Inhabited Information Spaces : Living with your Data (Computer Supported Cooperative Work)

Inhabited Information Spaces: Living with your Data

David N. Snowdon Elizabeth F. Churchill Emmanuel Frécon, Editors

Springer

Computer Supported Cooperative Work

011

011

011

011

11

Springer London Berlin Heidelberg New York Hong Kong Milan Paris Tokyo

i

Also in this series Gerold Riempp Wide Area Workflow Management 3-540-7643-4

Ralph Schroeder (Ed.) The Social Life of Avatars 1-85233-461-4

Celia T. Romm and Fay Sudweeks (Eds) Doing Business Electronically 3-540-76159-4

J.H. Erik Andriessen Working with Groupware 1-85233-603-X

Fay Sudweeks and Celia T. Romm (Eds) Doing Business on the Internet 1-85233-030-9

Paul Kirschner, Chad Carr and Simon Buckingham Shum (Eds) Visualising Argumentation 1-85233-664-1

Elizabeth F. Churchill, David N. Snowdon and Alan J. Munro (Eds) Collaborative Virtual Environments 1-85233-244-1 Christine Steeples and Chris Jones (Eds) Networked Learning 1-85233-471-1 Barry Brown, Nicola Green and Richard Harper (Eds) Wireless World 1-85233-477-0 Reza Hazemi and Stephen Hailes (Eds) The Digital University – Building a Learning Community 1-85233-478-9

Christopher Lueg and Danyel Fisher (Eds) From Usenet to CoWebs 1-85233-532-7 Kristina Höök, David Benyon and Alan J. Munro (Eds) Designing Information Spaces: The Social Navigation Approach 1-85233-661-7 Bjørn Erik Munkvold Implementing Collaboration Technologies in Industry 1-85233-418-5

Elayne Coakes, Dianne Willis and Steve Clark (Eds) Knowledge Management in the SocioTechnical World 1-85233-441-X

Related Title Richard Harper (Ed.) Inside the Smart Home 1-85233-688-9 A list of out of print titles is available at the end of the book

ii

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

David N. Snowdon, Elizabeth F. Churchill and Emmanuel Frécon (Eds)

Inhabited Information Spaces Living with your Data

With 94 Figures

011

011

011

11

iii

David N. Snowdon, BSc (hons) MSc, PhD Xerox Research Centre Europe, 6 Chemin de Maupertius, 38240 Meylan, France. Elizabeth F. Churchill, BSc, MSc, PhD FX Palo Alto Laboratory Inc., 3400 Hillview Avenue, Building. 4 Paol Alto, CA94110, USA. Emmanuel Frécon, MSc Swedish Insitute for Computer Science, Interactive Collaborative Environments Laboratory, Platforms for Collaborative Environments Group, Box 1263, 164 29 Kista, Sweden. Series Editors Dan Diaper, PhD, MBCS Professor of Systems Science & Engineering, School of Design, Engineering & Computing, Bournemouth University, Talbot Campus, Fern Barrow, Poole, Dorset BH12 5BB, UK Colston Sanger School of Management, University of Surrey, Guildford, Surrey GU2 7XH, UK

British Library Cataloguing in Publication Data Inhabited information spaces : living with your data. – (Computer supported cooperative work) 1. Human-computer interaction 2. Interactive computer systems I. Snowdon, David N., 1968– II. Churchill, Elizabeth F., 1962– III. Frécon, Emmanuel 004′.019 ISBN 1852337281 Library of Congress Cataloging-in-Publication Data A catalog record of this book is available from the Library of Congress Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms of licences issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers. CSCW ISSN 1431-1496 ISBN 1-85233-728-1 Springer-Verlag London Berlin Heidelberg Springer-Verlag is a part of Springer Science+Business Media springeronline.com © Springer-Verlag London Limited 2004 The use of registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant laws and regulations and therefore free for general use. The publisher makes no representation, express or implied, with regard to the accuracy of the information contained in this book and cannot accept any legal responsibility or liability for any errors or omissions that may be made. Typeset by Florence Production, Stoodleigh, Devon, England Printed and bound in the United States of America 34/3830-543210 Printed on acid-free paper SPIN 10910238

iv

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Foreword The Human Touch: Reflections on i3

011

011

011

011

11

The Machine-centred Mind Set At the Chicago World Fair of 1933, the official motto was: “Science Finds – Industry Applies – Man Conforms”. To many of us today this seems quite shocking, yet it has been the driving force of much development in the last century. In particular, if you look at the rise of computing over the last 50 years, you will see that, on the whole, development has been extraordinary, but fairly straightforward: it can be characterised as trying to make “faster and faster machines fit into smaller and smaller boxes”. Starting from the time of the ENIAC, one of the colossal computers of the 1940s, most IT progress has been driven from the point of view of the machine. Since then things have changed – but perhaps not really that much. Even if computers can today calculate many times over what was possible a few years ago, and the machines have become somewhat less obtrusive, much of the “mind set” has stayed the same. It is the visions of huge calculating machines spanning massive rooms, trying to recreate an absolute artificial intelligence, that still haunt much of the thinking of today. Clearly, it is difficult to shake off old mind sets.

Alternatives Alternatives to the idea of fitting computing into ever smaller boxes can mainly be attributed to Mark Weiser. In his paper, “The Computer for the 21st Century”, he outlined notions of how computing could become integrated into the fabric of everyday life by becoming completely distributed into the environment. In this way computing would become “ubiquitous”. More recently, similarly inspired work on “tangible media”, by Hiroshi Ishii has emerged from the MIT Media Lab. Apart from this, the technological revolution of GSM and the mobile phone has also had its share of making information technology come out of its “traditional shell”. Alternatives to the machine-centred view to computing were also starting around the same time, such as the “anthropocentric” ideas proposed by Mike

v

Foreword

Dertouzos at the MIT Computer Science Lab; and in a similar vein, cognitive scientist, Don Norman has been pointing out the lack of well-designed information environments. Roughly at the same time, but from a different perspective, we started thinking about how to give technology more of a “human touch”. Now, in principle, this should not be that difficult, as technology is after all, made by humans. In practice, however, one has to go quite far to break down the machine-centred and box-centred ways of thinking. We decided that the only way to attack the problem with any significance was to try to invert the picture completely – that is, to start thinking from the human point of view and work outwards. Our idea of “human centredness” was that it should nurture technological innovation but within a broader context of human values and aspirations. This was not the same as “user” driven, or “defined by user needs”, all of which tend to become stuck in improving the status quo, but not growing beyond it. At the same time, we also wanted to make sure to break out of the box-centred ways of thinking as much as possible and avoid doing “traditional HCI”, which was mainly involved in improving computers as they were. Our ideas were designed to balance questions of technically “how”, with questions of “why?” and “what for?”. And the aim was to see if we could start restoring the balance between people’s inventiveness to make new machines, with the essence of being human. Our questions became rather: How can we reach a better and more fulfilling balance between technology and people? What could be new ways of thinking about the problems? What could be the new paradigms that could lay the paths for further research and development?

The i3 Research Programme It is along these lines that we launched our first call for proposals back in 1995. Our general aim was to look at the relationship between people and information technology in the future: how could people access and use information, and exchange things with others using information technology as a medium? A clear break was needed to get out of stale thinking. Therefore, we called for new paradigms of interaction and research on new interfaces between people and the world of information. We also asked how such work could intertwine human, societal and technological elements into one dynamic research activity. One of the main quotes from our call for proposals was: The goal of i3 is to research and develop new human-centred interfaces for interacting with information, aimed at the broad population (1996).

To help define a specific research agenda, we first had a competition for more specific visions of the future. “Connected Community” and “Inhabited Information Spaces” were selected as the two visionary themes on which we based a subsequent call for research projects. Even though it took some time

vi

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Foreword

● 1 ●●●●●●●●●●●●●● to have an extra layer of calls for proposals, in retrospect it was better to “reculer pour mieux sauter”. The two selected themes had similar yet contrasting underlying philosophies. The Connected Community theme, proposed by a team headed by Irene Mac William (Philips Design, Eindhoven) and Marco Susani (Domus Academy, Milan) asked: forget about virtual environments and trying to fit people into some artificial world – how can we help people in their everyday environment, and integrate technology into this? The idea is to understand how information and communication tools start making a difference when they are embedded in a real context, and start being more meaningful for actual people and communities. How can technology enhance these environments and activities, rather than replace them? The other schema, Inhabited Information, proposed by a team headed by Tom Rodden (University of Nottingham) took a slightly different perspective. It stated: the Internet and the Web already represent a suspended reality, and people want to participate more in these spaces. Given that this is a reality, how could it evolve in the future? How could we make it more accessible to the broadest possible public, and make it socially interactive for large groups of people, in meaningful ways? And in similar spirit to the first theme, how can such environments link to the physical everyday world rather than be removed from it? At a later stage, we decided to supplement the research with an emphasis on learning. We wanted to explore new relationships between learning and technology. The idea was that a lot could be learnt about designing new interfaces by looking at how children interact, play and learn. Similar ideas had been experimented in a Lego context by Seymore Papert of the MIT Media Lab. In 1997, we decided to have a call on experimental school environments (ese). This centred around learning for very young children, in fact, the 4–8-year-old age range. This age range struck us as being particularly challenging because at this stage children don’t have too many of the adult preconceptions of the world, and are still open to new things. Young children have a different kind of “language” – a form of communication and expression from which adults can learn a lot. From this we wanted to gain insights about how to design meaningful interaction tools for the population at large. The header of our call was: The aim of i3-ese, is to research new kinds of IT-based tools designed to enable new approaches to learning, focussing on the age range of 4 to 8 (1997).

From each of these programmes we selected a number of individual research projects. Together these spanned many universities, research centres and companies across Europe, and involved a mix of people from many walks of life – artists, designers, computer scientists, game companies, technology companies, experimental schools, teachers and children, people in communities, etc. At the same time all these different outlooks were united by the common vision: of exploring new relationships between people and technology.

vii

Foreword

Grains of the Future In this book you will find some examples of work in particular from the Inhabited Information Spaces Grouping. It is interesting to see how some of these ideas are still “futuristic” and others have started to become part of mainstream thinking and made their way into products. Some people say that you can find “grains of the future” in the present today – the only problem is, where do you start to look? One of the potential advantages of this book is that by looking at the research developments stretching out into recent past, one can identify how some grains developed into trends of the present, and other are still just emerging. For those still interested in seeking out “grains of the future”, this book will be a valuable source. Jakub Wejchert Information Society DG European Commission Jakub Wejchert grew up in Ireland, with a family background of artists and architects, of Polish origin. He studied natural science at Trinity College Dublin, specialising in physics, and holds a doctorate (modelling of non-linear networks) from the same institution. Later he worked in the USA with IBM research, working on computer graphics and interface design. He joined the European Commission in 1992. At the Future and Emerging Technologies unit, he set up and managed a number of research programmes such as i3 – intelligent information interfaces; i3 – experimental school environments; and the “disappearing computer”. He now works as an advisor on vision and strategy to one of the Directors in the Information Society Programme. Jakub lives in Waterloo, south of Brussels, with his wife and three sons. The opinions expressed here are those of the author and do not necessarily reflect the position of the European Commission.

viii

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Acknowledgements

011 The editors would like to acknowledge the European i3 initiative and all the authors of the chapters in this volume for their contributions. Much of the work described in this volume would not have taken place without funding from the European Commission. We would also like to thank SICS, XRCE and FX Palo Alto Laboratory for supporting our activities within this domain. Rosie Kemp and Melanie Jackson of Springer also deserve thanks for their help and support throughout the process of preparing this book for publication.

011

011

011

11

ix

This page intentionally left blank

Contents

011

011

011

List of Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix Part 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1. Inhabited Information Spaces: An Introduction Elizabeth Churchill, David Snowdon and Emmanuel Frécon . . . . . . . . . . 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Chapters in this Volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.1 Pure Virtual Environments . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.2 Mixed Reality Environments . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.3 Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.4 Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.5 Community . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 3 5 5 6 6 7 8 8

Part 2. Pure Virtual Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2. WWW3D and the Web Planetarium Mårten Stenius and David Snowdon . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Producing a 3D Representation of a Web Page . . . . . . . . . . . . . . 2.3 Browsing the Web Using WWW3D . . . . . . . . . . . . . . . . . . . . . . . 2.4 Improving Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 The Web Planetarium: Creating a Richer Visualisation . . . . . . . . 2.5.1 Visual Differentiation of Nodes . . . . . . . . . . . . . . . . . . . . . . 2.5.2 The Web as a Road Network . . . . . . . . . . . . . . . . . . . . . . . . 2.5.3 Hybrid Browsing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

11 11 12 13 16 20 20 22 22 24

3. PlaceWorld, and the Evolution of Electronic Landscapes Steve Pettifer, Jon Cook and James Marsh . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Background: The Physical and the Abstract . . . . . . . . . . . . . . . . . 3.2.1 Watching a Cityscape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 The Distributed Legible City . . . . . . . . . . . . . . . . . . . . . . . . .

25 25 27 28 29

011

11

xi

Contents

3.3 3.4

3.5

3.6

3.2.3 Finding “Something to Do” . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.4 Abstract Influences: Nuzzle Afar . . . . . . . . . . . . . . . . . . . . . PlaceWorld . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 The Design of PlaceWorld . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 The User Interface and Presentation System . . . . . . . . . . . . Technological Challenges for Electronic Landscapes . . . . . . . . . . 3.4.1 Synchronising the Behaviour of Entities . . . . . . . . . . . . . . . 3.4.2 Distribution and Communications . . . . . . . . . . . . . . . . . . . . 3.4.3 Defining the Behaviour of Entities . . . . . . . . . . . . . . . . . . . . 3.4.4 Methods and Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.5 The Distribution Architecture . . . . . . . . . . . . . . . . . . . . . . . System Support for PlaceWorld . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 Menus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 Access Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.3 Exploiting Subjectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.4 Becoming a Place Where Places Meet . . . . . . . . . . . . . . . . . Conclusions

4. Using a Pond Metaphor for Information Visualisation and Exploration Olov Ståhl and Anders Wallberg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 The Pond . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 The Pond Ecosystem Metaphor . . . . . . . . . . . . . . . . . . . . . . 4.2.2 The Pond Example Application . . . . . . . . . . . . . . . . . . . . . . 4.2.3 The Hardware Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.4 The Software Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 The Pond Audio Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Observations from Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7 Summary and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

31 33 33 34 36 37 39 40 41 43 44 46 46 46 47 48

51 51 54 54 55 56 57 58 63 64 65 68

Part 3. Mixed Reality Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 5. City: A Mixture of Old and New Media Matthew Chalmers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Ongoing and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xii

71 71 73 77 82 86 88

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Contents

● 1 ●●●●●●●●●●●●●● 6. Soundscapes Tony Brooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 The Soundscapes System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Therapeutic Uses of Soundscapes . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Artistic Performances Based on Soundscapes . . . . . . . . . . . . . . . . 6.4.1 Interactive Painting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.2 The Four Senses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

89 89 89 92 94 94 96 99

011

011

011

011

7. The Computational Interplay of Physical Space and Information Space Enric Plaza . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 7.2 The Interplay of Physical and Information Spaces . . . . . . . . . . . 102 7.3 A Framework for Context-aware Agents . . . . . . . . . . . . . . . . . . . 104 7.3.1 Awareness and Delivery Services . . . . . . . . . . . . . . . . . . . . 105 7.3.2 Agents Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 7.4 The COMRIS Conference Centre . . . . . . . . . . . . . . . . . . . . . . . . . 107 7.4.1 Delivery Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 7.4.2 Awareness Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 7.4.3 Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 7.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Part 4. Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 8. Communicating in an IIS: Virtual Conferencing Adrian Bullock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Virtual Conferencing – a Historical Perspective: Past, Present and Future . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.1 What Do We Mean by Virtual Conferencing? . . . . . . . . . . 8.3 Approaches to Virtual Conferencing . . . . . . . . . . . . . . . . . . . . . . 8.3.1 Early Videoconferencing . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.2 MUDs and MOOs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.3 The Arrival of Graphics . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.4 Video Comes of Age . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.5 Graphics Come of Age . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4 Using Virtual Conferencing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.1 Understanding Collaboration . . . . . . . . . . . . . . . . . . . . . . . 8.4.2 The Importance of First Impressions . . . . . . . . . . . . . . . . . 8.4.3 Sharing Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.4 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.5 Real Versus Abstract: The Role of Video? . . . . . . . . . . . . .

115 115 116 117 117 117 118 118 120 122 124 124 125 125 125 126

11

xiii

Contents

8.5 8.6

Virtual Conferencing Versus Telephony . . . . . . . . . . . . . . . . . . . Guidelines for Using Virtual Conferencing Effectively . . . . . . . . 8.6.1 What Is the Task at Hand? . . . . . . . . . . . . . . . . . . . . . . . . . 8.6.2 Communication Media . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.6.3 Infrastructural Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

127 129 129 130 130 131

9. Getting the Picture: Enhancing Avatar Representations in Collaborative Virtual Environments Mike Fraser, Jon Hindmarsh, Steve Benford and Christian Heath . . . . 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.1 Awareness and Co-ordination . . . . . . . . . . . . . . . . . . . . . . 9.3.2 Anticipation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.3 Occlusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5 Reflections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.1 Scaleability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.2 Reciprocity of Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.3 Unrealism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

133 133 135 137 137 140 142 145 145 146 147 149 150

8.7

10. New Ideas on Navigation and View Control Inspired by Cultural Applications Kai-Mikael Jää-Aro and John Bowers . . . . . . . . . . . . . . . . . . . . . . . . . . 151 10.1 Introduction and Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 10.1.1 Challenges for Interaction Design . . . . . . . . . . . . . . . . . . . 152 10.2 Interactive Performances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 10.2.1 Lightwork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 10.2.2 Blink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 10.3 Inhabited Television . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 10.3.1 Heaven and Hell – Live . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 10.3.2 Out of This World . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 10.4 Production Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 10.4.1 Finding and Framing the Action . . . . . . . . . . . . . . . . . . . . 170 10.4.2 The Round Table: A Physical Interface . . . . . . . . . . . . . . . 172 10.4.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 10.5 Discussion: Navigation, Presence and Avatars . . . . . . . . . . . . . . 176 10.5.1 Avatar-centred Navigation . . . . . . . . . . . . . . . . . . . . . . . . 176 10.5.2 Object-centred Navigation . . . . . . . . . . . . . . . . . . . . . . . . 177 10.5.3 Activity-oriented Navigation . . . . . . . . . . . . . . . . . . . . . . . .178 10.5.4 Navigation as Montage, Dispersed Avatars . . . . . . . . . . . . 178 10.5.5 Accomplishing Presence and Intelligibility . . . . . . . . . . . . 179

xiv

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

Contents

● 1 ●●●●●●●●●●●●●● 11. Presenting Activity Information in an Inhabited Information Space Wolfgang Prinz, Uta Pankoke-Babatz, Wolfgang Gräther, Tom Gross, Sabine Kolvenbach and Leonie Schäfer . . . . . . . . . . . . . . . 11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Related Work and Requirements . . . . . . . . . . . . . . . . . . . . . . . . 11.3 User Involvement and Studies . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.1 Partner Settings and Evaluation Methods . . . . . . . . . . . . . 11.3.2 Do Users Meet at all in a Shared Workspace? . . . . . . . . . 11.4 The Tower Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5 Personalised Overview of Activities: The Tower Portal . . . . . . . 11.6 Awareness in a Working Context: Smartmaps . . . . . . . . . . . . . . 11.7 Symbolic Actions in a Context-based 3D Environment . . . . . . . 11.7.1 The Tower World . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.7.2 User Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.8 DocuDrama . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.9 Ambient Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.10 Lessons Learned About Awareness . . . . . . . . . . . . . . . . . . . . . . . 11.10.1 Awareness Is Something One Is Not Aware of . . . . . . . . 11.10.2 Synchronicity of Awareness . . . . . . . . . . . . . . . . . . . . . . 11.10.3 Walking and Talking Are Means to Achieve Awareness . 11.10.4 Peripheral Awareness in Electronic Settings . . . . . . . . . . 11.10.5 Awareness Is Double-situated: The Workspace’s and the Observer’s Situation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.11 Summary and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

181 181 182 184 185 186 188 189 191 194 194 196 198 201 203 203 204 205 205 206 207

Part 5. Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 12. DIVE: A Programming Architecture for the Prototyping of IIS Emmanuel Frécon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2 The Virtual World as a Common Interaction Medium . . . . . . . 12.3 Partial, Active Database Replication . . . . . . . . . . . . . . . . . . . . . . 12.4 Programming the System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.1 The DIVE Programming Model . . . . . . . . . . . . . . . . . . . . 12.4.2 Programming Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.3 Building your Application . . . . . . . . . . . . . . . . . . . . . . . . . 12.5 DIVE as a Component-based Architecture . . . . . . . . . . . . . . . . . 12.5.1 System Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5.2 User-oriented Components . . . . . . . . . . . . . . . . . . . . . . . . 12.5.3 The DIVE Run-time Architecture . . . . . . . . . . . . . . . . . . . 12.6 The London Demonstrator: An Example Application in More Detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

211 211 212 213 215 216 216 218 223 223 224 225 226

11

xv

Contents

12.6.1 Centre of London . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.6.2 Collaboration Services for Use by Groups . . . . . . . . . . . . 12.6.3 Tourist Information Data Visualisation Service . . . . . . . . 12.6.4 Real-time Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.7 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . .

228 229 229 230 231

13. Communication Infrastructures for Inhabited Information Spaces David Roberts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.2 Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.3 Avatars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.4 Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.5 Communication Requirements . . . . . . . . . . . . . . . . . . . . . 13.1.6 Resources: Computers and Networks . . . . . . . . . . . . . . . . 13.2 Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.1 Localisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.2 Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.3 Persistence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.4 Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.1 The DIVE Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.2 PING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4 Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4.1 Point-to-point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4.2 Tunnelled Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4.3 Hybrid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

233 233 234 235 236 237 238 240 240 241 246 251 252 256 256 260 263 264 265 266 266

Part 6. Community . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 14. Peer-to-peer Networks and Communities Mike Robinson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.2 Early Inhabited Information Spaces in CSCW . . . . . . . . . . . . . . 14.2.1 Rendering the Invisible Visible . . . . . . . . . . . . . . . . . . . . . 14.2.2 ClearBoard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.2.3 Feather, Scent and Shaker: Supporting Simple Intimacy . 14.2.4 Gesture Cam: The Nodding Robot . . . . . . . . . . . . . . . . . . 14.3 P2P Themes and Overall Direction . . . . . . . . . . . . . . . . . . . . . . . 14.4 Design for Community: Inhabited Information Spaces . . . . . . . 14.4.1 Communities: An Aside on Definitions . . . . . . . . . . . . . . 14.4.2 Communities: An Aside on Use . . . . . . . . . . . . . . . . . . . . 14.4.3 Communities: An Aside on Philosophy . . . . . . . . . . . . . .

xvi

271 271 274 274 275 276 277 278 281 281 282 284

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

Contents

● 1 ●●●●●●●●●●●●●● 14.5 P2P, Community and the Design of Inhabited Information Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286 14.6 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 15. Inhabitant’s Uses and Reactions to Usenet Social Accounting Data Byron Burkhalter and Marc Smith . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.3 Netscan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.4 Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.4.1 Social Accounting Data and Author-assessment Threads 15.4.2 Social Accounting Data and Newsgroup-assessment Threads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

291 291 293 294 295 295 301 305

References

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

Index

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325

011

011

011

11

xvii

This page intentionally left blank

List of Contributors

011

011

011

011

11

Steve Benford Mixed Reality Laboratory, School of Computer Science, University of Nottingham, Jubilee Campus, Nottingham, NG8 1BB, UK [email protected] John Bowers Department of Numerical Analysis and Computer Science, Royal Institute of Technology, SE-100 44, Stockholm, Sweden [email protected] Tony Brooks Aalborg University, Niels Bohrs Vej 8, DK 6700 Esbjerg, Denmark [email protected]

[email protected] Matthew Chalmers Computing Science, University of Glasgow, 17 Lilybank Gardens, Glasgow, G12 BQQ, UK [email protected] Elizabeth Churchill FX Palo Alto Laboratory, 3400 Hillview Avenue, Building 4, Palo Alto 94304, USA [email protected] Jon Cook Department of Computer Science, The University of Manchester, Oxford Road, Manchester, M13 9PL, UK [email protected]

Adrian Bullock Swedish Institute of Computer Science, Box 1263, SE-164 29, Kista, Sweden [email protected]

Mike Fraser Mixed Reality Laboratory, School of Computer Science, University of Nottingham, Jubilee Campus, Nottingham, NG8 1BB, UK [email protected]

Byron Burkhalter Department of Sociology, University of California, Los Angeles, 264 Haines Hall, 375 Portola Plaza, Los Angeles, CA 90095–1551, USA

Emmanuel Frécon Swedish Institute of Computer Science, Box 1263, SE-164 29, Kista, Sweden [email protected]

xix

List of Contributors

Wolfgang Gräther Fraunhofer Institute for Applied Information Technology, Schloss Birlinghoven, 53754 Sankt Augustin, Germany wolfgang.graether@fit.fraunhofer.de

James Marsh Department of Computer Science, The University of Manchester, Oxford Road, Manchester, M13 9PL, UK [email protected]

Tom Gross Fraunhofer Institute for Applied Information Technology, Schloss Birlinghoven, 53754 Sankt Augustin, Germany tom.gross@fit.fraunhofer.de

Uta Pankoke-Babatz Fraunhofer Institute for Applied Information Technology, Schloss Birlinghoven, 53754 Sankt Augustin, Germany uta.pankoke@fit.fraunhofer.de

Christian Heath Work, Interaction and Technology Research Group, The Management Centre, King’s College London, Franklin-Wilkins Building, London, SE1 8WA, UK [email protected] Jon Hindmarsh Work, Interaction and Technology Research Group, King’s College London, UK [email protected] Kai-Mikael Jää-Aro Department of Numerical Analysis and Computer Science, Royal Institute of Technology, SE-100 44, Stockholm, Sweden [email protected] Sabine Kolvenbach Fraunhofer Institute for Applied Information Technology, Schloss Birlinghoven, 53754 Sankt Augustin, Germany sabine.kolvenbach@fit.fraunhofer.de

xx

Steve Pettifer Department of Computer Science, The University of Manchester, Oxford Road, Manchester, M13 9PL, UK [email protected] Enric Plaza IIIA, Artificial Intelligence Research Institute, CSIC, Spanish Council for Scientific Research, Campus UAB, 08193 Bellaterra, Catalonia, Spain [email protected] Wolfgang Prinz Fraunhofer Institute for Applied Information Technology, Schloss Birlinghoven, 53754 Sankt Augustin, Germany wolfgang.prinz@fit.fraunhofer.de David Roberts Department of Computer Science, University of Reading, Whiteknights, Reading, RG6 6AY, UK [email protected]

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

List of Contributors

● 1 ●●●●●●●●●●●●●● Mike Robinson Sageforce Ltd, 61 Kings Road, Kingston-on-Thames, Surrey, KT2 5JA, UK [email protected] Leonie Schäfer Fraunhofer Institute for Applied Information Technology, Schloss Birlinghoven, 53754 Sankt Augustin, Germany leonie.schaefer@fit.fraunhofer.de Marc Smith Microsoft Research, Microsoft Corporation, One Microsoft Way, Redmond, WA 98052, USA [email protected] David Snowdon Xerox Research Centre Europe, 6, chemin de Maupertuis, 38240 Meylan, France [email protected]

Olov Ståhl Swedish Institute of Computer Science, Box 1263, SE-164 29, Kista, Sweden [email protected] Mårten Stenius Alkit Communications AB, Aurorum 2, SE-977 75, Luleå, Sweden [email protected] Anders Wallberg Swedish Institute of Computer Science, Box 1263, SE-164 29, Kista, Sweden [email protected]

011

011

11

xxi

This page intentionally left blank

011

011

011

011

11

Part 1 Introduction

This page intentionally left blank

011

011

011

011

Inhabited Information Spaces: An Introduction

1

Elizabeth Churchill, David Snowdon and Emmanuel Frécon

1.1 Introduction Studies of co-operative work have shown that, even when co-operation is not explicit, a surprisingly large amount of work relies on the knowledge of what other people are doing (or have done) so that work can be co-ordinated. Further, people collaborate over objects that are manipulated and exchanged: reports, diagrams, maps, books, models and drawings are all examples. People’s conversations and actions are part and parcel of producing the social frame within which work takes place (Giddens, 1984; Lave and Wenger, 1991). However, the technological circumstances within which that work takes place has a profound impact upon how it is achieved – technologies afford resources and constraints that affect practice. This volume offers readers an introduction to the field of Inhabited Information Spaces (IIS). We attempt to shed some light on the most important issues, including examples of representing information, how people interact in such systems, how IIS systems are constructed, and emerging notions of communities. There are already many books dedicated to “pure” information visualisation (techniques for visually representing information) and so in this volume we concentrate mainly on the value added by inhabited spaces rather than visualisation techniques per se. For a review of the field of “pure” information visualisation we recommend Card et al. (1999), who give a good overview of the field. The question addressed by IIS design is how best to design spaces and places where people and digital data can meet in fruitful exchange – that is, how to create effective social workspaces where digital information can be created, explored, manipulated and exchanged. IIS are the confluence of research into distributed, augmented and virtual reality (VR) spaces, information visualisation and computer-supported co-operative work (CSCW).

11

3

Inhabited Information Spaces

The term “Inhabited Information Spaces” derived from work being carried out within a number of European research initiatives (e.g. INSCAPE, COMIC) and laid out an agenda whereby virtual reality and information visualisation techniques were explicitly combined in support of collocated and remote collaborative work. Thus, in Inhabited Information Spaces (IIS) both information and people who are using that information (viewing it, manipulating it) are represented. This supports collaborative action on objects, provides awareness of others’ ongoing activities and offers a view of information in the context of its use. Thus, while information visualisation systems are useful tools in themselves, a representation of others who are also using the information and what they are doing with it could potentially add considerably to the value of such systems for co-ordination of collaborative work. Just knowing that many people are accessing a particular piece of information could be almost as useful as the piece of information itself. The specific representations can vary but in all cases they are manipulable: data can be interrogated, representations of people are mobile and interactive. Further, being shared, navigable, “live-in” spaces, information can be explicitly sought, can be “discovered by chance” (cf. Williamson, 1998), or “encountered” (cf. Erdelez, 1999) or tacitly consumed as part and parcel of navigating an “information neighbourhood” (cf. Savolainen, 1995). In Europe, the European Commission (EC) funds large programmes of research covering most fields of human endeavour (science, technology, medicine, culture etc.); one of the domains supported by the EC is Information Society Technologies (IST, http://www.cordis.lu/ist/). In 1996 the i3 (Intelligent Information Interfaces) network (http://www.i3net.org) was formed as a mechanism to create a community of people working on a number of research projects. One of the aims of this was to encourage researchers participating in European projects to exchange ideas and information and to allow collaboration on a larger scale. From 1997–2000 the i3 supported a programme called Inhabited Information Spaces and much of the work that appears in this book resulted from this programme. The field of IIS overlaps that of Collaborative Virtual Environments (CVEs) (see Churchill et al., 2001), as CVEs are one of the preferred implementation techniques for visualisation information in a collaborative way. However, IIS does not necessarily imply the use of online virtual environments technology – for example, it is possible to imagine a system that enables co-located groups to co-operatively work with information by using a display projected onto physical artefacts. Research into IIS also overlaps with work carried out in Social Navigation, which explicitly addresses social aspects of information seeking, searching and use (see Munro et al., 1999). The chapters in this volume cover all variants on IIS, the technology required to make it work and the social and psychological issues raised by such work. We also present recent innovations in “hybrid

4

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

An Introduction

● 11 ● ● ● ● ● ● ● ● ● ● ● ● ● ● environments” and augmented real-world environments. Our aim with this broad coverage is to offer readers the opportunity to reflect on the intersection of technology design, communication, representation, and collaborative work practices. Technological, psychological and sociological issues in the design and use of Inhabited Information Spaces are considered, including: design issues in the development of technology for human–human and human–system collaboration around information visualisations; applications demonstrating uses of the technology; and psychological/sociological analyses of the way such systems are used in practice. Our aim was to provide a broader perspective than solely graphical aspects of visualising information, systems aspects underlying the distribution and sharing of graphical and textual workscapes, or the communication and work practice aspects of the use of such systems.

1.2 Chapters in this Volume We have divided the chapters in this volume into a number of broad areas, although observations made by the authors often span these areas. The areas are pure virtual environments; mixed reality; communicationoriented systems and applications; construction; and community.

1.2.1 Pure Virtual Environments Stenius and Snowdon, in their chapter ‘WWW3D and the Web Planetarium’ (Chapter 2) describe the WWW3D 3D web browser, and how it turns browsing the web into exploration of a 3D space. The chapter describes the initial version of WWW3D and describes how it evolved into the Web Planetarium. Not only is the Web Planetarium more aesthetically pleasing and more scalable than WWW3D, but in this incarnation it has also been re-purposed to serve as a gateway between 3D environments. In Chapter 3, Pettifer, Cook and Marsh describe the Placeworld system, and its implementation in the Deva virtual environment. Placeworld was inspired by Jeffrey Shaw’s artwork “PLACE – A User’s Manual” and, like the Web Planetarium, aims to provide a connection between virtual spaces. The chapter describes the series of user trials that led to the final Placeworld design and how the Deva system is used to create a high performance virtual environment that implements the Placeworld design. This chapter also gives some insight into some of the issues that must be tackled to implement large scale IIS and CVE systems efficiently and serves as an appetiser for topics covered in more depth in Part 5 of this book. The final chapter in Part 2, Chapter 4, is by Wallberg and Ståhl, who use a pond metaphor for information visualisation and exploration. The Pond is a system that allows people to browse collections of multimedia data, such as music albums. The interface is presented in the form of a

5

Inhabited Information Spaces

large back-projected display on a table surface allowing several people to gather around it and use it collaboratively.

1.2.2 Mixed Reality Environments Chalmers, in his chapter entitled “City: A Mixture of Old and New Media” (Chapter 5), argues that one of the problems with “traditional” CVE systems is that they are disconnected from the physical environment and from other media (for a review of CVE systems see Churchill et al., 2001). He calls for an approach in which there is an explicit linking between different media, where people are considered to inhabit the “real” (physical) world, not the virtual, and have a number of media both “old” and “new” available to them. The chapter presents an experiment where visitors experience a gallery via different media – one by physically experiencing it, one via the web and one via an immersive 3D VR environment. All three are able to communicate via an audio link and are given awareness of the locations and actions of the others. The chapter presents the results of this experiment and the ways in which the participants used the features of the technology to interact and share their experiences. In Chapter 6, Brooks describes the Soundscapes system, which allows unencumbered interaction with visual and auditory systems projected into the physical world. This work illustrates a different form of information system, an auditory one, and offers an example of a mixed reality system, and example applications are covered including therapeutic use and public artistic performances. In the next chapter, Plaza illustrates the computational interplay of physical space and information space. Drives within mobile computing push computers further “into” the physical world (see also Mark Weiser’s vision for ubiquitous computing, Weiser, 1991). In order to design tools that are context sensitive and not inappropriately intrusive, it has been argued that such devices need to be “aware” of the activities of their users. This chapter describes the approach used in the COMRIS project in which wearable computers were linked with an information space composed of agents who attempt to find information useful to a person at a given moment.

1.2.3 Communication In Chapter 8, Bullock describes human–human communication via the medium of an IIS, considering design issues in the development of virtual conferencing. He makes the points that, for an IIS to function effectively, all technological elements need to work in concert. With video conferencing as a backdrop, Bullock explores opportunities and pitfalls of using IIS for mediated communication.

6

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

An Introduction

● 11 ● ● ● ● ● ● ● ● ● ● ● ● ● ● Fraser, Hindmarsh, Benford and Heath in their chapter “Getting the Picture: Enhancing Avatar Representations in Collaborative Virtual Environments” discuss how humanoid avatars are the most widely used type of avatar in VR systems. They note that although the avatars have humanoid forms they don’t have human-like perceptual abilities within the virtual worlds – the data that are relayed back to people from their avatar as prosthesis-in-the-virtual-world is in fact often misleading. This chapter examines some of the problems inherent in using different forms of avatar, and the problems this poses for collaboration in IIS. The authors describe some extensions the authors have made in an attempt to rectify some of the problems they have encountered. Jää-Aro and Bowers describe some new ideas on navigation and view control that have been inspired by cultural applications. This chapter describes the lessons learnt from a number of public VR and mixedreality performances. The authors discuss what they learnt in terms of the content of performances, the pacing and the means given to participants to navigate within the space. An important issue addressed is how such performances can be made accessible to non-interactive audiences who can only see a TV-like rendering of the event. In Chapter 11, Prinz et al. consider how to present awareness of the activities of others for better support of collaborative work. This chapter describes the TOWER (Theatre of Work Enabling Relationships) system, which provides a number of mechanisms to communicate awareness information to members of a work group both via 3D displays and also via Smartmaps integrated with the Basic Support for Co-operative Work (BSCW) document management system. Smartmaps are 2D displays based upon the tree-map visualisation technique. The TOWER world is an automatically constructed 3D environment that represents both users and documents and indicates the actions that users are taking with respect to the documents via symbolic actions and gestures performed by the avatars. DocuDrama allows 3D presentations of the past actions taken by members of a project team as a sort of 3D virtual theatre in which avatars look and turn towards one another to enhance the impression of an ongoing conversation, and camera navigation is carefully controlled in order to generate an interesting presentation. Finally, small simple robots were employed in order to give a tangible presentation of the activities of other users.

1.2.4 Construction Frecon’s chapter in Part 5 introduces readers to DIVE, a programming environment for prototyping IIS. The DIVE CVE system is described, with a focus on the mechanisms it provides to allow developers to rapidly develop VR applications, including IIS. DIVE provides several different APIs (application programmer’s interfaces) and mechanisms for

7

Inhabited Information Spaces

creating dynamic 3D content thereby allowing developers to choose the combination that works best for them. The chapter concludes with a number of examples that show how significant applications have been built using DIVE. DIVE is one of the oldest and most mature VR systems and the current version is the result of many years of experience; both the Web Planetarium and Pond systems are built in DIVE. Readers are urged to read these three chapters (2, 4 and 12) to fully understand what is possible with a mature VR system. In Chapter 13, Roberts considers communication architectures for IIS, describing the most important networking issues that need to be faced when trying to construct distributed IIS systems. He also details some of the techniques that can be used to create an illusion of a shared space in the face of delays caused by communications technology. A number of different CVE systems are described in order to give concrete examples of the techniques described in the earlier part of the chapter.

1.2.5 Community Robinson’s chapter deals with peer-to-peer networks and communities. Robinson argues that the concept of peer-to-peer and notions of community are heavily interdependent, and that the design of IIS would benefit by focusing more closely on community as the organising principle of peer-to-peer. The author first considers the early days of research into CSCW, analyses the metaphor of community and how it relates to electronically mediated communication, and draws parallels with the current state of peer-to-peer systems today. Burkhalter and Smith move away from what might be considered a true IIS in Chapter 15 to explore how the availability of social accounting data can help the auto-regulation of online communities. The authors consider Usenet news but the principles could just as well be applied to other systems. The intent of this chapter is to show how similar information might be used to help CVE-based communities such as those being created by there.com (http://www.there.com), if they ever reach the scale of communities such as Usenet.

1.3 Summary As noted above, the chapters in this volume cover a broad range of technical and social issues. All are focused on creating useful and “habitable” environments for information representations, seeking, searching and manipulation. There are many challenges to be faced at technical and at social levels, and far more research needs to be done on the long-term use of Inhabited Information Spaces, and how they co-evolve as a result of being regularly inhabited. We hope you enjoy reading about the work as much we have.

8

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

Part 2 Pure Virtual Environments

011

011

011

11

9

This page intentionally left blank

011

011

011

011

11

WWW3D and the Web Planetarium

2

Mårten Stenius and David Snowdon

2.1 Introduction This chapter will describe an Inhabited Information Space based around a 3D visualisation of a portion of the WWW. The system was originally called WWW3D (Snowdon et al., 1996), and evolved from an experiment in immersive 3D web browsing into a richer inhabited information space called the Web Planetarium during the i3 eSCAPE project (Stenius et al., 1998). We include it here as an example of an inhabited space supporting a common modern activity – web browsing – and also as an example of how a basic visualisation was extended to correct deficiencies in the earlier system. WWW3D started life as an experiment in 3D web browsing. We were thinking about some of the original ideas behind VRML (Virtual Reality Modelling Language, www.vrml.org), namely that it would be a sort of 3D web. In fact many people working on VRML were inspired by the Cyberspace detailed in William Gibson ’s novels: Cyberspace. A consensual hallucination experienced daily by billions of legitimate operators, in every nation, by children being taught mathematical concepts . . . A graphic representation of data abstracted from the banks of every computer in the human system. Unthinkable complexity. Lines of light ranged in the nonspace of the mind, clusters and constellations of data. Like city lights, receding (Gibson, 1986).

However, VRML became more a means to represent 3D models with hyperlinks between them, than a richer way of experiencing the web. It so happened that as we were thinking about this during the summer of 1996 we were asked to provide a demonstration for SICS’s (the Swedish Institute of Computer Science’s) new large-screen immersive VR system. This system, while having impressive graphical performance, did not support interaction via a standard 2D GUI (Graphical User Interface); we therefore decided to create a web browser that was entirely 3D. While

11

Inhabited Information Spaces

the result was not a web browser as usable as Netscapes ’s or Microsoft’s for standard web browsing, the prototype had several interesting features and evolved into a more capable system during the i3 eSCAPE project. In this chapter we will describe the original system, WWW3D, and its evolution into the Web Planetarium. Just as a normal web browser, WWW3D allowed users to follow links and view the web pages associated with those links. However, rather than simply showing the current web page the user was exploring, WWW3D also visualised the structure of the portion of the web that the users had explored, historical information showing when pages had been last viewed and (thanks to the DIVE environment for which WWW3D was written) other users browsing the web at the same time. Note that WWW3D was never intended to visualise the web – the web is far too large for this to be meaningful, instead we simply visualise the portion of the web that users have explored and are exploring. We assume that existing search engines are sufficient to locate new pages of interest. The two key features of WWW3D were its method for representing web pages in 3D and the method for organising multiple web pages in 3D space. We will describe how this was done before continuing with a description of how the basic prototype was improved.

2.2 Producing a 3D Representation of a Web Page Many HTML tags provide semantic information about the marked-up text – that is, what the text is rather than how it should be displayed on the page. 2D web browsers will typically use this information to determine what style (i.e. font, font style, font size, etc.) to use when drawing a particular piece of text. WWW3D uses the information contained in HTML tags to produce a representation of the document in 3D space. A web document is represented as a sphere that is labelled with the document’s title. The contents of the document is placed around the inside surface of the sphere. Displaying large amounts of text in a satisfactory way is difficult in current VR systems, so textual information is currently represented by icons that can be unfolded to reveal the entire text. The first few words of the piece of text are displayed under the icon to give some indication of the contents. Images are displayed by texture mapping them onto polygons on the inside surface of the sphere. Finally, links to other documents are represented as icons labelled with their destination. To reduce the visual complexity of the virtual environment WWW3D makes extensive use of level of detail (LOD) operations. When viewed from outside, a document is represented as an opaque sphere and the actual document contents are not displayed. When a user enters a document to view it, the sphere is drawn in wire-frame so that the rest of the virtual environment is still visible.

12

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

WWW3D and the Web Planetarium 2 ●●●●●●●●●●●●● ● 1 ●

011

011

011

Figure 2.1 A WWW3D representation of a web document seen from the inside. The spheres in the background represent other web pages.

Figure 2.1 shows the contents of a web document as displayed by WWW3D. Figure 2.2 shows the display generated when a user reads some of the text comprising a web document.

2.3 Browsing the Web Using WWW3D

011

11

When a user selects a link icon, WWW3D creates a new sphere representing the target document and places it near the document from which the user selected the link. In order to indicate the structure of the portion of the Web that the users have explored, WWW3D draws arrows between the spheres representing linked documents. If the documents are resident on the same Web server then the arrow is drawn in blue, otherwise

13

Inhabited Information Spaces

Figure 2.2 Reading part of the text of a web document.

it is drawn in green, thereby helping to provide additional information on the structure of the documents that the user has explored. In addition to this, the brightness of the arrow is dependent on the time since a user last followed that link thereby providing users with a visual representation of their browsing history. If WWW3D fails to fetch a document then a small red arrow is attached to the source document to represent the “broken” link . WWW3D is implemented using multiple lightweight threads so users do not have to wait for a document to be retrieved before selecting another link. This behaviour is essential if multiple users are to be able to browse independently. Users are also free to navigate through the space and browse other documents while waiting for a document to be retrieved.

14

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

WWW3D and the Web Planetarium 2 ●●●●●●●●●●●●● ● 1 ●

As WWW3D parses a newly retrieved document, it checks for links to documents that the user has already explored and draws arrows to represent them. This means that at any given moment the complete set of links between documents is displayed without users having to follow every link. This is intended to aid the user by indicating links between documents that the user might have been unaware of. This also has the result that several users can be browsing different parts of the web and yet any links between the sets of documents they are exploring will be displayed. This might be useful since users will then have a visual representation of possible common interests. To produce an acceptable layout of the set of linked documents an incremental version of the Force Directed Placement (FDP) (Fruchterman and Reingold, 1991) algorithm is used. Links between documents act like spring forces that result in linked documents being moved closer together. Documents exert repulsive forces on one another which prevents documents being placed closer together than a userspecified minimum separation. At regular (user-specified) intervals WWW3D applies the FDP algorithm to refine the inter-document layout for a specified number of iterations. The more links a document has, the greater “inertia” it is considered to have when the FDP algorithm is applied. This has the result that heavily referenced documents are less likely to move and provides some stability to the visualisation. In addition damping is applied to try and prevent large changes to the visualisation for a given iteration of t he FDP algorithm – this helps to prevent the user from becoming disorientated. The result of this is that the inter-document layout gradually evolves over time to produce clusters of inter-linked documents. Instead of running the FDP algorithm until it converges (which could take a long time) only a specified number of iterations are executed. The rational for this is that the space will change anyway as users browse new web pages so it is not necessary to force users to wait until the layout stabilises before allowing them to continue browsing. If there is no change to the space (no new web pages) then the system will gradually converge to a stable state. However, the FDP algorithm suffers from the disadvantage that it can take a great many iterations to converge to a stable state. Figure 2.3 shows the display produced by WWW3D after the user has browsed a number of documents. The FDP algorithm has resulted in the formation of clusters of closely linked documents. The colours of the documents provide some indication of how long ago the user last visited them. Between invocations WWW3D stores information on the current set of documents, the links between them and the current 3D layout to a file. When a new instance of WWW3D is started, it reads this history file and displays the structure as it appeared in the last session. The contents of the documents found from the history file are not retrieved until the user

15

Inhabited Information Spaces

Figure 2.3 A collection of web documents, showing the links drawn between them.

enters a particular document. Since the document representations are opaque this process is invisible to the user except for an occasional delay in seeing a document’s contents.

2.4 Improving Scalability One problem with the WWW3D prototype is that it suffers from scalability problems both in terms of display and number of users. Since a single DIVE application is responsible for management of the visualisation, it is vulnerable to becoming overloaded if several users simultaneously place demands on it. In addition to this the complexity of the display increases as documents are added, lessening the advantage of

16

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

WWW3D and the Web Planetarium 2 ●●●●●●●●●●●●● ● 1 ●

the displayed web structure information and making it harder to browse. Even though WWW3D makes extensive use of level-of-detail (LOD) operations for individual nodes, there will still be a point where the world becomes too complex to be rendered on even the most powerful hardware. For this reason we created a new version of WWW3D running on the MASSIVE-2 CVE (Benford et al., 1997a) that attempted to solve the display problem by making more extensive use of LOD effects combined with clustering of web pages. We decided to extend the use of LOD beyond the contents of a single web page to encompass groups of web pages (e.g. all pages on the same server, same domain name, owned by the same user etc.). Doing this meant that instead of showing all the pages in a website, or all the personal pages (home page and related pages) for a user, we could display an object that represented the whole group. The user would only see the contents of the group when a certain criteria was met – in this case getting close enough to the group. This meant that when a user was outside a cluster their client had no need to know anything about the web pages in the cluster or the users that were currently browsing web pages in the cluster. This would have the following advantages: ● ● ● ●

A reduction in the visual complexity of the world. A consistent metaphor since clusters looked and behaved similarly to individual web pages when users entered them. Increased interactivity since the computational load on the client machine was reduced. Reduced network bandwidth since MASSIVE-2 assigned separate multicast communication channels to each cluster. This meant that a user who could not see the interior of a cluster had no need to receive network updates concerning objects or users contained in that cluster.

We used a simple scheme based on URLs to perform our clustering. A more advanced alternative would be to explicitly consider the legibility of the CVE and cluster accordingly as is done by LEADS (Ingram and Benford, 1995) which is also capable of adding additional objects such as landmarks to aid users navigating through a CVE. Our new implementation was based on two fundamental building blocks, containers and links. Containers have the ability to enclose other objects in a spatial sub-region of the virtual space. Links connect the containers. A special form of container is the page, which represents a web page and its contents. A cluster is a container that is able to treat its contents as a sub-visualisation complete with layout, other containers and links. Figure 2.4 shows a cluster both from a distance and as the user gets sufficiently close to see the contents. In order to simplify the representation of links between clusters, we introduced compound links. A compound (or aggregate) link between two clusters is formed whenever there exists one or more links between the

17

Inhabited Information Spaces

a

b Figure 2.4 A cluster abstraction dissolves to display its contents when the user approaches: (a) the exterior of the cluster; and (b) the view seen by a user as they get closer to the cluster.

18

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

WWW3D and the Web Planetarium 2 ●●●●●●●●●●●●● ● 1 ●

A

B

c

f d AB g e

011

011

Figure 2.5 A compound link (AB) between two clusters (A, B).

contained objects of the two clusters. Thus, a compound link both serves as a link structure between two clusters and as an abstraction for one or more lower-level links between the contents of these clusters, this is illustrated in Figure 2.5. Figure 2.6 shows a collection of linked clusters. When refined, the current application structure approach should lend itself to parallelisation: letting different processes handle different clusters. This in turn would enable a distribution of calculations on different hosts to allow more complex visualisations.

011

011

11

Figure 2.6 A set of interconnected cluster abstractions. Their sizes roughly reflect the number of objects (here pages) contained within.

19

Inhabited Information Spaces

2.5 The Web Planetarium: Creating a Richer Visualisation The Web Planetarium extends the WWW3D concept with the idea of the system becoming a gateway between different virtual spaces. The metaphor of a planetarium was used to create a new, more informative and more visually appealing look and feel, since the system was also going to be used in public settings using a large, dome-like display (Schwabe and Stenius, 2000). Nodes in the Web Planetarium display can represent either web pages, 3D models (in VRML or DIVE format) or 3D DIVE worlds that can be jumped to. The Web Planetarium can therefore also act as a space of portals to other spaces and provide a means of navigation between different virtual worlds. Compared to WWW3D the Web Planetarium provides a more visually interesting virtual space and different layout options.

2.5.1 Visual Differentiation of Nodes One problem with the WWW3D visualisation is that all nodes look alike. The only distinguishing feature is colour and that colour represents the time since the last access and not a feature of the node itself. When creating the Web Planetarium, it was felt more important for users to be able to quickly scan the space for potentially interesting pages than to show the usage information. The Web Planetarium replaces the use of colour by texture maps extracted from the web pages. Alternatively, the sphere representing a node can be replaced by a 3D model. Not only does this result in a more interesting environment but it allows the users to dinstinguish more easily one node from another and gain some idea of what the node represents. Figure 2.7 shows an example Web Planetarium view in which the texture maps clearly distinguish nodes from one another. A simple algorithm is used to select the image to texture onto a sphere – the first image found in a web page is used as the texture. In the case of a user’s home page this image is typically a photo of the user – this is easily visible in Figure 2.8 in which several of the spheres clearly show people’s faces. Obviously, a simple algorithm such as this will sometimes fail to produce an interesting image for one of several reasons. ● ● ●

20

There is no image on the web page. A banner advert is selected instead of an image representative of the content of the page. The image selected is a piece of web decoration (a line, a bullet point) instead of something really representative of the subject of the web page.

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

WWW3D and the Web Planetarium 2 ●●●●●●●●●●●●● ● 1 ●

011

011 Figure 2.7 The 3D layout alternative emphasises the concept of a planetarium.

011

011

11

Figure 2.8 A screenshot of the Web Planetarium. Compared to the original WWW3D, one of the obvious differences is the use of texture mapping to give more information about the contents of web pages instead of a simple colour indicating the time since last access.

21

Inhabited Information Spaces

It would be hard, probably impossible, to produce an algorithm guaranteed to work, but there are a number of simple filtering operations that could be used to ensure that in many cases unsuitable images are rejected. ●

●

●

Reject images that are too small (according to predefined parameters) or are long and thin. This extremely simple approach was used successfully in the CWall electronic notice board (Snowdon and Grasso, 2002) and rejected most web decoration . Use a blacklist of URLs and URL components to reject banner advertisements or configure a proxy server that performs this function (Hood, 2000). There are websites that maintain such blacklists (@Man, 2000) so each user would not have to shoulder this burden. Reject images that do not come from the same server as the web page. This is a very simple approach that will work in many cases, but it may sometimes reject legitimate images.

An alternative approach could be to use a conventional web browser – embedded within the application or running in parallel – to generate a snapshot of the web page, including background colour, text layout, images and so on. The snapshot would then be used as an icon (or “thumbnail”) to be pasted on the outside of the 3D page representation. This method would have the advantage of creating a stronger connection to the original design of the web page as a whole.

2.5.2 The Web as a Road Network WWW3D and the Web Planetarium can generate very tangled and convoluted visualisations of nodes. Although users can manually reposition nodes this is not a solution that can be applied to large spaces. The Web Planetarium offers users the options of restricting the FDP layout algorithm to a horizontal plane, thereby giving the layout shown in Figure 2.9. A disadvantage of this approach is that in certain cases the FDP algorithm may take longer to converge.

2.5.3 Hybrid Browsing The original aim of WWW3D was to provide an acceptable and novel web-browsing experience using an entirely 3D interface. This made an interesting demo and made sense on a large screen or head-set based system but is less useful on the more common desktop-VR systems in which the VR system is just another window on the user’s 2D desktop. The Web Planetarium therefore allows users to combine the 3D view of the web structure with a traditional web browser view of the contents of a web page. This is illustrated in Figure 2.10 which shows a Web Planetarium view and a web browser (Netscape) view of the same web page side by side.

22

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

WWW3D and the Web Planetarium 2 ●●●●●●●●●●●●● ● 1 ●

011

011

Figure 2.9 The 2D layout alternative results in a road-like network of sites and links.

011

011

11

Figure 2.10 Using a conventional web browser as a side-viewer for HTML documents encountered in the Web Planetarium. Whereas the 3D view of the HTML document (to the right) gives an overview of the content and interaction points for links, Netscape provides a detailed close-up of the current page (to the left).

23

Inhabited Information Spaces

2.6 Conclusion This chapter has described WWW3D, a simple 3D web browser and its evolution into the Web Planetarium. Starting from a humble beginning WWW3D has been changed to address the issues of scalability and producing a more pleasing and informative visualisation. In addition it can now be seen as providing a means for navigating between disjoint 3D worlds as well as between web pages. However, the current version of the Web Planetarium is not without problems. One major issue is related to the viewing of 2D standard web content. Neither the current 3D view nor the separate browser and 3D views side by side really provide a convenient browsing experience. There is still work to be done to find the optimal way to provide information about web structure and a view on web page contents in an integrated way. The second problem with the Web Planetarum is that while it is perfectly possible for multiple users to share a single instance of the Web Planetarium there is no support given to the interrelation of different planetaria nor to different views on the same planetarium. The use of subjective visualisations (Jää-Aro and Snowdon, 2001) may provide a technique for merging planetaria since the same web page could be presented differently to different users, thus preserving individual layouts while allowing a degree of interaction between users.

24

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

PlaceWorld, and the Evolution of Electronic Landscapes

3

Steve Pettifer, Jon Cook and James Marsh

3.1 Introduction The eSCAPE project (ESPRIT Long Term Research Project 25377) set out to investigate the idea of an “electronic landscape” (e-scape): large-scale inhabited information spaces represented as a three-dimensional (3D) virtual environment (VE) in which users could interact with applications, find routes to other virtual worlds, access sources of data, and interact with one another at a social level. The project brought together computer scientists, artists, social scientists and graphics designers from Lancaster University, the University of Manchester, the Swedish Institute of Computer Science and the ZKM (Zentrum für Kunst und Medientechnologie – Centre for Art and Media Technology) in Germany. This chapter describes the evolution of PlaceWorld, an example e-scape. It is organised in four sections. We begin by chronicling the developments of the eSCAPE project that led to the implementation of PlaceWorld as its final demonstrator. We then consider in more detail the nature of PlaceWorld itself, and discuss how its development has influenced the underlying system technology. We conclude by tying together some of the technological issues with concrete examples of their use in the PlaceWorld landscape. The vision of an electronic landscape is easy to articulate, but difficult to make real. Distributed 3D virtual environments of any kind push the limits of today’s graphics, networking and processing technology; some of these issues will be considered in detail much later in the chapter. Add to these problems the social factors involved in making a virtual environment actually usable by the general public, and the issues become much more complex. The eSCAPE project’s approach to these difficulties is described in this chapter.

11

25

Inhabited Information Spaces

The project set out to address two problems. First there was the technological difficulty of integrating diverse environments and their applications. There is as yet no accepted standard technology for building a VE, and though the majority of implementers end up using OpenGL to render the visual part of the world at some point in their implementation, the sources of semantics and behaviour vary wildly from bespoke handcrafted pieces of code to those using higher level more generic systems such as MASSIVE (Greenhalgh, 1999), DIVE (Chapter 12) or DEVA (Pettifer et al., 2000). We will return to consider the way in which the DEVA VR system evolved to support electronic landscapes later in this chapter. The second, and much more open-ended, issue is that of how to make an environment really inhabitable and of practical utility to a user, and it is to this that we first turn. The methodology used in the eSCAPE project to develop effective electronic landscapes was iterative. Inspiration for the environments was drawn from two sources: artistic vision and ethnographic study of suitable real-world situations. The environments were then implemented and put in front of the general public for testing. Further ethnographic studies were then carried out on the installations, and the environments were refined and redeveloped in the light of the results from these studies. PlaceWorld combines lessons and technological perspectives discovered during the three-year period of this process.

Figure 3.1 Snapshots of PlaceWorld.

26

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

PlaceWorld, and the Evolution of Electronic Landscapes 3 ●●●●●●●●●●●● ● 1 ●●

Figure 3.1 shows snapshots from the final implementation of PlaceWorld, and will serve for now, with the brief description that follows, to give an overall flavour of the environment. Users entering PlaceWorld find themselves in an exotic landscape containing many artefacts. Situated here and there are cylindrical “buildings” covered with images from other worlds: these are portals that lead elsewhere. A network of pathways, hovering above the mist-covered landscape lead off into the distance in all directions. The scene is populated with towers, billboards, flying craft and other miscellaneous structures, and in the sky ethereal floating ribbons of light drift back and forth. Other users move around the landscape, represented by animated walking stick figure-like characters, and leave behind hazy coloured trails. Interaction with any of these entities brings up on screen a selection of possible actions. Touching a user’s trail, for example, even when that user is a long way away, reveals that it can be used as a shortcut to find its owner. Touching any other artefact allows it to be “picked up” and carried around, to be deposited later at another location. The remainder of this chapter explains why the esoteric nature of PlaceWorld is as it is, beginning with the project’s initial experiments with electronic landscape .

011

011

011

11

3.2 Background: The Physical and the Abstract eSCAPE examined two extreme styles of virtual environment, to extract from them appropriate features that would make an environment inhabitable. At one extreme, there was the physical or concrete “cityscape”: familiar, constrained, slow to change, easily understood. At the other extreme, the “abstract landscape”: unfamiliar, dynamic, unrestricted. Aspects from both these styles of environment were finally integrated into PlaceWorld. Virtual environments have the potential to include aspects of both these extremes: there is no technological reason why one should be confined to a groundplane in the virtual world just because gravity works in a particular way in the real world; there is no technological reason why one should be forced to walk from one part of a virtual world to another rather than simply “teleporting” to one’s destination, much like clicking on a link in a web page. On the other hand, it is easy to imagine getting completely disorientated in a space where there are no constraints on navigation, or of missing out on chance meetings and opportunistic discoveries if one’s only means of transport is by instantaneous hyperlink. Thus some balance needs to be achieved between these two extremes of physical and abstract. The evolution of the project is easier to tell from the perspective of the concrete cityscape metaphor rather than the abstract, simply because of familiarity with its nature and terminology. The following vignettes chronicle various attempts at building electronic landscapes, and lead towards the final design of PlaceWorld.

27

Inhabited Information Spaces

3.2.1 Watching a Cityscape The first environment built during the project was a straightforward cityscape (Figure 3.2), consisting of buildings, roadways, parks and street furniture generated by algorithms based on Hillier and Hanson’s (1984) studies of urban evolution. A set of ethnographic studies were completed to see how users behaved in this environment, setting the participants the task of finding a number of landmarks in the city such as fountains, statues or pavilions. Studies were carried out at a desktop computer, using a standard mouse as an interaction device. The results were in some ways surprising. In spite of the user being a lone inhabitant of the cityscape, and without prompting from the scientist, wayfinding in the city was almost invariably carried out on the pedestrian areas (pavements, parks etc.) in spite of there being not a single car in sight. Though there was no collision detection preventing passage through the buildings, users mostly followed the road system. Even though the mouse navigation included an ability to fly up in the air (a feature that the participants were told about, and which clearly would give them an advantage in terms of spotting land marks from on high), this was rarely used (indeed, one participant who did briefly levitate to a suitable

Figure 3.2 A virtual cityscape.

28

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

PlaceWorld, and the Evolution of Electronic Landscapes 3 ●●●●●●●●●●●● ● 1 ●●

vantage point rapidly came back down to ground level having felt uneasy up in the air). Although this evidence was anecdotal rather than definitive (and there were a small number of exceptions in terms of behaviour), the studies suggested that even with little effort by way of interaction device to induce the sense of presence, that the general affordances of a real cityscape, transferred rather strongly to the virtual world. A more detailed description of the study can be found in Murray et al., 2001. On this basis, a more ambitious city-like environment, the Distributed Legible City, was built.

011

011

011

011

11

3.2.2 The Distributed Legible City The artist Jeffrey Shaw, a partner in eSCAPE, had once developed an artistic installation piece called the Legible City. In this work, a solitary cyclist, seated on a real bicycle and situated in a darkened room with a large projected wall, could cycle their way around interpretations of the cities of Amsterdam, Manhattan and Karlsruhe. Rather than ordinary buildings, the cities were populated with towering, solid letters, forming extracts from texts associated with the areas of the cities. For example, a virtual tourist in the Legible City could follow the ramblings of a taxi driver around Manhattan, or find sections of poetry in Amsterdam. Informal observation of the piece, which is still in situ in the ZKM in the real Karlsruhe, showed that, even with now outdated rendering technology and simple interaction style, the piece would capture people’s attention, and they would happily tour around the cities for some time, often trying to find out which piece of text was associated with their favourite part of the world. For entirely opportunistic reasons (much of the technology already existed and the piece was known to be “engaging”), the project decided to extend this installation to form a distributed environment, and so the Distributed Legible City (DLC) was constructed (Figure 3.3). Using consumer-level graphical accelerators, the original database of city structures was reused, and modified exercise cycles were pressed into action as interface devices. Rather than the large screen projection system used in the original work, stands for monitors were built in front of the cycles, and where the first Legible City used an LCD panel attached to the handlebars to display maps of the cities, in this new version the map was displayed virtually as an overlay on the monitor screens showing now the layout of the world as well as the positions of other inhabitants. For the purposes of communication, the users were provided with headphones and a microphone. Other users were represented by animated cycling avatars. Two versions of the DLC interface bike were situated around the ZKM, and another across a wide area network in Vienna, and the installation was made available to the public during a number of exhibitions.

29

Inhabited Information Spaces

Figure 3.3 Images from the Distributed Legible City.

For several days, the technology behaved perfectly; however, the overall result of the piece was extremely disappointing. No longer did users engage with the environment and there was no interest in exploring the world for its own sake. Quickly, users realised that the buildings offered no resistance to them cycling through them, and the overlaid map was discarded, with users making a beeline towards one another and completely ignoring their surroundings. Once users had found each other, another disappointing interaction took place. Unexpectedly, users were resolute that they should be facing one another in order to carry out any conversation (this was not a requirement on the audio layer, which had no directional bias). The positioning of the monitor in front of the cycle, and the inability to pedal backwards meant that a significant amount of effort was put into achieving “conversational orientation”, cycling round in ever decreasing circles until this was achieved. With this effort expended, the users exchanged little more than a few perfunctory greetings, and then left the installation. Our expectations of multi-user cycling tours of the cityscape were far from met. With hindsight, the problem was one of a mismatch between the affordances of the environment, and the expectations of the users. First, the main novelty of the environment to its user turned out to be the potential to find another inhabitant, and perhaps then to tour the world.

30

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

PlaceWorld, and the Evolution of Electronic Landscapes 3 ●●●●●●●●●●●● ● 1 ●●

Flashing arrows on the overlaid map made it clear that other users were around somewhere, and once users realised that the “buildings” did not block their attempt to cycle through them, the map became a meaningless diversion, which was not returned to as a useful navigation aid later (in spite of its apparent successful application in the original piece). Similarly, on a real bike one would expect to be able to cycle side by side with a friend, or at least to look over one’s shoulder to carry out a conversation with a cyclist behind. The fixed position of the screen in front of the cyclist, and the difficulty of arranging conversational orientation with these limitations, made casual touring and chatting unrealistic. To solve these problems, two simple modifications were made to the installation. First, the map was discarded (the alternative of forcing the utility of the map by making the buildings impenetrable seemed unproductive since the cities were large and users simply wanted to find one another in the first instance). As a replacement, a bird was positioned in the sky above each participant, which would fly in the direction of the nearest other user, meaning that now finding one another was straightforward: simply follow the bird. The most significant alteration was the introduction of a tracked head mounted display. The original fixed monitor was retained so that bystanders could see interactions within the environment; however, the cyclist was now able to look around in the environment, and to see the world in a different direction from that in which the cycle was facing. The improvement to social interaction with the new version of the DLC, this time tested out at ESPRIT IST98 in Vienna, was dramatic. With the ease of finding other users and of achieving a convenient orientation for comfortable communication, participants were much happier to spend extended periods of time exploring the cities together. The conclusion at this stage was that cityscapes provide easily understood metaphors for interaction, but these metaphors must be matched with the actual affordances of the environment and its input devices with great care.

3.2.3 Finding “Something to Do” For all the lessons learned from the Distributed Legible City, it was far from being an e-scape: it lacked interaction with other environments, and had no “information” as such beyond the possible interest in the cities’ texts. A more concrete cityscape was built to address these limitations. An ethnographic study of a real-world tourist information centre, in the northern England seaside town of Morecambe was carried out, concluding that it was an interesting enough location of social interaction and information browsing to warrant further attention. In particular the question of “What can we do here?” seemed as relevant to a tourist turning up in a new holiday location as it did to an inhabitant of a new electronic landscape. A virtual Tourist Information Centre (TIC)

31

Inhabited Information Spaces

was constructed to see in what way the behaviours observed in the real world TIC were transferred to the virtual landscape, and studies of this new environment were carried out in situ. The virtual TIC (Figure 3.4) was based on a map of Morecambe, and included 3D representations of the various tourist attractions. The map and its contents distorted dynamically to bring objects or areas of interest to a particular user into view, according to search criteria. Once more, users tended to follow city-like conventions in their exploration of the virtual world, and there was evidence that to some extent the environment supported social interaction and exploration. Its limitation, however, was clearly one of content. The hand crafted nature of the virtual Morecambe and the relatively small number of attractions represented within the world meant that its real potential as a source of holiday ideas was nowhere near reached. More importantly, it was also obvious that it was unrealistically costly for any individual to attempt to populate such a world. From this it was concluded that an important aspect of any worthwhile e-scape was the ability for its inhabitants and stakeholders to straightforwardly introduce their own content and to modify existing artefacts to reflect their own experience.

Figure 3.4 A virtual Tourist Information Centre, showing the deformable map and buildings.

32

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

PlaceWorld, and the Evolution of Electronic Landscapes 3 ●●●●●●●●●●●● ● 1 ●●

3.2.4 Abstract Influences: Nuzzle Afar In parallel with the thread that has been described so far, informed very much by the “concrete” and “physical” end of the e-scape spectrum, a number of other studies and sub-projects were being explored under the eSCAPE banner. One of these in particular: a multimedia installation of an abstract nature called Nuzzle Afar had a significant influence of the nature of navigation and wayfinding in PlaceWorld. Nuzzle Afar, a multimedia installation by Masaki Fujihata, consisted of an enclosed room, with left and right side entrances. Within the room were two podia, in front of which were projection screens. Trackballs embedded on top of the podia allowed users to control movement through the computer-generated environment displayed on the projection screens, and microphones similarly located allowed distributed users to communicate. As users moved through the electronic environment, a string-like trace is left upon the virtual landscape, which may be locked onto and followed by another user. The enclosed space of the virtual world consisted of four walls, ground and sky plane. Upon the walls were images of “sense organs” (e.g. a hand, an eye, an ear, etc.). In addition to spherical “avatars” of unique colour, a sphere and a cylinder are placed within the virtual room. These latter objects were the means by which users could enter or depart a series of three rooms. Within each of the spaces in Nuzzle Afar navigation needs to be learnt anew: effort, space and travel have different relationships within each. When two or more users are in close approximation “within” any of the spaces, they are able to see a video image of one another mapped and wrapped, visor-like, around the middle of the sphere. This allows for recognition of the others’ identity. In meeting each other, any two users are able to enter a new virtual space which encompasses them, while locking out the previous environment and any other inhabitants. This new space is, however, visible to other users as a spherical object inside of which the colours of the two users “inside” merge. Once inside this new space, users are represented via their video images on a 2D square. When one or both of the users leave this space, a video still of the two users remains, along with details of the time and physical locations of the encounter. In spite of the very abstract nature of the Nuzzle Afar world, its means of wayfinding and of locating other users proved much more intuitive than our initial experiments with maps and other metaphors borrowed from the cityscape metaphor. The influence of this installation on the final implementation of PlaceWorld is described in the following sections.

3.3 PlaceWorld PlaceWorld became the final eSCAPE demonstrator and assimilated much of the work done in the earlier parts of the project. It aimed to

33

Inhabited Information Spaces

become an effective “place where places meet” building on the lessons learnt from studies of the smaller scale landscapes. Another multimedia installation piece by Jeffrey Shaw formed the basis for its artistic design, and a novel user interface device was commissioned for the purpose of public display of the new environment. The next sections describe the vision and rationale that underpinned the world.

3.3.1 The Design of PlaceWorld The design of PlaceWorld took as its point of departure many features of Jeffrey Shaw’s 1995 artistic work “PLACE – A User’s Manual” (Shaw, 1997). The paradigm that is embodied in both implementations is that of a navigable virtual environment that is populated by cylindrical artefacts that form portals into other virtual environments. The world is also characterised by pathways that provide identifiable routes between these places. As boundary architecture, cylindrical space is quite different from more typical rectangular spaces: the latter is perceived as an enclosure, whereas the former constitutes what is in effect a scaleable panoramic horizon. This aspect of the virtual cylinder is extended into the real world by the panoramic navigator interaction device described later. Cylindrical spaces are especially appropriate within PlaceWorld because they allow all the idiosyncratic virtual environments to define their own unique enclosures within a neutral and scaleable circumference. Despite the apparent exterior uniformity of these cylinders, attaching images to their surfaces offers an expressive way to signal their interior contents. In this manner, rather than by architectural variation, they constitute a characteristic and very expressive visual syntax for PlaceWorld. This cylindrical uniformity does not preclude the possibility to scale them variously in height and width, or to work with the relative transparency of their surface, and this allows further expression of spatial as well as temporal meanings as will be described later. From this initial vision, a graphic designer developed a design specification that took the earlier PLACE work, and extended it with new features that addressed the particular needs set by the objectives of an electronic landscape. One important aspect was the fact that an everyday urban landscape is characterised by the presence or absence of its population of citizens. PlaceWorld also socialises its unusual landscape with representations of its visitors and occupants, but in accord with the inherent properties of this information space, these avatars have unique design characteristics that enable them to effectively function as true citizens of PlaceWorld. A visitor in PlaceWorld would be expected to orient themselves to a large extent by the visual identities of the cylinders themselves – that is, by the individual images that would be textured on the outer surfaces of

34

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

PlaceWorld, and the Evolution of Electronic Landscapes 3 ●●●●●●●●●●●● ● 1 ●●

these cylinders (a form of advertising their interior contents) and the general stability of the layout. Much as in the real world, it seemed useful to also establish some method of signposting that could offer and guide visitors to the various locations. Such signposts could also be set up by persons who had built a place in PlaceWorld as pointers to their location. The issue of the relative permanence of the overall geography of PlaceWorld and its cylindrically bounded environments was felt to be an important issue if regular visitors to PlaceWorld are to be expected to readily gain familiarity with it. On the other hand, the ability to introduce new content to an environment means that some controlled modification of the world must be possible, as was made clear by our experiments with the virtual TIC. A description of the access model – a set of malleable guidelines and policies that could develop over time under the influence of the inhabitants of the world – was developed to support coherent and manageable changes to the underlying structure. A brief description of the access model is given later in this chapter. The issue of balancing familiar structure with the desire for modifiable content formed a fundamental feature of the temporal nature of PlaceWorld’s landscape, and reflects once more the desire to combine appropriate aspects of the concrete and the abstract. Masaki Fujihata’s project Nuzzle Afar provided an exemplary demonstration of the value and effectiveness of attaching traces to users’ movements through a virtual environment. Colour coded for each visitor, these lines gave temporal evidence of people’s presence and their paths of interest. And the added feature that a visitor in such an environment could “hook on” to someone else’s trace and then get a roller coaster ride along that convoluted line that brought him face to face with that other person was also very effective and enjoyable. However, further discussion of this methodology revealed a basic weakness – as soon as there would be a large number of visitors in the virtual environment these lines would become an unintelligible tangle. Because we had decided to restrict movement to ground level in PlaceWorld, such traces could then only be drawn in one plane, thus adding to the confusion if there were many visitors. On the other hand, the restriction of these markings to one plane revealed the possibility of another very interesting design strategy. We realised that these traces could in fact be merged with the geographical and physical articulation of roadways in PlaceWorld. These roadways could be programmed to dynamically and temporally express the volume of their usage, being a virtual reflection of “desire lines” in the real world that are created, say, when pedestrians create their own shortcuts across grassy areas in parks. The hierarchy could be say trail, path, road, highway, which would be defined by the density of traffic experienced along these routes, and would change according to the varying density. In this way a new feature of informative comprehensibility could be added to PlaceWorld: the nature of the pathways would themselves express a history of visitor interests, so one could, for instance, choose

35

Inhabited Information Spaces

the well-trodden paths or venture along trails into relatively unfrequented regions, or forge a new trail into previously unexplored territory. Following this line of thought about the informative value of temporal signifiers in PlaceWorld, the notion was extended to the cylinders themselves, which expressed the temporal condition and/or the popularity of their contents. As with the tourist information centre, cylinders that are of interest or frequently visited were designed to increase in scale over time, with those of little interest shrinking into obscurity. A point of debate was the notion of hyperlinking the individual cylindrically bounded environments so that a visitor could “teleport” instantly from one related place to another without having to negotiate the PlaceWorld landscape. A more radical extension of this facility would be that a visitor could at will completely reorganise the distribution of cylinders in PlaceWorld to suit a specified personal need or desire (e.g. he only wants to visit the popular places, so these all become clustered around him and the others disappear). However, going back to the premise of PlaceWorld as a physical electronic landscape that should be strongly anchored in real world physics and experiences, it was felt that such hyperlinks and hyper-rearrangements should be kept to a minimum, since they were not entirely consistent with the “physical landscape” objectives set for this environment and shown to be effective in the previous e-scapes. As a compromise, the ability for a user to generate a hyperlink between places that they had already visited was made available, with the link being visible only to that user as a glowing trail in the sky, anchored at both ends on the groundplane.

3.3.2 The User Interface and Presentation System The Panoramic Navigator (PN) is a patented technology developed by Jeffrey Shaw and the ZKM whose initial function was to provide a panoramic method of allowing visitors to the ZKM to preview and explore the contents of the ZKM building. Using a 360-degree rotatable touch screen coupled to a video camera, this augmented reality approach enables visitors to the ZKM to look around the building (via the live camera image) and use the touch screen to access additional multimedia information that is attached to specific locations in that building. While these past implementations of the PN have used the real environment as its frame of reference for interactive information delivery, it was realised that this technology could also be an ideal interface to explore and interact with wholly virtual environments, and that this would be eminently suitable for PlaceWorld. What was needed was a method of generating a representation of that virtual environment around the PN, so that the touchscreen could then be used to interact with it. The design solution for this was to attach a video projector to the back of the PN (behind the touchscreen) and to put a circular projection screen

36

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

PlaceWorld, and the Evolution of Electronic Landscapes 3 ●●●●●●●●●●●● ● 1 ●●

011

011

011

011

11

Figure 3.5 The Panoramic Navigator – Augmented Virtuality version, in use with PlaceWorld.

around the PN. In this way a new presentation and interaction method which embodies the innovative concept of augmented virtuality (PN-AV) was developed. This PN-AV technology is used in PlaceWorld to embody two distinct but interrelated functions: ●

●

to display the 360-degree immersive representation of virtual environment, wherein one also see the effects of user interaction with that environment. to give the user a touchable interface that allows him/her to explore and interact with that surrounding virtual environment by showing a parallel representation of that environment which is augmented by a dynamic set of user interaction tools.

The final version of the PN-AV, in use during a public demonstration, is shown in Figure 3.5. Having described the evolution of the PlaceWorld environment, we now turn our attention to the problem of its actual implementation, and describe the development of the DEVA VR system, which grew in parallel with the e-scape demonstrators. We will return to how aspects of PlaceWorld were supported by this technology at the end of this chapter.

3.4 Technological Challenges for Electronic Landscapes Until recently, technological limitations were the clear brake on progress for VEs (the graphics challenge is perhaps the clearest example). Beyond the technology the key task is that of writing software. The scale and complexity of this task is often underestimated, and it is here where we believe that the major problems lie. We do not at this time have adequate frameworks to simplify the task of implementing virtual worlds. Today, a person wishing to implement a challenging VE application has two broad options, as evidenced in current demonstrations. The most

37

Inhabited Information Spaces

sophisticated VEs are usually bespoke applications constructed from the graphics layer upwards; this is a substantial undertaking. The alternative is to use the VE system. These fall into different categories, (for example, VRML browsers at the lower end, up to VE systems such as DIVE and MASSIVE at the more ambitious extreme). The software support challenge to facilitate large-scale VE applications is twofold. First, to find techniques and algorithms to address specific needs in VEs. Collision detection, parallelism, distribution, synchronisation, navigation and so forth all require work of this kind. Secondly, and perhaps rather harder, it is to find frameworks that allow all the parts to be put together in “flexible yet powerful” ways. The rather trite nature of such a statement belies the difficulty of quantifying that task. Finding the “right” framework is particularly difficult in the case of virtual reality (VR), since it brings together a number of complex technical issues, and binds them with real-time constraints at the social/ perceptual interface. A desirable approach for the necessary flexibility to experiment is to build as little into the system as possible, so the system provides a set of mechanisms and policies and default behaviours that can be unplugged and tailored at each level. The issue of scale is an important one; simple small-scale VEs that do not challenge today’s hardware can be constructed by any number of means. Building large-scale complex applications raises a number of challenges. For shared VEs these relate, broadly speaking, to the following areas: number of entities; complexity of behaviour required of these entities; complexity of individual rendering techniques; number and geographical distribution of simultaneous users; and number of co-existing and interacting applications. Matters of scalability and synchronisation, and some architectural and network topologies for achieving this are discussed in Chapter 13. Here we present the approach implemented in the DEVA VR system, and arising from the observation that for a shared VE there is a natural distinction between the users’ perceptions of the world and what is actually “going on” within it. That is, between the distribution and simulation of application behaviours on the one hand, and the task of presenting a coherent perception of the VE to each user on the other. This is similar to the traditional philosophical distinction between subjective perception and the underlying objective reality. The reason that this is useful is that in VR it separates out the challenging graphics/interaction tasks from the semantics of the underlying world simulation. The distinction also legitimises efforts to make a perceptually smooth presentation of the world in the light of fundamental networking limitations. Architecturally we have addressed these two aspects of the VE (perception/reality) separately. First, rendering and spatial management seem to need special treatment that is different from current approaches to graphics toolkits. The MAVERIK system aims to address these issues. The unusual approach taken by the MAVERIK system is to avoid having any internal

38

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

PlaceWorld, and the Evolution of Electronic Landscapes 3 ●●●●●●●●●●●● ● 1 ●●

“structure” for the representation of VEs and their contents. Instead, an object-orientated framework is provided that supports an application builder in implementing rendering, interaction and spatial management routines that are tailor made and appropriate to their particular purpose. This is as much a performance issue as programming elegance, for key graphics optimisations are highly application specific, and are generally unavailable when the application must export its representation into the VR system. The system is described in detail in Hubbold et al. (2001). The more difficult problem is that of defining the underlying behaviour of entities in a way that can be distributed to multiple users. It is to this problem that we now turn our attention.

3.4.1 Synchronising the Behaviour of Entities The distribution of semantics or behaviour in shared VEs is a particularly difficult issue. The ideal solution would be to describe the required behaviour in a single location and to have it instantaneously sent to all participants (a pure “client/server” approach). Current network technology is far too limited to make this solution feasible. An alternative is to replicate behaviour locally for each participant (a pure “peer-to-peer” approach); this introduces extreme synchronisation problems. The approach taken in DEVA is a hybrid solution, part way between client/server and peer-to-peer. The model is one of an objective and subjective reality; the former being located (logically) on the server, the latter being represented on each client. Each user interacts with the objective reality via their own subjective view. This introduces the idea that each entity in the virtual environment has two definitions, which may differ significantly in their semantics. Our usage of the term “subjective” is not intended to imply that each user experiences an entirely different VE (this would hardly count as a shared experience). Rather, we argue that an amount of the user’s experience may be decoupled from the “objective” behaviour of the world without disturbing consequences. Given that absolute synchronisation is in any case impossible, it is our contention that so long as the length of the delay is not too large, and causal events occur in the correct order, it is possible to accept a degree of subjectivity without it affecting the users’ understanding of the semantics of the application. Under these circumstances, given no other frame of reference the users are unlikely to even realise they are not receiving the same view of the world as each other (as will be described later, PlaceWorld demonstrated this phenomenon in extreme ways). In general it would therefore be useful to differentiate between what is “actually” happening in a VE and what the users perceive. In this way it is possible to optimise the manner in which information is transferred to minimise causal discrepancies. In the DEVA system this notion is implemented by describing an entity as comprising of a single “object” and a number of “subjects”, the former

39

Inhabited Information Spaces

part being the “objective non perceived” aspect of the entity and the latter being a collection of the “subjective perceivable” characteristics (e.g. its visible, audible, haptic parts). An alternative view of this distinction is that the “object” represents what an entity does, while its “subjects” represent what the entity looks/sounds/feels like when it is doing it. Communication between an object and its subjects can be implemented using whatever high-level “vocabulary” is appropriate. In this way the system minimises the need for strict synchronisation while maximising the accuracy of causally important events. One example of this is the use of twines (Marsh et al., 1999). Twines are lightweight parametric curves that are used to “smooth out” spatially and temporally the discrete updates to the visible aspects (e.g. position and rotation) of appropriate entities by interpolation thus increasing their “plausibility” by eliminating disconcerting jumps. We are also currently investigating an infrastructure for providing “quality of service” based on balancing such smoothing against frequency of updates depending on the perceived importance of various events. The flexible nature of the communications is also advantageous for subject-to-object communications; for example when a user “grabs” and moves an entity. Such manipulation with a classic client–server architecture would involve a round trip: the event is sent from the client to the server to be processed and then the effects of the event are then transmitted back to the client to be visualised. With the commonly available network technologies we are targeting for the client–server connections such a round trip would introduce an unacceptable lag. It is perceptually important for the cause and effect of the manipulation to be as tightly coupled as possible. With the separation of subject and object, changes caused by the manipulation can be immediately perceived by the subject, with a userdefinable policy function being used to determine when the object is updated. For example, an arbitrary fraction of the changes (say, one every fifth of a second) are transmitted back from the subject to the object and thus onto other visualisers connected to the server. Alternatively, a change in position is only updated to the object when the subject has moved a certain distance from its previously synchronised location. The goal is to ensure that the entity behaves “correctly/optimally” on the visualiser performing the manipulation and also that it behaves “plausibly/acceptably” on any other visualiser connected to the cluster (while accepting that latencies and bandwidth issues rule out strict and absolute synchronisation in such a distributed system).

3.4.2 Distribution and Communications The programming model employed by the DEVA system is one of communicating “entities” which can represent objects in the virtual

40

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

PlaceWorld, and the Evolution of Electronic Landscapes 3 ●●●●●●●●●●●● ● 1 ●●

environment, the properties of the environment itself, or alternatively abstract programming concepts that have no direct representation to the inhabitant of the environment. These entities are coarse-grained programming objects, exporting a number of methods that can be called by other entities, and implementing these internally using optimised imperative code (currently written in C++). The DEVA programming model makes entity behaviour explicit, allowing entities to query one another’s features, those of the environment in which they exist, and to reason about these, rather than having behaviour emerge as an implicit side effect of a piece of code being executed. In this section we describe the makeup of an entity, and the mechanisms provided for enabling transparent and lightweight communication between entities distributed around the system.

3.4.3 Defining the Behaviour of Entities We can identify at least four conceptual sources of behaviour for each individual entity within a VE: 1. Behaviour unique to an entity. Entities each have their own particular role within a VE, for example the ability of a stopwatch to record elapsed time. 2. Behaviour common to a range of entities. Often many entities share some aspects of their behaviour through being of a similar type. For example, each pawn on a chessboard is subject to the same detailed rules governing movement; these rules do not apply in the same way to other pieces which have their own restrictions. However, all the pieces have in common the rules determining whether they may be “captured”; a notion that is specific to chess pieces but not necessarily to other entities in the same environment. Thus a hierarchy of common groups can be determined. 3. Behaviour common to all entities in a particular world. We generally consider gravity to be a phenomenon associated with everything found in the world around us rather than as a property of each individual entity. This also applies to social constructs such as “monetary value”. 4. Behaviour that is dynamically required at runtime. If an entity becomes inhabited by a user for example, it will behave differently – being controlled by a navigation device and so forth – to when uninhabited. Similarly an entity that is “set alight” will suddenly have properties previously unavailable to it (for example, being able to set fire to neighbouring entities or raising the temperature of an environment). Commonly, VR systems define the behaviour of an entity by attaching pieces of code, often written in Java, Tcl or some other scripting language. With little specific architectural support, however, it is often laborious to code and difficult to ensure consistency. The DEVA

41

Inhabited Information Spaces

programming model attempts to improve the situation by taking an object-orientated approach to the definition of behaviour and by providing features such as introspection (the ability to ask an object what methods it has, what parameters they take etc.) and run-time combination of environmental influences with entity-specific behaviour. This essentially allows the traditional inheritance graph to be modified while an entity is “live” in an environment, adding or removing properties and behavioural traits as necessary. Our solution to the problem of merging the various sources of behaviour that comprise an entity is to use “characteristics”. In DEVA a characteristic is a collection of methods and attributes relating to a single concept that can be attached or detached from an entity at runtime. DEVA supports three types of characteristic: 1. Innate: behaviours that define an entity and make it different to those around it, for example its physical shape. 2. Imbued: behaviours that are offered to an entity when it joins an environment, but can be over-ridden by that entity by a more specific innate behaviour (for example, an entity lacking mass in an environment that requires such an attribute, may inherit an imbued characteristic that approximates a default mass from its bounding volume; an entity that has a better idea of what its mass is can provide the functionality itself). 3. Enforced: behaviours that must be processed in order for an entity to conform to an environment’s requirements. The methods and attributes that comprise the characteristic are divided into a single “object” and multiple “subjects” as outlined in the previous section. While typically a characteristic will contain both parts, some characteristics are entirely abstract and have no directly perceivable representation in the virtual environment, that is, they have no “subject” part. The researcher trying out new low-level ideas is free to write characteristics directly in C or C++ that interface to the VR kernel at whatever level is appropriate. More general users are free to use the library of existing characteristics to construct entities without concerning themselves with their implementation. We speculate also that this characteristic system has the added benefit of being usable from an immersive “graphical user interface” allowing the user to “mix and match” characteristics at runtime. In order to facilitate efficiently moving entities between worlds with different behaviours, they contain two lists of characteristics, one inherited from the environment, and the other containing its own innate behaviour. Traditional object-oriented inheritance is supported through characteristics being able to load other characteristics when they are initialised. The order in which the characteristics are searched enforces the correct precedence.

42

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

PlaceWorld, and the Evolution of Electronic Landscapes 3 ●●●●●●●●●●●● ● 1 ●●

Income method call

Deva Entity

011

011

Characteristic 1

method 1 method 2 method 3

method 1 method 2 method 3

Characteristic 2

Characteristic 2

Characteristic 2

method 1 method 2 method 3

method 1 method 2 method 3

method 1 method 2 method 3

Characteristic 3

Characteristic 3

Characteristic 3

method 1 method 2 method 3

method 1 method 2 method 3

Imbued

Characteristic 1

method 1 method 2 method 3

Innate

Enforced

011

Characteristic 1

method 1 method 2 method 3

Optional return valve

Figure 3.6 Structure of a DEVA entity, showing the ordering of method processing.

When a method is called upon an entity, the two lists are searched in strict order. First of all the list of characteristics given to the entity by the environment is scanned for methods marked as being “enforced”. If one with the correct name is found, then this is called. Enforcing methods allows an environment to ensure all entities contain a particular method that cannot be overridden. Next, the entity’s innate characteristics are searched. If the method is still not found then the environment characteristics are searched again for methods marked as “imbued”. These are methods given to the entity by the environment but which can then be overridden by the entity itself. For example, an environment may enforce the notion of “solidity” upon all its entities (“you can’t pass through walls”). It may also “imbue” all entities with a concept of mass with a default value estimated based on its volume, but which the entity is free to override should it prefer. This process is illustrated in Figure 3.6.

011

11

3.4.4 Methods and Filters The strict order in which characteristics get searched for methods leads to another useful concept, that of the filter. As well as methods just returning results, they can be declared to be filters. These are allowed to return a new method call as their result, which then continues to

43

Inhabited Information Spaces

propagate along the characteristic lists. This is useful in a number of situations. Say, for example, it was necessary to constrain an entity to a plane. The entity inherits the imbued method setXYZ from its ThreeDSpace environment. In this case the entity would define a characteristic itself that defines setXYZ as a filter. Since this would be found before the imbued method, the entity’s filter is able to constrain the coordinates to a given plane before returning a new setXYZ message with the new co-ordinates that then propagates through to the environment imbued setXYZ method. Another example would be storing an entity’s state prior to migrating the entity to another server node process or prior to a complete shutdown of the system. Each characteristic defines a snapshot filter that adds any state variables it possesses to the input message and returns a new snapshot message. The final characteristic then returns a restore message containing all the variables that is sent to the new entity. The new entity contains a restore filter in each characteristic that originally stored variables, which then unpacks the message. Continuous behaviour is supported by permitting characteristics to define a special method called “activity” which is polled intermittently. Activities are not subject to the usual calling precedence, but always polled if defined.

3.4.5 The Distribution Architecture DEVA is logically a client–server architecture, which to a first approximation provides a single definitive locus of control for the VE using its server component, with “mirrors” of the entities being maintained in each client process. Behind the scenes, however, DEVA pragmatically manages the delegation of control dynamically to the most appropriate parts of the system, thereby achieving the highest fidelity of perceptual and causal coherency attainable for the application at hand. The “server” is in fact a cluster of processors running identical processes called “server nodes” that together form a single multithreaded parallel virtual machine capable of processing large numbers of entities. The intention is that the server provides a computing resource for multiple virtual environments, and maintains a far heavier processing load than any one user’s client could manage at any one time. A networking layer provides lightweight position independent messaging between entities. Entities are created in and managed by the server node processes, and client processes – such as a visualiser or user application – connect to the server to interact with and obtain state information about the entities. The server is persistent: it remains alive, processing any entities regardless of whether or not any clients are connected. Administrative tools exist to simplify the start-up, monitoring and shut down of the parallel server.

44

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

PlaceWorld, and the Evolution of Electronic Landscapes 3 ●●●●●●●●●●●● ● 1 ●●

Creating and Addressing Entities Each server consists of a (large) fixed number, M, of virtual servers. The M virtual servers are trivially mapped using a lookup table to the N server node processes that comprise the server. This conversion takes place at a low level within the system and essentially hides the configuration of the server. An arbitrary virtual server is chosen to create and manage each entity (currently a random virtual server is chosen but this selection process could take into account loading factors). Each server node potentially manages multiple entities. Each entity is assigned a unique “pool ID” – an offset into the list of entities managed by a given server node. The location of an entity in the server is uniquely defined by the virtual server and pool IDs. This pool ID is not strictly necessary and is provided for efficiency only, since each entity has a unique name that can be searched for in the list of entities managed by the server node. When an entity is created a hash function is applied to the entity’s name to obtain a second virtual server. This virtual server manages the name of that entity, that is, it definitively knows the virtual server of where the entity is actually located and its pool ID. The name of the entity and the entity itself are managed by separate virtual servers. The same hash function is used throughout the system allowing any DEVA process to obtain the location of a named entity. This data is cached for future use. Entity name to location lookup, while a lightweight process, is a central and frequent task in a distributed system. A scheme where this load is spread equally across all server nodes is therefore advantageous. The main advantage of the addressing mechanism is that it allows entities to dynamically migrate across server nodes to help balance the processing load. When an entity moves it only needs to inform its name manager of its new location. DEVA processes – which now contain out of date cached data – will receive an error the next time they try to communicate with the entity at its old location. This error is trapped internally and the new location of the entity is obtained from the name manager; the originator of the communications with the entity is oblivious to the migration. The name manager can always be relied upon to know the correct information and its location is trivially obtained.

Server Reconfiguration It is also possible to migrate the names managed by a given virtual server onto a different server node by updating the virtual server-to-server node lookup tables in every client and server process. The migration of both entities and name management allows server nodes to be dynamically added and removed from a running server.

45

Inhabited Information Spaces

Networking Protocol Currently, standard TCP/IP point-to-point socket communication is employed although some work has been undertaken investigating multicast, since for local area networks at least it promises improvements in performance for our application. Although not a strict requirement, the communications strategy is based upon the assumption that inter-server node communication is fast compared to server–client communication. For example, the server nodes are connected via a dedicated network or protected to some extent from superfluous traffic by a bridge; while clients connect to the servers via a high-traffic shared LAN (local area network) or modem connection.

3.5 System Support for PlaceWorld In this final section, we describe how a number of the artefacts and features of PlaceWorld, informed by the previous multimedia installations and ethnographic studies, were supported by the technology of the DEVA system.

3.5.1 Menus Every entity in PlaceWorld is able to respond to the user in one way or another. When the user touches the entity on the navigator screen, a menu appears showing the available features. Most entities, for example, can be picked up and put in the user’s “pocket”, and in this way moved to another part of the same world, or indeed somewhere completely different. Even entities that were coded before PlaceWorld was conceived have this feature “enforced” upon them by PlaceWorld’s master environment giving a degree of coherence to the interface. Other entities hand-crafted for the world, such as the generators or noticeboards offer more sophisticated behaviour via the “innate” mechanism. Filters are used to cascade through the possible methods provided by each entity, combining the imbued, innate and enforced facilities available to the user and presenting these graphically at the interface.

3.5.2 Access Model PlaceWorld’s access model demonstrates a more sophisticated use of the characteristic architecture. In order to maintain a degree of order in the environment, users are able to specify rules that govern the use of entities they have placed in the world. The mechanism for describing these rules is beyond the scope of this chapter, however the method of

46

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

PlaceWorld, and the Evolution of Electronic Landscapes 3 ●●●●●●●●●●●● ● 1 ●●

enforcing them can easily be explained. The PlaceWorld master environment enforces on its contents (which includes all its sub-environments and their contents) an “authenticate” filter that responds to all possible method calls. This effectively intercepts communication of any kind arriving at any entity in the world from any source. The authenticate filter examines the incoming method call, and matches the credentials of the caller against whatever rules are currently being requested by the individual entity via an innate method higher up the call chain. If the attempted invocation is valid according to these rules, the filter “releases” the original method call to cascade through the remainder of that entity’s methods, with the caller effectively unaware of the validation process that has occurred. Methods that are not authenticated are returned to the caller, which in turn has a method imbued by the environment to cope with such “return to sender” results. As before, entities that were coded without any knowledge of the access model automatically pick up a minimum amount of functionality to enable them to interact with the access model in a suitable manner; no extra effort on the part of the implementer is necessary, while those entities aware of their surroundings are able to easily extend the rule base in bespoke ways.

011

011

011

11

3.5.3 Exploiting Subjectivity PlaceWorld has comparatively relaxed synchronisation requirements. Interaction takes place mostly at a social level, and there is currently no detailed manipulation of shared entities in real-time such as might be necessary in a collaborative CAD package. The subject/object distinction embodied by DEVA allows us to take advantage of this by programming quite relaxed synchronisation routines for these entities. Though it is necessary to synchronise, say, two users attempting to pick up the same entity at the same time, or to reflect the change of text on a noticeboard to all users in a timely fashion, it is not necessary to synchronise the gentle floating effect of the PlaceWorld artefacts as they hover above the groundplane, nor, as another example, the swaying of the glowing hyperlinks in the sky. The public exhibition of PlaceWorld at the I3Net conference in Sweden and at the Sixth Doors of Perception conference in Amsterdam demonstrated that the relaxation of synchronisation and the use of subjectivity could be pushed further than we originally expected. Due to a technical problem with the graphics drivers at the time, it became necessary to use an extra rendering client to generate the display on the touch sensitive panel of the PN-AV device, rather than using the camera as was originally intended: the projected display on the external cylinder showed the user’s view of the world, and the display on the touch screen showed the same view, but augmented with the controls and menus etc. The idea of relaxed subjective behaviour had always been intended to overcome

47

Inhabited Information Spaces

network lag where users were geographically distant, and unable to see each other’s views into the environment: here, unusually, the same user was able to see one view on the touch screen in front of them, and “the same” view on the projected cylinder. The potential power of the subjective set-up was demonstrated when not a single user of the installation noticed that there were significant variations between the two displays that they could now see simultaneously: where their attention was focused (changes of text, avatars moving around the environment, entities coming and going), all appeared to behave normally. The aesthetics of the floating entities, which were in fact moving independently of one another with no attempt to synchronise the views, went unnoticed. More dramatically, one of the artefacts, a version of ‘QSPACE’ (Pettifer et al., 2001), is a graphical network much in the same vein as the Web Planetarium described in Chapter 2. To preserve bandwidth the distributed version of this artefact in DEVA was implemented with the force-placement routines as subjective behaviour, and the weights and connectivities that define the structure as objective and synchronised behaviour. What was overlooked during the implementation was that in the force placement routines, a small random factor was introduced to the positions of the nodes prior to application of the algorithm, as this seemed to produce more aesthetically pleasing results. The implication of this, however, was that although the logical and gross structure of the resulting 3D network was the same for each run of the algorithm, these small perturbations in the initial state of the network resulted in slightly different placement solutions. For the several weeks of implementation, and during the public demonstration of PlaceWorld, this went entirely unnoticed, even though, with hindsight, it was clear that the various visual representations of the QSPACE being manipulated by the collaborating users were very different. The fact that the logical structure of the artefact was coherent across views appeared to mask the visual differences in orientation and absolute positioning of the nodes.

3.5.4 Becoming a Place Where Places Meet PlaceWorld aimed to bring together many diverse virtual environments, not just trivially by allowing a user to “switch” from one application to another, but by making it possible for artefacts created in one place to be used elsewhere, analogous to cutting and pasting a spreadsheet into a word processor document. It was decided to integrate a number of existing virtual environments, including other multimedia art installations, into the PlaceWorld system. Artefacts and applications that are written with the DEVA framework in mind have this facility by their very nature; more challenging, however, was the problem of making legacy applications, such as the Legible City, or the Memory Theatre VR (an installation by Agnes Hegedüs) work within this

48

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

PlaceWorld, and the Evolution of Electronic Landscapes 3 ●●●●●●●●●●●● ● 1 ●●

framework (this last application, in particular, is interesting since it was written using Silicon Graphics Performer, rather than raw OpenGL, and therefore was already part of its own “system”). Figure 3.7 shows this and other environments embedded within PlaceWorld. MAVERIK’s (Hubbold et al., 2001) agnostic approach to data structures made this integration at the graphical level relatively straightforward, even to the extent of enabling the low-level graphics context to be shared between Performer and native MAVERIK objects, and DEVA’ s ability to combine behaviour at runtime based on enforced and imbued characteristics allowed legacy applications such as the Memory Theatre VR to pick up appropriate interaction features such as the menus and the ability to be carried around by a user.

011

011 a

b

c

d

011

11

Figure 3.7 Images from PlaceWorld and its embedded environments: (a) the QSPACE; (b) the Legible City; (c) Memory Theatre VR; and (d) the Advanced Interfaces Group’s laboratory.

49

Inhabited Information Spaces

3.6 Conclusions The vision of an electronic landscape remains an exciting one, and as graphics technology and higher performance networking become more affordable and available, there are increasing opportunities for the kind of research and development described in this chapter. The iterative approach of the eSCAPE project has been an effective way of understanding the social, aesthetic and technological problems associated with making these worlds meaningful and useful to the general public.

Acknowledgements PlaceWorld was influenced in many ways by all the partners and participants in the eSCAPE project: too numerous to mention, this chapter draws upon much of their work and our thanks go to them all. Special thanks are due to Paul Arnold, Annika Blunck, John Bowers, Andy Colebourne, Andy Crabtree, Timo Fleish, Simon Gibson, John Hughes, Andreas Kratky, John Mariani, Gideon May, Craig Murray, Andreas Schiffler, Jeffrey Shaw, Adrian West, and the project’s co-ordinator, Tom Rodden. The work was supported by a grant from the European Commission.

50

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Using a Pond Metaphor for Information Visualisation and Exploration

4

Olov Ståhl and Anders Wallberg

4.1 Introduction The constantly increasing amount of information and media available in electronic form leads to a growing demand for new methods for searching and browsing. Traditional text-based database queries can be limiting, often requiring a user to know exactly what it is she is looking for and to express this interest using predicates in a query language such as SQL. Furthermore, to be successful in locating the right information, a user will often have to be familiar with the standard interface metaphor of desktop computers and know how to use a mouse and keyboard efficiently. Approaches to improve access to online information and to visualise it in an intuitive manner have been under development for a long time. Examples of systems that display information graphically in three dimensions include VR-VIBE (Benford et al., 1995a), QPIT (Colebourne et al., 1996) and BEAD (Chalmers, 1993). All of these systems require (at least to some extent) the users to use navigation to access particular information objects, because objects may obscure each other or may be out of view. However, navigation within 3D spaces is known to be difficult (Ingram and Benford, 1995), especially if the navigation needs to be precise. In this chapter we describe The Pond, a system used to search for and visualise data elements on an engaging tabletop display. The Pond uses methods of unencumbered interaction and audio feedback to allow users to investigate data elements, and supports shoulder-to-shoulder collaboration with the physical Pond artefact mediating the collaboration between those people gathered around it. The user interface is based on an ecosystem metaphor, presenting data elements in the form of shoals of aquatic creatures inside a virtual 3D pond. The design makes use of a

51

Inhabited Information Spaces

static view of the information space, making viewpoint navigation unnecessary. Instead the information creatures move and form groups, allowing the user to easily identify related information and to distinguish results from different queries. The work draws heavily on our experiences in developing two previous systems, and concepts and approaches can be traced back to these systems. The Web Planetarium, a 3D interpretation of HTML structures in the form of a graph (Chapter 2), and the Library Demonstrator, an information landscape constructed from the contents of an online library database (Mariani and Rodden, 1999), were both interfaces to active online information. Common features of these systems were the spatial arrangement of data elements, navigation around these data elements and the introduction of new data elements into the display. While these systems were successful in presenting information to end users, they were not necessarily easy to use. When using the Web Planetarium, new information was only introduced as a result of explicit interaction (clicking) by the user, and novice users could be too shy to discover this fact and not load new information. The positioning of this new data could also be problematic. In the Planetarium the user must be observant otherwise she may miss the introduction of the new data element. The Library Demonstrator addressed this problem using animation and self-organising models to show the emerging relationships between information, and it is this approach we build upon in The Pond with its shoal metaphor. Navigation around the data used a point and click method, automatically transporting the user to (or close to) the selected object. The core of this navigation technique was to adopt an object centric philosophy where users were explicitly freed from the overhead of having to manage their navigation and movement in three dimensions. This restriction of the overall freedom of movement meant that users were able to focus on the exploration of the information space. However, users still had problems navigating the structures. It could range from getting completely lost to not being able to look upon the data in the way they wanted. These systems were also subject to single user interactions and did not encourage a social atmosphere for the exchange of gathered information. When we started the work on The Pond, the objective was to design a multimedia user interface where users without any prior knowledge or acquired competence should be able to easily handle both single as well as groups of objects in an affordable and easy way. The objects in turn should be able to represent any type of information or media. They should be able to present themselves in a natural fashion at the user’s convenience. Also, they should disappear in non-obtrusive fashion when no longer needed. The user should be able to select, move, sort and explore objects of interest without having to confront the rigid hierarchy that is the hallmark of traditional file handling and database applications. Instead of using colour, form or position to indicate group or class

52

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Using a Pond Metaphor for Information Visualisation and Exploration 4 ●●●●●●●●●●● ● 1 ●●●

membership we wanted to use motion dynamics of objects to indicate these properties. If new information is requested, objects should just float to the surface and present itself to the user. Objects no longer relevant should slowly sink to the bottom and quietly disappear. The viewpoint should stay fixed above the virtual pond surface, thus making it unnecessary for the users to engage in navigation within the information space. An observation made early in the design process was the difference in the behaviour of users when confronted with a horizontally placed display vis-à-vis a vertically placed one. When a group of people gathers in front of a vertically placed display – be it a big screen TV or a wall projection – immediately a somewhat authorial situation tends to develop. The person in control of the display content is perceived as a teacher or lecturer. The rest of the group will play the role of an audience or school class. When, on the other hand, the display or projection area is positioned horizontally, people will gather on a more equal basis. With inspiration from as disparate sources as roulette tables, billiard tables and the military’s “classical” tactical war gaming board, it was decided that The Pond should use a horizontal display and that none of the sides of the physical artefact should be more important than any of the others. Tabletop displays have been in use for a number of years now, for example in visualising information (Kreuger and Froehlich, 1994) for command and control scenarios (Chase et al., 1998) and augmenting physical objects (Ishii et al., 1999). The table provides a natural centre for interaction to take place around and encourages collaboration between the users while they are interacting with the table (Benford et al., 2000). The developments in plasma display technology coupled with touch-sensitive surfaces now make it possible to dispense with potentially clumsy projected displays in favour of a neat, compact display. Interaction with The Pond is object based, using physical tags to load information into the application by placing them around the edge of the table (Ishii and Ullmer, 1997) (Ullmer et al., 1998). Unlike other approaches where the physical objects are placed directly on the table and manipulated (Underkoffler and Ishii, 1999; Fjeld et al., 1999; Schäfer et al., 1997), the tags used in The Pond are passive. They make it possible to load and store information, but do not manipulate this information any further. Interacting with the contents of The Pond is supported through direct manipulation of the virtual pond objects (Shneiderman, 1983) using the touch screen display. Typically, interactions with the tabletop systems discussed above make use of stereoscopic glasses, data gloves, and magnetic position trackers. These techniques are not used in The Pond as our aim was to create as easy and direct an interface as possible and not to encumber the user with devices needed to experience the material presented. In the remainder of this chapter we will describe The Pond as it stands today, details of its implementation, and observations from a study of system in use. We conclude with a discussion on some design choices and their implications on the potential utility of The Pond in different settings.

53

Inhabited Information Spaces

4.2 The Pond We now describe The Pond system in detail, examining its construction, both in terms of the physical artefact and the underlying software, and the way in which it is used.

4.2.1 The Pond Ecosystem Metaphor The Pond uses a user interface based on an ecosystem metaphor. The objective was to present an aesthetic that would hide the work chore aspect and act as a complementary backdrop for dialogue. When interacting with The Pond, the users see a 3D presentation of a virtual pool or pond (see Figure 4.1), an aquatic environment in which the information objects resulting from queries are presented as shoals of marine creatures. The visual presentation is complemented by a sound environment, consisting of a number of bubbling, splashing and whirling sounds, indicating various active processes inside the pond virtual environment. Each information creature has a simple navigational behaviour that governs its movements within the virtual pond. The behaviour is inspired by the boid flock model described in Reynolds (1987). The basic steering rules are: ● ● ●

Avoid collisions with nearby creatures. Match velocity with nearby shoal mates. Stay close to nearby shoal mates.

Figure 4.1 The Pond virtual environment.

54

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

Using a Pond Metaphor for Information Visualisation and Exploration 4 ●●●●●●●●●●● ● 1 ●●●

These simple rules are then combined with additional influences like curiosity (a creature might temporarily break away from its shoal to “examine” its surroundings) and competitiveness (an urge to be in the lead), resulting in a very dynamic and fascinating pond environment, where shoals of information creatures move around in tight groups, avoiding each other as well as the pond walls, and reacting to interactions from the users. The movement of a shoal is entirely dependent on the combined navigational behaviour of its creatures, that is, there is no shoal “intelligence” that determines where the shoal is going or how its creatures should behave in certain situations. The argument for using a flocking algorithm for positioning the information creatures is not based on aesthetics alone. Our earlier experiences in information visualisation mixed with an ambition to explore a somewhat alternative user interface made us choose a self-animating system for the data elements. Not only animating during the creation/insertion stage but during the whole time the elements were available. In addition, the human visual system is quite good at separating objects with common velocity vectors making it easier to identify group belongings in a crowded environment. The Pond does automatic garbage collection, which means that one or several of the shoals that exists in the virtual pond may be removed. This may happen, for instance, if the environment is too crowded or because a particular shoal has not been interacted with for a long time. A shoal that is selected for removal by the system will sink down towards the bottom of the virtual pond, where it will disappear. However, should a user interact with a sinking shoal, for example, select one of its creatures, the shoal will return to the surface and another shoal might be removed instead.

4.2.2 The Pond Example Application

011

011

11

To investigate the concepts behind The Pond we developed an application that allows users to search for and browse music content on the Amazon.com web site. The music theme was chosen to maximise The Pond’s visual and sonic impact, being a theme common to most potential end-users. However, The Pond is not limited to this domain and is adaptable to other database resources. A search task is initiated by a user providing a keyword string, for instance the name of an artist, group, or a musical genre. The Pond communicates the keyword to Amazon.com and presents the resulting album hits as a shoal of creatures, each representing one specific album. The virtual creatures are represented by simple geometric shapes, which are texture mapped with the albums’ cover images. By interacting with a creature, users are able to access the album data (e.g., artist name, album and track titles, etc.) and play samples from some of the tracks. The virtual pond itself consists of a 3D model of a deep, narrow shaft that extends down from a watery surface.

55

Inhabited Information Spaces

The music-oriented Pond application was selected since we wanted an application that would allow us to explore and evaluate the ideas behind The Pond without having to implement a lot of advanced search functionality, and which would also be interesting and enjoyable to use (e.g., looking for and listening to your favourite music). Furthermore, the size of a “typical” query result was expected to be less than 50 items, which would allow us to present a number of simultaneous shoals in the virtual pond environment without making it appear too crowded.

4.2.3 The Hardware Platform The physical Pond artefact has the form of a desk, on top of which a large touch-sensitive plasma display is horizontally placed. On top of the display surface is a wide wooden frame with an irregular curved outline, representing the bank of the virtual water pond rendered on the display. The frame is covered by pieces of thick carpet so that users standing around it can comfortably lean over the display when interacting with the virtual pond environment (see Figure 4.2). Built into the frame are a number of speakers that are used to output various sounds and music samples. The use of audio is an important

Figure 4.2 The Pond desk.

56

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

Using a Pond Metaphor for Information Visualisation and Exploration 4 ●●●●●●●●●●● ● 1 ●●●

feature in interacting with The Pond, and the sound system consists of several devices including a sampler, sub-woofer and amplifier. The frame also encloses several RFID tag readers. Each reader is entirely embedded into the frame carpet and uses three light-emitting diodes to indicate its position and state to the users. Users may initiate queries by placing a RFID tag on such a reader. The tag’s identifier, sensed by the reader, will identify a query keyword or phrase and the query will be initiated.

4.2.4 The Software Platform The Pond software platform consists of two different components (see Figure 4.3): ● ●

The visualiser application renders the view from the virtual pond environment on the plasma display. The pond manager application accepts query keywords from the users and communicates these to the Amazon web server. The resulting information is used to introduce and control shoals of creatures in the virtual pond.

The visualiser and the pond manager are built using the DIVE (Chapter 12) distributed VR system from the Swedish Institute of Computer Science (SICS). The 3D virtual pond is in fact a DIVE virtual world, shared by the visualiser and the pond manager. When the pond manager Tag readers

Speakers

011 Dive pond virtual world

Pond Manager

Query and result via web services API

011

11

Amazon Web server

Visualiser

Figure 4.3 Overview of The Pond software components.

57

Inhabited Information Spaces

application is started it loads a file containing a model of the pond graphical environment. Initially the virtual pond is empty, that is, it contains no information creatures since these are only created as the result of user queries. When the visualiser is started it joins the world created by the pond manager and will receive a state transfer containing the graphical environment. From this point on the two applications share the pond environment, exchanging DIVE messages to notify each other of any world changes (e.g., the introduction of information creatures or creature movements). The visualiser is responsible for detecting user interactions on the plasma display’s touch sensitive surface. If a user clicks on a creature, the visualiser distributes an interaction event that will be received by the pond manager. After examining the event, the manager determines the appropriate action to take (if any), which typically involves some change to the virtual creature (e.g., a change of appearance or position). The change will generate a new DIVE message, which will be received and handled by the visualiser, thus making the world change visible to the users. The visualiser uses a number of rendering plug-ins to enhance the visual appearance of the pond environment. A ripple plug-in creates ripples and waves that graphically deform the perceived environment when fingers are pressed on and moved over the touch screen of The Pond. A second plug-in is responsible for generating a caustic lighting effect on all objects in the water. These plug-ins use hooks in the DIVE renderer and operates on the image it generates. User input in the form of query keywords is handled by the pond manager. Queries are initiated either by direct request, that is, a user placing a tag on a reader, or more indirectly through inferred user interest. The manager continuously monitors the tag readers embedded in The Pond table and immediately senses any change in their status (i.e., RFID tags being added or removed). To get data requested by these queries the manager uses the Amazon.com web services (Amazon.com, 2002). With its web services program, Amazon.com offers third party web sites and applications the ability to search or browse Amazon’s product database. The manager retrieves information like album titles, artist names and URLs to cover images through the web services using XML over HTTP. This information is then used in the process of creating representations of individual albums and shoals representing more than one hit on a query subject. The pond manager also handles the audio output to The Pond table speakers, which is described further in Section 4.4.

4.3 Interaction Users standing around The Pond table are able to interact with it in several ways to perform various tasks. In the absence of keyboard or mouse devices, the users perform most interactions by tapping or

58

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

Using a Pond Metaphor for Information Visualisation and Exploration 4 ●●●●●●●●●●● ● 1 ●●●

stroking the touch-sensitive display surface. Furthermore, prepared tags that are spread out on top of The Pond frame allow users to input information about queries without having to type on a keyboard. Queries are initiated by placing tags on the tag readers in table frame. Since each available query tag has a sticker attached to it showing the keyword text or an image of an artist or group, users are able to determine what keyword a certain tag represents (see Figure 4.4). When a tag is placed on a reader, the reader senses the tag’s unique identifier, which is pre-mapped to the keyword. As soon as a query is initiated, an empty shoal appears inside the virtual pond, representing the ongoing query. The shoal is indicated by a circle and a text string specifying the query keyword, as seen in Figure 4.5a. The circle will commence to float inside the pond environment, bouncing off the walls and avoiding other shoals. When the query results are initially delivered from the Amazon.com web server, creatures start to appear inside the empty query shoal. Each such creature represents an information element from the query result; in this case a CD album. As soon as a particular result creature has been created, it will begin to move around inside the virtual pond. However, since all creatures resulting from a particular query stay close together, different query shoals are easily identifiable, even with a vastly populated pond environment.

011

011

11

Figure 4.4 Query tags representing the keywords “Frank Sinatra”, “Billie Holiday”, “ABBA” and “Kraftwerk”.

59

Inhabited Information Spaces

a

b

c

d

e

f

Figure 4.5 Actions and events in The Pond: (a) To the right is an empty shoal, representing an ongoing query using the keyword “Nitin Sawhney”. To the left is a shoal of creatures representing a finished query. (b) Two shoals representing the results of queries using the keywords “Zeppelin” and “Queen”. (c) To the right a selected creature dragged a bit from its flock. (d) Playing sample one (out of five available) on the album “Siamese Dream” made by Smashing Pumpkins. (e) Three creatures dragged into a creel, limited by the pond walls and two buoys. (f) Getting related albums to an album called “Lamb” made by the group Lamb.

60

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Using a Pond Metaphor for Information Visualisation and Exploration 4 ●●●●●●●●●●● ● 1 ●●●

When all the results have been delivered and the corresponding virtual creatures created, the shoal circle label will change to only display the query keyword (see Figure 4.5b). Shortly thereafter the shoal circle and label will disappear, leaving behind only the creatures visible to the users. However, it is possible for users to make the circle and label visible again by tapping on any of the creatures belonging to the shoal. Each shoal member represents an information element that is part of the result of the corresponding query. The information includes name of group or artist, album title, and URL to the album cover image, as well as URLs to a number of short Real Audio® samples of some of the album tracks. Users are able to access this information by manipulating the creatures in different ways. A user selects a creature by tapping on it on the display, and is then able to see the information identifying the artist and title of the corresponding CD. This information is presented as a virtual text string, encircling the 3D creature and moving alongside it (see Figure 4.5c). The text will only be visible for a brief period (around five seconds) and will then disappear. The frame of the creature will become green to indicate the selection, and will remain so until changed back to white again when another creature is selected. Tapping once more on an already selected creature makes it float up to the surface and initiates playback of the corresponding album’s first Real Audio sample (see Figure 4.5d). By using the Real Audio player and the sample URL, the audio data is streamed over the network from the Amazon.com web site and output to the speakers embedded in table frame. By tapping repeatedly on the creature being played, users are able to step through the sample tracks available for that particular album. The text encircling the creature displays the number of the sample being played as well as the total number of available samples. By default, the viewpoint in the virtual pond environment is placed above The Pond surface, looking down, and at a distance from where the view is always guaranteed to include all the existing creatures. In this way the users are able to get a good overview of all the activity within the environment. However, this also constrains the creature representations to be rather small (as seen on the plasma display), which might present a problem when trying to identify the creature graphics of the CD albums they represent. In order to allow users to get a closer view of one of the shoals, a zooming mechanism allows for the translation of the virtual viewpoint to a position close to a shoal centre. The viewport is just large enough to encompass the whole shoal, with the benefit of making the creatures in that shoal together with their associated text strings and images appear larger. As a result, other shoals may end up out of view, not visible from the new viewpoint position. Another feature of the zooming mechanism is that, while zoomed, the viewpoint is attached to the shoal, which means that it will move as the creatures within the shoal moves. In this way the

61

Inhabited Information Spaces

viewpoint will always stay centred on the chosen shoal, even as this shoal changes its position within the virtual pond. While zoomed, the shoal creatures may be interacted with in the same way as before, for example, selected to initiate replay of the music samples. The zooming mechanism is triggered when a tag is placed on a reader while the corresponding shoal already exists in the pond environment. Removing the tag from the reader will reset the viewpoint to the default overview position. Thus, if a user places a tag associated with the keyword “Dylan” on a reader while a “Dylan” shoal exists, the viewpoint will change to a position close to this shoal, and stay there as long as the tag is on the reader. Whenever the tag is removed, the viewpoint is reset. Only one user at a time can use the zoom mechanism. If a zoom is active while a user initiates another zoom the viewpoint state will not be overridden and the second request is ignored. As users initiate more and more queries, older shoals may have to be removed in order to incorporate newer ones. To prevent a particular creature from being removed from the environment, it is possible for users to move individual creatures to safe areas, called creels. This is done by touching the creature with the finger, and then dragging the finger (and thus the creature) along the surface and releasing it over the creel area (see Figure 4.5e). Creels exist in several places in the virtual pond environment, close to the tag readers. Once inside a creel, a creature is constrained to move only within the creel boundaries. By moving several creatures, possibly from different shoals, into a creel, a selection shoal is formed consisting of creatures that a user finds interesting for some reason. Since this particular action shows an interest from the user in a specific data element, this instantiates an extra functionality. A more focused query is automatically launched pertaining to the corresponding album (usually resulting in fewer hits and consequently a smaller shoal), thus further populating the environment (see Figure 4.5f). In the case of Amazon.com the system issues a related albums query resulting in items that have a high chance of being interesting to the user as well. The creatures in the creels may be interacted with in the same way as other creatures (e.g., tapped on to play music samples), the only difference being that they won’t be removed from the environment as long as they stay inside the creel. Creatures that are dragged out of a creel will return to their native shoal, or form a new shoal if their native shoal no longer exists. By using a recordable tag, it is possible for a user to save the contents (i.e., creatures) of a creel selection shoal. A recordable tag is a RFID tag which is not pre-mapped to a search keyword. Placing such a tag on a tag reader next to a creel creates an association between the tag and the creatures within the creel area. When the tag is then removed from the reader, the creel’s selection shoal will disappear from the environment and may be regarded as being stored on the tag. By placing the same tag on a reader at a later stage, the creatures of the “saved” selection will

62

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Using a Pond Metaphor for Information Visualisation and Exploration 4 ●●●●●●●●●●● ● 1 ●●●

reappear, added to the corresponding creel shoal. Thus, creels and recordable tags allow users to save references to one or several albums of particular interest, and to gain quick access to these on a later occasion, as well as to share references with each other.

4.4 The Pond Audio Environment The sonic environment of The Pond consists of two different parts, the soundscape and the interface sounds. In accordance with the ecosystem metaphor, the nature of the soundscape is founded on a family of aquatic whirling sounds and a deep, obscure mumbling giving the impression of the data elements ascending from an abyss of ooze or mud. These ambient sounds fade out when samples from a selected creature starts to play, and fades smoothly in when the music stops. The interface sounds acts as a feedback mechanism to indicate user interactions like selecting/unselecting, clicking, dragging etc. This collection also originates from a number of concrete water sounds. Samples from different types of bubbles, a dripping tap, whirlpools etc. are heavily processed to suit their particular purposes. Examples include: ● ●

●

●

●

When a user initiates a query, the appearance of the query shoal is accompanied by the sound of sluggish bubbles rising from the bottom. When a sound file is retrieved over the network, the waiting time is masked with a bubble vortex that after a few seconds is smoothly merged with the music sample. When the user removes a tag from a reader, the action is accompanied by sounds of a cluster of bubbles being rapidly inhaled by The Pond itself. When the user draws his finger across the touch screen to drag an object, a glass organ sound reminiscent of drawing a finger along the damp edge of a crystal wineglass is heard. The visual zooming in of a shoal is illustrated by the familiar bubble sound gradually magnified through lowering of the pitch. When zooming out, the process is reversed.

Every time an interface sound needs to be played, the system will randomly choose a sound from a collection of sounds available for the particular interaction. There are ten query sounds, ten RFID sounds, ten dragging sounds, etc. The idea is to give the impression of the sonic interface being somewhat organic and unpredictable. The precise spatial placement of every sound is achieved through a builtin high quality four-channel sound system. A subwoofer in the table foot produces a deep and suggestive bass. The computer-controlled software mixer makes it possible to physically move sounds around and to create expressive musical gestures.

63

Inhabited Information Spaces

4.5 Observations from Use To test out The Pond a number of sessions were held with external participants in the expectation that some qualitative assessment of the effectiveness of system could be made. After being presented with an introduction to The Pond and the example application by one of the developers, the participants were told to explore the system, using the available tags on the table frame. The focus of the studies was on the technology-in-use (Button, 1992), that is, the sequences of interaction within which The Pond came to be used in real time. The developers stayed close the whole time to answer questions and to help out when problems occurred. Most sessions involved two or three participants and lasted for about an hour. Video was used to record each session and from this material a number of observations were drawn. The general impression is that the participants had very few problems in understanding the basic principles of The Pond design and its operation, for example, how to use the tags to initiate queries or how to interact with the creatures in order to play sample music. On very few occasions did the developers have to repeat the instructions given at the start of the session. The Pond is engineered to support hands-on experience and ease of use, and most participants did engage in various activities, together and alone, such as selecting CDs, listening to samples, transferring CDs to tags, and transferring CDs from tags after only a short time. During a discussion after one of the sessions, a participant stated: “The main good thing about The Pond is that the interface doesn’t require any special level of computer skills for using it.” Not only did the participants get the grasp of The Pond fairly quickly, but were also able to envisage its use for practical purposes in every day settings. For example, one of the participants suggested: “Place it in a Virgin Megastore, it can be used as a jukebox. Perhaps you should specialise it more so it fits into a category like film, books, articles, and radio. And you should be able to tell The Pond which CD you want and then be able to put it into your shopping basket.”

In some sequences of interaction it is noticeable that the sound worked as functional feedback. After the participants had recognised the connection between the function and the sound, the sound worked as a support for the user in interacting with The Pond. This can be seen in a sequence when a participant drags a creature out from a creel. In the sequence the sound that is heard as the dragged creature passes the creel boundary makes it clear to the participant that she has succeeded and the creature is released. The problems that participants had during sessions were mostly related to the hardware, and especially the touch screen. A serious drawback with the large area touch screen technology we currently use is a limitation when it comes to detecting multiple simultaneous touches.

64

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Using a Pond Metaphor for Information Visualisation and Exploration 4 ●●●●●●●●●●● ● 1 ●●●

This sometimes caused conflicts when more than one participant was trying to interact with creatures at the same time. For instance, it was impossible for two users to drag creatures at the same time. Also, if one user was dragging a creature while another user tried to select a different creature, the selection would most often fail. The reason for this is that when two positions on the touch screen are clicked simultaneously, a position in between the two will be returned by the driver software, most likely causing the second user to miss the intended creature. Future multi-user touch screens will alleviate this shortcoming. In the present set-up we try to work around the problem by making most interactions single click based, dragging being the exception. Another problem with the touch screen was that of sensitivity. Sometimes when the participants where dragging creatures they seemed to unintentionally lose contact with the touch surface, causing the creature to stop following the finger and return to its shoal. What happened was that even though their finger was actually in contact with the surface, they didn’t put enough pressure onto it, which made the driver report that the interaction had ended. A number of “phantom clicks” (i.e., the driver reported a click when there was none) were also experienced which had a similar effect on the interaction. We expect most of these hardware problems to disappear in the future as the technology matures. One of the software mechanisms that caused some problems was zooming. It was evident from the video material that zooming could cause confusion if being performed by one participant when others where trying to interact with creatures in shoals different from the one being zoomed (as these shoals would disappear from the view). Zooming probably requires synchronisation of the users’ activities in order to be useful. As it turned out, the navigational behaviour of the information creatures became a source of fascination for many of the participants when encountering The Pond, and it was not uncommon to see them standing silently at the side of the display for some time, simply watching the creatures swim back and forth.

4.6 Discussion One of the most frequently expressed comments from users was the desire to be able to search The Pond in a more dynamic fashion, that is, to issue custom search queries, which is not possible through the use of the tags at the moment. The seriousness of this limitation depends of course on the application and its intended use. The Pond is not primarily designed to be used for the task of quickly locating and browsing arbitrary known information items on the web, in which case the use of the pre-configured query tags would probably be unacceptable. Instead, the focus is more on applications where the size of the data set is fairly small (i.e., a product catalogue) so that the query tags more or

65

Inhabited Information Spaces

less cover the whole search space. In addition, if the database is hierarchical or allows items to somehow contain references to each other, the users can traverse the data not only by issuing direct queries using the tags, but also by exploring the creatures already existing within The Pond (following the inter-creature links to discover other “parts” of the data space). In such settings the tags can be seen as a way to introduce information objects that probably won’t be the end result of a user’s query, but merely a starting point for further exploration. Examples include using The Pond in a record store setting where the top 100 albums are to be displayed and played, or in a hotel lobby where menus from local restaurants, museum information, etc., can be examined in a collaborative fashion. In the Amazon.com data source set-up, a way of search refinement and database traversal was available through the use of related album queries. This introduces items that have a high chance of being interesting to the user. Combined with the fact that items considered uninteresting will sink and disappear, this has the effect of gradually refining the ecosystem to contain more and more interesting items. So even though very specific selections are being used as “seed” material, the contents of the environment might develop through time to contain a mix of several general tastes, specific favourites and complete wildcards. Our aim when designing the interface was towards a non-intrusive form of interaction, avoiding introducing keyboard type input. The RFID tags do serve well in cases where a hierarchical, finite and discrete database structure exists, as the music database in our case. Notice that the environment is in the most part populated by relations and not exact matches, so we force exploration on the users. It proved at times frustrating for users not to be able to directly summon the artist or track of their choice, and at times gratifying in discovering alternative music of their liking. A solution that would in fact allow users to input exact search keywords or query phrases and which wouldn’t entail the use of some sort of keyboard function would be to use voice as input. An issue that came up during the user studies was the possibility of modifying or even replacing the example application to add new functionality or to support different types of media. For instance, instead of saving references to albums onto recordable tags, one participant wanted to be able to save the actual sample data onto some kind of portable device, for example, a PDA, which could then be used for playback. This could be achieved by using some kind of point-to-point (e.g., infra-red or Bluetooth) connection to transfer the audio data from The Pond to the PDA. Also, since DIVE runs under PocketPC™ it would be possible to develop a PDA application which would join the pond virtual world to be able to access the information (e.g., music samples) represented by the virtual creatures. Modifying The Pond to support a different type of content media, or even combinations of different media, would require changes to the pond manager application (see Section 4.2.4 above) in two different ways. First, it would require the pond manager to be able to

66

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Using a Pond Metaphor for Information Visualisation and Exploration 4 ●●●●●●●●●●● ● 1 ●●●

communicate and issue queries to a database service different from the one provided by Amazon.com (assuming that Amazon.com does not support access to data elements of the “new” media type). This would be fairly straightforward since the code responsible for handling queries is isolated in one specific module, which can be easily replaced. It is also possible to add a second (or more) module to exist in parallel with the first if there is a need to connect to more than one database service during a session. Secondly, to support new types of content media would require new rendering mechanisms to present the creatures to the users when there is interaction with them. As is the case with the current Amazon.com web service version and the RealAudio player, special information rendering that goes beyond small pieces of text, images and texture-mapped videos will probably need external renderers. These can be in the form of DIVE plug-ins or external applications (e.g., Windows Media Player to render video material). The pond manager application defines a number of events (e.g., OnClick) that are fired when creatures are interacted with and needs to display details of their associated information (e.g., album title in the Amazon.com application). By modifying or replacing the current handling of these events in the pond manager code, it is possible to change the way the creatures present themselves to the users. One of the consequences of using a static view into the pond environment is that the display space is restricted, thus limiting the number of creatures that can be presented simultaneously. As described earlier, garbage collection can alleviate the problem to some extent by removing old shoals when new ones need to be introduced, but it does not help if the new query result is too large to fit into the virtual pond by itself. Making creatures smaller in size will make more of them fit before the pond becomes to cluttered, but also makes it harder to distinguish the album images, read the text strings, etc. One possible extension of the shoal concept that could help in presenting large data sets is hierarchical shoals. Assume that a shoal could be made up of sub-shoals as well as information creatures, and that a sub-shoal was initially represented by a single graphical object, just like an ordinary creature. When clicking on a sub-shoal object it would expand to display all of its elements, some of which might themselves be sub-shoals, and so on. An expanded sub-shoal might collapse into the single sub-shoal object after a certain amount of time, or possibly as the result of an explicit user interaction (e.g., clicking once more on the sub-shoal object). In this way it would be possible to make available a large number of information creatures to The Pond users without having to display them all at the same time. The rules governing the creation of sub-shoals are probably application specific, that is, the criteria used to determine when to create a sub-shoal and which creatures to add to it. Another possible solution to handle large query results would be to make each shoal present only a limited set of the resulting creatures at one time, and then allow the users to somehow (via some shoal interaction) make the shoal present

67

Inhabited Information Spaces

the next set of creatures, and so on. This is the method used by most Internet search engines where the user is presented with a list of about ten hits, and is then expected to use the “next” or “previous” links to move forward or backward in the result material.

4.7 Summary and Future Work We have presented The Pond, a multi-user system for browsing information (currently CD record data) on the Web using an engaging tabletop display. Users input musical search keywords (typically names of artists or groups) using RFID tags and are presented with matching Web information in the form of shoals of aquatic creatures inside a virtual 3D pond. The virtual pond environment is presented on a big touch-sensitive plasma display, which is placed horizontally to better support shoulder-to-shoulder collaboration between those people gathered around it. By touching the surface of the display users can interact with the creatures to access the information they represent, for example, play music samples from the corresponding CDs. A series of user studies have provided initial insights into the utility of The Pond. The results indicate that the device and metaphor are easy to understand and use, but also identify a number of problem areas. For instance, the touch sensitive display doesn’t currently support multiple simultaneous interactions, which sometimes caused the users’ actions to interfere with each other. The Pond has been demonstrated to members of the research community as well as to the public on numerous occasions. The feedback and observations from these sessions have been used to refine The Pond under an ongoing evolution process. Future ideas include support for multiple simultaneous interactions on the display surface, using a voice input system for entering search keywords, and using PDAs to extract and input information to and from The Pond environment.

Acknowledgements We would like to thank Lennart E. Fahlén, Jan Humble, Jenny Lundberg and Jonas Söderberg for their participation in the development of The Pond system. We would also like to thank Adrian Bullock for his contribution in documenting The Pond work as well as Andy Colebourne for his Pond-related efforts within the eSCAPE project. This work was undertaken as part of the Swedish Research Institute for Information Technology (SITI) funded Electronic Landscapes Project (ELP). We would like to thank all those who have experienced The Pond and provided us with invaluable feedback. The Pond work has also been described in (Ståhl et al., 2002).

68

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

Part 3 Mixed Reality Environments

011

011

011

11

69

This page intentionally left blank

011

011

011

011

11

City: A Mixture of Old and New Media

5

Matthew Chalmers

5.1 Introduction The majority of the chapters in this book describe what might nowadays be called “traditional” inhabited information spaces: collaborative virtual environments (CVEs) or virtual worlds. Although not all CVEs centre on computer-rendered 3D graphics, the paradigmatic CVE does so. Shared 3D virtual environments are emblematic of CVE research, but have only gained public acceptance in the form of computer games. The focused engagement in such games is designed to fit with the closed world of the virtual environment. A player can become immersed in a game – closed off from the “real” world – by attention as much as by apparatus. A personal computer (PC) at home can be as engaging as the head-mounted displays and immersive projection technologies in research labs. However, even a single-player non-networked game may be a resource for social interaction, for example played by one person while friends and family shout advice from the sofa, order pizza by phone and slip into the kitchen to get more drinks. A computer game is a resource for far more social interaction than the software’s architecture may suggest. In general, the wider context of use is hardly modelled or represented in the system. Games’ internal data structures are designed to be decoupled – that is, closed off – from the other media people use in everyday life, and this decontextualised design approach has worked well in this domain. Although many households, schools and workplaces have computers that could support 2D or 3D virtual environments, such CVEs are rarely used as a medium of family interaction, education or work. One reason for this may be the decoupling from the more traditional interactions of family members, the overall educational activities of the school and the business of the workplace. The information within the CVE would have to correspond with each user’s wider context, and this would require sensing and tracking of users’ activity in media beyond that of the computer. To paraphrase part of this book’s introduction, a large amount of

71

Inhabited Information Spaces

our activity relies on the knowledge of what other people do, and what people do in the home, street, school and workplace involves many noncomputational media. However, CVEs are designed on the basis of narrowly focused engagement, decoupled from their users’ wider context of family, friends, learning and work. Other than in games, CVEs’ decontextualised design approach has not led to popularity or widespread use. As the introduction to the book also points out, a number of researchers have begun to work on inhabited information spaces that are more “out in the world” than traditional CVEs. A rhetorical example the editors give is “a system that enables co-located groups to co-operatively work with information by using a display projected onto physical artifacts”. An IIS may include tangible artefacts in more traditional media, such as urban models and interaction devices made from wood, wire and plastic (Underkoffler and Ishii 1999), or tiles and pages made from toner and paper in augmented reality systems such as MagicBook (Billinghurst et al., 2001). The work discussed in this chapter is intended to go further in this direction. As part of the Equator interdisciplinary research collaboration (www.equator.ac.uk), the City project explores digital spaces that are peers with others, rather than a digital space that is the primary focus or locus of activity. For example, in our system, one person’s use of a 3D VR model of a museum exhibition is coupled with another person’s use of hypermedia describing the exhibition as well as a third person’s use of the traditional “bricks and mortar” exhibition. No one of the three is primary; each is part of the context of the other two. We combined the media of traditional exhibitions, mobile computers, hypermedia and virtual environments in one design, and support interaction between people using different subsets of this heterogeneous set of media. The project was initially theory led. A number of theoretical issues were outlined in a discussion document, and then exemplified by scenarios of technology use. As the project grew and developed, our theoretical issues, design scenarios, system development, evaluation of pilot trials, and observational studies all affected each other. Although I sometimes campaign for the rule of theory, not one of these areas is primary. Instead, each is part of the context of the others. The project has always aimed to get out into the streets of the city, but we decided to begin our work in a more controlled setting: the Mackintosh Interpretation Centre, a permanent exhibition devoted to the life and work of Charles Rennie Mackintosh (1868–1928). Mackintosh was a Glasgow architect, designer and artist, and several of his buildings and other institutions related to his work stand in the city. Often simply called the “Mack Room”, the Centre is comprised of textual and graphical displays with some original artefacts, as well as over 20 screens presenting video and interactive digital material. The Mack Room is in The Lighthouse, Scotland’s Centre for Design, Architecture and the City (www.thelighthouse.co.uk).

72

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

City: A Mixture of Old and New Media 5 ●●●●●●●●●● ● 1 ●●●●

More generally, we are exploring the way that digital information is just another part of an interwoven set of media, the use of which constitutes inhabiting the city. Unlike traditional CVEs, we aim for systems coupled with and contextualised in everyday activity, and hence in accord with contemporary theory of the use of language and space. A basic theoretical premise is that we can only use digital media because of such interweaving and interdependence, and we are looking for ways to increase and take advantage of the interdependence of traditional and new media. This theoretical standpoint is set out in the next section of this chapter. The subsequent section outlines a system design based on this standpoint, followed by a discussion of user experience in trials of an implemented research prototype. Some of the details of our ongoing and future work are then outlined before a concluding section that offers more general reflections on the project.

5.2 Theory This section focuses on conceptions of space and the media associated with work, and how we often conceive of space as a medium that stands above or apart from others. It is this usually implicit assumption that lets us talk of information spaces as being “inhabited”. I would like to present an opposing view that treats information spaces as merely one medium among the many used in everyday life. My approach is based on experience with information visualisation and virtual environments, as well as some borrowing of structuralist semiotics (Saussure, 1983; Nöth, 1995) and philosophical hermeneutics (Grondin, 1994). Part of this latter work was set out in a recent paper in the Journal of Computer Supported Cooperative Work (Chalmers, 2003). When discussing work, and designing systems for remote collaboration, we all too often concentrate on emulating the spatial aspects of the workplace, for example modelling spatial forms and supporting remote communication that appears to be like face-to-face interaction. All design has to be biased in some way, and the bias towards space in CVE research may be due to it being technologically led more than sociologically, semiologically or philosophically led. The arrival of cheap graphics hardware and the eye-catching novelty of 3D images gave rise to a good deal of work that focused rather narrowly on the construction of rather decoupled and decontextualised information spaces. This is true in some of my own work over the past twelve years, ranging from Chalmers (1991) to Morrison et al. (2002), and there are strengths, weaknesses and alternatives to such a bias (Chalmers, 1999). Newer technologically driven research is weakening or revealing CVE research’s implicit assumptions of space’s primacy and independence. Many of the characteristic design principles and assumptions were established before the current fashion for mobile computers and ubiquitous

73

Inhabited Information Spaces

computing. Nowadays, it is possible to obtain tolerable frame rates for 3D graphics on a wirelessly net-connected handheld computer. A person can thus be walking down a city street with a friend, chatting about a museum they intend to visit, while simultaneously watching the avatar of another friend moving through a CVE – with that “remote” friend also taking part in the conversation. In this case, it would seem difficult to claim that the person “inhabits” the information space. One might ask whether the person is in digital or virtual space, or in real or physical space, but the question is based on two false dichotomies: digital media are no more or less real than older media, and computers are just as physical as buildings and books. The workplace has always been affected by communication with people in other locations. Many traditional, everyday and non-digital media support remote interaction, for example letters, books, maps and the landline telephone. There are already digital media in the contemporary workplace that support remote interaction, such as email and mobile telephones. Nowadays, why do we not speak of “entering cyberspace” when we use email, as people did a decade ago? Why do we not inhabit telephone space, or speech space, or MacDonald’s employee name badge space? I suggest that a principle from philosophical hermeneutics is useful here: we don’t talk about these technologies in such marked ways because we have appropriated them into our everyday life and language. We no longer “enter cyberspace” because email is so interwoven in our everyday life and familiar in our experience that we don’t need to mark it out in such a way. We don’t inhabit telephone space because we understand telephones, in particular how to present ourselves through them and how to present ourselves to “spectators” nearby who can perceive our use of them. We only “inhabit” virtual worlds because their designs are so new and decoupled from other media. Experience and understanding of such coupling lets us focus on the task of communication, not on the tool for communication, just as a carpenter engaged in his work focuses on hammering and not the hammer. We continually mix and couple media in our everyday communication: walking, gesturing and pointing while talking, and referring to places and what people did in them as one writes. Space is an essential part of this mix. It has its unique characteristics that differentiate it from other media, but it has no privileged position above or apart from them. It does not stand alone as a paragon for computational media to emulate. More generally, a medium cannot be fully used or understood in an isolated or “singular” way. People’s activity continually combines and cuts across different media, interweaving those media and building up the patterns of association and use that constitute experience and understanding. A person’s work or activity may be influenced by the configuration of space around them and the interactions that space affords, but also by books, telephones, hypermedia, 3D computer graphics and so forth. People act and work through the full range of media they have

74

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

City: A Mixture of Old and New Media 5 ●●●●●●●●●● ● 1 ●●●●

ready to hand. A narrow emphasis on space as the paramount resource for activity underrates the influence of other media. Recent technological developments, such as mobile phones and email, heighten or highlight a phenomenon already familiar in the use of older media such as written text, maps and cinema, and well explored in older disciplines than computer science. For example, a city’s meaning is not just in its bricks and mortar, but also in our understanding and use of the information about it. At any time, one is likely to have symbols in a number of heterogeneous media available for interpretation and use. As I walk through a train station towards a city square, the map in my hand, the voice of a friend on my mobile phone, the signs informing me of exit routes and the posters advertising exciting shopping opportunities are all open for my interpretation and action. Temporally, symbols in an even broader range of media influence me, as my activity is influenced by my past experience and my expectations of the future. Past experience may include my previous visits to that city, my browsing of a web site with good maps to print out, and my experience of magazines, books and films about urban life, and so forth. My language and culture, spanning media old and new, affect me as much as the immediate perception of spatial form. Since Heidegger and Saussure, a fundamental tenet of philosophy and linguistics has been that language is constituted by all the symbols and all the media one uses, with each symbol interpreted through immediate individual perception as well as past social experience. Contemporary neurophysiology is in strong accord with this view (Churchland and Churchland, 1998; Edelman and Tononi, 2000), as is the field most obviously related to the design of space, architectural and urban design (Leach, 1997). The differences between media are usually very obvious. We can characterise media and treat each one as if it were an isolated individuated entity because of the senses we use in perceiving each one, and also because of our understanding of how to relate and to distinguish examples of each one. For example, it is easy to distinguish the spoken word “red” from the written word red because of the senses one uses in each case. Despite having the same letters, it is easy to distinguish tar from rat by looking at the order of letters within each written word. Simple rules about what one can immediately see, hear, etc. within a word begin to strain and then break when one considers, for example, how we distinguish homonyms such as rose. The written word rose can mean many things, including a flower and having risen. When spoken, the same syllables can also mean linear structures (rows), about or belonging to fish eggs (roe’s), moving in a boat (rows), small deer (roes) and multiple occurrences of the Greek letter  (rhos). The word’s usage is understood through its context – one’s understanding of the other symbols co-occurring with its use – rather than perception of the word’s pattern of syllables or letters.

75

Inhabited Information Spaces

Context becomes progressively more important as we turn from thinking about the differences between media, and distinguishing symbols, to considering the similarities of media and the relatedness of symbols. For example, the spoken word “red” and the written word red are related because we can use either of them in the context of rose blooms, fresh blood, the former USSR and so forth. We understand, relate and differentiate symbols through experience of contexts of use within a culture. As shown in the early twentieth century by Saussure, this understanding is not solely dependent on the form or medium of each symbol, but also on how we use each symbol in the context of other symbols – and this context includes symbols in other media. Taking fuller account of the interdependence of media enriches our understanding of space and of work. Consideration of how to make systems that are consistent with this standpoint opens up new possibilities for technology design and for computer-mediated social interaction. More particularly, it opens up a wealth of approaches based on coupling and contextualisation. For example, a museum exhibition might be associated with a set of web pages so that walking into a room of a particular architect triggers the display of text describing the life and work of that architect. Similarly, reading the text might trigger display of a map or visualisation of the room, affording access to a structured collection of blueprints, design sketches and building models. The space of the room would be coupled with the text of the page, with each becoming part of the context of the other. In terms of social interaction, a person walking into the room might be made aware of a friend’s reading of the web page, and hence open to conversation about the exhibition despite the two people being geographically remote from each other. Our intention is to support social interaction, as is familiar in traditional museums where co-visitors use awareness of each other’s interaction with exhibits as a resource for their interaction with each other, and use interaction with each other in interpreting the exhibits (Falk and Dierking, 1992; Galani and Chalmers, 2002). The City project explores the coupling of new and traditional media, weaving them together to form resources for social interaction and interpretation. In particular, our 2002 experiment explored social interaction between people in different locations and contexts where, by definition, they have different resources at hand. As they discuss and refer to contextual information, heterogeneity of media is inevitable: one person can use the nondigital resources of his or her location while others have only digital representations of that location. A case that is more easily handled is audio: each person will hear his or her own voice and sounds from other nearby sources differently to others, because of the digitisation and transmission of audio, but we have become relatively accustomed to handling this. A much more challenging heterogeneity is that of people’s position, orientation and gesture within rooms and buildings. For example, the Mack Room presents much greater visual and tactile richness than

76

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

City: A Mixture of Old and New Media 5 ●●●●●●●●●● ● 1 ●●●●

the room’s digital representations, e.g. maps and VR models. Unlike most earlier CSCW research, the City project addresses this inevitable heterogeneity by coupling media together, tracking activity in each medium and representing it in others, and so letting participants interweave these media in their social interaction.

5.3 System This section outlines our prototype system, beginning with its infrastructure: the EQUIP platform, the Auld Leaky contextual link server, the VR Juggler framework, and the Bristol ultrasonic positioning system. More detail of this system can be found in MacColl et al. (2002). The EQUIP platform is being developed within Equator to support information sharing between heterogeneous devices. It provides a runtime infrastructure to support interoperation between Java and C++, and supports extensibility, for example dynamic loading of C++ classes. The University of Nottingham is leading the development of EQUIP, with contributions from the various Equator projects. City uses it as a blackboard architecture through which VR Juggler, Auld Leaky and the Bristol ultrasonics interoperate. Data items representing user context, an underlying spatial model and context-dependent content are stored for manipulation by City clients and services. Additional EQUIP facilities support real-time 3D graphics and mathematical operations, and abstract, renderable scene graph nodes. In addition, interfaces between EQUIP and a number of other systems have been developed, including the University of Iowa’s VR Juggler. VR Juggler is used as the renderer for 3D graphics. It is described as a virtual platform for virtual reality (VR) application development. It is a high-level, cross-platform architecture supporting immersive and nonimmersive presentations. Both UCL and Nottingham have immersive projection facilities, and the UCL facility has been used for development and pilot trials in the City project. The 3D graphics rendering is used to provide an analogue to the traditional exhibition space visited by traditional visitors. For World Wide Web visitors, the space is represented as a 2D map. We also require a presentation of the information in the exhibition displays, and this is provided by Auld Leaky. Auld Leaky is a lightweight contextual link server being developed within Equator to store and serve hypermedia structures, using context to filter query results. The model used to define the structures is capable of representing a variety of hypermedia domains: link-based, spatial and taxonomic. Auld Leaky is being developed by the University of Southampton, and is written in Perl although it has a Java API. Information is encoded as an XML linkbase, loaded into Auld Leaky and queried using HTTP. The text and images of the hypermedia were taken from the Mack Room’s catalogue. Contextual queries are used to

77

Inhabited Information Spaces

generate location- and device-specific content to be delivered by the Apache World Wide Web server and servlet engine. For 2D or 3D renderings, a location can easily be derived from the position of a visitor’s avatar or map marker. For the mobile computer, an ultrasonic system is used to provide position information. The University of Bristol is developing a low cost indoor positioning system using a combination of radio frequency (RF) and ultrasonics (Randell and Muller, 2001) as part of its contribution to Equator. The system uses a single RF transmitter for synchronisation, with ultrasonic transmitters for positioning. The ultrasonics transmit at known intervals after the RF, and are received by a handheld or wearable receiver. Each second, the variations in flight time of the ultrasonic transmissions are used to calculate the spatial position of the receiver. The receiver incorporates a magnetic compass to provide orientation information. The City project installation involves eight ultrasonic transmitters covering the approximately 10 m by 20 m area of the Mack Room. The room is a challenging environment for ultrasonics, as it is split into two large areas by a partial “time line” wall and has some areas set up as cubicles within which ultrasonic reception is virtually impossible. For aesthetic and coverage reasons, the transmitters are set on top of walls, displays and cubicles so that ultrasonic transmissions are reflected off the ceiling. The system supports a shared visiting experience, with one visitor using a handheld or wearable computer in the Mack Room, a second visitor using the World Wide Web on a laptop or PC in another room, and a third using 3D graphics on a similar machine in a third room. These computers communicate through 802.11 wireless Ethernet. A separate audio subsystem, that we will not detail here, handles visitors’ speaking to and hearing each other. In discussion and development scenarios we name the visitors Vee, Dub and Ana respectively, and these will also be used in the remainder of this chapter. The names do have roots: Vee is for ‘visitor’, and Vee was the user in our first design scenario; Dub is from the first syllable of “double-U”, as in WWW; and Ana stems from ‘analogue’, playing with the way that the digital space of the VR is an analogue of the Mack Room. Some people have suggested that Vee should be the ‘virtual visitor’ but we decided to keep the name, so as to irritate those who think that Vee’s experience is not strongly influenced by digital media. Spatial awareness is supported by tracking activity in each of the 2D, 3D and handheld systems, sending position and orientation information for each one into EQUIP, and then rendering the information about all visitors to each individual visitor. The components of the prototype system operate similarly for each visitor, broadly as follows: 1. store spatial position and orientation in EQUIP; 2. retrieve and render positions of other visitors;

78

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

City: A Mixture of Old and New Media 5 ●●●●●●●●●● ● 1 ●●●●

3. store named location in EQUIP in response to position change; 4. store content from Auld Leaky in EQUIP in response to location change; 5. format content for presentation and advise client program of availability in response to content change. For Ana, position and orientation information is automatically published in the EQUIP data space by the VR Juggler client, and the positions of all visitors stored in the data space are automatically rendered as 3D avatars. Figure 5.1 shows a non-immersive spatial awareness display for Ana, with avatars representing Vee and Dub (displaying only heads rather than complete avatars). Vee uses a Hewlett-Packard Jornada that polls position and orientation sensors, and sends the results via a proxy into EQUIP. The proxy is also responsible retrieving the information about the other visitors, and the positions and orientations of all visitors are presented to Vee on a 2D map. Figure 5.2 shows a visitor the Mack Room, with a handheld and an ultrasonics receiver. The figure also shows a close-up view of the handheld as an inset.

011

011

011

11

Figure 5.1 A non-immersive VR display of the Mack Room for Ana, with avatars representing Vee and Dub (displaying only heads rather than complete avatars).

79

Inhabited Information Spaces

Figure 5.2 A visitor in the Mack Room, “Vee”, with a handheld computer and ultrasonics receiver. The figure also shows a close-up view of the handheld as an inset.

Dub interacts with a Java applet in a World Wide Web browser frame. The applet communicates via a proxy that converts mouse clicks on a 2D map of the Mack Room to position and orientation information. The applet also displays representations of all visitors. An example of Dub’s map is shown in Figure 5.3, corresponding to Ana’s 3D display in Figure 5.1. The red boxes on Dub’s map are trigger zones, discussed in the next section. Vee’s map is similar to, but simpler than, Dub’s. Shared visiting requires a sense of shared context, and hence some comparability of the information available to each visitor, but we also

80

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

City: A Mixture of Old and New Media 5 ●●●●●●●●●● ● 1 ●●●●

011

011

011

011

11

Figure 5.3 An example of Dub’s map. For paper publication, the image has been annotated with the names of the visitors.

wanted to maintain and explore a degree of the heterogeneity that is inevitable in remote collaboration. Vee has the rich environment of the Mack Room’s displays and artefacts, as shown in Figure 5.2. Providing hypermedia to Dub and Ana involves converting positions to named locations, querying Auld Leaky with the visitor’s device and location in order to generate informational content, and then formatting and presenting the content. In initial trials of our prototype system we did not deliver hypermedia content to Vee, encouraging her to use the rich content of the existing room when interacting with her friends. Dub and Ana have rich access to the web that Vee lacks, and they can move and jump between Mack Room locations in ways that Vee cannot. Ana’s

81

Inhabited Information Spaces

3D view of the Mack Room VRML model has greater visual richness than the 2D maps of Dub and Vee, but she also has visual occlusions to deal with. Positions are converted to locations by an EQUIP-connected service. These locations represent semantically significant volumes or extents within the spatial model (shown as red outlines on Dub’s map, as in Figure 5.3). Also, for each visitor a target is inserted into the EQUIP data space, currently equivalent to a 10 cm cube held in the hand. Detection of a collision between a target and a sensor invokes code that inserts a new (user, location) item into the data space. Adding or updating a userlocation item in the data space triggers a query to Auld Leaky and the results – a set of hypermedia fragments – are stored in the EQUIP data space. Adding or updating such a set of hypermedia fragments in the data space triggers formatting for delivery to the visitor. The content fragments are retrieved and combined into an HTML page. Dub’s applet displays the HTML page in a separate browser frame set aside for this purpose. Ana also runs a browser and applet that, when advised, displays HTML pages, i.e. Ana has no map display but does have a textual display. The overall effect, then, is that all three visitors have location information about their co-visitors’ shared awareness of location. Dub and Ana each have spatial and textual information about the exhibition, with the text updated as they use the 2D map and the 3D graphics. Vee has the traditional information of the exhibition room. Many of the artefacts and exhibits have corresponding representations for each of the three visitors. We sometimes refer to such artefacts and exhibits as “hybrid objects” because of the visitors’ tight interaction around corresponding heterogeneous representations. Space, text and audio afforded sufficient interaction, context and reference to support a shared visiting experience, as the next section discusses.

5.4 Use In the summer of 2002, the City project carried out a set of system trials in the Mackintosh Interpretation Centre. Rather than trying to make the best possible system for the Mack Room in particular, our focus was on general lessons we could learn for the design of systems involving heterogeneous representations and interactive media. We aimed to increase our experience and understanding of how these systems serve as constraints and resources for users’ interaction. We had already studied the use of a number of cultural institutions, including the Mack Room, without our technology (Galani and Chalmers, 2002), and were interested in comparing the Mack Room with and without our technological intervention. Fuller discussion of the trial can be found in (Brown et al., 2003). The trials involved 34 participants: ten groups of three and two groups of two. The groups of three consisted of a Dub (using the web), an Ana

82

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

City: A Mixture of Old and New Media 5 ●●●●●●●●●● ● 1 ●●●●

(using VR) and a Vee (using a mobile computer). Dub was in the Interpretation Centre, while Ana and Dub were in separate rooms on a different floor of The Lighthouse. The pairs explored different combinations: Dub and Ana visiting without the physical visitor (one trial), and Vee and Ana (one trial). For the first half of the trial, participants were asked to explore the Centre together, to familiarise themselves with the technology and how they could co-visit. Since we were specifically interested in how the system supported social interaction, we introduced an artificial task for the second half of the trial. Each participant was given three questions, and the group was asked to answer these questions together. Some questions were designed to provoke open-ended discussion and interaction between the participants. For example, participants were asked “What is the group’s favourite Mackintosh painting?” and “What contribution has Mackintosh made to Glasgow?” as well as more factual questions such as “What was Mackintosh’s birthday?” This combination of open and task-centred behaviour allowed us to study activity that was typical of a museum visit, such as finding exhibits, and to observe how the system supported the shared aspects of visiting a museum. During the trial, use of the system was heavily logged and each Dub was video taped. After the trial, the participants were interviewed as a group in a recorded semi-structured debriefing. For analysis, we combined the map view used by participants, with the video and audio recordings. We analysed transcripts of the post-trial debriefings, and the logs of the visitors’ use of the system. We paid close attention to the details of how users interact with each other and with technology, especially through video analysis. With particular interest in the use of location and of exhibits, we looked for “critical moments” where the system was used in a way that would let us reveal design lessons, consistencies and inconsistencies with theory, and comparisons with earlier studies. The participants engaged in rich social interaction around the hybrid or coupled exhibits. When participants found that objects corresponded in this way they quickly were able to move on to using them in their shared tasks and activities, for example discussing the qualities of the object and comparing it to the other exhibits. In the following extract the participants discuss a set of Mackintosh pictures to decide which one they like the most. Square brackets (e.g. [pet]unias) show overlapping talk and italics shows a speaker’s emphasis. Vee: Petunias is errm better for me than Rosemaries Ana: Ok [pet]unias Dub: [hhh] Petunias it is Vee: Early work Ana: Hey guys see: this other one it’s really nice. It’s called Fort Mailly hhh Fort Mailly in nineteen twenty seven Vee: Nineteen twenty seven Ana: Yeah, it’s got the light

83

Inhabited Information Spaces

Vee: Yeah I know but I like [Roses] Ana: [Can you] see it? Dub: Fort Mailly? Ana: Hmmmm Dub: Yeah that’s quite nice Vee: I still prefer Roses

The visitors do not focus on the system or the media involved, such as the differences between the digital and the printed reproductions of the Fort Mailly painting. Instead they focus on its aesthetic qualities and the task of deciding which picture they like most. They don’t focus on the tool but on the task. However, interacting around these hybrid exhibits was not without its problems. In ordinary face-to-face interaction, we assume a degree of commonality in the objects that we can see, hear, touch and so forth. Each user of our system had to build an understanding of the perspectives the other users had of the Centre. They could often see similar things, or related things, but not the same things from the same perspective. Dub and Vee frequently guided Ana to specific exhibits verbally. We designed the system so that Ana did not have an overview map, and therefore might find that occlusion of objects in the exhibit was a problem. In turn, since Dub and Ana could shift attention between artefacts placed far apart in the room much more quickly than Vee, they frequently found information and then guided Vee to the corresponding location. Again this was often done verbally, but more spatial or graphical guiding is discussed later. Participants also made use of shared location and orientation, using icons on an outline map for Vee and Dub, and avatars in Ana’s 3D display. Shared awareness of location also allowed users to quickly move to their friends, and to quickly find or confirm the exhibits being discussed, that is, to quickly find what their friends were looking at and then move so as to look at the same or a closely related thing. They developed simple means to gesture. In one case, Ana moved her avatar back and forth in a “wiggle” so as to confirm which icon represented her and to show her location to Dub. This gesture was something like the wave of a hand used by someone to show he or she is in a crowd. Global location could be seen “at a glance” on the map, without the need for the visitors to use talk, but examples such as Ana’s wiggle show participants’ awareness of how they would be perceived by each other in different media. Participants learned about each other’s perspectives through questions and observations, building understanding of what they shared and what they did not, and thus how to more smoothly interact with each other through the resources at hand. Indeed, participants put considerable effort into designing their interactions to take into account the characteristics and limitations of their varying views of the Centre. For example, the above extract shows Vee emphasising and confirming the year in

84

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

City: A Mixture of Old and New Media 5 ●●●●●●●●●● ● 1 ●●●●

which “Fort Mailly” was painted (“nineteen twenty-seven”). One of the largest exhibits in the Mack Room is a long wall of panels, with each of the chronologically ordered panels showing a number of images associated with a year of Mackintosh’s life. Mentioning a year to Vee would let her move quickly to a corresponding part of the exhibit. The information presented to Ana and Dub about this wall was broken up into a page for each year, so any one of the visitors could help guide the others to a particular image in that exhibit by specifying the year. The use of our system involved more talk, and louder talk, between the co-visitors than we observed in conventional museum settings. For example, during a post-visit discussion two participants were asked: Q: Is it different to a museum visit? A: Yeah, it’s really talkative. B: You kind of go “Mmmm, that’s nice” [. . .] If you find something interesting, you go “Look”, and “That’s over here”

Another commented on being able to talk without disturbing others: I quite enjoyed the social engagement . . . being able to talk about everything more and not feeling that you are disturbing. Not thinking about other users in the gallery, you know it’s kind of liberating . . .

I offer here a few of the many possible reasons for this, stemming from the use of multiple rooms, the audio hardware used and, perhaps more interestingly, the set of media available for interaction. Two of the participants were in rooms other than the traditional exhibition room, and so were less influenced by the normal hushed reverence adopted inside museums. All three participants were engaged in a trial of an unfamiliar technology, without established social norms for speech levels. Vee often talked more loudly and paid less attention to other members of the public in the Mack Room. Much like a telephone, the loudness of speech needed for audibility through the audio hardware may not be the same as in faceto-face interaction. Lastly, the participants often used shared audio because the coarse-grained representations of position and orientation did not afford familiar use of gesture and posture. However, participants were not solely using talk in order to circumvent the tools given; they were often talking while engaged in the tasks. Also, they developed gestures for the media at hand, for example Ana’s wiggle: a gesture made in a 3D VR to be seen on a 2D map. In everyday face-to-face situations, interaction between two people can be impeded for a number of reasons such as one or both of the participants choosing to interact with others (e.g. on the telephone) or to interact with objects, or by participants being forced to interact with other people or objects because of interruption, breakdown, occlusion and so forth. In the trial, interaction would often pause when participants found a difference between the visitors’ representations of the Centre. For

85

Inhabited Information Spaces

example, the room’s interactive video displays were only available to Vee. When a visitor started to use and talk about a display or exhibit that was not shared, the other participants would refrain from interacting and move on to other exhibits. Similarly, Dub and Ana generally used movement on the map or VR in order to access information, in preference to the more conventional hyperlink navigation. This seems to have been partly due to the fact that such spatial movement was more a part of the shared experience, in terms of conversational references to locations but also as a way of avoiding future confusion: following links to a web page about a new exhibit did not move the participant’s icon or avatar to the corresponding new position. (This capability was implemented during the trial period but, to maintain consistency, was not deployed.) This meant that “web movement” could leave a visitor’s icon in a potentially confusing place. While there were occasional interactional breakdowns, they were not fatal for the sense of a shared visit or for interaction. Overall, participants showed skill in finding ways to handle the differences between the different representations, and exploiting corresponding and coupled features. The system successfully supported a shared experience by enabling users to talk about and interact around the exhibition, offering a socially engaging experience beyond that available to a conventional web site visitor.

5.5 Ongoing and Future Work We continue to explore remote collaboration in cultural information and cultural institutions, in particular collaboration involving heterogeneous media. We support social context as a resource for the interpretation of information, and contextual information as a resource for social interaction. We are extending our system to be used in more of the city than The Lighthouse, adding GPS (Global Positioning System), dead reckoning, GPRS (General Packet Radio Service), and further 802.11 aerials. We are using the publicly available VRML model of central Glasgow from the University of Strathclyde, and 2D maps from services such as EDINA (www.edina.ac.uk). We have been undertaking field studies of visitors in a range of locations in Glasgow, seeing how their visits include far more than traditional cultural institutions, and how they use resources such as tourist information centres, maps, guidebooks and signage. We aim to run another field trial that involves a number of participants visiting the city, each of whom has a wearable computer on which tightly coupled 3D VR, 2D maps and hypermedia can be used. Each participant’s activity will be available to the others synchronously, much as in the Mack Room, but also asynchronously. One reason for this latter development is to move beyond the traditional objectifying or “scientific” systems of classification and retrieval

86

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

City: A Mixture of Old and New Media 5 ●●●●●●●●●● ● 1 ●●●●

that too often are the only means of access to digital information. Influenced by the theoretical standpoint outlined in Section 5.2 and the Recer recommender system (Chalmers et al., 1998), we have built a central information resource, connected to EQUIP and thus accessible via a variety of media and devices, and which stores a growing and evolving body of individuals’ paths or narratives through a range of symbols: our own images and fragments of hypertext, annotations made by users, locations in the city, and other locations and web pages worldwide. The system uses this resource to make contextually specific recommendations of people, places and things by comparing each person’s recent activity with similar sections of the past activity of selected others. Paths can also be more directly shown on maps, in VR models and woven into web pages. We will allow this body of information to grow as people use it, making new associations between symbols and adding in new ones. Information access based on evolving inter-subjective patterns of contextual association and use will complement access based on more static and objective interpretation. We are also experimenting with bridging between the two, using pre-written “official” explanations of the connections between symbols as a means to enrich the dynamically created recommendations. Remote collaboration brings abstraction and approximation as a system monitors and senses activity such as a person walking across a city. Issues such as sampling, resolution, delay, disconnection and uncertainty have to be faced as one decides how to represent the activity inside a system, even before one considers how to represent the activity to a remote collaborator. There is no getting away from the fact that activity is going to be interpreted through sensors and transducers such as cameras, GPS and ultrasonic systems, and any digital medium has characteristic losses and uncertainties. While we want to make new resources for interaction available to people, combining new media with old in a perfectly seamless way is not going to happen. Uncertainties and inaccuracies (“seams”) are an inherent part of any communicative medium, and people often learn to use these characteristics for their own ends. For example, mobile phones can be set to display the current cell, if the service provider permits, and some people choose to enable this facility. This is an elegant ambient or peripheral presentation of potentially useful information: users can choose what use to make of it, for example seeking a stronger signal by moving to a location that forces handover to another cell. Cell boundaries and signal strengths are interactional resources of the medium. Similarly, long-term use of videomediated communication was reported by Dourish et al. (1996) to lead to “complex patterns of behaviour built up around the interactional details of the video medium . . . When the medium changes, the mechanisms change too; but the communicative achievements remain.” Recalling a term used by Mark Weiser (1994), we see seamfulness as an important design goal for our future work. We plan to design in

87

Inhabited Information Spaces

explicit representations of the errors and uncertainties in our systems, letting people take account of the characteristic heterogeneity, errors and limitations of the systems we offer them. For example, we are starting to develop explicit presentations to accommodate uncertainty due to ultrasonic and GPS-based positioning, showing a person’s sensed position as a spatial extent, rather than as a point, and showing estimates of sensing accuracy and communications bandwidth on our city maps and models.

5.6 Conclusion The City project emphasises the interdependence of media, such as computer graphics and audio, with others. We have explored the combination of CVE technology with hypermedia and mobile computers, and also with the architecture and exhibits of the Mackintosh Interpretation Centre. Supporting broad social context in remote collaboration involves heterogeneity, and our project aimed to address this through coupling and correspondences between media. Users of a mixed collection of interactive media were able to enjoy a shared visit experience, engaging in collaboration through awareness of each other’s activity and through more focused talk and interaction around “hybrid” objects. By presenting the theoretical issues underlying much of this work, as well as system design and experiences of use, this chapter may serve to complement many other chapters of this book. Rather than considering that users inhabit our information space, we see people as inhabiting cities and towns, and using new technologies and older media to interact with friends, relatives and colleagues. Looking to the near future, CVE technologies will be widely available via phones and mobile computers. I suggest that taking fuller account of their use among a wider set of technologies and media, and designing for contextuality, heterogeneity and seamfulness, will greatly enrich our work.

Acknowledgements Special thinks go to all the City project members past and present, especially Barry Brown, Areti Galani, Chris Greenhalgh, Ian MacColl, Dave Millard, Cliff Randell and Anthony Steed. Also, we are all grateful for the generosity of our hosts at The Lighthouse, especially Lynn Bennett and Stuart MacDonald. Thanks also to Ziggy Stardust, Fugazi and Low. Equator is an Interdisciplinary Research Collaboration (IRC), supported by the UK Engineering and Physical Sciences Research Council, and the City project was supported by a donation from the HewlettPackard Art and Science programme.

88

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Soundscapes

6

Tony Brooks

011

011

011

011

11

6.1 Introduction Although purely virtual information spaces receive much attention and are indeed useful in certain situations, it can be difficult to integrate them with real-world activities and awkward to interact with them. As pointed out in Chapter 5, even notionally single-user activities often happen together with other people and a purely virtual environment tends to cut people off from their surroundings making it difficult for several colocated people to share an experience. Fraser et al. (Chapter 9) also show that collaborative interaction within a CVE is not quite as intuitive as it might first appear. For these reasons purely virtual environments may not be desirable in cases where the participants have a disability or where the environment is intended to be shared with other people, as in a public performance . The creative process is something that is inherent in most people, even the severely disabled, yet it is an often uncharted channel within which to explore their expressive human potential. “Productive creativity” is becoming recognised as a beneficial therapeutic treatment for people with disabilities. In this chapter the Soundscapes system, which is built to allow unencumbered creative expression for both able-bodied and handicapped people, will be discussed. The next section will describe the system itself and then further sections will show how it has been used therapeutically and also to stage public performances.

6.2 The Soundscapes System The origins of the Soundscapes system lie in a MIDI bass (a modified bass guitar that can generate MIDI – Musical Instrument Digital Interface – signals allowing the control of electronic synthesisers) and an expression pedal (a foot-controlled sound effects processor that can change the sound of an audio signal). From interactions with his severely

89

Inhabited Information Spaces

handicapped uncle in which the author played the bass and his uncle manipulated the expression pedal the author realised that even a simple means of expression could open up ways for people to communicate and could provide immense satisfaction. The author started searching for a richer means of expression and it soon became clear that the MIDI bass itself was too intimidating and required too much physical skill to be satisfying for people like his uncle. This led to a search for a rich expression medium that required no training to use. Although immersive reality hardware could track peoples’ movements in a natural way it was expensive and uncomfortable to use to the equipment since it had to be physically worn by the user. After much experimentation the author selected infrared movement sensors. Within the range of the sensor, movement from the flicker of an eye to a full body movement can be detected. The Soundscapes system uses three such sensors to allow triangulation of movement or triggering of three separate operations. Each sensor head is mounted on a flexible “gooseneck” support to allow for different configurations. The

Figure 6.1 Photograph of the first three-headed infrared movement sensor used by the Soundscapes system.

90

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

Soundscapes 6 ●●●●●●●●● ● 1 ●●●●●

sensors output MIDI information that have been used to control movement and navigate through 3D space as well as controlling filters applied to computer-generated images. The Soundscapes system is formed from a library of tools for body function capture and a number of programs for creating the results that each user response determined. The library of input tools ranges from biofeedback sensors for brain activity, muscle tension, GSR (galvanic skin resistance), heart rate, to infrared (see Figure 6.1), ultrasound and video tracking (the EyesWeb http://www.eyesweb.org/ is used to perform the video capture) technologies. Most often the captured body movement data is transmitted using the MIDI protocol and the MAX program (http://www.cycling74.com/products/maxmsp.html) is used as a central hub directing and manipulating the data as needed. In conjunction with a physiotherapist the Soundscapes operator calibrates the system such that the range of velocities measured by the sensors matches the movement capability of the individual who will be participating in the session. The system is configured so that movement triggers audio and visual feedback with the exact nature of the feedback depending on the individual – we have found that some individuals respond better to audio stimulus and others to visual stimulus. System settings as maintained between sessions and movement data are logged so that the therapist can measure progress from session to session.

011

011

11

Figure 6.2 Version two of the three-headed infrared sensor system in use. The three sensor heads are visible at the centre of the image in between and a little below the performer’s hand and the image projected on the screen.

91

Inhabited Information Spaces

6.3 Therapeutic Uses of Soundscapes One of the uses of the Soundscapes system is to act as an “expression amplifier” for people with physical disabilities – it provides them with the possibility to generate sounds and images from whatever movement they are able to control and this can be a powerful therapeutic experience. The therapeutic use of Soundscapes is illustrated by the i3 funded future probe project called The World Is As You See It (TWI-AYSI, http:// www.bris.ac.uk/Twi-aysi/). TWI-AYSI was a successor to the CARESS project, which was also funded by i3 as part of the Experimental School Environments programme. The TWI-AYSI project members consisted of members of the Bristol University Electrical Engineering Department, and Stefan Hasselblad both of whom were involved in the EU CARESS project, and Tony Brooks. The CARESS project successfully motivated and empowered children to develop creativity, imagination and expression through interactive acoustic environments. The objective of TWI-AYSI was to answer the question “Can immersion in a visual environment hold similar potential for such children in terms of the aesthetic resonance they might derive from movement within such a visual space?” During the project we brought young children from a school for multi-handicapped people into the Centre for Advanced Visualisation Projection screen Video camera

Full scene camera

Infrared motion sensor

Video projector Disabled child Infrared motion sensor Video camera

Computer

Face shot camera

Camera feed quad splitter monitor & VCR

Figure 6.3 Typical set up as used in TWIAYSI (Sweden 2001) with Multiple Camera Analysis (MCA) utilising six video cameras and three infrared motion sensors.

92

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Soundscapes 6 ●●●●●●●●● ● 1 ●●●●●

and Interactivity (CAVI, http://www.cavi.dk) in Denmark. The children ranged in age from three-and-a-half to five-and-a-half. A common experience for all of the children involved was the manipulation of sounds, robotic lights and coloured images. However, in CAVI we also wanted to explore the possibilities for using the movement sensors to enable 3D navigation. MIDI signals from the movement sensors were received by a Linux workstation and translated to movement information that was used to control the rendering of the 3D world on a SGI Reality Monster (Silicon Graphics Inc.). In this experiment, the movement sensors allowed a person to control the movement of a rocket ship that was projected onto a large screen in front of them. It was possible to view the image in stereo using LCD shutter glasses . However, the children did not want to wear the necessary head sets and shutter glasses that were used to view the virtual objects and so we made the decision to project the ship in mono and the youngsters once placed with their head inside of the active sensor space, were able to control the various degrees of movement of the spaceship by small head gestures (a video of this is available online at http://media.nis.sdu.dk/video/ twi-aysi.html). A small gesture to the right and the rocket ship moved to the right, a movement left and the rocket ship moved to the left, a head movement down and the ship’s nose tipped down as if to dive, and a head movement backwards and the ship’s nose tipped up as if to climb. As an example of the results of this experiment we shall describe a session with a multi-handicapped five-year-old boy. At first we tried to place the 3D shutter glasses on him so he could view the scene in stereo but he shook his head to remove them. We removed the glasses and changed to a mono projection. This was a big success as the young boy was totally immersed for around six-and-a-half minutes. The young disabled boy was only able to express through a single verbal expression (a sort of “AYYE” phrase). While in the session and immersed in the interaction he could clearly be heard making his sound, which was translated by

Figure 6.4 Freja, a severely disabled girl with her helper Maggie. Freja is painting via her movement and can see herself in the window on the right. The colour painted depends on the velocity of the movement, with each individual having his/her own settings to match their ability. Freja’s face tells the story. Reproduced with permission from Stefan Hasselblad.

93

Inhabited Information Spaces

his helper as an expression of a joyful experience, and this was probably due to the fact that he quickly understood the functioning of the interactive space and was able to control the image on screen – an experience of controlling his environment that was probably completely new for him. As a result of the CARESS and TWI-AYSI projects we learned that when audio only or visual only feedback was present some people reacted better to audio and some better to video. When both audio and video were present in the environment a greater degree of immersion was observed. This work resulted in another European project to take these ideas further, called CARE HERE (http://www.bris.ac.uk/carehere), whose results are due to become available as this book goes to press.

6.4 Artistic Performances Based on Soundscapes 6.4.1 Interactive Painting The inspiration for “The Interactive Painter” came from a discussion at a party in which it was stated that some people believed that painting on canvas was “dead” since computers enabled us to do so much more with Video camera Projection screen Infrared motion sensors Back projected image

Mirro

Painter

Video camera Computer

Video camera

Video projector

Figure 6.5 Diagram of the equipment used in the interactive painting performances. The three sensors capture motion via infrared beams. In this diagram the video image is back projected but front projection is also an option to allow working by casting shadows and to overlap the video on a canvas.

94

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

Soundscapes 6 ●●●●●●●●● ● 1 ●●●●●

images and colours. We decided to try to investigate a way for a traditional painter to utilise new technology while painting on canvas in the traditional way. The resulting performance involved the painter Manu Rich from Paris, France and within Tony Brook’s COIL (Circle of Interactive Light) interactive installation, it toured a number of museums of modern art in Scandinavia in 1998 and 1999 culminating at the Danish NeWave festival performed at the Gershwin Hotel, in Manhattan, New York 1999. More recently we have experimented with a therapeutic version of the installation for elderly and handicapped people. Interactive painting is designed to be a live experience with an audience and takes place in a darkened room, which provides a challenge to the painter since painting normally requires a well-lit room. The event makes use of projected computer-generated imagery and synthesised sounds that change according to the movement of the painter. In turn the painter may be influenced by the sound and images thereby creating a feedback loop. In order to give maximum freedom of movement, infrared and ultrasonic movement sensors were used to capture the painter’s movement, thereby allowing him to move freely without trailing wires. Owing to the size of the canvas and the limited range of the infrared sensors a jacket with reflective strips attached to it was worn by the painter, thereby allowing movement to be detected from a greater distance. The sensors were strategically set up around the canvas and translated light reflected from the jacket into movement information that a computer translated into sounds, colours and images. One of the primary aims of the performance was to see how the painter would be influenced by images projected onto the canvas. The light from the projected image would alter the appearance of the painted image and change how both the artist and audience perceive it. In order to do this a video projector is set up to project images onto the canvas. The movement sensors control filters applied to the image such that when there is no movement there is no image. As the painter moves then portions of the image are projected onto the canvas.

011

11

Figure 6.6 Photograph of an interactive painting session Gershwin Hotel, Manhattan, New York 1999.

95

Inhabited Information Spaces

In addition to the projected image, the painter’s movement also triggers sounds from a sound synthesiser. The author changed the synthesiser’s patch data in real-time to match the “mood” of the composition and also to try and influence the mood of the painter. The audience of an interactive painting performance therefore has many elements to focus on: ● ● ●

The painting that the painter is actually producing on the canvas. The movements of the painter, which the painter may change in order to change the sound and visuals as well as the painting. The sound and images caused by the movement of the painter.

The artist naturally tends to focus on the creation of the painting and not the multimedia experience. However, it was apparent that the sound and images had an effect on the painter and the painting. From experience we noted that smooth “pad” sounds tended to result in a corresponding smooth motion of the artist’s brush and passive choice of colour and similarly a hard-edged “Hendrix” guitar lick resulted in an aggressive motion and severe choice of colour.

6.4.2 The Four Senses In April 2002 the first collaboration between artist/researcher Tony Brooks and artist Raewyn Turner resulted in a series of multisensory performances called The Four Senses in Auckland, New Zealand, with the Aotea Youth Symphony, Touch Compass, (a mixed able-bodied/ handicapped dance company), and HANDSS (Hearing ANd Deaf Sign Singers, a deaf signing choir). The performances were an improvisation in light, sound and olfactory information. Brooks and Turner created a real-time translation of sound and the gestures of making that sound, into light and colour and multiple layers of smell. The concerts aimed to engage and reframe perception of music and to play with subjective experiences and simulated synesthesia. Each sensory element was constructed from information relating to the other elements. The associations and correspondences of the elements made by the audience were according to their own individual and personal experiences. Tony Brooks utilised movement sensors and video cameras to capture body part movement and translate it into painting with coloured light. In this way the orchestra conductor was able to “paint” the scene through his gestures. Similarly, orchestra members, dancers and a special signing choir for the deaf images were blended into the backdrop in real-time such that their velocity of movement affected the colour of image generation and collage composition.

96

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Soundscapes 6 ●●●●●●●●● ● 1 ●●●●●

Projection screens Percussion / Timpani

Primary camera capture areas

011

011

011

011

Lead cello

Lead violin

Deaf signing choir Basses

Singer Conductor Dancers

3 × roving video cameras

Figure 6.7 Diagram showing the locations of performers and projection screens for the Four Senses performance.

Raewyn Turner translated the music through drawing and proportional relationships into coloured light and olfactory stimulation. This was done using computer-controlled lighting rigs and by releasing aerosols into the ventilation system at key moments during the performance. The light collage thus created was a play of interaction between live video feeds and sensors, and coloured light pre-programmed to an interpretation of sound, each affecting the other in a dynamic visual loop. The fourth sense employed was touch. In order to give hearing impaired people a feel (pun intended!) for the concert experience vibrating cushions were used. These cushions were developed as an accessory for the Sony Playstation® and can transmit vibration at various levels of intensity. These cushions were coupled to the PA system so that they would vibrate according to the music played by the orchestra and a number of them were distributed throughout the auditorium. Hearing impaired people could sit on the cushions or hug them to their chests in order to get a “feel” for the music. In addition a deaf signing choir participated in the performance to counterbalance the conventional (but sight impaired) singer also performing on stage.

11

97

Inhabited Information Spaces

a

b Figure 6.8 Two photos of the Four Senses performance: (a) dancers in front of the orchestra; (b) an example of the projected video generated during the Four Senses performance. Reproduced with permission from Milan Radojevic annd Mirjana Devetakovic Radojevic.

98

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Soundscapes 6 ●●●●●●●●● ● 1 ●●●●●

6.5 Conclusion We believe that there will one day be a programmable ion layer surrounding our body which will be able to stimulate our individual senses so that we are enabled towards a truly augmented reality without wearables and special hardware such as screens. This is not pure fantasy since a number of people in the Nanotechnology community such as Storrs Hall (http://discuss.foresight.org/˜josh/UFog.html) and Drexler (1992) are exploring methods that would allow the creation of a “Utility Fog” (Storrs Hall) that would, among other things, allow our senses to be stimulated and our movement sensed. Our human-centred work with people with special needs, who are disabled, elderly or in rehabilitation allows us, in certain circumstances and with certain individuals, to get closer to higher nuances of the senses and what the senses mean. By exploring that research and working in crossdisciplinary teams including neuropsychologists, computer scientists, human–computer interaction (HCI) researchers and others, we hope to work slowly towards that goal. We believe that the immersive “play” within an interactive environment has much potential to do good. Specific to the current work is the belief that everyone is so individual that a system is required that can be tailored to each desire, facility and requirement – this entails adaptability with a capital A. Libraries of input HCI devices, together with libraries of mapping devices and libraries of output software is the optimal way forward. Through non-fixation on interactive spaces through invisible system components – hardware (disappearing computers, sensors embedded in environments etc.) – we will obtain the mapping of body function subconsciousness that can help people. The subliminal pervasive aspects – or really more so terms such as “proactive computing” (that which focuses on improving performance and user experience through speculative or anticipatory actions) and “autonomic computing” (which focuses on improving user experience through the system’s self regulation) in relation to this work are preferred as both relate to the user experience rather than the artefacts often referred to in pervasive computing – are obvious. We also believe that responsive audio/visual/haptic feedback may have much more to offer in the future over and above what we now utilise, in fact we believe that correspondences between synchronised feedback especially sonic and visual (in the first instance) are only just scratching the surface and that there are many new discoveries waiting to be made.

Acknowledgements The CARESS and TWI-AYSI projects were funded by the European Union via the i3 network.

99

This page intentionally left blank

011

011

011

011

11

The Computational Interplay of Physical Space and Information Space

7

Enric Plaza

7.1 Introduction There is a current trend in computer science to develop new devices and applications that make the transition from desktop (and laptop) computing to computing devices that are embedded in the physical and social environment in which people live. Several approaches have been proposed in this direction and they have different names and focus on related but distinct issues. The first, known as ubiquitous computing, pervasive computing or the disappearing computer, focuses on embedding computing devices into the physical objects and surroundings where people work and live. A second related trend is that of wearable computers, that focuses on embedding personal computing services in devices that people can carry or wear while moving around in their everyday activities. Next, augmented reality focuses on enriching people’s perception of physical surroundings with computer-generated information. And finally, to be brief, there is a trend for developing autonomous agents that take on people’s goals and try to achieve them on their own. However different these approaches are, a common issue they all have to deal with is the awareness of the physical (and social) surroundings in which people interact with computing devices. Traditionally, computers (ranging from mainframes to personal computers) operate in a purely informational world – typically, screens and printers, plus a customised connection to manufacturing machines or task-specific sensors. The advent of the Internet and the World Wide Web links these computers into a common (or rather shared) information world. The relation of the Internet, as an information space, with physical space is a research issue that essentially deals with the issues of context awareness: who is where and when, with whom, doing what. This chapter focuses on the interplay of physical space where people act and live and

101

Inhabited Information Spaces

an information space where software programs reside, interact among them, perceive some properties of the physical world, and perform some tasks and actions on both the informational and physical spaces.

7.2 The Interplay of Physical and Information Spaces Since all computers, and the software they run, are potentially connected over the Internet, we can consider this as an “information space”. Then, the computing devices that populate the physical world, from personal digital assistant (PDAs) to the emerging “ubiquitous computing” devices, can be considered as the interface between the physical spaces people inhabit and the “information space” inhabited by software programs. The most critical issue to improve this interface is that nowadays software programs have little or no awareness of the physical space and of the activities in which people engage in that physical space. In this chapter we will first discuss the general issues that need to be addressed to improve the interfacing of physical spaces by the “information space” inhabitants, in particular awareness of the physical and social context of people. Later, we will present the COMRIS project and explain how these general issues are addressed and solved for a specific physical space (a conference centre) and a series of informational tasks useful in that space. Awareness of physical space involves more than merely spatial or geographic reasoning. People perform activities and interact with other people while moving in physical space, and the more aware they are of those activities the better the interface with the information space. Two levels of context awareness are required: 1. Physical sensors, determining the granularity of the perception of physical space activity. The physical sensors can be a Global Positioning System (GPS – giving a co-ordinate point); wireless tags that people wear to detect who is close by or in which room; or even more sophisticated speech capture and analysis systems (e.g. trying to determine topic, mood, etc., of ongoing activities). 2. Common sense knowledge, determining the inferences a system can make on the “world situation” given the physical sensor information. For instance, if a wireless tag indicates that a person is in a room of type Meeting Room, there is a series of inferences that can be made from a knowledge base that models business activities (e.g. the person is in a business meeting with other people and should not be interrupted unless there is something urgent). Perception using physical sensors establishes the baseline for awareness capabilities. For instance, GPS can be used together with personal devices like PDAs to yield information customised to the person (using the personal profile in the PDA) in that spatial situation (by a server that

102

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

The Computational Interplay of Physical Space and Information Space 7 ●●●●●●●● ● 1 ●●●●●●

has a database of services available near that location). However, this approach is very centralised, depending on a service provider holding geographical information. For some tasks it is better to add more sensors that allow the person’s behaviour to be determined: a microphone can be used to determine if the person is busy (talking with somebody) or idle. Currently, project Esprit “IT for Mobility” 26900 is developing a sensor board that could be integrated with mobile phones or PDAs. The sensors on this board include two microphones, a dual axis accelerometer, a digital temperature sensor and a touch sensor. With them, a computing device can locally infer different contexts where this device is situated – such as “sitting in a pocket”, “lying on the desk”, “in user’s hand” – allowing the device to adapt its behaviour to each context. Even discussing the physical sensors used we have had to include the idea of a computational model that transforms the raw input data into some interpretation of the state of affairs in the world. This is because context-awareness is essentially an interpretation of the world situation and, as such, what is needed is plenty of knowledge about what the world is like, in other words, common sense knowledge is needed. There has been a lot of research in artificial intelligence (AI) in the last 10–12 years, concerning the issue of common sense, with the Cyc project (http://www.cyc.com) being the most well-known endeavour. The current understanding of common sense can be summarised as: 1. an ontology, defining the objects existing in the world that we want to talk about, and 2. an inference engine, capable of using a model of the state of affairs in the world expressed in that ontology to conclude new facts or statements about that state; new facts that are “obvious” or “implicitly” known by people, by what we are calling “common sense”. Moreover, context aware applications need to have some properties that differ from current applications: they need to be persistent, responsive and autonomous. We will call this collection of properties continuative computing1 because they set apart context-aware applications from the usual applications oriented to input–output. First, a context-aware application needs to be persistent, that is, persistently in runtime state, non-terminating. Commonly, an application accepts input and produces output (in fact, the same definition of algorithm is based on the idea of transforming an input to an output) – an exception is programs that are operating system services (that are difficult to model in the algorithm paradigm based on termination). A common application is a file that, when a user needs it, becomes runtime, receives an input and after some processing time yields a result and then goes offline. 1

Continuative: tending or serving to continue.

103

Inhabited Information Spaces

A context-aware application needs to be non-terminating, awake and running persistently, much like an operating system or a PDA. Moreover, it needs to be persistent in order to be responsive: able to adapt and produce adequate responses when something changes in a context or the context changes to become a new context. Finally, context-aware applications need to be autonomous in the sense of having an identity persistent in time and a memory (or internal state) that is individual. Since changing context is one of the most important pieces of information a context aware application can deal with, it makes no sense that each particular physical space has a context-aware application that is independent from other locations. Since people move around it is better to think of a context-aware application as centred on users, like a PDA that a user carries around. In this way the context-aware application can know the past history of contexts of the user, and even learn to anticipate the most likely future contexts of the user and prepare for it. There is a current paradigm and associated technology that fits these requirements, which we have outlined as continuative computing: intelligent agents. Agents and multi-agent systems being developed in artificial intelligence are conceived of as autonomous, permanent entities capable of using ontologies to perform inferences for solving problems, and for co-operating and/or competing with other agents or people. Intelligent agents are classified as reactive or deliberative, depending on some design properties, but we will show later in the chapter that designing agents with a particular architecture like the one proposed, means they can also be responsive and thus exploit awareness of physical and social context to improve their performance on behalf of users.

7.3 A Framework for Context-aware Agents Our framework is composed of a collection of context-aware personal information agents (CAPIAs) working in an information space and a collection of human users interacting in the same physical space. A useful way to visualise this distinction is the dual space schema depicted in Figure 7.1. Human users, on the right-hand side of Figure 7.1, are in a location, interacting with other persons (who may or may not be users) in the context of social activities. Information agents, on the left-hand side of Figure 7.1, inhabit an information space where they interact with other agents and gather information in the interest of the users. Moreover, we have mediation services connecting the information space of agents and the physical space of human users. Specifically, we are currently using two mediation services, namely an awareness service and a delivery service (see Figure 7.1).

104

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

The Computational Interplay of Physical Space and Information Space 7 ●●●●●●●● ● 1 ●●●●●●

CAPIA

CAPIA

CAPIA

CAPIA CAPIA

Delivery services

Awareness services

Information space

Physical space Social activity

Figure 7.1 A schema of the dual space of information agents and human users with the mediation services between them.

7.3.1 Awareness and Delivery Services The awareness service takes charge of pushing information from the physical space to the information space. Specifically, the awareness service provides real-time information about the physical location and movements of users to CAPIAs. The specific data provided depends on the particular sensors available in the awareness service for a particular application. For instance, in the conference centre application the awareness service provides a real-time tracking of attendees’ location as well as the group of other attendees nearby a given attendee – see Section 7.4 for the features of the awareness service in the COMRIS Conference Centre application. The delivery service offers mediation and brokerage capabilities (subscribed by the human users) for delivering information from the information space to the physical space. Specifically, the delivery service provides the channels for delivering the information gathered by the CAPIAs to their corresponding users. For instance, in the conference centre application the delivery service allows information such as audio output to be sent by means of a wearable computer and HTML pages by means of screen terminals scattered through the conference building.

011

11

7.3.2 Agents Requirements The society of agents has to be able to communicate using a common ontology for a specific application, and they have to share a collection of interaction protocols appropriate for that application. Our approach is to use the notion of agent-mediated institution (Noriega and Sierra, 1999)

105

Inhabited Information Spaces

to specify the ontology and interaction protocols to be used by a society of agents for a particular application. In addition, to support the ontology and interaction protocols of an agent-mediated institution the agents should be able to handle contextawareness information. That is to say, a context-aware agent should be able to react dynamically when new physical context information is received from the awareness service. Moreover, since the future physical and social context of the user is not known, a desired feature of CAPIAs is the capability of gathering information that may become relevant in a future context. For instance, in the conference centre application, when an attendee is at a specific exhibition zone, the CAPIAs use the knowledge provided by the conference about the physical distribution of booths for trying to anticipate the next movement of the attendee. In our framework, CAPIAs are based on the distinction between two kinds of information valuation, namely interestingness and relevance. Information interestingness measures the intersection of a given piece of information with the user model a CAPIA has for the tasks with which it is charged. That is, interestingness: Info × UM |→ eI where Info is a given piece of information; UM is the user model; and eI is the estimation of the interest that the user has in Info. For instance, in the conference application a preliminary criterion for determining the interestingness of a given paper presentation is performed comparing the user interests (described as a collection of topics with different weights) and the keywords associated with the presentation (also described as a collection of topics with different weights). Then, other criteria such as the knowledge about the speaker or the user’s agenda increase or decrease the initial assessment. Depending on the physical and social context of the user and on the time, however, some information may be more or less relevant for the user at each particular point of time. Information relevance measures this intersection of a given information with the time and the context of the user. That is, relevance: Info × Time × UC |→ eR where Info is a given information; UC is the user context; and eR is the estimation of the relevance of Info in UC. For instance, in the conference application when an attendee is near an exhibition booth, the information related to the booth is estimated as more relevant. Another example of increase of relevance is when a conference event is close to start: a CAPIA has a time constraint for deciding if that event is useful for the user interests. We can say that, basically, the CAPIAs go on their tasks, interacting with other agents (and other accessible information resources), to gather information that is interesting for their users. Concurrently, each CAPIA uses an awareness service (see below) to keep track of the whereabouts of its user and decides which information is relevant for the user in a particular physical and social context. Clearly, interestingness and relevance are not completely independent, and the information gathering is correlated with the information the agent expects to deliver to the user, but for exposition purposes it is handy to talk about them separately.

106

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

The Computational Interplay of Physical Space and Information Space 7 ●●●●●●●● ● 1 ●●●●●●

At this point it is useful to put this framework into a concrete application to illustrate the dual space and the personal information agents’ exploitation of context awareness.

7.4 The COMRIS Conference Centre This section applies the framework of context-aware information agents to a particular application, that is to say in the physical location and the social activity context of a conference centre in the COMRIS project.2 We view the conference centre (CC) as an agent-mediated institution where a society of information agents work for, and are aware of, the attendees of a conference (Plaza et al., 1998). The ontology of the CC institution defines the conference activities that take place in the CC. Examples of conference activities are exhibition booths and demo events, plenary and panel sessions, and so on. The ontology also defines the roles that a person takes in different locations while performing different activities, such as speaker, session chair, attendee, organisation staff, etc. Other important elements defined by the CC ontology are the different locations of the conference such as the exhibition areas, the conference rooms, and the public areas – i.e. halls, cafeterias and restaurants. This information is used by the agents for reasoning about the movements of users in the conference. The schedule of conference events is also defined in the CC ontology. Finally, the CC ontology supports the definition by each user of the “instruction set” that their CAPIA should follow. The instruction set is entered by the conference attendee using a WWW browser while registering, and basically includes (1) an interest profile (specifying the topics, with weights, in which the attendee is interested); (2) those tasks the user commissions the PIA to do on their behalf (e.g. if they are interested in making appointments); and (3) the delivery modes that the CAPIA will use to communicate with the user. We implemented two types of CAPIAs in the CC application: CAPIAs representing interests of attendees and CAPIA advertisers. There is a CAPIA for each attendee, a CAPIA advertiser for each exhibition booth, and a CAPIA advertiser for each paper session. The goal of CAPIA advertisers is to convince people to attend the conference event they are representing.

7.4.1 Delivery Service The delivery service in COMRIS allows the users to receive information in two ways: by means of a wearable computer with text and audio output 2

COMRIS stands for Co-habited Mixed-Reality Information Spaces. More information is available at: http://arti.vub.ac.be/~comris/

107

Inhabited Information Spaces

and by screen terminals scattered throughout the Conference Centre. The wearable computer is used to convey short messages that are relevant for the user with respect to their current physical and social surroundings. The user can walk to a terminal if they wish to have more information about this message or other recent messages they have received. When the user approaches a screen the wearable computer detects this terminal’s identifier, and then sends this identifier to the user’s CAPIA. Once the CAPIA is aware of this situation, the agent sends to that screen the report of the performed tasks and the report of ongoing tasks. The delivery service comprises several components. The first component is the natural language generation (NLG) component. The NLG component receives the message sent by a CAPIA and generates an English sentence explaining the message content and taking into account the current attendee context and the sentences previously generated. Then, when the message has to be delivered as audio, the sentence structure is sent to a speech synthesis component that produces the actual audio heard by the user. Similarly, there are components that transform CAPIA’s messages into HTML or VRML in order to be delivered to the screen terminals.

7.4.2 Awareness Service The awareness service keeps track of the whereabouts of the attendees in the Conference Centre. In the COMRIS CC the detection devices are a network of infrared beacons (marking the different rooms, places and locations in the CC) and the wearable computers carried by the attendees. The COMRIS wearable computer (shown in Figure 7.2 and commonly called parrot) detects the infrared beacons and thus informs the

Figure 7.2 The wearable computer also known as “the parrot”. The CPU is on the front unit while the back one hosts sensors and batteries.

108

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

The Computational Interplay of Physical Space and Information Space 7 ●●●●●●●● ● 1 ●●●●●●

awareness service of the location of its user. Moreover, the wearable device possesses an infrared beacon, allowing the detection of other persons, wearing a parrot, located nearby. In order to have access to this information, each CAPIA in the information space “subscribes” its user to the awareness service. As a result, the CAPIA receives messages about the changes in location of that person and a list of other people close to that person. When the CAPIA interacts with other CAPIAs (representing other conference attendees), and decides that those CAPIAs are interesting persons, it subscribes those persons to the awareness service. Consequently, the CAPIA is aware of the location of the most interesting persons for its user and detects, for instance, when one of these persons is in the same location as the user – a most relevant situation to push to its user the information concerning that person who is interesting and nearby.

7.4.3 Tasks The tasks that the COMRIS Conference Centre supports are at the core of the activity in the CAPIAs. It is important to remark here that, in order to perform these tasks, the information agents use both the CC ontology and the awareness service to infer the situation of the user. That is to say, knowing that the user is in a particular place, the current time and the activity scheduled by the Conference for that place at that time, the information agent can infer the social activity in which the user is involved. The tasks performed by COMRIS CAPIAs and the scenes in which they are involved are summarised below. ●

011

011

11

●

Information gathering: this task is responsible for establishing initial conversations with other CAPIAs for estimating the interestingness of the attendees or conference events they represent. We say that the information gathering task constructs the interest landscape of a given attendee. The interest landscape holds all the information considered to be useful for the interest of the attendee and is used and refined in the other tasks. When the information-gathering task assesses a conference event with a high interestingness valuation, the information is directly delivered to the attendee. This delivery strategy was adopted for biasing the future decisions of the attendee. In CAPIA advertisers, this task has been specialised for attracting persons who might be interested into the conference events (exhibition booths or conference sessions) they represent. Appointment proposal: in this task, using the interest landscape, the CAPIAs try to arrange an appointment between two attendees. First, CAPIAs negotiate a set of common topics for discussion (the meeting content). When they reach an agreement, CAPIAs negotiate on the appropriate meeting schedule.

109

Inhabited Information Spaces

●

●

Proximity alert: in this task an attendee is informed that she is physically near to another person with similar interests – or near an exhibition booth or a thematic session with similar topics. Commitment reminder: this task is responsible for checking if attendees are aware of their commitments. The CAPIA uses context to determine that the user may be unaware of a commitment, e.g. if she is not near the location of an appointment (or a bus scheduled to leave) a few minutes before. Commitments of attendees are only noticed when the context information available to CAPIAs indicates that the attendee is not aware of the commitment (e.g. it is five minutes before the starting of a session chaired by the attendee and the attendee is physically in another place).

For each task several activities are launched in the CAPIA. For instance, when an agent in COMRIS is discussing appointments with several CAPIAs, each thread of interaction is managed by a distinct activity. The activities can start either by an internal decision of a CAPIA or because a CAPIA has received a request from another CAPIA.

7.5 Conclusions We have shown a specific context-aware application (the COMRIS Conference Centre) developed in the framework of the COMRIS project that illustrates the interplay of physical space and information space. We have seen that the physical infrastructure, consisting of an individual wearable computer and localisation beacons, was used as a “awareness service” by a society of agents inhabiting the information space. We have focused on the kind of software required to develop a context-aware application, showing that using an agent-based approach we can fulfil the properties we required in what we called continuative computing. We also discussed the kind of agent architecture that can exploit context awareness. The approach we presented was based on the idea of having a personal agent per person. This allows the continuative dimension of processing to be user-centred. As the user changed context, the personal agent received the corresponding perception data from the awareness service and followed the user to a new physical context. In addition to this, services that need to be provided to the users (services provided by the Conference organisation in our example) are agentified, that is, they are also provided by agents (and they also are aware of the contexts in which they are interested). Moreover, this approach is scalable: the agent could use another awareness service at a different conference centre. Moving from one application context to another (from one conference to another) requires a standardisation effort of the awareness services, but this effort is reasonable since it can improve the performance and lower the costs.

110

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

The Computational Interplay of Physical Space and Information Space 7 ●●●●●●●● ● 1 ●●●●●●

We see thus that context awareness can be integrated into an agentbased paradigm in a well-understood way. A better infrastructure on context perception, as we can expect to be developed in the next decade, can be integrated in the agent-based paradigm without major problems. The main reason for this is the AI approach to agents that employ ontologies describing the world. Clearly, with a better perception infrastructure the agents could perform better inferences about the state of the world. For instance, in the COMRIS conference centre the agents knew when two people were in the same room (using beacons) and when two people were in front of each other (using the wearable computer beacons), but since there was no microphone on the wearable computer there was no way to know if the user was already busy talking with someone or not. In addition to improved awareness services and perception infrastructures, a second issue that was considered in the COMRIS project but never tried was that of learning. Agent learning is developing into an active area of research, initially focused on reinforcement learning but it is rapidly broadening. Agents in context-aware applications should be able to adapt to new contexts but also to learn from the user satisfaction (or not) of the agent’s action. However, learning from examples requires a sufficient number of examples to be worthwhile, and the experiments conducted in the framework of the COMRIS project assumed that the Conference only lasted one day – and the amount of data was too sparse to allow significant learning. It turned out that learning would be interesting if personal agents were carried over by the user to different conferences, allowing the collection of examples significant enough in number and variability. Only when awareness infrastructures are more proficient and more readily available (and awareness services are more standardised, allowing agents to continue from one application context to the next) the AI and agent technologies will be able to respond ubiquitously and intelligently to the requests and the needs of people.

Acknowledgements The research reported in this paper is partly supported by the ESPRIT LTR 25500-COMRIS (Co-habited Mixed-Reality Information Spaces) project.

011

11

111

This page intentionally left blank

011

Part 4 Communication

011

011

011

11

113

This page intentionally left blank

011

011

011

011

11

Communicating in an IIS: Virtual Conferencing

8

Adrian Bullock

8.1 Introduction This chapter considers the role that virtual conferencing has to play in realising a successful Inhabited Information Space (IIS). For any IIS to be successful it needs to weave together many different constituent elements and present these in a coherent and seamless manner. For example, Maher et al. (2000) describe how many different components are used together to create a virtual design studio for architectural collaboration. For the IIS to function, all the elements must work both individually and collectively. Communication is one of the basic building blocks for an IIS, and can be in many modes across many media. Approaches to virtual conferencing offer support for communication across a number of media and can be utilised in an IIS. These approaches are also starting to offer support for collaboration. By providing an introduction to, and overview of, various possibilities for virtual conferencing this chapter aims to show how these solutions can provide the required and appropriate support for communication and collaboration between inhabitants in a shared information space. Of course virtual conferencing solutions exist at many levels of sophistication and fidelity. Communication media can range from text through 3D graphics to video representations. The aim of this chapter is to present these many and varied possibilities, drawing on the experience of the author as well as insights into the past, present and future. In this way it is possible to see how diverse a range of IISs can make use of virtual conferencing functionality. We talk about virtual conferencing as an activity that takes place within an IIS, the aim being to examine a range of communication and collaboration possibilities available for use in an IIS. Many of the approaches could well be termed IISs in their own right, whereas others provide only part of the functionality required by an IIS. Our aim is not to simply overview virtual conference approaches, but to demonstrate

115

Inhabited Information Spaces

how these approaches can be incorporated, borrowed or used to complement other applications and so aid in the construction of an IIS. We begin by considering just what we mean by the phrase “virtual conferencing”, and then we look at a number of approaches, demonstrating the range of areas that virtual conferencing covers. Next, we examine how virtual conferencing can be used, and identify some issues that have a bearing on this use. We also consider the roles of video and graphics in realised solutions for virtual conferencing. These approaches are then compared to telephony, arguably the most pervasive virtual conferencing solution in use today in global terms, and we suggest areas that need to be addressed if conferencing is to approach these success levels in the future. We end by presenting general guidelines for making use of virtual conferencing approaches effectively.

8.2 Virtual Conferencing – a Historical Perspective: Past, Present and Future We have come a long way since the first telephone call in 1876 in Boston (Farley, 2001), the first videoconference meeting in 1930 in New York (Rosen, 1996) and the first TV broadcasts in March 1935 by the German Post Office. The subsequent developments in these technologies have concentrated on reproducing lifelike representations of shared audio or/and video between participants. The introduction of computers in the 1960s saw the development of more abstract forms of communication: email, text chat and MUDs / MOOs (see Section 8.3.2) are early examples. As computer graphics developed so did the possibility of creating simple shared environments, something picked up by authors at that time. W. Gibson (1986) coined the term ‘cyberspace’ in his writing in the 1980s to describe the virtual world in which we will be as comfortable occupying in the future as we are the physical world now. Stephenson’s (1992) Snowcrash further describes how physical and virtual worlds may become intertwined, undertaking actions in both worlds at the same time. In Star Trek we see a “Holodeck”, where the ship’s crew walk into an empty room, speak to the computer and suddenly it is as if they are in a completely different physical location, able to experience all that it has to offer – though we are still some way from achieving this today (Raja, 1998, Leigh et al., 1999). While early shared environments such as AlphaWorld (http://www.activeworlds.com/) encouraged thousands of users to use them and indeed have been used to host international conferences (Jones, 2000), it is perhaps with online gaming that we are starting to see the potential of IISs with hundreds of thousands of users interacting together in shared gaming environments, such as EverQuest (2003). By examining past and current approaches to virtual conferencing, providing examples and experiences of use, we aim to present virtual

116

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Communicating in an IIS: Virtual Conferencing 8 ●●●●●●● ● 1 ●●●●●●●

conferencing as an abstract area, fulfilling the requirements of many different environments and scenarios, including those listed above. It is an activity that takes place in the IIS to support the work that is the primary purpose of the system.

8.2.1 What Do We Mean by Virtual Conferencing? We define a virtual conference as a meeting between two or more participants, where the participants are physically located at different places. Typically participants communicate and collaborate through some form of visual and audio communication channel, along with the sharing of task-related information. Videoconferencing systems such as the H.323based Polycom ViaVideo (Polycom, 2003) and the IP–based Marratech (2003), telepresence systems such as MASSIVE (Greenhalgh and Benford, 1995) and DIVE (see Chapter 12) and text-based systems such as MUDs and MOOs (Burka, 1995) are common forms for supporting this communication. An important aspect of virtual conferencing is that there is a shared space that supports appropriate communication across a number of media in as natural a manner as possible; an IIS is such a space.

8.3 Approaches to Virtual Conferencing We now examine how virtual conferencing has developed, considering its origins in video in the late 1960s and multi-user dungeons in the late 1970s, the effects of developments in graphical, computational and network capabilities, and application areas that benefit from and drive forward the development of virtual conferencing systems.

8.3.1 Early Videoconferencing Early videoconference developments dating from the 1960s placed the emphasis on communication, enabling two people in different geographical locations to talk to and see each other. The Picturephone system (Massey, 2003; Schindler, 1969) developed by AT&T was an early example of an attempt to enhance the experience of talking on the telephone. A telephone was augmented with an accompanying video display unit with a built-in camera, enabling the user to see the person to whom they were talking. Functionally the system was sound, but people felt that the unit was too bulky, the controls too unfriendly, the cost of use too expensive and the picture too small. Another difficulty was the high bandwidth requirements for a video call, 333 times higher than a standard phone call. Clearly this would not scale very well with the telephone

117

Inhabited Information Spaces

infrastructure of the early 1970s and the Picturephone disappeared very quickly. However, with advances in compression techniques, improved networks (both Internet and Telecommunications) and increasing availability of low-cost bandwidth we are seeing conditions where videophones are becoming viable. Indeed, we increasingly see their use by the media when reporting from countries with little infrastructure (e.g. satellite video phones from Afghanistan in 2002).

8.3.2 MUDs and MOOs The roots of more abstract forms of virtual conferencing can be traced back to MUDs (Multi User Dungeons) and MOOs (Object Oriented MUDs) in the 1980s (Bartle, 1990). At this time computational resources were limited and expensive (mainframes), so people had to rely on simple text message passing and, perhaps more interestingly, the use of text to construct some of the first IISs (even if they were not thought of as such then). Despite the impoverished appearance of such environments (only a screen of text after all), they were extremely effective at engendering a sense of community and presence – a very effective IIS indeed! Emoticons were used to enrich communication (Rivera et al., 1996), perhaps the most famous being the smiley :-) (Fahlman, 2003). These textual environments showed that effective collaboration and community building was possible. Embodiment, information representation and technological issues were addressed in the minds and imaginations of the end users, and textual descriptions aided in this process (a textual description of how and where someone appears). It is not surprising that this works well, given the way in which it is possible to immerse oneself while, for example, reading a book. These environments had a range of uses, some being for games and role-playing, some a place to simply hang out and meet friends (Evard et al., 2001), while others were support environments for work (Evard, 1993, Churchill and Bly, 1999).

8.3.3 The Arrival of Graphics In the early 1990s graphical add-ons to MUDs and MOOs started to appear, as graphics technology became something that was available on the desktop and was no longer the preserve of expensive, high-powered servers. However, these early approaches to bolt on graphics (e.g. BSXMUD, 1994) did not really work (see Figure 8.1). The interesting and useful interaction continued to take place in the simple text exchanges, while the graphics were simply a series of updated still images and scenes that bore little resemblance to the ongoing dynamic activity. More successful approaches to graphical MUDs and MOOs were ActiveWorld’s

118

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

Communicating in an IIS: Virtual Conferencing 8 ●●●●●●● ● 1 ●●●●●●●

AlphaWorld (http://www.activeworlds.com/) and Blaxxun’s Contact (http://www.blaxxun.com/), realisations of the kinds of online communities described in Neal Stephenson’s (1992) Snowcrash (indeed Blaxxun takes its name from the virtual club from the book). Here there was a more direct correlation between the user and their embodiment (Cuddihy and Walters, 2000), albeit in a very constrained and rather unnatural way. Typed text was still the main communication channel, but now gestures could be performed by the embodiment as well as emoticons such as :-). These environments appealed to teenagers in particular, offering a customisable place to hang out with friends and meet new people. Creativity could also be expressed, with parallels between the early settlers in the USA and the early settlers in AlphaWorld, arguably the world’s largest virtual reality world, being easily made through the creation of homes and communities. Activeworlds Maps (2003) charts the development of this world from December 1996 to August 2001 in pictorial form. Currently the world is approximately the same size as California, i.e. 429,025 square kilometres. Today these companies have grown from their MUD roots to offer a 3D home on the Internet with sophisticated possibilities for interaction .

8.3.4 Video Comes of Age Away from MUDs and MOOs, work in the late 1980s was also being undertaken on shared media spaces, where video technology was used to create a real shared space inhabited by real representations of people. Xerox EuroPARC pioneered this work with their RAVE media space (Gaver et al., 1992), in tandem with research into media spaces at PARC (Stults, 1986, Bly et al., 1993). ISDN-based videoconferencing was the

011

11

Figure 8.1 A screenshot from BSXMUD. Reproduced with permission from Henrik Rindlow.

119

Inhabited Information Spaces

standard for business and commercial use, and a new suite of software tools that ran over the Internet were developed in the academic community at this time: the MBone tools (Macedonia and Brutzman, 1994). However, workstations and peripherals to support video were expensive and not in widespread use, even by the mid-1990s. All this was to change with the arrival of inexpensive USB devices and dedicated video hardware in the latter part of the 1990s. Video became accessible on the desktop to all users, software became more widely available (e.g. Microsoft bundled NetMeeting with their operating system), plug and play all-in-one hardware devices such as the Polycom ViaVideo removed the complexity from local peripheral configuration, and network infrastructures made it possible to support the bandwidths required for video communication. We give two examples of different types of videoconferencing solutions that are most commonly in use today. Figure 8.2 shows a roombased solution, where dedicated hardware is installed in a conference room, and a large screen displays the other members of the videoconference. This is purely videoconferencing, and not really an IIS in the way it is being used. However, if the other screens were used to display shared workspaces, e.g. using Virtual Network Computing (VNC) (Richardson et al., 1998) to share a computer desktop between all the sites then this simple communication space is quickly transformed into an IIS.

Figure 8.2 A typical room-based videoconference session

120

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

Communicating in an IIS: Virtual Conferencing 8 ●●●●●●● ● 1 ●●●●●●●

Our second example shows how a desktop-based videoconference solution can be thought of as an IIS. Figure 8.3 shows what a typical desktop looks like when holding a meeting using the desktop videoconference system Marratech Pro (http://www.marratech.com). Four windows make up the session and we describe them moving from left to right. The viewer enables the user to load HTML web pages. By default these pages are local to the viewer, though users can choose to transmit their content to others in the same session. In this case the project web site, being discussed in the meeting, is displayed. The next window is a shared whiteboard that supports the import of images, Microsoft Word and PowerPoint documents as well as standard whiteboard functionality. This holds the meeting agenda and is updated with annotations as the meeting progresses (who is responsible for developing which sections of the web site in our example). The video window shows who is currently talking, and the video image follows the audio source, changing dynamically. Finally, the participant window shows thumbnail video images of everyone who is in the meeting. Participants join the meeting by contacting a portal and joining a session. Sessions can be encrypted and/or password protected for privacy purposes. It is also possible to communicate privately with other participants in the meeting while still being present in the main meeting room.

011

011

11

Figure 8.3 A meeting in a desktop videoconference system.

121

Inhabited Information Spaces

8.3.5 Graphics Come of Age From the mid-1990s onwards, technological advances were such that 3D graphical environments that supported audio communication between participants were possible on desktop workstations, opening the door to potentially rich interactions between standard users. Early examples of these types of systems included DIVE (see Chapter 12), a fully programmable distributed interactive virtual environment, and MASSIVE-1 (Greenhalgh and Benford, 1995), a 3D graphical teleconferencing system. Another important driving force in the development of CVEs at this point was the gaming market. Doom (1993, Id Software) was a revelation, demonstrating that engaging 3D environments into which users could immerse themselves for literally hours on end were possible, and on very lowly specified PCs. OK, so these were hand-crafted, specific and optimised solutions, but if such levels of realism and involvement were possible in a game then surely it would only be a matter of time before such environments were possible for business and leisure use? The main drawback in the development of realistic environments in the early 1990s was the available computational power. Despite pushing

Figure 8.4 A virtual conference in DIVE (c. 1995).

122

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

Communicating in an IIS: Virtual Conferencing 8 ●●●●●●● ● 1 ●●●●●●●

the technological capabilities to the limits, the early environments supported by systems such as MASSIVE and DIVE appeared primitive and impoverished (see Figures 8.4 and 8.5). Textures helped increase realism, but at the time this had a major effect on performance. Unlike the predefined scenes from games, these new systems were programmable on the fly and very much dynamic in nature. The collaborative nature also had a major impact on networking, especially with the bandwidth requirements of audio communication between participants. Initial experiences showed that the configuration of such distributed environments was most definitely non-trivial, even for those who simply wanted to run a client. However, once sufficient experience and training has been acquired it has been observed that people could collaborate well in such environments, could build personal and social relationships with others, and adapt their behaviour quite flexibly (Greenhalgh et al., 1997, Bullock, 1997). It was possible to examine collaboration and usability issues, and not just be content with the technical “success” of merely giving everyone the possibility to communicate, though this was a result in its own right.

011

011

011

11

Figure 8.5 Participants examine a representation of network connectivity in MASSIVE-1. Reproduced with permission from The University of Nottingham.

123

Inhabited Information Spaces

8.4 Using Virtual Conferencing In the previous section we examined the development of systems that have overcome the technical challenge of simply enabling people to meet together in a shared environment. This stable base allows the investigation into a number of interesting research questions and challenges for supporting virtual conferencing and meeting in an IIS. Some of these are quite technical in nature – such as how can scalable systems that support many thousands of simultaneous users be constructed and how complex can the environments and users be? – while others are concerned with social factors – such as how do we convey subtle, involuntary communications inside an IIS, and how can we assess the successfulness of the IIS as a whole? Perhaps an easy way to summarise is to say that we need to understand the process of collaboration and identify the key aspects and challenges to be addressed.

8.4.1 Understanding Collaboration When we meet together with other people we are naturally immersed in a shared environment; it is the environment in which we spend our entire life and we spend many years learning how to understand it and interact in it. Through the use of computing and telecommunication technology for supporting virtual meetings we introduce an explicit obstacle between each of the users, in that each person must first interact with the artificial shared environment and then interact with the occupants and contents of that environment (Churchill and Snowdon, 1998). There have been many studies of how people collaborate together in both virtual (Fraser et al., 2000) and real (Heath et al., 2001) environments. One of the major challenges of teleconferencing, and one that is still far from being solved, is the ability to interpret the many and complex interactions that take place in real life when two or more people meet together. Information is exchanged in what is said, how it is said, gestures, facial expressions, body orientations as well as people’s natural ability to somehow sense how the other person is feeling (possibly from the sources listed above but maybe from other factors as well). This information is very difficult to capture and process, though there are attempts to build dedicated installations to do this such as in the EU IST-funded Virtue project (Schreer and Kauff, 2002) and the Office of the Future project (Wei-Chao Chen et al., 2000). There are also continuing developments in the area of image recognition and tracking, so that systems can be developed that automatically capture not just the voice and appearance of a user, but also their gestures and other subtle interactional components and automatically translate these into input for the shared meeting system (e.g. Fuchs, 1998 and Intelligent Spaces in MIT’s

124

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Communicating in an IIS: Virtual Conferencing 8 ●●●●●●● ● 1 ●●●●●●●

Project Oxygen (2003)). Another possibility to capture user data would be to expect someone to explicitly control a virtual embodiment to represent their range of interactional possibilities. However, the complexity and cognitive load in using such an interface would most likely make it impossible to use – users would spend all their time trying to control their avatar to the extent that they do not concentrate on the actual interaction itself, the whole reason for being in the environment in the first place!

011

8.4.2 The Importance of First Impressions While meeting someone for the first time should be a real-life experience, repeat meetings can benefit from virtual conferencing approaches. There is already a shared experience on which to build (the first meeting), and people can use that to bridge the gap between the real and the virtual (be it graphical or video) world. These techniques are equally valid for graphical environments and videoconference systems.

011

011

011

11

8.4.3 Sharing Context Another important aspect of collaboration is the ability to share the context or setting in which each user is situated. The communication itself is important, but so is the work that is going on (Luff et al., 2000). We need to share work as well as communication, and tools that let us do this are being continually developed, e.g. shared CAD design software, whiteboards and application sharing systems such as NetMeeting and VNC. We also need to be able to share experiences between local and remote participants, an example being the TELEP system from Microsoft (Jancke et al., 2000) that allows remote viewers to attend seminars, having a presence in the room and an ability to interact.

8.4.4 Scalability When it comes to scale there are big problems to be overcome. What does it mean to be in a virtual audience of a thousand users listening to a presenter give a speech? Hard choices have to be made as to how much awareness of the rest of the audience we have while we naturally concentrate on the speaker. There should be the potential to interact with any individual in the room (e.g. in a plenary session at the ACM CHI conference anyone is able to ask a question, and that person is heard by everyone in the room, but only at the instant the question is asked. At the same time we are aware of what the people immediately around us in the audience are saying.). Techniques to scope and limit

125

Inhabited Information Spaces

the interactions are necessary (Chapter 13, Benford et al., 1997a), and these techniques must be flexible enough for easy switching between the different scenarios, and ideally done without the awareness of the user. Here the technical issues have to be dealt with while considering the social implications of any given choice.

8.4.5 Real Versus Abstract: The Role of Video? Approaches to teleconferencing using 3D graphical systems (Greenhalgh et al., 1997, Lloyd et al., 2001) offer a “real” shared environment, graphically rendered with polygons and textures, where users are embodied in the same shared space and it is possible to observe the interactions between the users. This offers potential for developing and supporting the more involuntary aspects of communication that we touched on previously. As the embodiments can be quite crude, however, and even when they are very realistic as in computer games or state of the art research (VR Lab, 2003; Mira Lab, 2003) (see Figure 8.6), it is still not natural to immediately associate the embodiments with actual people. There are also very real performance issues when using more realistic embodiments, as significant computational resources are needed as the levels of realism increase, especially if the embodiments are to react dynamically in real time. And how should such embodiments be animated and controlled? One danger with a lifelike embodiment is that there is an expectation of lifelike behaviour from the avatar. If an avatar has legs is it not unreasonable to expect them to move when the avatar moves? Often this is not the case. With video the association between people and embodiment is automatic and we immediately get a sense of the person we are talking to. However a simple video image is quite poor in the depth and quality it can offer and much depends on the local user configuration (i.e. where the camera is placed, lighting, the behaviour of the person themselves).

Figure 8.6 Realistic human embodiments. Reproduced with permission from Daniel Thalmann.

126

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Communicating in an IIS: Virtual Conferencing 8 ●●●●●●● ● 1 ●●●●●●●

So while we recognise and relate to the person on one level, much of the subtle interactional methods are lost with a purely video approach. Therefore the immediate way forward would seem to be a hybrid approach where video-based avatars are placed in a shared 3D space and can interact with each other (examples being Benford et al., 1996; Insley et al., 1997), or mixing the real and virtual worlds together (Koleva et al., 2001).

8.5 Virtual Conferencing Versus Telephony In spite of the advances in support of virtual conferencing at a number of levels across a number of media, perhaps the most used form of virtual meeting today remains the humble telephone and conference calls, where there is no user embodiment nor support for shared information. In this section we examine some of the reasons why the telephone enjoys widespread success while the seemingly superior virtual conferencing services are lagging some way behind. At first it appears strange that more use is not made of virtual conferencing, given the potential savings in time and money if people were not to travel so often (Townsend et al., 2002). However, a Scandinavian survey (SAS, 2002) undertaken by Gallup for SAS, the region’s major airline, has found that for business, personal meetings are preferred to virtual ones (maybe not too surprising given who the survey was commissioned by). It is interesting that the same survey also found that women were more positive towards virtual meetings than their male counterparts. With the introduction of broadband (0.5Mb and above) to homes across Europe and the United States, one might think that the days of regular telephony were numbered (if everyone has a computer and it is always connected then why use the telephone?). This is definitely not the case. Let us consider why the telephone is so successful. People are familiar and comfortable with the telephone, they know the limitations and know what to expect when using it. They simply lift the phone, dial a number and talk to the person at the other end. Technical glitches are rare and not the fault of the individual. Also, the interface has changed little over the past 50 years – indeed if a 50-year-old phone is connected to today’s system it is more likely to work than not. Contrast this to computers, relatively new and complex devices that serve many and varied purposes, that are constantly being improved and developed, offering little backwards compatibility in terms of both software and hardware (things quickly become obsolete). With virtual conferencing we are often not dealing with a single device (the telephone), but a collection of devices: microphone, camera, screen, computer and the connections between all these devices. The software that holds everything together is often not as stable as the telephone systems and much configuration and fine-tuning is necessary to have

127

Inhabited Information Spaces

everything perform optimally. Also, much of this configuration is a local issue that the users have to deal with themselves. Unfortunately many people have poor experiences of trying to use virtual conferencing applications, and these experiences make them shy away from using such approaches in future, be it consciously or unconsciously. However, many of the problems of the past concerning interoperability and configuration are being solved. USB devices, such as cameras, are standard and easily used across platforms, and software wizards make the configuration of applications a relatively easy task. Also, instant messaging systems such as ICQ and Microsoft Messenger are educating people to the potential advantages for virtual conferencing, even if it does not explicitly seem that way. These tools offer facilities much like the standard telephone where people can select an icon and then start a conversation with another user. While this is not much different to a standard telephone call, it is providing the user with experience of using his or her PC for communication, and perhaps more importantly some confidence and benefit from the process. The next step of experimenting with conferencing applications should hopefully be easier with such a user, as they are aware of potential benefits and have some confidence in the technology in use (i.e. they have used it successfully and know the benefits to be gained at first hand). Long-term usage of room-based videoconferencing (Bullock and Gustafson, 2001) identified a number of potential problems that need to be addressed if people are to concentrate on their meeting and on each other, and not on the infrastructure supporting the meeting. A consistent experience is necessary, as with the telephone. Participants rated their experiences very highly the first time they experienced the system, but for subsequent meetings they rated their experience less highly, even though there were no perceivable differences between configurations. We suggested that people were excited and taken in by the set-up the first time they used it, and subconsciously compensated for small glitches in audio and video. Once familiar with the set-up, however, they were more susceptible to minor imperfections and noticed these, and these made people feel more tired. So, while systems might appear to be good in the short term, they need to offer consistency and reliability like the telephony system if regular long-term use is to occur. Perhaps the most significant problem in maintaining a sense of presence between the participants concerned audio, and feedback in particular. Audio headsets help eliminate this, but in a room setting these are not practical. While microphone/speaker combinations do exist that help eliminate feedback, these can be expensive, or else the quality they offer is relatively poor. In order to take advantage of the increased quality that computational approaches offer (compared to the telephone), then audio solutions offering feedback-free communication need to be developed.

128

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

Communicating in an IIS: Virtual Conferencing 8 ●●●●●●● ● 1 ●●●●●●●

8.6 Guidelines for Using Virtual Conferencing Effectively When deciding on what type of virtual conferencing solution to use, there are a number of factors that need to be taken into consideration. Perhaps most important is the task that is being undertaken. It is also important to think about what the most relevant media are for communication, and what types of communication need to be supported. Other factors to be aware of are the supporting infrastructures that will be used (e.g. networking, display possibilities) and issues of heterogeneity. We bring this chapter to a close by drawing up a set of guidelines to help in selecting appropriate approaches to adopt for communication in an IIS.

8.6.1 What Is the Task at Hand? The most important factor to consider is what activity is being undertaken in the IIS. A scheme for classifying different types of activity involves the activity itself and two parameters describing the activity. The first parameter describes whether the activity naturally occurs in real, physical space or in a virtual, constructed space, and can take the value real or virtual. The second parameter concerns information that will be shared between participants in the activity, and again takes the values real and virtual depending on whether the information is real (e.g. word documents) or virtual (e.g. a CAD model of a physical object). We give examples for each of the four resulting application areas and show how virtual conferencing can best be used to support these areas.

Activity(Real, Real) An example of this kind of activity is a project meeting using desktop conferencing that supports a shared workspace. Returning once more to the system shown in Figure 8.3, video and audio provide the necessary communication between participants, and shared editing and browsing tools allow web pages and office documents to be shared between participants. There is no abstraction in the virtual conference, with real life meeting characteristics and properties mapped directly onto the virtual conference.

011

11

Activity(Real, Virtual) An example of this kind of activity would be a system similar to the one described above, but where support is provided for sharing more abstract information between participants, rather than simply documents. The DING project (Törlind et al., 1999) is a good example of such

129

Inhabited Information Spaces

an environment, where a videoconference system is augmented with a shared 3D virtual world where participants are embodied and can act on shared 3D CAD models.

Activity(Virtual, Real) This kind of activity would take place in a shared virtual environment, but the information shared in the environment would be real. The Web Planetarium (Chapter 2) is a good example of an IIS that supports this kind of activity. The shared 3D environment is provided by DIVE but the information inside is real web pages.

Activity(Virtual, Virtual) Finally, truly virtual activities inside an IIS would involve things like shared browsing of information visualisations or similar activities.

8.6.2 Communication Media Above we gave examples of different types of real and virtual activities. What we didn’t explicitly mention was the possibilities for communication inside these IISs. Audio is arguably the most useful form of realtime communication in an IIS, and is equally applicable to any of the scenarios listed above. The one drawback with audio communication is the way it suffers from interference through dropped audio packets or poor availability of resources. Simple text messages are a very useful and very effective form of communication, supporting both synchronous and asynchronous communication, and providing an explicit history mechanism in the process. Video imagery is useful, and the more realistic the image the better the sense of presence will be, but video can normally be given a lower priority than the other media unless there is an explicit need for it in the activity.

8.6.3 Infrastructural Support In order to support high quality audio or/and video conferencing, high network bandwidths are required. If such networks are not available then simpler forms of communication need to be used, such as typing text messages or using lower quality encodings for the media streams. However, given the increasing availability of network infrastructures and supported bandwidths, it is better in the long run to make design choices based on infrastructures existing in the future (e.g. Smile! – Johansson,

130

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

Communicating in an IIS: Virtual Conferencing 8 ●●●●●●● ● 1 ●●●●●●●

1998)) rather than work around problems that are not really problems and not have chance to examine what are the interesting issues.

8.7 Final Remarks There is no doubt that virtual conferencing has a big part to play in future business and leisure scenarios and applications. As interfaces to the computer improve, and the computer as an artefact disappears to be replaced by intelligent devices with built-in computational and communication facilities, we will no longer think of running virtual conferencing applications or configuring infrastructures for communication – we will simply take part in virtual meetings as if they were happening in reality. Of course it will take some time before an experience of actually “being there” with the other person is possible, but the use of conferencing techniques in much the same way that the telephone is used today would represent a significant step forward.

011

011

011

11

131

This page intentionally left blank

011

011

011

011

11

9

Getting the Picture: Enhancing Avatar Representations in Collaborative Virtual Environments Mike Fraser, Jon Hindmarsh, Steve Benford and Christian Heath

9.1 Introduction Inhabited Information Spaces (IISs) can be either realistic or abstract. Realistic IIS systems presenting 3D graphics are usually termed “Collaborative Virtual Environments” (CVEs). However, difficulties encountered by designers of abstract spaces such as Populated Information Terrains (PITs) still arise in “realistic” CVEs. Notably, both abstract and quasi-realistic information spaces suffer the problem of supporting “natural” interaction in an unnatural world. Users of abstract spaces encounter difficulties because interactions in an unfamiliar world must be learnt. Users of CVEs encounter difficulties because interactions in a familiar world are assumed where many remain unsupported. In this chapter, we focus on the realism of inhabitant representations. The use of “avatars” is a fundamental part of CVE design. Users are represented in the 3D environment so that they can see their own representation and also see one another (Fraser et al., 1999). CVE systems usually take the approach of providing “realistic” avatars (e.g. GuyeVuillème et al., 1999; Salem and Earle, 2000). Designers mirror human appearance and behaviour as closely as possible, even though there is some debate about how close designers are to creating a virtual experience that is “truly realistic” (e.g. Brooks, 1999). Visual representation and animation of human-like 3D figures has made great advances over the past few years. Avatars can look and move like realistic humanoid figures more than ever (e.g. Badler et al., 1999; Faloutsos et al., 2001). This approach has pervaded the representation of people in CVEs, and has commonly meant that CVE avatars are, at the least, pseudo-humanoid. Actions that can be performed at the interface are represented through

133

Inhabited Information Spaces

the use of a human-like metaphor or model. However, problems with the use of realistic representations remain. Rendering realistic 3D graphics must take into account constraints of the system such as graphics processing speed, network bandwidth and so on. Another often-overlooked factor, however, is that the user’s control and perception is not completely realistic. For example, a human-like figure can suggest realistic perceptual and movement capabilities to other users. These capabilities are simply not supported by current, or even prototype, display and tracking technologies. Nor is it likely that many problems, particularly the speed-of-light limitation on data transmission, will be removed over time (Brooks, 1999; Fraser et al., 2000). Recent studies (Hindmarsh et al., 1998; Fraser et al., 1999; Valin et al., 2000) have shown how traditional approaches to representation in CVEs can cause problems for users. Communications technologies have frequently ignored the importance of the contents of the world to users. Designers have tended to focus on supporting face-to-face communication rather than providing the ability to refer to the environment that users share. This does not take into account all manner of interactions that we regularly rely on in co-present communication. In the case of CVEs, designers have tended to work around these kinds of communication by focusing on support for simple meetings or coarse informal interactions, rather than attempting to allow users to collaborate around objects or features of interest within the world. The limitations of fieldof-view, speed of movement, and ease of gaze changes, mean that other users’ avatars are often off-screen, a key problem for coherently working together around shared objects. This makes it difficult to use pointing gestures and references, not only because movement can be slow, but also because it is hard to show (parts of) an object to someone whose position and/or view are unclear. The focus on representational realism of 3D shared spaces contrasts directly with that of 2D shared spaces, for which usability of the interface has been the key design focus. Research in 2D groupware has produced a number of concepts to describe how users’ views are represented to each other, which do not rely on the realism of representation. In particular, approaches to handling multiple views often involve “relaxed WYSIWIS” (What You See Is What I See) (Steffik et al., 1987). In this approach, each user’s view is outlined with a rectangle to indicate that view to the other user(s). As with CVEs, the use of independent viewpoints in relaxed WYSIWIS systems offers greater flexibility for individuals to act by providing adaptable views of the shared space (Gutwin and Greenberg, 1998). This chapter presents an investigation of an alternative approach to that of realistic simulation. Removing a focus on realism, we aim to reconsider representation of people in CVEs using an analogous approach to 2D groupware systems, by investigating how people use and interact with particular representational forms. We have begun to experiment

134

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Getting the Picture: Enhancing Avatar Representations in Collaborative Virtual Environments 9 ●●●●●● ● 1 ●●●●●●●●

with different representations of people in CVEs. Instead of simply providing a human-like avatar, we provide a representation that embodies the capabilities of the particular user’s interface. We have adapted the use of an outlined field-of-view, as provided in relaxed WYWIWIS systems, for 3D environments. A user’s view is graphically embodied in the virtual world as a semi-transparent frustum. The extent of the frustum matches the size of the user’s horizontal and vertical field of view. We then present an observational analysis of the use of this technique. Our data is drawn from recordings of pairs of users performing a design task. Analysis reveals instances of detailed co-ordination of talk and gestures, anticipation of problems, and examples of difficulties with occlusion of the environment. On the basis of this investigation, we outline considerations for design, as well as reflecting upon our approach more generally.

9.2 Method There are a number of ways of representing views in virtual environments. The traditional method has been to provide an avatar; a humanlike representation whose head/eyes presumably represent the view that the corresponding user sees. Approaches that are less realistic and more explicit might include representation on the virtual environment itself, perhaps through subtle highlights or shadows on objects in view. However, the increased graphical complexity of rendering lighting or shadows on a moment-by-moment basis, as the avatar moves around the environment, would severely compromise system performance. Additionally, while lights or shadows would be rendered on the graphical features of the world, they would not be visible in the intervening environment. We have extended the MASSIVE-2 CVE system (Greenhalgh, 1999) to support a different form of representation. The field of view is revealed in the virtual environment by outlining the user’s view frustum. This method provides information on the extent of the field of view, and also embeds this information onto and within the environment, connecting the avatar to the objects in view. Our approach can therefore make a viewpoint visible to others who might not be able to see the source of the view (i.e. the avatar itself). The outline consists of a semi-transparent frustum. This “view frustum” technique is shown in Figure 9.1. Note how the black hi-fi object is bisected by the frustum, allowing observers to determine that it borders the other’s view. The use of semi-transparency explicitly highlights the viewing area while showing the objects in the field of view. There are two further aspects to the interface used in our study. Distorted windows called “peripheral lenses” (Robertson et al., 1997; Fraser et al., 1999) are included at the sides of the main view. An example

135

Inhabited Information Spaces

Figure 9.1 User’s “traditional” avatar (circled) with field of view made visible as a semi-transparent frustum.

is shown in Figure 9.2. Our design of the view frustum does not display the bounds of the peripheral lenses, only the main view. Bounding the distorted areas could mislead others to believe that objects in the peripheral view are easily seen, when in fact objects are fairly difficult to see in distorted conditions. Additionally, a complicated representation would be required to bound both the distorted and undistorted views. Our impression was that this representation might prove too complex and have the effect of confusing an observer. Instead, the representation displays only the undistorted view. Arms that can be stretched out into the environment are provided on the avatar – a conceptually similar approach to stretching the representation of the field of view. Participants could point by stretching a single arm out into the world. Picking up and moving objects was represented by both arms stretching to touch an object, and that object turning into a wire-frame for the duration of the grasp. This design helps pointing and grasping actions to be visible across the environment. Using these representations, we asked pairs of volunteers to perform a design task. They were told to collaboratively arrange furniture in a virtual room (pictured in Figure 9.1). This task was designed to investigate geographically distributed interaction and has been used to study CVEs and other communications technologies (Gaver et al., 1993; Hindmarsh et al., 1998; Heath et al., 2001). Participants were given conflicting opinions on the ultimate layout of the room to encourage

136

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Getting the Picture: Enhancing Avatar Representations in Collaborative Virtual Environments 9 ●●●●●● ● 1 ●●●●●●●●

The left-hand edge of Fred’s view frustum is in Harry’s view

The right-hand edge of Fred’s view frustum appears in the right of Harry’s view

011

011

011

011

11

Figure 9.2 Screenshot from Harry’s screen in Example A as he says “you’re behind me”.

discussions. However, we were not interested in the final outcome or success of the participants’ design. Rather, our data selection focused on elements of the data in which participants specifically used or discussed the view frustum as part of their interaction. Our method of data collection closely mirrored that reported by Hindmarsh et al. (1998). We collected audio-video data of each participant’s screen and of their physical environment, including what was being said through their microphone and heard through their headphones. We asked 16 participants to perform the task. Each pair took about an hour to finish designing the room. Our method of analysis has been to account for particular examples of interaction that are recurrent within the data we have collected.

9.3 Analysis Our study reveals three key issues with regard to the use of the frustum representation, which are discussed in the following sections. The first section describes how users are able to intricately co-ordinate their talk and gestures, implicitly using the frustum to understand views of the virtual environment. The second section describes how users are able to anticipate and circumnavigate potential problems that others may have with the visibility of the shared environment. Finally, the third section describes how view frusta may occlude actions without that occlusion being obvious to the frustum’s “owner”.

9.3.1 Awareness and Co-ordination The depiction of the user’s field of view extends into the virtual environment to make it visible to other users. As a result, it is possible that

137

Inhabited Information Spaces

another user’s position and orientation can be understood without recourse to fast and intricate gaze movement, and indeed without recourse to the other’s avatar. Consider Example A, which follows a long period of silence in which Fred and Harry1 have been moving objects in separate parts of the virtual room. Fred’s avatar is located behind Harry. Fred rotates so that they are both facing in approximately the same direction (i.e. Fred can see the back of Harry’s avatar).

Example A2 (Audio Perspective3: Harry) Harry: you’re behind me Fred: yeh (.) I’m gonna grab the white chairs and put em round the table Harry: oh go (on then)

At the initiation of the talk, Fred’s avatar has not been in Harry’s view for some time. However, Fred’s frustum has intermittently appeared in Harry’s view as it extends across the virtual environment. At the start of the example, the left edge of Fred’s frustum is in Harry’s main view. As Fred rotates his view to the left, the right edge of his frustum appears in Harry’s right peripheral lens (see Figure 9.2). At this point Harry says, “you’re behind me”. Harry uses Fred’s frustum as a successful resource in locating Fred and understanding where he is looking, without having to re-orient to find Fred’s avatar. Fred and Harry can easily proceed with their course of action regarding the design task – grabbing and placing the white chairs around the table. This example shows how the view frustum is used as a resource in locating others, even when their avatar is completely out of view. Two key aspects of the example are worth noting. First, direct visual contact does not need to be explicitly established. The frustum can be used to maintain awareness of the other participant’s movements over time. At key moments such as in Example A, this resource can be used 1 All participants in these trials have been called Fred and Harry for purposes of anonymisation. Actual participants are different pairs of users. 2 Examples show the turns at talk that participants have. Numbers in parentheses are the length of pauses in talk. A single period in parentheses means a pause of 0.2 seconds or less. Letters in parentheses show uncertainty by the transcriber in what is said at that point. Square brackets show overlaps in conversation. Italics shows emphasis of those syllables with volume. Colons show elongation of the previous sound, the number of colons being proportional to the elongation. 3 Small delays mean that the time at which words are said and heard differs slightly, as audio is transmitted across the local network. This, in turn, means that timings may differ if talk is transcribed from one participant’s perspective compared to the other’s perspective. In examples presented in this chapter, there is minimal discernible difference.

138

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Getting the Picture: Enhancing Avatar Representations in Collaborative Virtual Environments 9 ●●●●●● ● 1 ●●●●●●●●

in interaction. Secondly, the resources provided by the frustum often feature very plainly in conversations between participants. Thus, it is perhaps what is not said that is the most interesting feature of this example. An implicit recognition of viewpoint is displayed with “you’re behind me”. There is no need to explicitly discuss each other’s position and orientation for the purposes at hand. This implicit recognition shows that talk can be co-ordinated with the use of the frustum without detracting from the task of designing the virtual environment. Example A illustrates how easily interaction with the other participant occurs without the requirement of seeing their traditional avatar representation. The extension of the view into the environment means that users have a resource to collaborate without having to explicitly talk and find out about each others’ viewpoint, even when the avatar itself is not visible. However, the example is one in which understanding the other’s view to pinpoint accuracy is not demanding for the activity at hand. The frustum simply provides an understanding of the location from which the other is seeing, and therefore for co-ordinating conversation between one another. In addition, however, the frustum representation also gives the opportunity to accurately co-ordinate around “visual” features of interaction, such as pointing gestures. In Example B, Fred and Harry are discussing where to put the table and chairs. Harry has stretched an arm to point towards an area of the room (Figures 9.3a and 9.3b). Fred has rotated his view from looking directly at Harry’s avatar, following Harry’s pointing arm (Figures 9.3c and 9.3d).

Example B (Audio Perspective: Harry) Fred: yeh i- i- i see where you mean (0.4) right in the corner yeh? Harry: well somewhere over here cos they’re not really (.) that much use for much else but some people could sit (and eat out) Fred: yeh (1.0) Harry: play a game or something (0.8) cos the chairs and the table go together

As Fred and Harry discuss the placement of the furniture, Fred rotates his view left, following Harry’s arm to look at the table and chairs, and the proposed location. Figures 9.3c and 9.3d show Fred and Harry’s respective views at this point. As Harry says “go together”, Fred’s frustum passes the end of Harry’s pointing arm. Harry seems to align to the frustum moving past his pointing gesture. He releases his mouse button and drops his arm. Compare their views again in Figures 9.3e and 9.3f. The most noticeable aspect from Harry’s perspective is the movement of Fred’s frustum along his pointing arm. The edge of Fred’s frustum progresses along the outstretched arm. The arm is slowly “revealed”, as it is progressively less obscured by the frustum. As soon as his arm is

139

Inhabited Information Spaces

a

b

c

d

e

f

Figure 9.3 Screenshots from Example B: (a) Fred’s view: begins to follow Harry’s gesture, rotating left; (b) Harry’s view: points to “somewhere over here”; (c) Fred’s view: rotates left, following Harry’s arm; (d) Harry’s view: his arm is still just inside Fred’s frustum; (e) Fred’s view: Harry’s pointing arm has dropped; (f) Harry’s view: Frustum passes Harry’s arm and he drops the point.

wholly revealed, Harry drops his pointing gesture. He is able to co-ordinate the production of his gesture with whether Fred can see it. The detailed timing of holding and dropping the gesture relies on the frustum representation to indicate how, whether and when Fred can see the point. This allows the intricate co-ordination of their talk with the gesture. Again, as with Example A, the participants’ talk remains focused on the design task. Co-ordination of talk and gesture occurs without the need to explicitly talk about how to achieve those activities with the system. In other words, the use of a frustum representation can allow the participants to focus on design work rather than the technology itself.

9.3.2 Anticipation So far, we have seen how collaboration can be achieved by virtue of “knowing” what the other can see. The frustum allows gestures to be produced in conjunction with understanding how those gestures are seen. Participants use the representation to design their actions for their

140

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Getting the Picture: Enhancing Avatar Representations in Collaborative Virtual Environments 9 ●●●●●● ● 1 ●●●●●●●●

visibility by others. However, as the CVE interface does not provide rapid gaze movement, there are also many cases where the frustum makes it obvious that things will not be seen. In this case, the frustum allows a course of action that might be called “anticipation”. Participants can circumnavigate potential situations in which others are not able to see relevant features of the environment. This aspect of frustum use is shown in example C. Harry begins by asking which of two large chairs can be disposed of. Fred is busy trying to place the television in the corner (he is finding this a tricky operation).

011

011

011

011

11

Example C (Audio Perspective: Harry) Harry: which do you think is a- best of these two chairs the big three seater or the (0.3) big one comfy one (0.6) Fred: (errl- l- let me) wait there (trying to) put the TV there Harry: cos I don’t think we want this umm (0.3) this big seat that I’m moving now (.) I’ll move it into yer view so you can see it (0.7) oh: (.) sort of (1.0) Fred: hhh Harry: do you think we need this? (0.5) Fred: need what Harry: this- this one I’m selecting (0.4) its in yer view hhh Fred: well just leave it out (of) the way for the time being (0.5) Harry: well no do you think we need it at all (0.3) Fred: we could do (0.3)

This example shows how participants can use the frustum to circumnavigate potential problems in which a referent might not be seen. Harry can see the chair is not in Fred’s main view, as it is not within the bounds of his frustum (Figure 9.4a). Therefore, in order to show the chair to Fred, he moves it into his frustum, so that they can discuss whether it can be disposed of (Figure 9.4c). The frustum allows Harry to anticipate problems with pointing the chair out to Fred, and instead follows a course of action to ensure Fred can see which chair the “comfy one” is, before discussing its use. Harry has some difficulties with moving the chair (displayed to Fred by saying “so you can see it (0.7) oh: (.) sort of (1.0)”). However, once the chair is visibly within Fred’s frustum, Harry re-states his question “do you think we need this?” He proceeds to select and deselect the chair by repeatedly grasping and letting go of it. This changes the appearance of the chair from filled-in polygons to wire-framed polygons. This representation is supposed to show that an object is being moved. Here, however, it seems as if Harry is making the chair “flash” in order to make it more visible to Fred. As Fred continues to ignore the question (and object) Harry’s voice becomes irritated, as he “knows” that the chair is now visible. Harry’s talk in this example seems to show how the frustum provides a responsibility for its “owner” to attend to the object at hand. It is not

141

Inhabited Information Spaces

a

b

c

d

Figure 9.4 Screenshots from Example C: (a) Harry’s view: at the start of the example; (b) Fred’s view: at the start of the example; (c) Harry’s view: holding the wire-framed “comfy one” inside Fred’s view frustum; (d) Fred’s view: Harry’s arms and the wire-framed chair (outlined) are obscured from Fred by his view frustum.

that the object must be viewed, but rather that there is a social, rather than technical, explanation for non-attendance. The possibility of technical reasons for not being able to see the object are reduced, because the frustum implies that what is contained within it can be seen. Harry seems to assume that technical problems with seeing things are no longer a potential “get-out clause” for ignoring the chair. The frustum renders its “owner” accountable to attend to the reference, once work has been carried out to reconcile that the potential referent can be seen. When we see this example from Fred’s view (Figure 9.4d), however, the object and arm (circled) are indeed relatively difficult to see. There are representational issues which make the visibility of the object very different between Fred and Harry’s views. Specifically, the edge of Harry’s view extends between the chair and Fred’s location, and thus Fred’s view of the chair is obscured to some extent by Harry’s frustum. This is the issue to which we turn in the next section.

9.3.3 Occlusion The previous examples have shown that the co-ordination and course of activities can be supported by the frustum. In effect, the frustum beneficially transforms the ways in which gestures and object grasps are accomplished. The last example, however, showed that the effect of rendering a representation of one’s own view on a co-participant’s view could not be seen. Despite this problem, participants are often wary of

142

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Getting the Picture: Enhancing Avatar Representations in Collaborative Virtual Environments 9 ●●●●●● ● 1 ●●●●●●●●

blocking each other’s line of sight. In this particular CVE, sight blocking is especially relevant for a number of reasons: ● ●

●

there is no “solidity” programmed into the environment and thus participants can walk through each other’s avatars; there is no haptic feedback that an avatar collision has occurred. For example, one participant can stand in front of another and reverse into them without realising; and it can be difficult to quickly glance around to check another’s line of sight when their frustum is completely or partially out of view. For instance, having one side of the frustum in view may not indicate which side it is, leaving vastly different alternatives about where the other participant may be looking.

Example D shows how, as a matter of course, Fred and Harry organise sight-blocking movements. Harry anticipates blocking Fred’s line of sight to the object he is moving (the “desk thing”), and notifies him of the possibility.

Example D (Audio Perspective: Fred) Fred: I’m gonna move that desk thing (0.3) [out the way Harry: [I’m just walking across your (0.4) line of sight (I’ll) be out the way in a minute

Harry is walking towards the edge of Fred’s frustum, near to his avatar. At this moment, he interrupts Fred before he has finished speaking. He talks in an apologetic tone, and in a way that implies his turn is an “aside” from the main business of talking about the desk thing. He says “I’m just walking across your” and then pauses for 0.4 seconds as he crosses Fred’s frustum, continuing with “line of sight”. Harry is able to provide an apology for moving across Fred’s line of sight, and obscuring the “desk thing” by virtue of the frustum representation. Blocking Fred’s line of sight onto the “desk thing” is seen by Harry as something to warn him about. However, despite the accountability of obscuring another user’s line of sight, sight blocking occurs more than one might expect within the video data. This is because the frustum represents the exact edges of the participant’s view, and thus cannot be seen by that participant (the frustum is not even rendered on the local user’s view, as it would be invisible and slow their speed of movement through unnecessary rendering). It is simply a representation to other users, and only the frusta representing the views of others can be seen. As a result, it can be difficult to determine the effect of moving your own view on someone else’s view. Example E illustrates how Fred and Harry encounter this problem. Fred rotates his view until it centres on Harry’s avatar. He then rotates to centre the desk in his view.

143

Inhabited Information Spaces

Example E (Audio Perspective: Fred) Fred: what you d- d- what you doin with that desk (0.7) Harry: puttin it near a pink chair again (0.4) will you stop blinding me with that (.) flashlight! Fred: what flashlight? Harry: your [vision Fred: [oh

Fred’s perspective is that he moves so that his view shows Harry’s avatar and then the desk. His reply “what flashlight?” seems perfectly reasonable. On the other hand, when the example is viewed from Harry’s perspective, Fred’s frustum appears across his entire view just prior to his exclamation (compare Figures 9.5a and 9.5b). Perhaps more importantly, it bisects Harry’s view from the object that he is grasping. The effect of his movement on Harry’s world is unavailable to Fred. However, it makes definite changes to Harry’s view, to the extent that he asks Fred to “stop blinding” him. Participants can be unaware that their own movements are causing significant changes to what another sees. The occlusion effect that a participant is having on another’s view may be particularly hard to understand. Fred and Harry are standing side on to each other, a common configuration for discussing the same object. It may be hard to imagine that one’s representation would affect another who is not directly in one’s line of sight. In order to blind a person with a flashlight, it makes sense that they would be facing you, not standing to one side. Example D showed how obscuring with the avatar can be anticipated and accounted for. However, the view frustum is entirely invisible to its owner. Thus, it causes problems where apparently insignificant movements of a participant’s view can cause significant changes to a co-participant’s view. If we now return to Example C, it can be seen that, as Harry is selecting the chair (by grasping and releasing it to make it flash between wire-frame and solid rendering), the chair arguably becomes less visible to Fred behind Harry’s frustum (Figure 9.4d). Whether Fred is not seeing the chair on purpose is not an issue here.

a

b

Figure 9.5 Screenshots from Example E: (a) Harry’s view: “Putting it near a pink chair again”; (b) Harry’s view: “stop blinding me with that flashlight”.

144

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Getting the Picture: Enhancing Avatar Representations in Collaborative Virtual Environments 9 ●●●●●● ● 1 ●●●●●●●●

Rather, it is clear that one participant sees an object as highly visible when, in fact, it is barely visible to the other participant.

9.4 Summary Our study suggests three key issues to consider when representing users’ views in CVEs. 1. Establishing locations and views. Talk about design between participants can be made possible by subtly co-ordinating references to objects or spaces. The explicit frustum representation enables this co-ordination of talk and gesture. Frusta, unlike traditional avatars, can give precise details about what can and cannot be seen by others. This allows simple cases to be assumed, such as whether an object is visible to another user. However, it also allows intricate co-ordination that is not possible with avatars; it offers the ability to design gestures for their visibility by others; and to observe gestures in such a way that the gesturing user knows their movements are potentially visible. 2. Anticipation of problems with visibility. The frustum representation does not comprehensively fix all problems with the visibility of the virtual world; it simply transforms the problem space encountered by users. However, one key feature of the frustum is that it can allow anticipation of such problems. In turn, it provides the necessary resources with which users can work around their issues. Thus, it is possible to insert simple sequences of action (e.g. moving an object into someone else’s view) to make features of the world visible to others. 3. Orientation to occlusion by traditional avatar, but not by frustum. The traditional human-like avatar representation is made relevant in interaction as a potential obscuring device. For example, users are very aware of blocking the line of sight on which another user may be acting or planning to act. Similar alertness is not generally apparent with the frustum representation, which often partially obscures the others’ views.

9.5 Reflections In this section, we review the implications of our study for the broader design of representations in shared 3D systems. First, some suggestions are derived for addressing problems of sharing viewpoints and other forms of 3D representation. Secondly, we discuss the principle of reciprocal perspectives, including a treatment of “out-of-body” camera approaches to shared applications. Thirdly, we consider the ways in which views of information spaces can be shared in “unrealistic” ways.

145

Inhabited Information Spaces

In particular, we describe some problems with the pervasive notion of “real-world metaphor’.

9.5.1 Scaleability Although our study clearly shows some benefits of explicit view representations, we need to consider the scaleability of enhanced representations. We have provided explicit information about the user’s perception and actions by graphically embedding their properties within a representation. Such an explicit approach, however, can provide challenges for scalability. The more users we represent, the more visual information we must provide. We have observed problems with occlusion with two users. It is most likely that multiple representations to support multiple users will compound difficulties. Providing so much information about so many activities could blind a user from the activities themselves. Additionally, consider our use of peripheral lenses (Robertson et al., 1997; Fraser et al., 1999) on the interface. Simply providing extra information about the extent of these distorted views would treble the number of graphical representations required. Alternative representations of viewpoints are also likely to suffer from scaleability problems. For example, spatialised audio representations might be used to indicate locations or boundaries without occluding visual information. These sounds will soon compound, however, to “occlude” conversations between users. Another approach might be to allow users themselves to define their interest in explicit representations (Dyck and Gutwin, 2002). In this case, however, we need to ensure that the user controls themselves do not distract from the task at hand. Thus, we propose that representations of users (and, indeed, user controls) need to consider key contextual factors in order to address scaleability problems. Designers need to decide just what representations are appropriate, and when. These decisions about the context in which information should be presented must include the use of semantics and subtlety. Semantically, we need to consider whether the choice of representation matches the application or task. For example, in the case presented in our study, the task involves decisions about objects and places. Perhaps our system can show which objects are being viewed at any particular moment in a way that fits the task itself? We might derive examples from studies of interior designers experimenting with scale models, observing how they look at and show objects to one another. In terms of subtlety, we would like the message conveyed to appear as part of the virtual task; to appear as a natural metaphor for conveying that kind of information. However, it seems clear that our definitions of appearance of the representation, and of the context in which that representation is relevant transform the ways in which that representation is used.

146

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Getting the Picture: Enhancing Avatar Representations in Collaborative Virtual Environments 9 ●●●●●● ● 1 ●●●●●●●●

9.5.2 Reciprocity of Perspective Schutz (1970) describes how in everyday life we tacitly assume that our individual perspectives are irrelevant for the collaborative task at hand; that everyone encounters objects in the same way, and that if you were me, this is what you would see. The assumption holds until evidence to the contrary appears. This evidence arises frequently in cases involving occlusion with the view frustum. The assumptions that are made by participants regarding reciprocal perspectives are suddenly made problematic, and require work to overcome. This is not to say that participants are not extraordinary in their abilities to “interact their way out of troubles”. Rather it is to say that the reality of CVE interaction is one where reciprocal perspectives are difficult to tacitly maintain, especially with regard to the mutual availability of features of the virtual environment. In the case of our study, the frustum technique was provided to overcome problems of location and viewpoint confusion. However, rather than completely solve these problems, the frustum transforms the problem space that participants encounter in assuming the perspective of the other. The provision of the frustum makes it more difficult for participants to see the effect of moving their view on the world of the other. In Gutwin and Greenberg’s (1998) terms, “power” has been exchanged for “workspace awareness”. Providing awareness information of what you can see can make it harder for you to understand what others can see. This raises important points about the provision of techniques to provide awareness in CVE systems. If we consider providing reciprocal visual awareness, we can broadly categorise CVE techniques into three kinds: camera viewpoints; exchanged viewpoints; and augmented viewpoints.

Camera Viewpoints A common technique used in commercial and research CVEs is the use of out-of-body camera views. These generally take the form of a bird’seye view over the avatar, or an “over-the-shoulder” view to give a wider perspective on the environment, framing the scene or activity at hand. These techniques are commonly known as tethered viewpoints. Some computer games have implemented this technique with automated viewpoints in relation to the focus of activities within the “task”. For example, a martial arts game can be framed with both avatars “in shot”, or racing cars’ relative positioning shown. However, the success of tethered and automated views may be tied to the activity being supported. Racing in a driving simulation is quite a different pursuit from discussing objects with a colleague and making one’s actions intelligible. The problems that may be caused in such situations can, in any case, be incorporated into

147

Inhabited Information Spaces

a game as an additional challenge in winning the fight or race; participant problems can be useful additions for gaming. In a similar vein, activityoriented techniques are suggested in this book for working with virtual cameras in real-time production activities (Chapter 10). Unlike the situation presented in our study, however, the viewpoint of the camera is also the locus of work in this case. In other words, viewing activities from different angles is part of the work of camera operators, and therefore the separation between bodies, objects and views is integral to the production process.4 The problems in “seeing what others can see” in our study suggest that difficulties could occur when using these camera techniques for coordinating real-time work. Whether participants are provided with the ability to change views with respect to their avatar, or whether the interface uses algorithms to change those perspectives, problems in interaction are likely to arise. Consider how difficulties in discerning the extent of another’s view through their representation can disrupt participants’ interaction. If camera views relative to the avatar are used to improve the user’s perceptual experience of the world, then the avatar becomes misleading for others not just with respect to their field of view, but also to the very location from which the represented user is viewing the environment.

Exchanged Viewpoints Techniques in CVEs have been outlined in which different views of participants can be chosen at any moment, in order to support exchange of reciprocal perspectives; in other words, the ability to actually adopt another user’s view on the 3D information space, to see what they are seeing (e.g. Valin et al., 2000). However, the changes in metaphor between individually controlled views and shared views cause the representation of users again to become a critical issue. Just what does an avatar represent in cases where the individual perspective metaphor is abandoned?

Augmented Viewpoints The frustum technique we have presented here comes under an alternative category, “augmentation”, in which reciprocal perspectives are supported by augmenting the realistic appearance of the environment

4 Nonetheless, and for the same reasons we discuss in this chapter, the camera operators themselves may still need to be aware of each other, cf. Drozd et al. (2001).

148

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Getting the Picture: Enhancing Avatar Representations in Collaborative Virtual Environments 9 ●●●●●● ● 1 ●●●●●●●●

with additional information about the “reality” of the user interface (cf. Fraser et al., 1999). Our study indicates that these kinds of technique are feasible, in the sense that participants are able to use the augmented information as part and parcel of their strategies for collaboration. Nonetheless, there are also ways in which the augmentation itself becomes part of the task collaboration. All three techniques to support reciprocal perspectives may be useful in one or another situation. However, our study at least tentatively indicates that techniques of augmentation can support reciprocal perspectives well in situations of “closely coupled” collaborative work.

9.5.3 Unrealism Although technologies to support virtual reality (VR) will continue to develop, it is likely that a range of obstacles will prevent an allencompassing realistic experience. There remain severe difficulties in developing environments that satisfactorily support interaction between individuals within a convincing virtual world. Current haptic interfaces are crude and expensive enough to be uncommon. Even though the realism of 3D graphics is rapidly advancing, interfaces to realistically capture movement and real-time expression for these illustrations lag behind. Network bandwidth (and the speed of light) means that different versions of events and causality are presented to different users. Yet, fundamentally, the success of the virtual reality programme as it stands depends on the success of the realism metaphor; it depends on the willing belief that one is sensing and acting in a computer-generated environment. It may be worth reconsidering whether the successful design of VR applications rests upon our ability to completely simulate ordinary realities and physical worlds. This is particularly true of CVEs, where the nirvana of a “truly real” virtual world may cloud the fact that a communications technology exists which, however rudimentarily, has the potential to be deployed on currently available hardware. While the overriding interest in achieving a realistic sense of virtual presence continues, the shorter-term potential of CVEs to provide effective tools to support distributed collaborative work is being undermined. Note that here we are not suggesting that shared spaces should necessarily present less realistic representations in favour of abstract information. Some might see the frustum representation as more appropriate for an abstract data space in which coping with a realistic world is less of an issue. In some cases this may be true; however, we see this as an application-specific issue, to be considered in those terms alone. As outlined in the introduction to this chapter, whether abstract visualisations or real-world simulations are portrayed in CVEs, success will always depend on participants’ ability to collaboratively make sense of, and act

149

Inhabited Information Spaces

in, the environment. We would like to emphasise that our studies simply question the commonly assumed correlation between realism and usability. Perhaps it is time to directly investigate the usability of CVEs rather than measuring their realism through sense-of-presence and co-presence metrics. We have shown that representations that might be termed unrealistic (in the sense of simulating human embodiments) can still, in fact, be used. These representations actually turn out to be realistic (or, rather, practical) in the sense that, unlike humanoid forms, they are a viable proposition for accomplishing collaborative tasks in virtual spaces.

9.6 Conclusions The implication of the approach we have outlined here is to become far more pragmatic than trying to achieve a perfect representation of reality in information spaces. CVEs provide a technology that will currently support interaction between distributed participants and enable them to engage and co-operate in particular activities. Our study has shown that users are able to accomplish collaborative tasks in this way, given the right resources. Rather than assume that we need to simulate conventional realities, we have examined one way in which users can accomplish certain forms of collaborative work through CVEs. Instead of concealing the limitations of the technology through photorealistic portrayals, we have tried to provide individuals with a sense of the constraints of the CVE system. Our approach has further implications for designers of abstract inhabited information spaces. The conclusion of our study must be that realistic portrayals of inhabitants must be tempered with representations that convey the in vivo characteristic properties of an interaction, rather than its “natural” properties. Unlike quasi-realistic approaches to inhabitant representation, therefore, designers might apply our strategy to both abstract and quasi-realistic spaces alike. Our study has shown some benefits and considerations of constructing representations that allow users to identify and deal with the characteristics of the system. We hope to have shown an alternative approach to designing information spaces, by shifting focus away from the general trend of hiding their properties and towards providing an enabling technology for remote collaboration.

150

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

New Ideas on Navigation and View Control Inspired by Cultural Applications

10

Kai-Mikael Jää-Aro and John Bowers

10.1 Introduction and Overview In this chapter we describe some of the work which was conducted within the eRENA project of the EU’s Inhabited Information Spaces research activity. This project combined the expertise of partners from computing, social scientific, artistic and television backgrounds to investigate the application of advanced media technology in cultural settings. Our “inhabited information spaces” were “electronic arenas” for participation in cultural events of an entertaining or artistic nature. In this chapter we focus on how, in these application areas, we have rethought some familiar interaction issues in human–computer interaction (HCI) to offer the beginnings of an approach which may be useful in such complex settings. Let us give a little more of the flavour of the application areas which concern us and the approach we have adopted. We organise this around five key themes. 1. Mixed reality technologies. All the demonstrators, applications and other developments within eRENA hovered on the border between the virtual and the real. While we were particularly interested in designing inhabited information spaces which were rendered with 3D graphical techniques familiar from virtual reality (VR) research, we were concerned to do so in ways which were sensitive to our final applications settings (galleries, performance spaces, television studios). This required us to be concerned about how interaction with the system might be impacted by the real world with all its interruptions, non-computer-mediated communication channels and physical limitations. Furthermore, as we shall see, some of our demonstrators directly explored physical, embedded interaction techniques in relation to VR.

151

Inhabited Information Spaces

2. Real-time applications. We were not developing offline methods. All the rendering, interaction, sound manipulation and other techniques we were working with had to give real-time results. To set ourselves tough test cases, we often worked with formats that allowed for a degree of improvisation on the part of performers or production crew. 3. Large scale participation. eRENA applications went beyond the “one person-one computer” paradigm of classical human–computer interaction into interaction between crowds, over large virtual spaces and physically separated nodes. We were particularly interested in formats which enabled public participation, potentially on a large scale. This in turn set us challenges for formulating techniques that could be used with minimal training by a heterogeneous set of users. 4. Media rich environments. Nor was interaction restricted to graphics presented on a computer screen and input through a keyboard, but rather whole-body interaction with physical props, display on multiple screens (and alternative projection technologies) and multichannel audio were more commonly our concern. 5. Cultural context. The applications were intended for culture, art, performance and entertainment. We made a practice of regularly displaying our work to the public through relevant cultural outlets (television programmes, events at arts festivals, exhibitions, public performances). Again this set us challenges for delivering high-quality content appropriate to those settings and the expectations of audiences within them. It also gave us the opportunity to flexibly explore the roles of spectators, performers and producers – while obtaining direct feedback on the quality of our work.

10.1.1 Challenges for Interaction Design As we have already noted, such settings and applications set core challenges for interaction design. In this chapter we describe our contributions to addressing these challenges in three areas: 1. Navigation and view control. How does a user choose what to see of an interactive artistic installation or of a performance? How, within an interactive digital television show, does a participant’s view relate to the television audience’s view, and both of these to the views available to production and direction personnel? What ordering is given to how views are selected and scenes navigated between? As we shall see, in eRENA, we experimented with a number of techniques for supporting navigation and view control – in particular, ones which extend the conventional notion of an avatar as indicating a locus of unitary view control. 2. World and experience design. How are “electronic arenas”, the “inhabited information spaces” of art and entertainment, to be designed?

152

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

New Ideas on Navigation and View Control Inspired by Cultural Applications 10 ● ● ● ● ● ● 1 ●●●●●●●●●

What are their constituents? How are they assembled? In particular, how does the structure and organisation of such environments serve to make them arenas for experience, that is, places where certain incidents might be encountered or effects enjoyed? 3. Production management. How does one produce an experience, and yet again, if it is a large-scale experience, how does one co-ordinate the possibly very many people and computer processes involved in the production? What are the “behind the scenes” activities like and how in turn might we offer support for them? Over the course of our work on eRENA, we came to interrelate our answers to such questions rather closely. We began to develop, for example, world design techniques in tandem with navigation methods for such worlds and do so in such a way as to facilitate real-time production work. We unfold some specific examples of this approach in this chapter. This is done most effectively through presentation of a series of examples of cultural events realised in the project.

10.2 Interactive Performances Many of the cultural events within the eRENA project were realised as interactive performances in one sense or another (see, e.g. Carion et al., 1998; Bowers et al., 1998a, 1998b; Benford et al., 1999c). The underlying ideas and themes of these spanned a wide spectrum, both aesthetically and technically. We will first look at the scope of work in general with regards to interactive performances, and then look in detail at some specific examples which involved the improvisation of intermedia environments – that is, where the construction of graphical (and, in some cases, sonic) environments took place live within the performance itself. Interactive performances can be classified along several dimensions: How tightly scripted are the performances? What can be affected by the interactor(s)? What is the number of performers? What is the relation between “performer” and “audience”? What is the overall physical setting of the performance (staged, promenade, public space . . .)? Some examples of interactive performances within eRENA demonstrate these factors. ●

CyberDance (Carion et al., 1998) consisted of a human dancer equipped with motion capture sensors and a set of realistically rendered computer-animated dancers. The virtual dancers would move in synchrony with the human dancer. The choreography was carefully planned beforehand, yet the behaviour of the computer-generated parts was created in real time, in response to the movements of the human dancer. The setting was one of a “traditional” staged dance performance, but where some of the performers were computer-generated, the audience were off-stage and not involved in the interaction.

153

Inhabited Information Spaces

●

●

To the Unborn Gods (for a description, see Norman et al., 1998) was a “virtual reality opera”, where a human singer interacted with computer-generated figures. Their behaviour was indirectly affected by the singer’s actions, by an off-stage operator choosing suitable paths. Here too, the setting was very much in the way of a traditional staged music performance, but where some of the actors were screenprojected. Lightwork (Bowers et al., 1998c) was an intermedia performance where two human musicians interacted with abstract computergenerated forms which were generated in the real time of the performance. The graphics were created through the actions of one player and responded to by the other. An audience faced a performance area in a conventional manner. We will describe Lightwork in more depth below.

While the above three examples involve interactivity between performers and computer-realised material, they involve quite conventional stagings. In each example, an audience has a space associated with them, the performers have theirs (in the first two examples, this was provided by a theatre stage). Other eRENA performances involved rather more flexible relations between the performance space, audience and performers. ●

●

154

Desert Rain (Benford et al., 1999c; Shaw et al., 2000; Rinman, 2002) featured a number of discrete spaces through which participants moved as they engaged with a game-like activity based upon events in the Gulf War. At a number of points, actors directly engaged with the participants, giving them instructions or “debriefing” them. At one striking moment in the production, each participant interacts with a virtual environment projected onto a “rain curtain” – a fine water spray which holds a back-projected image. At a key moment, an actor appears between the projector and the rain curtain and slowly moves towards a participant, casting a silhouette in the projected image. The actor passes through the rain curtain and gives a swipe card (used later in the performance) to the participant. Desert Rain manifests a complex set of relationships between participants, actors and production crew. Performances have not been realised in conventional theatre settings (the debut of Desert Rain was in a disused factory). The crowd behaviour simulation and animation system ViCrowd (Hoch et al., 1999b; Lee et al., 2000) was used in a number of performances/installations for interaction by groups of people with computer-animated figures. People entering the interaction space were tracked and their combined behaviour caused various scripted behaviours in the computer-generated figures. As the tracking was only position based, movements in the space could be made arbitrarily complex for the benefit of the other humans present, but the virtual humans would react the same way regardless of how the triggering conditions were achieved.

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

New Ideas on Navigation and View Control Inspired by Cultural Applications 10 ● ● ● ● ● ● 1 ●●●●●●●●●

Let us give a little more detail of one aspect of all this endeavour, where we explore interactive performances with an eye to opening up the scope for improvisation by the performers.

10.2.1 Lightwork While being an artistic performance combining electro-acoustic music and the real time construction of graphical environments, Lightwork also served as a “live” occasion to test many of the ideas for navigation and interaction in virtual environments proposed within the eRENA project, all the more so as the format of an improvised performance would sharply bring to the fore any problems with these ideas and their implementation. The design goals, intermingling technical and aesthetic issues, included the following: ●

●

● ●

●

The use of a minimum of encumbering apparatus – rather the adaptation of the standard tools that our performers would most typically use (i.e. electronic music control devices). A large element of improvisation in the performance – we wished to avoid the use of prepared world models and instead generate material algorithmically on the fly. The gestures of interaction with the environment being made public and a legible, meaningful part of the performance. “Compressing” and “decompressing” interactive gestures, so that small causes may get large effects and vice versa – the challenge being to allow experimentation with this, while still ensuring the legibility of performer activity. The creation of an “infinite collage”, where sound, graphics and text are recombined and perceivable from a variety of viewpoints – any particular realisation of Lightwork is just one of an indefinitely large number of possible ones.

This being a real-time situation where any interaction would have immediate and publicly visible effects, a classic direct manipulation approach to interaction was felt to be insufficient in that it both would require the following of similar steps to achieve similar results, and small excursions from the intended path may lead to unintended and unrecoverable effects (in the sense of requiring explicit, publicly visible and time-consuming actions to undo). Rather, we followed an indirect principle of “algorithmically mediated interaction”. The idea was to interpret the actions of the performers as input parameter values to content generation algorithms that would create visually rich, animated graphics. At the same time, the interaction methods were chosen so that all actions would result in meaningful effects – thus movement, while being adjustable, would happen along paths that were guaranteed to always

155

Inhabited Information Spaces

have something interesting in view. In a sense the performers would be shaping material independently flowing by. The initial sequence of the performance consisted of a 90-second flythrough through large squares displaying textures representing the various themes of the performance. Finally, the viewpoint entered the large sphere that was the container for the graphics during the rest of the performance. This sphere was texture-mapped with maps and satellite images that were automatically switched every 80 seconds. The viewpoint orbited around the centre of this sphere in either of two types of paths: a. A cloverleaf pattern according to the function r = m + n × sin t, where r is the radius modulated by the sinusoidal function, n determines the maximum deviation from the base radius m and  gives the period of oscillation. See Figure 10.1 for some samples of this function. b. A back-and-forth motion through the centre of the sphere.

a

b

c Figure 10.1 Three parameter settings for the “clover leaf” path: (a) m = 10, n = 10,  = 3; (b) m = 20, n = 10,  = 4; (c) m = 30, n = 10,  = 5.

156

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

New Ideas on Navigation and View Control Inspired by Cultural Applications 10 ● ● ● ● ● ● 1 ●●●●●●●●●

The plane of motion was slowly tilted back and forth, in order to give as many viewpoints as possible – yet the angle of tilt was kept within ±/4 radians in order to retain a sense of up and down. For the type a orbits, the view direction could be chosen to point either in the direction of motion or towards the centre; for the type b orbits the view could point either towards or away from the centre. The position of the viewpoint would also determine the spatialisation of the sounds. While this still was a geometrical navigation method, we had started moving towards a more target-centred approach, where the path was not under the direct control of a performer, but rather the viewpoint control was mediated by analyses of performer input. Within the sphere various objects could be created. One of the performers (V) could select between five form generation algorithms with the parameters of the algorithms being set by means of analyses of performer input. Specifically: ● ●

011

011

011

11

● ● ●

scaffold, a grid of “pipes” – cylinders or parallelepipeds. The colour of the pipes as well as their density and lengths could be varied. formModulator, a tessellated sphere, the vertices of which were displaced according to a frequency modulation algorithm. The same fundamental algorithm was used to create an immersiveForm, sculptureForm or cave, using different parameter ranges. These forms could then be animated, giving the impression of an organic, “breathing” shape. chamber, a cube with rotated and scaled cubes inserted in its surfaces. plenumbulator, simple rectangles with text or images, set at random positions and orientations. orbiting forms, groups of textures and text strings orbiting around a common centre.

All shapes except scaffold were texture-mapped with images selected from one of 13 themes. Each new object would step to the next image theme. Only one of each kind of object could be present at any time, but any combination of them could be displayed simultaneously. V could create and destroy objects, as well as determine the type of path (a or b orbits, above) by pressing foot switches. V’s activity was analysed by a program we called “The Interactive Narrative Machine”, which computed magnitude and irregularity parameters over various time-scales. These (indirectly) determined m and n, the view control parameters, while also setting parameter values for the form generation algorithms and affecting the shape and motion of the created objects. For a full overview of the set-up, see Figure 10.2. At the end of the 15-minute performance, the viewpoint automatically retraced its path through the initial textured screens.

157

Inhabited Information Spaces

V

S

T

Figure 10.2 The connections between the units. The data from the wind instrument and foot switches are passed to the Interactive Narrative Machine, a Max program on an Apple Macintosh, making analyses of the behaviour of V over three different time periods, compressing these into two values, fed to T’s SGI O2, which runs the visualisation software. The foot switches determine what objects are to be created at the centre of the sphere as well as which type of orbit is to be used, while the data values generated from the wind instrument set the parameters for object creation as well as for the orbits. The spatial position data are forwarded to the Mac of S, where SO-2, yet another Max program, uses these to spatialise the music improvised by S.

The sound was also created and mixed using algorithmically mediated interaction methods. There was a large set of prepared sounds, both physically modelled and sampled, that could be mixed together live – see (Bowers et al., 1998c) for more details.

Performance Experience The first performance was realised in December 1997 at the Royal Institute of Technology in Stockholm and revealed a number of difficulties with Lightwork as initially conceived. While the quality of the graphical images (cf. Figure 10.3) and the electro-acoustic sounds was admired,

158

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

New Ideas on Navigation and View Control Inspired by Cultural Applications 10 ● ● ● ● ● ● 1 ●●●●●●●●●

011

011

011

011

11

Figure 10.3 A chamber in Lightwork.

the indirect connections between elements often made it difficult to understand what was going on. Our techniques of “algorithmically mediated interaction” involved the analysis of performer activity in a number of time windows. Effectively, this introduces a smoothing and a delay to relations that V’s activity might have to graphical consequence. As the worlds were quite complex and the analyses of performer activity had several different consequences, this made it hard for the audience (and performers for that matter) to pick up on the interactive relationships in the work. V found himself exaggerating his pressing of the foot switches which introduced new world contents or removed objects which were present as this gesture had a more immediate and legible effect. In short, while our methods of analysing performer activity and indirectly using this to parameterise world building and navigation algorithms worked in a technical sense, the degree of indirection in interactivity was excessive for enabling the audience and performers to track what was happening and why. In Lightwork there were multiple mappings requiring calibration. Performance data needed to be measured in different time windows. The results of these analyses needed to be mapped to parameter values for

159

Inhabited Information Spaces

the world building and navigation algorithms. In principle, we could vary how the performance data were measured and scaled, the number and size of the time windows, the functions which mapped analysis results to parameter values and so forth. Ideally, one would wish to explore many different alternative mapping functions under different performance conditions. We were careful to set the ranges of values so that (for example) generated graphical objects would all be placed within the enveloping sphere and that the navigation paths would never be out of an intended range. That is, a number of calibration issues could be settled a priori. However, this alone does not guarantee that the range of worlds that will be generated will be appropriate and that the views obtained of them will be interesting. This requires an empirical approach to calibration and, before the performance deadline, we were not able to deal with all these matters fully. Accordingly, it was left to performers to troubleshoot calibration problems in performance itself. For example, V felt forced to adopt a playing style that would be guaranteed to keep a reasonable path, but which was less expressive than desired. However, our indirect, algorithmically mediated approach to interaction did not always give performers adequate resources to troubleshoot such problems. For example, rather more often than one would wish, the projected view was uninteresting and could not be swiftly adjusted.

10.2.2 Blink Continuing the ideas from Lightwork, the next performance was Blink (Bowers and Jää-Aro, 1999). Many of the elements of Lightwork were retained, but expanded with new graphical themes and new texts (including randomly shuffled three-line poems which could appear superimposed upon 3D image material). Importantly, we introduced some new interaction and navigation ideas to avoid some of the problems encountered earlier. In particular, the viewpoint in virtual space could be attached to any of three cameras moving within the space. In such a way the risk that nothing interesting would be visible was minimised. Cutting between views was supported with a direct interaction technique – so that “degenerate” views could be cut away from there and then. In short, Blink complemented our ideas of indirect, algorithmic interaction (e.g. to generate world materials and to control some features of camera movement) with direct interaction (e.g. to cut between views and in other ways override the algorithms if necessary). This notion of navigation by cutting reveals an influence of a cinematic or television metaphor for view control and indeed, for us, was a direct result of our participation in the television-oriented applications of virtual environments which we will describe later. In Blink we defined three cameras which would tend to produce related but different views (again by analogy to a cinema–television convention of giving different

160

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

New Ideas on Navigation and View Control Inspired by Cultural Applications 10 ● ● ● ● ● ● 1 ●●●●●●●●●

camera operators different responsibilities). Two of the cameras circled the centre of the virtual environment, one near, the other further out, while an “explorer” would move radially from the centre to random points on the periphery and back, giving a further variety of views. The images from one of these cameras could be selected for “transmission” or “TX” in the vocabulary of television. Another virtual camera could be selected for “preview” without altering the TX view. In this way, shots could be prepared and meaningful edits made from one camera to another. (Ideally, perhaps, all three cameras should have their images continually available with TX selection being a matter of just choosing the camera required. With the graphics hardware available to us, however, we found that computing four views – the cameras plus TX – was not feasible at the resolution and frame rate required.) Furthermore, based on experiences both from Lightwork and the Real Gestures, Virtual Environments workshop (Hirtes et al., 1999), it was decided to make the physical environment of Blink visually richer by using a large number of screens surrounding the audience, rather than having them as a backdrop for the performers. Perhaps the biggest change was that instead of relying only on computed data for the input, the software was now equipped with a user interface with manual overrides for all functions, so that technical problems, bad parameter settings, and so forth could be bypassed during the performance. See Figure 10.4 for a view of the interface. Blink was performed as part of the Digital Clubbing event at the Now98 arts festival in Nottingham, England in October 1998. The venue was The Bomb, a nightclub in the city. On a small stage at The Bomb, 4hero performed in collaboration with Carl Craig, who was located in his studio in Detroit, through a video and data link. Two graphics operators were seated at separate SGI O2 workstations, each running the Blink software, though unsynchronised with each other. Twenty-one monitors were distributed over the premises in seven groups of three. In each group, one monitor showed the live transatlantic video link into Carl Craig’s studio while the others displayed images from the two workstations. In addition a large screen was placed adjacent to 4hero’s performance area, showing a video mix of these three sources. The groups were positioned so that the audience would typically have a line of sight of at least one set of monitors no matter where in the somewhat cavernous environment of The Bomb they were located. The initial intention was that Carl Craig and 4hero were to exchange MIDI data with each other, and thus interact with each other’s music during the performance. The combined MIDI stream was to be analysed to yield input parameters to the graphics generation algorithms. The analysis was simpler than the one used for Lightwork in order to make the relation between music and objects more perceivable. In the event, the MIDI connection turned out to be unreliable and it was necessary to use the manual overrides throughout to set the graphics parameters. The

161

Inhabited Information Spaces

Figure 10.4 The manual interface to Blink. At the top are a number of buttons, sliders and selection lists that define all the available functions and their parameters and in addition the buttons that determine what views will be shown. The buttons marked “Near orbit”, “Far orbit” and “Explorerorbit” determine which camera view will be shown, “Left”, “Front”, “Right” and “Back” determine which direction the camera will look towards. “Set TX” will then actually put this image on the public screens. In this manner the view space can be explored to find an optimal shot without the audience being subject to the browsing. Texts and images can also be pasted onto the “visor”, i.e. placed right in front of the camera and moving along with it. These can be tested separately (using the buttons at the lower right) and when a suitable combination has been found, it can be sent to the TX screen.

162

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

New Ideas on Navigation and View Control Inspired by Cultural Applications 10 ● ● ● ● ● ● 1 ●●●●●●●●●

graphics operators thus worked out a schedule in which they would gradually exercise the abilities of the software and independently, but in a generally co-ordinated way, increase the tempo of cutting and the complexity of objects during the progress of the performance. In this way, the dynamics of the performance of the Blink software approximately followed the dynamical shape of 4hero and Carl Craig’s musical performance.

Performance Experience The audience reception to the event was extremely enthusiastic, though this of course is due in large part to the participation of (in their field) popular and well-known musicians, yet it seemed that the graphical work (see Figure 10.5) was considered to add to the event and make it even more exciting.

011

011

011

11

Figure 10.5 The TX window in Blink. We see a scaffolding and within it an immersive form. We can also see a box-like object, half hidden behind the scaffolding; this is one of the other cameras in the environment.

163

Inhabited Information Spaces

There were, however, problems with the system, mainly concerned with the pace of interaction and cutting. A mouse-based interface with lots of on-screen buttons, of which many had to be pressed in a particular sequence, was not conducive for the rapid cutting usually associated with (for example) the music videos which typically accompany the kind of music performed. The synchronisation of the “preview” window and the TX view was only accurate to within a second or so, so the actual transmitted view would tend to be slightly different from the one intended. This made it hard to perform closely co-ordinated edits (e.g. where there’s a cut from one view to another at the moment when the camera collides with an object). Overall, we felt our concept of multiple virtual cameras and supporting live editing between them was validated in Blink. The software operators rarely found that they had nothing interesting to cut to. The camera paths had been selected to contrast with each other while yielding occasional overlaps of material to permit meaningful edits to suggest themselves. The interface to the Blink graphics software combined direct and indirect algorithmic techniques, enabling the operator to override inappropriate algorithmically generated material, while making edits that could be synchronised with the ongoing music. Furthermore, the combination of directly actioned cuts with machine-controlled camera paths allowed operators to “buy time” for themselves: a camera could be left to explore the environment while the operator thought about what to do next.

10.3 Inhabited Television A different interpretation of the concept of interactive performances is that of inhabited television (Benford et al., 1998, 1999b; Greenhalgh et al., 1999; Craven et al., 2000). Here the concern is (in some way) to hybridise some of the traditional concerns of television (e.g. broadcast to a potentially mass audience) with the interactive possibilities of online virtual environments. The sources we have just cited describe a number of experiments in inhabited television. In its fullest form, inhabited television manifests the following features: ● ●

● ●

The set for the show is a shared virtual environment. The actors-performers-presenters as well as (at least some) audience members access the shared environment through a computer interface. The audience can take an active role in the unfolding events. Views from the virtual set are broadcast to those audience members who cannot be present in the virtual environment.

The persons involved in such a production can thus be divided into the following groups: performers, those who have been engaged to create the show – this can be taken to also include staff not visible in the

164

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

New Ideas on Navigation and View Control Inspired by Cultural Applications 10 ● ● ● ● ● ● 1 ●●●●●●●●●

environment but controlling it, such as directors, camera operators and support staff of various kinds; inhabitants, members of the public present in the virtual environment and able to interact with it and viewers, who receive images and sound from the virtual environment, corresponding to the traditional television audience. Inhabited television, then, can be regarded as a contribution to research on the possibilities of digital television and allied new media – one which particularly emphasises public participation in broadcast events through (characteristically) activity in an “inhabitable” virtual environment. In what follows we develop the theme of this chapter by particularly highlighting the interaction and navigation innovations that have been explored in this application of VR technology. We do this by contrasting two inhabited television shows: “Heaven and Hell – Live” and “Out of this World”.

10.3.1 Heaven and Hell – Live Heaven and Hell – Live was a game show with a theme based loosely on Dante’s Inferno. It was broadcast live on British television’s Channel 4 on 19 August 1997. The show was realised as a collaboration between three eRENA project partners: the University of Nottingham, Illuminations Television and British Telecom. Two celebrities and the programme host were the performers, and 135 members of the public who had received copies of the software client for the virtual environments (Sony’s Community Place) were the inhabitants. The two performers did a treasure hunt, an avatar stacking game, a quiz and a gambling game, the intention being that the inhabitants would help the contestants with their tasks. The interest level by the public can be gauged by noting that not only the 135 inhabitants but an estimated 200,000 others stayed up in the middle of the night to watch the show. While it was a great technical achievement, Heaven and Hell – Live was problematic as a television show. There were insufficient resources to do a full-scale test beforehand. Thus both performers and inhabitants were working through the game environments, controlling their avatars and performing their tasks as best they could live. This resulted in the ostensible game content – rule enforcement, point counting etc. – often not being taken very seriously, as the task of working in the environment while trying to make adequate improvised television of one’s efforts became the performers’ primary concern. The loose relevance of the game format to the show was also noted by the inhabitants, many of whom tended to not take the game very seriously either. Furthermore, as the software only supported text-based communication (the performers in the studio could of course speak directly to each other), interaction between performers and inhabitants was slow. As a consequence the inhabitants tended to ignore the game and instead drifted away to chat with each other.

165

Inhabited Information Spaces

Figure 10.6 A scene from Heaven and Hell – Live. Inhabitants and actors in a graveyard setting. Reproduced courtesy of Illuminations.

The properties of the interface also made for a strange mix of paces – while the actors were working hard at their respective machines, typing text and moving through the environment, the view for the television audience was quite a lot slower than normal for an entertainment programme. Furthermore the Community Place software placed constraints on the number of other avatars a user could be aware of at any one time. Thus, inhabitants did not have the impression they were part of a massparticipatory event and, in some cases, became suspicious about the authenticity of the occasion (e.g. if they failed to see themselves on television, in spite of being in the camera field). Similarly, for the television audience, the environment often looked quite desolate, in spite of there actually being well over a hundred people “on set” (see Figure 10.6). Whatever else Heaven and Hell – Live was, it was scarcely recognisable as the participatory game show it was intended to be.

10.3.2 Out of This World Out of this World was a determined attempt to avoid the problems encountered with Heaven and Hell – Live and bring off an event which

166

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

New Ideas on Navigation and View Control Inspired by Cultural Applications 10 ● ● ● ● ● ● 1 ●●●●●●●●●

011

011

011

011

11

Figure 10.7 The opening briefing of Out of this World. The robot team stands in the foreground with the alien team in the distance. On the video screen is the game show host and producer, John Wyver of Illuminations. Reproduced courtesy of Illuminations.

was more recognisable in television terms. Again the game show format was followed but various production and navigation facilities were added into the show’s software to encourage (or even, if necessary, mandate) appropriate participation. Out of this World was not a television broadcast but was shown to a theatre audience as part of the International Symposium on Electronic Art in Manchester, England in 1998. (Full descriptions of the technologies in the show can be found in Benford et al., 1999b, while Bowers, 2001 recounts the story of the behind-the-scene production work.) The game scenario was of two teams, one composed of robots, one of aliens, being pitted against each other in a series of games on a doomed space station, the winning team being allowed to escape (see Figure 10.7). There were still two performers that were the main contestants, but in this case they used an immersive interface with motion tracking and were placed on stage either side of a projection screen showing a live edited TX view of the show. The inhabitants were eight volunteer members of the audience who were seated by workstations in a separate room, using

167

Inhabited Information Spaces

a joystick interface to move in the environment. MASSIVE-2, the system used, allowed all inhabitants and performers to be connected by audio and be visible to each other. Two specific applications had been built in the MASSIVE-2 system to help address earlier difficulties. A production support application allowed the show to be represented as a series of phases with varied capabilities assigned to avatars and objects on a per phase basis. In this way, the movement capabilities of the avatars could be constrained so as to aid their participation in a game depending on, for example, whether groundplane movement only needed to be supported or whether the avatars could climb upwards. This allowed a single familiar interaction device to be used (a joystick) without the user pressing modifier buttons. At key moments also control could be removed from the team members so as to enable the smooth movement of their avatars to a new location. For example, travellators took the avatars from one game environment to the next. The transition from one phase to another was actioned by a member of the production crew. This enabled reasoned decisions to be made about the pacing of the action in discussion with the show’s director. In addition, some special purpose phases were defined to speed things along or cover technical failures without suspending the show as a live event. In addition, a virtual camera interface was developed to assist members of the production crew in obtaining meaningful views of the action. Shots could be targeted on specific avatars or objects. The distance from the target and movement around it could be manually controlled to permit various zooming and panning effects. Free navigation could be engaged to enable “roaming” shots but the cameras could always “snap” to a particular target if the operator received an instruction from the director or picked up on a cue from the action itself. Out of this World was much more successful in realising an event which was recognisably a game show than was Heaven and Hell – Live. The pace of the action was more consistent with what one would expect from such a piece of entertainment. The inhabitants were much more focused on the games within the show and their role in competing against the other team in playing them. In large part this was due to furnishing the production crew of the show with applications which were specifically designed to assist in production management and camera control. It is also important to emphasise, though, the role of the production crew’s professional competence in working with these technologies. For example, there are, in principle, many different ways in which a game show in a virtual environment could be shot and edited from the views offered by the four virtual camera operators. Not every way of working with the cameras or of selecting shots would have led to a recognisable production of a game show. Indeed, the director experimented notably with different sets of instructions for the camera operators and different principles for making edits over the course of her work on Out of this

168

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

New Ideas on Navigation and View Control Inspired by Cultural Applications 10 ● ● ● ● ● ● 1 ●●●●●●●●●

World (Bowers, 2001) before settling on a style and pace of shot composition and editing which seemed appropriate. The main difficulties with Out of this World as identified by audience members centred on three problems: ●

●

011

011

011

011

11

●

A lack of empathy with the show and its characters. It was hard for audience members to engage with the concerns of the robots and aliens. The cover story for the show was too thin. A lack of a sense of history and context. In particular, no effort was given to elaborating any sense of history for the characters, why they came to be on the space station, what their past relationships with each other might have been, and so forth. Objections to the game show format. Many audience members thought that the choice of a game show format was unambitious.

In more recent work on inhabited television, these problems have been directly addressed by embedding the broadcast within a whole series of preparatory activities which help elaborate a participatory narrative. Avatar Farm/Ages of Avatar prepared for a live Internet broadcast show by providing a series of online worlds and allowing a sense of “community” to develop over the course of several months. Characters were introduced into these online worlds and narrative fragments were suggested before four inhabitants were invited to participate in the live event itself (for details of this work, see Craven et al., 2000).

10.4 Production Management Whether we are concerned with cultural events in an artistic or entertainment tradition, it is clear that some major challenges are presented to virtual and mixed reality research if those technologies are to be used to realise the events. In the events described so far, we have addressed particular issues concerned with world design, and navigation and view control. In Lightwork and Blink, we were designing virtual environments of a visually rich and (hopefully) engaging nature. However, we needed also to ensure that the worlds were presented in such a way that interesting views on them would be relatively easy to find. In other words, there was a reciprocal relation between world design and view control issues. Technically, we addressed this problem on geometrical and optical grounds. For example, we “layered” world content within a sphere in such a way that chambers could be created and viewed inside and out by switching between an inner and an outer orbiting path. Practically, we provided performers with a variety of resources for creating worlds and, in Blink, choosing between cameras. Our intention was to give performers a rich range of resources for world creation and navigation but to structure them in such a way as to make them usable within the real time of an improvised performance setting.

169

Inhabited Information Spaces

As in the research on inhabited television, we realised early on that it was impractical and unnecessary for performers, inhabitants or the operators of virtual cameras to have unconstrained six degrees of freedom (6DOF) motion when trying to frame shots in the environment. It was all too easy to overshoot the target or end up at an inconvenient location. To remedy this, more application-specific movement methods were developed for the later inhabited television productions to enable camera operators to focus on particular persons or objects and move around them without losing them from sight. These camera vehicles could, for example, constrain their motion to the surface of a sphere centred on the relevant object, thus simplifying movement to a 2DOF (plus zoom) operation. In Out of this World, the virtual cameras interworked with a production management application so that, for example, cameras could be moved to set locations at the beginning of a new phase or be able to take as the object of their shot a particular entity which might be important to the events in the current phase. In all these respects, production design and management is a matter of conjointly designing virtual environments and navigation control so as to simplify the real time burden on participants in live performance.

10.4.1 Finding and Framing the Action We have already mentioned that the director of Out of this World experimented with a number of ways of organising the camera operators and selecting from the shots they gave. One of the reasons for this is that in television, as in cinema shot composition, it is not just a matter of selecting the right target and framing it. One has to compose shots and edit between them to convey a sense of the action in the scene. Sometimes it is enough to select a particular target (e.g. an actor or avatar performing a critical and clearly legible action). At other times it is vital to show characters in relationship to each other, or in mutual relationship to objects in their environment, or juxtapose one shot with another to capture the action. The problem of finding the action and framing it is even more acute when there is a large number of participants (e.g. in Heaven and Hell – Live). Clearly, this is a different order of view control problem from those classically discussed in 3D computer graphics or virtual reality. How can we find and frame the action? Is it possible to design view control and navigation techniques which directly support this requirement? To begin to investigate these issues, we developed SVEA (Sonification and Visualisation for Electronic Arenas), a tool which enabled us to view the patterns of people moving in the space and to place cameras in the environment. The intent was that these cameras would have semiautonomous behaviour, left to themselves they would roam the environment, finding the hotspots of activity and frame a shot so that it would

170

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

New Ideas on Navigation and View Control Inspired by Cultural Applications 10 ● ● ● ● ● ● 1 ●●●●●●●●●

capture the group of actors at that point, attempting to (depending on the settings) maximise either the number of faces or profiles in view. It was also possible to add additional weight to specific performers, so that they would be “favoured” in view. A human operator could then take over to make any adjustments, or even move the camera somewhere else deemed more important at the time. We give further details of our techniques for camera deployment shortly. Attempting to further enrich the impression one could have of the behaviour of a populous crowd we experimented with sonifying a number of parameters, so that one would be able to hear, for example, the levels of aggregation. (Further details of our sonification strategies and a preliminary experimental test of the sound model we used can be found in Bowers et al., 1999). To find the potential areas of interest in an environment, we made a novel use of the Spatial Interaction Model (Benford et al., 1995b). The central ideas in this model are that in a virtual environment (potentially) every object has a focus and a nimbus, where the former is (approximately) an abstraction of an object’s “attention”, and the latter (approximately) an abstraction of an object’s “projection of presence”. Typically these are taken to be functions over space, but it is possible to extend the notion over time as well (S¸andor et al., 1997). The combination of focus and nimbus can then be used to model an object’s awareness of another object: the awareness A has of B is some function of A’s focus on B and B’s nimbus on A. An ordinary language approximation of these notions might be: my awareness of you depends upon my level of attention to you and the degree to which you are making your presence felt. Benford et al. (1995b) and S¸andor et al. (1997) show how (given certain assumptions) awareness levels can be quantified in continuous and graphed “spaces” respectively. These notions have been used in a number of systems for collaborative virtual environments to control the level and kind of detailing in rendering in shared environments among other applications. To support camera control and view selection in virtual environments on the basis of finding and framing activity, we make use of the spatial model on the assumption that objects (most specifically avatars) in the world will be differentially aware of the various parts of the scene and will themselves orient towards things of current interest. In principle, we can compute various “activity landscapes” using focus, nimbus or calculated awareness. For example, we can determine the sum for every point in space of the nimbus at that point contributed by every object in the environment, giving a “nimbus landscape”. An awareness landscape could be determined in the corresponding way using the awareness level for every object of others at each point. A nimbus landscape would give high values at points where many objects are nearby, the latter gives high values at points where, in addition, many objects are focusing upon each other. Such landscapes can then be visualised as a “heat map” of

171

Inhabited Information Spaces

the environment, where the levels in the activity landscape are displayed. In SVEA, we simplified this notion in a number of ways. We visualised only avatars. As motion in these particular settings was taking place in a horizontal plane we could justifiably ignore the avatars’ heights above the ground plane. We computed the heat maps for just the points where avatars were located. This allowed us to visualise the avatars as small triangles (showing position and orientation) in a 2D “overview” environment and colour them to show the landscape value at their location. Naturally, other visualisation techniques could be experimented with – see (Hirtes et al., 1999) for discussion. A visualised activity landscape (map) could be used for navigation and view control in a number of ways. First, directly: a user could move to areas of interest on the activity map by directly interacting with it. That is, one might click on a location and be teleported to it. Alternatively, the map could be used to inform the use of conventional navigation controls. That is, one might see an area of interest on the map and move towards it using whatever one’s current navigation vehicle might be. Finally, one could use the activity map algorithmically. For example, a region of the map might be selected (e.g. by drawing around it) and a camera location and orientation computed which maximises the number of avatar “faces” or “profiles” in the region that are in shot. In Hirtes et al. (1999), we describe a number of algorithms which could do this and similar view computations given a selected set of objects to display in shot. To give another example of how an activity map can be used algorithmically, consider its use as a map of dynamical “potential” which would govern an autonomously determined camera movement. As an example, we implemented what we termed a “puppy camera”, a camera which would follow the gradient of awareness until reaching a local maximum and, if this point was stable, stay there for a while until the camera gets “bored” and sets off in a random direction to find a new local maximum – a behaviour analogous to that observed in puppy dogs that the authors have encountered. This camera thus roams an environment seeking out interesting places and capturing views from these. We experimented with a population of four puppy cameras. In order to keep them from flocking to the same highly interesting event (something that was a concern even with human camera operators in the inhabited television shows we studied), the cameras could avoid each other by associating a force of repulsion to each of them. As a proof of principle, we used SVEA to present data captured during Heaven and Hell – Live (for more information, see Hirtes et al., 1999 and Figure 10.8).

10.4.2 The Round Table: A Physical Interface To concretise our work with SVEA, we needed to take the application beyond a proof of principle and prototype a use scenario. In the work

172

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

New Ideas on Navigation and View Control Inspired by Cultural Applications 10 ● ● ● ● ● ● 1 ●●●●●●●●●

011

011

011

011

11

Figure 10.8 SVEA visualisation of a crowd based on the data recorded from Heaven and Hell – Live. The inhabitants are represented by coloured triangles, showing both their awareness levels (represented by saturation changes rather than colour scales in order to accommodate colour-deficient users) and their orientation in the plane. Open triangular objects represent cameras and their immediate field of view. As we did not have access to the geometry data of Heaven and Hell – Live, the background is empty, but the geometry is of course trivially added. At the bottom is a timeline showing the current time in the data set – in the prerecorded data set a user could move back and forth in time at will, otherwise the timestep would be updated at the rate of incoming data.

on inhabited television we have discussed, shot selection and vision mixing was done by the director using conventional television equipment, a video line entering a mixing desk from each workstation running virtual camera software. In our initial version of SVEA, shot selection and camera deployment were accomplished by mouse operations. While our camera algorithms worked very well, it was awkward to manually orient cameras in the environment with a mouse-driven interface. Furthermore, if the mouse was moved to select a new camera, time delays were introduced in comparison with the simple button press used to

173

Inhabited Information Spaces

select an input channel at a conventional video mixer. The direct yet speedy manipulation of a SVEA visualisation to compute camera locations and select views seemed to require a novel interface solution. In collaboration with Michael Hoch (then at the ZKM in Karlsruhe in Germany), we worked with the Round Table as a physical interface approach. The Round Table itself is composed of a projection surface and a camera (see Figure 10.9). Optically tracked physical objects (“phicons”) can be manipulated to accomplish interaction operations, their positions and orientations being reported on by image analysis software. Bowers et al. (2000b) describe a number of applications of the Round Table including a virtual space sound mixer and as a presentation device for interactive artworks. We projected SVEA visualisations on the table, placing differently shaped physical objects to represent cameras, to select groups of objects for framing by algorithmically deployed cameras or for zooming up for detailed scrutiny. These latter two facilities made another use of the idea of an awareness landscape. Selection of a group for the camera would be done by placing a phicon at a particular point. Those avatars that were aware of that point would then be selected for a camera shot. Likewise a zoom would be done so that the view grew to contain just those objects aware of the given point. By default there were four puppy cameras roving around, but the placement of a camera phicon would snap one of the cameras to that view overriding the puppy behaviour. If a marker was placed on the camera phicon, that camera was selected for transmission (TX). In the real world environment of the Round Table, we

Figure 10.9 Michael Hoch using the Round Table. Inside the cylindrical bottom is a projector displaying images on the matt glass tabletop. In the can above the table is an infrared lamp and a camera with an IR filter. The manipulable props are coated with IR-reflective tape making them visible to the camera. Reproduced with permission from Michael Hoch.

174

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

New Ideas on Navigation and View Control Inspired by Cultural Applications 10 ● ● ● ● ● ● 1 ●●●●●●●●●

placed other screens to display views from cameras and TX. Thus, we are envisaging a scenario in which users can share the physical environment of the table and co-operatively work using it as a shared display to manipulate views of a large-scale virtual environment. Depending on the application these users could be behind-the-scenes production crew, selected inhabitants, or other participants. We found that interacting with physical props, while allowing all the advantages we had foreseen – rapid manual interaction by multiple users in close proximity – raised new issues. Of course, the phicons would not move of their own accord when the display was updated. This meant, for example, that when zooming in on a section of the view, any previously placed camera markers would remain where they were while the objects they were presumably trained at would move away. Obviously this could not be taken to mean that their corresponding cameras would jump to those new points, so instead we chose to de-assign those cameras and let them continue moving on their own until picked up again by a camera phicon. This meant we had to change the metaphor from placing representations of cameras to placing a tool that would in turn place a camera. This is somewhat indirect, but perhaps not more so than the indirection inherent in using a mouse to move objects on a separate screen. If such a solution seems opaque, then alternatives would be to disallow zooming or to compute zooming algorithms which distorted the display to maintain at least some of the camera-phicon associations, among other possibilities.

10.4.3 Conclusions The most important points of our work with production management are the following: ●

●

011

11

●

Real-time interaction. As noted in the introduction, all applications had real-time requirements, thus our camera-control algorithms also had to operate in real-time. We could not prepare camera paths beforehand but had to be able to find at least approximate shots automatically as events unfolded. Large-scale participation. We wanted to be able to accommodate large numbers of participants in virtual worlds, and therefore could not restrict ourselves to only following a few select performers, but rather we had to be able at any time to move the viewpoint to somewhere else in a possibly large space in order to capture an interesting event and also being able to notice that such an event was taking place. Understanding rules of practice. We need to start from how television actually is produced. Our suggested interfaces indeed are quite different from what is used by television producers today, but they do

175

Inhabited Information Spaces

●

●

●

not in themselves preclude the tasks that need to be done, even if they suggest alternative fashions in which to perform them. Division of labour. Our systems are created with the goal of allowing a working group to fluently and in real time divide the work between themselves as well as between computer system and human workers – a camera can be passed from autonomous algorithm to human camera operator, to a different operator and back to the computer process, all by simply moving a wooden block over the projection surface. Improvised action. A script can help camera operators and directors to plan their work but an inhabited television event is by necessity at least partially improvised on account of the invited inhabitants which make every show a live event, so it must be possible to fluently adapt one’s work as needed. Activity-orientation. Camera direction is not concerned with geometry but with activity. Therefore we have developed methods for finding activity and orienting cameras in relation to it.

10.5 Discussion: Navigation, Presence and Avatars We have been discussing some of the work from the eRENA project which was concerned with supporting large-scale real-time interaction in cultural and entertainment events using virtual and mixed reality technologies. In these settings we have described the interaction techniques and world design principles we have worked with in a number of artistic and entertainment applications. While we have presented some specific applications, these demonstrate some general design principles. By way of discussion, we situate our design principles in relation to general questions of interaction, navigation and avatar design for virtual and mixed reality environments.

10.5.1 Avatar-centred Navigation Classically, navigation in virtual environments is taken to mean something like “move my avatar in a certain direction with a certain orientation at a certain speed”. Various restrictions may be placed on what directions and speeds are allowed – e.g., one may use a “walking” or “driving” metaphor where the avatar is restricted to move along a ground plane, walls and other objects may be impenetrable, or one may use unrestricted six-degree of freedom motion, passing through any object at will. A typical experience is that it is preferable to have additional constraints on motion, in order to avoid ending up at an awkward angle halfway through the groundplane, unsure of how to manipulate one’s

176

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

New Ideas on Navigation and View Control Inspired by Cultural Applications 10 ● ● ● ● ● ● 1 ●●●●●●●●●

interaction device so as to straighten up and stand on the ground again. In part this may be due to mismatches between the input devices and the available degrees of freedom, mouse-based interaction, for example, requiring modifiers and/or modes in order to support all six degrees of freedom. Still, even when using 6DOF devices, it is often difficult to reach one’s intended goal at the intended orientation. We also questioned whether it is necessary to actively steer one’s avatar every single step of the way, if one already knows where one is going. A common experience is that most of the time in virtual environments is spent on navigation, getting desired objects into view and then moving to them, both taking up valuable time and being a cause of frustration when one’s path goes astray or the intended target cannot be found. Accordingly, when we have encountered avatar-centred navigation in the work reviewed above (e.g. to support the inhabitants in inhabited television), we have found it most effective to constrain motion and map it in an activity-informed way onto simple (low DOF) navigation devices. Not only does this make for easier participation, it also makes the distribution of avatars more predictable, thus facilitating production and camera work. In short, rather than work with generic motion and selfrepresentation notions, we believe that the capabilities of an avatar should be specifically configured for the activities it is to engage in and so that its conduct can be appropriately picked up by others.

10.5.2 Object-centred Navigation Another common navigational paradigm is based on objects in the environment. Viewpoints are predefined points that the designer of the environment deemed likely to be interesting for a visitor. By choosing the name of a viewpoint in a list, one’s avatar is transported to that spot and encounters an intended set of objects there. The transportation may be a smooth animation through the environment, or it may be an instantaneous relocation, a teleport. A teleport may also be undertaken to some arbitrary co-ordinates; this requires that there is some kind of map of the environment, or that the co-ordinates of some likely spot can be easily ascertained and saved, for example, as a URL. Target-based movement allows the user to point at some visible object in the environment and be brought there, again either through a smooth animation or through a teleport. We have reported explorations which have further refined objectcentred navigation so that, as in the later inhabited television experiments, significant individual avatars or the “centre of gravity” of a group of them can be targeted. The camera control application used in Out of this World also supported viewpoints but, significantly, these were integrated with a production management application to support a working division of labour between a show producer and a camera operator.

177

Inhabited Information Spaces

Furthermore, once targets were selected, view control was informed by the need to produce coherent camera shots, rather than only allowing unconstrained motion. Our experience, then, is again that navigation and view control need to be informed by the practical activities that participants are performing (e.g. as a camera operator or as a show producer) and specifically configured with those ends in mind.

10.5.3 Activity-oriented Navigation We wanted to go further in the direction of informing interaction in terms of the practical activities that participants are engaged in by supporting navigation and view control where the intended target is not a geographical spot as such, but a place of interest and activity. We have described an approach which employs a model of awareness and activity in virtual environments to make inferences about where the “hot spots” might be so as to enable transportation there – in effect like a helpful taxi driver in a foreign city when asked: “Take me where the action is.” We described how navigation informed by representations of activity (activity maps/landscapes) might interwork with algorithms for view computation, for example, to maximise the number of faces of a group of avatars who are at a particular hot spot. While we presented these notions in terms of the deployment of cameras in the production of events, similar tools could be made available to inhabitants and other participants so that, for example, they can visualise an overview of activities and occurrences that they happen to be interested in and manipulate this to control their movement in the environment.

10.5.4 Navigation as Montage, Dispersed Avatars A consequence of the above is that we may loosen the idea that navigation is the moving of an avatar through space and make it more akin to real-time image cutting in television or film production, selecting from multiple viewpoints. A teleport still uses the metaphor that there is a single avatar, though one that can be instantaneously transferred from one place to another. But consider the following scenario: there are multiple users, each represented with an avatar, indicating their position in the environment and serving as a focus for the interaction of others with them; at the same time there is a pool of cameras that any user can access to check out what’s going on in the rest of the world, and on finding an interesting place, can ask to be teleported there. Now, in a sense the cameras in this scenario are parts of the user’s avatar, as the user is able to perceive the environment from the viewpoint of a given camera, yet it is not a unique access point, as several users may be peering through the same camera. If the cameras are

178

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

New Ideas on Navigation and View Control Inspired by Cultural Applications 10 ● ● ● ● ● ● 1 ●●●●●●●●●

extended with the ability to manipulate the environment, we have completed the dispersion of the avatar. Furthermore, from this perspective it can be noted that our Round Table implementation of SVEA disperses views not only over multiple cameras, but also their control over several physical devices which in turn can be shared between several users. In this way, we allow for flexible mappings between users, peripheral devices, displays, views, cameras and avatars.

011

011

011

011

11

10.5.5 Accomplishing Presence and Intelligibility We are prepared, therefore, to entertain some radical consequences of our work for interaction concepts in virtual and mixed realities. Components of the avatar “function” might be distributed and disembodied (e.g. there is no necessity for cameras to be “owned” or visualised). Of course, we may wish to constrain the picture for particular purposes in design. Our emphasis throughout has been on informing navigation and related issues on the basis of general event design and production considerations. Particular events may indeed require a traditional avatar concept. Other events may allow or require very different approaches. Furthermore, one may need to support different realisations of the relationships between navigation, view control and embodiment for different participants in the same event (e.g. a traditional avatar for an inhabitant, a dispersed capability for production work). By decoupling navigation from view control and by varying all these matters with respect to avatar-like embodiment, it might be argued that we are inviting complexity and unintelligibility. After all, much conventional thinking about traditional avatars and interaction in multi-user virtual environments is based around assumptions about how avatars can be used as a resource for mutual inferencing about participants’ conduct and perception. For example, if I can see your avatar at such a spot oriented in such a way, I should be able to make inferences about what you can see and, perhaps, what you are doing. On our more distributed and variably embodied model, how can this be true any more? Interestingly, Bowers et al. (1996) present ethnographic evidence that it is not the case that such inferencing occurs automatically and nonproblematically in all circumstances in conventional shared virtual environments. Participants commonly check whether their sight of another’s avatar does indicate the other’s presence before interacting with them, for example, by pre-calling the other’s name. Avatars might be creatively used to indicate the momentary unavailability of their user, for example, by laying them down. Avatars might change “ownership” as someone else in the real world takes over at the workstation. Other communication media might be used alongside the virtual environment itself to troubleshoot problems and sort out identities. In short, the presence of

179

Inhabited Information Spaces

persons in a virtual environment and the intelligibility of their actions are matters which are accomplished using designed technologies alongside whatever other resources participants have for making sense of what is going on. Participants “naturally” decouple navigation, view control, embodiment and activity, and reassemble them in their practical activity within virtual environments in whatever way is appropriate to the task at hand. From this point of view, our advocacy of an approach to interaction which regards navigation as montage and disperses the traditional capabilities of the avatar over multiple perceptual loci turns out to be a modest proposal.

180

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

11

Presenting Activity Information in an Inhabited Information Space Wolfgang Prinz, Uta Pankoke-Babatz, Wolfgang Gräther, Tom Gross, Sabine Kolvenbach and Leonie Schäfer

11.1 Introduction

011

011

011

11

A group of people that work together in the same spatial environment relies on various possibilities to observe the activities of partners, to monitor the progress of a project or to recognise the development of group structures. Such activities are often recognised peripherally; nevertheless, they stimulate spontaneous and informal communication. Although the computer supported co-operative work (CSCW) research area yielded a number of systems and solutions that enable and support distributed co-operation processes, distributed work is still significantly more difficult to manage than co-located work. A significant reason for this is the missing perception of the activities and actions within a distributed group. Therefore, distributed groups often suffer from a lack of awareness of the common activities. Co-operation partners are often not aware of activities of other co-operation partners that are relevant for their own work. The synchronisation problems resulting from this often lead to decision problems, misunderstandings or duplicated work. Thus, effectiveness, spontaneity, and social orientation possibilities in distributed teams are limited. The social forces which facilitate the behaviourmilieu synomorphy in an environment – that is, in a behaviour setting (Barker, 1968) – are very limited in electronic spaces. Awareness support can make the difference between an electronic behaviour setting and a pure electronic space (Pankoke-Babatz, 2000). Apart from the lack of awareness of actions that could be co-operative, there are limited opportunities for chance meetings. In the local working environment, coincidental meetings often initiate communication and the exchange of experience and knowledge. Prussak (1997) describes this phenomenon very appropriately: “If the water cooler was

181

Inhabited Information Spaces

a font of useful knowledge in the traditional firm, what constitutes a virtual one?”. Social contacts that are initiated by chance encounters at copiers, printers or coffee machines are important for the social orientation, the mutual exchange of information and knowledge, and the coordination of shared activities (Swan et al., 1999). The Tower system that is presented in this chapter addresses this problem. Tower (Theatre Of Work Enabling Relationships) provides different approaches for the support of awareness and the creation of chance encounters for local and distributed teams. First, we describe requirements and the methods we applied for requirements analysis. This is followed by the architecture of our awareness environment. Then we introduce different means for the presentation of activity information using a portal, Smartmaps, a 3D environment, or ambient interfaces. The chapter concludes with lessons learned about awareness in distributed electronic settings.

11.2 Related Work and Requirements The importance of awareness for CSCW was initially described and analysed for synchronous co-operation processes (Dourish and Bellotti, 1992). Later, different approaches were presented to stimulate awareness of shared activities through different awareness widgets such as multiuser scroll bars, or radar views (Roseman and Greenberg, 1996). For the generic support of synchronous applications, infrastructures have been developed that support the exchange of synchronisation and notification events between different applications by the provision of a notification server (Patterson et al., 1996; Segall and Arnold, 1997). Awareness is of equal importance for the support of asynchronous co-operation processes. For successful co-operation it is essential that users are informed about relevant activities of co-operation partners in a situated and intuitive way (Schlichter et al., 1998). Most approaches that provide asynchronous awareness visualise user actions by appropriate awareness icons, that indicate current or past activities on a shared object (Sohlenkamp et al., 2000). Infrastructures to support asynchronous awareness have been presented in Lövstrand (1991) and Prinz (1999). Video-based media space (Gaver, 1992; Lee et al., 1997) or video walls have been developed to bridge spatial distance between different locations and to support chance encounters and ad hoc communication between different places. In Benford et al. (1997c) such an approach is applied in a VR environment to provide chance encounters for people who browse the web. The Session Capture and Replay system is an example of a system that allows other users to replay past actions. This is particularly helpful for users who join the group process later. The system captures users’ interactions with an application and stores data into a session object. Group

182

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Presenting Activity Information in an IIS 11 ● ● ● ● ● 1 ●●●●●●●●●●

members can annotate, modify and exchange the session objects. Session objects consist of data streams representing captured interactions and audio annotations, which users can add while their interactions are captured. When a session is replayed, data streams are re-executed by the application (Manohar and Prakash, 1995). Begole et al. (2002) did statistical analyses on the availability of users of an instant messenger system and found out work rhythms. These work rhythms are used to project the current or future availability of other users. Common to all these approaches is the concentration on a particular co-operation type (synchronous/asynchronous) or a restricted application domain. However, co-operation processes are not limited to a particular application or a specific co-operation type. This is why Tower aims to develop a model that supports awareness across the boundary of different applications and that can visualise activity information using different presentations. The system provides an application-independent infrastructure that receives event information submitted by user activities or activity sensors. It stores and administers the event database and forwards events for the presentation by different indicators to users based on user-specific interest. Such an infrastructure is able to provide a comprehensive awareness support for different co-operation situations. In the following, we describe different scenarios and the resulting requirements for an awareness environment. A prerequisite for the smooth operational sequence of a co-operative process is the seamless interlinking of the actions of different team members. The availability or explicit provision of information or working objects that are required for the next activity often initiate user activities. To minimise meta-communication by which co-operation partners tell each other what they have done, it is necessary that notifications about relevant activities are produced automatically and presented to the user in the appropriate situation (Pankoke-Babatz and Syri, 1997). In addition to the immediate notification, users require a presentation that provides an overview of the current activities in the co-operative environment. Such an overview must be user configurable to allow each user to adapt it to their specific needs. We describe such a user configurable awareness portal later in this chapter. Users who work on a shared object or within a shared workspace require object-related information about the activities of others. Most systems provide activity symbols that indicate recent activities by others on an object. In addition, it is helpful to get an overview of all activities in a shared workspace or environment. We will show how such an overview can be provided using Smartmaps indicators that are based on tree-maps (Johnson and Shneiderman, 1991). In addition to the indication of activities, they also support an activity-based navigation in a shared workspace. This is exemplified by the integration of Smartmaps with BSCW shared workspaces (Appelt, 1999).

183

Inhabited Information Spaces

The relevance of peripheral awareness to support mutual orientation was identified in Heath and Luff (1991). In addition to the summarised presentation of awareness information via a portal, an environment such as Tower must provide possibilities for the presentation of user activities in the user’s periphery. For this purpose, Tower applies a 3D environment in which user activities are visualised automatically. The layout of the 3D environment is based on shared working contexts. Thus, the environment also represents meeting places where people who work in a similar context but at different geographical locations meet coincidentally. We will show in this chapter how such a world is created and how user activities are represented in the 3D environment. The settings discussed so far describe synchronous communication patterns. However, the support for long-term project work requires the support of asynchronous awareness. As also shown for video-based media spaces (Fussell et al., 2000), the continuous observation of remote partners does not suit team needs. Therefore, support for the presentation of past activities was developed in Tower. This supports users who need information about recent activities after a temporal absence from the co-operation process. A similar approach is described in Greenhalgh et al. (2000a). In the DocuDrama section of this chapter (Section 11.8) we describe how Tower addresses this issue. Although awareness is studied extensively in synchronous and collocated work situations (e.g. Heath and Luff, 1991), needs in asynchronous and distributed settings are still under-researched. Tower has provided the opportunity to study awareness needs in different settings and to use a tangible artefact (Brave et al., 1998) to elicit further needs and to discuss the potentials of technical support in more detail with potential users. The results of these user studies augment the description of the presentation tools described in the following. Finally we present the lessons learned about awareness.

11.3 User Involvement and Studies From the very beginning of Tower development, user groups participated in the project team. Their participation enhanced the project team’s understanding of user needs with respect to awareness. Planned Tower features or prototypes were discussed with the users and their early feedback improved the system design. We selected such an iterative design process based on qualitative evaluations since the co-operation processes supported by Tower were also long term and asynchronous. Studies of awareness needs and requirements for technical support were performed in two different ways. On the one hand, discussion of needs and the usability of awareness features were discussed with potential users of application partners. Here, needs for awareness support were studied by means of interviews and workshops. On the other hand, the

184

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Presenting Activity Information in an IIS 11 ● ● ● ● ● 1 ●●●●●●●●●●

team process of the distributed Tower team itself was observed throughout the whole course of the project.

11.3.1 Partner Settings and Evaluation Methods One application partner was a small German company with about 20 staff members. Its major aim was the provision of web-based support mainly to engineers. The other partner was one of the world’s leading providers of civil engineering consultancy and support services currently expanding to about 10,000 employees. Representatives of the two Tower application partners also became members of the Tower team. The collaboration began with expert interviews and expert workshops. We discussed the Tower features planned and under development as well as particular needs at their application partners’ sites. This was followed by five user workshops performed at the application partners’ sites. Between three and ten staff members from the respective application partners participated per workshop. Workshops comprised three kinds of topics: the need for awareness support in the respective settings, the demonstration of Tower features and finally the discussion of the potential usability of those features. Thus, Tower features were used as tangible interfaces to elicit further user needs. The Tower development team was distributed across four companies in two countries. In addition one expert from each application partner joined this team. In total, about 40 people had access to the Tower workspace that was used to co-ordinate the project work. Among them about 15 used it regularly. In the course of the project we could study how awareness needs changed depending on the team process and on particular work situations. Seven persons joined the project at a later stage. From them we could study the particularities of the awareness needs of a newbie. The course of the Tower team process was documented continuously from the members’ perspective. In addition, technically measured data – that is, recorded event data – were analysed to understand the team process, the relevance of media usage, and the requested awareness support in the respective phases of team work. This enabled a detailed view into the awareness needs but also into the way in which awareness was achieved in a distributed work setting. This also informed our study on how work activities are reflected in the recorded event data and how these can be processed to provide suitable awareness notifications. The diversity of application partners in Tower has provided the opportunity to study various different work settings and to find out about their particularities with respect to needs for awareness support. With the introduction of early prototypes we were able to test potential effects on users. Where there was positive acceptance of a feature, we could learn from users how to improve it. Where users objected to or rejected a

185

Inhabited Information Spaces

feature, more detailed analysis of the underlying rationale was needed. We had to analyse whether the rejections were due to shortcomings of the feature, which could be overcome, or whether they were due to misfits with their needs and misassumptions about awareness. Interestingly, any feedback we gained from the users also disclosed user-specific needs for awareness support on the other hand. Thus we could learn a lot about awareness needs in general. We found that the needs for awareness depend on the different settings, the particularities of the co-operation cultures, the modes of working and the current work process and the individual interests of an actor in a particular situation. When Tower started the major aim was to design a theatre of work, i.e. a 3D stage which augments an existing shared workspace system with the requested awareness about ongoing activities. In the course of the work, however, awareness turned out to be much more complex than we had thought and we realised that it cannot be supported with a single tool. In real world settings, awareness is a multi-channel phenomenon, thus it requires multiple means for the indication of activity information in an electronic setting. Therefore, in the course of the project several different features were developed, from among which a user may choose the most appropriate one.

11.3.2 Do Users Meet at all in a Shared Workspace ? Being present in the same place usually provides an opportunity for chance encounters. One motivation for Tower is the provision of an awareness space that notifies users about the presence of others in their working context. We have analysed the log files of user activities in a shared project workspace to check whether situations occur where more than one user has been present at the same point in time. In such a case, knowing about the local presence would have opened up a chance to meet in the context of the actual work. Such functionality goes beyond the services offered by simple presence awareness systems such as ICQ (“I Seek You”, a service to get in touch with people across the network: http://www.icq.com/). The log file contains data for each user action in the shared project workspace for the period from January until October 2001 (approximately 200 working days). In this time period, about 15 users had access to the workspace, but there were only 8 active users who used the workspace frequently. The project workspace contained more than 1000 objects, e.g. documents, presentations, figures, etc. The result of our analysis is shown in Figures 11.1 and 11.2. Each figure shows the number of meetings between two or more users within a certain time interval. A meeting is defined by two consecutive actions of different users on the same object. The x axis indicates the time interval between two actions in minutes. The y axis denotes the total number of

186

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

Presenting Activity Information in an IIS 11 ● ● ● ● ● 1 ●●●●●●●●●●

meetings that occurred within a time interval. Figure 11.1 indicates that only 20 meetings occurred within a short time space of 1–2 minutes, while 60 meetings took place over a 10-minute period. This means that every third day two users met on exactly the same object. If we consider folders within the project workspace as meeting places, then the number of meetings increases drastically. The grey line in Figure 11.1 indicates that within 5 minutes 60 meetings between two or more users took place in the same folder. Using 15 minutes as the time interval we count 90 meetings, i.e. every second day people have met coincidentally in the same folder. These numbers increase further if we consider the whole project workspace as a meeting place. In this case more than 400 meetings took place within a 5-minute period, i.e. two or more meetings on every working day. This frequency of coincidental meetings is similar to the number of meetings between two people in the same spatial environment. The statistics provide evidence that users actually meet in a shared virtual project or workspace and they further indicate that it is important to choose the right level of detail for the construction of the meeting

011 Same object 140

100 80 60 40 20

011

011

11

Same folder

120

– –

10

20

30

40

50

60

Figure 11.1 Number of meetings on the same object and in a folder.

Same project space 1200 1000 800 600 400 200 0 –

10

20

30

40

50

60

Figure 11.2 Number of meetings in the project workspace.

187

Inhabited Information Spaces

places. If the level of detail is too high, then the number of coincidental meetings is too low. If it is too low, more meetings will occur but then the context of the meetings is too general. The empirical observations are confirmed by our practical experiences with the construction of the Tower world that are reported later in this chapter.

11.4 The Tower Architecture In order to support awareness in a distributed electronic environment, the events that take place at one location or application, must be conveyed to the locations of the other users sharing the environment. First, this requires a means to detect events caused by user actions. Secondly, these events need to be recorded and processed. Finally, the events must be presented and situated in the local action field of the other users. Consequently, the Tower system provides three basic components: a set of various sensors that recognise user actions; an event and notification server that stores, administers, filters, and distributes events; and a set of different visualisation tools. In the following we describe the architecture of Tower. Figure 11.3 illustrates the components and architecture of the Tower system. Different sensors recognise user actions. These are either integrated in applications or realised as agents that observe user actions, e.g. modifications of shared files systems or web pages. Sensors forward events to the central event and notification server (ENI) by calling appropriate common gateway interface (CGI) methods of a web server. This web-based approach was chosen to provide a simple, yet powerful interface that can be used by almost all applications (Prinz, 1999). The ENI-server stores and administers events. It forwards events to appropriate indicators that have a registered interest in events using predicates over event attributes. In addition, the server implements functions for the authentication of users, the authorisation of access rights, and the aggregation of events in a history. The transformation module allows a semantic transformation of events to satisfy requirements of different applications. Further, it supports the interworking of different ENI servers by enabling servers to exchange events. The reciprocity module informs users about the interest of other users. This provides transparency to avoid the misuse of the awareness environment for control purposes. Additional modules can extend the ENI-server. For example the context module has been developed to contextualise incoming events. Examples for contexts are a project, a task or collection of document folders that belong to a work package. The classification is done by matching event attributes with a predefined context description (Gross and Prinz, 2000). In the following sections different indicators are described for the presentation of activity information in inhabited information spaces.

188

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Presenting Activity Information in an IIS 11 ● ● ● ● ● 1 ●●●●●●●●●●

Work environment Activity sensors

Agent sensors

Event processing modules

Theatre of work Ambient interfaces

HTTP-server

011

011

011

011

11

3D multi-user environment

DocuDrama

Symbolic acting module Space module

CGI-scripts Event processing modules Authorisation Key action module

Context module

Event and notification server Interest profiles

Access control

Reciprocity

Aggregation

History

Transformation

Figure 11.3 Tower architecture.

11.5 Personalised Overview of Activities: The Tower Portal Our user studies revealed that an individual user may like to be made aware of several electronic locations. For example, a user may be involved in a project organising an electronic seminar and finally being a shareholder. Such a user may want to perceive awareness information about all relevant contexts. In case that these contexts are relevant over a certain period of time, this user may assemble awareness information about all these contexts in a personalised Tower portal. Figure 11.4 shows such a portal of a Tower team member. It provides awareness about people currently present in the Tower project space and the activities in the Tower workspace as well as information about the current German stock market. The Tower portal provides users with a personalised visualisation of awareness relevant for their personal work context. For this purpose users configure their own Tower web page, called MyTower. For the individual configuration of MyTower users specify interest profiles to express which events and awareness information they are interested in (Fuchs, 1999; Fuchs et al., 1995). This may be the notification of changes to documents in shared workspaces, activities of group members, or

189

Inhabited Information Spaces

Figure 11.4 An example of MyTower.

information delivered by agents observing web contents. In addition to the selection of contents in the Tower web page, users decide on the visualisation of the event information. For this purpose the Tower portal offers a multitude of configurable indicators. The configuration of MyTower in Figure 11.4 contains five different indicators. In the first line the RandomImage-Indicator shows the photos of all users currently online in Tower. The Smartmaps gives an overview of the user actions in the Basic Support for Co-operative Work (BSCW) workspace of the Tower project. In the next row a ticker tape informs in more detail about the project activities of the Tower team members. The last indicator is a URL-Indicator. It displays the picture of the current stock development of the German stock exchange. Figure 11.5 illustrates the concept and the architecture of the Tower Portal. MyTower is an HTML document containing Java applets that inform users about new events and represent this awareness information in the web page. One of these Java applets is responsible for the communication with the ENI server. It administrates the user-defined interest profiles, asks the ENI server for new events, and distributes these events to the appropriate indicators, likewise Java applets. With the modular architecture and the implementation in Java, the Tower Portal is application independent. All indicators can also be integrated in external applications, so that existing systems can be extended with the Tower awareness widgets.

190

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

Presenting Activity Information in an IIS 11 ● ● ● ● ● 1 ●●●●●●●●●●

TOWER portal

Indicator

User login

MyTOWER User registration

Indicator

Communication applet

Indicator Indicator

Credentials

Event query MyTOWER

User authorisation

Events CGI-scripts

ENI server Personnel myTOWER user web page

Figure 11.5 Concept and architecture of the Tower portal.

011

011

011

11

11.6 Awareness in a Working Context: Smartmaps The users asked for simple means to provide awareness. To them awareness information and the possibility to access the respective objects were integral. When connecting to a shared workspace the traces of activities which took place since the last time an actor was connected were requested. In addition, while connected, awareness about current activities was required. Like in real world settings, situated action (Suchman, 1987) requires awareness information about the working space in which action takes place but, in contrast, in electronic workspaces situations may span over time. To meet these requirements, Smartmaps were developed. Smartmaps provide both task-oriented and social awareness (Prinz, 1999), i.e. they yield information about the state of artefacts as well as the presence and activities of people. Furthermore they provide a twodimensional layout of the electronic workspace which may ease the local orientation in the workspace itself. Traces of past or current actions are indicated as “footprints” in the Smartmaps. Smartmaps thus span synchronous and asynchronous work situations. Past actions as well as current actions are indicated in the space. Smartmaps provide an overview of all activities in shared information spaces like shared file systems, web sites, or shared workspaces in 2D graphics. Figure 11.6 shows the Smartmap of a large file system. All files in the tree of folders are represented as small rectangles. The hierarchy of sub-folder becomes visible through the thick lines. Files in the same

191

Inhabited Information Spaces

Figure 11.6 A Smartmap representing the files and user activities in a large file system.

sub-folder are represented close to each other. Some rectangles, i.e. artefacts, are highlighted in a different colour to indicate activity. Smartmaps is a Java applet implementation based on the tree-map visualisation technique (Johnson and Shneiderman, 1991). It shows artefacts of information spaces, preserving a lexicographic ordering. Different actions like read, write, rename, create, delete, etc. in information spaces are reported by the ENI-server as events to the Smartmaps. The events carry, for example, data about producer, artefact name, artefact type and operation. This information is interpreted and indicated in the Smartmaps by colouring the corresponding rectangles. The default presentation mode conveys the overall activity and their distribution in the information space. Tool tips, which are activated when users move the mouse over the corresponding region, indicate the artefact’s name. The tool tips also present the names of the persons currently acting on the artefacts. This enables people to know that there are currently others present and working in the same information space, the same part or even on the same artefact. To explain the application of Smartmaps for the provision of awareness in shared workspaces, we describe the integration of Smartmaps into BSCW shared workspaces (Figure 11.7). Integrating it into another groupware platform, e.g. Lotus Notes, is also possible. When combined with shared workspaces, Smartmaps augment the often-applied list mode presentation of the shared objects with the 2D spatial representation of

192

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Presenting Activity Information in an IIS 11 ● ● ● ● ● 1 ●●●●●●●●●●

011

011

011

011

11

Figure 11.7 A Smartmap showing activity in a BSCW shared workspace.

the complete workspace. When navigating in the shared workspace, the Smartmap highlights the actual position of the user in the overall workspace by an orange rectangle in the Smartmap. Thus, beyond the provision of the supply of awareness information it also eases navigation in the BSCW workspace. Users have the following options to interact with the Smartmaps. When moving the mouse over a Smartmap the complete name of the represented artefact or information about recent user activities is displayed. A mouse click presents the pathname of the artefact in the status bar of the browser window, a shift-mouse click opens the artefact itself, a control-mouse click opens the enclosing “folder”. A control-right-mouse click opens a pop-up menu to open the artefact, the enclosing workspace, and further enclosing workspaces up to the top-level workspace. These interactions enable an activity-based navigation in the information space and reduce the amount of time that is normally needed to navigate hierarchical structures. Several parameters configure the visualisation. The colour for highlighting can be chosen, the highlighting intensity can be specified, only artefacts, or artefacts and enclosing “folders” up to a configurable level can be highlighted with decreasing intensities. Furthermore the granularity of the visualisation is adjustable: complete (default); only enclosing “folders”; only the first x-levels of the hierarchy; and up to the last y-levels of the hierarchy. In addition the duration of highlighting can be set. Usage experiences have shown that a length of three minutes for Smartmaps representing busy web sites, and 4 hours for shared workspaces and shared file systems is a good duration. An overview of the activities that happened during the last day can be achieved by selecting a duration of 24 hours.

193

Inhabited Information Spaces

We have integrated the Smartmaps applet with several large project workspaces and we could observe that users quickly apply the Smartmaps to navigate the information space as well as for activity-based navigation. For example, they first check the places for which activity of other users is indicated. They move the mouse to the corresponding highlighted rectangle to see who did which operation on the object. Then, often that link is followed and the corresponding folder or object is opened. The Smartmaps applet in the header of a workspace provides users with an overview of the complete workspace structure, easy navigation, direct access to workspaces and artefacts, and informs about group members carrying out actions, currently or in the past. In fact such awareness-enhanced workspaces become an inhabited place for social encounters and activity-based communication.

11.7 Symbolic Actions in a Context-based 3D Environment All these presentation modes are text-based or use 2D graphics. To come closer to an intuitive way of presenting awareness information the Tower world was developed. It presents awareness information in a 3D world. Actions in the electronic space are visualised by symbolic actions of avatars representing the respective actors. The avatars are located at the representation of the corresponding object. In the project the Tower world was understood as the “Theatre of Work”, i.e. the stage on which the activities are played out that take place in an electronic environment.

11.7.1 The Tower World The Tower world realises context-based presentation of user activities in a multi-user 3D environment. The environment consists of a landscape containing representations of shared working objects such as documents, folders, etc. In the following figures, boxes are used for the representation of these objects. This representation can be very detailed, e.g. each shared document is represented by one object in the 3D world. The construction of such a detailed world can be done automatically using the space module (Gavin et al., 2000). This module allows users to select the type of objects that shall be represented as well as the grouping and arrangement of objects based on their semantics. That is, objects can be grouped based on their location in a shared folder, or on other attributes such as keywords, owner, modification dates, etc. More abstract worlds represent only contexts such as folders or an aggregation of folders in a work package or task context. Figures 11.8 and 11.9 show examples for detailed and abstract worlds. Our experiences have shown that most users prefer a more abstract representation, since this allows a better overview.

194

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Presenting Activity Information in an IIS 11 ● ● ● ● ● 1 ●●●●●●●●●●

011

011 Figure 11.8 An abstract overview showing different project contexts.

011

011

11

Figure 11.9 A detailed Tower world representing individual documents.

195

Inhabited Information Spaces

The Tower world is populated with avatars that represent users and their actions in a symbolic way (McGrath, 1998). This is realised by automatically moving avatars to places in the 3D world that correspond to the working context or the document the user is currently working on. Symbolic gestures of the avatar represent the current action of the user. For example, a read operation is visualised by an avatar reading a newspaper, while an avatar indicates a write operation with a large typewriter. The automatic placement of the avatar and the symbolic actions are controlled by the space and symbolic acting module of the Tower system (see the architecture in Figure 11.3). Team members are represented in Tower only when they are working on shared documents or when they perform activities in a public space. This distinguishes the Tower approach from video-based media spaces, where users are visible independently of their current working context. The exaggerated presentation of user activities by symbolic actions provides a good overview on the activities in a world and the overall situation, also from a distance. Users can easily recognise which documents, tasks, or colleagues currently perform related processes. Users whose avatars are close to each other in the Tower world are also working in a similar working context. Communication channels such as audio or chat are provided to enable a spontaneous conversation. Thus, the Tower world could serve as a chance encounter and context-based meeting space that facilitates coincidental meetings. For a peripheral awareness on user activities, the Tower world can be projected in the user’s office environment. Alternatively, it can also be included as a plug-in in the user’s web browser (Figure 11.10). Camera agents offer a guided tour inside Tower world. The agents are configurable and provide a personalised entry to Tower world and the represented events. A history camera agent guides the visitors to places at which in the past interesting events have taken place.

11.7.2 User Feedback User feedback gave evidence that the easy orientation in the world representing a workspace is of high relevance. Therefore, simplified views had to be provided. Now several worlds representing different levels of detail are available. One overview world displaying the major folders in a workspace and in parallel a detailed world with all the documents is provided. An experience with using the Tower world as well as feedback from users indicated that specific navigation support is needed. Users want to navigate to points of interests or to points of activities to take a more detailed look. This is an interesting finding, since at the beginning of the project, it was considered to be beneficial, that in contrast to other 3D worlds, we “let the system do the walking”. That implies that the world was intended to be a display medium, but not

196

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Presenting Activity Information in an IIS 11 ● ● ● ● ● 1 ●●●●●●●●●●

011

011

011

011

11

Figure 11.10 Tower world as an integral part of the web browser.

for navigation. This is still true for the control of the avatars that represent the actions of its corresponding user. But in the position of a visitor, users considered it necessary to have easy means available to select suitable viewpoints. To this end, users requested that the positions of avatars were clearly indicated. The position of an avatar indicates a location of interest, that is, a location of activity. Therefore, users requested large avatars relative to the size of the work objects. Furthermore, the feedback from various user groups pointed out that the appearance of avatars may depend on the formality of the social context. Workshop participants suggested starting with simple representations. Nevertheless, the avatars should be complex enough to be recognised and personalised to some degree. Participants demanded that it be easy to identify the person represented by the avatar. Furthermore, the avatars should look friendly. Among the participants, there was a clear preference for stylised avatars as opposed to naturalistic ones. Participants proposed to use team colours for the avatars’ clothing. To satisfy these needs, Tower provides a tool where

197

Inhabited Information Spaces

users can select among six different avatars and change colours of hair, eyes and dress and thus shape their personal avatar. Another interesting finding was that the users wanted to immediately access the workspace objects represented in the Tower world, although we intended the world in the first instance only as a visualisation and not as an interaction medium. All this gives a clear indication of the need to integrate the supply of awareness tightly with the action options of an actor. Actually, the users are actors and not spectators. They want to interact through navigation but also through interaction with the presented objects, just like they would interact in a 3D game environment. We can conclude that like in real worlds, awareness is an integral part of action planning (Lantermann, 1980; Leont’ew, 1977). The various user groups proposed to use the Tower world mainly for places with high levels of activity. For example, one user group suggested using it in a public space to show the activities on the web portal they provide. Another group suggested displaying the Tower world in their coffee room to show the overall team activity in their office. However, in the usual team setting the level of activity was too low to benefit from the synchronous presentation of awareness in the Tower world. The same finding also occurred for the usage of video spaces to support ongoing team work (Fussell et al., 2000). Instead, the support of medium and longterm teamwork requires that awareness spans activities over time. To meet these needs, we tried to replay the recorded event data over a given time period. However, the pure temporal order of events did not convey the right information about the activities in the electronic workspace either. Instead, more complex processing of event data is required. A possible solution for this requirement is presented in the next section.

11.8 DocuDrama DocuDrama aims at recording the history of desktop events and activities generated by a project team in the Tower collaborative work environment. Avatars that enact the events as they occurred in the shared workspace visualise the replay of the team’s interactions with documents. DocuDrama Conversation, a DocuDrama approach that focuses on interaction between people on documents, refines the idea of the history replay. DocuDrama Conversation rearranges the order of events for the history replay instead of a play-out strictly organised by time. This approach enables the user to focus on activities that have taken place on a certain document. For example, a team member uploads a document in the team’s workspace and emails the project team about its existence and location. As a follow-up activity, other team members will open the document and read, change or annotate it. DocuDrama Conversation groups all these activities by different team members into a single scene. The replay of the events in that scene is performed in the order of their

198

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

Presenting Activity Information in an IIS 11 ● ● ● ● ● 1 ●●●●●●●●●●

occurrence although the scene is not thematically interrupted by concurrent activities on a different topic. In DocuDrama Conversation, avatars and their symbolic actions are at the centre of interest. The avatars look and turn towards each other to enhance the impression of an ongoing conversation. To enrich the story and to keep the user’s attention, special focus has been given to camera navigation and positioning. At the beginning of a scene, the camera approaches the centre of activity, the box of the current document, and remains in an overview position. The avatars appear one after the other and perform their symbolic actions. The camera chooses randomly between a variety of close-up views on the avatars. Figure 11.11 shows an example of possible camera positions and views in DocuDrama Conversation. The position of the avatars is dynamically determined. In scenes with only two or three avatars the avatars are grouped facing each other. If there is a larger number of actors involved, the avatars are grouped in circles on top of the document boxes. Figure 11.12 shows an arrangement of avatars in circles to give the impression of a conversation. To start DocuDrama users define a timeframe, which, by default, includes all past events. The user chooses a timeframe of the past, a day or a week, and defines subjects, authors or activities, which are relevant to him. Figure 11.13 shows the user interface of DocuDrama Conversation. The replay of events takes place in a single-user version of Tower world. Text fields in the middle section give information about user actions currently played out by avatars, the document as the centre of activity and about the period in time, in which activities on this document have taken place. The section below includes the elements for the user’s personal configuration of DocuDrama.

011

011

11

Figure 11.11 Camera positions.

199

Inhabited Information Spaces

Figure 11.12 Avatars positioned in circles.

Users of DocuDrama Conversations reported that the history replay enabled a better understanding of the process of events and their coherence, compared to a sequence of event reports of the same timeframe. The choice of selection criteria has proven to be useful and will be refined and extended in a future version of DocuDrama.

Figure 11.13 DocuDrama user interface.

200

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Presenting Activity Information in an IIS 11 ● ● ● ● ● 1 ●●●●●●●●●●

11.9 Ambient Interfaces Ambient interfaces support presenting and capturing information beyond the PC. They go beyond the classical PC with the traditional hardware and the desktop metaphor. Ambient interfaces include the whole environment of the user as a medium for the interaction between the user and the system. The subtle presentation of information exploits people’s ability to be peripherally aware of others, thus enabling users to be aware of activities without disturbance. Furthermore, ambient interfaces allow the user to capture information permanently – even if they are not working with their PC (Gross, 2002). Ishii et al. at the MIT Media Lab developed some early systems (e.g., ambientROOM). However, these systems were called ambient displays and primarily focused on presenting information and not on capturing information (Wisneski et al., 1998). We explicitly use the term ambient interfaces to denote systems that use the physical environment of the user to present information and to capture events. For the Tower environment we developed two types of ambient interfaces: binary ambient interfaces, and AwareBots. The technological bases for binary ambient interfaces are relayboards, which can be connected to the parallel or serial port of the PC. A special client controls the individual relays. Examples of binary ambient interfaces that we developed are a fish tank with plastic fish (Figure 11.14a), a fan, a coffee machine, and so forth. The system can release bubbles into the fish tank, which can be seen and heard by the users. The different binary ambient interfaces can be configured in various ways. For instance, releasing bubbles can indicate that the user’s web pages are accessed; the fan can blow air into the user’s face when a document is uploaded into the shared workspace; and the coffee machine can be switched on when the user logs into the Tower system from within the office. AwareBots are ambient interfaces that have the shape of robots. The Tower AwareBots were built based on the LEGO Mindstorms Robotics Invention System. The first generation of AwareBots was RoboDeNiro (Figure 11.14b). RoboDeNiro uses two motors for the presentation of information and a touch sensor for capturing information. It can lift its hat, rotate its torso, and senses when its arm is pressed. RoboDeNiro was used both in individual users’ offices and in a public space. The following configuration was chosen: RoboDeNiro can wait for a specific user, when this user logs in RoboDeNiro lifts its hat; when changes in a specific shared workspace occur, RoboDeNiro rotates its body; when its arm is pressed, it sends a specific message to the ticker tape. On the whole, the user reactions were positive and the users could understand the semantics of the individual movements of RoboDeNiro. However, more and more users stated that they would like to personalise their robot or build their own robot.

201

Inhabited Information Spaces

a

b

c d Figure 11.14 (a) Fish tank; (b) RoboDeNiro; (c) EyeBot; (d) ClassificatorBot.

Therefore, we distributed LEGO Mindstorm packages to the users and asked them to build their own AwareBots. This resulted in the second generation of AwareBots. One user built the EyeBot, which can roll its eyes and has a switch with a flag that can be moved to the front and to the back (Figure 11.14c). Whenever changes occur in the shared workspace, the EyeBot rolls its eyes. The switch can indicate the user’s availability. Another user built the ClassificatorBot – a mobile robot that can rotate to the left and to the right (Figure 11.14d). The ClassificatorBot can be used to indicate any kind of relationship (e.g., it was used to indicate access to web pages; on an access to the personal pages of the user the ClassificatorBot turns to the left, on access to other web pages of the research group it turns to the right).

202

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Presenting Activity Information in an IIS 11 ● ● ● ● ● 1 ●●●●●●●●●●

On the whole, both the binary ambient interfaces and the AwareBots allow the subtle presentation of up-to-the-moment information and events as well as capturing information over time. Furthermore, several AwareBots remain in a certain position and can indicate status information that the users can capture later (e.g., when they return to their office). The users of the ambient interfaces can not only design and construct their ambient interface, but they can also specify their individual mapping of the TOWER events to the symbols of the ambient interfaces – they do not need to use the default mapping. For instance, a user can specify which action the pressing of the arm of the RoboDeNiro triggers. Another user might want to have a login or logout event triggered, because she imagines that pressing the arm symbolises a greeting of saying hello or goodbye; another user might want to trigger a message to the ticker tape, asking if some colleagues are in the same building and would like to have a short coffee break together. Another example is rotating the body: the default mapping is that rotating the body shows changes in a specific shared workspace. Some users preferred the position of the torso to represent the number of new emails in the inbox: for each new email, RoboDeNiro rotates the torso a little more (some users even draw a scale, so they could see exactly at which position the torso was standing). For the public places, such as a coffee room, the default settings were kept in order to avoid confusion. Users of personal ambient interfaces also reported that they used a personal mapping for privacy reasons: when other persons were in their office, the guests could see the changes in the ambient interfaces, but they could not tell what the mapping was. Therefore, only the respective owner could interpret the mapping.

11.10 Lessons Learned About Awareness The demonstrations of the Tower features and the discussions with the different groups of users have shown that the needs for awareness depend on many different factors. Specifically, we found differences between the needs of tele-workers compared with those of collocated team members or those of members of distributed teams. Another interesting finding was that not only the events that occurred were of interest but also those that were expected but did not occur in time. For example, if a deadline in a project approaches and a requested contribution does not arrive in time, users wanted to be notified about those outstanding events.

11.10.1 Awareness Is Something One Is Not Aware of All discussions with users in the various settings, however, have confirmed that awareness is something they are not aware of. Instead of

203

Inhabited Information Spaces

permanently watching awareness information, they are performing task-related activities. “Seeking awareness information is an out-of-task activity”, a user said. Any extra effort is considered to be troublesome. Furthermore, the team observation disclosed that – like in real world settings – all available information was read as conveying awareness information almost unconsciously. For example, the stream of emails in the project was interpreted as indicating the level of activity. It conveyed presence information about the senders of the mails. Information about the topics of relevance was read from the subjects used in the emails. This was combined with the information gained in the BSCW daily activity report to achieve awareness about the team and the work process. In interviews the team members confirmed this intuitive interpretation of email as awareness means. From real world environments one could learn that awareness is one of the most unconscious and intuitive ways of human orientation in a surrounding environment. One is always aware without knowing. It is impossible not to be aware since this would imply not perceiving the environment. Tightly interrelated with perception is interpretation, that is, associating meaning to what one perceives (J. Gibson, 1986). Perception and interpretation are preconditions for being aware also in electronic settings. Accepting the unconsciousness of awareness it is not surprising that requirements analysis for electronic support of awareness using verbal discussions, interviews and so on, did not give a complete picture. The availability of tangible interfaces in Tower often disclosed new aspects. They helped to improve the understanding of awareness needs in electronic environments considerably.

11.10.2 Synchronicity of Awareness Purely synchronous supply of awareness information turned out to be much less useful than expected (see also Fussell et al., 2000). In particular it was not suitable for the work of distributed teams as the team observation disclosed. Also the fixed time interval of the daily BSCW activity report that is compiled and distributed every night was not suitable in all situations. Our team observations revealed changing interaction rhythms depending on the actual work processes. For example, in case of an approaching deadline and in case of joint document production, a high level of awareness was requested to support co-orientation in the team. While in the phase of programming, continuous mutual awareness was irrelevant but immediate reactions of other partners were requested once a problem occurred and immediate help from others was needed. The provision of awareness information should acknowledge these rhythms and also changes of the rhythm. Thus in phases with more activities, awareness information may be provided more often than in phases

204

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Presenting Activity Information in an IIS 11 ● ● ● ● ● 1 ●●●●●●●●●●

with low level of activities. Actually, real world places also have their own rhythm and tempo, depending on the surrounding culture (Levine, 1997). Simultaneous awareness is required in situations where one wants to establish a synchronous contact. To this end awareness about people and their actual availability is required. Orientation in the work process requires asynchronous awareness. This implies compiled stories about time periods.

11.10.3 Walking and Talking Are Means to Achieve Awareness In order to understand particularities of asynchronous awareness better, we asked team members how they achieve awareness in their local real world settings after a time of absence. “I gain awareness from going for a walk”, an interviewee said. Walking around and talking to colleagues were named by all interviewees as an important means to gain awareness. After some time of absence they ask colleagues about what happened in the meantime: “I like to listen by first getting the current state of things, what is going well, what is going badly, are there any hitches, and effects of hitches on the overall program.” Length of time for discussion is for most of them about 5 minutes. This gives clear evidence that summaries are requested, rather than detailed lists of all events. The feedback from interviewees explains why the pure presentation of events in the temporal order failed. Instead the challenge is to process the stream of events as recorded in Tower into meaningful stories about what happened over time. It also explains why the basic motto of the project, “Let the system do the walking”, did not comprise the whole story. Instead, the environment should “talk” to the users. In effect, this also means getting summaries and an overview kind of awareness stories. In case a user wants to get more detail, they want to be able to “walk around” in the electronic environment and ask the environment for more awareness information, i.e. to retrieve awareness information on request. The Docudrama development is a first step towards this direction, but further research is needed.

11.10.4 Peripheral Awareness in Electronic Settings With Tower we wanted to provide means for peripheral awareness similar to real environments. Instead of permanently watching events in the workspace, awareness should be immediately at hand, when needed and it must be adjusted to temporal fragmentation. For presence awareness this implies that a table of people present must be accessible with the level of temporal accuracy that is requested for a personal contact. For workspace awareness this implies that the environment provides on request the story of intermediate events for the location a user is

205

Inhabited Information Spaces

interested in. Peripheral awareness in electronic environments means immediate situative provision. Also in these cases the awareness information spans over time, that is, comprises awareness information over a time interval.

11.10.5 Awareness Is Double-situated: The Workspace’s and the Observer’s Situation Instead of constant need for the same kind of awareness information, the relevance of such information depends on two situations, the situation in the actual shared electronic workspace and the current situation of an observer, respectively a potential actor. For example, to keep all members of an electronic setting informed about the progress of work, a daily activity report that is sent out every day to all members has proven to be helpful. All team members in Tower found the BSCW daily activity report useful. “I look at BSCW daily activity reports everyday, it gives you information about who is doing what”. However, after an absence of several days, reading all intermediate reports does not suit the user needs. Instead, this user situation requires summaries of all intermediate events. The environment should compile a story from those events that took place since the actor’s last presence in the environment. This implies that instead of receiving a report giving the awareness information in fixed intervals of time, users want them to be compiled dynamically according to their individual observation rhythm. On the other hand, when something unusual happens and the progress of work in the workspace needs the actor’s attention, awareness information must be pushed – e.g. by email or by activating an ambient interface in the local work setting – to the respective actor. Thus, the awareness support in a workspace must keep track of what happens in the environment to provide the respective information to its actors. The stories compiled should be adjusted to the interaction rhythm of the environment as well as to the observation rhythms of its actors. We can, for example, differentiate the following potential situations of an actor who joins a workspace: taking a break and looking around; finishing a document under time pressure; supervising a project’s progress; searching for a particular document or for information on a particular topic; or a newbie may want to find out about places of relevance and about the history of the project. All these situations differ with respect to the awareness needed. The awareness information supplied must be dynamically adapted to these user situations. Instead of configuring tools, a user may be enabled to choose among the most suitable means of presenting awareness information, just like the choice of features offered by Tower. Furthermore, the environment should be able to detect as far as possible the user’s

206

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Presenting Activity Information in an IIS 11 ● ● ● ● ● 1 ●●●●●●●●●●

situation (by means of the events it has recorded from the user and by means of the presentation tool selected) and to adapt the provision of awareness information respectively. This requires more research to investigate concepts of adaptability to the situation of the shared environment, the actual situation of the user and the group. Mechanisms to compile events into meaningful episodes and stories need further investigations. Similarly, the requirements of synchronicity for the provision of awareness have to be studied in more detail.

011

011

011

011

11

11.11 Summary and Conclusion This chapter described an awareness environment for co-operative activities that is based on inhabited information spaces. It provides a number of user-configurable indicators. The 3D environment creates a shared space presenting users and their working context as an inhabited information space. The portal summarises all notifications in an overview presentation. We have described Smartmaps as a compact visualisation for an inhabited environment. They can be easily integrated into the information space itself to build social spaces. Furthermore, we presented ambient interfaces. The complexity of the Tower system became evident upon each demonstration and workshop and so does the complexity of its usability. Many items arose which could have not been foreseen until such a tool really existed. For example, the relationship of size between avatars and documents, the suitable level of detail with respect to documents and folders, a suitable mapping which facilitates navigation in the file system and the 3D world in parallel, the navigation in the 3D world to find points of interest had to be revised. In particular the relevance of DocuDrama became evident and many new requirements could be raised. Discussions with the different groups of users have shown that the needs for awareness depend on the different settings, modes of working and the particularities of the co-operation cultures. Awareness may contribute to project management, to knowledge management – e.g. for finding documents by patterns of their usage – and provide chance encounters in non-collocated teams. However, all discussions with users in the various settings have confirmed that awareness is a phenomenon of which one is not aware. Therefore any extra effort required to the provision of awareness is considered to be troublesome. Instead, awareness notifications must be compiled into meaningful stories that span over time. Details must be available on request. The presentation of activity notifications must fit with the changing local work situation of the particular observer. The availability of medium or long-term history of awareness may be useful in many situations. All this requires sophisticated mechanisms to aggregate and select the relevant data.

207

Inhabited Information Spaces

The effects of awareness support became evident when an interviewee said that he now worries “how I may appear to others, when they see what I am doing”. This is an aspect that requires further attention. Awareness is not only a means to see what others are doing. Instead, it will become a means for presenting oneself to a distributed team.

Acknowledgments The research presented here was carried out by the IST-10846 project TOWER, partly funded by the EC. We would like to thank all our colleagues from the TOWER team at Aixonix, Atkins, blaxxun, BTexact, Fraunhofer FIT, and UCL for their co-operation.

208

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

Part 5 Construction

011

011

011

11

209

This page intentionally left blank

011

011

011

011

11

DIVE: A Programming Architecture for the Prototyping of IIS

12

Emmanuel Frécon

12.1 Introduction This chapter presents the Distributed Interactive Virtual Environment (DIVE) system. DIVE realises an architecture and a programming toolkit for the implementation and deployment of wide-area, Internet-based multi-user virtual environments. The platform is designed to scale to both a large number of simultaneous participants and a large number of objects populating the environments, while ensuring maximum interaction at each site. Started as a lab tool in 1991, DIVE has now reached a stability and state that allows its use outside the research niche that it has occupied since its birth. DIVE can be compiled for a number of UNIX and Windows platforms. It benefits directly from the recent advances in three-dimensional (3D) graphics hardware that have moved 3D capabilities from professional workstations into everyday home personal computers (PCs) and probably soon to mobile terminals (see ATI, 2003; and Intel, 2003). In this chapter, we present the DIVE system from the perspective of Inhabited Information Spaces (IIS). Our intention is to describe an example system in order to give a better understanding of the technological issues that are involved by the realisation of large-scale shared interactive 3D spaces: networking solutions to keep interaction high and ensure the illusion of a shared space, various interaction methods for input and output (including live audio and video communication), support for large spaces both in extent and detail, openness of the platform towards the outside world, etc. Our focus on programming interfaces to the system is justified by the very eclectic nature of IIS and the recurrent necessity to couple the space with a database. At many different levels, DIVE tries to ensure an adequate trade-off between complexity for the programmer and capability. For example, the

211

Inhabited Information Spaces

different programming interfaces that it offers will allow the programmer to pick the interface and language that are most appropriate for the tasks at hand. However, offering such a palette of interfaces lends to complexity in mastering all or some of the interfaces and integrating all the components forming an application together. Similarly, DIVE provides a number of building blocks to interface its communication architecture. Assembling these blocks in various ways will let the application developer experiment with different interest management techniques at the cost of more programming complexity compared to other systems. However, the “standard” DIVE-based application instance will generally be developed without the need to focus on such complex problems. The remainder of this chapter is structured as follows. In the next section, we describe the conceptual abstraction through which DIVE applications transparently communicate with each other, namely via virtual worlds. We then focus on the wide-range of programming interfaces and techniques and describe their use for IIS. Following this, we present the run-time architecture and a summary of the main modules composing the system. Finally, we examine a specific application in more detail to hook its architecture and design choices into the various key techniques presented in the rest of this chapter.

12.2 The Virtual World as a Common Interaction Medium DIVE provides both an architecture and a programming model for the implementation of multi-user interactive virtual environments over the Internet. The architecture focuses on software and networking solutions that enable highly responsive interaction at each participating peer, that is, interaction results are whenever possible immediately shown locally at the interacting peer but slightly postponed at remote peers. The programming model hides networking details, allowing the programmer to focus on the space, its content and its application logic. By peer, we mean an application process running at a specific host (computer) connected to the Internet. An application or process is any active program interfacing to the virtual environment by presenting and modifying entities (see below), monitoring and reacting on different types of events, and so on. A typical application is the 3D browser that handles interaction and visualisation of the environment for a specific user. Other applications may, for instance, perform animations and complex simulations, and present 3D user interfaces for different purposes. A number of IIS examples are presented at the end of this chapter and elsewhere in this book, e.g. the Web Planetarium (see Chapter 2) and the Pond (see Chapter 4). A central feature in the programming architecture of DIVE is the shared distributed world database. All user and application interactions take place through this common medium. The world database acts as a

212

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

DIVE: A Programming Architecture for the Prototyping of IIS 12 ● ● ● ● 1 ●●●●●●●●●●●

world abstraction, since DIVE applications operate solely on the database and do not communicate directly with each other. This technique allows a clean separation between application and network interfaces. Thus, programming will not differ when writing single-user applications or multi-user applications. This model has proven to be successful since DIVE has changed its inter-process communication package a number of times without applications requiring any redesign, only minor code tweaking. A DIVE world is a hierarchical database of what are called entities. DIVE entities can be compared to objects in object-oriented programming, although DIVE is written in plain ANSI C. The database is hierarchical to ease ordering of information and rendering. In addition to graphical information, DIVE entities can contain user-defined data and autonomous behaviour descriptions. We believe that the 3D rendering module should not dictate a structure to the content of the database. By making this distinction the system becomes more flexible, and can more easily be adapted to a large variety of rendering techniques. Entity persistence is ensured as long as one application is connected to a world. When the last application “dies”, the world dies and entities will stop living. The next time an application connects to this world, it will reload its state from some initial description files or a Uniform Resource Location (URL). Currently, active persistence is achieved by running monitoring processes that periodically save the state of the world to files, from which the world database can be restored if the system needs to be restarted.

12.3 Partial, Active Database Replication The DIVE architecture is based on active replication of (parts of) the database, so that a copy resides at each application process (see Figure 12.1). This model allows applications to access the world database directly from memory, which provides low-latency user interaction. Typically, entity additions, removals and modifications are done on the local copy first, then distributed to all connected peers through network messages and applied to the local copy at each receiving peer. By this we mean that the replication of the database is active. Conceptually, programmers can think of a “global” central database residing somewhere on the network, but the database is indeed replicated at each process. To achieve scaleable real-time interaction, DIVE: ● ●

uses a peer-to-peer multicast communication model where partitions of the world are associated to multicast groups; tolerates world copies that differ slightly and implements services that ensure their equality over time. Dead-reckoning techniques and runtime object update mechanisms are used to achieve this;

213

Inhabited Information Spaces

P2

P1 P6 W2

W1 W2

W1

P3 H1

H3

W1

W2

W2

P5

W1

W2

Network

W1

H2

P4

Figure 12.1 The replication architecture of DIVE. A number of processes Px, hosted on several machines Hx are sharing two different virtual worlds W1 and W2, conceptually located “on the network”. Only P3 and P6 share both worlds, all other processes share one or the other world.

●

divides the world into sub-hierarchies that are only replicated and used in-between the small number of applications that have expressed an interest in a particular hierarchy.

Modifications to the database are applied locally first, then transmitted and applied to all receiving copies. Thus, on the receiving side, world events are captured and transcribed into the database some time after they occur. The length of this time is highly dependent on the networks that packets will cross. At all levels, DIVE is tuned to accommodate varying round-trip times and packet losses. Despite network latency, DIVE does not introduce differences between peers that are excessively large. The distribution principle fits with our experience of virtual environments: typically, entities will be modified in bursts and then stabilise into a common “inertia”, that is, a common state that lasts. Differences introduced by latency and lost network packets are “healed” over time by periodic synchronisation using sequence numbers to keep track of entity versions, followed by possible update requests. As a result, with time, all connected peers will achieve the same “inertia” for these entities. DIVE pursues an idea originally in Scaleable Reliable Multicast (SRM) (Floyd et al., 1995) to reduce the amount of message passing and thereby minimise network load and increase scalability. The method uses multicasting heavily, makes communication entity-based, and bases reliability on a negative acknowledgement request/response scheme.

214

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

1 11

DIVE: A Programming Architecture for the Prototyping of IIS 12 ● ● ● ● 1 ●●●●●●●●●●●

To remedy problems with network congestion and traffic overhead, and to offer the possibility for hundreds of participants to share a common place, DIVE provides a mechanism for dividing the world into sub-hierarchies that are only replicated and used in-between the small number of applications that are actually interested in them. This mechanism forms logical and semantic partitions. Each sub-hierarchy is associated with a multicast communication channel, called a lightweight group. As a result, processes that are not interested in these sub-hierarchies can simply ignore this branch of the entity tree and reduce network traffic. The top-most entity, i.e. the world, is itself always associated with a multicast channel, and every world member process must listen to it. More information on lightweight groups can be found in Frécon and Stenius (1998). On top of lightweight groups, DIVE offers an abstraction called “holders”. This abstraction deals with the initialisation of the database branch encapsulated by the group, for example through a URL. Holders can be associated to empty multicast groups, which implies total locality of part of the virtual world and can be used for scenery or other parts of an environment that are guaranteed to never change. Holders, described in more detail in Frécon and Smith (1999) and Frécon et al. (2001), can also be used to drastically reduce network traffic through semantically richer application-driven protocols and the realisation of predictive behaviours.

12.4 Programming the System In this section, we present a summary of the palette of techniques offered to DIVE application programmers. We will see that DIVE offers a wide range of programming interfaces, from a high-level scripting language to low-level C programming. The variety of languages, the openness of the platform and the variety of architectural options (from stand-alone applications to distributed scripts) make DIVE a toolkit of choice for IIS systems. It fits requirements such as advanced rendering techniques, e.g. through specific plugins, and connectivity to external databases, e.g. through its ability to embed application data in the environment and its component-based approach. Above all, DIVE offers a framework for the implementation of applications. Years of research development have led to a number of features that are present in few systems, such as the ability to support a wide range of avatars, spatialised audio communication between connected users, visual environment subjectivity (Jää-Aro and Snowdon, 2001). This makes it a software platform ideally adapted to the prototyping of IIS systems.

215

Inhabited Information Spaces

12.4.1 The DIVE Programming Model DIVE applications perform three distinct steps to exist and interact within the environment. Each of these steps is optional. An application will: ● ● ●

introduce one or more (shared) entities to the environment; register for events on selected parts of the environment, usually the entities that it has added to the environment itself; react to events occurring in the environment through modifications to the database, i.e. modify the environment.

Consider the example of a multi-user whiteboard application such as the one described in Ståhl (1992). Such an application follows the steps above. It introduces a 3D model of a whiteboard together with icons for the different drawing tools available. It listens to user interaction events on the surface and icons and reacts by drawing graphical elements on the surface or changing the drawing tool, e.g. from ellipse to free-hand tool.

12.4.2 Programming Interfaces Given this very general principle, DIVE, written itself in ANSI C, offers a wide range of programming interfaces (see Figure 12.2). Monolithic applications can be implemented by compiling some or all of the DIVE component libraries into a DIVE stand-alone application (or part of another application if necessary). Examples of such applications are earlier versions of VR-VIBE (Benford et al., 1995a) the Web Planetarium (Chapter 2), and The Pond (Chapter 4). DIVE offers interfaces for two major groups of languages: C/C++ and Java. ● ●

The C/C++ interface gives full access to the component libraries of the system. This leads to both power and complexity in some cases. The Java interface translates the DIVE object-based approach into true object orientation. For example, all DIVE entities are mirrored by Java objects that instantiate a class in a class hierarchy that mirrors the DIVE class hierarchy (see Chapter 4 in Pettifer, 1999).

Applications can also be loaded dynamically at run-time. These typically need one or several monolithic applications to exist. There are two major interfaces for dynamic loading of applications: ●

216

DIVE has support for plug-ins written in C or C++. By construction, plug-ins have full access to the components libraries of the system without any restrictions. Plug-ins can be either loaded when a standalone application starts or later on, programmatically or as a result of user actions. The standard 3D browser that comes with the system offers the user the option to select a plug-in from the local file system and load it into the 3D browser.

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

DIVE: A Programming Architecture for the Prototyping of IIS 12 ● ● ● ● 1 ●●●●●●●●●●● ●

More importantly, DIVE offers a high-level programming interface in the form of a scripting language. This interface is based on TCL (Tool Command Language). Any entity in the database can be associated with a script that will describe its behaviour over time. Scripts follow the DIVE programming model: they will register for events and react to events. To this end, the set of standard TCL commands is enhanced with a number of DIVE specific commands for event registration, modification of the environment, etc. This forms DIVE/TCL (see Frécon and Smith, 1999 for more information).

Finally, external applications are allowed to connect to a running standalone application through the DIVE Client Interface (DCI). In this case, the DIVE application acts as a server and the external application as a client (note that the communication that takes place is totally different from the standard DIVE communication mechanisms: it is a pure client–server solution tuned for usage within a machine or on a local network). External applications will be represented in the environment by an entity of a specific type. This entity will operate in the environment on their behalf. The language used here is, again, DIVE/TCL. This interface is tuned for situations where the external application cannot be easily modified but needs some DIVE facilities. Finally, standard implementations of the client interface exist in many languages: C, TCL, Java, Oz, Prolog.

Java inverface cation interface Appli o C re libraries

011

DCI

DIVE/TCL

C/C++ Interface

11

Plugin

011

External application

Figure 12.2 DIVE offers a wide range of programming interfaces. Most of them operate on the hierarchical database as a central abstraction. The Java interface mirrors part of this database in a pure object-oriented manner. The script interface (DIVE/TCL) is part of the database in the sense that scripts are associated to the replicated objects and describe their behaviour in a distributed manner.

217

Inhabited Information Spaces

12.4.3 Building your Application Given the wide palette of programming interfaces and architectures, we discuss now the various ways of writing DIVE applications and the typical applications that these target.

“Monolithic” Programming It is possible to write DIVE applications that apply the programming model in one sweep, i.e. within a single piece of code. There are different ways to implement these types of applications: ● ●

●

C, C++ or Java applications can be programmed and linked together with the DIVE core libraries. It is also possible to build whole applications using the plug-in interface. These will be hosted within a stand-alone application such as above (typically the 3D browser). External applications programmed in any language of choice that has support for TCP/IP and connected to a running DIVE application through the DCI.

Such applications will be integrated to a DIVE world through event registration. This offers transparent access to a networked environment. Monolithic DIVE Programming is suitable for applications such as simulation engines, complex applications or simple renderers. An example of IIS application that uses this model is VR-VIBE (Benford et al., 1995a). VR-VIBE creates visualisations of sets of documents. Users specify keywords that they wish to use to generate the visualisation and place these keywords in 3D-space. Representations of the documents are then displayed in the space according to how relevant each document is to each of the keywords (this relevance is computed by searching the documents for the keywords specified by the user and recording the number of matches). Earlier versions of VR-VIBE were written as a stand-alone application written entirely in C. This is mainly for scalability reasons. Above all, VRVIBE is typically a compute intensive application, e.g. computation of document icons positions upon interaction and initialisation. It postulates the acceptance of networking delays, for example, the time for new document position to arrive at all visualising sites. Along the years, VRVIBE has benefited from the incorporation of DIVE/TCL. Consequently, it has now become a mixed mode application (see below) where the core component is still written in C for scalability and the user interface uses the behaviour language. Another example of IIS application that uses this centralised programming model is the Pond (Chapter 4). The Pond is a multi-user horizontally projected system for access and manipulation of information

218

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

DIVE: A Programming Architecture for the Prototyping of IIS 12 ● ● ● ● 1 ●●●●●●●●●●●

elements from the Internet as well as for communication and collaboration between users being present both remotely and physically. The current demonstrator provides a visualisation of a record database and focuses on touch and sound interfaces, “prohibiting” textual entry through a keyboard. The Pond is mostly implemented as a stand-alone application (in Java), for performance reasons. The logic of the Pond application, e.g. flocking behaviour, Internet search initiated by users, is on purpose separated from the visualisation (a specific DIVE 3D browser, with a number of rendering plug-ins). Yet another application largely following this technique is the Web Planetarium (Chapter 2). The application visualises the structure behind World Wide Web documents and hyperlinks as a 3D virtual world of planet-like abstract objects and connection beams. An object is a 3D representation of a web page and, once the user is inside, displays the hyperlinks on that page as additional small objects on which the user is able to click in order to fetch new pages, and thus extend the 3D graph with new site representations. The Web Planetarium is implemented as a stand-alone application (in C), for performance reasons. As for the Pond above, the logic of the application (parsing of Web pages, placement of documents, interaction with the users) is contained in an application separated from the visualisation. This allows for a number of different visualisation set-ups: standard desktop, multi-screen displays, etc.

Script Programming A radically different way of programming consists of creating a DIVE world (or set of objects) with scripts attached to objects in the database. The TCL layer isolates clients from executable changes to the DIVE, which is of importance as DIVE/TCL has stabilised over the years and is generally kept backward compatible. Of interest to this style of programming is the fact that the standard 3D browser provides a generic actor environment (with audio and visual input and output). On top of this generic environment, there can be any number of TCL/TK-based “skins”, presenting different user interfaces. This environment allows applications that live within the presented DIVE worlds and programmed in the form of scripts to modify the layout of the 2D user interface at run-time, for example to add application specific menus (see Steed et al., 1999 for an example). Script programming is tuned for rapid prototyping, simple applications, interface experiments and animations. Script programming is ideal when the behaviour of the application can be described through a number of somewhat independent visual objects that consume little CPU power. It is particularly suitable to adapt to changing environments of

219

Inhabited Information Spaces

all sorts: adding new objects with new scripts to an environment will make it behave differently. This can even happen at run-time and is therefore very dynamic by nature. An example of an IIS that uses this programming model is the DIVE Room System (Frécon and Avatare-Nöu, 1998). This application actively supports collaborative as well as individual work through the concept of rooms. The application uses a real-life metaphor and introduces virtual counterparts of objects that are usually found in meeting and shared group rooms: overhead projectors, notebooks, screens, documents, etc. Every object in the environment has only a slight dependence on other objects. This design fact has driven the choice towards a scripting programming model. Some objects have to know about one another, for example, placing an overhead on the projector will have the effect of showing it in a bigger format on the screen. To achieve this, the application uses script-to-script communication, which is part of the scripting interface.

Mixed Mode As already alluded to several times already, a key design of the DIVE system is the possible interconnection of all programming components. As such, the system encourages application programmers to mix

Figure 12.3 Several views of the DIVE room system. Main picture: At the forefront is the remote control to allow remote operation of the screen that is currently enlarging the document that has been placed on the overhead projector. Close to the projector is a set of virtual folders containing various documents. Inset: An avatar sitting at the conference table and examining a document.

220

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

DIVE: A Programming Architecture for the Prototyping of IIS 12 ● ● ● ● 1 ●●●●●●●●●●●

programming interfaces in order to best suit the needs of the various components on which they are working. A typical example is the combination of plug-ins for services and DIVE/TCL for application logic. This combination provides a very powerful prototyping process. Plug-ins will register functions with the DIVE/TCL layer and the TCL layer is able to inspect loaded plug-ins or request a plug-in to be loaded/unloaded. Another example is external applications connecting through DCI. Such applications will often extend the entity that represents them in DIVE with some application-specific DIVE/TCL code in order to relieve the burden placed on the socket connection between DIVE and the application. WebPath (Frécon and Smith, 1998) is an example of such an application. Mixed mode offers many possibilities; examples of use are face and body animation, new navigation styles, simulation of crowds, etc. This mixed mode is used in many of the latest IIS based on DIVE, since it combines rapid application development (using the scripting language), a component-based approach (allowing components of the application to be associated to specific programmers) and component reuse (both at the plug-in and scripting level). An example of an IIS that uses this model is the London Traveller application described later on in this chapter (Section 12.6). The application supports travellers by providing an environment where they can explore London, utilise group collaboration facilities, rehearse particular journeys and access tourist information data. To adapt to the amount of data to visualise (3D model over 16 × 10 km of London) the core DIVE rendering engine was modified. Additionally, the model is brought to life using avatar crowd and face animations. For performance reasons, this is implemented using several plug-ins. Finally, the different applications that are embedded within the London model were implemented using a set of objects and scripts, designed and implemented by different programming teams. Another example of IIS that uses this model is the Library Demonstrator application (Mariani and Rodden, 1999). The overall goal of this application is to provide visitors to the library with an alternative search method, that will help in “fuzzy” searches, that is, searches that typically would require the visitor to speak to a librarian to get a grip of what to look for in the library. Rather than replacing the existing computer-based search methods, which work well for the initiated and focused users, the demonstrator provides a complementary means of searching, adapted to the users outside of this category. The core of the library demonstrator is built on top of the Java interface in order to benefit from Java’s ability to interact easily with external databases. A large part of the user interface is written using the DIVE/TCL interface, which makes it easy to experiment with a number of different approaches in order to find a suitable design.

221

Inhabited Information Spaces

Figure 12.4 The main window shows the visualisation part of the library demonstrator. At the centre of the window is a focusing circle. The information for the document contained within that circle is shown in a semi-transparent legend in the top left corner of the visualisation window. To the right are a number of windows to perform queries within the library database.

Remote Dynamic Programming The DIVE/TCL layer supports a remote procedure declaration and activation protocol that allows distributed script execution. Indeed, the dynamic nature of scripting languages and the distributed nature of the DIVE system make possible the real-time extension and modification of existing scripts within existing entities. Building applications in such a context becomes interesting. For example, this allows for per-user customised behaviour, e.g. 2D versus 3D presentation or the ability to extend the user interface of other participants by sharing scripts. Finally, it allows distributed application development where the code of the application can be edited between multiple machines interactively from within the system. There are a number of situations where this model can be useful. For example, the various parts of the London Traveller application dynamically add a menu to the standard 3D browser to improve control over

222

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

DIVE: A Programming Architecture for the Prototyping of IIS 12 ● ● ● ● 1 ●●●●●●●●●●●

the information shown. The code for menu extension is contained in the code of the London Traveller application and brought to all users successively connected. Similarly, later versions of VR-VIBE added an extra window to any person (process) entering the world.

12.5 DIVE as a Component-based Architecture To complete our description of the system, the following sections discuss the main modules that compose the DIVE system and that are offered as part of the standard libraries.

12.5.1 System Components In DIVE, an event system realises the operations and modifications that occur within the database. Consequently, all operations on entities such as 3D transformations will generate events to which applications can react. Additionally, there are spontaneous and user-driven events such as collision between objects or user interaction with input devices. An interesting feature of the event system is its support of high-level application-specific events, enabling applications to define their content and utilisation. This enables several processes composing the same application (or a set of applications) to exchange any kind of information using their own protocol. Most events occurring within the system will generate network updates that completely describe them.

User

011

011

11

Input devices

Output devices Rendering

User interface

Audio and video i/o

MIME

Tcl scripting

Event system

3D i/o Database Distribution SRM mechanisms Network

Figure 12.5 The different modules composing the system and their approximate layering and dependencies between the user and the network.

223

Inhabited Information Spaces

In any application, the content of the database must be initialised. DIVE uses a module that manages several three-dimensional file formats and translates them into the internal data structures that best represent their content. Usually only one peer will load, and parse, a particular file and the resulting entity hierarchy will be distributed to other connected peers through a series of (multicast) updates that describe the resulting entities. This specific mechanism differs from many other systems that rely on being able to access the description files or URLs from all connected peers. DIVE has an embedded scripting language that provides an interface to most of the services of the platform. Scripts register an interest in, and are triggered by, events that occur within the system. They will usually react by modifying the state of the shared database. Moreover, these modifications can lead to other events, which will possibly trigger additional scripts. A series of commands allow the logic of the scripts to gather information from the database and decide on the correct sequence of actions. For example, the simplistic script below would move the associated object 1.0 m upwards at every interaction with the mouse (2D or 3D) or any other interaction device connected. The procedure on_interaction is bound to any interaction event on the object (which identifier is returned by [dive_self]). proc on_interaction {event_type object_id type\origin_id src_id x y z} { dive_move [dive_self] 0.0 1.0 0.0 LOCAL_C } dive_register INTERACTION_SIGNAL DIVE_IA_SELECT\[dive_self] “” on_interaction

12.5.2 User-oriented Components The services described previously are independent of any DIVE application. This section focuses on the different modules present within the 3D browser. The primary display module is the graphical renderer. Traditionally, the rendering module traverses the database hierarchy and draws the scene from the viewpoint of the user. This module has several implementations, on top of various graphical libraries such as Performer, OpenGL, Direct3D or PocketGL. Some versions support a constant frame-rate rendering mode (Steed and Frécon, 1999). DIVE has integrated audio and video facilities. Audio and video streams between participants are distributed using unreliable multicast communication. Audio streams are spatialised so as to build a soundscape, where the perceived output of an audio source is a function of the

224

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

DIVE: A Programming Architecture for the Prototyping of IIS 12 ● ● ● ● 1 ●●●●●●●●●●●

distance to the source, the inter-aural distance and the direction of the source. The audio module supports mono-, stereo- or quadra-phonic audio rendering through speakers or headphones connected to the workstation. Input can be taken from microphones or from audio sample files referenced by a URL. Similarly, the video module takes its input from cameras connected to the workstations or video files referenced by URLs. Video streams can either be presented to remote users in separate windows or onto textures within the rendered environment. Users may also be presented with a two-dimensional interface that offers access to rendering, collaboration and editing facilities. The interface itself is written using the same scripting language as offered by the world database. Consequently, applications can dynamically query and modify the appearance of the 2D interface. For example, the London Traveller application exploits this feature by adding an applicationspecific menu to the standard interface of the DIVE browser (see Section 12.6). Finally, a MIME (Multimedia Internet Mail Extensions) module is provided to better integrate with external resources. It automatically interprets external URLs. For example, an audio stream will be forwarded onto the audio module where it will be mixed into the final soundscape.

12.5.3 The DIVE Run-time Architecture Not all components are present within the different programs of the DIVE run-time architecture: ●

011

●

011

11

●

Session initiation: Sitting directly on top of the networking components, the name server allows other DIVE applications to enter worlds and sub-hierarchies of worlds controlled by holders. The name server listens for requests on a well-known multicast group and is requested once and only once for each world and holder. Upon request, a multicast address is returned and this address will be used for all further communication. Several name servers, tuned to different addresses, can coexist on the Internet, thereby allowing a number of completely separate virtual universes to exist. Supporting architecture: On top of the networking component, the proxy server can interconnect sub-islands with multicast connectivity and/or single local networks. A thorough discussion of the proxy server and an analysis of typical DIVE traffic can be found in Lloyd et al. (2001). Environment evolution: On top of the system components, persistence managers ensure that the content of an environment will continue to exist and evolve even when no user is connected. One persistence manager is responsible for the state of one world, but there can be any number of managers ensuring this very task on the Internet. The

225

Inhabited Information Spaces

AP1

3DB8

NS PS1

3DB7

PS2

3DB6

3DB9

PM1

3DB1

PM2

DIVE hybrid multicast backbone PM3

3DB5

PM4

PS3 PM5

3DB2 PS4

3DB4

PS5

3DB3

AP2

Figure 12.6 An example of a running DIVE session with all involved processes, represented by discs of different grey tones for each class of process. Direct connections between processes are represented with a joining straight line; otherwise connection is through IP multicast. The DIVE application-level multicast backbone is formed by a number of proxy servers Px. The name server, NS, is used by all 3D browsers, 3DBx, for session initiation. Persistence managers, PMx, whose existence is sometimes controlled by monitoring applications APx, ensure that the environments will continue living even when a 3D browser is no longer connected.

●

managers guarantee persistence with evolution for environments where the application logic is described using DIVE/TCL. Session participation: Finally, the 3D browser uses most components to give its user a presence within the environment. It introduces a new entity called an actor to the shared environment. This is the virtual representation of the real user. Additionally, it handles interaction with entities, and allows the user to move freely (or in a constrained way) within the environment, listen to all audio sources and talk through the mouth of the avatar. DIVE supports a number of input and output devices: from 3D trackers to standard mice, from multiscreen displays (Steed et al., 2001) to standard workstation screens and even personal digital assistants (PDAs).

12.6 The London Demonstrator: An Example Application in More Detail In this section we describe the London Demonstrator; our description focuses on the programming aspects of the application itself and how these relate to the core services offered by the DIVE system. A better description of the application and implementation choices can be found in Steed et al., (1999). The broad aim of the demonstrator is to present an application to a group of users enabling them to specify and rehearse a meeting of any sort. This includes supporting the selection of features at a given location (hotels, conference venues, etc.), using both abstract and facsimilebased information visualisation approaches. The demonstrator provides

226

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

DIVE: A Programming Architecture for the Prototyping of IIS 12 ● ● ● ● 1 ●●●●●●●●●●●

users with the ability to navigate through a large virtual cityscape (representing a real location). Their navigation is aided by a number of dynamic information visualisation systems. A suite of collaborative features aids the users in constructing, rehearsing and participating in both virtual and real meetings. The demonstrator consists of four main geometric and functional layers. All these services are integrated into a single coherent environment. ● ● ● ●

A 16 × 10 km geometric model of the centre of London. Collaboration services for use by groups. Tourist information data visualisation service. Simulations of public transport and crowds.

011

011

011

11

Figure 12.7 A view of the enhanced 3D browser window that was developed for the London demonstrator. At the top of the figure the application menu that is built by the environment for each new user that connects to it is currently open. To the left are, from top to bottom, a list of view points to quickly navigate and jump to named points within the environment and a list of all present users with a real-time summary of their activities (here: one user, stand still and talking). At the bottom of the figure is text chat window.

227

Inhabited Information Spaces

Any user entering the London world is automatically supplied with an application menu that adds itself to the standard menus of the 3D browser. The menu offers a number of application-specific options such as toggling the visibility of a personal compass or a global orientation map. To achieve the modification of the menu bar, a script associated to the object of the world with no geometrical representation watches for the events generated when new users enter the world. The script sends the combination of DIVE/TCL and TK code required to generate the menu to the entering actor. Additionally, this object adds the geometry for the compass and the map to the visor, a logical object close to the eyes of the avatars. Both these objects have associated scripts that control their behaviour.

12.6.1 Centre of London The model of London is based on two different types of buildings. The vast majority of the model is based on automated extrusions of the contours of the buildings, according to their height. The appearance is then controlled by a number of heuristics (texturing, shaping, etc.) to give an appropriate illusion. Additionally, a number of buildings are modelled at a higher level of detail, i.e. external and internal architecture and furnishings. London is divided into 16 × 10 tiles of one square kilometre each. Every tile is composed of an object whose representation is an invisible sphere. This object is associated with a script that listens to collision events in order to react to avatar presence within the sphere. Upon collision, the script will automatically toggle the loading or unloading of the content of the tile into or from the local database. This arrangement allows keeping in memory only the parts of the static model that are within the vicinity of the avatar. The application specific menu offers a 2D graphical user interface to manually toggle the loading or unloading of tiles for machines with sufficient memory. Within the conference centre and its detailed buildings, a number of scripted objects represented by invisible boundaries (boxes for most of them) toggle the local visibility of judicious parts of the model. For example, internal architecture is made progressively more visible as a user approaches and enters a building and its rooms. The implementation makes use of the collision and subjective views mechanisms that are built in the DIVE platform. Around the conference centre there are a number of invisible volumes that embrace the shape of the streets. These volumes will react to collision with avatars by sending the name of the street to an information text object that has been placed on the visor of the avatar when the user first entered the environment. The visor is a logical object that is always placed in front of the current eye through which the avatar is looking into the environment.

228

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

DIVE: A Programming Architecture for the Prototyping of IIS 12 ● ● ● ● 1 ●●●●●●●●●●●

12.6.2 Collaboration Services for Use by Groups Within the conference centre, a number of rooms are furnished with virtual counterparts of the different items that can generally be found in meeting rooms: notebooks, hands-outs, overhead projectors, etc. These rooms are furnished with all the elements that form the DIVE Room System described in more detail in Frécon and Avatare-Nöu (1998). The room system is based on a number of scripted objects that understand one another when necessary. For example, when placed on top of an overhead projector a virtual document will present its content to the nearest screen. This type of action is implemented through communication between the scripts associated with each object. Additionally, there are a number of invisible boundaries to the rooms. Every conference room is associated with specific multicast groups for audio and event communication within it. The content of the rooms is not present in the database of every peer. Loading of database content, subscription to multicast groups and other related operations is controlled by the scripts that are associated with the room boundaries. This arrangement provides for scalability in the usage of the rooms and the number of users that the overall application can host.

12.6.3 Tourist Information Data Visualisation Service The DIVE city visualisation tool has been developed to help users select places of interest according to predefined requirements. To enable this, the tool retains a database of attributes (e.g. hotel prices, star ratings, etc.) for each attraction type and provides for a 3D interactive visualisation of the data above the city. The visualisation service is programmed using a number of DIVE/TCL scripts. It reuses some of the techniques and components described above. For example, an invisible scripted boundary encloses the whole visualisation space so that all users within its vicinity will be able to talk to one another without sound attenuation. By default DIVE uses a model of sound based on distance and inter-aural distance, where the perceivable sound level diminishes with distance. However, this is undesirable in the demonstrator where collaboration is required over large distances: the attenuation would prevent users from collaborating and talking to one another about the data being visualised. A controlling window is brought up when a user interacts with the visualisation cube. The scripting code for its construction and operation is contained in the visualisation script and dynamically sent at the very moment of interaction. Interaction with this window will let users select hotels, bars and other attractions from a number of preferences. Iconic representations for the attractions are dynamically moved within the visualisation cube when parameters are changed. This operation is very

229

Inhabited Information Spaces

Figure 12.8 A view of the main conference site. In the top right corner, a view of the tourist information data visualisation service and its controlling 2D window which is brought to users by a simple click. In the top left corner, a view over the Canary Wharf. In the bottom left corner, a view of one of the interiors of the conference centre.

dynamic by nature and can lead to the movement of a large number of 3D objects in real-time. The implementation uses aggregation of 3D transformations to relieve the burden on network throughput and ease real-time transmission of visualisation modifications at all sites.

12.6.4 Real-time Simulations The demonstrator includes three real-time simulations that enhance the travel scenario. The first is a simulation of journey on the London Underground that arrives at a station close to the conference centre. The second is an audience in a seminar room in order to enable talk rehearsals or support for those that suffer from a fear of public speaking (see Slater et al., 1999 for more information). Finally, the third simulation component is a virtual crowd of avatars. The crowd simulation was designed with scalability in mind. The crowd itself is controlled through a plug-in implemented with the DIVE C interface. This plug-in, hosted within one and only one process only, controls the position of the avatars that are present in the crowd. Animation of limbs is performed locally at all processes that have an interest in the crowd. This animation is implemented using a DIVE/TCL script associated to the main crowd avatar objects. Animation information is transmitted at a slow rate to give other processes a chance to catch up and avoid swamping the network with animation messages.

230

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

DIVE: A Programming Architecture for the Prototyping of IIS 12 ● ● ● ● 1 ●●●●●●●●●●●

12.7 Conclusion and Future Work In this chapter, we have presented DIVE and focused on its communication architecture and the programming models and interfaces that it offers. We have seen that the various programming interfaces are of importance to many IIS systems. This variety and the composite aspects that are supported by DIVE allow application writers to focus on performance at crucial points only and let them experiment with different interfaces by reducing the cost of development through the introduction of a scripting language. We believe that a component-based approach that offers a number of interconnected programming interfaces is key to the development of an IIS application. It brings modern application development techniques into a novel domain and lets people experiment with an increased number of varying interfaces and test these on final users before the finalisation of the application. DIVE addresses this approach and adds a scalable communication architecture that allows for real-life Internet trials and deployment. For the last few years and for a number of years to come, work within the system is heading in two distinct but complementary directions. First, we have engaged in a finer componentisation of the system, through modularisation and the widespread use of plug-ins. Our goal is to reduce the system to a minimalist kernel on top of which a number of key components are built and through which they interface and communicate. This work is similar in some respects to platforms such as Bamboo (Watsen and Zyda, 1998), JADE (Oliveira et al., 2000) or NPSNET V (Kapolka et al., 2002). Secondly, we are slowly moving towards a number of modules that will allow the system to serve as the core for applications that require realtime interaction, that are distributed over the Internet, but that are not necessarily three-dimensional by nature. Part of this work consists in being able to support different communication models and toning down the multicast orientation of the system and making multicast only one of the possible communication architectures available. For example, we are currently looking into the latest development on peer-to-peer technologies such as CAN networks (Ratnasamy et al., 2001), SCRIBE (Castro et al., 2002) and the work derived from OceanStore (Chen et al., 2002).

011

11

231

This page intentionally left blank

011

011

011

011

11

13

Communication Infrastructures for Inhabited Information Spaces David Roberts

13.1 Introduction Inhabited Information Spaces (IIS) require advanced communication infrastructures that address issues arising from the use of limited computational and network resources to place people within an interactive information space. This chapter describes these issues along with the ways in which they are typically addressed. IIS situate people in a social and information context where they can interact with each other and with the information itself. These users, possibly in remote geographical locations, access the environment through a variety of display devices through which they gain distinct levels of presence and immersion (Slater et al., 2001). Some may be co-located, seeing each other in the real world while immersed in the virtual environment, while others may be in some geographically remote location and represented locally as an avatar, a 3D graphic character capable of representing human-like communication, appearance, identity and activity. Information presented to the user may be shared or private, objective or subjective. It may be abstracted from live data in the real world or from simulation. Users may interact with the information to adapt its presentation, content or behaviour. Information objects often provide a focus for group activity (Greenhalgh and Benford, 1999). IIS merge the real and the virtual. Ideally the latter should possess the richness and naturalness of the former. We would like to be able to interact with remote users as if they were standing next to us. Verbal and non-verbal communication and the use of objects in the environment are primary methods of social human communication in the real world (Burgoon et al., 1994). In IIS, shared interactive information objects may be observed, used to inform, explain, teach, heal, experiment or as a basis for discussion. Sometimes it is important to see not only what

233

Inhabited Information Spaces

each participant is doing, in relation to the shared information, but how he or she is feeling. Expressive human communication should include speech, gesture, posture and facial expressions. Shaking hands and passing task-related artefacts, from business cards to a model of a new product, are important activities in real world group activities. We are, however, constrained by technology, physics and cost. The constraints of computers, networks, displays and acquisition devices introduce a gap between what we would like to achieve, and what is currently realisable. In practice, we need to make trade-offs, by reducing realism, naturalness and content where they are not needed, in order to maximise performance where necessary. This is typically addressed in terms of what each user can see and in what detail, as well as the objectivity and responsiveness of interactions with shared information. IIS applications are numerous and have diverse requirements. Specialised IIS communication architectures attempt to strike the balance within various application genres. It has been found that striking the balance within one or more genres requires a complex architecture comprising many co-operating optimisation and balancing mechanisms. Common mechanisms will be dealt with in detail later. This chapter is concerned with the systems issues of communication in IIS. That is, how we make best use of computers and networks to support co-located and geographically distinct users in an IIS.

13.1.1 Requirements We set the scene by briefly introducing a number of application genres along with the balance the communication infrastructure needs to set for each. A detailed discussion of application genres is beyond the scope of this chapter and we restrict our description to Table 13.1. The remainder of the section discusses common requirements in detail. Some architectures provide a level of configuration, and sometimes adaptation, to cope with differential application requirements, various computational resources and dynamic network characteristics. Before we discuss the complexities of IIS communication architectures, it is important to understand what this technology can give us, what we can do with it and what kind of information needs to be communicated. The remainder of this section introduces some functional and non-functional requirements of an IIS communication infrastructure. Our discussion of functional requirements focuses on situating inhabitants in a social and information context. Non-functional requirements are taken from the various communication media used by IIS as well as the computers and networks that these media must run on. Information objects act as foci for activity and often collaboration. Users can collaboratively affect the presentation, content and behaviour of shared information. Simple interaction with information is often

234

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

Table 13.1. Typical compromises for various application genres. Application genre Maximise Reduce Tele-conferencing Expressive avatar Group size, complexity and communication interactability of shared information Scientific visualisation

Faithfulness of simulated behaviour, consistency

Group size, avatar communication

Cohabited agent spaces

Communication between agents and between agents and users

Avatar communication, responsiveness

Social meeting places

Group size

Avatar representation, responsiveness, consistency

Games

Responsiveness

Faithfulness, avatar representation, group size

Training and planning

Faithfulness, responsiveness, consistency, repeatability

Group size, avatar communication

011

011

011

011

11

achieved through selection and manipulation tools, allowing the representation to be moved to a more suitable viewing perspective. Application specific 3D toolbars give additional control and adaptation. The representation of information may itself incorporate handles or tools for natural interaction. The presented information can often only be understood in the context of how a group is working with it. It is therefore important to demonstrate how others are interacting with the data as well as supporting instructive and expressive communication within the group.

13.1.2 Information Information may represent anything and be represented in many abstract forms. A reasonable question is, what can IIS offer in terms of information representation, over and above what we had before? IIS is a combination of advanced technologies and is not restricted to a particular set of these. Let us briefly look at the way in which some component technologies are changing what we can do with information. Access to unprecedented scales of data and processing is now available through technologies such as the e-Science GRID (GRID). 3D graphics, simulation and display devices give unprecedented naturalness in the viewing and steering of such information. Mobile and social agents provide powerful ways of finding, assessing and combining information. A Collaborative Virtual Environment (CVE) allows us to share information and observe those with whom we share it. IIS technology encompasses all of this and thus gives us novel ways of presenting and interacting with shared information in a distributed group setting. So what does this

235

Inhabited Information Spaces

information look like? An advantage of computer graphics and virtual reality is that we can tailor the representation of information to any abstract form that best suits the user, application and display device. What can we do with it? We can alter the representation, change the detail, content and state, or steer the simulation. Most importantly we can share information and share the way we work with it.

13.1.3 Avatars The spoken word is often the most important medium for communication between users. It is, however, not sufficient to demonstrate how others are interacting with the information. When combined with video streaming, or a 3D avatar capable of reflecting gesture and posture, we have an effective tool for instructive and emotive communication. Viewing a remote user through a video window is, however, not effective for demonstrating how the remote user is interacting with information. Representing both the information and remote user through 3D graphics gives a much better impression of how each user is interacting within the team and with the data. All that is left to situate the inhabitants in a social and information context is the support of expressive communication. This brings us to the topic of user-controlled, computergenerated characters (avatars). Video avatars can provide high levels of detail, realism and expression. They faithfully reflect the actions and emotions of their user. Although there is little technical difficulty in placing a stereo video in a 3D world, it is much harder to capture imagery of the user. Problems of camera placement within a display system are exacerbated by freedom of movement of the local user and any number of observers. Other problems include isolation of the user from his environment, occlusion of the displayed image by the cameras, and the high bandwidth requirements of multiple streams of video across a network. For these reasons, most IIS systems use avatars generated from 3D graphics. These are typically humanoid with movable joints that provide a basic reflection of body movement. Although such avatars are not as realistic, they can provide instructive and emotive communication sufficient for many applications. In the real world we look at posture, gesture, subconscious movement and facial expression to gauge emotion. All of these can be represented through an avatar. The problem again relates to capture. A typical display device takes sparse input from the user to control the avatar. For example, a desktop system may use a mouse to control movement, mouse keys to interact with objects and keyboard to chat. An immersive display system, such as a Head Mounted Display (HMD) or CAVE would typically track the head and a wand held in the dominant hand. The wand provides additional input for moving long distances in the environment and interaction with objects. Talking would

236

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

normally be communicated through streamed audio. Such input is sufficient for demonstrating how a user is interacting with data. Showing any emotion through a current desktop interface is almost impossible without additional input. The combination of audio and freedom of single-handed gesture does allow a base level of expressive communication from an immersive device. It has been found that when desktop and immersive users work together, the latter take dominant roles, presumably from their greater ability to express themselves (Slater et al., 2000b). Further to this we have found that where two immersive users share an environment with desktop counterparts, the former team up mostly ignoring the latter. Greater levels of emotive communication may be achieved for any device by allowing the avatars to improvise (Slater et al., 2000a). Here the avatar will attempt to fill in the gaps left by the lack of input. Context and profiled personality may be used to interpret user input, or lack of it, and drive suitable emotive behaviour. Other behavioural techniques may further enhance the believability and realism of avatars. For example, behavioural reasoning may combine concurrent simple autonomous behaviours such as fidgeting, shifting weight between the feet, breathing and eye movement. Reactive behaviour is useful to define how objects, including avatars, react to given interactions. Diverse behaviour can be achieved through applying polymorphism, allowing something of given type to behave in new ways to given stimuli.

13.1.4 Interaction Some basic requirements for interaction within IIS are responsiveness, detail and intuitiveness. As they are of prime importance to the usability of the environment, they will now be discussed in more detail.

Responsiveness A key aspect of usability and believability is maintaining responsiveness of interactions close to the level of human perception. Changes in the presented information must be represented as soon as a user affects it. Low responsiveness will make the system feel unnatural and cause frustration. Immersive displays render the environment from a new perspective every time the user moves his or her head. A low responsiveness in updating perspective causes disorientation and sometimes feelings of nausea. IIS introduce the issue of responsive sharing. This is a particular concern where users are in geographically distinct locations connected over a network. The communication infrastructure must provide sufficient responsiveness to support, and not confuse, the natural sequences of conversation and interaction.

237

Inhabited Information Spaces

Detail Some interactions will require more detail than others. For example, to interact with another person it is often important to communicate both complex language and emotion. Even email users exchange icons to represent how they feel. The detail of presented information will be a balance between the data from which it is derived and what is useful and perceivable for each user. The communication infrastructure must support a wide range of detail in interactions and should do so in an optimum manner.

Intuitiveness Interaction must be both natural and intuitive. A user should be able to interact with an object or peer, without having to worry about overcoming shortcomings in the technology. Furthermore, an object should react in a believable way regardless of how or where it is implemented. This places requirements on both the device and the infrastructure. Display devices offer various input/output capabilities that may be mapped to interaction scenarios. For example, in an immersive system, users may use a joystick or a wand to move up to an object, their own body movement to position themselves at the correct aspect, and the wand to select and manipulate the object. Both the physical device and the way its inputs are interpreted must map to natural and believable behaviour in the virtual environment.

13.1.5 Communication Requirements So where does this leave us in terms of communication? Representing information using computer-generated graphics gives unprecedented power to tailor its presentation. Virtual Reality (VR) uses 3D graphics to allow the user to control his or her position in the environment, giving natural access to spatially organised information. CVE socially situate a group of users around information within a familiar spatial context. Although video avatars would offer a potentially higher level of realism, computer-generated avatars are easier to situate in an environment where users have freedom to walk around. It is not surprising that the majority of IIS systems rely primarily on 3D computer-generated graphics to present visual information. Unlike video, 3D graphics scenes, comprising the geometry and appearance of many objects, can be downloaded in advance. Where users are distributed, the scenegraph may be replicated at each user’s computer with incremental changes sent across the network. This massively reduces bandwidth usage and also increases

238

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

the responsiveness. Without such replication, any user movement would require a perspective recalculation of the scene on a server before the resultant images could be streamed back to the user’s machine. This approach is generally unusable as the network delays result in feelings of disorientation and nausea as the user’s visual inputs lag behind their internal senses of balance and proprioception. 3D graphics may be the primary medium for IIS but it is often combined with others. Natural language has been shown to be of vital importance to collaborative tasks. Audio streaming has been found much more effective than chat in IIS settings and does not require the use of a keyboard. Streaming of video and 3D graphics across the network is useful for rich and detailed images provided observer perspective is constrained. An exception to this is tele-presence, which allows a single user to see through the eyes of a movable robot, but this is outside the scope of this chapter. Table 13.2 shows how various media are typically used together, when they are used and how they impact on available network bandwidth. The degree to which each medium is used is application dependent. We have assumed so far that sharing the use of the information is the primary goal of an IIS communication infrastructure. Other applications, for example, may place more emphasis on emotive communication and thus use video as the primary medium. The remainder of this chapter deals with the typical and leaves specialisation to other works. We therefore restrict our discussion to systems that primarily use 3D graphics for vision, audio streaming for speech, and video and 3D graphics streaming for occasional supplementary, high detail, imagery.

011

011

011

011

Table 13.2. Usage of mediums in IIS and the effect on the network. 3D graphics – Audio Video Streamed 3D replicated graphics scene graph Purpose Primary Primary Supplementary Supplementary visual natural visual and visual e.g. for medium language audio for high end medium perspective graphics on constrained desktop high fidelity Usage

Continuous

Bandwidth High during usage initial download then medium

Text chat

Alternative to audio streaming for desktop, public systems

When user Occasionally is speaking as required

Occasionally as required

When user is chatting

Medium

High

Low

High

11

239

Inhabited Information Spaces

13.1.6 Resources: Computers and Networks Let us now take a brief look at the relevant characteristics of computers and networks. It is, after all, these that must underpin an IIS communication infrastructure. Computers have limitations on the amount of information they can store and process. An IIS will often contain users supported by computers of widely differing capabilities. These computers may be connected to various network technologies such as Ethernet, ATM and wireless. These networks are often part of the greater Internet and will communicate through intermediate networks of various technologies. These technologies have widely differing bandwidth, delay characteristics and reliability. IIS communication infrastructures use the Internet Protocol (IP) that deals with the heterogeneous nature of the Internet by making low assumptions about the guaranteed service. That is, IP assumes that messages may be fragmented and individual fragments may arrive late, out of order or be lost. The Internet, and often super computers running display devices and processing information, are shared resources offering highly dynamic levels of throughput depending on localised load. A final important point is that the speed of light will introduce perceivable network delays for many intercontinental links. An IIS communication infrastructure must be designed to run over a set of heterogeneous computers and networks, each with possibly very different dynamic, throughput and reliability.

13.2 Principles IIS situate inhabitants in a social and information context that extends interaction in the real world in a natural manner. Technology, physics and cost create a gap between this ideal and reality. This section is concerned with balancing throughput limitations of computers and networks with the requirements of IIS applications. IIS communication infrastructures employ a set of co-operating mechanisms and algorithms that effectively concentrate resources by maximising the fidelity of sharing where it is needed by reducing it where it is not. We have looked at what might reasonably be expected in terms of perception and interaction and how this may be supported through a combination of existing communication media. We have explained why 3D graphics with replicated scenegraphs have become the primary medium of communication in IIS and how these may be supplemented with other media. We restrict our discussion here to the mechanisms for improving the fidelity of sharing through 3D graphics and replicated scenegraphs. A key requirement of IIS and VR is the responsiveness of the local system. Delays in representing a perspective change following a head movement are associated with disorientation and feelings of nausea. An IIS system supports a potentially unlimited reality across a number of

240

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

resource bounded computers interconnected by a network which induces perceivable delays. Key goals of an IIS communication infrastructure are to maximise responsiveness and scalability while minimising latency. This is achieved through localisation and scaling.

13.2.1 Localisation Localisation is achieved through replicating the environment, including shared information objects and avatars, on each user’s machine. Sharing experience requires that replicas be kept consistent. This is achieved by sending changes across the network. Localisation may go further than simply replicating the state of the environment and can also include the predictable behaviour of objects within it.

Object Model The organisation and content of a scenegraph is optimised for the rendering of images. Although some systems, for example Cavernsoft (Leigh et al., 2000) and Arango (Tramberend, 2001), directly link scenegraph nodes across the network, most systems introduce a second object graph to deal with issues of distribution. Known as the replicated object model, we will from here on refer to it as the replication and its nodes as objects. Objects contain state information and may link to corresponding objects within the local scenegraph.

Behaviour A virtual environment is composed of objects which may be brought to life through their behaviour and interaction. Some objects will be static and have no behaviour. Some will have behaviour driven from the real world, for example by a user. Alternatively, object behaviour may be procedurally defined in a computer program. In order to make an IIS application attractive and productive to use, it must support interaction that is sufficiently intuitive, reactive, responsive, detailed and consistent. By replicating object behaviour we reduce dependency on the network and therefore make better use of available bandwidth and increase responsiveness. Early systems replicated object states but not their behaviour. Each state change to any object was sent across the network to every replica of that object. This is acceptable for occasional state changes but bandwidth intensive for continuous changes such as movement. Unfortunately, movement is one of the most frequently communicated behaviours in IIS. A more scalable approach is to replicate the behaviour model and only send changes to that behaviour. Such changes are known as events.

241

Inhabited Information Spaces

Deterministic Behaviour Behaviour may be characterised as deterministic or non-deterministic. Deterministic behaviour need not be sent across the network provided it can be calculated independently at each replication. Most procedural behavioural descriptions such as reactive, improvisational and emergent may be defined in a repeatable and deterministic manner. Events can simply identify the name and possibly arguments to a procedure, the execution of which will be replicated at each machine. Even non-deterministic behaviour can be approximated as deterministic provided the effect of bad approximations are not catastrophic.

Dead Reckoning Constrained movement, such as that of a vehicle, may be determined approximately using a technique called dead reckoning (IEEE1278.1, 1995). One of the earliest applications for large-scale virtual environments was battlefield simulation (SIMNET). Here embodiment was originally confined to vehicles, such as tanks, where the vast majority of communicated behaviour was movement around the battlefield. Dead reckoning was introduced to reduce bandwidth consumption of movement information. A dead reckoned path represents a predicted approximation of near future parametric movement based on recent samples of position over time. Paths are sent to other replicas in events. A remote replication then calculates the probable position of the vehicle based on current time. Divergence is checked at the sender by comparing actual and predicted position by running the same algorithm as the receiver on the path it has sent. When divergence exceeds a threshold, a new path is calculated and sent. The algorithms for calculating the path are based on Newton’s Laws and Hamilton’s quaternion expressions from the seventeenth century. Variations on the approach deal with first and second order integration, time constants and smoothing (Miller et al., 1989). The remote user is presented with an approximation of movement, the most noticeable aspect of which is sudden jumps in position when a new event is received (see Figure 13.1). The magnitude of this discontinuous jump is the product of the difference in velocity described in two adjacent events and the network delay.

Consistency Public switched networks, such as the Internet, introduce both dynamically changing delays and the possibility of loss. This can adversely affect the synchronisation, concurrency, causality and responsiveness of events. Synchronisation ensures that events are replicated within realtime constraints. Causal ordering ensures that causal relationships are maintained. Concurrency defines the ability of the system to allow

242

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

Local movement

Remote representation

Figure 13.1 Effect of dead reckoning.

events to occur simultaneously. Lastly, responsiveness is the delay in the user perceiving the effect of an action on the system. Concurrence and therefore responsiveness are reduced as the level of consistency is increased. This all leads to the need for consistency management, the role of which is to provide sufficient synchronisation and ordering while maximising concurrence and thus the responsiveness of the system. The optimal balance between sufficient synchronisation, ordering and responsiveness is application and scenario dependent. An ideal ordering mechanism provides a compromise between synchronisation and ordering on one side and responsiveness and concurrence on the other.

Synchronisation Behaviour may be described parametrically. For example, dead reckoned paths describe movement through time. Some early systems based time on frame rate. This can be seen in some single user computer games, where the movement of objects slows down as the complexity of the scene increases. This approach is unsuitable for IIS as shared behaviour should be consistent and not represented differently to each user dependent on the performance of the local machine. A common approach is to use the system clock on each computer to provide a continuous flow of time. Movement can then be described in terms of metres per second and will be represented at the same rate to each user. As well as progression it is important to synchronise the start of replicated events. Some systems, for example, NPSNet (Macedonia et al., 1994), set the start time of a received event to the time at which it was received. This removes the need to synchronise local clocks accurately, which is a non-trivial task. The disadvantage with this approach is that any behaviour is offset by the network delay. Through synchronising local clocks it is possible to synchronise the state of objects from the time an event arrives until the time a subsequent overriding event is sent. The PaRADE system, developed as part of the author’s PhD (Roberts, 1996),

243

Inhabited Information Spaces

allows locally predictable events to be sent in advance, thus overcoming the network delay and synchronising from the start.

Concurrency Control Concurrency control is an important subset of consistency management that deals with the prevention of concurrent conflicting updates. This is most apparent where two users try and move a given object in conflicting directions. Without concurrency control it is difficult to determine the outcome but it will at least cause confusion and frustration and at worst an unrecoverable divergence between replicas. Many existing infrastructures do not include concurrency control. Those that do, employ algorithms that themselves are adversely affected by network latency. This in turn affects the responsiveness of interaction between user and shared information objects. A conservative concurrency algorithm, as used in some analytical simulations, would lock the whole world and allow updates on a turn basis. This unnecessarily restricts responsive interaction to a level that is unworkable for general IIS applications. An optimisation is to increase the granularity of locking, to either sets of objects, object or object attribute level. A common mechanism for concurrency control is transferable object ownership, where a user can only affect an object once ownership has been transferred across the network. The effect of such latency is normally apparent in a delay in being able to interact with an object recently affected by another user. Optimisations have been developed for predicting interactions and transferring ownership in advance (Roberts et al., 1998).

Causality Events sent over the Internet may be lost or arrive in a different order to which they were sent. In many cases the current state is more important than history and can be derived from an old state and a new event, even when some preceding events have been missed. For example, a new dead reckoned path overrides the last and is not dependent upon it. Ordering is, however, often vital. A lack of ordering can cause complete confusion when collaborating with remote users and sharing objects. It is therefore surprising that the majority of IIS systems do not guarantee it. This is most likely a throwback to the conventional applications of collaborative virtual environments that did not properly support shared interaction. Order must be balanced against responsiveness. The greater the level of ordering, the lower becomes the concurrence and thus the responsiveness. A true objective state of an environment cannot be guaranteed until all events have been received and processed in the correct order. Generating a new event before the objective environment state is known is dangerous and requires some strategy for dealing with events generated on the basis of an untrue state. To guarantee objectivity all

244

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

replicas must be frozen while waiting for events to arrive, thus lowering the concurrence. Lamport developed an optimisation called causal ordering which removed the need to order events that could not have been related (Lamport, 1978). The definition of causal relationship was based on the subjective view of a replication. Total ordering and Lamport causal ordering work well in distributed analytical simulation but are not generally suited to IIS applications which require continuous and responsive interaction with the environment. One solution is to allow the IIS infrastructure to decide when to apply and where to apply ordering. This decision may be based on application knowledge of causality and importance of ordering, awareness (see below) and network conditions. Such approaches have been applied to various degrees in PaRADE, MASSIVE 3 (Greenhalgh et al., 2000b) and PING (Section 13.3.2).

011

Application of Consistency Now that we have introduced synchronisation, concurrency control and ordering as the basic components of consistency, we can look at how they are applied. Table 13.3 compares common alternative mechanisms for each, describing each mechanism, giving an application level example of use, comparing typical delay in terms of level of human perception and giving some example infrastructures in which they are used.

011

011

Table 13.3. Comparison of consistency mechanisms. Synchronisation Concurrency

Ordering

Description

Behaviour of an object Object replicas affected is synchronised over concurrently replicas

Order of object events over replicas

Mechanism

Wall clock

Causal

Total

Effect

Remote Replicas Diverging object update states are follows in step converged parametric behaviour

Prevents divergence through unique key

Based on potential causality

All events are ordered

Example

Dead reckoning

Crowd Tug of war walking with elastic in step rope

Passing a business card

Player’s activity ordered in ball game but not with spectators

Player action delayed until earlier spectator action observed

Induced delay None

Medium Low

Low

Medium

High

Example Most Infrastructure

RTI

DIVESpelunk, PaRADE,

MassiveIII, RTI PaRADE, PING

Tick

Convergence Ownership

011

11

DIVE

245

Inhabited Information Spaces

13.2.2 Scaling Scaling allows the amount of information in the environment, including the number of users, to increase, without reducing the fidelity of experience to any one user. This is achieved by balancing each individual’s need for information with what can be achieved given available computational and network resource (Benford and Fahlén, 1993a). Awareness management is the mechanism used to balance an individual’s ideal awareness with resources. The scale of information provided to any one user or process may be controlled in terms of extent, granularity and detail. These define awareness in terms of object subsets of the environment, aggregation of many objects into few, and the attributes of a given object.

Extent The majority of effort in attaining scalability has focused on subdivision of the environment and population according to interest. This is often referred to as interest management. Awareness of remote objects is determined by context-dependent interest. Distinct resources such as servers or communication groups, discussed later, are used to support each area of interest. The interest of a user is dynamic and context dependent. For example by walking into another virtual room a user becomes aware of its contents and occupants. A number of technical issues must be addressed in order to support this dynamic awareness. Subdivision should be natural and appear transparent otherwise it can affect a user’s behaviour. To be effective it must balance resource usage across the areas of interest. Changing awareness may require much data to be transferred. This can result in delays in the presentation of a new area, which may reach the order of seconds. Different application genres are suited to distinct definitions of interest and methods of subdivision. The granularity of subdivision may be tackled at world, object or intermediate level. We now survey some classic approaches to subdivision used in IIS which are summarised in Table 13.4.

Multiple Worlds A simple method for dividing the environment and population is to provide distinct multiple worlds. Each world is typically supported by a distinct server and hosts a distinct set of objects and users. As discussed in the deployment section, this is straightforward to support over the Internet and thus is prevalent in current systems used by the general public, for example, in games such as Ultima Online (Electronic Arts, 2003) and social environments. Users typically inhabit a single world at a given time and, in some systems, may move between these worlds using portals (Snowdon et al., 2001). The disadvantage of this approach is the

246

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

Table 13.4. Overview of classic subdivision approaches. Approach Description Granularity Multiple worlds Separate world connected World through portals

Example systems Active worlds, Ulitima, DIVE

Static spatial subdivision

Divide world surface into tiles

Intermediate

NPSNet

Dynamic spatial subdivision

Flexible mesh of tiles that stretches to balance tile membership

Intermediate

VIVA

Löcales

Rooms

Intermediate

SPLINE

Aura

Aura, focus and nimbus

Object

DIVE, MASSIVE I and II

Regions

Abstract spaces

Intermediate

MASSIVE II, DIVE (COVEN version)

difficulty in balancing the number of users in each world. Figure 13.2 shows multiple worlds interconnected through portals and demonstrates the potential problems with balancing population.

Static Spatial Subdivision Increasing the granularity of subdivision allows worlds to be split into areas of interest. An approach developed for battlefield simulation and training was to divide the environment into areas of interest in the shape of equal hexagonal tiles and map each to a communication group (Macedonia et al., 1995). A process sends information to the group associated with the tile occupied by its user and receives information from that group and those associated with adjacent tiles. The supporting process dynamically joins and leaves groups as the user moves between tiles. Receiving information from adjacent tiles removes the problem of not seeing spatially close objects across a border. Group communication provides a mechanism for limiting awareness at a message distribution level with the added bonus of removing the need for a server.

Worlds

011

11

Portal

Figure 13.2 Multiple worlds, showing portals and possible population loading.

247

Inhabited Information Spaces

Figure 13.3 Static spatial subdivision.

It is not yet, however, generally supported on the Internet and so this method adds complexity to deployment which is discussed later. Again, the static nature of this method can also produce an unbalanced population of areas (see Figure 13.3).

Löcales Environment plays an important role in restricting and focusing human interaction. Spatial subdivision approaches are suited to open spaces but do not take advantage of the awareness limits imposed by buildings. Löcales (Barrus et al., 1996) are areas of interest that map to physically divided spaces such as rooms in a building (Figure 13.4). This approach relies on the adequate provision of resources to support a crowded room and again suffers from its static nature. It is, however, sufficient for many applications.

Dynamic Space Subdivision The above approaches rely on an even distribution of users across statically defined areas of interest. This suits them to particular implementations and restricts their general applicability. Dynamic space subdivision attempts to redefine divisions between areas of interest in order to balance the number of users in each (Figure 13.5). Robinson et al. (2001) divide the environment into a 2D mesh or 3D lattice and move

Figure 13.4 Löcales.

248

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

Figure 13.5 Dynamic spatial subdivision.

011

011

011

011

11

the boundaries between the areas of interest to balance membership. Boundary movement is considered when an area becomes over populated and is determined through negotiation between servers dedicated to adjacent interest areas. Robinson’s algorithm considers the cost of moving a boundary to both servers and clients.

Aura Interest may be determined at the granularity of object pairs by determining their potential for interaction based on spatial proximity. Spatial proximity may be efficiently detected by placing auras around objects and checking for aura intersection. In the case of avatars, this potential for interaction is increased when they face each other. Benford and Fahlén (1993b) encapsulate avatars in auras and use aura collision as a perquisite for interaction. Within the aura, focus and nimbus spatially define attention and projection respectively (Figure 13.6). Both focus and nimbus reach out in front of the avatar but have distinct shapes.

Regions Both tiles and Löcales are specific definitions of how to divide the environment and are applicable to distinct forms of interaction and application genres. MASSIVE 2 combines aura-based awareness within abstract regions which may be mapped to application-specific definitions of interest. Figure 13.7 depicts one possible way of dividing an environment into regions.

Focus Nimbus

Figure 13.6 Aura – focus and nimbus.

249

Inhabited Information Spaces

Figure 13.7 Regions.

Granularity In the real world people are able to reason at different levels of granularity. For example, a lecturer must be aware of the attention and understanding of each student during a lecture whereas a university chancellor sees the institute in terms of departments. This approach of aggregation may be adopted in IIS systems to further increase the scalability. Aggregation reduces not only the rendering but also the amount of information needed by some observing processes. For example, in a battlefield simulation, the driver of a tank is interested in other tanks whereas a general is more concerned with tank divisions (Singhaland and Cheriton, 1996). Another example is that of a crowded stadium represented by a single avatar (Greenhalgh, 1998). The size of the group, the team they support, and the sound they produce, are represented through the avatar’s size, colour and aggregated audio streams respectively. Emergent behaviour may be replicated and communicated in aggregated form to reduce the load on the network. For example, the behaviour of a flock of birds could theoretically be replicated by simply communicating the size of the group and then continuing to communicate the movements of whichever bird is in front. A reasonable flocking behaviour can then be replicated at each site through application of local rules based on following and collision avoidance. This aggregated emergent behaviour may be applied to many other group behaviours, for example the behaviour of a human crowd. A similar principle can be applied to an avatar allowing the majority of body movement to be calculated locally and driven by the communication of movement of selected body point, such as head and hand. Here, a combination of kinematics and selections of previously recorded motion tracking data can be used to improvise reasonable local behaviour based on head and hand movement. In order to reduce network traffic through aggregation it is necessary for the sender to know the level of aggregation. Although aggregation can increase the scalability of a receiving process it can decrease the scalability of the sender and the use of the network when many receivers require distinct levels of aggregation for the same objects (Roberts, 1996).

250

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

Detail We have seen how scalability may be increased by reducing the number of communicating objects held on each machine. Scalability can be further increased by managing the detail at which individual objects are replicated. Heuristics of interest such as distance or the relationship between the role of the observer and the use of the observed may be applied. Many graphics languages, such as Inventor, Performer and VRML, support Level of Detail (LOD) modelling where a sufficient frame rate is maintained by reducing the graphical complexity of distant objects. The scalability of communication and computation can be greatly increased by applying this reasoning to the communication of behaviour. Objects may be defined in terms of attributes in which remote processes can dynamically express and decline interest, for example as defined in IEEE 1516 and implemented in the DMSO RTI. Balancing the detail of communicated behaviour with the interest of remote users is an important, if under researched, topic. The amount of information being received may be reduced through local filtering or sending control messages back to the sender. The latter approach again suffers from the potential need to send distinct levels of information to different receivers. A hybrid approach might send the highest detail required by any to all and allow receivers to filter further.

13.2.3 Persistence Users can join, leave and rejoin collaborative virtual environments at will. When in the environment, they can affect its state through interacting with, and introducing, objects. A real world analogy is a bank account. When someone deposits money into a bank cash machine, the money should not be lost as soon as the card is withdrawn. Persistent environments will maintain the effect of changes when the user leaves. Supporting persistence is straightforward when the underlying CVE infrastructure hosts all master objects on servers. Where a localised approach has been adopted to increase scalability or responsiveness, master objects will be held in the memory of a user’s machine. These must be moved to a participating machine when the user leaves. Provided the behaviour of an object is known at the target site, it is only necessary to move the current state and master status of the object. There are two basic forms of persistence: state and evolutionary. State persistence maintains an object in a static state once its owner has left. Evolutionary persistence will support the ongoing behaviour of an object. For example, in a lecturer’s bank account, which is always overdrawn, the money deposited will be reduced over time by interest payments.

251

Inhabited Information Spaces

So far we have only considered what happens to objects when a user leaves. We must also consider the effect of the environment going offline. Such an occurrence may be planned or accidental. In either case we may wish to guarantee persistence. One solution is to store object state information to disk on a persistency server both periodically and, where possible, when an imminent failure is predicted.

13.2.4 Communication Previous sections have introduced the kind of information that must be passed through an IIS and we have described object level mechanisms for managing this information in order to maximise responsiveness and scalability. We now move down into the message level to examine how to actually communicate this managed information.

Requirements The communication requirements of an IIS are complex. Those of responsiveness, reliability and scale of information transfer differ greatly depending on application, context and scenario. Before we describe the method of communication we must look at the content. We now examine some typical forms of information and their requirements on the underlying communication system. This is broken down into discovery of objects; events; audio and video.

Discovering Objects When a client alters its awareness by entering a new world or area of interest, it must discover the objects within. Some mechanism is required for the client to obtain all the information about every object it discovers. This information includes state, behaviour and graphical appearance. Behaviours, and particularly appearance descriptions, tend to be much larger than state, but in most systems remain unchanged throughout the lifecycle of an object. Such information is typically in the order of kilobytes per object. Usually such data only needs to be sent to one client at a time and it must be sent reliably, in order and preferably efficiently. Users frequently move between areas of interest, which results in traffic bursts as the local system downloads object state and possibly appearance and behaviour. In turn this can result in delays often reaching several seconds. It is therefore important to use an awareness management scheme that minimises movement between areas as well as the number of objects in each. Some systems download from an existing peer process but this can cause that process to lock, which is disorientating for its user. The responsiveness of remote peers may

252

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

be maintained by obtaining all object information from a persistence server.

Events The behaviour of objects is driven and communicated by events. Events need to be propagated to any interested process as quickly as possible. They are typically very small in terms of network bandwidth. Many events are frequent and quickly superseded. Others are infrequent and their loss might cause applications or users to act in an erroneous way. The majority of events typically describe movement. Constant latency is important as it improves the realism of remote movement. As discussed above, in the context of event ordering, it is typically more important to reflect the current position as opposed to how the object arrived at it. Since we may typically send many movement events for a given object in a second and that the probability of message loss is low, lost movement events will seldom be noticed. An important exception to this rule is introduced by dead reckoning where the frequency of path generation is considerably lower. We presented a scheme for addressing this problem by reliably re-sending dead reckoned paths that had not been superseded within a time limit (Roberts, 1996). Tracking systems allow natural non-verbal communication but generate quantities of events that are difficult to support over the network. During trials between networked reality centres in UK and Austria, we found it difficult to realistically approximate human movement with dead reckoning but have had greater success limiting the frequency of outgoing events for given objects by simple filtering. Bursts of events typically accompany interaction with other avatars, objects or both. For example, avatar communication may well include gesticulation and talking. This results in bursts of movement events and audio traffic. Such exchanges can occasionally swamp bandwidth and overrun receive buffers, resulting in high message loss. This is particularly the case for groups of interacting users. Remote events can sometimes be delayed for seconds while the IIS system attempts to catch up with the receive buffer resulting in a temporary loss of responsiveness. In this case the loss of movement events is preferable as it brings the system back to a synchronised state in a shorter time. Some systems, for example PING and DIVE, limit this time through a Bucket algorithm. Some events may be vital particularly where they affect the result of, or ability to process, subsequent events. This includes any event that changes the structure of the scenegraph. Such events are commonplace where users interact with objects. Losing such events can cause significant divergence between users’ views. For example, one user sees that he has taken an object out of another’s hand, while the other sees herself still holding it. At best this causes confusion and at worst, an unrecoverable divergence.

253

Inhabited Information Spaces

Audio Verbal communication considerably improves the performance of general collaborative tasks as well as the feeling of co-presence. In order for audio communication to support human conversations it must be continuous and have a constant rate and sufficient resolution. Network bandwidth and message loss can reduce resolution. Network jitter, where heavy network traffic causes temporary high delays, can alter the rate at which the data is delivered. The COVEN trials suggest that audio traffic is in the order of kilobytes per second for each user (Greenhalgh, 2001).

Video Video has similar requirements for continuity and rate but high resolution images can require much higher bandwidths. Typical IIS use video sparingly, mapping low resolution streams to polygons. For example, a low resolution video avatar might require tens of kilobytes per second.

Solutions Now that we have described how information needs to be communicated we will look at ways in which this is achieved in typical IIS. In particular we focus on how data is prepared for sending over the network and how it may be disseminated to one or many recipients with various Qualities of Service (QoS) of delivery.

Preparation Before being sent over the network, data is marshalled into a flattened message. This message is split into packets and sent across the network. Transport level network protocols convert between messages and packets. The size of a packet is determined by the underpinning link level protocol. The Internet Protocol (IP) adopts a maximum packet size from the underlying network technology. In an IIS system a packet might contain one or several events and a continuous flow of packets might support an audio or video stream.

Dissemination Dissemination determines if a message is sent to a single or a group of recipients. Distinct forms of dissemination are offered by various transport level protocols. Group communication multicast is often used in IIS to scale the number of users. Multicast allows hosts to express interest in any set of communication groups. A message sent to a group will be distributed to every member at no extra cost to the sender. The

254

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

scalability of the sender is maximised while that of the receiver can be increased by mapping awareness to groups. Group dissemination may alternatively be implemented above point-to-point protocols by sending a message to a set of connections, for example in Spline (Waters et al., 1997) and PaRADE (Roberts et al., 1999).

Delivery Not all packets that are sent arrive, or arrive in the correct order. Their subsequent assembly into a message, and delivery to the application, may be delayed while these errors are overcome. QoS determines what criteria, in terms of reliability, order and timeliness, will be met before delivery. Generally the higher the reliability and level of ordering, the lower the responsiveness and scalability. This is particularly the case for group communications. Different transport level protocols offer distinct QoS in addition to dissemination. Some systems, for example, DIVE and PaRADE, implement additional or improved qualities of services above the transport level.

011

011

Mapping Information to Dissemination and Delivery We have seen how various types of information in an IIS require distinct levels of dissemination and QoS. Some IIS systems simplify their design by using single dissemination and delivery methods and accept the drawbacks. Others, for example HLA RTI, PaRADE, PING and DIVE, combine various dissemination and QoS delivery methods to optimise performance. Table 13.5 suggests how an IIS might map information type dissemination and QoS. This table is derived from combining best practice of PaRADE, PING and DIVE.

011

011

11

Table 13.5. How an IIS might map information type dissemination and QoS. Type Example Reliability Order ResponDissemina- Throughput siveness tion Downloads Object High High Low One High discovery Regular events

Movement

Irregular events

Low

Latest

High

Many

Medium

Object creation High

High

High

Many

Low

Audio

Verbal Low communication

Latest

Constant

Many

Medium

Video

Facial expression

Latest

Constant

Many

Medium

Low

255

Inhabited Information Spaces

Channels Managing the mappings between information, dissemination and QoS becomes complex when an environment contains many users dynamically moving between areas of interest. The channel abstraction may be used to map dissemination to QoS. In PING, for example, events are routed to channels according to their type and the current area of interest. Some events may theoretically be sent down many channels, for example unreliably to user machines and reliably to a persistence server.

13.3 Architecture We have introduced the basic requirements and realities of communication within IIS and outlined principles used to balance the two sufficiently to support fruitful collaboration between users socially situated in an information context. This section provides case studies of two example systems, DIVE and PING, describing each in terms of modulised architecture and use of principles. The Distributed Interactive Virtual Environment (DIVE) is a widely adopted CVE platform that implements most of the principles we introduced. The Platform for Interactive Network Games (PING) attempts to bring together best practice from CVE architecture. Although the latter is still in beta prototype stage its design provides a good tool for explanation.

13.3.1 The DIVE Architecture The Distributed Interactive Virtual Environment (DIVE) has a classic architecture consisting of seven modules (Table 13.6). Each represents a conceptual level and is implemented as a unique library. This provides flexibility when updating the platform.

Table 13.6. Modules of the DIVE architecture. Modules Description Video Allows video to be texture mapped to polygons in the scene Audio

Audio supporting conversations between users as well as attaching sounds to objects

Graphics

3D rendering of the graphical representation of objects and thus scene

Aux

Tools for the application building including the scripting language

Core

Object database and supporting functionality such as time and events

Sid

Communication

Threads

Thread library provides concurrence at each computer

256

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

DIVE introduces, adopts and adapts most of the principles described in the previous section. Best practice solutions have been added and iteratively improved for more than ten years. Widely used in research, this platform has proved the principles.

Localisation DIVE uses localisation to maximise local responsiveness and make best use of the network. We will now look at the particular design decisions taken in implementing this localisation within a framework of the principles outlined in the previous section.

Object Model The responsiveness of a user’s interactions with the environment is maximised through object replication, negating the need for events to be passed across the network before the local model is updated. The replicated object database resides on participating machines according to awareness management. A replication is organised into a hierarchy of objects, each of which contain state information and may be attached to behaviour scripts and graphical appearance. A scenegraph is coupled to a local replica and mirrors those qualities of objects necessary for rendering. An application reads and writes to a replica regardless of the fact that other replicas may exist.

Behaviour A simple reactive behaviour associates triggers and responses to objects. An object’s behaviour is defined in an attached script. The script language, DIVE/TCL, extends the Tool Command Language (TCL) to include useful commands for monitoring and updating objects. Typed events may be triggered through a user input device, collision with another object, timer and world entry. An interest in events may be expressed through event call-backs and responses mapped to event types. For example, an application programmer can register an interest in collision events for an avatar and define distinct responses to various types of collided object. Behaviour scripts are replicated along with the object. This allows objects to react to local interactions without network induced delay. Remote scripts are not called directly but through the communication of the same event that triggered them locally. The concept of dead reckoning is supported but the implementation of the algorithm left to the application programmer. Each object is able to store a parametric path from which the current position may be calculated. Use of the path to communicate and calculate current position is, however, optional. This feature is good, as not all objects move in a predictable way.

257

Inhabited Information Spaces

Consistency High responsiveness comes at the cost of low consistency. Replicas are loosely coupled allowing divergence and attempting convergence over time. For example, when a user moves an avatar the remote representation will follow, delayed by the network, and catch up when the avatar stops moving. There is no specific concurrency control within DIVE. Hence an object may be affected in conflicting ways by multiple users, causing the replicas to diverge. Mechanisms are provided to settle an object to a mean position after being pulled in opposing directions. Users, however, observe the object jumping wildly between them until the steady state is reached. A loose form of ownership allows an object to be attached to an avatar. Other users can still affect the object, for example, by changing its relative position to the carrying avatar. An immersive extension to DIVE, Spelunk (Steed et al., 2001), implements concurrency control through object mastership. Partial casual ordering is implemented at the communication level and is therefore described below.

Scaling Awareness is managed at the world level as well as within the world through division of the object model hierarchy. All replicas must hold the route object but can be selectively pruned by local interest. Branches may be assigned to interest groups to which the application may express interest. This low-level approach can support any higher-level awareness management scheme that maps to the organisation of the hierarchy. Both subjective views (Snowdon et al., 1995; Jää-Aro and Snowdon, 2001) and aura-based focus and attention (Benford and Fahlén, 1993b) have been implemented on top of DIVE. Level of Detail (LOD) is partially supported. Composite objects comprise a tree of objects within the hierarchy; therefore, interest-based tree pruning may be used to reduce their complexity. However, this does require some scripting. The LOD of an atomic object can only be switched within the graphics module. Thus, without custom scripting, LOD affects appearance and rendering performance but not behaviour and network traffic. The default renderer supports distance-based LOD switching. Adaptive rendering was incorporated in the COVEN extension to DIVE (Frécon et al., 2001). Here, distance culling and iterative rendering techniques can alter the detail of the rendered scene to meet specified frame rates. Aggregation is not directly supported but again could be implemented at the application level by making use of interest management, this time to switch from a sub-tree to an alternative atomic object.

258

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

Persistence Objects are not owned and thus can be created by an application and left in the world once the application has closed. Any application can remove the object from the world. By default, clients are responsible for persistency. Early versions had no persistency servers but an object would remain in the world as long as one copy of it exists. Later versions of DIVE incorporated persistency servers. Object behaviour can be defined by scripts that are replicated along with the object at each host. Evolutionary persistence is maintained through the continued triggering and execution of scripts. The triggering events can come from the object itself or from other objects in the world.

011

Communication DIVE uses a combination of point-to-point and group communication. The point-to-point protocol (TCP) is reliable and ordered. Group communication is supported at two qualities of service: unreliable, and partial reliability and order. IP Multicast provides the former and is extended into Scalable Reliable Multicast (SRM) for the latter.

011

Discovering Objects The first client to enter a world downloads the initial world from an Internet location. Subsequent clients entering obtain the current world from a peer. This approach allows an up-to-date world to be downloaded immediately without the need for a world server. A downside of this approach is that the peer from which the world is obtained freezes while sending data. This typically takes tens of seconds depending on the complexity of the world. Later versions of DIVE address this problem by allowing downloads from persistency servers instead of from clients. Clients can create objects at any time and must inform peers on doing so. When a client discovers a new object, either through a creation or update message, it may request the object. With the exception of the first client download, all requests and downloads are done over SRM. An algorithm attempts to transfer objects from the nearest client in terms of network delay.

011

011

11

Events All events are sent using SRM. Partial reliability with ordering are mapped to three event categories: movement, geometry and general. By default, all three are set to reliable. Each object has a causal counter which is stamped to outgoing messages. Partial ordering and reliability of events are implemented within SRM. Partial ordering ensures that two

259

Inhabited Information Spaces

events from the same object are delivered in the order they were sent but the same is not guaranteed for events from distinct objects. Partial reliability guarantees that if a lost event is detected through arrival of a later event, the object state may be requested. The assumptions made are that event loss and disorder are rare and that the partial reliability and ordering are thus sufficient to converge the databases over time. Both reliability and ordering are achieved through object sequencers thus providing a high level of concurrence and thereby reducing the effect on responsiveness. Receiving an unexpectedly high sequencer detects message loss. When this occurs, the state of the object is requested rather than the set of lost updates. The downside is that loss is not detected until a subsequent event from the same object arrives. The loss of events for infrequently updated objects can make some applications unworkable. For example, a door might be unlocked by one user but remain locked to another. Furthermore, a dead reckoned path can result in considerable divergence if a subsequent path event is lost.

Audio and Video Both audio and video are streamed across unreliable multicast. Responsiveness and consistency are slackened to allow constant delivery rates suitable for human communication. Each world has a unique multicast group for audio and another for video. Sound is spatialised so that objects and avatars are heard from where they are seen.

13.3.2 PING The Platform for Interactive Network Games (PING) was developed by a European consortium headed by France Telecom. It combined many best practice principles into a scalable architecture implemented as a communications infrastructure for support of massive multi-player games. Figure 13.8 and Table 13.7 summarise the PING architecture in terms of modules.

Table 13.7. Modules of the PING architecture. Modules Description Entities Interfaces replicated persistent objects to the application program Replication Manages the replication of objects including life cycle and synchronisation Persistence Maintains persistence using stable storage Consistency Balances synchronisation with responsiveness Interest Manages awareness in terms of world subdivision Communication Supports message passing between processes Core Provides core services used by and linking the other modules.

260

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

Application Entity Behaviour

011

011

011

011

11

Replication

Consistency

Interest

Persistence

Network

Figure 13.8 The PING architecture.

Localisation Object Model At each process, objects are replicated by the replication service according to awareness determined by the interest service. The entities management service interfaces replicated persistent objects to the application program. It provides selective levels of transparency of distribution and replication. The replication service is responsible for the life cycle management of replicas and makes use of the consistency service to update them. The object model comprises both data objects and reactive objects. Data objects hold state information. Reactive objects are data objects that embed a reactive behaviour. Data objects may be shared and may also be made persistent. Sharable objects contain a selection of sharable attributes.

Behaviour Two forms of behaviour support are provided: reactive and reflective. Reactive objects are associated with a reactive program. Within a given process, reactive objects communicate through local broadcast of events. The reactive behaviour is defined at the application level and then replicated within the object model by the replication service. The reactive program defines triggers and responses in terms of typed events. Reflective behaviour allows the behaviour of objects to adapt to the availability and condition of computational and network resources. This facility is not core to the PING infrastructure but may be placed between it and the application as a filter.

261

Inhabited Information Spaces

Consistency The consistency service is highly configurable supporting a range of time management services. The consistency module sits below that of the replication and above that of event router and in turn above communication. Its purpose is to balance synchronisation with responsiveness and this is achieved by delaying the sending or delivery of events according to some interchangeable time management policy. Each iteration of the local simulation process is synchronised by a tick. This tick causes events held in the consistency module to be delivered to the replica according to the time management policy. Supported policies fall into two categories: non-causal and causal. Non-causal strategies are: receive order; time stamp; and predictive. Receive order simply delivers events to the replication in the order received. Time stamp delivers them in the order that they were created. Predictive delivers predicted events at the predicted time thus overcoming some effects of network latencies. The sending of predicted events may be delayed to reduce the likelihood of erroneous predictions. Causal order may be guaranteed with policies that define causality in terms of awareness or interaction. Some causal policies are based on object sequencers and so use the exchange of sequencers to provide concurrency control. More general concurrency control is offered as a core service of PING from outside the consistency module. These include read and write locking of objects using either a pessimistic or optimistic approach as well as a choice of explicit or implicit locking. Pessimistic concurrency control prevents inconsistencies whereas the optimistic approach resolves them. The former is generally better for human in the loop real time systems and is used in PING by default.

Scaling The interest management service provides support for world subdivision policies which may be defined at the application level. The role of this module is to manage dynamic grouping, determine the set of object replicas needed within the local process and informing other processes of changes in interest through the generation of events. Neither control of Level of Detail or aggregation is supported within the infrastructure.

Persistency Persistency is provided at two levels relating to static and evolutionary persistence. Static persistence is supported over stable storage and is guaranteed when all processes have exited. Evolutionary persistence maintains and evolves objects as long as one replication of them exists in any process.

262

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

Communication Discovering Objects The discovery of objects is directed by the interest management service. The replication service is responsible for the life cycle of each replica and must thus fetch an object to a process when it is originally discovered. A local caching service is provided so that an object need only be fetched once even though an interest border may be traversed several times.

011

011

Events Events are used to synchronise replica updates as well as communicate system messages. These events are synchronised by the consistency service described above. An event router service takes outgoing events from the consistency service and uses interest management to direct them to appropriate communication channels. Channels provide Application Level Framing (ALF) to map events to particular dissemination groups and qualities of service. The granularity of the ALF is that of an object. The event router maps unique object identifiers to channels using tables that are updated according to the interest management service. Various underpinning transport level protocols are used including UDP, TCP and IP multicast. SRM offers object level reliability and ordering above the latter. Channels hide all the choice of protocol from the services above. The infrastructure may be configured to implement reliability and ordering at either the consistency or communication level. The basic requirement of reliability on the communication service is that it will not deliver an incomplete or corrupted event to the consistency service.

011

011

11

13.4 Deployment IIS bring together people, possibly from distinct geographical places, into a shared information space. We have shown how the environment may be replicated across many processes and synchronised through event communication. So far we have assumed that all the machines are connected to some network of reasonable bandwidth which allows them to communicate using a combination of peer-to-peer and group communication and using varying qualities of service. Unfortunately the use of a current wide area network, such as the Internet, introduces problems that must be addressed when deploying an IIS over them. This section considers the impact of real world problems of deployment on the existing Internet. These include firewalls, modems and the lack of Multicast capability on the Internet. We consider three idealised approaches to deployment: point-to-point, tunnelled group, and hybrid.

263

Inhabited Information Spaces

Firewalls have become essential to maintain the security of corporate and academic networks connected to the Internet. A firewall restricts access to selected port numbers, protocols and remote sites. It is unlikely that an IIS process can communicate through a firewall without some adjustment or help. IIS systems may allow inclusion of users from home or mobile computers. Such computers connect to the Internet using modems which typically offer low bandwidth compared to corporate and academic networks. Furthermore, modems offer only a point-to-point connection. Multicast is supported on most local area networks (LANs) but is currently not supported on much of the Internet. This is because of problems with scaling routing strategies and global management of the address space as well as the large number of legacy routers in use.

13.4.1 Point-to-point The traditional approach to distributed processing on the Internet is based on the simple client–server model (Figure 13.9). This approach has been popular in supporting public IIS applications such as social meeting places and games. This popularity arises from the simplicity of access and reliability offered by restricting communications to point-to-point connections as well as the simplicity of security, maintenance and consistency offered by servers. Clients connect to servers that maintain the current state of the environment. Scalability is increased by mapping servers to worlds or awareness management subdivisions. Servers decide the true state of the environment and thus simplify concurrency control. Many offer persistence. Home or wearable computers connect to the Internet through modem links and Internet Service Providers. Those on LANs connect through corporate routers. Although this model is

Client Corporate LAN

Client

ISP

ISP

Server Corporate LAN

Client Client

Figure 13.9 Point-to-point deployment across the Internet.

264

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

fundamentally less scalable than those using group communication, some games applications have boasted tens of thousands of simultaneous users by mapping awareness management to servers and relaxing consistency (Ultima Online, 2003).

13.4.2 Tunnelled Group Group communications mapped to peer-to-peer distribution is generally more scalable than point-to-point (Figure 13.10). It does, however, complicate the development and deployment of an IIS system. This approach is dominant in research and defence simulation training, which both aim for optimal rather than simple solutions and, furthermore, do not make wide use of low cost modems. To use multicast across the Internet it is currently necessary to join some Multicast backbone, such as MBone (Berkeley Laboratory, 2002), or to deploy a private equivalent. Multicast backbones use an approach called tunnelling. Each connected LAN has a tunnel process that converts between multicast and point-to-point network packets. Multicast packets are captured by a tunnel process, encapsulated in IP packets, through firewalls and across the Internet, to peer tunnel processes on remote LANs that strip off the IP headers and redistribute as multicast. Private tunnels typically offer high security and low latency compared to tunnelling across public backbones. The servers can be placed at any LAN or stand-alone computer connected via a tunnel. Servers provide either initial or persistent worlds but are not generally responsible for maintaining the true state of the environment. Maintaining this true state is the responsibility of the clients with the help of distributed consistency control.

011 Tunnel Tunnel LAN Client

LAN Client Server Tunnel

011

11

Client

Client

LAN

Client

Server

Figure 13.10 Tunnelled group deployment across the Internet.

265

Inhabited Information Spaces

Client Tunnelled multicast network IISP

IISP

IISP

Client

Server

IISP Client

Client

Figure 13.11 Hybrid deployment across the Internet.

13.4.3 Hybrid A hybrid solution, pioneered in DIVE (Frécon et al., 1999) and refined in PING, is to allow private computers to link to multicast connected service providers (Figure 13.11). Let us call these IIS providers or IISPs. Tunnels link the IISPs and other servers. An IISP is responsible for converting point-to-point communication from a client into group multicast. Awareness management mapped to group addresses determines which clients, IISPs and other servers receive a given message. IISPs are positioned to minimise latency across the point-to-point link.

13.5 Conclusion Inhabited Information Spaces (IIS) situate users in a social information context. In the real world, these users may be co-located or at different geographical locations. The unique combination of IIS technology provides us with unprecedented access to information, and ways of processing, presenting, interacting and sharing it. The technology maps well to social human communication supporting not only verbal and non-verbal communication but also unprecedented communication through information objects in the environment. Both information, and the way in which users interact with and around it, must be supported in a natural and intuitive manner. This requires the issues of responsiveness, fidelity, consistency and scalability to be addressed. A multi-level architecture is required to focus these issues on representation, behaviour, synchronisation and communication. We have described the principles of supporting these issues at each level and how this is done in example systems. Deploying systems over

266

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

Communication Infrastructures for IIS 13 ● ● ● 1 ●●●●●●●●●●●●

the Internet introduces additional problems of security, bandwidth and dissemination. We have shown three idealised models of deployment to explain how these issues may be addressed for different applications and networks. For reasons of space, this chapter has focused on the support IIS that allow people to inhabit information spaces through primarily graphical interfaces. We have not discussed important systems such as COMRIS, where the emphasis is placed on the large-scale co-habitation of agents and people within information space and the primary use of audio interfaces. The underpinning technology of IIS, and particularly the communication systems, are reaching maturity. Simple networked IIS are already in daily public and commercial use. More advanced systems in research offer considerably higher levels of realism and richness. Core to this is the shared interaction with dynamic and steerable information. Most of the core principles of IIS communication are well developed and a deep understanding of the usability of such systems is being gained. An IIS communication infrastructure that addresses all the issues well is yet to emerge. The time has now come for the IIS research community to consolidate and bring together best practice at each architectural level to develop systems to a commercial standard. Distinct applications have diverse requirements and it is unlikely that one system will fit all applications for the foreseeable future. However, we are yet to achieve true realism in social interaction with information in any system. We are some way from being able to work together in an IIS without constantly thinking about the effects of the system but the light at the end of the tunnel is growing close.

Acknowledgements The author would like to thank his PhD students, particularly Robin Wolff and Oliver Otto, as well as Anthony Steed and his colleagues at UCL, Emmanuel Frécon and his colleagues at SICS, and Frederic Dang Tran and his colleagues within the PING consortium.

011

11

267

This page intentionally left blank

011

Part 6 Community

011

011

011

11

269

This page intentionally left blank

011

011

011

011

Peer-to-peer Networks and Communities

14

Mike Robinson

14.1 Introduction Inhabited Information Spaces, and associated Virtual Communities, are by definition based on some form of computing and networking technology. The social phenomena displayed are constrained and conditioned by the available underlying technologies. There is a link between application design, actions and activities available to people using the technology, and the underlying technologies. This link is the “organising concept”. It acts as a boundary object (Star and Griesemer, 1989; Kaplan et al., 1994) between the three realms. It enables a conceptual grasp of the underlying technology as it relates to, and emerges into the social world of people and activities. This chapter proposes that the organising concept of peer-to-peer is community, and that peer-to-peer thus has a special relevance to Inhabited Information Spaces. In the mid-1980s, local area networks (LANs) became associated with the organising concept of “group”. This provided a vital lever for selling a novel and expensive leading-edge technology to organisations. “Group” enabled managers to understand what they could use a LAN for – email and file-transfer between, file and printer sharing for “group” members, etc. From here, “group” also constrained and directed the genre of applications that were developed. Ideas for applications could be justified (and financed) if they “supported groups”. Group thus set up a form of positive feedback between LAN development, LAN diffusion through customer acceptance, and application design. Although “group” has correctly been much critiqued, nevertheless it led to many interesting and innovative applications and experiments in Computer Supported Co-operative Work (CSCW) – within a time frame that lasted from the late 1980s to the early 90s. We may call this period seven fat years for CSCW. As a collection, the CSCW innovations of these years met Thompson’s criteria for a major cultural change.

11

271

Inhabited Information Spaces

In his view (Thompson, 1972), the move from speech, to writing, to print effected three significant changes in the surrounding culture – a change in the ease with which stored human experience can be accessed, an increase in the size of the common information space shared by the communicants, and an increase in the ease with which new ideas can be propagated throughout society. As these features are difficult to measure directly, he proposes a “test of significance” for each as follows: 1. Must affect the way in which people index information. 2. Must increase the range of strategies open to the communicants for the interrupt act. 3. Must increase the probability of transmitting or receiving an interesting but unexpected message (Bannon, 1989). In the mid-1990s, the general focus of development shifted from LANs to the Web. Technically, the Web can be regarded as one of the major innovations of the twentieth century. At the social level it has transformed the notion of “information”. The Web has undoubtedly met Thompson’s first and (if the ratio of junk to interesting messages is discounted) third criteria. Nevertheless, we should not lose sight of what the Web did not do. The Web did not meet Thompson’s second criterion, and it did not change, add to, facilitate, or provide important tools for the ways people communicate and interact with each other. Group and CSCW software was extended (sometimes successfully) to the Web, but “groupware” was not a main focus of interest or commercial success. Generally there was a lack of richness and innovation in the Web vis-à-vis interaction between people. This is not surprising as the organising concept was “library”: one person in silence in front of a screen of information, linked to millions of other screens of information. This metaphor is neither an accurate model of real libraries (Bowker and Star, 1999) nor is it conducive to CSCW, or to virtual community (despite some heroic attempts on the VR front). It would not be an overstatement to classify the years of the Web explosion, the mid-1990s to 2001, as seven lean years for CSCW. They were also seven lean years for community, as the widespread nostalgia for the community spirit of the early Internet shows. Is the next period likely to be any better, in terms of support for interactions between people? This chapter takes the optimistic view that there could be another seven fat years. “Peer-to-peer” – better known simply as P2P – is a technology under development. It has some proven applications and some interesting promises and potential areas of application. The term P2P includes networks that expect end users to contribute their own files, computing time or other resources to the benefit of themselves and/or others. Just as interesting is the emerging organising concept: “community”. First let us admit that there are some thin notions of community knocking about. It is stretching credibility (or at least the English language) to

272

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Peer-to-peer Networks and Communities 14 ● ● 1 ●●●●●●●●●●●●●

talk about Napster or Gnutella “communities” as communities. (It can also be noted that Napster is not a genuine P2P application either, although cited in most articles). Music search and download may be popular, and it may be convenient to find the sounds on someone’s personal hard disk, not on an official server. To classify the people doing this as a “community” is beyond the current meaning of the term. But it does not do to be too didactic in the critique. Language is changing, and easily comes to include and reflect new uses. Similarly practices are not static. A community that starts as a statistical or functional aggregate may easily develop forms of interaction, and may even stabilise as a community of practice. And even if the notion of community, in some cases, is thin, this does not mean it will not be useful. The concept of “group” was often used in equally stretched and bizarre ways: consider for instance the sleight of hand in Decision Support System research that considered “nominal group” as equivalent to a face-to-face discussion This did not prevent “group” making a significant contribution to innovative and stimulating experiments in the context of the LAN. There is a better example than Napster of the way in which “community” is needed as a P2P organising concept. Peer-to-peer technology itself poses some hard technical questions that need “community” to lay out the terrain of possible answers. “Search” is an area which server-based Web technology has addressed extremely well. There are search engines to suit almost all needs. They are very powerful, and generally effective. If, however, the Internet expands to include the majority of personal hard disks as well as all the current Web servers, then the searchable space will expand about thirty times, on a conservative estimate. Such an order of magnitude change is beyond the capacity of current search engines, and, worse, it is beyond the current technology of search engines. In addition to the problem of pure scalability, there is another feature that current search technology cannot address. The hard disks in the expanded space are not “always on”. They connect and disconnect, come and go in unpredictable ways. Applying current search techniques in an intermittentconnection context would lead to a situation where at least 90 per cent of results led nowhere. This is not acceptable by any standard. How can these issues of scalability and intermittent connectivity be addressed? “Community” as an organising concept provides the necessary lever to subdivide Web space, and deal technically with the “on/off” nature of the content providers. A general scenario could run like this. The Web remains the domain of the search engine as we know it. Within this space there is one additional entity, community. Thus existing engines identify community. To search a community then becomes the work of a community search engine, again similar to those in existence today. These already (often) have the capability to search the limited subspace quickly and frequently. Thereby they do what the Web engines cannot do: maintain an up-to-date awareness of which files are currently available and which are not.

273

Inhabited Information Spaces

The practice of community can be formalised and utilised by combining both types of search engine to extend searchability into P2P space. This is driven by technical necessity. Socially, the search scenario suffers from the same thin notion of community as the earlier Napster and Gnutella examples. We need to understand how the gap between this and a richer notion of community can be bridged. This will be addressed in the following sections. The next section will address in more detail early (CSCW) inhabited information spaces. It will present some examples of applications. It will explore how the notion and practice of design for “community” can be strengthened by this part of history. The third section will explore the various meanings of P2P, and its overall directions. The fourth section will come back to the notion of community, the strengths and weaknesses of current usages, and design aspects. It will draw on the historical strengths and weaknesses of “group” as an organising concept in CSCW. We will conclude that the notions of community and P2P are symbiotic in a profound sense. P2P has no raison d’être without community, and the development of P2P networking will develop our understanding of community. Lastly, we conclude that the design project of Inhabited Information Space may be fruitfully, although not exclusively, continued in the area of P2P under a CSCW perspective.

14.2 Early Inhabited Information Spaces in CSCW 14.2.1 Rendering the Invisible Visible Many of the early inhabited information spaces in CSCW were quite real spaces. A large proportion of them were control rooms of one sort or another. Heath and Luff studied London Underground control (Heath and Luff, 1991, 1996) and a City of London trading room (Heath et al., 1993). Suchman (1982, 1997; Suchman and Trigg, 1997) studied office work, airport ground control, and “centres of coordination”. Goodwin and Goodwin (1996) studied aircraft ground control. The Lancaster Group studied air traffic control (Harper et al., 1989, 1998). Nardi et al. (1993) studied medical co-ordination in complex neurophysiological operations. Bowers et al. (1995) studied the subtleties of workflow systems on the print industry shop floor. All these authors pioneered the tradition of ethnographically informed CSCW. In less grandiose terms, basing system design on people’s day-to-day practices. There was, however, a more far-reaching social consequence of these ethnographic investigations. They changed the meaning of the word co-operation. Historically there had been a great deal of theory (much of it rather speculative) about co-operation. There was a public agreement that “co-operation is a good thing” (something we will see later

274

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

Peer-to-peer Networks and Communities 14 ● ● 1 ●●●●●●●●●●●●●

explicitly claimed for collaboration). In some extremely penetrating articles, Fairclough (1986, 1987) demonstrated, with practical examples, how co-operation meant almost anything (good) to almost everyone. It was indeed a very special sort of word, more like an absolute moral imperative than a description or an activity. Thus one of the weaknesses of practitioners (especially in organisations terming themselves co-operatives) was plenty of ideology and a lack of feel for practice (Landry et al., 1986). From within the heart of the CSCW movement arose a new understanding of co-operation in practice. Not in terms of ideology, principles and precepts, but in terms of everyday, minute-by-minute working life. Researchers such as those mentioned above rendered the previously invisible minutiae of co-operation visible. They showed that co-operation was not an external to the work process, and had little to do with organisational rules about equality. Co-operation was that through which the work process was constituted. It was central and could not be deleted without deleting the work itself. Co-operation was thus regrounded as a very normal activity, as a part of what it is to be human (a social animal). Co-operation ceased to belong to the world of moral abstraction, and took its place at the heart of everyday activity. To this author it seems very likely that the activities and investigations around embedding P2P in community and community in P2P will result in a similar seismic displacement and development of the notion of community. Hence a certain level of optimism about the coming period for inhabited information spaces beyond the confines of the Web. To try and give some idea of the creativity in the fat years of CSCW, I have chosen three examples. All of them were experimental, and none are especially well known. Each of them is especially well suited to P2P technology. Examples of virtual reality IISs can be found in Chapters 2, 3 and 4.

011

011

11

14.2.2 ClearBoard In a series of papers, Hiroshi Ishii and co-workers developed ClearBoard (Ishii et al., 1992; Ishii and Kobayashi, 1993). The initial metaphor was a simple glass screen between two people (Figure 14.1). They could both draw on the screen and both could see what the other was looking at. This was a breakthrough in the concept of video communication. It was possible to attend to “the work” and at the same time see what part of the work the other was attending to. Such awareness is a precondition of co-operation. It is a great pity that, 10 years later, most designers of videoconference systems still have not learned this simple lesson. The final version of ClearBoard (Figure 14.2) was rather sophisticated. It was networked, and used a large screen angled like a drawing board. By various tricks of video reversal, both the writing and the partner’s gaze were the correct way round for both parties (think about it!).

275

Inhabited Information Spaces

Figure 14.1 The ClearBoard prototype. Reproduced with permission from Hiroshi Ishii.

Figure 14.2 ClearBoard. Reproduced with permission from Hiroshi Ishii.

14.2.3 Feather, Scent and Shaker: Supporting Simple Intimacy In 1996, at the end of the CSCW cusp, Strong and Gaver (1996) of the Royal College of Art presented a demonstration of some simple devices to support intimacy (Gaver, 2002). The word love hovers in the background. The scenario was that one partner, thinking of the other, picks up a framed photo of them. The movement of the photo triggers a simple pulse over the Internet. Somewhere else, another continent, another country, the beloved is at home. In the corner, decoratively, is a waist high, slim glass tube. As the first partner thinks wistfully of the other, a feather floats softly up the tall glass tube, and hovers, fluttering, near the top. Sadly no video grabs are sufficiently clear to reproduce here, and the reader will have to rely on words for the images. These ideas were never taken up in CSCW since they are somewhat outside the idea of “work”. But what better application could P2P develop for acceptance at the most fundamental and intimate level of community?

276

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

Peer-to-peer Networks and Communities 14 ● ● 1 ●●●●●●●●●●●●●

14.2.4 Gesture Cam: The Nodding Robot Kusuoka and co-workers work in the field of remote instruction. They faced the normal difficulties when supporting instruction in a complex engineering context. The instructor (remote) needed to know the layout of the equipment he was dealing with in real time; what the student was doing and what they were attending to; and the instructor needed to be able to point out physical items such as pulleys and knobs, and be able to indicate the direction they should turn in. Undaunted by the impossibility of the requirements, the researchers produced GestureCam (Kuzuoka et al., 1994; Yamazaki et al., 1999). This was fundamentally a small robot with an eye (camera) and a finger to point with (laser). See Figure 14.3. The robot’s joints and motors were synchronised with an identical twin that the instructor could control in order to look about and point in the remote location. It suffices to say here that the experiments were successful. Both instructors and students were satisfied. Three issues are of special interest here. First, this is an ideal P2P application. There is no reason why servers should be holding files that are only of interest to the parties concerned. Second, just as CSCW had to do, P2P needs to broaden its scope from PCs, and look at other devices, including person surrogates (Buxton, 1993) and robots. Third, returning to the specifics of GestureCam, the interactions took on a specially rich character that the designers had not anticipated. For instance, in order for the instructor to see the direction of the student’s gaze, and the item they were looking at, the little robot had to turn frequently from one to the other. Sometimes the instructor would ask if the student had understood, and the student would nod. Reciprocally (although it was visually dysfunctional for it to do so) the robot learned to nod to the student as an affirmative answer. Then, at the end of the session (and outside the design parameters), the robot and the student would do a little bow to each other, a gesture of farewell and mutual respect (Figure 14.4).

011

11

Figure 14.3 GestureCam. Reproduced with permission from Hideaki Kuzuoka.

277

Inhabited Information Spaces

Figure 14.4 GestureCam and a student. Reproduced with permission from Hideaki Kuzuoka.

14.3 P2P Themes and Overall Direction Peer-to-peer technology can be confusing (Red Herring, 7 May 2001). As already noted, P2P includes all networks that expect end users to contribute their own files, computing time, or other resources to the benefit of themselves and/or others. The notion is set to become even wider, since the P2P Working Group merged with the distributed computing community (Global Grid Forum) in April 2002. Most research and companies in the P2P area specialise in either distributed file sharing or in distributed computing. There are many well-known examples of systems to support (mainly music) file sharing: Napster, Gnutella, Limewire, KaZaA, Morpheus, Grokster, and others. Less well known are these companies seeking to exploit the businessto-business potential of P2P interactive and collaborative file sharing. Groove is the largest, claiming 200 partners including Microsoft in its development program. The keyword for these B2B applications is collaboration. The Groove Website explicitly says “Collaboration is Good”. It focuses on supporting small group and cross-enterprise collaborations, and, importantly, emphasises collaboration in context. Here it says: Collaboration rarely happens in a vacuum. People interact with each other within the context of well-defined and ad hoc business processes, using the content in which they are swimming all day. Process and content “surround“ these daily business activities (http://www.groove.net/pdf/ backgrounder-product.pdf).

Figures 14.5 and 14.6 show two typical CSCW scenarios involving coordination – but not as naively conceived. All the actors (office workers, journalists) are immersed in their own work, as can be seen from their gaze directions. At the same time, and not so obviously, they are mindful of the activities of others, synchronising with them, and being ready to change action in mid course should it be needed. In these areas of

278

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Peer-to-peer Networks and Communities 14 ● ● 1 ●●●●●●●●●●●●●

011 Figure 14.5 Office work in Greece.

011

011

011

11

Figure 14.6 Community of journalists in Finland.

collaboration, P2P appears to have learned these lessons of CSCW, and is targeting many of its application areas: There are tools in Groove Workspace for sharing content (files, images, maps), having conversations around that content (discussions, instant messages, live voice, text-based chat), and working together on shared activities (real-time co-editing and co-viewing of documents, co-browsing, group project management and tracking, meetings management) (ibid.).

The overlap with CSCW is not surprising since the founder of Groove is Ray Ozzie, one of the creators of the popular Lotus Notes Program, which itself incorporates many findings from CSCW. In an interview, Ozzie outlined the origin of his P2P vision: The epiphany of sorts was that I kept watching my daughter, Jill, doing her homework with her friends over [AOL Instant Messenger ], and my son Neil playing Quake [a search-and-destroy game that can be run on a multiplayer network] with his online friends.

279

Inhabited Information Spaces

In watching Neil in particular, I found that Quake was an immensely effective collaborative environment for a shared task: His team had to “capture the flag” of the other team. It used every bit of horsepower of the PC and network to help each player be efficient and effective at that one task. In business, we commonly have projects that require multiple people to selforganize and solve problems. But why are we stuck using e-mail, when technology is being used to serve these kids so much more effectively? (Kharif and Salkever, 2001).

On the other side, there are resource sharing companies and researchers. The principle of resource sharing is to use idle PCs for distributed processing. P2P connections tap cheap, under-utilised processing power and bandwidth. Proof of concept came from David Anderson at the University of California at Berkeley. He recruited 3.2 million volunteers, each of whom downloaded a small program that parses radio telescope data as part of a massive search for extraterrestrial life. The program runs while the volunteers’ PCs are idle and sends the processed data back to Anderson every time they log on. The network created in this way has the processing power of 3.2 of IBM’s $100 million ASCI White supercomputers. Several companies are basing distributed processing for corporations on this experiment: for instance, United Devices, DataSynapse and Entropia. Like the file share systems, distributed processing ranges from the practical to the utopian. One especially beautiful idea was put forward by Todd Papaioannou, CEO of Distributed Artificial Life Inc. He suggested simulating the Indian Ocean on millions of desktop computers and other devices. Each screen simulates a tiny part of the Indian Ocean. Virtual wildlife – autonomous digital organisms with their own needs – swim from desktop to desktop of their own free will. Users create fish and turn them loose in the virtual sea – where they can pass through other computers, PDAs, cell phones, or any other Java-enabled device. Papaioannou hoped to use this experience as a stepping stone to creating large-scale distributed P2P games. So far we have laid out what any book chapter or news article says about P2P. Namely, that under a general heading of distributed computing there are three generic types of application: file sharing; coordination support; and distributed processing. What unifies these three diverse application areas as a single research field? Certainly not the technology, which is diverse, often proprietary and application specific. Certainly not the idea of a network: the idea of machines acting as both server and client is as old as the Internet itself, and pre-Web, was the usual state of affairs. What unifies P2P is the organising concept of community. A quick scan of P2P texts and commentaries shows “community” to be the key concept. It is often used instinctively and unreflectively and there is rarely a recourse to definition.

280

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Peer-to-peer Networks and Communities 14 ● ● 1 ●●●●●●●●●●●●●

14.4 Design for Community: Inhabited Information Spaces Before we design for community, it is as well to look at some definitions, uses and issues connected with Inhabited Information Spaces (IIS). The term IIS originated in the early to mid 1990s in the context of “mixed reality” applications. These combined multi-person Virtual Reality into non-VR environments, such as television (Benford et al., 1999a), or (real life) offices (Robinson et al., 2001; Büscher et al., 2001) or theatres (Benford et al., 1997b). As the number of IIS projects blossomed (e.g. the i3 network), the term took on a more generic meaning: any social or human activity domain or set of domains with a technologically mediated information structure or system. There was much research interest in embedding new information technologies (e.g. VR, hand-held) in existing technological (e.g. television) or physical (e.g. theatre, neighbourhood) spaces. “Community” is widely used in IIS research, but with differences from the uses in P2P. In P2P, as we have seen, community is the organising concept, without which the area fragments into disparate elements. Community is core to understanding the technology, the designs, the social domains, and the linkages between them. In IIS, community is more often a description of the target domain for a particular technology. The technology itself is usually prior, and given externally to the domain. The suggestion in this chapter is that IIS should learn the lesson of P2P, reconceiving community as a technical and design concept, as well as a social criterion. At best this would enable the co-evolution of technologies and social domains. At worst, there is little to lose. Before looking more closely at IIS and P2P design issues, we need to explore the strengths and weaknesses of “community” more closely.

14.4.1 Communities: An Aside on Definitions In a seminal and programmatic paper for CSCW, Schmidt and Bannon (1992) deconstructed the idea of “group” as used to define co-operative work. Since much of what they say has direct relevance for, or can be extended to the idea of community, it is worth reflecting on some of their points. With respect to groups, they argue that “the very notion of a ‘shared goal’ is itself murky and dubious”. This is prima facie even more relevant to community, despite the fact that “community goal” is often used in the literature. There may be partial objectives for some members of a community some of the time. Common goals cannot be used to define communities. “Cooperative ensembles” say Schmidt and Bannon “are either large, or embedded within larger ensembles . . . [they are] often transient formations. Membership . . . is not stable and often even

281

Inhabited Information Spaces

non-determinable. Cooperative ensembles typically intersect. Cooperative work is distributed physically, in time and space [and] logically, in terms of control, in the sense that agents are semi-autonomous . . . Cooperative work involves incommensurate perspectives (professions, specialities, work functions, responsibilities) as well as incongruent strategies and discordant motives. There are no omniscient agents . . .” All of these observations are pertinent for the deconstruction of naïve notions of community. Communities have neither common perspectives, nor motives, nor goals, nor strategies. Membership is often nondeterminable, and boundaries intersecting, arbitrary and transient. Activity is physically and logically distributed over shifting eddies of temporary alliances. There are no omniscient agents to co-ordinate, set the agenda, or even to write the history afterwards. If there is a rule, it is simply that everything and anything can be contested, and that the dialectics of what gets contested and how (and what does not) set a trajectory that may become apparent with hindsight. Whatever general definition of community is offered, it is easy to come up with a counter-example. This is partly because the term has many meanings: my Shorter Oxford Dictionary provides five meanings, each with several variants. It is partly because we have a very incomplete understanding of what community is. Tony Salvador (1998) makes some similar points in the excellent, brief article “Communities, Schumanities”. He exhibits three attempts to define, then build software to support communities. He shows that the definitions of community cover too little and too much, managing to be under and over inclusive at the same time. Even more trenchantly, he remarks “In no way that I can tell does their definition influence their design”. Two definitions are cited with approval in the article: Mark Jeffrey, whose experience is not in academia but rather in business, perhaps has the optimal definition of community: “a group of individuals, typically geographically dispersed, who communicate electronically in an on-line environment for some common purpose of activity”. If only he left off the bit about common purpose, he’d have a fairly harmless and thus beneficial definition. In fact, in his talk, [Mark Jeffrey] argues that “anything that allows people to get together can be a community building tool’, which for design purposes seems to be on the right track.

14.4.2 Communities: An Aside on Use Most people manage to use the word community without becoming ensnared in definitional questions. In this section, we will look briefly at some everyday uses in one P2P article. The choice is arbitrary, but has been influenced by the inherent interest of the article, which is well worth reading outside the linguistic perspective.

282

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Peer-to-peer Networks and Communities 14 ● ● 1 ●●●●●●●●●●●●●

Minar and Hedlund (2001) provide an outline of the use of peer-topeer models through the history of the Internet. Their position is that the Internet was peer-to-peer for most of its history, between 1969 and 1995. It was an inhabited information space: a medium for communication for machines that shared resources with each other as equals. Their argument is that today’s P2P applications could learn a great deal from the protocols and methods of previous distributed Internet systems like Usenet and the Domain Name System (DNS). The article contains much interesting analysis. For instance, how Usenet’s NNTP protocol avoids a flood of repeated messages – which Gnutella would do well to learn from – and how DNS distributes its load naturally over the whole network, so that any individual name server need only serve the needs of its clients and the namespace it individually manages. The article also displays a sharp understanding of the issues that have led to firewalls, dynamic IP addresses, and finally Network Address Translation (NAT), all of which present serious obstacles to P2P and potentially to IIS applications. These obstacles give rise to a host of more or less unsatisfactory work-arounds – the most common being the abuse of port 80 (supposedly the port that allows simple Web access). The following extracts from the article illustrate some everyday ways in which community may be used sensibly and constructively. The first extract includes a rather strange, but perfectly understandable use: Since 1994, the general public has been racing to join the community of computers on the Internet, placing strain on the most basic of resources: network bandwidth.

The article repeatedly emphasises the critical importance of non-automated, human control of networks as the most effective but nevertheless flawed management method. The beauty of Usenet is that each of the participating hosts can set their own local policies, but the network as a whole functions through the cooperation and good will of the community . . .

but Usenet has been enormously successful as a system in the sense that it has survived since 1979 and continues to be home to thriving communities of experts. It has swelled far beyond its modest beginnings. But in many ways the trusting, decentralized nature of the protocol has reduced its utility and made it an extremely noisy communication channel.

The next extract shows a relation between community and accountability, and is a good example of the way community and P2P technology co-evolve (rather than, as in much IIS work, taking the technology as a given).

283

Inhabited Information Spaces

A key challenge in creating peer-to-peer systems is to have a mechanism of accountability and the enforcement of community standards. Usenet breaks down because it is impossible to hold people accountable for their actions. If a system has a way to identify individuals (even pseudonymously, to preserve privacy), that system can be made more secure against antisocial behavior. Reputation tracking mechanisms . . . are valuable tools here as well, to give the user community a collective memory about the behavior of individuals.

The notion of community develops to include technical and social meanings of community standards and collective memory. The shared communication channel of news.admin allows a community governance procedure for the entire Usenet community. These mechanisms of local and global control were built into Usenet from the beginning, setting the rules of correct behavior. New breed peer-to-peer applications should follow this lead, building in their own social expectations.

Thus we see the unreflective, but entirely appropriate usage of the notion of community in one article on P2P. Moreover, the postulated control mechanisms for P2P (as did the search mechanism outlined earlier in this chapter) depend crucially on a practical notion of community. In Chapter 15, Burkhalter and Smith discuss uses of social accounting data in Usenet. Various data are tabulated for newsgroups, such as number of messages, number of postings per author, numbers of responses to each posting, etc. These statistics are available to Usenet participants. Various reactions are noted, from strong suspicion about the interest of Microsoft in sponsoring such information to strong interest in the profile of the newsgroup: its place in the activity league of newsgroups, and participants’ own rankings as “posters”. Burkhalter and Smith use the terms “online community” and “newsgroup” more or less interchangeably. Part of the interest of the chapter is that gathering “social accounting data” depends on an assumption of community – yet when the data is taken up by participants it plays a role in transforming the assumption into a reality. This simple example shows the need in IIS for the technical (the social accounting system) to co-evolve with the social (Usenet community). Although co-evolution may be the authors’ intention, this is not made clear in the chapter – where the technology is treated as prior and given. This brings us to some philosophical considerations.

14.4.3 Communities: An Aside on Philosophy Before assessing the roots of IIS in community, and some implications, we need to delve into a little philosophy. Earlier we claimed that our understanding of community was incomplete. Using a word correctly

284

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

Peer-to-peer Networks and Communities 14 ● ● 1 ●●●●●●●●●●●●●

does not guarantee there is a corresponding referent in the world. “Unicorn”, “The Jabberwok”, and “the present King of France” can all be used correctly but have no referent. More interesting than pure fictions are objects that can only be seen through a glass darkly: complexes in process whose dynamics and attributes can never be directly inspected, and which can only be known through representations: e.g. (Lynch, 1991a, 1991b). We will never, to paraphrase Sartre (1965) meet a community “in person”. The representations we construct of it, consciously or unconsciously, will largely determine our attitudes, actions and, in the case of P2P and IIS, our designs. All we really know directly is that the idea of community is meaningful, that it does have serious ramifications in the way we mould our lives and on the paths we follow, and that it (whatever it is) is being changed by networking technologies. Until our experience is more complete, and our representations more adequate, then Salvador’s prescriptions for a “fairly harmless and thus beneficial definition” seems most likely to avoid logical blunders. But this is unsatisfactory from the point of view of those who wish for a closer understanding, a better representation of community in order to build more appropriate community support networks. Philosophy can help a little. We need representations in order to design applications and systems. We need to take design and implementation actions on the basis of the representations. But we also need to be able to suspend belief in our representations, to maintain flexibility. It helps with this difficult, even contradictory task, if we take a consciously philosophical position, and regard community as a metaphor. Richard Rorty cites Nietzche’s famous remark that truth is a mobile army of metaphors. He explicates this in a late twentieth-century context by saying: I take its point that sentences are the only things that can be true or false, that our repertoire of sentences grows as history goes along, and that this growth is largely a matter of the literalization of novel metaphors (Rorty, 1991, p. 3).

Later he expands this view of metaphor, saying: A metaphor is, so to speak, a voice from outside logical space, rather than an empirical filling up of a portion of that space, or a logical-philosophical clarification of the structure of that space. It is a call to change one’s language and one’s life, rather than a proposal about how to systematize either (ibid., p. 13).

011

11

Following Rorty, we may regard community as a metaphor. The repertoire of sentences we can articulate using it are growing and will grow with time. This explains our inability to produce useful definitions, combined with our appropriate use of the term in context. It also explains the attraction and excitement of the idea: there is a lot more to find out. This will not be achieved by attempts at “logical-philosophical

285

Inhabited Information Spaces

clarification”, but by active exploration. This is the point at which we may be able to understand the role of P2P in community, and the role of community in P2P and IIS much better. Jeffrey was right to argue that “anything that allows people to get together can be a community building tool”. P2P, and the family of community building tools it will probably lead to in IIS, will help us unroll that set of sentences that develop our understanding of what community is. In an epistemological sense, community comes into being as a result of ongoing and deepening interactions framed by the intention of build or support community. Hence the difficulty of providing useful definitions and representations – road maps of continents where roads have not yet been built. Conversely, without the metaphor of community as an organising concept, P2P loses its coherence and falls apart into diverse technologies and applications with little obvious application or raison d’être. P2P and community are deeply symbiotic. This type of deep co-evolution is not yet a feature of IIS. There is an obvious case that it should be.

14.5 P2P, Community and the Design of Inhabited Information Spaces The previous sections have shown that “community” is deeply involved with P2P, but its use is uncritical, unreflective, intuitive and often utopian. None of this disqualifies the concept once its metaphorical nature is understood. “Group” and “co-operation” were similarly flawed, yet played a key role in the development and acceptance of CSCW applications. The lack of tight definition combines with fluent, natural and intuitive usage to mean that “community” can be a common currency between developers of different strains of P2P and IIS. It can act as a boundary object: plastic enough to adapt to local needs and constraints of the several parties employing them, yet robust enough to maintain a common identity across sites . . . weakly structured in common use, and become strongly structured in individual-site use (Star, 1992).

Thus the loose but common notion of community in P2P is a source of dialogue, new insights and (undoubtedly) plausible mistakes. All of which will help the development of the field. In addition, ethnographic investigation of community will help to further our understanding of what community actually is. It may even result in a paradigm shift similar to that for co-operation in CSCW: the understanding of “co-operation” moved from the structural-ideological to the second-by-second ubiquitous process underlying most work activity. But is P2P the appropriate technology for the design of Inhabited Information Spaces? It would be extremely foolish to argue this from a

286

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Peer-to-peer Networks and Communities 14 ● ● 1 ●●●●●●●●●●●●●

technical point of view. The functionalities of different types of IIS demand different architectures in order to function. Within this book, Roberts (Chapter 13) shows the advantages and disadvantages of different architectures in terms of the compromises that each implies for applications built on them. He also shows how different application genres have different priorities to be useful. For instance, the more immediately responsive an application needs to be, the less detail can be transmitted. Nevertheless, from a design point of view, it can be argued that P2P is the most appropriate technology for IIS – despite limitations or awkwardness of functionality for any particular application. These will turn out to be fewer than we might anticipate, since P2P is not a single architecture, but an evolving set of architectures. P2P is the technology that offers the most potential for IIS because it is the technology that offers the most interesting problems within the conceptual frame of community. In doing so, it is likely to result in solutions that bootstrap our understanding of community itself. IIS cannot be defined by a specific technology, any more than CSCW can be defined by a specific technology (Schmidt and Bannon, 1992). However, CSCW was conditioned, constrained and located within a technology framework of the LAN. Similarly, I would argue, IIS is best located within a technology framework of P2P – but not of course any specific P2P technology. This means no more (and no less) than that every participant’s machine is a server as well as client. At a conceptual level, within the sociological framework of community, it is quite obvious that each person, each community member, is a source as well as a sink for information, viewpoint, prejudice, emotion, action, and so on. It does not seem too much to ask that their mode of connectivity (machine) to an IIS should support being a source as well as a sink. In other words, that the IIS network should be a P2P network. Some guesses about how this process might go as follows. Some of us remember how in CSCW the ideological notion of co-operation resulted in bad social engineering and very troublesome applications. Similar things are likely to happen with community. A definitional approach, eschewing metaphor, will result in confusion, as illustrated by the earlier sections. We need to remember how real progress was not made in CSCW until the insights of ethnographers – called in as fire-fighters and consultants when applications were not being accepted, and it was not obvious to the designers why – showed that co-operation was a momentby-moment business, found almost everywhere where people interact, and having very little to do with ideology. The guess is that a similar process will happen with community. Grand sociology, rooted in searches for roles and rules, grand schemas and plans is unlikely to do any better than it did in CSCW. Probably community will be found in the moment-by-moment interactions between distantly related, even unrelated in any direct way, people.

287

Inhabited Information Spaces

The secret of co-operation in groups lies with history, mutual knowledge and learning from experience. Almost all the co-ordinations described by, for instance, Heath and Luff, or by Suchman and coworkers, or by Button, Bowers and colleagues, are deeply skilled. The skill is a result of learning over time with the same people in the same (usually physical) context. Community is prima facie a different context, since the ongoing interactions between the same people in the same place or context over long periods of time is absent. The interesting interactions will be between relative strangers. They may know each other in some way, yet lack an interpersonal history. That such people can interact, and interact in such a way that is mutually beneficial, or that can be labelled “the kindness of strangers”, is in need of empirical and detailed examination – not theoretical explanation. If these micro-processes can be studied, and, at least to some extent, understood, then it may be possible to start building community-supporting applications based on them. Inhabited Information Spaces will only be inhabited if there is a reason for people to inhabit them. Cities are attractive because they offer rich opportunities that are lacking in villages and the countryside generally. But cities are not built by plonking a large pile of “opportunity” in the middle of a field. “Something” starts, and that “something” replicates and mutates. That something is a form of interaction. Design for community, which is, in all but name, design for IIS, needs by analogy, to start with simple supports for interaction, as recommended by Jeffrey (in Salvador, 1998) and Burkhalter and Smith (Chapter 15).

14.6 Concluding Remarks This chapter has argued that computer support for human interaction blossomed from the mid-1980s to the mid-1990s. There were seven fat years. Most of the advances and insights and novelty of CSCW – the research field on supporting co-operation – were generated in this period. Similar evidence could be drawn from other fields devoted to supporting forms of human interaction, such as HCI or Groupware. Then came the Web. While this was a massive advance in universal access to information provision, it did little to further support for interaction. The support applications that were generalised from the mid-90s to date (e.g. email, instant messaging, shared editors and whiteboards) were developed much earlier. The chapter argued that this change was not primarily a matter of intention, but of the underlying technologies and organising concepts. The early period was based on the spread of LANs, with the organising concept of group providing coherence and a trajectory that supported interaction. The later period was based on client–server Web technologies, and the organising concept of the library (most e-shopping takes

288

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

Peer-to-peer Networks and Communities 14 ● ● 1 ●●●●●●●●●●●●●

place in library-like structures). Neither the technology nor the organising concept were conducive to developing new ways to support human interaction. The emergence of the new field of P2P, with its organising concept of community, gives hope of another seven fat years for supporting human interaction – but this time centred on community rather than group. The chapter argued that community plays a central role in the very coherence of P2P. It is technically necessary for the development of appropriate search engines, and for maintaining overall control. It is organisationally necessary, since it provides a common perspective, direction or ambition for otherwise diverse, file-sharing, collaborative and distributed processing applications. It is suggested that community should play a similar role in IIS, where it currently impacts the social, but not the technical. Conversely the technical attempts to provide community support are likely to result in interesting developments in, and far greater understanding of communities. The conclusion is to try to move both P2P and “community” to the heart of IIS. There is much still to discover. That is what makes it exciting.

011

011

011

11

289

This page intentionally left blank

011

011

011

011

11

Inhabitant’s Uses and Reactions to Usenet Social Accounting Data

15

Byron Burkhalter and Marc Smith

15.1 Introduction Usenet and other text interaction systems are inhabited by populations of tens of millions of active contributors and potentially even larger populations of invisible observers. While text systems lack the explicit representation of bodies common to many graphical spaces, most text interfaces represent individuals as a corpus of messages over time. With effort using existing tools, users reading collections of individual messages can piece together a distinct sense of the many participants in particular and the character of different spaces in general. New systems are extending the basic features of text interfaces to introduce measures and maps of spaces like newsgroups and email lists as well as their populations of contributors. Summary reports and visualisations of individual participants and spaces can be produced from the analysis of collections of messages exchanged in text spaces bringing new resources for users and contributors to these environments. How do participants make use of enhanced context about text spaces and the contributors and conversations with whom they interact? In the following we report examples of the ways social accounting data are incorporated into threaded conversations created in Usenet newsgroups. Netscan (http://netscan.research.microsoft.com) is a research project that generates and publishes extensive social accounting data about the public Usenet. Netscan contains information on each of more than 103,000 newsgroups and 20 million unique authors who contributed messages since the end of 1999. In effect Netscan can take masses of conversational data and render a series of summary metrics that can be reflected back directly or as a way to select content to the readers of and contributors to Usenet newsgroups. Thirty-eight thousand unique visitors have used the Netscan web site since the start of 2000. Some of these users go

291

Inhabited Information Spaces

on to discuss the Netscan service or the data it reports by posting messages in Usenet itself, in some cases including URLs pointing to specific reports in the system or copying segments of the reports published on the Netscan site directly into messages posted to their favourite newsgroup. We have discussed interface components and visualisations as well as the value of social accounting data for selecting content from newsgroups in other papers (Fiore et al., 2001). Here we want to explore how social accounting data are used by those who participate in Usenet. We do not address how people have used the Netscan web site itself but rather how they have made use of Netscan data by posting it or making references to it in and as the context of their particular newsgroup. Our intent is to explore the ways new representations of social context are made use of in the very spaces they represent and to document the ways these representations of participants and places are used by and between participants in these spaces. Graphical representations of bodies and geometry are not the only way information spaces are inhabited. Groups of people who exchange simple ASCII text through online conversation systems like email lists and Usenet newsgroups, also come to inhabit an information space. The approach of the Netscan project is to provide tools that support social awareness in text interaction spaces like Usenet by combining measures of each newsgroup, author and thread’s activity with data visualisation and reporting interfaces that present these patterns back to community participants and interested observers. The underlying assumption is that social accounting measures of the activity of newsgroups, authors, and conversation threads can be used as a social resource providing context in support of social processes related to boundary maintenance, status contrasts between individuals and newsgroups, and the characterisation of conversation partners. To assess the impact of publishing social accounting data back into each space we examined all mentions of the Netscan web site in Usenet messages. Since the Google Groups (http://groups.google.com, formerly Deja News) service has saved and indexed large collections of Usenet messages dating back to the first years of Usenet in 1981 we were able to search the contents of millions of Usenet discussion threads for mention of Netscan and its data. Using this tool it is simple to retrieve the collection of messages containing a particular set of words, including those containing a particular web URL address. We searched for fragments of our system’s URL (“netscan.research”). Our search through April of 2003 found 255 threads containing 7,430 messages starting in June of 1997 with the bulk (87per cent) occurring since 2002. These messages were reviewed and the forms of social use they were put to is discussed in the next section. This strategy provided a broad general understanding of the types of messages that contained Netscan social accounting metadata and allowed us to see how the data was used in various ways in the selected discussion threads.

292

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Inhabitant’s Uses and Reactions to Usenet Social Accounting Data 15 ● 1 ●●●●●●●●●●●●●●

15.2 Related Work Several related systems visualise patterns of activity of authors and conversations. These systems focus on the representation of the social history of online conversation spaces and the members of their contributing (and in some cases observer) population.1 PeopleGarden (Xiong and Donath, 1999), for example, visualises message boards as a collection of flowers where each flower represents a user in the conversational space and its petals represent his/her postings. The ordering of petals and the saturation of each petal indicate time – a message posted far in the past is less saturated than one posted more recently. Finally, the height of each flower represents how long a user has been active in the message board. Even though PeopleGarden’s focus was web-based discussion boards and not Usenet newsgroups, it represented conversational spaces in terms of its participants’ histories. The Loom project focused on visualising social patterns within Usenet newsgroups. It highlighted patterns that indicated varying patterns of participation, for example highlighting rowdy, vociferous users as well as the number of participants in different threads over time. Although its focus was not on the authors per se, Loom managed to uncover interesting author dynamics found in newsgroups – for instance the marked difference between the average numbers of participants per thread in technical versus social newsgroups. A related project, Conversation Map (Sack, 2000), represents authors present in Usenet newsgroups as nodes in a social network in which patterns of reply form the links between contributors. It computes a set of social networks detailing who is talking to whom in the newsgroup, which visualises the centrality degree of each of the authors in the newsgroup. The system also analyses the text of messages to uncover sets of “discussion themes”. Here, as in Loom, remarkable patterns emerge that are related to people’s interactions in the conversational space. Babble (Erickson et al., 1999) is a similar effort to visualise the behaviour of groups of people interacting through a networked conversational system, in this case a propriety message board system in use at a major corporation. The system attempts to provide a form of “social translucence”, a rapidly graspable representation of the state and pattern of interaction of people participating in the space. Babble represents participants in a collection of chat/message boards as circles within a series of concentric rings which indicate how recently they have been active. 1

Newsgroup threads are not face-to-face conversations (see Garcia and Jacobs, 1999). The insights that informed this research are the notion of recipient-design, which references the way that orderly coherent conversations are constructed by both the messages that explicitly mention the Netscan data and the responses to that message.

293

Inhabited Information Spaces

15.3 Netscan Netscan generates and publishes social accounting data about the Usenet. It provides a web interface for information about the relationships and activities of the three major elements of Usenet: the newsgroups themselves, the authors and the threads their messages create through patterns of turn and reply over time. Netscan collects messages but unlike a search engine extracts and retains only the FROM, NEWSGROUPS, DATE, MESSAGE-ID, LINES and REFERENCES headers from each message. Using these message elements Netscan creates aggregations of the multiple dimensions of the Usenet over time. Message bodies are retained for a few weeks or months but are not permanently stored. The main Netscan interface takes a keyword and matches it against the name of newsgroups in Usenet instead of against the content of their messages. It then displays a report on the number of messages each matching newsgroup received in the given day, week or month selected. In addition, the number of authors (also referred to as “posters”) is listed along with the number of those authors who had also posted in the prior time period (called “Returnees”). Measures of the way each newsgroup links to others is presented in the form of the count of the total number of messages “crossposted” (shared with another newsgroup) and the count of the total number of other newsgroups that are linked by even a single message. From this report users can access a more focused report on a single newsgroup in a selected period of time. This “report card” displays daily activity in the newsgroup and its change in activity in terms of the number and type of messages and participants over the prior time period. These reports are very macro level and address some of the overall structure and dynamics of the newsgroup. The newsgroup report card also displays two examples of content selected from newsgroups on the basis of the size and structure of the threads and the behaviour pattern of authors who contributed messages to them. The “thread tracker” reports the 40 largest threads in terms of the number of messages that were added to the chain of turns and replies in the selected time period. This report, therefore, displays the most active and possibly most controversial topics in the newsgroup. A related report, the “author tracker”, selects content by the behaviour of the authors who contributed to them. The report lists 40 authors in the newsgroup selected by the number of different days each author contributed at least a single message in the time period selected. In some newsgroups the most active authors contribute messages nearly every day in a month and have consistently done so for many months or even, in rare cases, years. This measure, therefore, represents a kind of “costly signal” – a hard to falsify quality that is a relatively reliable indicator of the tenure of the contributor in the newsgroup. If users select a listed author the Netscan system displays the ten threads to which

294

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

Inhabitant’s Uses and Reactions to Usenet Social Accounting Data 15 ● 1 ●●●●●●●●●●●●●●

the author contributed the most number of messages in the time period selected. This way the threads that most attracted the contribution of the most dedicated participants are easily accessed.

15.4 Findings Within newsgroups discussions of individual participants is quite common. In a study of soc.culture newsgroups, evidence suggested that discussants often attempt to categorise other discussants in order to “spin” the arguments in terms that are more preferable (Burkhalter, 1999). In newsgroup arguments one may be able to “win” an argument simply by successfully categorising the opponent in discrediting fashion. Certainly in newsgroups where advice and other forms of help are offered, advisee’s have a need to understand the type of person with whom they are dealing in order to assess information that they are likely to be unable to evaluate on its face. The massive variety of newsgroups and respondents makes it very difficult to make these crucial judgments with confidence, however, without significant investment of effort over time as a pattern of messages is pieced together into a overarching picture of the author and an assessment of how reliable, trustworthy and valuable they are. The most common resource for categorising other participants is to consider the content of their most recent post. A more effective means of understanding who is whom within the newsgroup comes from longstanding newsgroup members who provide an institutional history. The problem is identifying longstanding members. Social accounting data is an alternative that goes beyond listing all the messages that contain the keyword or author name used as a query. For example, data on which newsgroups an author has participated within, the number of posts per time period in each, the first and last date of posting for a particular address, the number of threads initiated and how often their message generates a response, are all ways of typifying a particular author through structural data. As a result social accounting data provides easy access to the most prolific and longstanding newsgroup members as well as finding the newcomers or visitors from other discussion spaces.

15.4.1 Social Accounting Data and Author-assessment Threads

011

11

The existing interfaces to Usenet and related discussion spaces present an overwhelming welter of individual messages from a potential population of thousands or tens of thousands of posters. The diversity of participants and the limits of most Usenet interfaces create a situation in which people, accustomed to the richness of face-to-face interaction, have difficulty typifying co-participants. Typifications are a regular

295

Inhabited Information Spaces

practice in interaction and are used to formulate responses and reactions appropriate to the social status of the participants and the setting (see Burkhalter, 1999; Sudnow, 1972). In other words, participants seek to put other participants “in context” in order to conduct each subsequent interactional turn. Thus, a common occurrence in newsgroup messages is a search for information about particular authors. For example: . . . Do any of you know Toni, the guy who told you who the perpetrator is? Is he a regular here? . . .

The post above speaks to an important type of newsgroup participant, “the regulars”, who have long-standing relationships to the newsgroup. Regulars are replied to more often than new members. Given the disproportionate amount of messages regulars produce, in a very real sense, what regulars do in their messages is what the newsgroup, effectively, is. Regulars can be important for a number of reasons. Regulars in “Q&A/help” newsgroups have presumably answered a number of questions and the accuracy and helpfulness of their answers may be validated by their continued presence and the absence of repudiation. Regulars are also an important sign of what the normal procedures are for the newsgroup. Regular authors, even those who fill large controversial threads, even in negative and hostile roles, are part of the normal operations of the newsgroup. The point is not that a person by virtue of being a regular is virtuous, the point is that by being a regular the person is a known quantity – for better or worse. The importance of known quantities should not be overlooked. Knowing the nuances of another’s identity is a necessary for competent responses. Messages can be evaluated through social accounting data about their authors. For example, the fact that an author is a relative newcomer or is a prolific poster to hundreds of other newsgroups is made visible by reports from the Netscan reports on that author. While such measures are open to interpretation (an identity with only recent activity could be the result of a newcomer or a previously active user seeking to create a new identity and lose the baggage of a prior one) these histories, however fragmentary, combine to offer a picture of the social context of newsgroups, authors and threads. Those who post in undesirable newsgroups can be seen as outsiders or visitors. Thus, information about an author’s position in the social structure of a newsgroup in particular and Usenet in general can be used to characterise particular posters. In effect, Netscan’s author profiles are reputation measures creating a definition of a particular poster as, for example, prolific or troublesome: Most of politics belong in national local news groups. Foreign people can participate on existing local discussions. But [POSTER A] is posting in 115 news groups and initiates foreign issues. And just like politicians he suffers from shortness of life. He only started to use the name “[POSTER A]” begin September 2001

296

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

Inhabitant’s Uses and Reactions to Usenet Social Accounting Data 15 ● 1 ●●●●●●●●●●●●●●

[link to Netscan statistics] October 2001 [Posts Replies FirstSeen ThreadsTouched Other NG’s [A] 721 554 09/03/2001 336 58 ]

The particular thread excerpted above involves a call for [POSTER A] to cease posting messages not relevant to the newsgroup’s topic. For unmoderated newsgroups (which most are) this is a serious issue since anyone can post anything to any newsgroup at anytime. A reoccurring practice of newsgroup maintenance involves “patrolling” the newsgroup for “offensive posts”. This may include admonishing those who are advertising, asking questions already answered in a FAQ guide, posting a large number of messages or, as in this case, posting irrelevant and offensive messages. However, the message excerpted above does not draw out all the potential conclusions possible from the data utilised. The message specifically mentions that the address in question has only been in use for a short time. Researchers and participants must remember that messages are signed with an address that may or may not correlate to a particular person. The fact that an address is new might suggest that the participant is new; it might also suggest that a poor reputation was connected to a previous name. For this reason measures beyond the length of participation are important. For example, the number of posts and the ratio of initial turns (posts that start a new thread) to replies (posts that are responses to previous messages) can help to distinguish between a person who over-posts (also known as spam) to a newsgroup, in which case the number of initial turns is likely to be large in proportion to the number of replies which contrasts with someone who is an active participant who contributes replies to other poster’s messages. An author is further placed in context by reference to their crossposting data. At the extreme, a large number of newsgroups also suggest spamming behaviour, which might be evidence of a troublemaker or advertising or other “outsider” work. Participants are aware of these implications. Some messages expressed a concern that their own author profile might be misinterpreted: I went to this [Netscan] URL and it worked (I’m using IE 6). searched myself, and apparently I’ve been posting in groups I don’t post in. May have been accidental crossposting in replies, though. But I couldn’t figure out if there was x-no “damage”. I couldn’t get to the actual posts. Nor did I find a way to look up stuff I know I x-nayed. Oh well.

011

11

Many users of Usenet do not examine the cross-posting header line of the messages they reply to which frequently means that they “inherit” the cross-posting pattern of the initial message. As a result, their replies are not confined to the particular newsgroup in which the responding author found the message. In fact, users may be inadvertently crossposting to ten, twenty, or more newsgroups. Thus the cross-posting data

297

Inhabited Information Spaces

may not be completely indicative of posting intent; however, frequent posting to a particular newsgroup does suggest intentional participation. Stray cross-posts, as indicated by few or even just one message, may be entirely unintended or accidental but that does not mean that others will accept that account. This is important because where a person posts is taken to reveal information about the person. Participants who examined their own profile constituted half of all author-oriented posts. And a large portion of these messages concerned perceived inaccuracies in the reports of cross-posting: that’s freaky! – try checking out user profiles. i see a whole load i’ve never been near (unless i’ve contributed to some crossposted thread) such as alt.obituaries,uk.gay.london(!),soc.culture.pakistan(!!)

The social accounting data included in author-oriented threads frequently involves discussions of the reputation of participants and/or a person’s position within a particular newsgroup. This data provides a sense of whether a person is known to other (established) members of the newsgroup. Like message archives, social accounting data provides a way of tracing an individual identity’s activity over time. However, social accounting data places the individual identity in multiple contexts at once. The date the identity was created (the date of its first posted message), the relation of this identity to other newsgroup authors, the amount of cross-posting and the names of other newsgroups touched by the individual name, along with the frequency of posting within each newsgroup, helps to establish the position and history of each author. This context can be used to characterise and typify participants and allow other participants to design responses appropriate to their status. Noting variation in the number of each author’s contributions was the most common use of social accounting meta-data. For many newsgroups this was a one-time occurrence. However, a fair number of newsgroups have made this a monthly practice. Monthly or even weekly rankings were a regular part of the proceedings, at the end a winner is declared: Ah, yes, the results for the whole of March are available. I’ve slipped into 19th place (uh-oh) but the top ten are: . . . [results deleted] Dom Robinson wins the Queen Mother Memorial Award for Most Helpful Poster

As with newsgroup comparisons (discussed below), the most frequent author comparison is in terms of the number of posts per member. Following these posts were self-evaluations accounting for the different positions won, eight short examples follow: I’m at a lowly 40!! Okay, who won in October? I am hardly surprised I posted the most in December, as I was unemployed the entire month.

298

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Inhabitant’s Uses and Reactions to Usenet Social Accounting Data 15 ● 1 ●●●●●●●●●●●●●●

I think my trivia posts have inflated my position a little The question is though, will I be the Xmas No. 1 . . . Damn I’ll have to try harder next time, I know I can make the top 3! not me i aint even made the list, not even 5 posts?????? Ahh, there i am in 6th :o) I was so close to breaking the top ten :(

However, not everyone in the newsgroup can appropriately justify their position. One poster suggested that more work was necessary to raise their own number of posts, a rebuke came quickly simply stating “who are you?” This is more than a putdown; it is in many respects the point of these “competitions”. They are competitions over who has the requisite status to participate in the newsgroup as full members. A person might rank as the fortieth ranked poster, yet still be able to join this discussion because they have a certain standing in the newsgroup. Although these messages focus on the quantity of posting by particular authors, the issue is not one of characterising a particular author but with a comparison between authors who frequently contribute to the conversations in the newsgroup. Quite often the posting of comparative social accounting meta-data created a discussion of participation in the newsgroup. Occasionally people would express disbelief in the numbers, some took pride in the quantity of messages they posted, and others used the opportunity to claim that they were actually the most prolific newsgroup contributors. On numerous occasions, parties apologised for their lack of participation. While it may seem that more is better, the quantity issue is not clear-cut: Average 7 per day for the last seven day (and that’s counting a busy Thurs jousting with Virt). You’re ahead of me this weekend and you’ve yet to say anything interesting, just the usual name calling.

As this example illustrates being the most prolific is hardly a guarantee of being beloved by all. We find that those who post frequently become the objects of discussion and attention themselves. There is status involved in being the object of discussion (even if the discussion is largely negative). Interesting stuff, looks like they put Tracker as the number one contributor this month, Little ‘ol me didn’t even make the top 40, but you have to post stuff for that, right? ;p Yes posting helps but as in all things quality as opposed to quantity is whats really important. Leaves you know who you know where.

Social accounting data provides an objective justification for doing the work of acknowledging participants, an important function of maintaining interactional newsgroups. Different participants occupy particular positions within the newsgroup. Newsgroups maintain their cohesion

299

Inhabited Information Spaces

when their populations agree to a definition of the situation, including who is a member and who is not. Social accounting data allows roles connected to participation to be continually re-established through objective measures. This can be done informally by members; however, the objective data curtails arguments about the measures (although such arguments may also have a function within the newsgroup). Instead the discussion immediately moves to the characterisation of participants. These characterisations rely on the increasing complexity of social accounting data beyond the counting of posts: Did you know that on 9th of September 2001, the Flonk got 1202 messages? The daily average in September was 713. The favorite crosspost destination was alt.fan.karl-malden.nose, with about 67% of the traffic crossposted there. The top five was rounded out by alt.flame, alt.usenet.kooks, alt.flamenet and alt.fan.scarecrow. The busiest poster was anonyme with 2334 messages, not counting the numerous morphs. That’s well over 70 articles a day. OpI was second with 1897, and Dave Hillstrom third with 333. mimus started the most threads, 40 in all. That’s all mimus does, is start threads. Then he sits back and cackles while people jump all over them and get into fistfights and stuff. He’s a Bad Man.

Those who make the list in terms of participation may be open to characterisation along different metrics, in this case the number of initial posts (messages that create new threads). Another member uses the longest post for the month to suggest another category won by that member: . . . You’ll reportedly want to use IE for best results (big surprise). For those interested in such information as: During the month of November, alt.coffee had: 4014 posts from 556 posters with an average line count of 26 3457 were replies from 438 repliers 96 posts went unreplied, and 76 were crossposted 245 posters were returnees from the previous month, while 241 were drive-bys Woohoo! I am NOT the mostestposter for the month! (Barry was!) But I am tops in most stimulating conversations created . . . uh “Threads Initiated”. :P Warning signs (a thread I started – go figure) had the most traffic -83 posts. This thing is cool. Too bad there’s no easy way to see how much reader traffic the ng has . . .

Along similar lines, those who initiate without garnering responses can be seen as holding a lower status within the newsgroup: Well you are top with “most posts”, but you fall down because many of your posts remain unreplied to and you don’t start a lot of your own threads, which is what the “chart” is calculated on, I think.

300

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Inhabitant’s Uses and Reactions to Usenet Social Accounting Data 15 ● 1 ●●●●●●●●●●●●●●

15.4.2 Social Accounting Data and Newsgroup-assessment Threads The overwhelming focus of Usenet messages using Netscan data focused on newsgroup level metrics. For the purposes of this analysis we will separately discuss those messages where social accounting data is used to primarily discuss single newsgroups from those where multiple newsgroups are compared. “Intra-newsgroup assessments” contain messages where some perceived characteristic of the newsgroup is discussed by reference to social accounting data. This might include a newsgroup that seemed to by dying or one which has experienced a sudden large influx of messages. The second type, inter-newsgroup assessments, includes those where a potential participant is searching for an appropriate newsgroup among a collection of potential newsgroups using social accounting data. In each case, social accounting data is used to gain a perspective not possible from within existing newsgroup browsers which simply present long lists of available messages. Usenet consists of at least 103,000 groups which makes finding a suitable newsgroup a daunting task. Social accounting metrics allow searching for a newsgroup by properties of the patterns of interaction within each newsgroup. The excerpt below is an example: Hey, I just used this tool for this news group http://netscan.research.microsoft.com and I see that the majority of threads here are related to talking about one console vs. the other, and the most posts and replies come from someone who maintains a FAQ for the news group, and there are some dedicated regulars and some amount of trolling. Can any of you frequent posters tell me the scoop with this news group? Looking back at a week of messages here, am wondering if there is enough substance here to pay it close attention. It seems to be a lot better than the alt. newsgroup, and am willing to put up with some noise to find out what people are doing with XBox.

This poster details characteristics that, for them, comprise a worthwhile newsgroup: a particular topical range and focus, a group of regulars and few troublemakers or “trolls”. Social accounting data allows this participant to see the newsgroup from a perspective otherwise difficult or impossible to assemble manually. From this vantage point an informed choice about which newsgroups to join can be effectively made. Central to the question of newsgroup health appears to be the issue of an adequate base of regular participants: Anyone good at looking at stats from newsgroup software? How many regular posters do we have in chi.general? Just wondering . . . . netscan.microsoft.com (doesn’t work in Netscape, surprise!) December stats: 3,485 posts from 260 posters, 106 of which were returnees (posted the month before) November: 2,424 posts from 259 posters, 101 were returnees

301

Inhabited Information Spaces

October: 2,521 from 338, with 106 returnees September: 3,470 from 358, 98 returnees I’d say we have about 100 regulars.

The primary benefit for users (particularly users new to Usenet or to a particular newsgroup) is the ability to enter discussion space not as a complete neophyte but having already acquired a perspective on the newsgroup (see Lawley, 1994). The number of posts and the ratio of replies to total messages can allow new users to characterise the population of the space, identifying prolific users with different patterns of contribution. Conversely those who build and promote a particular newsgroup may orient to this feature and concern themselves with social accounting data as a sign of a newsgroup’s health. The “regulars” within a newsgroup are certainly concerned with the social accounting data related to cross-posting and replies: Of course this means poor uk.media.tv.misc gets lumped with thousands of off-topic posts a week . . . . but I don’t think anyone really cares any more.

Regular newsgroup participants often express deep concern about their newsgroup; it takes effort to post over long periods of time as many regulars do. Many newsgroups have existed for a decade or more and members often express concerns about the health and vitality of their newsgroups. Social accounting data can be used to gain a perspective on the health and vitality of newsgroups. In the following example, a poll where participants vote on their favourite rollercoaster had fewer participants in the current year than in the past. An initial post expresses concern over the declining participation: The message below is in reply to that concern: My assumption is that there are actually fewer people participating in roller coaster forums (or websites) when (February) there are not many roller coasters available to be ridden. Is there statistical data (posts/day etc.) available to prove or refute my assumption? In any case, that would not explain why fewer people participated in the 2001 poll than in the 1999 poll (which was also in February). That month had roughly 2000 more posts than Jan 2002. Jan 2000 was slightly higher than Jan 2002, but lower than Feb 2000. Both 2000 months were corrupted slightly by what appears to be a slight feed loss in both months. Both of these months had average and maximum posting days quite a bit higher than Jan 2002. So . . . it’s entirely possible that 2000 just had more posters than 2002. And Mitch, October 2001 was less than either Feb 2000 or Jan 2002, for what that’s worth.

The social accounting data accounts for the declining participation based on a model where the availability of roller coasters corresponds to the participation in the newsgroup. Thus, the concern over participation is eased.

302

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Inhabitant’s Uses and Reactions to Usenet Social Accounting Data 15 ● 1 ●●●●●●●●●●●●●●

Newsgroup health and vitality is also established by comparison to other newsgroups. Newsgroups were most commonly contrasted in terms of the number of posts each received, for example, the two excerpts below: wow, I like this site, thanks Kent look up windowsxp.general it did 34,000 posts in Oct. next one is only 20,000 Autism averages in the top 20 “alt.support” groups. Alt.support.depressed seems to have 4 times the number of posts as any other group, over virtually any period of time, which is a bit of a sad statement about our world.

011

011

011

011

11

Other messages engaged in more complex formulations, adding discussions of interactivity such as replies and poster to post ratios. Here are two examples: Yeah, there is a separate mostly dead newsgroup for Bollywood films. It’s not all that dead – I did a indian.research. Indian.ft.com report on alt.asian-movies and rec.arts. movies.local.indian, and the Indian group had about 78% of our number of posts, 40% of our number of posters and about the same number of replies that we had (that ought to mean their threads runlonger). I put in “motorcycles” in the search field and out of the *37* motorcycle related newsgroups the top three were: (totals derived from Nov 1, 2001 to present): 1. uk.rec.motorcycles total posts: 19,546 with 780 unique posters 2. aus.motorcycles total posts: 8,262 with 781 unique posters *and* 3. rec.motorcycles.harley total posts: 7,349 with 787 unique posters which beat out our nr 4 newsgroup: 4. rec.motorcycles total posts: 6.965 with 838 unique posters. What does all mean? Hell if I know, but it seems that LESS folks in the rmh groups post MORE than the rm newsgroup. That and those Brits on the other side of the pond are chatty as hell. Probably because of useless crap like I just posted.

This is not merely an opportunity for a structural perspective; many of these messages (and those in the next section) are explicitly competitive. The notion that members on one newsgroup “beat” another newsgroup on some dimension comes across in many of the messages. This contrast presents an opportunity to characterise outsiders and insiders, to express the difference between “us” and “them”. The drawing of newsgroup boundaries by the establishment of difference is a crucial aspect of newsgroup identity and maintenance. Thus the ability to compare newsgroups becomes an opportunity to cast structural differences as evidencing the moral differences between newsgroups. Thus the account of different structural outcomes is explained by a certain “chattiness” that distinguishes one newsgroup from another. In the next example, a comparison of cross-posted messages in a fitness newsgroup becomes an opportunity to affirm membership:

303

Inhabited Information Spaces

Newsgroup Shared MSGS % Shared Total Neighbors: 67 [distinct groups] 1282 42 % #1 Neighbor alt.sport.weightlifting 741 57.8 % #2 Neighbor misc.fitness.misc 342 26.68 % #3 Neighbor sci.med.nutrition 332 25.9 % #4 Neighbor uk.rec.bodybuilding 143 11.15 % #5 Neighbor alt.fitness.weights 132 10.3% The only really suprising thing is that MFW appears to rate #1 or #2 on every measure of activity. I was always attracted to it by the high energy level – but now I know why it takes more time to read about weights than actually to lift them.

However, social accounting data is not only an opportunity to cheer for one’s own team. Indeed as the complexity of the data increases, the potential interpretations of that data become increasingly complex. The next excerpt features social accounting data about the number of posts, the number of posters, the poster-to-post ratio, the number of returnees to a newsgroup, the average number of lines in each message, the number of replies and repliers, the number of unreplied to messages and the number of cross-posted messages and cross-posted newsgroups. All of this brought out a discussion not simply of the pros of the newsgroup, but also the cons: OK, I’m sure this will be garbled, but here’s a comparison of ASA to our more genteel counterpart, RBC for November: P—Ps—PP—Rt—ALC—R—Rrs—S—UnR—X-Xp 3 #A 6225 181 .03 95 26 5799 137 80 113 39 37 #B 2258 445 .20 172 33 2015 354 218 57 640 900 We had triple the posts, but less than half the posters – a much more active group. Our linecount was less, probably caused by one very terse poster whom I won’t name ([name omitted]). One other item stands out – more than a quarter of [Group B’s] posts were crossposted, with 900 targets! What’s going on there?? [Group A] had very few crossposts. For the personal stats, [name omitted] was the most consistent poster, posting every day in November! But she only initiated 3 threads. Actually, as a group we were very consistent – 30 people posted 14 days or more during the month. RB started the most threads, followed by MC and [name omitted]. Since CKW defected to AS, MC is on vacation, and RB is preoccupied, the number of interesting threads has diminished of late. Content is harder to establish since so many of our threads meander. Amazingly, the great anchor chain debate was less than 10% of the posts. Beverages seemed to hold second place. Very interesting! This is more a “club” group. These stats suggest that the “company” is good! Hmmm. Now I see that as meaning we have the same people posting again and gain with less to say of anything with any depth. It seems that rec.boat.cruisng has a broader base of contributors with something other than chit-chat to post. RBC tends toward the serious side of boating subjects and do not tolerate funnybusiness . . . they kicked ole’ McNeal off months ago.

304

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

Inhabitant’s Uses and Reactions to Usenet Social Accounting Data 15 ● 1 ●●●●●●●●●●●●●●

The structural perspective in comparative mode precipitates a discussion of how wonderful and club-like the newsgroup is and also how off topic subjects are pursued. No doubt such discussions can be irritating to members, more consequentially they can lead newsgroups to splinter. However, a newsgroup’s interaction style and topic evolves over time, and the structural data provides an opportunity for these topics to be broached.

15.5 Conclusion Netscan social accounting data is applied both to newsgroups and participants to distinguish between useful newsgroups and those that are noisy or fractious and between authors who may be regulars or interlopers, nice people or not so nice, and those authors who have quality responses as well as a quantity of responses. Through these and other structural measures newsgroups and their members are typified, categorised and understood from a perspective not possible (or excessively costly to manually construct) through a simple archive of messages. Social accounting tools present the historical and sociological tracks of the newsgroup and are used to perform functions that seem extremely similar to those used by offline groups and organisations. Social accounting data is not merely useful for our understanding of Usenet newsgroups but may be becoming a vital and commonly used tool by the members of these kinds of discussion spaces themselves.

011

011

11

305

This page intentionally left blank

References

011

011

011

011

@Man (2000) Web Ad Blocking Under Linux/Unix, BeOS, MacOS and Windows. http://www.ecst.csuchico.edu/~atman/spam/adblock.shtml Activeworlds (2003) Activeworlds Maps, http://www.activeworlds.com/ community/maps.asp Adler, D. (1996) Virtual Audio: Three-Dimensional Audio in Virtual Environments. Swedish Institute of Computer Science (SICS), Internal Report ISRN SICS-T—96/03-SE. Amazon.com (2002) http://www.amazon.com/webservices Appelt, W. (1999) WWW Based Collaboration with the BSCW System, In SOFSEM’99, Springer Lecture Notes in Computer Science 1725, Milovy, Czech Republic, pp. 66–78. ATI (2003) Wireless Solutions. http://www.ati.com/products/builtwireless.html Badler, N., Palmer, M. and Bindiganavale, R. (1999) Animation Control for RealTime Virtual Humans. In: Communications of the ACM, 42 (8): 64–73, ACM Press. Bannon, L. (1989) Shared Information Spaces: Cooperative User Support Networks. Mutual Uses of Cybernetics and Science, Amsterdam, 27 March–1 April, University of Amsterdam. Barker, R. (1968) Ecological Psychology, Stanford University Press, Stanford. Barrus, J. W., Waters, R. C. and Anderson, D. B. (1996) Locales: Supporting Large Multiuser Virtual Environments. IEEE Computer Graphics and Applications, 16(6), 50–57. Bartle, R. (1990) Early MUDHistory, http://www.ludd.luth.se/mud/aber/mudhistory.html Begole, J. B., Tang, J. C., Smith, R. B. and Yankelovich, N. (2002) Work Rhythms Analysing Visualisations of Awareness Histories of Distributed Groups, In Proceedings of the ACM 2002 Conference on Computer-Supported Cooperative Work – CSCW 2002, ACM, New Orleans, LO, pp. 334–343. Benford, S. and Fahlén, L. E. (1993a) A Spatial Model of Interaction in Large Virtual Environments. Paper presented at the 3rd European Conference on Computer Supported Cooperative Working, Milan, Italy. Benford, S. D. and Fahlén, L. E. (1993b) Awareness, Focus, Nimbus and Aura – A Spatial Model of Interaction in Virtual Worlds. Paper presented at the HCI International 1993, Orlando, FL. Benford, S., Snowdon, D., Greenhalgh, C., Inrgam, R., Knox, I., and Brown, C. (1995a) VR-VIBE: A Virtual Environment for Co-operative Information Retrieval, Computer Graphics Forum 14(3) (Proceedings of Eurographics ’95), 30 August–1 September, NCC Blackwell, pp. 349–360.

11

307

References

Benford, S., Bowers, J., Fahlén, L E, Greenhalgh, C., Mariani, J. and Rodden, T. (1995b) Networked Virtual Reality and Cooperative Work, Presence 4(1): 364–386. Benford, S., Brown, C., Reynard, G. and Greenhalgh, C. (1996) Shared Spaces: Transportation, Artificiality and Spatiality. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW’96), Boston, pp. 77–86, ACM Press. Benford, S., Greenhalgh, C. and Lloyd, D. (1997a) Crowded Collaborative Virtual Environments. In Proceedings of ACM CHI’97, Atlanta, GA, USA, March 1997, pp. 59–66. Benford, S., Greenhalgh, C., Snowdon, D. and Bullock, A. (1997b) Staging a Public Poetry Performance in a Collaborative Virtual Environment. In J. Hughes, W. Prinz, T. Rodden and K. Schmidt (eds.), Proceedings of the Fifth European Conference on Computer Supported Cooperative Work – ECSCW’97. 9–11 September, Lancaster, England. Kluwer Academic Publishers, Dordrecht, pp. 125–140. Benford, S. D., Snowdon, D. N., Brown, C. C., Reynard, G. T. and Ingram, R. J. (1997c) Visualising and Populating the Web: Collaborative Virtual Environments for Browsing, Searching and Inhabiting Webspace, In JENC’97 – Eighth Joint European Networking Conference, Edinburgh. Benford, S. D., Brazier, C.-J., Brown, C., Craven, M., Greenhalgh, C., Morphett, J. and Wyver, J. (1998) Demonstration and Evaluation of Inhabited Television, eRENA Deliverable 3.1. Benford, S., Greenhalgh, C. et al. (1999a) Broadcasting On-Line Social Interaction as Inhabited Television. In Bodker, S., Kyng, M. and Schmidt, K. (eds.), Proceedings of the Sixth European Conference on Computer Supported Cooperative Work – ECSCW’99. 12–16 September, Copenhagen, Denmark. Kluwer Academic Publishers, Dordrecht, pp. 129–198. Benford, S., Bowers, J., Craven, M., Greenhalgh, C., Morphett, J., Regan, T., Walker, G. and Wyver, J. (1999b) Evaluating Out of this World: An Experiment in Inhabited Television, eRENA Deliverable D7a.1. Benford, S., Norman, S. J., Bowers, J., Adams, M., Row Farr, J., Koleva, B., Rinman, M.-L., Martin, K., Schnädelbach, H. and Greenhalgh, C. (1999b) Pushing Mixed Reality Boundaries, eRENA Deliverable D7b.1. Benford, S., Bederson, B., Åkesson, K., Banyon, V., Druin, A., Hansson, P., et al. (2000) Designing Storytelling Technologies to Encourage Collaboration between Young Children. In Proceedings of ACM CHI’00, The Hague. Berkeley Laboratory (2002) Introduction to the MBone Distributed Systems Department Collaboration Technologies Group at Ernest Orlando Lawrence Berkeley National Laboratory, http://www-itg.lbl.gov/mbone/ Billinghurst, M., Karo, H. and Poupyrev, I. (2001) The MagicBook: A Transitional AR Interface. Computer Graphics, 25: 745–753. Bly, S. A., Harrison, S. R., and Irwin, S. (1993) Media Spaces: Bringing People Together in a Video, Audio, and Computing Environment, Communications of the ACM, 36(1): 28–47. Blunck, A. (1998) The World Generator – The Engine of Desire, an Interactive Installation by Bill Seaman, eRENA Deliverable. Bowers, J., Button, G. and Sharrock, W. (1995) Workflow from Within and Without: Technology and Cooperative Work on the Print Industry Shopfloor. In Marmolin, H., Sunblad, Y. and Schmidt. K. (eds.), Proceedings

308

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

References

● 1 ●●●●●●●●●●●●●● of the Fourth European Conference on Computer Supported Cooperative Work – ECSCW’95. 10–14 September, Stockholm, Sweden. Kluwer Academic Publishers, Dordrecht. Bowers, J., Pycock, J. and O’Brien, J. (1996) Talk and Embodiment in Collaborative Virtual Environments. In Proceedings of CHI ‘96, pp. 58–65. Bowers, J., Hellström, S.-O., Jää-Aro, K.-M., Söderberg, J., Bino, H. P. and Fahlén, L. E. (1998a) Constructing and Manipulating the Virtual: Gesture Transformation, Soundscaping and Dynamic Environments for Extended Artistic Performance, eRENA Deliverable 2.2. Bowers, J., Hellström, S.-O. and Jää-Aro, K.-M. (1998b) Making Lightwork: The Algorithmic Performance of Virtual Environments. In Bowers et al. (1998a), pp. 6–20. Bowers, J., Norman, S. J., Staff, H., Schwabe, D., Wallen, L., Fleischmann, M. and Sundblad, Y. (1998c) Extended Performances: Evaluation and Comparison, eRENA Deliverable D2.3. Bowers, J. and Jää-Aro, K.-M. (1999) Blink: Exploring and Generating Content for Electronic Arenas. In Hirtes et al. (1999), chapter 6. Bowers, J., Hellström, S.-O. and Jää-Aro, K.-M. (1999) Supporting Event Management by Sonifying Participant Activity. In Hirtes et al. (1999), chapter 4. Bowers, J., Jää-Aro, K.-M., Hellström, S.-O., Lintermann, B., Hoch, M., Drozd, A., Taylor, I. and Whitfield, G. (2000a) Production and Management of Events in Electronic Arenas, eRENA Deliverable 4.5. Bowers, J., Jää-Aro, K.-M., Hellström, S.-O., Hoch, M. and Whitfield, G. (2000b) Production Support Tools for Electronic Arenas: Using Tangible Interfaces for Media Editing. In Bowers et al. (2000a), chapter 3. Bowers, J. (2001) Crossing the Line: A Field Study of Inhabited Television, Behaviour and Information Technology 20(2): 127–140. Bowker, G. and Star, L. (1999) Sorting Things Out: Classification and its Consequences. MIT Press, Cambridge, MA. Boyd, D., Lee, H-Y., Ramage, D. and Donath, J. (2002) Developing Legible Visualizations for Online Social Spaces, 35th Annual Hawaii International Conference on System Sciences, Vol. 4, p. 115. Institute of Electrical Engineers. Brave, S., Ishii, H. and Dahley, A. (1998) Tangible Interfaces for Remote Collaboration and Communication. In Poltrock, S. and Grudin, J. (eds.), CSCW ‘98 Computer Supported Co-operative Work, AACM Press, Seattle, pp. 169–178. Brooks, Jr, F. P. (1999) What’s Real About Virtual Reality?, IEEE Computer Graphics and Applications, 19(6): 16–27, IEEE. Brown, B., MacColl, I., Chalmers, I., Galani, A., Randell, C. and Steed, A. (2003) Lessons from the Lighthouse: Collaboration in a Shared Mixed Reality System. To appear in Proceedings of the ACM Computer–Human Interaction (CHI03), Fort Lauderdale. BSXMUD (1994) http://www.lysator.liu.se/mud/bsxmud.html Bullock, A. (1997) Inhabiting the Web: Highlights from a Series of VR Meetings, Video Proceedings Fifth European Conference on Computer Supported Cooperative Work (ECSCW ‘97), 7–11 September 1997, Lancaster, UK. Bullock, A. and Gustafson, P. (2001) The VITI Program: Final Report, SICS Technical Report T2001:02, March 2001, ISSN 1100–3154.

309

References

Burgoon, M., Hunsaker, F. G. and Dawson, E. J. (1994) Human Communication (3rd edn). SAGE Publications, London. Burka, L. P. (1995) http://www.apocalypse.org/pub/u/lpb/muddex/mudline. html Burkhalter, B. (1999) Reading Race Online, Communities in Cyberspace. Routledge, London. Büscher, M., Krogh, P., Mogensen, P. and Shapiro, D. (2001) Vision on the Move: Technologies for the Footloose. Appliance Design 1(1): 11–14. Button, G. (1992) The Curious Case of the Vanishing Technology. In Button, G. (ed.), Technology in Working Order: Studies of Work, Interaction and Technology. Routledge, London, pp. 10–28. Buxton, W. A. S. (1993) Telepresence: Integrating Shared Task and Person Spaces. In Baecker, R. M. (ed.), Readings in Groupware and Computer Supported Cooperative Work: Assisting Human-human Collaboration. Morgan Kaufmann, San Mateo, CA, pp. 816–822. Card, S., Mackinlay, J. and Schneiderman, B. (1999) Readings in Information Visualization: Using Vision to Think. Morgan Kaufmann. Carion, S., Beylot, P., Magnenat-Thalmann, N., Emering, L., Raupp Musse, S. and Thalmann, D. (1998) Mixed Reality Dance Performance, eRENA Deliverable 2.1. Castro, M., Druschel, P., Kermarrec, A.-M. and Rowstron, A. (2002) SCRIBE: A Large-scale and Decentralised Application-level Multicast Infrastructure. IEEE Journal on Selected Areas in Communications (JSAC) (Special issue on Network Support for Multicast Communications). Chalmers, M. (1991) Seeing the World through Word-Coloured Glasses. In Proceedings of the Second International Conference on Cyberspace, University of California Santa Cruz. Chalmers, M. and Chitson, P. (1992) Bead: Explorations in Information Visualisation. In Proceedings of the ACM Conference on Information Retrieval (SIGIR’92), Copenhagen. Published as a special issue of SIGIR Forum, June 1992, ACM Press, pp. 330–337. Chalmers, M. (1993) Using a Landscape Metaphor to Represent a Corpus of Documents. In Proceedings of the European Conference on Spatial Information Theory, Elba. Chalmers, M., Rodden, K. and Brodbeck, D. (1998) The Order of Things: Activitycentred Information Access. In Proceedings of the World Wide Web (WWW98), Brisbane. Published as Computer Networks and ISDN Systems, 30: 359–367. Chalmers, M. (1999) Comparing Information Access Approaches. J. ASIS 50th Anniversary Issue, 50(12): 1108–1118. Chalmers, M. (2002) Awareness, Representation and Interpretation. Journal of Computer Supported Co-operative Work 11: 389–409. Chase, P., Hyland, R., Merlino, A., Talant, A., Maybury, M. and Hollan, R. (1998) Semantic and Content Visualization. Coling-Acl 98 workshop: Content Visualization and Intermedia Representations, Montreal, Canada, August 1998. Chen, Y., Katz, R.-H. and Kubiatowicz, J.-D. (2002) SCAN: A Dynamic Scalable and Efficient Content Distribution Network. In Proceedings of the International Conference on Pervasive Computing (Pervasive 2002), Zurich, Switzerland.

310

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

References

● 1 ●●●●●●●●●●●●●● Churchill, E. F. and Snowdon, D. (1998) Collaborative Virtual Environments: An Introductory Review. Virtual Reality: Research Developments and Applications, 3: 3–15. Churchill, E. and Bly, S. (1999) It’s All in the Words: Supporting Work Activities with Lightweight Tools. In Proc. Group ‘99, Phoenix, AZ, November 1999. Churchill, E.F., Snowdon, D. and Munro, A. (2001) Collaborative Virtual Environments. Digital Places and Spaces for Interaction. Springer Verlag, London. Churchland, P. M. and Churchland, P. S. (1998) On the Contrary: Critical Essays 1987–1997. MIT Press, Cambridge, MA. Colebourne, A., Mariani, J. and Rodden, T. (1996) Q-PIT: A Populated Information Terrain. In Proceedings of Visual Data Exploration and Analysis III, San José. Craven, M., Benford, S., Greenhalgh, C. and Wyver, J. (2000) Third Demonstration of Inhabited Television, eRENA Deliverable 7a.3. Craven, M., Taylor, I., Drozd, A., Purbrick, J., Greenhalgh, C., Benford, S., Fraser, M., Bowers, J., Jää-Aro, K.-M., Lintermann, B. and Hoch, M. (2001) Exploiting Interactivity, Influence, Space and Time to Explore Non-Linear Drama in Virtual Worlds. In Proceedings of CHI 2001, pp. 30–37. Cuddihy, E. and Walters, D. (2000) Embodied Interaction in Social Virtual Environments. In Proc. ACM CVE 2000, San Francisco, September 2000. Donath, J. (1999) Identity and Deception in the Virtual Community, in Communities in Cyberspace. Routledge, London. Donath, J., Karahalios, K. and Viégas, F. (1999) Visualizing Conversations, 32nd Annual Hawaii International Conference on System Sciences. Institute of Electrical Engineers. Dourish, P. and Bellotti, V. (1992) Awareness and Coordination in Shared Workspaces, In Turner, J. and Kraut, R. (eds.), CSCW 92 – Sharing Perspectives, ACM Press, Toronto, Canada, pp. 107–114. Dourish, P., Adler, A., Bellotti, V. and Henderson, A. (1996) Your Place or Mine? Learning from Long-Term Use of Audio-Video Communication. Journal of Computer Supported Co-operative Work, 5(1): 33–62. Drexler, E. (1992) Nanosystems: Molecular Machinery, Manufacturing and Computation, John Wiley & Sons, New York. Drozd, A., Bowers, J., Benford, S., Greenhalgh, C. and Fraser, M. (2001) Collaboratively Improvising Magic: An Approach to Managing Participation in an On-Line Drama. In Proceedings of ECSCW 2001, pp 159–178, Kluwer. Dyck, J., and Gutwin, C. (2002) Groupspace: A 3D Workspace Supporting User Awareness. In: Extended Abstracts of CHI 2002, Minneapolis, MN, pp. 502–503, ACM Press. Edelman, G. and Tononi, G. (2000) Consciousness: How Matter Becomes Imagination. Allen Lane Penguin Press. Electronic Arts (2003) Ultima Online, http://www.uo.com/ Erdelez, S. (1999) Information Encountering: It’s More Than just Bumping into Information, Bulletin of the American Society for Information Science, 25: 25–29. Erickson, T., Smith, D. N. and Kellogg, W. A. (1999) Socially Translucent Systems: Social Proxies, Persistent Conversation, and the Design of “Babble”. Proceedings of the Conference on Human Factors in Computing Systems. ACM Press, New York.

311

References

Evard, R. (1993) Collaborative Networked Communication – MUD as Systems Tools, in Seventh USENIX Systems Administration Conference Proceedings, pp. 1–8, Monterey, CA, November. Evard, R., Churchill, E. and Bly, S. (2001) Waterfall Glen: Social Virtual Reality at Work. In Churchill, E., Snowdon, D. and Munro, A. (eds.), Collaborative Virtual Environments, Springer. EverQuest (2003) http://www.station.sony.com/ Fahlman (2003) http://www-2.cs.cmu.edu/~sef/sefSmiley.htm Fairclough, M. (1986) The Webbs’ Revenge ? Conditional Degeneration and Producer Co-operatives: A Reappraisal of the Socialist Tradition. National Conference for Research on Worker Co-operative, Co-operatives Reseach Unit, Open University, London. Fairclough, M. (1987) Mondragon in Context, Department of Sociology, University of Bristol. Bristol. Falk, H. and Dierking, L. (1992) The Museum Experience. Whalesback Books, Washington. Faloutsos, P., Vanne de Panne, M. and Terzopoulos, D. (2001) Composable Controllers for Physics-based Character Animation. In: Proceedings of SIGGRAPH 2001, pp. 251–260, ACM Press. Farley, T. (2001) TelecomWriting.com’s Telephone History Series http://www. privateline.com/TelephoneHistory/History1.htm Fiore, A. T., Lee Tiernan, S. and Smith, M. (2001) Observed Behavior and Perceived Value of Authors in Usenet Newsgroups: Bridging the Gap. In Proceedings of the Conference on Human Factors in Computing Systems. ACM Press, New York. Fjeld, M., Voorhorst, F., Bichsel, M., Lauche, K., Rauterberg, M. and Krueger, H. (1999) Exploring Brick-based Navigation and Composition in an Augmented Reality. In Gellersen, H.-W. (ed.), Handheld and Ubiquitous Computing. Vol. 1707, Springer-Verlag, Berlin, pp. 102–116. Floyd, S., Jacobson,. V., McCanne, S., Liu, C.-G. and Zhang, L. (1995) A Reliable Multicast Framework for Light-Weight Sessions and Application Level Framing. In Proceedings of ACM SIGCOMM 95, ACM Press, New York, pp. 242–256. Fraser, M., Benford, S., Hindmarsh, J. and Heath, C. (1999) Supporting Awareness and Interaction through Collaborative Virtual Interfaces. In Proceedings of UIST’99, pp. 27–36, ACM Press. Fraser, M., Glover, T., Vaghi, I., Benford, S., Greenhalgh, C., Hindmarsh, J. and Heath, C. (2000) Revealing the Reality of Collaborative Virtual Reality. In Proceedings of the Third ACM Conference on Collaborative Virtual Environments (CVE 2000), San Francisco, CA, September 2000, pp. 29–37, ACM Press. Frécon, E. and Stenius, M. (1998) DIVE: A Scaleable Network Architecture for Distributed Virtual Environments. Distributed Systems Engineering Journal (DSEJ), 5: 91–100, Special Issue on Distributed Virtual Environments. Frécon, E. and Avatare Nöu, A. (1998) Building Distributed Virtual Environments to Support Collaborative Work. In Proceedings of ACM Symposium on Virtual Reality Software and Technology (VRST ‘98), Taipei, Taiwan, pp. 105–113. Frécon, E. and Smith, G. (1998) WebPath – A Three-dimensional Web History. In Proceedings of the IEEE Symposium on Information Visualization

312

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

References

● 1 ●●●●●●●●●●●●●● (InfoVis ‘98), part of IEEE Visualization 1998 (Vis98), NC, USA, pp. 3–10. Frécon, E., Greenhalgh, C. and Stenius, M. (1999) The DiveBone: An ApplicationLevel Network Architecture for Internet-Based CVEs. Paper presented at the VRST’99 – Symposium on Virtual Reality Software and Technology 1999, 20–22 December, University College London, UK. Frécon, E. and Smith, G. (1999) Semantic Behaviours in Collaborative Virtual Environments. In Proceedings of Virtual Environments ‘99 (EGVE’99), pp. 95–104, Vienna, Austria. Frécon, E., Smith, G., Steed, A., Stenius, M. and Ståhl, O. (2001) An Overview of the COVEN Platform, Presence: Teleoperators and Virtual Environments, 10(1): 109–127. Fruchterman, T. M. J. and Reingold, E. M. (1991) Graph Drawing by Forcedirected Placement. Software Practice and Experience 21(11): 1129–1164. Fuchs, H. (1998) Beyond the Desktop Metaphor: Toward More Effective Display, Interaction, and Telecollaboration in the Office of the Future via a Multitude of Sensors and Displays, AMCP:98, pp. 30–43, Osaka, Japan. Fuchs, L., Pankoke-Babatz, U. and Prinz, W. (1995) Supporting Cooperative Awareness with Local Event Mechanisms: The GroupDesk System, In Marmolin, H., Sundblad, Y. and Schmidt, K. (eds.), Fourth European Conference on Computer-Supported Cooperative Work: ECSCW ‘95, Kluwer Academic Publishers, Stockholm, pp. 247–262. Fuchs, L. (1999) AREA: A Cross-application Notification Service for Groupware, In Bødker, S., Kyng, M. and Schmidt, K. (eds.), ECSCW’99: Sixth Conference on Computer Supported Cooperative Work, Kluwer Academic Publishers, Copenhagen, pp. 61–80. Fussell, S. R., Kraut, R. E. and Siegel, J. (2000) Coordination of Communication: Effects of Shared Visual Context on Collaborative Work, In Whittaker, S. and Kellog, W. (eds.), CSCW 2000, ACM, Philadelphia, PA, pp. 21–30. Galani, A. and Chalmers, M. (2002) Can You See Me? Exploring Co-Visiting between Physical and Virtual Visitors. Proc. Museums and the Web. Archives & Museum Informatics, Boston, USA. Garcia, A. C. and Jacobs, J. B. (1999) The Eyes of the Beholder: Understanding the Turn-Taking System in Quasi-Synchronous Computer-Mediated Communication, Research on Language and Social Interaction, 32(4): 337–367. Gaver, B. (2002) Provocative Awareness. Computer Supported Cooperative Work: The Journal of Collaborative Computing (Special Issue on Awareness) 11(3–4): 475–493. Gaver, W. W. (1992) The Affordance of Media Spaces for Collaboration. In Turner, J. and Kraut, R. (eds.), CSCW ‘92: Conference on Computer Supported Cooperative Work – Sharing Perspectives, ACM Press, Toronto, Canada, pp. 17–24. Gaver, W., Moran, T., MacLean, A., Lövstrand, L., Dourish, P., Carter, K., and Buxton, W. (1992) Realizing a Video Environment: EuroPARC’s RAVE System, Proceedings of CHI’92. ACM, New York, pp. 27–35. Gaver, W., Sellen, A., Heath, C., and Luff, P. (1993) One Is Not Enough: Multiple Views in a Media Space. In Proceedings of INTERCHI ‘93, pp. 335–341, ACM Press. Gavin, L., Mottram, C., Penn, A. and Kueppers, S. (2000) Space Module – TOWER Deliverable D3.1.

313

References

Gibson, J. J. (1986) The Ecological Approach to Visual Perception, Lawrence Erlbaum Associates, Hillsdale, NJ. Gibson, W. (1986) Neuromancer, Victor Gollancz, London. Giddens, A. (1984) The Constitution of Society, Polity Press, Cambridge, p. 402. Goodwin, C. and Goodwin, M. (1996) Formulating Planes: Seeing as a Situated Activity. In Engestrom, Y. and Middleton, D. (eds.), Communication and Cognition at Work. Cambridge University Press, New York, 61–95. Google Groups, http://groups.google.com/ Greenhalgh, C. M., and Benford, S. D. (1995) MASSIVE: A Virtual Reality System for Tele-conferencing, ACM Transactions on Computer Human Interfaces (TOCHI), 2 (3), pp. 239–261, ACM Press, September. Greenhalgh, C., Bullock, A., Tromp, J. and Benford, S. (1997) Evaluating the Network and Usability Characteristics of Virtual Reality Tele-conferencing, BT Technology Journal, 15(4), October. Greenhalgh, C. (1998) Analysing Awareness Management in Distributed Virtual Environments. Paper presented at the Second Annual Workshop on System Aspects of Sharing a Virtual Reality, at CVE’98, Manchester, UK. Greenhalgh, C. M. (1999) Large Scale Collaborative Virtual Environments. Springer-Verlag. London. Greenhalgh, C., Bowers, J., Walker, G., Wyver, J., Benford, S. and Taylor, I. (1999) Creating a Live Broadcast from a Virtual Environment. In Proceedings of SIGGRAPH ‘99, pp. 375–384. Greenhalgh, C. and Benford, S. (1999) Supporting Rich and Dynamic Communication in Large-Scale Collaborative Virtual Environments. Presence: Teleoperators Virtual Environments, 8(1): 14–35. Greenhalgh, C., Purbrick, J., Benford, S., Craven, M., Drozd, A. and Taylor, I. (2000a) Temporal Links: Recording and Replaying Virtual Environments, In Proceedings of the Eighth ACM International Conference on Multimedia (MM 2000), ACM Press, pp. 67–74. Greenhalgh, C., Purbrick, J. and Snowdon, D. (2000b) Inside MASSIVE-3: Flexible Support for Data Consistency and World Structuring. Paper presented at the Third ACM Conference on Collaborative Virtual Environments (CVE 2000), San Francisco, CA. Greenhalgh, C. (2001) Understanding the Network Requirements of Collaborative Virtual Environments. In Churchill, E., Snowdon, D. and Munro, A. (eds.), Collaborative Virtual Environments: Digital Places and Spaces for Interaction. Springer Verlag, London, pp. 56–76. Grondin, J. (1994) Introduction to Philosophical Hermeneutics. Trans. J. Weinsheimer. Yale University Press. Gross, T. (2002) Ambient Interfaces in a Web-Based Theatre of Work, In Proceedings of the Tenth Euromicro Workshop on Parallel, Distributed, and Network-Based Processing – PDP 2002, IEEE Computer Society Press, Gran Canaria, Spain, pp. 55–62. Gross, T. and Prinz, W. (2000) Gruppenwahrnehmung im Kontext, In Reichwald, R. and Schlichter, J. (eds.), Verteiltes Arbeiten – Arbeit der Zukunft, Tagungsband der D-CSCW 2000. B.G. Teubner, Stuttgart/ Leipzig/ Wiesbaden, pp. 115–126. Gutwin, C. and Greenberg, S. (1998) Design for Individuals, Design for Groups: Trade-offs in Power and Workspace Awareness. In Proceedings of CSCW’98, pp. 207–216, ACM Press.

314

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

References

● 1 ●●●●●●●●●●●●●● Guye-Vuillème, A., Capin, T. K., Pandzic, I. S., Thalmann, N. M. and Thalmann, D. (1999) Non-verbal Communication Interface for Collaborative Virtual Environments. The Virtual Reality Journal, 4: 49–59, Springer. Harper, R. H. R., Hughes, J., Randall, D., Shapiro, D. and Sharrock, W. (1998) Order in the Skies: Sociology, CSCW, and Air Traffic Control. Routledge, London. Harper, R. R., Hughes, J. A. and Shapiro, D. Z. (1989) Working in Harmony: An Examination of Computer Technology in Air Traffic Control. EC-CSCW ‘89. Proceedings of the First European Conference on Computer Supported Cooperative Work, Gatwick, London, 13–15 September. Harrison, S. and Dourish, P. (1996) Re-Place-ing Space: The Roles of Place and Space in Collaborative Systems. In Proceedings of the ACM Conference on Computer Supported Co-operative Work, ACM Press, pp. 67–76. Heath, C. and Luff, P. (1991) Collaborative Activity and Technological Design: Task Coordination in London Underground Control Rooms, In Bannon, L., Robinson, M. and Schmidt, K. (eds.), Second European Conference on Computer Supported Cooperative Work, Kluwer, Amsterdam, pp. 65–80. Heath, C., Jirotka, M. Luff, P. and Hindmarsh, J. (1993) Unpacking Collaboration: The Interactional Organisation of Trading in a City Dealing Room. In Michelis, G. de, Simone, C. and Schmidt, K. (eds.), Proceedings of the Third European Conference on Computer Supported Cooperative Work – ECSCW ‘93. 13–17 September, Milan, Italy. Kluwer Academic Publishers, Dordrecht. Heath, C. and Luff, P. (1996) Convergent Activities: Line Control and Passenger Information on the London Underground. In Engestrom, Y. and Middleton, D. (eds.), Communication and Cognition at Work. Cambridge University Press, New York, pp. 96–129. Heath, C. C., Luff, P., Kuzuoka, H., Yamazaki, K. (2001) Creating Coherent Environments for Collaboration, Proceedings of ECSCW 2001, Bonn, Germany, pp. 119–138, Kluwer. Hillier, B. and Hanson, J. (1984) The Social Logic of Space. Cambridge University Press, Cambridge. Hindmarsh, J., Fraser, M., Heath, C., Benford, S. and Greenhalgh, C. (1998) Fragmented Interaction: Establishing Mutual Orientation in Virtual Environments. In: Proceedings of CSCW’98, Seattle, WA, USA, pp. 217–226, ACM Press. Hirtes, S., Hoch, M., Lintermann, B., Norman, S. J., Bowers, J., Jää-Aro, K.-M., Hellström, S.-O. and Carlzon, M. (1999) Production Tools for Electronic Arenas: Event Management and Content Production, eRENA Deliverable D4.3/D4.4. Hoch, M., Jää-Aro, K.-M. and Bowers, J. (1999a) Round Table: A Physical Interface for Virtual Camera Deployment in Electronic Arenas. In Hirtes et al. (1999), chapter 5. Hoch, M., Schwabe, D., Shaw, J., Staff, H., Raupp Musse, S., Garat, F., Thalmann, D., Jää-Aro, K.-M., Bowers, J. and Hellström, S.-O. (1999b) Individual and Group Interaction, eRENA Deliverable 6.3. Hood, E. (2000) Ad blocking. http://www.nacs.uci.edu/indiv/ehood/gems/adblocking.html Hubbold, R., Cook, J., Keates, M., Gibson, S., Howard, T., Murta, A., West, A., and Pettifer, S. (2001) GNU/MAVERIK: A Micro-kernel for Large-scale Virtual Environments. Presence: Teleoperators and Virtual Environments, 10: 22–34.

315

References

Hughes, J. A., Prinz, W., Rodden, T. and Schmidt, K. (eds.) (1997) Proceedings of the Fifth European Conference on Computer-Supported Cooperative Work. IEEE1278.1. (1995) Standard for Distributed Interactive Simulation – Application Protocols. In I. Institute of Electrical and Electronics Engineers (Ed.). Ingram, R. and Benford, S. (1995) Legibility Enhancement for Information Visualisation. In Proceedings of Visualization’95, Atlanta, GA, November. Insley, J., Sandin, D. and DeFanti, T. (1997) Using Video to Create Avatars in Virtual Reality, Visual Proceedings of the 1997 SIGGRAPH Conference, Los Angeles, CA 08/01/1997–08/01/1997, pp. 128. Intel (2003) Intel Graphics Performance Primitives. http://developer.intel.com/ design/pca/applicationsprocessors/swsup/gpp.htm Ishii, H., Kobayashi, M. and Grudin, J. (1992) Integration of Inter-Personal Space and Shared Workspace: ClearBoard Design and Experiments. Proceedings of ACM CSCW ‘92 Conference on Computer-Supported Cooperative Work, pp. 33–42. Ishii, H. and Kobayashi, M. (1993) ClearBoard: A Seamless Medium for Shared Drawing and Conversation with Eye Contact. In R. M. Baecker (ed.), Readings in Groupware and Computer Supported Cooperative Work: Assisting Human-human Collaboration. Morgan Kaufmann, San Mateo, CA, pp. 829–836. Ishii, H. and Ullmer, B. (1997) Tangible Bits: Towards Seamless Interfaces between People, Bits, and Atoms. In Proceedings of ACM CHI’97, pp. 234–241, Atlanta. Ishii, H., Wisneski, C., Orbanes, J., Chun, B. and Paradiso, J. (1999) PingPongPlus: Design of an Athletic-Tangible Interface for ComputerSupported Cooperative Play. In Proceedings of CHI’99, Pittsburgh. Jää-Aro, K.-M., Bowers, J. M. and Hellström, S.-O. (1999) Activity-Oriented Navigation. In Hoch et al. (1999b), pp. 45–52. Jää-Aro, K.-M. and Snowdon, D. (2001) How Not to Be Objective. In: Collaborative Virtual Environments, Springer-Verlag, London, pp. 143–159. Jancke, G., Grudin, J. and Gupta, A. (2000) Presenting to Local and Remote Audiences: Design and Use of the TELEP System, Proc. CHI’2000, April, ACM Press, pp. 384–391. Johansson, M. (1998) Designing an Environment for Distributed Real-Time Collaboration, IWNA ‘98, Kyoto. Jones, M.L.W. (2000) Collaborative Virtual Conferences: Using Exemplars to Shape Future Research Questions, ACM CVE 2000, San Francisco, September. Johnson, B. and Shneiderman, B. (1991) Tree-Maps: A Space-filling Approach to the Visualization of Hierarchical Information Structures. In Second International IEEE Visualization Conference, IEEE Press, San Diego, CA, pp. 284–291. Kaplan, S., Leigh Star, S., Tolone, W. J. and Bignoli, C. (1994) Grounding the Metaphor of Space in CSCW: Meta-Structures and Boundary Objects. DRAFT MS. Kapolka, A., McGregor, D. and Capps, M. (2002) A Unified Component Framework for Dynamically Extensible Virtual Environments. In Proceedings of the Fourth International Conference on Collaborative Virtual Environments (CVE 2002), Bonn, Germany, pp. 64–71. Kelly, S. U., Sung, C. and Farnham, S. (2002) Designing for Improved Social Responsibility, User Participation and Content in On-line Communities,

316

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

References

● 1 ●●●●●●●●●●●●●● Proceedings of the Conference on Human Factors in Computing Systems. ACM Press, New York. Kharif, O. and Salkever, A. (2001) A Chat with the Master of P2P in Business Week: Special Report: Peer to Peer. 1 August. Koleva, B., Schnädelbach, H., Benford, S. and Greenhalgh, C. (2001) Experiencing a Presentation though a Mixed Reality Boundary. In Proc. ACM SIGGROUP Conference on Supporting Group Work (GROUP’01), Boulder, CO, pp. 71–80, ACM Press. Kreuger, W. and Froehlich, B. (1994) The Responsive Workbench. Computer Graphics and Applications 14(3): 12–15. Kuzuoka, H., Kosuge, T. and Tanaka, M. (1994) GestureCam: A Video Communication System for Remote Collaboration. CSCW 94: Transcending Boundaries, Chapel Hill, NC, ACM. Landry, C., Morley, D., Southwood, R. and Wright, P. (1986) What a Way to Run a Railroad: An Analysis of Radical Failure. Comedia, London. Lamport, L. (1978) Time, Clocks, and the Ordering of Events in a Distributed System. Communication of the ACM, 21(7): 558–565. Lantermann, E. D. (1980) Interaktionen – Person, Situation und Handlung, Urban und Schwarzenberg, Munich. Lave, J. and Wenger, E. (1991) Situated Learning. Legitimate Peripheral Participation, Cambridge University Press, Cambridge. Lawley, E. (1994) The Sociology of Culture in Computer-mediated Communication: An Initial Exploration. Rochester Institute of Technology, available at: http://www.itcs.com/elawley/bourdieu.html. Leach, N. (ed.) (1997) Rethinking Architecture: A Reader in Cultural Theory. Routledge, London. Lee, A., Girgensohn, A. and Schlueter, K. (1997) NYNEX Portholes: Initial User Reactions and Redesign Implications. In Hayne, S. and Prinz, W. (eds.), Group ‘97 Conference, ACM Press, Phoenix, AZ, pp. 385–394. Lee, W., Goto, T., Raupp Musse, S., Aubel, A., Garat, F. and Davary, M. (2000) Participants: Individual Virtual Humans and Crowds Simulation in Collaborative Virtual Environment, eRENA Deliverable 5.4. Leigh, J., Johnson, A., DeFanti, T. et al. (1999) A Review of Tele-Immersive Applications in the CAVE Research Network. In Proceedings of IEEE VR ‘99, Houston, TX 13–17 March 1999, pp. 180–187. Leigh, J., Johnson, A. E., Park, K. S., Cho, Y. J., Scharver, C., Krishnaprasad, N. K. and Lewis, M. J. (2000) CAVERNsoft G2: A Toolkit for High Performance Tele-Immersive Collaboration. Paper presented at the Symposium on Virtual Reality Software and Technology, Seoul, Korea. Leont’ew, A. N. (1977) Tätigkeit, Bewußtsein, Persönlichkeit, Klett, Stuttgart. Levine, R. (1997) A Geography of Time. Basic Books, New York. Lloyd, D., Steed, A., Bullock, A., Greenhalgh, C. and Frécon, E. (2001) Making Collaborative Environments Work, Presence: Teleoperators and Virtual Environments, 10(2), April. Lövstrand, L. (1991) Being Selectively Aware with the Khronika System, In Bannon, L., Robinson, M. and Schmidt, K. (eds.), 2nd European Conference on Computer Supported Cooperative Work, Kluwer Academic Publishers, Amsterdam, pp. 265–277. Luff, P., Hindmarsh, J. and Heath, C. C. (eds.) (2000) Workplace Studies: Recovering Work Practice and Informing System Design. Cambridge University Press, Cambridge.

317

References

Lynch, M. (1991a) Pictures of Nothing? Visual Construals in Social Theory. Sociological Theory 9(1): 1–21. Lynch, M. (1991b) Science in the Age of Mechanical Reproduction: Moral and Epistemic Relations Between Diagrams and Photographs. Biology and Philosophy 6: 205–226. MacColl, I., Millard, M., Randell, C., Steed, A., Brown, B. et al. (2002) Shared visiting on EQUATOR City. Proc. 4th International conference on collaborative virtual environments (CVE 2002), Bonn, Germany, pp. 88–94. ACM Press. Macedonia, M. R. and Brutzman, D. P. (1994) MBone Provides Audio and Video Across the Internet, IEEE Computer, 27(4): 30–36. Macedonia, M. R., Zyda, M. J., Pratt, D. R., Barham, P. T. and Zeswitz, S. (1994) NPSNET: A Network Software Architecture for Large Scale Virtual Environments. Presence Teleoperators and Virtual Environments, 3(4). Macedonia, M. R., Zyda, M. J., Pratt, D. R., Brutzman, D. P. and Barham, P. T. (1995) Exploiting Reality with Multicast Groups: A Network Architecture for Large Scale Virtual Environments. Paper presented at the IEEE Virtual Reality Annual Symposium, RTP, North Carolina, 11–15 March. Maher, M.L., Simoff, S.J. and Cicognani, A. (2000) Understanding Virtual Design Studios, Springer Verlag. Manohar, N. and Prakash, A. (1995) The Session Capture and Replay Paradigm for Asynchronous Collaboration, In Proceedings of the Fourth European Conference on Computer-Supported Cooperative Work – ECSCW’95, Kluwer Academic Publishers, Stockholm, Sweden, pp. 149–164. Mariani, J. and Rodden, T. (eds.) (1999) The Library Abstract eSCAPE Demonstrator. eSCAPE Esprit Project 25377, Deliverable 4.1. Marratech (2003) http://www.marratech.com Marsh, J., Pettifer, S., and West, A. (1999) A Technique for Maintaining Continuity of Perception in Networked Virtual Environments. Proc UKVRSIG’99. Salford University Press. Massey, D. (2003) http://www.bellsystemmemorial.com/picturephone.html McGrath, A. (1998) ACM SIGGROUP Bulletin, 19: 21–24. Miller, D. C., Pope, A. C. and Waters, R. M. (1989) Long-Haul Networking of Simulators. Paper presented at the Tenth Interservice/Industry Training Systems Conference, Orlando. Minar, N. and Hedlund, M. (2001) A Network of Peers: Peer-to-Peer Models Through the History of the Internet. Peer-to-Peer: Harnessing the Power of Disruptive Technologies. A. Oram, O’Reilly & Associates. Mira Lab (2003) http://miralabwww.unige.ch/ Morrison, A., Ross, G. and Chalmers, M. (2002) A Hybrid Layout Algorithm for Sub-Quadratic Multidimensional Scaling. In: Proc. IEEE Information Visualisation, Boston, pp. 152–160. Murray, C. D., Bowers, J. M., West, A. J., Pettifer, S. R. and Gibson, S. (2000) Navigation, Wayfinding and Place Experience within a Virtual City. Presence: Teloperators and Virtual Environments, 9: 435–447, MIT Press. Munro, A., Hook, K. and Benyon, D. (1999) Footprints in the Snow. In Munro, Hook and Benyon (eds.), Social Navigation of Information Space, pp. 1–14, Springer. Nardi, B., Schwartz, H., Kuchinsky, A., Leichner, R., Whittaker, S. and Sclabassi, R. (1993) Turning Away from Talking Heads: The Use of Video-as-Data in Neurosurgery. Proc. INTERCHI ‘93, Amsterdam, 22–29 April, ACM.

318

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

References

● 1 ●●●●●●●●●●●●●● Netscan: A Social Accounting Search Engine. http://netscan.research. microsoft.com Noriega, P. and Sierra, C. (1999) Towards a Formal Specification of Complex Social Structures in Multi-agent Systems, 1624 in Lecture Notes in Artificial Intelligence, pp. 284–300. Springer-Verlag. Norman, S. J., Staff, H., Schwabe, D. and Wallen, L. (1998) Extended Performance Staging: Background and Evaluation. In Bowers et al. (1998c), chapter 2. Nöth, W. (1995) Handbook of Semiotics. Indiana University Press, Bloomington, IN. Oliveira, M., Crowcroft, J. and Slater, M. (2000) Component Framework Infrastructure for Virtual Environments. In Proceedings of the Third International Conference on Collaborative Virtual Environments (CVE’2000), San Francisco, CA, pp. 139–146. Pankoke-Babatz, U. and Syri, A. (1997) Collaborative Workspaces for Time Deferred Electronic Cooperation, In Hayne, S. and Prinz, W. (eds.), GROUP ‘97: International ACM SIGGROUP Conference on Supporting Group Work, ACM Press, Phoenix, AZ, pp. 187–196. Pankoke-Babatz, U. (2000) Electronic Behaviour Settings for CSCW. AI and Society, 14(1): 3–30. Patterson, J. F., Day, M. and Kucan, J. (1996) Notification Servers for Synchronous Groupware. In Ackermann, M. S. (ed.), Conference on Computer Supported Cooperative Work (CSCW’96), ACM Press, Boston, MA, pp. 122–129. Pettifer, S. (ed.) (1999) eSCAPE Systems, Techniques and Infrastructures. eSCAPE Esprit Project 25377, Deliverable 5.1. Pettifer, S., Cook, J. Marsh, J., and West, A. (2000) Deva3: Architecture for a Large-scale Virtual Reality System. In Proceedings of the ACM Symposium in Virtual Reality Software and Technology 2000, pp. 33–39. ACM Press. Pettifer, S., Cook, J. and Mariani, J. (2001) Towards Real-time Interactive Visualisation Virtual Environments: A Case Study of Q-space. In Proceedings of the International Conference on Virtual Reality 2001, pp. 121–129. ISTIA Innovations Laval, France. Plaza, E., Arcos, J. L., Noriega, P. and Sierra, C. (1998) Competing Agents in Agent-mediated Institutions. Personal Technologies, 2: 212–220. Polycom (2003) http://www.polycom.com/products_services/products_groups/ 0,1422,pw-186–186–72,00.html Prinz, W. (1999) NESSIE: An Awareness Environment for Cooperative Settings, In Bødker, S., Kyng, M. and Schmidt, K. (eds.), ECSCW’99: Sixth Conference on Computer Supported Cooperative Work, Kluwer Academic Publishers, Copenhagen, pp. 391–410. Project Oxygen (2003) http://oxygen.lcs.mit.edu/E21.html Prussak, L. (1997) Knowledge in Organizations, Butterworth-Heinemann, Oxford. Raja, V. (1998) The Cybersphere, http://www.vr-systems.ndtilda.co.uk/sphere1. htm Randell, C. and Muller, H. (2001) Low Cost Indoor Positioning System. In Proc. UbiComp 2001: Ubiquitous Computing, pp. 42–48, Springer. Ratnasamy, S., Handley, M., Karp, R. and Shenker, S. (2001) Application-level Multicast using Content-Addressable Networks. In Proceedings of the Third

319

References

International Workshop on Networked Group Communication (NGC ‘01), London. Reynolds, C. (1987) Flocks, Herds, and Schools: A Distributed Behavioral Model. Computer Graphics, 21(4): 25–34. Richardson, T., Stafford-Fraser, Q., Wood, K. R. and Hopper, A. (1998) Virtual Network Computing, IEEE Internet Computing, 2(1): 33–38. Rinman, M.-L. (2002) Forms of Interaction in Mixed Reality Media Performance – A Study of the Artistic Event DESERT RAIN, Fil.lic. thesis, Royal Institute of Technology, TRITA-NA-0214. Rivera, K., Cooke, N. J. and Bauhs, J. A. (1996) The Effects of Emotional Icons on Remote Communication, CHI Conference Companion, April. Roberts, D. J. (1996) A Predictive Real Time Architecture for Multi-User, Distributed, Virtual Reality. Unpublished PhD, University of Reading, Reading, UK. Roberts, D. J., Lake, T. W. and Sharkey, P. M. (1998) Optimising Exchange of Attribute Ownership in the DMSO RTI. Paper presented at the Simulation Interoperability Workshop, SISO, Orlando, USA. Roberts, D. J., Strassner, J., Worthington, B. G. and Sharkey, P. (1999) Influence of the Supporting Protocol on the Latencies Induced by Concurrency Control within a Large Scale Multi User Distributed Virtual Reality System. Paper presented at the International Conference on Virtual Worlds and Simulation (VWSIM), SCS Western Multi-conference ‘99, San Francisco, CA. Robertson, G., Czerwinski, M. and van Dantzich, M. (1997) Immersion in Desktop Virtual Reality. In Proceedings of UIST’97, pp. 11–19, ACM Press. Robinson, M., Pekkola, S., Korhonen, J., Hujala, S., Toivonen, T. and Saarien, M.J. O. (2001) Extending the Limits of Collaborative Virtual Environments. In Snowdon, D., Churchill, E. F. and Munro, A. J. (eds.), Collaborative Virtual Environments: Digital Places and Spaces for Interaction. Springer Verlag, London, pp. 21–42. Rorty, R. (1991) Essays on Heidegger and Others: Philosophical Papers Volume 2. Cambridge University Press, Cambridge. Rosen, E. (1996) Personal Videoconferencing, Manning Publications Co. Roseman, M. and Greenberg, S. (1996) TeamRooms: Network Places for Collaboration, In Ackermann, M. S. (ed.), Conference on Computer Supported Cooperative Work (CSCW’96), ACM Press, Boston, MA, pp. 325–333. Sack, W. (2000) Conversation Map: A Content-Based Usenet Newsgroup Browser, Proceedings of the International Conference on Intelligent User Interfaces. New Orleans, LA, ACM Press. Salem, B. and Earle, N. (2000) Designing a Non-verbal Language for Expressive Avatars. In Proceedings of CVE 2000, San Francisco, CA, pp. 93–101, ACM Press. Salvador, T. (1998) Communities, Schumanities. SIGGROUP Bulletin 19(2): 37–39. S¸andor, O., Bogdan, C. and Bowers, J. (1997) Aether: An Awareness Engine for CSCW. In Hughes et al. (1997), pp. 221–236. Sartre, J. P. (1965) The Psychology of Imagination. The Citadel Press, New York. SAS (2002) SAS press release 10th April 2002, “Personal meetings lead to more business” available from http://www.scandinavian.net/EC/Appl/Core/ Templ/PRView/0,3463,SO%253D0%2526CID%253D434109%2526MKT%25 3DSE,00.html

320

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

References

● 1 ●●●●●●●●●●●●●● Saussure, F. de (1983) Course in General Linguistics. Trans. Wade Baskin. McGraw-Hill. (Originally published in 1906.) Savolainen, R. (1995) Everyday Life Information Seeking: Approaching Information Seeking in the Context of “Way of Life”. Library and Information Science Research 17: 259–294. Schäfer, K., Brauer, V. and Bruns, F. (1997) A New Approach to HumanComputer Interaction: Synchronous Modeling in Real and Virtual Spaces. In Proceedings of the DIS’97, Amsterdam. Schindler Jr, G. E. (ed.), Bell Laboratories RECORD, Vol. 47, No. 5, May/June 1969, available online at http://www.bellsystemmemorial.com/pdf/picturephone.pdf Schlichter, J. H., Koch, M. and Bürger, M. (1998) Workspace Awareness for Distributed Teams. In Conen, W. and Neumann, G. (eds.), Coordination Technologie for Collaborative Applications – Organisations, Processes and Agents, Lecture Notes in Computer Science, Springer, Berlin, pp. 197–218. Schmidt, K. and Bannon, L. (1992) Taking CSCW Seriously: Supporting Articulation Work. CSCW 1(1): 7–40. Schreer, O. and Kauff, P. (2002) An Immersive 3D Video-Conferencing System Using Shared Virtual Team User Environments, ACM CVE 2002, Bonn, Germany. Schutz, A. (1970) On Phenomenology and Social Relations. University of Chicago Press, Chicago. Schwabe, D. and Stenius, M. (2000) The Web Planetarium and other Applications in the Extended Virtual Environment EVE. In Proceedings of the 16th Spring Conference on Computer Graphics, Budmerice, Slovakia, 3–6 May. Segall, B. and Arnold, D. (1997) Elvin Has Left the Building: A Publish/subscribe Notification Service with Quenching, In AUUG, http://www.dstc.edu.au/ Elvin/, Brisbane, Australia. Sharkey, P. M., Roberts, D. J., Tran, F. D. and Worthington, B. G. (2000) PING – Platform for Interactive Networked Games: IST Framework V. Shaw, J. (1997) PLACE – A User’s Manual: From Expanded Cinema to Virtual Reality. Hatje Cantz. Shaw, J., Staff, H., Row Farr, J., Adams, M., vom Lehm, D., Heath, C., Rinman, M.-L., Taylor, I. and Benford, S. (2000) Staged Mixed Reality Performance, “Desert Rain” by Blast Theory, eRENA Deliverable 7b.3. Shneiderman, B. (1983) Direct Manipulation: A Step Beyond Programming Languages. Computer 16(8): 57–69. Singhaland, S. K. and Cheriton, D. R. (1996) Using Projection Aggregation to Support Scalability in Distributed Simulation. Paper presented at the International Conference on Distributed Computing Systems ICDCS´96. Slater, M., Pertaub, D.-P. and Steed, A., (1999) Public Speaking in Virtual Reality: Facing an Audience of Avatars. IEEE Computer Graphics and Applications, 19(2): 2–5. Slater, M., Howell, J., Steed, A., Pertaub, D.-P., Garau, M. and Springel, S. (2000a) Acting in Virtual Reality. Paper presented at the ACM Collaborative Virtual Environments. Slater, M., Sadagic, A., Usoh, M. and Schroeder, R. (2000b) Small Group Behaviour in a Virtual and Real Environment: A Comparative Study. Presence: Teleoperators and Virtual Environments, 9(1): 37–51.

321

References

Slater, M., Steed, A. and Chrysanthou, Y. (2001) Computer Graphics and Virtual Environments: From Realism to Real-Time. Addison Wesley Publishers, Harlow. Smith, M. (2000) Invisible Crowds in Cyberspace: Measuring and Mapping the Social Structure of USENET, Communities in Cyberspace, Routledge, London. Snowdon, D., Greenhalgh, C. and Benford, S. (1995) What You See is Not What I See: Subjectivity in Virtual Environments. Paper presented at the Framework for Immersive Virtual Enviroments (FIVE’95), QMW University of London, UK. Snowdon, D., Fahlén, L. and Stenius, M. (1996) WWW3D: A 3D Multi-user Web Browser. In Proceedings of WebNet’96, San Francisco, CA, October. Snowdon, D., Churchill, E. F. and Munro, A. J. (2001) Collaborative Virtual Environments: Digital Spaces and Places for CSCW: An Introduction. In Snowdon, D., Churchill, E. F. and Munro, A. J. (eds.), Collaborative Virtual Environments. Digital Places and Spaces for Interaction. Springer Verlag, London, pp. 3–17. Snowdon, D. and Grasso, A. (2002) Diffusing Information in Organisational Settings: Learning from Experience. In Proceedings of ACM CHI2002, Minneapolis, MN, April, pp. 331–338. Sohlenkamp, M., Prinz, W. and Fuchs, L. (2000) AI and Society – Special Issue on CSCW, 14: 31–47. Ståhl, O. (1992) Tools for Cooperative Work in the MultiG TelePresence Environment. In Proceedings of the 4th MultiG Workshop, StockholmKista, Sweden, pp. 75–88. Ståhl, O., Wallberg, A., Söderberg, J., Humble, J., Fahlén, L., Bullock, A. et al. (2002) Information Exploration Using The Pond. In Proceedings of CVE’02, Bonn. Star, S. L. (1992) The Trojan Door: Organisations, Work, and the “Open Black Box”. Systems Practice 5: 395–410. Star, S. L. and Griesemer, J. R. (1989) Institutional Ecology, “Translations” and Boundary Objects: Amateurs and Professionals in Berkeley’s Museum of Vertebrate Zoology, 1907–39. Social Studies of Science 19: 387–420. Steed, A. and Frécon, E. (1999) Building and Supporting a Large-scale Collaborative Virtual Environment. In Proceedings of 6th UKVRSIG, University of Salford, UK, pp. 59–69. Steed, A., Frécon, E., Avatare, A., Pemberton, D. and Smith, G. (1999) The London Travel Demonstrator, In Proceedings of VRST’99 – Symposium on Virtual Reality Software and Technology, University College London, UK, pp. 50–57. Steed, A., Mortensen, J. and Frécon, E. (2001) Spelunking: Experiences using the DIVE System on CAVE-like Platforms. In B. Frohlicj, J. Deisinger, and H-J. Bullinger (eds.), Proceedings of Immersive Projection Technologies and Virtual Environments 2001 Springer-Verlag, Vienna, pp. 153–164. Steffik, M., Bobrow, D. G., Foster, G., Lanning, S. and Tatar, D. (1987) WYSIWIS Revised: Early Experiences with Multiuser Interfaces. Transactions on Office Information Systems, 5 (2): 147–167, ACM Press. Stenius, M., Frécon, E., Fahlén, L., Simsarian, K. and Nord, B. (1998) The Web Planetarium Prototype – Visualising the Structure of the Web. In Mariani, J., Rouncefield, M., O’Brien, J. and Rodden, T. (eds.), eSCAPE Deliverable

322

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

References

● 1 ●●●●●●●●●●●●●● 3.1 Visualisation of Structure and Population within Electronic Landscapes, Esprit Long Term Research Project 25377, Lancaster University, pp. 117–125. Stephenson, N. (1992) Snowcrash, Bantam, New York. Storrs Hall, J., Utility Fog: The Stuff that Dreams are Made Of, http://discuss. foresight.org/~josh/Ufog.html Strong, R. and Gaver, B. (1996) Feather, Scent, and Shaker: Supporting Simple Intimacy. In M. S. Ackerman (ed.), Proceedings of the ACM 1996 Conference on Computer Supported Cooperative Work. New York, ACM, p. 444. Stults, R. (1986) MediaSpace, Technical Report, Xerox PARC. Suchman, L. A. (1982) Systematics of Office Work. Office Studies for KnowledgeBased Systems, Digest. Office Automation Conference, San Francisco, 5–7 April. Suchman, L. A. (1987) Plans and Situated Actions: The Problem of Humanmachine Communication, Cambridge University Press, Cambridge. Suchman, L. A. and Trigg, R. H. (1991) Understanding Practice: Video as a Medium for Reflection and Design. In Greenbaum, J. and Kyng, M. (eds.), Design at Work. Lawrence Erlbaum, London and New Jersey, pp. 65–89. Suchman, L. (1997) Centers of Coordination: A Case and Some Themes. In Resnick, L. B., Säljö, R., Pontecorvo, C. and Burge, B. (eds.), Discourse, Tools, and Reasoning: Essays on Situated Cognition. Springer-Verlag, Berlin, pp. 41–62. Sudnow, D. (ed.) (1972) Studies in Social Interaction. Free Press, New York, pp. 229–258. Swan, J., Newell, S., Scarbrough, H. and Hislop, D. (1999) Knowledge Management and Innovation: Networks and Networking. Journal of Knowledge Management, 3: 262–275. Thompson, G. (1972) Three Characterizations of Communications Revolutions. Computer Communication: Impacts and Implications: International Conference on Computer Communication. S. Winkler, New York. Törlind, P., Stenius, M., Johansson, M. and Jeppsson, P. (1999) Collaborative Environments for Distributed Engineering, CSCWD’99 – Computer Supported Cooperative Work in Design 99, Compiègne, France, 29 September–1 October. Townsend, A.M., Hendrickson, A.R. and DeMarie, S.M. (2002) Meeting the Virtual Work Imperative, CACM 45(1). Tramberend, H. (2001) Avango: A Distributed Virtual Reality Framework. Paper presented at the Afrigraph, ACM. Underkoffler, J. and Ishii, H. (1999) Urp: A Luminous-Tangible Workbench for Urban Planning and Design. In Proceedings of the ACM Conference Computer–Human Interaction (CHI99), pp. 386–393. Ullmer, B., Ishii, H. and Glass, D. (1998) MediaBlocks: Physical Containers, Transports, and Controls for Online Media. In Proceedings of SIGGRAPH’98, Orlando. Valin, S., Francu, A., Trefftz, H. and Marsic, I. (2001) Sharing Viewpoints in Collaborative Virtual Environments. In Proceedings of HICSS-34, Hawaii, IEEE. VR Lab (2003) http://vrlab.epfl.ch/

323

References

Waters, R. C., Anderson, D. B., Barrus, J. W., Brogan, D. C., Casey, M. A., McKeown, S. G., Nitta, T., Sterns, I. B. and Yerazunis, W. S. (1997) Diamond Park and Spline: A Social Virtual Reality System with 3D Animation, Spoken Interaction, and Runtime Modifiability. Presence: Teleoperators and Virtual Environments, 6(4): 461–480. Watsen, K. and Zyda, M. (1998) Bamboo – A Portable System for Dynamically Extensible, Real-Time, Networked, Virtual Environments. In Proceedings of the Virtual Reality Annual International Symposium (VRAIS’98), Atlanta, GA, pp. 252–259. Wei-Chao, Chen, Towles, H., Nyland, L., Welch, G. and Fuchs, H. (2000) Toward a Compelling Sensation of Telepresence: Demonstrating a Portal to a Distant (static) Office, IEEE Visualization 2000, pp. 327–333. Weiser, M. (1991) The Computer for the Twenty-first Century, Scientific American, 265(3): 94–10. Weiser, M. (1994) Creating the Invisible Interface (abstract). In Proc. ACM User Interface Software and Technology, p. 1. Williamson, K. (1998) Discovered by Chance: The Role of Incidental Information Acquisition in an Ecological Model of Information Use. Library and Information Science Research, 20(1): 23–40. Wisneski, C., Ishii, H., Dahley, A., Gorbet, M., Brave, S., Ullmer, B. and Yarin, P. (1998) Ambient Displays: Turning Architectural Space into an Interface between People and Digital Information. In Streitz, N. A., Konomi, S. and Burkhardt, H.-J. (eds.), Cooperative Buildings – Integrating Information, Organization, and Architecture. Springer – LNCS, pp. 22–32. Wittgenstein, L. (1958) Philosophical Investigations. 3rd edn, trans. G.E.M. Anscombe, Oxford University Press. Xiong, R., and Donath, J. (1999) PeopleGarden: Creating Data Portraits for Users. Proceedings of the 12th Annual ACM Symposium on User Interface Software and Technology, pp. 37–44. ACM, New York. Yamazaki, K., Yamazaki, A., Kuzuoka, H., Oyama, S., Kato, H. et al. (1999) Gesture Laser and Gesture Laser Car: Development of an Embodied Space to Support Remote Instruction. In Bodker, S., Kyng, M. and Schmidt, K. (eds.), Proceedings of the Sixth European Conference on Computer Supported Cooperative Work – ECSC W’99, 12–16 September, Copenhagen, Denmark. Kluwer Academic Publishers, Dordrecht.

324

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

Index

011

011

011

A abstract landscape, 27 access model, 46 accountability, 143 activity landscape, 171, 172 activity-based navigation, 193 Activity-oriented Navigation, 178 affordances, 29, 30, 31 agent-mediated institution, 105 aggregate, 17 aggregation, 250 AlphaWorld, 116 Amazon.com, 55, 61 Ambient interfaces, 201, 203 Aotea Youth Symphony, 96 Apache, 78 appropriated, 74 architectural collaboration, 115 artefact, 27 attention, 71 audio communication, 130 audio feedback, 51 augmented reality, 36, 72, 101 Auld Leaky, 77 Aura, 249, 258 authenticity, 166 autonomous, 104, 170 avatar, 236, 253 Avatar-centred Navigation, 176 avatars, 133, 197, 199 AwareBots, 201, 203 awareness, 101, 102, 171, 184, 186, 189, 203, 204, 206, 246, 249 awareness service, 104, 105

011

11

B banner advert, 20, 22 BEAD, 51 Behaviour, 41, 103 blacklist, 22 Blink, 160, 161, 164 boid flock model, 54

boundary architecture, 34 boundary object, 271, 286 brain activity, 91 “broken” link, 14

C CAPIAs, 106 CARE HERE, 94 CARESS, 92, 94 Causal ordering, 242, 245 causality, 242, 244 CAVE, 236 challenges of teleconferencing, 124 chance encounter, 196 characteristics, 40 Charles Rennie Mackintosh, 72 City project, 72, 76 city’s meaning, 75 cityscape, 27, 29 ClearBoard, 275 client/server, 39 closed world, 71 cluster, clustering, 17 coherent perception, 38 collaborate, 3 collaboration, 115 Colour coded, 35 Common goals, 281 common sense, 103 Common sense knowledge, 102 community, 272 Community Place, 165 compound link, 19 Computer Supported Co-operative Work, 271 COMRIS, 102, 105, 107, 108, 109, 110, 111 conceptions of space, 73 Concurrency control, 244 conference calls, 127 conference centre application, 106 Consistency, 242, 243 container, 17

325

Index

context, 75 context aware, 103 context of use, 71 contextualisation, 76 continuative computing, 103, 104, 110 control rooms, 274 conversational orientation, 30 co-operation, 3, 274 co-presence, 150 costly, 32 creative process, 89 creels, 62 crowd, 154, 171 CSCW, 77, 271, 274, 277, 281 Cultural context, 152 CWall, 22 CyberDance, 153 Cyberspace, 11, 74, 116 Cyc, 103 cyclist, 29

D dead reckoning, 213, 242 decontextualised information spaces, 73 delivery service, 104, 105 Desert Rain, 154 design, 287 desktop-VR, 22 DEVA, 26, 39, 40, 44, 47 disability, 89 Dispersed Avatars, 178 distributed computing, 278 distributed file sharing, 278 Distributed Legible City, 29 DIVE, 12, 16, 20, 26, 57, 66, 123, 211, 212, 215, 218, 223, 224, 252, 255, 256, 266 DocuDrama, 184, 198, 199, 205 Domain Name System (DNS), 283 Doors of Perception, 47 dual space, 104

E ecosystem metaphor, 51, 54 electronic, 50 electronic arenas, 151, 152 electronic landscape, 25, 27 Embodiment, 118, 126 Emoticons, 118, 119 emotive communication, 237 Enforced, 42 Equator, 72, 78 EQUIP, 77 eRENA, 151, 152, 153, 155 eSCAPE, 11, 12, 25, 26, 27, 29, 33

326

e-scape, 25, 33 ethnographic, 179 ethnographic studies, 26, 28, 31 expectations, 30 experience, 32 experience design, 152 expression amplifier, 92 EyesWeb, 91

F facial expression, 236 familiar structure, 35 FDP, 15, 22 feedback loop, 95 field of view, 134, 135 filter, 43 Firewalls, 264 flocking algorithm, 55 flocking behaviour, 250 fly, 28 focus, 171, 249 focused engagement, 71 Force Directed Placement (FDP), 15 Four Senses, 96

G game show, 166, 168 GestureCam, 277 Gnutella, 273 Grokster, 278 Groove, 278 groundplane, 27, 36, 47, 168, 176 groupware, 272

H haptic feedback, 143 Head Mounted Display, 236 head-set, 22 Hearing ANd Deaf Sign Singers, 96 heart rate, 91 Heaven and Hell {-} Live, 165, 168 Heidegger, 75 heterogeneity, 81 heterogeneity of media, 76 hierarchical database, 213 hot spots, 178 HTML, 12 HTTP, 58, 77 human{-}computer interaction, 151 human-like representation, 135 hybrid objects, 82 hyperlink, 27 hyperlinking, 36

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

11

Index

● 1 ●●●●●●●●●●●●●● I icon, 12 Imbued, 42 immersive, 11, 37, 237, 238 implementing virtual worlds, 37 Indian Ocean, 280 informal interactions, 134 information creatures, 52 Information gathering, 109 infrared, 91, 95 infrared movement sensors, 90 Inhabited Television, 164, 165, 170 Innate, 42 input devices, 31 Instant Messenger, 279 Intelligibility, 179 interactional breakdowns, 86 Interactive painting, 95 interest landscape, 109 interest management, 246 intimacy, 276 introspection, 42

K KaZaA, 278

L LAN, 271 landscape, 35 Large scale participation, 152 large screen, 22 LEADS, 17 legibility, 17 Legible City, 29 LEGO Mindstorm, 202 LEGO Mindstorms, 201 Level of Detail, 251, 258 level of detail (LOD), 12 level-of-detail (LOD), 17 levitate, 28 light collage, 97 Lightwork, 154, 155, 158, 160 Limewire, 278 line of sight, 143 Linux, 93 local area networks, 271 Löcales, 248, 249 LOD, 17 Loom project, 293

M Mackintosh Interpretation Centre, 72 marked-up text, 12 Masaki Fujihata, 33

MASSIVE, 26, 123 MASSIVE 2, 249 MASSIVE 3, 245 MASSIVE-2, 17, 135, 168 mass-participatory, 166 MAVERIK, 38 MAX, 91 MBone, 120, 265 Media rich, 152 mediation services, 104 Memory Theatre VR, 48 metaphors, 31 Microsoft, 12 MIDI, 89, 91, 93 MIDI bass, 90 migrate, 45 migrating, 44 Mixed reality, 151 modify existing artefacts, 32 mood, 96 MOOs, 116, 118 Morecambe, 31 Morpheus, 278 MUDs, 116, 118 multi-agent systems, 104 multicast, 213, 224, 254, 259, 263 multiple simultaneous touches, 64 multiple users, 14 muscle tension, 91 mutual availability, 147

N Nanotechnology, 99 Napster, 273 natural language generation, 108 navigable, 34 navigate, 194, 196 navigating, 24 navigation, 53, 152, 176 NetMeeting, 120 Netscan, 291, 292, 294 Netscape, 22 Netscapes, 12 Network Address Translation (NAT), 283 neuropsychologists, 99 nimbus, 171, 249 Nuzzle Afar, 33, 35

O object, 39 object behaviour, 241 Object-centred Navigation, 177 objective reality, 38 objectivity, 234 OpenGL, 26, 49

327

Index

organising concept, 271 Out of this World, 166, 168, 170, 177 out of view, 138 outlined field-of-view, 135 own content, 32

P P2P, 272 Panoramic Navigator, 36 PaRADE, 245, 255 parallelisation, 19 parrot, 108 patch data, 96 pathways, 27 pedestrian areas, 28 peer-to-peer, 39, 213, 272 PeopleGarden, 293 perceptions, 38 Performer, 49 peripheral awareness, 184, 196, 205 peripheral lenses, 135, 146 Persistence, 251 persistent, 103 pervasive computing, 101 phicon, 174, 175 phicons, 175 physical landscape, 36 Physical sensors, 102, 103 physical space, 104 physical tags, 53 Picturephone, 117 PING, 245, 253, 256, 260, 266 PlaceWorld, 25, 27, 39 plasma display, 56 plug-in, 58, 216, 230 PocketPC(tm), 66 popularity, 36 Populated Information Terrains, 133 portals, 20, 27 Presence, 179 Production management, 153, 169 proprioception, 239 proxy server, 22 pseudo-humanoid, 133 public performance, 89, 152 puppy camera, 172, 174

Q QPIT, 51 Q-SPACE, 15 Quake, 279

328

R radio frequency, 78 realism, 134 Real-time applications, 152 reciprocal perspectives, 147, 148 recommender system, 87 relaxed WYSIWIS, 134 remote collaboration, 81, 87 representation, 133 representations, 285 responsiveness, 237, 242 RFID, 57, 58, 62, 66 ribbons of light, 27 Round Table, 174, 179

S Saussure, 75 scalability, 16, 146, 251 scalable, 241 scenegraph, 238, 241, 253, 257 Script Programming, 219 seamfulness, 87 search engine, 273 searchability, 274 sense-of-presence, 150 seven fat years for CSCW, 271 seven lean years for CSCW, 272 SGI Reality Monster, 93 Shared awareness, 84 shared awareness of location, 82 shared environment, 124, 126 shared location, 84 shared objects, 134 shared visiting experience, 78 shared whiteboard, 121 Shared Workspace, 186 shoal, 55, 59, 61, 63 shutter glasses, 93 Smartmaps, 183, 190, 191, 192, 193 social accounting, 284, 292 Social accounting data, 295, 298 Social contacts, 182 social interaction, 71, 76, 83 social surroundings, 108 social translucence, 293 society of agents, 105 solidity, 143 sonifying, 171 Sony Playstation(r), 97 Spatial awareness, 78 Spatial Interaction Model, 171 spatialised, 224 spatialised audio, 146, 215 speed of movement, 134 Spelunk, 258

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611

011

011

011

011

Index

● 1 ●●●●●●●●●●●●●● Spline, 255 SQL, 51 structuralist semiotics, 73 subjective perception, 38 subjective view, 39, 228, 245, 258 subjective visualisations, 24 subjects, 39 Symbolic Actions, 194 Symbolic gestures, 196 synchronisation, 47 synomorphy, 181 synthesiser, 96

T tangible artefacts, 72 technological limitations, 37 telephone space, 74 teleport, 177 teleporting, 27 telepresence, 117 tele-workers, 203 tethered viewpoints, 147 text chat, 116 texture, 20 texture mapping, 12 texture maps, 20 The Pond, 51, 52, 216, 219 theatre of work, 186 3D browser, 212 3D world, 93 To the Unborn Gods, 154 Touch Compass, 96 touch screen, 36 touchable interface, 37 touch-sensitive, 56 Tourist Information Centre, 31 trace, 33 Trackballs, 33 traditional media, 72 trails, 27 travellators, 168 tree-map, 192 trigger zones, 80 TWI-AYSI, 92, 94 twines, 40

ultrasonic, 95 ultrasonics, 78 unencumbered interaction, 51 urban design, 75 urban evolution, 28 urban models, 72 Usenet, 283, 291, 292, 294, 296 Utility Fog, 99

V Video avatars, 236, 238 Video imagery, 130 Videoconferencing, 117 view control, 152, 170 Virtual Communities, 271 virtual conferencing, 115, 127 virtual creatures, 61 virtual reality, 275 visual complexity, 17 visual syntax, 34 visualiser, 40, 57, 58 VR Juggler, 77 VRML, 11, 20, 86, 251 VR-VIBE, 51, 216, 218

W wayfinding, 28, 33 wearable computer, 101, 105, 108 web, 12 web browser, 11, 12, 22 web decoration, 20, 22 web page, 27 Web Planetarium, 52, 216, 219 whiteboard, 216 William Gibson, 11 working life, 275 workplace, 73

X XML, 58, 77

Z ZKM, 29, 36, 174 zooming, 61, 6

U ubiquitous computing, 101 Ultima Online, 246

11

329

Out of print titles Mike Sharples (Ed.) Computer Supported Collaborative Writing 3-540-19782-6 Dan Diaper and Colston Sanger CSCW in Practice 3-540-19784-2 Steve Easterbrook (Ed.) CSCW: Cooperation or Conflict? 3-540-19755-9 John H. Connolly and Ernest A. Edmonds (Eds) CSCW and Artificial Intelligence 3-540-19816-4 Duska Rosenberg and Chris Hutchison (Eds) Design Issues in CSCW 3-540-19810-5 Peter Thomas (Ed.) CSCW Requirements and Evaluation 3-540-19963-2 Peter Lloyd and Roger Whitehead (Eds) Transforming Organisations Through Groupware: Lotus Notes in Action 3-540-19961-6 John H. Connolly and Lyn Pemberton (Eds) Linguistic Concepts and Methods in CSCW 3-540-19984-5 Alan Dix and Russell Beale (Eds) Remote Cooperation 3-540-76035-0 Stefan Kirn and Gregory O’Hare (Eds) Cooperative Knowledge Processing 3-540-19951-9 Reza Hazemi, Stephen Hailes and Steve Wilbur (Eds) The Digital University: Reinventing the Academy 1-85233-003-1 Alan J. Munro, Kristina Höök and David Benyon (Eds) Social Naviation of Information Space 1-85233-090-2 Mary Lou Maher, Simeon J. Simoff and Anna Cicognani Understanding Virtual Design Studios 1-85233-154-2 Elayne Coakes, Dianne Willis and Raymond Lloyd-Jones (Eds) The New Sociotech 1-85233-040-6

1 2 3 4 5 6 7 8 9 1011 1 2 3 4 5 6 7 8 9 2011 1 2 3 4 5 6 7 8 9 3011 1 2 3 4 5 6 7 8 9 4011 1 2 3 4 5 611