Advances in Human Factors/Ergonomics, 20B
Symbiosis of Human and Artifact: Human and Social Aspects of Human-Computer Interaction
Advances in Human Factors/Ergonomics
Series Editor: Gavriel Salvendy, Purdue University, West Lafayette, IN 47907, U.S.A. Vol. 4 Vol. 5
Vol. 6 Vol. 7 Vol. 8 Vol. 9 Vol. 10A Vol. 10B Vol. 11 Vol. 12A Vol. 12B Vol. 13 Vol. 14 Vol. 15 Vol. 16 Vol. 17 Vol. 18A Vol. 18B Vol. 19A Vol. 19B Vol. 20A Vol. 20B
Engineering Physiology: Physiologic Bases of Human Factors/Ergonomics (K.H.E. Kroemer, H.J. Kroemer and K.E. Kroemer-Elbert) Human Factors Testing and Evaluation (D. Meister) Applications of Fuzzy Set Theory in Human Factors (W. Karwowski and A. Mital, Editors) Human Reliability: Analysis, Prediction, and Prevention of Human Errors (K.S. Park) Human Aspects of Occupational Vibration (D.E. Wasserman) Human Factors Research: Methods and Applications for Architects and Interior Designers (J.E. Harrigan) Social, Ergonomic and Stress Aspects of Workwith Computers (G. Salvendy, S.L. Sauter and J.J. Hurrell, Jr., Editors) Cognitive Engineering in the Design of Human-Computer Interaction and Expert Systems (G. Salvendy, Editor) Occupational Safety and Accident Prevention: Behavioral Strategies and Methods (C. G. Hoyos and B. Zimolong) Work with Computers: Organizational, Management, Stress and Health Aspects (M.J. Smith and G. Salvendy, Editors) Designing and Using Human-Computer Interface and Knowledge Based Systems (G. Salvendy and M.J. Smith, Editors) Designing User Interfaces for International Use (J. Nielsen, Editor) Human Factors in Product Design (W.H. Cushman and D.J. Rosenberg) Workspace, Equipment and Tool Design (A. Mital and W. Karwowski, Editors) Connective Networks in Ergonomics: General Methodological Considerations (E.A. Franus) Psychology of Systems Design (D. Meister) Human Aspects in Computing: Design and Use of Interactive Systems and Work with Terminals (H.-J. Bullinger, Editor) Human Aspects in Computing: Design and Use of Interactive Systems and Information Management (H.-J. Bullinger, Editor) Human-Computer Interaction: Applications and Case Studies (M.J. Smith andG. Salvendy, Editors) Human-Computer Interaction: Software and Hardware Interfaces (G. Salvendy and M.J. Smith, Editors) Symbiosis of Human and Artifact: Future Computing and Design of HumanComputer Interaction (Y. Anzai, K. Ogawa and H. Mori,Editors) Symbiosis of Human and Artifact: Human and Social Aspects of HumanComputer Interaction (Y. Anzai, K. Ogawa and H. Mori, Editors)
Advances in Human Factors/Ergonomics, 20B
Symbiosis of Human and Artifact H u m a n and Social Aspects of Human-Computer Interaction
Proceedings of the Sixth International Conference on Human-Computer h~teraction, (HCI International '95), Tokyo, Japan, 9-14 July 1995, Volume 2 Edited by Yuichiro Anzai Department of Computer Science Keio University Yokohama, Japan Katsuhiko Ogawa Nippon Telephone and Telegram Tokyo, Japan Hirohiko Mori Musashi Institute of Technology Tokyo, Japan
l[ ELSEVIER 1995 Amsterdam - Lausanne- New York- Oxford- Shannon - Tokyo
ELSEVIER SCIENCE B.V. Sara Burgerhartstraat 25 P.O. Box 211, 1000 AE Amsterdam, The Netherlands
ISSN 0921-2647 ISBN 0-444-81795 6 © 1995 Elsevier Science B.V. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior written permission of the publisher, Elsevier Science B.V., Copyright & Permissions Department, P.O. Box 521, 1000 AM Amsterdam, The Netherlands. Special regulations for readers in the U.S.A. - This publication has been registered with the Copyright Clearance Center Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923. Information can be obtained from the CCC about conditions under which photocopies of parts of this publication may be made in the U.S.A. All other copyright questions, including photocopying outside of the U.S.A., should be referred to the copyright owner, Elsevier Science B.V., unless otherwise specified. No responsibility is assumed by the publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material herein. This book is printed on acid-free paper. Printed in The Netherlands
PREFACE This book presents the latest advances in the research of ergomomics, human factors and social aspects of work with computer systems. The 171 papers presented in this volume were selected from those submitted to the Sixth International Conference on Human-Computer Interaction (HCI International '95) held in Tokyo, 9-14 July 1995 with the support of a grant from the Commemorative Association for the Japan World Exposition (1970). A total of 1,298 individuals from 36 countries submitted their work for presentation at this first major international meeting on human-computer interaction held in Asia. Among the submittals, only those judged to be of high quality were accepted for presentation. The papers accepted for verbal presentation, totaling 354, present recent advances in human interaction with computers and related machines in a variety of environments. The selected papers in the areas of future computing and system design in human-computer interaction are included in the accompanying Volume 1 entitled Symbiosis of Human and Arti.fact: Future Computing and Design.for Human-Computer Interaction. We are grateful for the help of many organizations which made the congress successful, and would like to thank the following sponsors of the conference: Information Processing Society of Japan Institute for Electronics, Information and Communication Engineers Japan Ergonomics Research Society Public Health Research Center The Society for Instrument and Control Engineers and the following cooperating organizations: Architectural Institute of Japan Atomic Energy Society of Japan Chinese Academy of Sciences Chinese Ergonomics Society EEC-European Strategic Programme for Research and Development in Information Technology- ESPRIT Ergonomics Society of Taiwan Finnish Institute of Occupational Health IEEE Systems, Man & Cybernetics Society IEEE Tokyo Section Indian Society of Ergonomics Institute of Management Services (UK) International Ergonomics Association
Japan Association of Industrial Health Japan Industrial Management Association Japan Institute of Office Automation Japan Management Association Japan Society for Software Science and Technology Japan Society of Health Science Japanese Cognitive Science Society Japanese Society for Artificial Intelligence Japanese Society for Science of Design Korea Research Institute of Standards and Science National Institute for Occupational Safety & Health (USA)
vi National Institute for the Improvement of Working Conditions and Environment (Thailand) National Institute of Industrial Health (Japan) Society of Biomechanisms (Japan) Software Psychology Society The Ergonomics Society of Korea
The Illuminating Engineering Institute of
Japan The Institute of Electrical Engineers of Japan The Japan Society of Mechanical Engineers The Japanese Association of Rehabilitation Medicine The Society of Heating, Air Conditioning and Sanitary Engineers of Japan.
We are most grateful to the following Board members for their fine contributions to the organization of the conference: General Chair Y0shio Hayashi, Japan Vice Chair Hiroshi Tamura, Japan
Advisory Committee Chair Kageyu Noro, Japan Organizing Committee Chair Takao Ohkubo, Japan
Advisory Board Hideo Aiso, Japan Shun'ichi Amari, Japan Takaya Endo, Japan Hal Hendrick, U.S.A. Atsunobu Ichikawa, Japan Kazumoto linuma, Japan Hiroshi Kashiwagi, Japan Akinobu Kasami, Japan Kakutaro Kitashiro, Japan
Kazutaka Kogi, Japan Takao Shirasuna, Japan Sadao Sugiyama, Japan Yotaro Suzuki, Japan Kei Takeuchi, Japan Thomas J. Triggs, Australia Keiichi Tsukada, Japan Masao Ueda, Japan Jiirgen E. Ziegler, Germany.
We thank, in particular, the Program Committee members who made their best contributions to organizing the program: Ame Aar~is, Norway Munehira Akita, Japan Yuichiro Anzai, Japan (Chair) Kazuo Aoki, Japan Albert G. Arnold, The Netherlands Eiichi Bamba, Japan Nigel Bevan, U.K. John M. Carroll, U.S.A. Yam San Chee, Singapore Marvin J. Dainoff, U.S.A. Miwako Doi, Japan Wolfgang Dzida, Germany Ray Eberts, U.S.A. Klaus-Peter F/ihnrich, Germany Emiliano A. Francisco, The Philippines
Hiroshi Hamada, Japan Hiroshi Harashima, Japan Susan Harker, U.K. Martin Helander, Sweden Herbert Heuer, Germany Michitaka Hirose, Japan Erik Hollnagel, U.K. Ken Horii, Japan Tohru Ifukube, Japan Koichi Inoue, Japan Kitti Intaranont, Thailand Hiroo Iwata, Japan Hiroyasu Kakuda, Japan Katsuari Kamei, Japan John Karat, U.S.A.
vii Osamu Katai, Japan Takashi Kato, Japan Yosuke Kinoe, Japan Bengt Knave, Sweden Richard J. Koubek, U.S.A Masaharu Kumashiro, Japan Masaaki Kurosu, Japan Nahm Sik Lee, Korea Soon Yo Lee, Korea Xu Liancang, China Holger Luczak, Germany Thomas L~iubli, Switzerland Marilyn Mantei, Canada Marvin Minsky, U.S.A. Naomi Miyake, Japan Hirohiko Moil, Japan Masaki Nakagawa, Japan Jakob Nielsen, U.S.A. Kazuhisa Niki, Japan Shogo Nishida, Japan Takeshi Nishimura, Japan Donald Norman, U.S.A Katsuhiko Ogawa, Japan Takao Okubo, Japan Choon-Nam Ong, Singapore
Olov Ostberg, Sweden Peter G. Poison, U.S.A. Jens Rasmussen, Denmark Kazuo Saito, Japan Susumu Saito, Japan Steven L. Sauter, U.S.A Dominique L. Scapin, France Pentti Seppala, Finland Thomas B. Sheridan, U.S.A. Ben Shneiderman, U.S.A. Michael J. Smith, U.S.A. T.F.M. Stewart, U.K. Yasuo Sudoh, Japan Yuzuru Tanaka, Japan Yoh'ichi Tohkura, Japan Kim J. Vicente, Canada Tomio Watanabe, Japan Runbai Wei, China Sakae Yamamoto, Japan Eiichiro Yamamoto, Japan Michiaki Yasumura, Japan Atsuya Yoshida, Japan Hidekazu Yoshikawa, Japan Richard Young, U.K.
This book, as well as the conference program, could not have been completed without the outstanding effort of Ms. Yoko Osaku, the secretariat for HCI International '95, and Mr. Akira Takeuchi of the Musashi Institute of Technology. Yuichiro Anzai, Keio University Miwako Doi, Toshiba Corporation Hiroshi Hamada, NTT Hirohiko Mori, Musashi Institute of Technology Katsuhiko Ogawa, NTT Susumu Saito, National Institute of Industrial Health
This Page Intentionally Left Blank
ix
CONTENTS Analysis, Design and Evaluation in Human-Computer Interaction
M. 1 Interaction in Context Context in Interaction Interaction in Context Context in Interaction William Edmondson, Jim Alty, Patrick Brezillon, Erik Hollnagel, John Meech, Katsuhiko Ogawa, Dan Suthers -
111.2 Winning the Market of HMS Winning the Market of Human-Machine Systems (HMS)
Elena A. Averbukh
21
Different Approaches in Visual Interactive Software Construction Olivier Esteban, St6phane Chatty, Philippe Palanque
27
Toward a Comprehensive Manipulation Solution on 3D Workspace Nobuo Asahi, Kazuhisa Okada, Akira Maenaka, Eun-Seok Lee
33
Tailoring Non-Visual Interaction in a Graphical Environment C. Stephanidis, R. Gogoulou
39
Command-Line Prediction and Error Correction Using Generalized Commandline YasuhiroUeda, Toshikazu Nishimura, Michihiko Minoh, Katsuo Ikeda
45
111.3 Interaction Design i
111.4 Interaction Design 2 FRADS: A System for Facilitating Rapid Prototyping by End Users Irvin R. Katz User Interface Development Environment for End Users CUIDE Yoshiichi Tokuda, Eun-Seok Lee, Norio Shiratori
53 59
Supporting Computer Users through Dialogue Akira Ito, Tadahiko Kumamoto, Tsuyoshi Ebina
65
A Denotational Approach for Formal Specification of Human-Computer Dialogue Kohji Matsubayashi, Yoshio Tsujino, Nobuki Tokura
?1
A Case-Based Method to Support Creative Design Incorporating Intention Recognition Takayuki Yamaoka,ShogoNishida
77
Designing Interfaces for Computer-based Assessments Randy M. Kaplan, Irvin R. Katz
83
III.5 Interaction Design 3 WMH Methodologyfor HCI Design
Christian Co~ff6
91
Guided Understanding for Problem Solving Process Using the Refining Self Explanation
Kazuhide Kanenishi, Yoneo Yano
97
A Strategy and Technology for Front End System Development Linda Candy, Ernest Edmonds, Susan Heggie, Bryan Murray, Nick Rousseau
103
A Methodology for Developing New Interaction Techniques Deborah Hix, James N. Templeman, Ankush Gosain, Kapil Danderkar
109
Basic Models for User Interface Design: Tasks, Users, Data, and Interaction Devices Chris Stary
115
Ill.6 Screen Design 1 The Effects of Realistic Versus Unrealistic Desktop Interface Designs on Novice and Expert Users
Laura A. Miller, Kay M. Stanney
Rendering Calligraphy Words with 'Kasure' Variations
123
Qinglian Guo
129
Kuniharu Takayama, Hiroyuki Kano, Yoshiharu Maeda, Kazuo Misue, Shinya Hosogi, Kozo Sugiyama
135
Dynamic Font: Its Concept and Generation Method
A Spatial Data Structure for the 3D Graphical Facility Management System Takashi Tamada, Teruhiko Teraoka, Minoru Maruyama, Shogo Nishida
Cryptographic Human Identification
Tsutomu Matsumoto
141 147
xi
III.7 Screen De sign 2 Adjustment Mechanism for a Drawing System with Stationery Metaphors Naoki Kato, Natsuko Fukuda, Masaki Nakagawa
155
Analysis Tool for Skill Acquisition with Graphical User Interfaces Based on Operation Logging Nobuko Kishi
161
The Role of Screen Parameters in Visual Communication Masaaki Kurosu, Hitoshi Yamadera, Itaru Mimura
167
Re-sampling of 3-D Object Range Data by Cube-Based Segmentation Sheng Jin Wang, Yi Cai, Makoto Sato, H. Kawarada
173
Harmonic Curve Design
John R. Rankin
179
GUIs and SUIs: More of the Same or Something Different? Alison Black, Jacob Buur
187
Ill.8 Screen Desi
3
Vision-Based Human Interface System with World-Fixed and Human-Centered Frames
Kang-Hyun Jo, Yoshinori Kuno, Yoshiaki Shirai
Fuzzy Reasoning Approach to Iconic Interface Design
193
Rungtai Lin
199
Takeharu Tanimura, Tsukasa Noma, Naoyuki Okada
205
Inferring Graphical Constraints from Users' Modification Enhancing Fold Manipulation Techniques
Ying K. Leung, Richard J. King
211
Providing Diagram User Interfaces for Interactive Theorem Proving Jun Hart, Tao Lin
217
111.9 Active Interface Active Interfaces for Useful Software Tools Amedeo Cesta, Daniela D'Aloisi, Vittorio Giannini
225
Interacting with Real Objects Real Object Interface and Transferred Object Interface Soichiro Iga, Michiaki Yasumura
231
User Identification in Human Robot Interaction Using Identification Pendant Kaoru Hiramatsu, Yuichiro Anzai
237
xii
Applying Personal Robots and Active Interface to Video Conference Systems Nobuyuki Yamasaki, Yuichiro Anzai
243
An Object-Oriented G UI for the Specification of Robotic Systems Raymond K. Wong
249
Augmented Interaction" Interacting with the Real World through a Computer Jun Rekimoto
255
Itiro Siio
261
An Object Oriented Methodology for Man-Machine Systems Analysis and Design A. Mahfoudhi, M. Abed, J-C. Angu6
267
An Analysis of Relationship between Human and Information System by Quantification Theory III Tsuneki Mukahi, Ken Murasugi, Tetsuo Ui
273
InfoBinder: A Pointing Device for a Virtual Desktop System
III. 10 Evaluation and Analysis I
Towards an Effective Subjective Measurement Method Based on Fuzzy Set Theory
Hiromi Terashita, Mieko Ohsuga, Futomi Shimono, Mamiko Toda
279
The Design and Experiment of an Evaluation Function for User Interaction Cost in the Interactive Semantic Disambiguation Masaya Yamaguchi, Nobuo Inui, Yoshiyuki Kotani, Hirohiko Nisimura
285
An Analysis of the Human-Computer Interfaces to High-Energy Physics Control Systems at CERN J.F. Meech, P. Huuskonen, E. Wagner, M. Meri, J.M. Le Goff
291
III. l 1 Evaluation and Analysis 2 PDS Analysis for Evaluating Procedural Usability on Conversational Systems Akinori Komatsubara, Masayuki Kobayashi
299
Quantitative Evaluation of Media Quality by Method of Competitive Priority Hiroshi Tamura, Jun Wu
305
Evaluation of Control Strategies in a Complex Space-Vehicle Control Task: Effects of Training Type Ranvindra S. Goonetilleke, Colin G. Drury, Joseph Sharit
311
Development of the Analysis Support System for Incidents and Troubles; "ASSIST"
Yuriko Yoshizawa, Keiko Mutoh
317
xiii
Discount Video Analysis for Usability Engineering Mark H. Chignell, Tetsuro Motoyama, Venicio Melo
323
User Interface Evaluation: Is It Ever Usable? Christelle Farenc, Philippe Palanque,JeanVanderdonckt
329
III. 12 HCI Evaluation Methodologies Software Tools for Evaluating the Usability of User Interfaces
Sandrine Balbo
337
How Usable are Usability Principles, Criteria and Standards? J.M.C. Bastien, D.L. Scapin
343
Usability is Quality of Use
Nigel Bevan
349
Deborah Hix Standards and Software-Ergonomics Evaluation Harald Reiterer, Reinhard Oppermann
355
Usability Evaluation: How Does It Relate to Software Engineering?
Using Ergonomic Rules for Evaluation by Linguistic Ergonomic Criteria Franqois Bodart, JeanVanderdonckt
361
367
HI.13 Usability Engineering A Teaching Method as an Alternative to the Concurrent Think-Aloud Method for Usability Testing Pawan R. Vora, Martin G. Helander
375
Tools for Iterative User Interface Design: UI-Tester and OST Toshiyuki Asahi, Hidehiko Okada, Osamu lseki, Ryoichi Matsuda
381
A Composite Measure of Usability for Human-Computer Interface Designs Kay Stanney, Mansooreh Mollaghasemi
387
Why Choose? A Process Approach to Usability Testing Troy Kelley, Laurel Allender
393
Usability and Quality Control of Human-Machine Interaction Elena A. Averbukh
399
Color Coordinate Supporting System with Navigating State of User's Mind Yasushi Yagi, Tomohiko Yagyu, Yoshihiko Hisamori, Masahiko Yachida
405
III. 14 Cognitive Engineering
xiv
Comparison between Three Human-Interfaces in Hospital Information System Kotaro Minato, Akira Endoh
411
Explaining Plant Design Knowledge through Means-End Modelling Pertti Huuskonen, Karl Kaarela
417
Method of Ecological Interface Design Applied to Interactive Diagnosis Support System YokoAsano, Shun-ichi Yonemura,Hiroshi Hamada,Katsuhiko Ogawa
423
III. 15 Computer Modeling of Mental Processes Computer Analysis of Characteristics of CreativeThinking and Self-esteem Level A.E. Kiv, V.A. Molyako, Stephen T. McHale, V.G. Orishchenko, L.A. Polozovskaya
431
Computer-based Testing of Reflective Thinking: Executive Control of Erroneous Performance in 9 to 12 Year Old Children Uri Shafrir
437
The Creative Thinking Testing by Using of Testing Problems Based on Different Logical Schemes A.E. Kiv, V.A. Molyako,V.L. Maloryan, I.A. Polozovskaya,Zelina I. Iskanderova
443
From Novice to Expert Decision Behaviour: A Qualitative Modelling Approach with Petri Nets
Matthias Rauterberg
449
Modeling and Simulation of Human Operator in Mental Task Handling Qualities Celestine A. Ntuen
455
The Interface Improvement for the Creative Thinking Computer Testing V.V. Chislov, V.L. Maloyran, I.A. Polozovskaya,G.V. Shtakser,A.I. Uyemov, I.G. Zakharchenko, Mafia Athoussaki
459
Evaluating Human Operator Models in Tool-based User Interface Design Maria Athousaki
463
Qiyang Chen, A.F. Norcio
471
111.16 Modeling I Associative User Modeling: A Neural Network Approach Personality Engineering: Applying Human Personality Theory to the Design of Artificial Personalities
Linda S. Endres
477
XV
Using the Template Model to Analyse Interface Specifications Christopher R. Roast, J.l. Siddiqi
483
Task Model-System model Towards an Unifying Formalism Philippe A. Palanque,R6mi Bastide, Val6rie Senges
489
III. 17 Modeling 2 Scenario Based Specification of Interaction Metaphors C. Stephanidis,C. Karagiannidis, A. Koumpis
497
Cocktail-Party Effect with Computational Auditory Scene Analysis- Preliminary Report -
Hiroshi G. Okuno, Tomohiro Nakatani, Takeshi Kawabata
503
The Effects of Rehearsal on Visual Memory Mamoru Umemura, Hiroshi lchikawa, Kenichi Teguchi
509
Mechanisms of Slips in Display-Based Human-Computer Interaction" A ModelBased Analysis Muneo Kitajima, PeterG. Poison
515
Computation Model for Human Communication Masahiro Hiji, Hiroshi Nunokawa, Masatoshi Miyazaki
521
III. 18 Voices and Faces Delivering the Promise of Speech Interfaces
Charanjit K. Sidhu, Gerry Coyle
529
VOICEDIC: A Practical Application of Speech Recognition Technology Kenji Kita, Kazuhiko Ashibe, Yoneo Yano, Hiroaki Ogata
535
An Operation Analysis of an Address Input System with Speech Recognition Kazuhiro Arai, Osamu Yoshioka, Shigeki Sagayama,Noboru Sugamura
541
A Menu-Guided Spoken Dialog System and Its Evaluation Mikio Yamamoto, Takashi Koike, Seiichi Nakagawa
547
Face Observation Using an Active Camera Qian Chen,Takeshi Fukumoto, Haiyuan Wu, Masahiko Yachida
553
Facial Features and Configurations Affecting Impressions of Faces Takashi Kato, Masaomi Oda, Misami K. Yamaguchi, Shigeru Akamatsu
559
Anthropomorphic Media Approach to Human-Computer Interactive Communication Using Face Robot Hiroshi Kobayashi, Furnio Hara
565
xvi
Ergonomics and Health Aspects of Work with Computers IV.1 Health Aspects Symptom Clusters among VDU- Workers Knut lnge Fostervold, Ivar Lie, Stig Larsen, Gunnar Horgen, Arne Aar~is, Arid V&gland
575
Construct Validity of Computer Anxiety as Measured by the Computer Attitudes Scale Dearie, F.P., Henderson,R.D., Barrelle, K., Saliba, A., Mahar, D.
581
Sick Building Syndrome: Are UK Libraries Affected? Anne Morris, Peter Dennison
587
Head-Coupled Display System- Research Issues on Health Aspects Wolfgang Felger
593
Establishment of an Expert System for Visual Display Terminals (VDT) Workers' Periodic Eye Checkups Hitoshi Nakaishi, Masaru Miyao
599
W.2 Workstation and Work Environments Ocular Motility of 72,000 VDU Operators Bruno Bagolini,Femando Molle, Marco Turbati, Domenico Lepore, Luigi Scullica
607
The Vertical Horopter and Viewing Distance at Computer Workstations Dennis R. Ankrum, Earl E. Hansen, Krisrie J. Nemeth
611
Recommendation for VDT Workstation Design Based on Analysis of Ocular Surface Area Midofi Sotoyama,Shin Saito, SasitornTaptagaporn,Susumu Saito
617
Lighting and Visual Ergonomics for the Display Screen Environment M.J. Perry, P.J. Littlefair
623
Berman Kayis, Khoi Hoang
629
Computerised Analysis of Prolonged Seated Posture Indoor Air Quality Evaluation by Continuous Measurement in Computerized Offices Akiyoshi Ito, Makoto Takahashi, Kazuhiro Sakai, Kazutaka Kogi
635
xvii
IV.3 Human Factors in Display Technology Effects of Ambient Lighting Conditions on Luminance Contrast and Color Gamut of Displays with Different Technologies Satoru Kubota
643
Display User Response by Task Lighting/Office Configuration: Implications for Flat Panel Display Users G. Sweitzer
649
Computer Workstations and Ergonomic Standards" Issues in Science and Engineering R.E. Granda, J. Greeson Jr.
655
Measurement of TFT/LCD Flicker for ISO Compliance Ryohji Yoshitake, Rieko Kataoka
661
A Psychometric Scale of TFT/LCDs with a Few Defecting Sub-Pixels Tohru Tamura, Yuhji Gohda
667
VI.4 Psychosocial Stress among VDU Workers Research Frameworks of Stress among VDU Workers - Impacts of Computerization and Task Characteristics of Computer Workers Yuko Fujigaki
,675
The Impact of Computerization on Job Content and Stress" A Seven Year Follow-Up in the Insurance Sector Tuula Leino, Kirsi Anola, Pekka Huuntanen, Irja Kandolin
681
The Impact of Office Computerization on Job Characteristics, Physical and Mental Health of Japanese Office Workers: Gender Difference Takashi Asakura
687
Effect of Computer System Performance and Other Work Stressors on Strain of Office Workers Pascale Carayon
693
Job Stressors and Depressive Symptoms in Japanese Computer Software Engineers and Managers Takashi Haratani, Yuko Fujigaki, Takashi Asakura
699
Job Stress Characteristics of Computer Work in Japan Norito Kawakami, C.R. Roberts, T. Haratani
705
xviii
VI.5 Input Devices An Integrated Haptographical User Interface Using a Force-Feedback Mouse Allan J. Kelley, T. Higuchi, S.E. Salcudean
713
Discussion on Method for Predicting Targets in Pointing by Mouse Atsuo Murata
719
The Difference of Information Input Method on Psycho-physiological Reaction of VDT Work Takao Ohkubo, MichiyoshiAoki, Mitsugu Sawa, Moritoshi Ikeda, Keun Sang Park
725
Rotating Objects Using Dials Devices
Atsumi Imamiya, Tadaaki Sakamoto
731
A New Integrated System to Assess the Amount of Information of Pointing Devices for Motor-Disabled Person Toshiyasu Yamamoto, Tetsuya Yamashina, Jyunichi Ohshima, Masafumi Ide
737
IV.6 Musculoskeletal, Postural, Visual, and Psychosocial Outcomes Resulting from Ergonomics and Optometrical Intervention Musculoskeletal, Postural, Visual, and Psychosocial Outcomes Resulting from Ergonomic and Optometric Intervention A. Aar~s, G. Horgen, M. Thoresen,A. Bugajska, A. Wolska, R. Danuta, M. Widerszal-Bazyl, M. Konarska, M.J. Dainoff, B.G.F. Cohen, M.H. Daonoff
745
A Method to Consider Ergonomic Conditions at VDT Workplaces Annika Johansson, Houshang Shahnavaz
749
IV.7 Physiological Measurements 1 Task-Related Musculoskeletal Disorders in Computerized Office Work Pentti Sepp~ila
759
Analysis of Mental Workload during the Work with Computer Using R-R Intervals Time Series Kiyoko Yokoyama, Masanori Moyoshi, Yosaku Watanabe, Takayoshi Yoshioka, Isao Tawamura, Kazuyuki Takata
765
xix
Assessment of Mental Workload Based on a Model of Autonomic Regulations on the Cardiovascular System Mieko Ohsuga, Hiromi Terashita, Futomi Shimono, Mamiko Toda
771
Experimental Study on R-R Intervals of Heart Rate by Wavelet Analysis Satoshi Kishino, Mitsuru Katoh, Yoshio Hayashi
777
IV.8 Physiological Measurements 2 CFF Values for Stress Caused by VDT Work and Relationship among Analysis of Uric Properties Masaharu Takeda, Yoshio Hayashi, Kaoru Suzuki
785
Development of a New Hand-Grasp Measurement System Yoshikazu Seki, Sigeru Sato, Makoto Shimojo, Akihiko Takahashi
791
On a Simple Method to Measure the Intensity of Keystrokes
Kaoru Suzuki
797
A Support System for Handwriting for the Blind Using a Virtual Auditory Screen Kazunori Itoh, Yoshihiro Inagaki, Yoshimichi Yonezawa, Masami Hashimoto
803
A System for 3D Motion and Position Estimation of Hand from Monocular Image Sequence Yoshio lwai, Yasushi Yagi, Masahiko Yachida
809
IV.9 Physiological Measurements 3 A Case Study on Evaluation Method for VDT Workload Using with Face Skin Temperatures Yoshinori Horie
817
Measurement of Work Load Using Brain Potentials During VDT Tasks Akihiro Yagi, Mika Ogata
823
The Relationship between Human Mental Variation and Its Application to Communication Aids Sakae Yamamotol Shigeaki Matsuoka, Sumio Yano
827
64-Channel EEG Measurement System- Applying to Stress MeasurementShin'ichi Fukuzumi
833
Analysis of Brain Activity for HCI
Mariko Fujikake Funada, Satoki P. Ninomija
839
Detection of the Event Related Brain Potential and Its Application to Communication Aids Takashi Kawakami, Michio Inoue, Yasuhiro Kobayashi, Kenji Nakashima
845
XX
IV. 10 Organizational and Psychological Aspects A Basic Experimental Study on Mental Workload for Human Cognitive Work at Man-Machine Interface Hidekazu Yoshikawa, H. Shimoda, Osamu Wakamori, Yoshinori Nagai
853
Workflow Technology Based Project Management Carlos K.H. Leung, Heloisa Martins Shih, Mitchell M. Tseng
859
Involving Workers in the Transformation of Work Organizations: Problems and Tools Irene Odgaard
865
Emotional Workload: Its Operationalization, Measurement, and Consideration in the Design of Human-Computer Interfaces Irwin Matin
871
The Psychological Impact of Computerised Production Feedback Systems: A Comparative Study of the U.K. Subsidaries of U.S. and Japanese Multinational Companies Cliff Oswick, David Grant
877
IV.I 1 HCI Standard Human-Computer Interaction Standards
Nigel Bevan
885
Frederik Dehlholm
891
Application of Ergonomic Standards to the EC Directive on Requirements for Display Screen Equipment Henrik Hopff
895
The Applicability of the ISO User Interface Standards
Structured Human Interface Validation Technique- SHIVA Jtirgen Ziegler, Michael Burmester
899
Interface for Physically Challenged V.1 Interface for Physically Challenged Composition of Messages on Winking by ALS Patients Naoyuki Kanou, Michio Inoue, Yasuhiro Kobayashi
911
xxi
Development of Language Training System for Developmentally Handicapped Children Kumiko Itoh, Kyoko litaka
917
INTERACT: An Interface Builder Facilitating Access to Users with Disabilities C. Stephanidis,Y. Mitsopoulos
923
Supporting Blind and Sighted User Collaboration through Dual User Interfaces Using the HOMER System Anthony Savidis, Constantine Stephanidis
929
Development of Human-oriented Information Systems - Learning with Mentally Handicapped PeopleYasuko Kaminuma Personal Information Appliances
935
Social Aspects, Management and Work VI.1 Information Technology Personal Information Appliances Peter J. Thomas,John F. Meech,Robert D. Macredie
945
Efficient Development of Organisations and Information Technology- A Design Approach
Jan Gulliksen, Mats Lind, Magnus Lif, Bengt Sandblad
951
Integration of People, Technology and Organization: The European Approach Christina Kirsch,Peter Troxler, Eberhard Ulich
957
Dynamic Changes of Human Systems under a Simple Task of HCI Mariko Fujikake Funada, Satoshi Suzuki, Takao Tanaka, Yusuke Yazu, Kyoko Idogawa, Chieko Hukuda, Satoki P. Ninomija
963
Temporal Organisation of Human Centred Systems V.A. Chernomorets,S.V. Kirpich
969
VI.3 Job Design Job Satisfaction in the Computer-Assisted Work Environment Andrew A. Mogaji
975
A Study on Shifting Time to Low Awakening Conditions on Monotonous VDT Works Chieko Fukuda, Satoshi Suzuki, Takao Tanaka, Keiko Kasamatsu, Yusuke Yazu, Mariko Fujikake Funada, Kyoko ldogawa, Satoki, P. Ninomija
983
XXII
Complementary Allocation of Functions in Automated Work Systems Gudela Grote, S. Weik, T. W~ifler, M. Z61ch
989
From Taylorism to Tailorability: Supporting Organizations with Tailorable Software and Object Orientation Helge Kahler
995
VI.3 The Esprit Project 8162 QUALIT, Quality Assessment of Living with Information Technology Human Oriented Management of Change. A Conceptual Model Federico Butera
1003
S. Downing, G. Ryan, A. McNeive, M. Mariani, O. Parlangeli
1011
User Requirements for Tools to Support Human Oriented Management of Change Irene Odgaard
1017
New Forms of Empowerment Using Simulation Games and Learning Form Cases K. Mertins, B. Schallock, P. Heisig
1021
The Quality of Working Life Concept
VI.6 The I CHING and Modem Science The I Ching Onto-/Axio-Genesis and the Analytic Hierarchy Process" Decisions, Negotiations and Conflict Resolutions Chung-ying Cheng
1029
Philosophy of Unity in Diversity - The Dance of Quantum and the I-Ching's Symbol -
Thomas In-sing Leung
1033
The I Ching and Non-Linear Mapping" A Meta-Binary Approach to Reflective Choice, Decision-Making, and Hierarchical Information Systems M. Secter
1037
Exploring Self-Developing Models in Computerized, Interactive Learning Environments D.A. Smith
1041
Business Rules, Revolutionary Discourse, and Multilogical Information Systems G. Tropea The I Ching as a Paradigm for Understanding Corresponding States in Fundamentally Different Systems J.W. Walls Nonlinear Computation in the I Ching Biomathematics Derived from the I Ching
1043 1047
K. Walter
1053
J.F. Yan
1059
xxiii
Author Index
1061
Keyword Index
1064
This Page Intentionally Left Blank
III. Analysis, Design, and Evaluation in Human-Computer Interaction
This Page Intentionally Left Blank
III.1
Interaction in Context" Context in Interaction
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
INTERACTION Participants: Panel chair:
EXTENDED
IN CONTEXT
- CONTEXT
IN INTERACTION
Jim Alty, Patrick Brezillon, William Edmondson, Erik Hollnage|, John Meech, Katsuhiko Ogawa, Dan Suthers William Edmondson
ABSTRACTS
C O N T E X T IN M U L T I M E D I A I N T E R A C T I O N S Prof. Jim Alty
Head of Department
LUTCHI Research Centre, Dept. Computer Studies, Loughborough University of Technology, Leics, LE11 3TU, UK. Tel: +44-1509-222648 Fax: +44-1509-211586
[email protected] Over the past five years we have been investigating the relevance of Multimedia interfaces in the process control area (Alty 1991, Alty & McCartney 1992, Alty & Bergan 1995). At the outset we assumed that the need was obvious. We saw the combination of knowledge-based support, coupled with a high bandwidth on the interface, as providing two powerful solutions to operator problems. However, the process operators were not so sure. Process operators are, all the time, balancing the volume of information available against its usefulness in a particular situation - in fact, the problem of context. As process operators see it, the only possible value in providing information in alternative media comes from the ability to: • do the task faster • do the task with fewer errors • make it easier (and more pleasant to do the task) • make learning easier and they had a suspicion that Multimedia interfaces would simply increase the amount of information being presented. Once we started to examine the literature it became clear that real examples of benefits (as opposed to features) of Multimedia interfaces were thin on the ground. The literature is full of wild claims and lacks real evaluations. We therefore decide to carry out a set of experiments both in laboratory conditions and in real plants to try to determine if Multimedia interfaces really did provide thebenefits claimed by largely unsubstantiated reports (Alty, Bergan, Craufurd & Dolphin 1994). We used a variant of the Crossman's waterbath (Crossman & Cooke 1974) experiment (used heavily in process control) and developed a set of different media interfaces to the experiment combining sound, speech, graphics, animation, colour, text and still video. Subjects had to carry out a large number of trials to achieve steady state conditions in the bath. In some conditions warnings were given (either audio or visually), in others no warnings were given. This abstract does not allow a full discussion of the results, but here are some highlights: • We found that the nature of the task and the subjects' knowledge were crucial in determining whether different media were important or not. For example, in a situation
where the task is obvious, then choice of medium has little effect. In cases where the task is difficult and the subjects have little understanding or knowledge of the problem, the choice of medium has little effect also. However, the choice of medium could be critical for success. This area of applicability is an important one since it covers the situation where operators are trying to solve problems at the limits of their competence, i.e. many emergency situations. • Whilst warnings improved task completion time they seriously affected subjects' ability to learn about the interface, particularly the more difficult concepts. • Use of realistic sound was not well received. The important result here (which seems rather obvious in retrospect) was that the medium must be able to convey the information needed to carry out the control action. When the results were analysed according to task difficulty, operator performance in difficult tasks using sound was better than without sound. In the latter situations, much larger differences were occurring, and it may be that these were such as to allow subjects to carry out realistic actions using the sound. • Subjects performed equally well with spoken and visual warnings and there was considerable positive feedback about the use of speech. However there was a bi-modal distribution in the ratings of speech and textual warnings. Those who rated textual warnings important perceived the interface task as difficult, whilst those who found the interface task easy gave the textual warnings a low rating. • The simple textual interface worked remarkably well for straightforward tasks. • The graphically oriented interfaces yielded better performance than the textual ones. This was due, in part, to much finer tuning being used in the textual interfaces presumably because of a lack of overview. The graphical advantage increased with task difficulty. The experiments in the real plant - a large Chemical Plant in Holland - were carried out over a three month period (Alty, Bergan & Schepens 1995). In a real plant it is virtually impossible to carry out controlled experiments, so our results are much more speculative. Although the operators did not have to use the interfaces at all, our multimedia interfaces were extensively used. This, in itself, must mean that the interfaces were not only usable, but at least as effective as the traditional interface. Another indirect measure of usability came from a request by the operators for our work-station to be positioned more centrally in the control room. The operator feedback was very positive for most aspects of the interfaces provided, but it was difficult to separate out the positive effects resulting from interconnected applications with the positive effects of the media themselves. Overall, we can conclude that our use of Multimedia interfaces, in specific situations, did improve performance, learning, or satisfaction, or sometimes all three aspects. However, the difficulty of the task, the knowledge of the operator, the sophistication of the surrounding software, and whether the experiments are laboratory-based or in a real plant, are all relevant issues and affect the interpretation of the results.
References Alty, J. L. (1991) Multimedia: What is it and how do we exploit it? Presented at HCI '91. Published in the proceedings, edited by D. Diaper and R. Winder, pp31 - 41. Cambridge: Cambridge University Press.
Alty, J. L., and McCartney, C. D. (1992) Design of a Multimedia Presentation System for a Process Control Environment. In L. Kjelldahl (ed.), Proc of the EUROGRAPHICS Workshop on Multimedia, pp293- 306, Springer-Verlag. Alty, J. L., Bergan, M., Craufurd, P., and Dolphin, C. (1994) Computing and Graphics, 17(3):205 - 218. Alty, J. L., and Bergan, M. (1995) Multimedia Interfaces for Process Control: Matching Media to Tasks. J. Control Eng. Practice, 3(2):241 - 248. Crossman, E. R., and Cooke, F. W. (1975) Manual Control of Slow Response Systems. In The Human Operator in Process Control, Edwards, E., and Lees, F., (Eds.), pp51 - 64. London: Taylor and Francis. Alty, J. L., Bergan, J., and Schepens. (1995) The Design of the PROMISE Multimedia System and its use in a Chemical Plant. In Multimedia Applications, Earnshaw, R., (Ed.). London: Academic Press.
TAILORING EXPLANATION CONTEXTUALIZATION
TO
USER'S
NEEDS
THROUGH
Patrick Brezillon
LAFORIA, Box 169, University Paris 6, 4 place Jussieu, 75252 Paris Cedex 05, France. Tel: +33-1-44-27-70-08 Fax: +33-1-44-27-70-00
[email protected] An explanation is a process that allows the user to assimilate a new piece of knowledge. The keystone of assimilation is explaining the plausibility of new knowledge with respect to the knowledge that the user already possesses. This must be a cooperative process where the answer given by the system is progressively adapted by both participants. We focus in this panel on the relationships between explanation and context of the cooperation. We think that a system must produce context-sensitive explanations to enhance its intervention in its cooperation with a user. Making explicit the context of the cooperation enables one to: tailor explanations to the users' needs; simplify correctly complex information for the user; paraphrase; structure the explanation; manage counter-examples; guide the research and the focus of attention; resolve ambiguities; correct misunderstanding; help learning; fill possible gaps; reduce cognitive load; develop a model of the "user-in-situation"; make sure that the participants' mental models match; and improve qualitative and quantitative performance of the explanation. Our purpose will be developed around the following example where CK stands for chunk of knowledge, SK the set of CKs that is considered at a given time, and CKo is a particular CK with which the user has a problem. I may say to a person (CKo): "I heard a lion from the window of my office this morning". This utterance has two parts: "I heard a lion roar" and "I was in my office". This is the link that may be unknown for the other person (it will be selfexplanatory for a person that is knowledgeable of me). If the person is surprised (the person has some trouble with CKo), I must develop my utterance, saying: "My office is in a university near a zoo that I can see from the window of my office. There are lions in the part of the zoo near the university. I often hear lions roar". Here, CKo is introduced after various CKs are first presented (the knowledgeable person knows these CKs: that I work in a university near a zoo, that I can see the zoo from my office and that there are lions in the zoo). However, I must make clear the links between the two parts of my utterance and share them with the person when explaining my first statement.
Note that CKs are introduced in the SK progressively, beginning with the CK that may be accepted easily by the other. In the example, I begin with "I work in a University", and then "there is a zoo near my University". Each piece of knowledge, which is presented to the other person, is explained within the context of the pieces of knowledge already shared and integrated by them with explicit explanations, i.e., with links with other CKs in the SK. The context is the shared knowledge and the mechanisms to select relevant knowledge to improve the interaction. It changes dynamically each time a CK is added or removed from the SK during this evolution. Any participant (i.e., the user or the system) may introduce a CK in the context, first to establish a link with CKo, and second to share that CK with the other participant. The dynamic aspect of context implies that it is not possible to plan in advance the whole explanatory dialogue. The context helps the user and the system to focus on the minimal number of relevant CKs at a time by making explicit the focus of attention of the explanation. Supplying a context to a system permits structuring of an important problem in explanation, namely the management of questions and answers. We note that context may enable to: support the user in formulating and asking questions; anticipate users' need for information; organize user's questions, making relationships among questions explicit; determine the fight meaning of a question in the current context; and facilitate the explanation production of different types, at different levels of details and abstraction. The user may navigate easily in the explanatory hyperspace and thus avoid the "lost-in-space" problem because the context will restrict the search space. Conversely, the explanation process may be a mechanism for managing the context of the interaction. The explanation process aims at the building of the SK in which an equivocal CK, CKo, may be assimilated by the user.
CONTEXT IN TASK ANALYSIS William Edmondson
Cognitive Science Research Centre, School of Computer Science, The University of Birmingham, Edgbaston, B15 2TT, UK. Tel: +44-121-414-4763 Fax: +44-121-414-4281
[email protected] Task analysis formalisms for HCI either misrepresent context - it is read off a higher level description (as in HTA diagrams) - or ignore it altogether (e.g. TAG). Recent approaches to menu design (e.g. Lean Cuisine (Apperley & Spence 1989), Decision Track (Edmondson 1990, Edmondson & Billups 1992), SMD (Edmondson & Spence 1992)) implicitly locate an action in the context of possible actions. However, what needs to be addressed in task analysis for HCI is the development of formalisms which explicitly address issues of context (Edmondson & Meech 1994). There are two such issues. The first concerns the modelling of the process of contextualization within the user, the second concerns descriptors for context, so that context can be incorporated in formal systems or notations. These two issues are addressed here. The process of contextualization is best understood, and thus modelled, in the following terms. Information is the process of contextualizing data. In the general case, and thus in humancomputer interaction, the human being is constantly exposed to many data from many sources. These data are not simply received or 'taken in' as information, they are perceived in the context of other data and processes and thereby become informative, and thus used. Some examples illustrate this perspective. An aircraft is fitted with many devices which produce data concerning height, speed, direction, etc. These data can be displayed on instruments, recorded in the flight recorder, and fed as input to the autopilot. The pilot contextualizes these data- there is an information process here. The autopilot also contextualizes the data - there is an information process here too (but not necessarily the same as in the human). The flight recorder
simply records the data (which may be subsequently retrieved and contextualized, by human or machine - say in an accident investigation). This model is valuable in HCI because it forces attention onto the processing. Simply displaying many data- in a cockpit, on a control panel, on a CRT - does not contextualize any of them; mere 'copresence' is not enough. The user, in a task dependent fashion, will be contextualizing the data- the information process - and may also be producing more data as 'output'. For the user to be supported in their interaction (and thus their task) the need is for the system designer to understand how to present the data in such a way as to promote, change, prompt, contextualization. This could be as simple as arranging dials in a row, so as to prompt perception of difference (visual 'pop-out', as it is called). On the other hand, contextualization support could be complex and reliant on the successful incorporation into the system of a model of the user's contextualization processes. Where this is correctly done the data are drawn appropriately and coherently to the user's attention because the system 'knows' that they are conjointly required for successful contextulaization. The model of contextualization advocated is insight promoting, but incomplete, which brings us to the second issue raised earlier. Context can be viewed as a property of data (ultimately; objects, events, etc., may 'present' the data) in that they are contextual for some other data. Thus cruising altitude, cabin lighting level, and galley oven temperature may be aircraft data with miniscule mutual relevance (they don't jointly participate in any conceivable contextualization - except this one !). However, other data will be mutually relevant in one (or more) process(es) of contextualization - and somehow this must be formalizable if a set of context descriptors is to be made available in system design. The challenge appears to be that of taking the general insight concerning contextualization and allying it with notions of task analysis to yield a formal approach which ensures that the system designer is incorporating contextualization support. In plain English this must mean moving away from task analysis and HCI as concerned with the 'what' of activity, toward a notion of the 'how' of it (cf. Edmondson 1993). But even here the challenge is toughened by the ever more urgent need to ensure that HCI is given formal methods for turning perspicacity into code. References
Apperley, M. D., and Spence, R.
(1989)
Lean Cuisine: a low-fat notation for menus.
Interacting with Computers, 1(1):45-68. Edmondson, W. H. (1990) Decision Track: A formalism for menu structure and user's selection behaviour. Presented at Human-Computer Interaction - INTERACT'90, Cambridge, U.K., 27 - 31 August, 1990. Published in the proceedings, edited by D. Diaper, G. Cockton, D. Gilmore, and B. Shackel, pp441-446 Amsterdam: North-Holland. Edmondson, W. H. (1993) A Taxonomy for Human Behaviour and Human-Computer Interaction. Presented at HCI International '93 - The Fifth International Conference on HumanComputer Interaction, Orlando, Florida, 8-13 August, 1993. Published in the proceedings: Advances in Human Factors/Ergonomics 19B: Human-Computer Interaction: Software and Hardware Interfaces, eds G. Salvendy & M.J. Smith, pp885-890. Amsterdam: Elsevier. Edmondson, W. H. and Billups, I. R. (1992) The Decision Track Formalism. Unpublished report CSRP-92-6. Edmondson, W. H., and Spence, R. (1992) Systematic Menu Design. Presented at HCI '92. Published in the proceedings, edited by A. Monk, D. Diaper, & M.D. Harrison, pp209-226. Cambridge: Cambridge University Press.
10
Edmondson, W. H., and Meech, J. F. (1994) Putting Task Analysis into Context. SIGCHI Bulletin, 26(4):59 - 63, October 1994.
S T R U C T U R A L AND F U N C T I O N A L A P P R O A C H E S I N T E R A C T I O N IN HCI Erik Hollnagel
TO M O D E L L I N G
OF
Honorary Professor of Psychology, University of Manchester & Technical Director, Human Reliability Associates
1 School House, Higher Lane, Dalton, Lancs., WN8 7RP, UK. Tel: +44-1257-463-810 Fax: +44-1257-463-121
[email protected] Structural and functional approaches The interaction between a user (human) and a machine (computer) is usually described in terms of states and state transitions. This description is very convenient for the application of formal languages and formal methods to support design. From a humane perspective it is, however, more appropriate to describe the interaction as a set of events which in prospect are loosely coupled to the user's intentions and in retrospect are strongly coupled to time. The analysis of human-computer interaction can accordingly be seen as the mapping of the set of events onto a classification scheme. A specific objective of this mapping is to determine the principles that governs the sequence of the events - or, in other words, the rules for the state transitions. The classification scheme must necessarily refer to a set of concepts about the user, in particular about the mental or cognitive functions, commonly expressed in terms of a psychological or cognitive model of the user. The necessity arises because the objective of the mapping is to find the principles that explain the temporal organisation of the events. Psychological theories and models are typically either structural or functional - although mostly the former. A structural approach refers to a set of basic assumptions about how the mind works, in terms of the components or 'mechanisms' and their individual functional characteristics, expressed in accordance with the accepted theories or models of the day. This way of explanation is in good agreement with the classical principle of decomposition, which dominates the Western approach to science. In psychology, and not least in cognitive science, the dominating theory or model is that the mind can be explained as information processing mechanism. (In the strong version of this view, the mind IS an information processing mechanism or physical symbol system (Newell 1980)). Several variations of the information processing view exist, which put the emphasis on particular types of structures, such as linguistic structures, knowledge structures, etc. The basic principle, however, remains the same, and the analytical principle is a classical decomposition In contrast to the structural approach, a functional approach refers to the dominant manifestations that can be provided through systematic observations. The emphasis is thus on WHAT happens rather than on HOW it happens. (I am aware that this somewhat begs the question, since one cannot make a systematic observation without looking for some things rather than others. However, the issue can be resolved in practice by referring to the notion of requisite variety, as it is known from control theory.) The functional approach starts from the phenomena rather than from the explanations, i.e., that it ends with a model but does not begin with it (Broadbent 1980). It thus conforms well with the maxims of minimal modelling (Neisser 1976, Hollnagel 1993), which means that it is less dependent on the model. This specifically means that the functional approach is less tempted to try to control the environment according to the presumptions of the underlying model or theory, hence less likely to construct an impoverished environment where only the responses expected by the model are wont to be noticed.
11
Cognition and context Structural approaches have always been attractive because they provide a seemingly objective frame of reference which allows us to describe the HCI as reciprocal information processing information processing in man and information processing in the machine. The disadvantage is that structural theories refer to an information processing mechanism in splendid isolation, i.e., to the hypothetically pure information processes of the brain, which are set in motion by events in the external world. Cognition is therefore seen as a kind of higher level information processing which occurs entirely within the human brain, and every effort is made to unravel the mechanisms of pure cognition. In the technological domain information processing can exist as a pure process, for instance the function of an adder or a pattern matching algorithm. But it does not make sense to speak of basic human information processes in the same manner. The fact that the information processing metaphor is useful to understand some fundamental features of human thinking does not mean that the mind is an information processor. In the late 1980s many people began to realise that cognition actually oents. This means that functional theories are intrinsically linked to the context, since the manifestations or the regularities by definition only exist in a given context. Functional theories therefore avoid the problems that stem from the notion of pure mental processes, and in particular do not make the mistake of assuming that cognition is a epiphenomenon of information processing. Thus, whereas a structural approach forces us to account for the context separately from the processes of the mind, the functional approach makes this problem disappear. The advantages of that should be obvious.
Implications for HCI The issue in HCI research is, however, not the modelling of cognition but the design of HCI. A specific HCI design is expected to facilitate the occurrence of a set of events, corresponding to meeting the functional goals of the joint system. Conversely, the design is also expected to hinder the occurrence of other events, which could put the functioning of the system into jeopardy. The value of making a distinction between a structural and a functional approach should therefore show itself in the ways we approach HCI design, specifically in how we describe and analyse events. It is a fundamental assumption for HCI design that the interaction has a purpose or an objective, i.e., that the user is trying to achieve a specific goal and that the interaction with the computers or machines is a necessary part of this. The goal could be writing a document, landing a plane, washing dirty clothes, controlling an industrial process, buying a train ticket from a machine, etc. The proliferation of computers into practically every aspect of ber of objects (buttons, menu lines) through which the interaction can take place. This solution may, however, not be practical for non-trivial tasks. Firstly, it requires that a complete analysis of all possible events has been made. Secondly, it only works if there are no significant constraints on the time available to the user. There are, in fact, only very few applications where both of these assumptions are fulfilled, even if they are relaxed somewhat. Undesired and unexpected actions cannot generally be avoided by constraining the interaction. Instead, efforts should be made to ensure that information and controls are unambiguous, both within and between different contexts. Thus, rather than assuming that the user will interpret the information in a specific way, we should consider what the possible range of interpretations is. The same goes for controls; we should carefully consider how the functioning of a control object can be understood. In particular, users may be unaware of certain options, or interpret a function differently from what the designer intended. Within contexts, the experience from a large number of event analyses have shown that it is important to avoid multiple mode indicators or multiple mode operations. Between contexts it is important that an interface attribute or object does not mean A in one context and non-A in another. A simple example is the meaning
12 of colours red and green; a more complex example is the meaning of symbols and icons. The structural approach assumes that users deliberately try to contextualise the information and controls that are part of the interface. This is a consequence of seeing cognition as a separate, internal mental process. According to the functional approach this step is unnecessary; there is no need for contextualisation because there can be no cognition without including the context. There is therefore no "pure" interface to be considered, only the interface as it is perceived. The interface, together with the tasks, the demands, etc., IS the context. The essence of interaction design is therefore to anticipate the possible contexts that can occur, and to remove any ambiguities from the information and controls that the user is exposed to. This can most effectively be achieved by means of a functional approach, because it looks at what actually happens rather than at what should happen. It furthermore has the advantage of avoiding academic discussions about whether one structural model is more correct than another. References Broadbent, D. E. (1980). The minimization of models. In A. J. Chapman & D. M. Jones (Eds.), Models of man. Leicester: The British Psychological Society. Hollnagel, E. (1993). Requirements for dynamic modelling of man-machine interaction.
Nuclear Engineering and Design, 144, 375-384. Neisser, U. (1976). Cognition and reality. San Francisco: W. H. Freeman. Newell, A. (1980). Physical symbol systems. Cognitive Science, 4, 135-183.
INTERFACES THAT HELP THE USER CONTEXTUALIZE I N F O R M A T I O N John F Meech
Centrefor Personal Information Management, Faculty of Computer Studies and Mathematics, University of the West of England, Bristol UK Tel: +44-1179-656261 x3331 Fax: +44-1179-763973
[email protected] In complex real time systems such as avionics, process control and similar environments the user or operator of the system is provided with a huge quantity of data. These data relate to current, present and possibly future system states and from this the user must decide what control action to take in order to optimise system performance. Because of the complexity of the systems in question it is generally impossible to display all relevant operating parameters to the user. 'Information Automation' is used as a technique by which a subset of the total possible data is presented to the user based on a concept of 'task' (e.g. task-based paging displays in nuclear power plants and 'glass-cockpit' aircraft). In current systems the user is driven by the behaviour of the system, and when things go wrong (and alarms are generated etc.) the user must be able to correctly identify the current state of the system (i.e. what the problem is) by placing the supplied data into context. The dynamic nature of system behaviour, particularly when alarms happen, provides different workloads for the user in order to search for the relevant data that may be displayed on several different task-based displays. This sudden transition of user workload (low when monitoring the system, high when intervening to diagnose alarms)is frequently a contributing factor to errors made in diagnosis (Meech 1992a). In order to aid the user in correctly identifying system behaviour and reduce 'operator error' I have argued elsewhere (Meech 1992b) that adaptive or intelligent user interfaces may be used to
13 constrain the data set supplied to the user and hence reduce the apparent complexity of the system. This would go some way towards constraining the set of all possible user-interface data to a relevant sub-set. In essence this process of intelligently restricting the data is one of correctly identifying the context that the system is currently operating within. The important distinction between this technique and task-based interface systems is that this is a way of composing data sets dynamically, according to the system state. This reduces the display navigation that the user must perform and may be used to supply context-sensitive user assistance. Further research (Edmondson & Meech 1994) into the nature of context as seen in humancomputer interaction has suggested that this use of dynamic context identification mirrors the process undergone in the user: context is not a static thing but a dynamic process of contextualisation. An intelligent/adaptive interface may therefore be viewed as a means of precontextualising the data for the user. As part of a preliminary investigation into the effectiveness of intelligent/adaptive user interfaces as contextualisation aids a series of experiments are currently being conducted on a representative domain. The objectives of these experiments are: ° To implement an intelligent/adaptive interface which is based on a dynamic model of user tasks which can be used to dynamically provide user aiding, • To evaluate the effectiveness of such an interface in aiding contextualisation when compared with a standard warning/alarm system and with no aiding at all. The preliminary results of these experiments will be presented at the panel, including the accuracy of the models used to contextualise by the computer. The ability to track context in this manner will therefore enable enhanced user performance in a variety of real-time systems.
References Meech, J. F. (1992a) Addressing Operator Errors in Supervisory Systems. Proceedings of Information-Decision-Action Systems in Complex Organizations IDASCO 92, IEE Digest 353, April 1992, Oxford U.K. Meech, J. F. (1992b) Intelligent Aiding in Complex, Supervisory Systems. Paper presented at the 5th IFIP/IFORS/IFAC Conference on Man-Machine Systems, The Hague, The Netherlands. June 1992. Edmondson, W. H. and Meech J. F. (1994) Putting Task Analysis into Context SIGCHI Bulletin, 26(4):59 - 63, October 1994.
INFORMATION SEARCH TO SUPPORT USER'S TASK Katsuhiko Ogawa
Nippon Telegram & Telephone, 1-1-6 Uchisaiwai-cho, Chiyoda-ku, Tokyo 100-19, Japan Tei:+81-3-3509-5944 Fax: +81-3-3503-4523
[email protected] Many database systems have been widely used to support users' tasks. They are usually very useful. But sometimes they are inconvenient if the users cannot easily operate the systems. The book metaphor approach is created to simplify database access. The user interacts with the
14 databases through a 'book like' screen and 'book like' functions such as the table of contents, the index, or browsing. Computer functions, such as string searches are also supported. A database system has been developed using the book metaphor interface to a set of about 400 human-computer interface design guidelines. The aim of the system is to support the software design review task more effectively and efficiently than the original paper version. The interface is more comfortable, but users do not quickly find guidelines appropriate for their designs. The problem is that inexperienced designers (users) do not know the details of the guidelines. Users often take a lot of time to browse guidelines and to find the most appropriate words for string searching. Although the browsing technique is inefficient, it is used more frequently than string searching. It is necessary for users to locate the appropriate guidelines effectively to produce high quality interfaces. This paper will discuss improving the access to the guideline databases based on the context of the designer's task. The most efficient search method would reproduce the experience of many users. Neural networks are a logical method of achieving this. We first propose a method to find appropriate guidelines from words employed by users using neural networks. The user can select the preferable words from the words listed by HCI experts, and then get the guidelines through the neural networks. The network is trained with the knowledge of the experts in advance. Each word from the experts corresponds to each unit in the input layer, and each guideline corresponds to each unit in the output layer. This method, however, do not always find appropriate guidelines for supporting the users' work, because the usage and meaning of the words employed by the experts and by the inexperienced designers are different. This method often suggests too few or too many guidelines. A new method is proposed: to use the list of the words used by the inexperienced designers. The list is composed of three categories: 'where is the problem', 'what is the problem' and 'how to improve it.' The method displays three categories of lists with three prompt messages: 'where','what' and 'how.' Users select each word from each category, and then the selected words are input to the neural network. This idea comes from observing many designers' behaviour in design review work with guidelines, checklists, or design specifications. Several designers, who have no human factors experience, participated in an experimental design review task. They were provided with a representation of a bad interface design. One group was instructed to improve the design by using the new neural searching, and the other group was instructed to improve by the ordinary string searching. The result indicates that both groups make similar number of improvements and task completion times, but the neural searching group produces high quality improvements and high users' satisfaction. We often find the same problem on other database applications. We believe, as the first step to the context based information search, that an approach of the neural networks with task analysis would be useful to support the user's task.
DESIGNING FOR INTERNAL vs. EXTERNAL DISCOURSE G R O U P W A R E F O R D E V E L O P I N G C R I T I C A L DISCUSSION SKILLS
IN
Dan Suthers
Learning Research & Development Center, University of Pittsburgh, 3939 O'Hara Street, Pittsburgh, PA 15260, USA. Tel: +1-412-624-7036 Fax: +1-412-624-9149
[email protected] As future "consumers" of science, today's students need to understand the process by which the claims of science are generated and revised. Towards this end, we experimenting with a
15 software environment, called "Belvedere" (Paolucci, Suthers & Weiner 1995), for supporting student's learning to engage in critical discussion of competing scientific theories. Herein we consider the locus of relevant student discourse with respect to the resources provided by supporting software, and with the implications of this locus for the design of that software. Our experience suggests that in applications where a nontrivial portion of this discourse is external to the software, we may find exceptions to standards of "good" interface design. A number of researchers have experimented with hypertext systems and graphical interfaces for supporting argumentation (Conklin & Begeman 1987, Fischer, McCall & Morch 1989, Smolensky, Fox, King & Lewis 1987, Streitz, Hanneman & Thuring 1989). For the most part, these systems are designed to provide either a medium for a genetic competent reasoner, or support for an expert user in a specific professional practice. The "Belvedere" effort seeks to support the development of scientific argumentation skills in young students. These students can't be presumed to have either general skills of constructing arguments or the specific knowledge of a domain. Therefore, the design of Belvedere has had to address the cognitive and motivational limitations and requirements of these unpracticed beginners, as presented in the psychological literature and as we encountered them in formative testing with students. Briefly, these limitations include (1) difficulty recognizing abstract relationships implicit in scientific theories and arguments, (2) difficulty focusing attention on particular problems encountered in the construction and evaluation of complex arguments, (3) lack of domain knowledge, and (4) lack of motivation. Belvedere addresses these limitations by (1 and 2) giving arguments a concrete diagrammatic form, (2) providing an automated, on-demand argumentation advisor, (3) providing access to on-line information resources; and (3 and 4) supporting students working in small groups to construct documents to be shared with others (Scardamalia & Bereiter 1991, Slavin 1990). Superficially, Belvedere is networked groupware for constructing representations (Paolucci et al 1995). The interface looks like a drawing program, but using it feels more like assembling components into desired configurations. However the utility of Belvedere's representations are primarily in their stimulus value rather than their modeling value. When Belvedere stimulates productive discourse, in some student groups much of this discourse occurs external to the representations that result. Because of this, our emphasis is primarily on designing representations the production and inspection of which stimulate critical discussion, and secondarily on representations that are adequate in themselves as a medium of communication or as the basis for final-product representations of a debate.
Examples of design issues The emphasis on stimulating critical discussion complicates the criteria for interface design. Although we design to make it easy to construct diagrammatic representations of the dialectical aspects of science, we also design to stimulate external discourse that need not be recorded in the diagram. We have found that the latter goal can overrule the utility features we would otherwise provide in support of the former. This point is illustrated with a few examples. Statements in Belvedere are embedded in shapes that represent their epistemological status (e.g., as "theory," "hypothesis," or "data"). Users often discuss the epistemological status of a statement before representing it in the diagram. An object can only have one shape at a time; therefore their discussion of the epistemological status cannot be part of the diagram. Is this a design flaw of the graphical language? Should we use an epistemologically noncommittal representation for statements, and provide annotations with which users can record any disagreement concerning the epistemological status of a statement? If the goal is to "push" all discussion into the interface, perhaps these questions are answered in the affirmative. However, it may be useful to force a decision prior to entry in the diagram precisely because it stimulates discussion towards making the decision. Otherwise, for example, students might
16 never care to discuss the difference between "data," "hypotheses," and "theories". We have not resolved this issue, but the present point is that it is a nontrivial issue. It illustrates the danger of assuming that optimizing a representation with respect to criteria of epistemological adequacy will constitute an optimization of the representation with respect to the larger task of interest. Enforcement of semantic constraints provides another example. In some versions of Belvedere, semantic constraints on the links are not enforced. For example, an "explains" link can be drawn from data to theory as well as from theory to data. (Instead of enforcement, we provide an "Advisor" that, at the user's request looks for these and other semantic anomalies that can be detected on a purely formal basis, and makes suggestions for improvements). If Belvedere were a tool for use by expert members of some community of practice, there would be no point in allowing users to make such errors. However, Belvedere is not such a tool because its users do not yet share standard terminology and practice in argumentation. Furthermore, in a learning environment we must consider the role of "errors" in the learning process. Some errors may be so superficial that they are not likely to result in a useful learning experience. Perhaps the interface should be designed to make these errors impossible, or they should be ignored as unimportant. On the other hand, delayed or absent feedback is clearly more appropriate for incomplete or problematic patterns of argumentation. Immediate "correction" could prevent users from engaging in processes of theory criticism and revision that are encountered in the real world. Investigation of whether immediate enforcement of semantics, feedback on request, or no feedback at all has a better qualitative effect on the user's discussions is ongoing. The point, again, is that this is a nontrivial issue, illustrating that interface features that are considered to be "good" for certain subtasks may be suboptimal for the larger task of interest.
Conclusions The specific requirements of our application may be somewhat unusual, but the lesson can be generalized. Design to support discourse processes must transcend the representational environment of the software itself, even in software that specifically relies on the utility of online representations for discourse. User's discourse processes take place in the social environment as well as within the representational and computational resources provided by support software. Thus, the utility of software features should be evaluated in terms of how well they stimulate the right kind of activity in the total human-computer system. We do not assume that local optimization of software support for isolated subtasks (e.g., making "correct" argument diagrams) always optimizes overall task performance. Rather, our main question is: What kind of discourse is facilitated or stimulated by each feature of the interface and of the task posed to the students, and what kind of discourse is inhibited?
Acknowledgements This research was conducted while supported by grant MDR-9155715 from the NSF Applications of Advanced Technology program. The author also thanks Violetta CavalliSforza, John Connelly, Alan Lesgold, Massimo Paolucci, and Arlene Weiner for valuable input and support.
References J. Conklin and M.L. Begeman. (1987) gibis: A hypertext tool for team design deliberation. In Hypertext '87, pages 247--252, Chapel Hill, NC, November 1987.
17 G. Fischer, R. McCall, and A. Morch. (1989) Janus: Integrating hypertext with a knowledgebased design environment. In Hypertext '89, pages 105--117, Pittsburgh, PA, November 1989. M. Paolucci, D. Suthers, and A. Weiner. (1995) Belvedere: Stimulating students' critical discussion. In CHI95 (to appear), Denver, CO, May 1995. M. Scardamalia and C. Bereiter. (1991) Higher levels of agency for children in knowledge building: A challenge for the design of new knowledge media. The Journal of the Learning Sciences, 1(1):37--68, 1991. R. E. Slavin. (1990) Cooperative Learning: Theory, Research, and Practice. Englewood Cliffs: Prentice-Hall. P. Smolensky, B. Fox, R. King, and C. Lewis. (1987) Computer-aided reasoned discourse, or, how to argue with a computer. In R.Guindon, editor, Cognitive Science and its Implications for Human-Computer Interaction. Lawrence Erlbaum. N.A. Streitz, J. Hannemann, and M. Thuring. (1989) From ideas and arguments to hyperdocuments: Traveling through activity spaces. In Hypertext '89, pages 343--364, Pittsburgh, PA, November 1989.
This Page Intentionally Left Blank
111.2 Winning the Market of HMS
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
21
W i n n i n g the M a r k e t of H u m a n - M a c h i n e S y s t e m s ( H M S ) Elena A. Averbukh Laboratory for Man-Machine Systems (IMAT-MMS) - University of Kassel (GhK), D-34109 Kassel, Germany Fax: +49 561 804 35 42 Tel: +49 561 804 27 54 e-mail: elena@ imat.maschinenbau.uni-Kassel.de This panel discussion is motivated by the increasing competition among both research and development institutions and companies in today's Human-Machine Systems marketplace. Only those who manage to respond rapidly to market conditions with a high degree of HMS differentiation and valuebased customer services adapted to the broader geography of the marketplace, efficiently save resources and environment, and - improve social health aspects, can really winn the HMS Market. -
-
This competitiveness problem challenges the enginuity of all members of enterprices and demands - clear understanding of the burning problems and goals of the analysis, design and evaluation of the human-machine systems, - role of the human factor within the whole life-cycle of the systems, - innovative techniques and strategies which support entire HMS life cycle, including its' further reuse and technology transfer, usage of new emerging technologies, e.g., multimedia human interface technology etc., appropriate criteria, technologies and standards. -
-
The structure of this market is schematically shown in Fig. 1. Normally both parties which negotiate about systems' requirements/services comprise at least two main hierarchical levels, i.e., managers at the top level and designers for HMS producers and end-users for HMS customers (industry and government) at the subordinate level. The following five main phases of the systems' life cycle are executed by these parties i.e., Establishment of the systems (safety)design goals, e.g., of plant operation and control, Rough HMS Requirements Formulation, HMS Requirements Specification, and Design & Development with End-User Participation, and Marketing and Maintenance. These phases are highly intersect with each other, particularly through the related internal and external Quality Control loops as shown in Fig.1.
• • • • •
Unfortunately, significant distance between hierarchical levels in both HMS producers' and customers' organisations, as well as cultural differences are still critical issues for winning HMS market and effective technology transfer. Hense, it demands sound discussion of the
22 - burning problems, - criteria, technologies and standards, as well as future perspective research trends related to all the above mentioned phases of HMS development and customisation at least in two "dimensions"- job organisation vs cultural aspects. Table 1 supports our preferences in selection the panelists who, to our mind, can consistently and soundly highlight the main points of the whole scope of proposed discussion within the selected "coordinates" from different points of view. Advanced design concepts, as, e.g., integrated user-centered design (G. Johannsen), vital technologies for knowledge industry (E.Averbukh), "evolutionary ergonomics" concept (K.Kawai) etc. will be presented and discussed
Table I Culture/
Producers
Geography
Designers
Japan
ToshioFukuda
Europe
•unnar Johannsen
Customers
Managers
Managers
End-Users
KensukeKawai
RajkoMilovanovich
Alberto Stefanini
Chair: Dr. Habil. Elena Averbukh (IMAT-MMS, University of Kassel, Germany) Panelists:
Prof. Dr. Toshio Fukuda (Nagoya University, Japan) Kensuke Kawai (Toshiba Corporation, Fuchu Works, Japan) Prof. Dr. Gunnar Johannsen (IMAT-MMS, University of Kassel, Germany) Dr. Rajko Milovanovich (British Telecom, Great Britain) Dr. Alberto Stefanini (CISE, Italy)
The presented work is partly supported by the Commissionof the European Union under the BRITE/EURAMII programme in Project 6126 (AMICA: Advanced Man-MachineInterfaces for Process Control Applications). The consortium partners are: CISE (Italy), ENEL (Italy),FLS Automation A/S (Denmark), Universityof Kassel (Germany), Marconi Simulation and Training (Scotland).
MARKET OF I-IUMAN-MACHINESYSTEMS
b
Rough Prqoct
F
Deslgn & Development with Encl User Pcir ticipatiori
-b
__ . .. .
I2equircrnenls f’orrnulation
--
---
b - >:-
Project Requlrcmcnis Speclflcatlnns Internal and Fxteriiol Qucllit y Control
cc, N
This Page Intentionally Left Blank
111.3 Interaction Design 1
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
27
Different A p p r o a c h e s in Visual Interactive S o f t w a r e C o n s t r u c t i o n Olivier Esteban a, St6phane Chatty a and Philippe Palanque ab aCentre d'l~tudes de la Navigation A6rienne 7 avenue Edouard Belin, 31055 Toulouse Cedex, FRANCE bLIS, University of Toulouse 1, Place Anatole France, 31042 Toulouse Cedex, FRANCE
In this paper we focus on tools and environments for visual interface development that have been proposed recently. The aim is to introduce Whizz'Ed, an experimental editor for construction of highly interactive or animated applications in order to highlight the original contribution of the Whizz'Ed approach. Whizz'Ed provide an easy-to-use environment using elementary components for visual design and development of an highly interactive interface, allowing rapid constructions by non-programmer users.
1. INTRODUCTION Since the beginning of the 80's, many tools have been developed to support user interface construction. Most of these tools can be divided in two main categories: user-interface toolkits and user-interface development systems. Though there are many of them, tools from those two categories only offer a partial solution to the problem of building graphical interfaces and more precisely to the description of their behaviour. Indeed, toolkits are difficult to use even for experienced programmers, and user-interface development systems commonly known as interface builders (e.g. Visual Basic TM) suffer from a lack of programming possibilities (programming can usually only be performed through a restricted programming language) and genericity (they are often dedicated to a certain kind of interactive systems). Hence, the design of visual programming tools (such as the ones described in [1]) for interface construction is still a stimulating challenge [ 14], especially when it comes to allowing non-specialists to develop highly interactive, direct manipulation interfaces [13]. This paper is dedicated to the visual construction of highly interactive software. The first section describes previous work on user interface construction and on visual programming. The second part presents an overview of Whizz'Ed, an experimental visual tool devoted to the construction of highly interactive or animated interfaces. In this section a snapshot of the tool is presented, and a simple example is shown. The paper ends up with a prospective view on Whizz'Ed in order to cope with non conventional interfaces such as the ones including multimodal or sound interaction.
28 2. RELATED W O R K
Most difficulties in describing the behaviour of objects in highly interactive interfaces have long been identified. In addition to the interface builders mentioned earlier, several approaches have been proposed to solve these difficulties. These approaches can be put in three main classes: constraint based systems, the state-transition diagrams, and data-flow systems. The use of constraints is well demonstrated by ThingLab, which implements that notion by providing a general simulation environment where physics laws are described by means of constraints [9]. Constraints in ThingLab are bi-directional to allow objects to be attached and updated simultaneously. Garnet [ 11 ] is another constraint-based system where constraints are Common Lisp expressions stored in the slots of objects. Garnet offers a set of tools to assist the design and rapid prototyping of user interfaces: an interface builder called Lapidary, an automatic dialog box and menu creation system called Jade and a spreadsheet for constraint specification called C32. Lapidary permits the drawing of the layout of the interface and the design of the dialogue by describing the user interface in terms of the user's actions. Jade is used to automatically create dialog boxes with a textual description. C32 uses the spreadsheet paradigm. It allows designers to create relations between the objects of the interface. NoPumpG II [ 16] is a spreadsheet-like visual language that attempts to combine the power of interactive graphics with the ease-of-use of a spreadsheet as it is possible to define constraints using a spreadsheet-like interface. It does this by allowing users to create applications that consist of spreadsheet-like cells used to control and report various attributes of interactive graphical primitives (behaviour and appearance) created in a simple MacDrawrM-like environment. However, NoPumpG II does not provide a general approach to events. More generally, constraint based systems do not allow designers to describe behaviours in a very natural way. Indeed, it is more difficult for them to express behaviours in an abstract way (such as needed with constraints) rather than describing causality links between objects (such as needed with data-flow based systems). Moreover, constraints are usually expressed in textual languages, that may be harder to understand (e.g. first order logic or temporal logic) than graphical ones [ 10] (e.g. Petri nets or State diagrams). RAPID/USE [15] uses state-transition diagrams in order to represent the behaviour of an interactive system. The nodes of the diagram represent the messages to be displayed. The arcs represent the transitions and thus fully state the appearance of messages. Transitions may occur either due to the user's actions or to external events. Iconit [4] is an environment based on state-transition diagrams for the development of iconic interfaces. It is composed of two subsystems: Iconit-D and Iconit-X. Using Iconit-D it is possible to design and specify the behaviour of the interface. The overall scheme of the user-application dialogue is represented by a state-transition diagram drawn interactively. Each node of the diagram is asssociated to a window. An arc between two nodes (nl and n2) represents the possibility to navigate from the window associated to n l to the window associated to n2. The layout of windows (e.g. menus, icons) is created independently. The windows are stored in a window base linked to the state-transition diagram. Iconit-X is composed of a verifier and an interpreter used to verify and test the interface specified using Iconit-D. Using this environment it is possible to distinguish between actions corresponding to the semantics of the application (usually known as Callbacks) and navigation actions. Such tools based on state-transition diagrams are very useful for creating interfaces as they make explicit both the representation of states and state changings. Iconit
29 describes in an efficient way the overall organization of the windows for highly interactive interfaces while RAPID/USE describes menus, buttons, dialog boxes, etc. for conventional interfaces. However, in most highly interactive interfaces the variety of choices at the user disposal is much larger than in traditional interfaces as the user can manipulate freely interfaces objects. So, the description of the sequencing of actions using state-transition diagram becomes over complicated (problem usually referenced as combinatory explosion). In order to describe in a declarative way those complex behaviours (using geometrical or temporal constraints) the data-flow paradigm has been introduced in user interface construction environments. Systems based on data-flows make programs easier to construct due to the natural understandability of data-flow diagrams. Such systems can be used by a broad range of people with different programming levels. NL [6] is a visual programming language, based on a data-flow programming model. A NL data-flow program is a directed graph. The arcs represent the paths over which tokens move between nodes, where they may be transformed into other tokens. NL uses a data-driven firing rules: a node is fired when each of its input ports holds a token. NL provides composite nodes which enable programmers to recode groups of nodes into comprehensible chunks. InterCONS [14] is another visual data-flow language, in which certain primitives are associated with interactors like buttons or sliders. Show and TellrU[8] is a visual programming language for school children. The aim of Show and Tell is to develop the programming knowledge of school children. The model used by Show and Tell is based on concepts of data-flow and completion. The basic idea is to increase the learning process by providing mechanisms for direct manipulation of values. Besides, Show and Tell uses the data-flow's intrinsic concurrency which is very useful to model communication systems. The completion is used to unify the concepts of communication, computation and data query. The Show and Tell system is presented to the children as a tool used to build and solve puzzles corresponding to their goals. Graphically a puzzle consists of boxes and arrows which are connected together. The completion shows them how far they are from the goal. Other approaches based on data-flow can be found in Prograph [5], an object-oriented visual programming language and a software development environment for the Macintosh TM. Fabrik [7] enhances the traditional data-flow model with a bidirectional data-flow. This extension permits the use of nodes that combine several functions (typically a function and its inverse).
3. WHIZZ'ED
The purpose of such a tool is to allow the creation by direct manipulation of interactive objects to build highly interactive or animated interfaces. The conceptual model of Whizz'Ed is based on the data-flow model. In order to enhance this model, Whizz'Ed uses the building games metaphor promoted by the Lego-Logo [12] construction kit which is a very interesting example of the use of this concept. Lego-Logo is a rich construction environment which allows designers to construct creatures with electronic bricks like sensors, motors, lights, logical gates, flip-flop, timers, etc. A parallel can be made between the creatures of Lego-Logo and the highly interactive applications to be built with Whizz'Ed. Assuming that point of view, the behaviour of a Lego-Logo creature corresponds to the behaviour of the components of the application. Whizz'Ed provides a set of predefined components (called elementary bricks) that can be wired together to build new components. Graphically connecting bricks together results either in creating the data-flow or in dynamically
30 reconfiguring it. Whizz'Ed proposes a visual language representing objects by icons and dataflow by lines connecting these icons. Plugs are represented by small rectangles coupled with the icon representing the object. The shape of the plug varies according to its type. An example of a Whizz'Ed construction can be seen on Figure 1. Whizz'Ed consists of three main parts: the palette, the edition area and the simulation area. • The palette contains the graphical representation of the bricks (cf. left part of Figure 1) that are supplied to the designer. The chaining of these elementary bricks in order to build complex behaviours avoids the limitations that could be introduced if only complex predifined bricks were available. Furthermore, compound bricks can be exported as elementary bricks, thus increasing the set of bricks primarily proposed. This allows designers to cope with reusability as well as hierarchical refinement in order to handle complex real world systems. • The edition area allows to build the highly interactive or animated interfaces with elementary bricks. The interface designer uses the palette (as in classical drawing tools) by direct manipulation, selecting an icon in the palette, dragging it in the edition area and dropping it at the desired place. The data-flow between bricks is also built using direct manipulation, by selecting a plug of a brick, dragging it and dropping it above another one. Syntactic constraints are automatically checked by editor in order to ensure that the types of the plugs are compatible (thus eliminating many errors at design time). An example of the edition area is shown on the middle part Figure 1. • The simulation area (cf. right part of the Figure 1) aims at graphically representing the execution of the visual program built in the edition area. This area can be used either for debugging, rapid prototyping or simulation of the construction. This example presented Figure 1 in aims at describing a rectangle moving on a screen according to a predefined elliptic trajectory, each of its movements occurring at a given time interval. Icons from left to right corresponds to: a tempo which is designed to synchronize animated objects, a rotor which is a point instrument that sends positions on an ellipsis and the rectangle itself. The data-flow is graphically represented by an arc between the tempo and the rotor and by another one between the rotor and the TopLeft input plug of the rectangle. This small visual program behaves as follows: according to its initial values in its input plugs, the tempo produces a pulse in its output plug that is put in the input plug of the rotor. The rotor reacts to this pulse by calculating the next position for the rectangle. This position is put in its output plug that is related to the input plug of the rectangle. As soon as the rectangle receives the position (the new position for its topleft comer) it changes its position on the screen. In this window, the elliptic trajectory of the rotor is shown but it could be hidden if necessary. This corresponds to the internal parameters of an object, and they can be accessed interactively by double-clicking on their icon. This basic example is dedicated to animation. However it can be easy made interactive by replacing the tempo brick by a mouse brick devoted to the management of the mouse position. Hence, the resulting construction will be able to handle user's actions.
31
Figure 1. Whizz'Ed visual construction of the animated rectangle.
4. CONCLUSION AND FUTURE WORK In this paper, we have presented different ways investigated in the area of user interface construction. Then we have rapidly presented our environment called Whizz'Ed which is a visual programming environment for highly interactive interfaces. Whizz'Ed is based on Whizz [2] a C++ library ported on most X Window systems (e.g. Sun, HE DEC). The underlying model of Whizz is based on the data-flow paradigm to which Whizz'Ed provides a direct manipulation interface. The use of graphical notation such as data-flow diagrams make Whizz'Ed usable by a broad range of people with different programming levels. The work currently done at CENA is to widen the range of applications that can be designed using Whizz'Ed. This is three fold: • extending the underlying library Whizz such as in [3] where two handed manipulation has been introduced, • extending the set of predifined bricks in order to improve the usability and the efficiency of the editor • integrating others media such as sound and video in new bricks.
REFERENCES 1. S.K. Chang. Visual languages: A tutorial and survey. IEEE Software, Jan. 1987. 2. S. Chatty. Defining the behaviour of animated interfaces. In Proceedings of the IFIP WG 2.7 working conference, pages 95-109. North-Holland, Aug. 1992.
32 3. S. Chatty. Extending a graphical toolkit for two-handed interaction. In Proceedings of the A CM UIST, 1994. 4. M. Costabile and M. Missikoff. Iconit: an environment for design and prototyping of iconic interfaces. In Journal Of Visual Languages and Computing, pages 151-174, June 1994. 5. P. Cox, F. Giles, and T. Pietrzykowski. Prograph: A step towards liberating programming from textual conditioning. In IEEE Workshop on Visual Languages, pages 150-156, Oct. 1989. 6. N. Harvey and J. Morris. NL: A generic purpose visual dataflow programming language. Technical report, University of Tasmania, Australia, Oct. 1993. 7. D. Ingalls, S. Wallace, Y. Chow, F. Ludolph, and K. Doyle. The Fabrik programming environment. In IEEE Workshop on Visual Languages, pages 222-230, Sept. 1988. 8. T.D. Kimura, J. W. Choi, and J. M. Mack. Show and tell: A visual programming language. In E. P. Glinert, editor, Visual Programming Environments: Paradigms and Systems, Jan. 1990. 9. J.H. Maloney, A. Borning, and B. N. Freeman-Benson. Constraint technology for userinterface construction in ThingLab II. In OOPSLA'89 Proceedings, pages 381-388, Oct. 1989. 10. T. Moher, B. Blumenthal, and L. Leventhal. Comparing the comprehensibility of textual and graphical programs: the case of Petri nets. In Fifth workshop on empirical studies of programmers. Albex Publishing Company, 1993. 11. B. A. Myers et al. Garnet, comprehensive support for graphical, highly interactive user interfaces. IEEE Computer, pages 71-85, Nov. 1990. 12. M. Resnick. Behavior construction kits. Communications of the A CM, pages 66-71, July 1993. 13. B. Shneiderman. Direct manipulation: a step beyond programming languages. IEEE Computer, pages 57--69, Aug. 1983. 14. D.N. Smith. The interface construction set. In Visual Languages and Applications. Plenum Pub, 1990. 15. A. I. Wasserman. Extending state transition diagrams for the specification of humancomputer interaction. IEEE Transactions on Software Engineering, SE-11:699-713, Aug. 1985. 16. N. Wilde and C. Lewis. Spreadsheet-based interactive graphics: from prototype to tool. In CHI'90 Proceedings, pages 153-159, 1990.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
33
T o w a r d a C o m p r e h e n s i v e M a n i p u l a t i o n S o l u t i o n on 3 D W o r k s p a c e Nobuo Asahi a, Kazuhis a Okada a, Akira Maenaka ~, Eun-Seok Lee b and Keiji Kobayashi ~ apersonal-use Electronics Laboratory, Mitsubishi Electric Corporation, 5-1-1 Ofuna, Kamakura, Kanagawa 247, J a p a n bDepartment of Inform ation Engineering, Sung-Kyun-Kwan University, Seoul, Korea
1. I N T R O D U C T I O N
We have already developed a system for construction and execution of 3D animated metaphor environments, named MECOT (Metaphor Environment Construction Tool), which provides i) easy and efficient construction model for designers by explicitly separating environments from graphical objects, and ii) application transparent environments for users by arranging application functions in a consistent environment [1]. As a result, it was confumaed t h a t it enables rapid construction of a variety of three dimensional graphical user interfaces (hereinafter, 3D GUIs). Since MECOT can adopt tmlimited variety of 3D objects as their interface metaphors and show animation presentation including viewpoint change triggered by user's operation and applications' events, the 3D GUIs constructed with MECOT are more intuitive to users than ordinary 2D GUIs, however, the manipulation on the 3D GUIs has become more difficult than 2D GUIs. Generally, the manipulation on 3D graphics is much harder than on 2D graphics, because there are six degrees of freedom in 3D while there are only three in 2D. Moreover, since the MECOT assumes t h a t the environment spreads over wide 3D space, users are required to move their viewpoint to accomplish their task. In order to give a good look and feel for the users, providing an easy 3D manipulation method is essential, because the feel of the manipulation strongly affects the usability of user interfaces. There are a lot of work relating to this problem [2-6], however, most of them try to solve specific issues such as object rotation and viewpoint control. In order to provide users a natural way of manipulation on the 3D GUIs, a comprehensive solution of the 3D manipulation issues should be investigated. We have categorized the issues on 3D manipulation to viewpoint control issue, short distance object placement issue, and long distance object placement issue. In this paper, we will report the result of the experiments we hadfor the viewpoint controlissue and the short distance object placement issue. The experiments were executed by
34 implementing several manipulation methods and evaluating them in terms of time spent to complete the given t a s k a n d e r r o r rate during the execution. For these experiments, we did not use special input devices for 3D manipulation.
2. ISSUES ON 3D MANIPULATION 2.1. Categorizing Issues 3D GUI assumed in this paper is a 3D graphical environment where there are various 3D graphical objects. Some of the graphical objects are translatable and/or rotatable in the environment. When users would like to accomplish their t a s k s in this environment, they should manage the combination of object translation/rotation and viewpoint movement. For example, consider a 3D GUI consists of a desk object with drawers and some document objects on the desk, and the given t a s k is to store one of the document objects on the desk to a drawer which is out of view now. The user's manipulation strategy m a y firstly move one of the documents close to the drawer, change the viewpoint to see both the document and the drawer in the same view, then open the drawer and drag off the document to the drawer. Independent of the input devices to be used, from our experiences, there are three categories of issues on 3D manipulation: viewpoint control, short distance object placement, and long distance object placement. Viewpoint control is a combination of viewpoint translation and view vector rotation. Generally, view angles used to generate scenes on screens are smaller t h a n our actual eyes so t h a t users often lose their point of interest (POI, afterwards) while they are changing their viewpoint and view vectors, and it is usually hard to recover from the lost situation. To prevent users from POI lost is an important issue. Short distance object placement is a combination of view vector rotation, object translation and object rotation. The t a s k in this category is to move objects to a different place and align them. In order to move and/or align objects, users need to rotate their view vectors because the source and the destination are not arranged to be seen in one fixed view. When users change their view vectors, generally the x, y and z direction of object movement seems different and this m a k e s users confuse& As the direction of movement depends on the local coordinate which the moving object is based on, we assume t h a t the choice of the local coordinate is a key of this category. Long distance object placement is a kind of combination of the above two. Depending on what input devices are used, all of the manipulation on the viewpoint, the view vector and objects need to be defmed as actual operations on the input devices. Optimal combination of the view control and the object placement manipulation, and optimal arrangement of the actual operations could provide a fully comprehensive 3D manipulation solution for users.
2.2. Proposed Manipulation Methods for Each Category We have developed a test environment for 3D manipulation on which several
35 different manipulation methods can be evaluate& With this test environment, we have investigated the viewpoint control issue and the short distance object placement issue to find out the best manipulation method for each issue. As for the viewpoint control issue, one of the important techniques is to give a good manipulation metaphor to users to prevent users from POI lost. According to Ware et. al., for the control of viewpoint movement in a wide-spread graphical environment like maze, flying vehicle control which gives users a control like driving a flying vehicle is better than the other two, eyeball in hand and environment in hand [2]. In this paper, we examine the flying vehicle control in more detail by adding some options. The proposed methods for viewpoint control are: (1) Flying vehicle control only (2) Flying vehicle control with R&L translation The way to move to the right and left directions is added to the basic flying vehicle control. (3) Flying vehicle control with fire-lock control It provides the way to lock the viewpoint to a specific object when users hit space key. (4) Flying vehicle control with wide-view It provides a wide-view to show more right and left view for users. Regarding the short distance object placement issue, since the choice of local coordinate is one of the important issues, we have firstly examined some local coordinate systems. The following is the local coordinate candidates. (1) World coordinate (2) Local coordinate in terms of moving object (3) Local coordinate in terms of destination object (4) Local coordinate in terms of view vector Furthermore, as we have noticed that positional guidances would be helpful for users while they are dragging objects, some of them are examined for each local coordinate. (a) Grid guidance It shows the horizontal position of the moving object on the grid on the floor. (b) Beam guidance It shows the x, y and z axes extending from the center of the moving object. (c) Both grid and beam
3. EXPERIMENT 3.1. Experiment System Figure 1 shows the block diagram of the experiment system used for the experiments. Experiment environment defmition defmes graphical environments, so we have only to change the defmition to change the experiment environments. Event manager reads a n experiment environment defmition and shows an environment according to the current viewpoint. Manipulation control manager is
36 an add-on module of the event manager. In order to test various manipulation methods, a manipulation control manager is replaced to another. The experiment system is working on IRIS 4D/340VGX and now it adopts only the three button mouse as an input device.
ManipulationControlManager I
EventManager ~-.e"
f Experiment 1 Environment t Definition
Figure 1. Block Diagram of Experiment System
3.2. E x p e r i m e n t Method Five subjects were given a t a s k for viewpoint control evaluation and a task for short distance object placement evaluation. For each task, subjects are required to accomplish it with all the proposed manipulation methods described in section 2.2. The evaluation is done by measuring time spent to accomplish tasks. All subjects did this experim ents on everyday for a week. The viewpoint control task is to see six ornaments. Each of them is put in a box only one of whose facet is open. The directions of the open facets are varied so that the subjects need to turn around the boxes to see the ornaments. The short distance object placement task is to move three plates from one place to another, and pile and align the plates. The source place and the destination place are not so far but they cannot be seen in one view. The subjects can see the destination place just by rotating their view vectors, so no viewpoint translation is require& 3.3. Result Figure 2 - 4 show the result of the experiments. Each graph shows average, minimum and maximum time spent for each manipulation metho& We can say that better manipulation method is of smaller average time and of smaller difference between maximum and minimum time. The difference between maximum and minimum time can be regarded as one of the indicators of error rate, because manipulation errors make the accomplish time longer. As a results of the manipulation experiments: (1) Flying vehicle control with fire lock control is the best for the viewpoint movement (Figure 2). (2) Giving local coordinate based on moving object is the best for the object movement (Figure 3). (3) As for the positional guidance, beam or grid depends on individual preference (Figure 4).
37 Sec. 900 800 700 600 500 400 300
0
/11 FV
I
I
+R&L
I
+FL
I
+WV
Figure 2. Viewpoint Movement Task with Various Control Methods Sec. Sec. 250
250 200
200
150
150
100
100
50
50
world
view
object
destination
Figure 3. Object Movement Task under Various Coordinate Systems
beam
grid
beam+ grid
Figure 4. Object Movement Task with Various Positional Guidances
4. D I S C U S S I O N S
According to Ware et. al., they have investigated three types of manipulation metaphors for the viewpoint control, and found that flying vehicle control is the best for walk-through and environment in hand is the best for object investigation [2]. The task given to the subjects for the viewpoint control investigation can be regarded as a combination of the two: walk through type and object investigation type. The flying vehicle control with fire lock control, which is the best way according to our experiment, is a good manipulation metaphor for this type of task, because this method provides easy switch between flying vehicle control for
38
walk-through and a similar control to environment in hand for object investigation. Furthermore, with this manipulation, a speed control of viewpoint movement can be easily implemente& Mackinlay et. al. suggested a way to control the viewpoint movement speed according to the distance toward POI [3]. The subjects mentioned after the experiments that they feel the operation for the POI setting is so natural that they can use fire lock control easily. This means, with this method, the system can get users' POI without disturbing their task. With regard to the short distance object placement issue, we could not get a significant difference among the four local coordinate, but according to the subjects' opinion after experiments, most of them said that the local coordinate in terms of moving object provides the best feeling. As for the positional guidances, we can say that giving some positional guidance significantly reduce the operation time compared with no guidance environment. It might be a good idea to show a guidance only when the user is grabbing an object. The long distance object placement needs to be studied next time based on the result of this research. We are now implementing a combined method incorporating the best of viewpoint control and the best of short distance object placement, and applying it to MECOT.
REFERENCES
1. Asahi, N" An Environment for Developing Metaphor Worlds - Toward a User-friendly Virtual Work Space based on Metaware, Proceedings of the 1994 FRIEND21 Symposium, 1994. 2. Ware,C., & Osborne,S.: Exploration and virtual camera control in virtual three dimensional environments., Proceedings of the 1990 Symposium on Interactive 3D Graphics, In Computer Graphics 24, 2, pp175-183, ACM, 1990. 3. Mackinlay,J.D., Card, S.K., & Rovertson,G.G.: Rapid controUedmovement through a virtual 3D workspace., SIC~RAPH'90 Conference Proceedings, In Computer Graphics 24, pp171-176, 1990. 4. Chen,M., Mountford, S.J., & Sellen,A.: A study in interactive 3-D rotation using 2-D control devices., Proceedings of SIC~RAPH'88, In Computer Graphics 22,4, pp121-129, 1988. 5. Houde,S.: Iterative Design of an Interface for Easy 3-D Direct Manipulation, Proceedings of CHI'92, pp135-142, 1992. 6. Bire,E.A.: Snap-dragging in three dimensions., Proceedings of the 1990 Symposium on Interactive 3D Graphics, In Computer Graphics 24,2, pp193-204, 1990.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
39
Tailoring non-visual interaction in a graphical e n v i r o n m e n t C. Stephanidis and R. Gogoulou Institute of Computer Science, Foundation for Research and Technology - Hellas, Science and Technology Park of Crete, P.O. Box 1385, GR-71110 Heraklion, Crete, Greece
The provision of alternative, non-visual interaction techniques is necessary in order to enhance non-visual interaction possibilities in a graphical environment, and fulfd the needs and preferences of blind users. This paper addresses issues related to the non-visual reproduction of a graphical environment and describes a dialogue configuration system (CONFIG) for tailoring non-visual interaction on the basis of individual blind user's needs and preferences, through 'easy-to-use' interactive facilities. 1. I N T R O D U C T I O N The proliferation of Graphical User Interfaces (GUIs) and multimedia technologies in human-computer interaction has introduced additional problems to blind users in accessing computer-based applications. This is mainly due to the fact that GUIs aim towards the exploitation of the capacity of the human visual channel; however, no alternative provision is made to ensure continued access by blind users to the same computer-based systems and applications as for sighted users (until recently, blind users had the possibility to access textbased interfaces through screen reader systems'). Currently available solutions, which address this problem through adaptations, are considered of restricted scope and applicability [ 1], [2], [3], because: • They take a rather narrow view of the problem domain; they adopt special purpose techniques for the reproduction of the user interface components in non-visual form, based either on auditory cues or tactile output; and, they do not provide alternative input methods which would facilitate blind users' interaction (i.e. mouse substitutes, speech recognition, etc). • They address specific graphical environments. • They provide limited access to the graphical interface (e.g. access only to the text within windows). • They do not support customisation of the non-visual graphical environments to the individual blind user's needs and preferences. This paper focuses mainly on the last issue by addressing: (i) Issues related to non-visual reproduction of a graphical environment utilising different special purpose devices. * A screen readeris a software system which, during user-computer interaction: (i) extracts information regarding the display content, and (ii) selectively represents the dialogue in non-visual form.
40 (ii) Interactive facilities for enabling configuration of blind user interaction in the graphical environment. In this context, we have developed a dialogue configuration system (the CONFIG system) for tailoring the graphical environment to blind users' needs and abilities. 2. NON-VISUAL INTERACTION IN A GRAPHICAL ENVIRONMENT Interaction in a graphical environment necessitates appropriate handling and manipulation of the various interaction objects of the application (e.g. menus, buttons, text) as well as navigation in the graphical environment (e.g. exploration of the screen contents). The various toolkits that may be used for the implementation of graphical applications (e.g. Motif, Athena) introduce different layout styles and interaction techniques (e.g. keyboard interaction, mouse interaction). In order to overcome the various difficulties that may arise during blind users' interaction, it is necessary to provide appropriate non-visual interfaces by combining alternative output media, different navigational facilities, and to support additional input methods. Non-visual reproduction can be based on the following approaches [4], [5]: (a) spatial representation of the screen contents, (b) hierarchical representation of the graphical interface objects, and (c) both of the above. The GUIB project (see Acknowledgements) has addressed, amongst other issues, the accessibility of the X Windows environment by blind users. In this context, we have carded out an in-depth examination of the various objects supported by the various graphical environments, and in particular, those employed in the X Windows environment, in order to identify critical issues related to non-visual reproduction of interaction objects. The various presentational and behavioural attributes of the low-level physical entities (e.g. windows, text cursor) as well as of various widget classes (e.g. pull-down menus, command buttons) have been investigated. This work has led to an identification of objects and attributes meaningful to blind users, and pointed out a number of critical issues that need to be considered for the reproduction of the interaction dialogue and of the lexical structure of the user interface in a non-visual form. The amount of graphical information to be reproduced varies depending on the approach followed (i.e. spatial representation, hierarchical representation, or both) and on the utilised media. In certain cases, specific objects and attributes concerning layout policies can be ignored as irrelevant. For example, the separator widget which is used for grouping options together (e.g. menu options referring to operations on files such as open/save/etc a file can be separated from the "exit" option using a separator) or simply for decoration purposes, can be ignored, since it does not provide any meaningful information for the non-visual reproduction. Additionally, various widget attributes (e.g. margins, shadow of a push button widget) can also be ignored, if either speech or tactile output is used; moreover, geometry attributes can be ignored in the case of speech output, although this should not happen in the case of tactile output. Another interesting case is the representation of the windows structure; if the nonvisual reproduction is based on the spatial structure of the graphical interface, then overlapping windows need to be taken into account and appropriately represented, while in a hierarchical structure such issues can be ignored. Additionally, different navigation facilities and user-support mechanisms need to be provided in each approach. For instance, special key accelerators can be used to facilitate the exploration of the hierarchical presentation of the graphical interface objects (e.g. "help" keys
41 for switching between the various active applications, for traversing menu bars), while "help" messages can be used to inform the user on the current status of the dialogue. Furthermore, the combination of alternative output media can further enhance the quality of presentation of interaction objects; for instance, the utilisation of both tactile and speech/non-speech output can facilitate the exploration of the interaction objects that constitute a dialogue box. 3. THE CONFIG SYSTEM From the above, it becomes apparent that blind users' interaction in a graphical environment can be significantly enhanced, if alternative non-visual reproductions of the graphical environments are provided and interactive facilities for combining alternative interaction techniques are also supported. In this context, the CONFIG system has been designed to support: • the selection or combination of different non-visual interaction techniques (e.g. exploration of the screen contents utilising speech, non-speech audio and tactile output) through the specification of alternative media for the various graphical objects, and • the configuration of the utilised media (e.g. speech output parameters), through the modification of the device parameters. The system is implemented in the X Windows environment using the Motif toolkit, and provides easy-to-use interactive facilities for the customisation of the non-visual interface.
3.1.
Specification of interaction techniques
On the basis of the above examination, a generalisation of the most commonly used interaction objects has been proposed, resulting into seven basic classes of objects: Windows, Text, Menus, Buttons, Dialogue Boxes, Icons, and Text Fields. This generalisation was considered necessary because of the availability of a number of toolkits that can be used for the development of graphical applications in the X Windows environment; these toolkits support nearly the same sets of interaction objects with similar behavioural attributes, but with different presentational attributes. Additionally, the different toolkits may provide different kinds of objects with minor variations with respect to their presentational and/or behavioural characteristics, although all of these objects belong to the same general category (e.g. there are different kinds of menus - popup, pulldown, menu bars - but all of them follow some common behavioural and presentational characteristics, so they are all treated as menus). As an example, Figure 1 depicts the "Motif' and "Athena" widgets that are mapped to the "Buttons" generalised class.
Motif Widgets XmPushButton XmToggleButton XmArrowButton XmDrawnButton
Generalised Object Class .1~
Buttons
~,"
Figure 1. Mapping of Motif and Athena Widgets to the Buttons Class
Athena Widgets Command
42 The dialogue configuration system supports the specification of alternative media for these general classes of objects; the output media that are supported for non-visual interaction include speech and non-speech auditory cues, and Braille output (Figure 2). The selection of one of these media, results into the utilisation of the corresponding device, while the non-visual reproduction of the current object is based on the appropriate associated attributes. For example, the selection of all the output media for windows, has the following effect in the nonvisual interface when entering an application window: (a) the window title is spoken; (b) a special sound is played to denote the "enter event"; and, (c) the window title as well as the window frames are "shown" on the transient Braille display. In the case that only speech output is used, there are three possible non-visual representation scenarios: (i) both the window title and geometry attributes (x and y position on the screen as well as width and height) are spoken; (ii) only the window title is spoken and the blind user is given the possibility to retrieve more information (about the window geometry) by pressing a special key; and, (iii) only the window title is spoken - there is no facility supported for getting any further information for the specific window. The above example supports the view that the r~ CONe'K;=
[]
i l ( ; ] i:[-~ ~[.] i 1 0 ] ~ [ ; - I - I [ / [ i [ . ]
U IHDOUS R wlr~w
including R number
iulndows.
.
.
|i! | i t ! I i ] [-]:Xl [.] i i * ] i~:! ,! , I ;] t
.
can
conEain
diFFerenE
concepts
In this
and
context,
user
-- R o v e , Reslzep Enter, TouchPad, RoutingKeMs,
-
a
the
InFor~mEion
are
related
to
also the ~allable
Fo||owing
Focus. [coniFti: House and in
Interaction
I~rFor~d via certain cams KeMboard
window)
,.
! Dia,o0=
:.. !
| Text-F-ieid--i.i ~
[-o-K-I
oF
Feedback is = i r o n v i a t h e Braille Co.=. w i n d o . F r a e t e ) , S p ~ c h (o.l~. u i n d o u nacre) and ,'~w:w,mnds ( e . g . w h i l e ~ n t : o r i n g
DeFault
[Menus
actions
regarding
I~B conewJnicatlon channels, i~ues are considered=
Im Button:
kinds
EexE, E r a p h i c s and even v i d e o .
oF
~DocIFications
For" o u t p u t :
~ra,lle
Br'aillep
Speech
and
~unds.
uu,ou,.
O
Input:
TouchPad / Routing Keys / Mouse / Keyboard
Output:
[ ] Braille
Input:
TouchPad / Routing Keys / M o u s e / Keyboard
Output:
n
Input:
Routing Keys t M o u s e / Keyboard
output:
M Braille
Braille
ml ~ p e e c n
[ ] Speech
[ ] Speech
r-I Speech
LmJ :5ouna
[ ] Sound
[ ] Sound
r-I Sound
I-CANcEtl
Figure 2. A snapshot of the CONFIG system for specifying the desired output media for the seven general classes of objects; on-line help for the Windows class
43 combination of various output media can further enhance the quality of the non-visual representation of interaction objects and empowers user's understanding of the presented screen contents. Input operations (e.g. button activation, menu exploration) are performed by means of the standard devices (keyboard or mouse) and special devices (mouse substitutes and routing keys of a Braille device, and touchpad).
3.2. Specification of device parameters The configuration of the utilised media is performed by appropriately setting the associated parameters. For example, the system supports the specification of the speech parameters (i.e. language, voice, pitch and speed) to be used according to the different modes of non-visual interaction (Figure 3). The German language may be selected, if the applications provide messages/text (e.g. button label, menu options) written in German, while messages coming from the screen reader (various help messages informing the user about the window title, entering the root window, etc) can be spoken in English. The utilisation of different speech parameters enables the blind user to be always aware of the current state of the dialogue. IE~KJ~Ii: _ell
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
L a n g u a g e for a p p l i c a t i o n s
Voice for applications ...
Pitch 1..10"
5
.
.
.
.
.
II._
.
Ill I' DEVICEPA METERS !11
I i.=.A-CT,o. .
.
...
IG~rman-=--I !
I-~
Male ca I ~
I! I
L a n g u a g e for the s c r e e n r e a d e r ...
I English ~ I
Voice for the screen
[ Male ~]
Speed 1..10"
5
reader
,..
r~
!
r~
Figure 3. A snapshot of the CONFIG system for specifying the desired speech parameters
3.3. Help facilities The CONFIG system provides a general "HELP" facility regarding the use of the system. It also provides on-line help, which is interactively activated by pressing the fight mouse button, for the seven classes of objects; brief descriptions of the main characteristics of these classes and of the basic features (e.g. object attributes, state of the object such as active, iconified) which are supported for non-visual reproduction are provided.
44 4. CONCLUSIONS The provision of flexible and effective solutions which enable customisation of the nonvisual environment on the basis of individual blind user's needs and preferences is considered very important for enhancing non-visual interaction. The CONFIG system has been designed and developed to support alternative non-visual representations of the graphical interface, as well as facilities for modifying the various non-visual interaction characteristics. Preliminary tests, which have been carried out with a small number of sighted and blind users, conf'wmed the practical value of such a system in enhancing blind users' interaction in a graphical environment. More systematic evaluation is currently under way. ACKNOWLEDGEMENTS This work has been carded out in the context of the GUIB-II (TP 215) project, partially funded by the TIDE Programme of the Commission of European Union (DG XIII). Partners of this consortium are: IROE-CNR, Italy; Institute of Computer Science-FORTH, Greece; Vrije Universiteit Brussels, Belgium; Department of Computer Science-FUB, Germany; Institute of Telecommunications-TUB, Germany; IFI, University of Stuttgart, Germany; VTT, Finland; RNIB, England; F.H. Papenmeier Gmb & Co KG, Germany. REFERENCES
1. TIDE-GUIB (TP 103) Deliverable No 5: Analysis of Textual and Graphical User Interfaces, European Commission, TIDE Office DG XIII, Brussels, December 1992. 2. E.D. Mynatt and W.K. Edwards, Mapping GUIs to Auditory Interfaces, Proceedings of the UIST '92 Conference, ACM Press, Monterey, California, 61-70, November 15-18, 1992. 3. A. Savidis and C. Stephanidis, Developing Dual User Interfaces for Integrating Blind and Sighted Users: the HOMER UIMS, Paper to appear in the Proceedings of the CHI '95 Conference on Human Factors in Computing Systems, Denver, Colorado, May 7-11, 1995. 4. G. Weber, D. Kochanek, C. Stephanidis and G. Homatas, Access b y Blind People to Interaction Objects in MS Windows, Proceedings of ECART-2, Stockholm, Sweden, May 26-28, 1993. 5. E.D. Mynatt and G. Weber, Nonvisual Presentation of Graphical User Interfaces: Contrasting Two Approaches, Proceeding of the CHI '94 Conference on Human Factors in Computing Systems, ACM Press, Boston, Massachusetts, New York, 166-172, April 24-28, 1994.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
45
Command-line Prediction and Error Correction Using Generalized Command-line Yasuhiro UEDA * and Toshikazu NISHIMURA* and Michihiko MINOH* and Katsuo IKEDA* *Department of Information Science, Faculty of Engineering, Kyoto University, 606-01, Japan A UNIX shell that maintains the history of not only command-lines but also "generalized command-lines" is proposed. A generalized command-line consists of a command name and argument symbols, and represents the syntax of a command. The system can detect an erroneous command-line and correct it, even if the system designer does not give such syntactic data to the system in advance. Moreover, by analyzing the history of generalized command-lines, the system can get information useful for command-line prediction and thus achieve better prediction. 1. I N T R O D U C T I O N Many systems are command-driven systems that interpret a command-line typed by a user and execute it. the command. To use such a system, a user must learn the function and usage of commands, and type command-lines correctly. Therefore, particularly for a novice, command-driven systems are hard to use. A shell, a command interpreter, is a typical command-driven system. Conventional shells such as the UNIX csh maintain the history of the command-lines typed by a user, and the user can retrieve a command-line typed in the past. This history mechanism is very useful. To use the history mechanism more effectively, we propose that the system maintains a history of "generalized command-lines" and use it for command-line prediction, error detection and correction. A generalized command-line consists of a command name and argument symbols such as FILE or SWITCH. The generalized command-line represents the syntax of a command. The system can detect erroneous command-lines and correct them using the generalized command-lines submitted in the past, even if the system designer does not give such syntactic data to the system in advance. Moreover, by analyzing the history of generalized command-lines, the system can get information useful for command-line prediction and thus achieve better prediction. 2. G E N E R A L I Z E D
COMMAND-LINE
A command-line which a user of the UNIX shell enters is a string that consists of a command name and zero or more argument words. We generalize a command-line by replacing the argument words with argument symbols. Considering ease of classification, frequency of use and effectiveness of prediction, error detection and correction, we define
45 the following argument symbols. SWITCH command option FILE file name DIR directory name WILD wild-card R E S T F I L E file name (it may not exist before command-line execution) RESTDIR directory name (it may not exist before command-line execution) REST the rest The procedure for replacement is carried out as follows. • If the argument word exists as directory, it is replaced with DIR; but, if it does not exist before command-line execution, it is replaced with R E S T D I R . • If it exists as file, it is replaced with FILE; but, if it does not exist before commandline execution, it is replaced with R E S T F I L E . • If it includes wild-card characters such as '*' and '?', it is replaced with W I L D . • If its first character is '-' or '+', it is replaced with S W I T C H . • Otherwise it is replaced with R E S T . 3. C O M M A N D
LINE PREDICTION
Command-line prediction is to predict the command-line that a user wants to submit next and to show the predicted candidates before he types. The prediction is performed in two steps. In the first step, the system predicts generalized command-lines. In the second step, the system predicts argument words assigned to the argument symbols in the predicted generalized command-lines. 3.1. Generalized C o m m a n d - l i n e Prediction Many commands of the UNIX system have rather limited power and a user must submit several commands to achieve a task. Therefore, many similar command-line sequences can exist in the history. As the following example, the similar command-line sequences whose difference is only argument words have the same generalized command-line sequence. command-line generalized command-line mkdir tmp mkdir R E S T D I R cd tmp cd D I R :
mkdir work cd work
mkdir R E S T D I R cd D I R
The preceding generalized command-lines can be an important key to predict generalized command-lines. Let C-i be t h e / t h preceding generalized command-line, and let n be the length of referencing sequences. The same sequences as C-n...C-1 is searched for in the history. If C_(j+,,)...C_(i+I ) is one such sequence, C_ i is a candidate. But there are many such candidates, so they should be ordered. Two of simple methods of ordering are frequency based and recency based. We call the former nD-FRQ and the latter nD-LRU. A user performs one task at a time and his submission of command-lines has locality, so LRU is superior to FRQ in general[l]. But in the case where interrupting command-lines, such as reading online manual, are inserted,FRQ is superior to LRU, because LRU is apt to be affected by these interrupting command-lines. So it will be effective to integrate LRU- and FRQ-candidates. This can be realized as follows. Let the initial evaluation
47 value e for any generalized command-line be 0. Every time a user enters a command-line, the evaluation values for all generalized command-lines are reduced by c~(___ 1) times. This operation adds the feature of LI%U. Then ~ is added to the evaluation value for the generalized command-line obtained from the command-line entered by the user. This operation adds the feature of FRQ. The ordering is made with this evaluation value. We call this ordering method WFRQ. We call the rate at which each candidate is hit, i.e. corresponds to what the user wants to enter, single hitting rate and the rate at which any one of the candidates is hit total hitting rate. The single hitting rate increases as n increases, but the number of candidates and the total hitting rate decreases as n increases (Figure 1). Therefore, we try to integrate the candidates of the various nD-WFRQ. The evaluation value of a candidate is not absolute among the various nD-WFI%Q methods and cannot be compared directly. Instead, we use expectant hitting rate(EHrate) for comparison. Let nl be the number of the candidates with evaluation various e in the nD-WFRQ method, and let n2 be the number of times that they are hit. The EH-rate of the candidate with evaluation value e in the nD-WFRQ method is then defined as ~nl. We conducted experiments to examine the effect of the integration. We compared the hitting rate of each of the single prediction methods with the integrated method. We used the command-line histories of eight users who had been using the UNIX system more than one year. These histories were 5,000 ,,~ 10,000 lines in length. We found out by the pre-experiments that integrating mD-WFRQ(m ~ 3) was ineffective, so we integrated 0,1,2D-WFRQ. We set c~ to 0.9 based on the pre-experiments. The result can be seen in Figure 1. As can be seen, the hitting rate is improved by the integration. 3.2. A r g u m e n t W o r d s P r e d i c t i o n To predict the argument words for an argument symbol, we use the 0D-WFRQ method based on the history of argument words that were assigned to the argument symbol. However, words that were not assigned to the argument symbol are not predicted in this method. Therefore even if a generalized command-line candidate is predicted successfully from the similar command-line sequences, the prediction for the argument symbols may fail. In the previous example, if 'cd D I R ' is predicted, 'work' may not be predicted for ' D I R ' . To predict 'work', the system must know that the ' D I R ' of the cd command and the ' R E S T D I R ' of the preceding mkdir are often the same word. To realize this, we use RP(A, B), the relevance of argument symbol A for B. Let n3 be the number of times that A is found in the preceding M lines including B, and let n4 be the number of times that A and B are assigned the same word in this situation. RP(A, B) is then defined as ~n 3. The candidates using WFRQ and RP are integrated also by the EH-rate. Let n~ be the number of candidates with evaluation value e, and let n6 be the number of times they are hit. The EH-rate of the candidates with the WFRQ method is then defined as On the other hand RP(A, B) is the hitting rate of the candidate for A that was the same argument word assigned to B, so the EH-rate of this candidate is just RP(A, B). 3.3. C o m m a n d - l i n e P r e d i c t i o n A command-line candidate is a combination of the generalized command-line candidates and the argument word candidates. The command-line candidates are also ordered
48 I i , 50
:
40
i
:
....
i, •
~- ~o
.1/~.
~s g°
"
//:
7
i ,
I | °
i
:
~ t ~ I K
: *
~0
,?:L~o
o...= . . . . . . . . . . . . .
i~
_" . . . .
:
,
,
0
.o.~....~zg
.
gener~,zea
..... I I ~il
o0
iii
.... ::.... : :
*
*
/
,
.
,"
~
,
.....l . . . . . .
~
,
.,~.--.~~..
,
,v--- . . . . .
",
,
- RU
:
i
. . . . . . . . . . . . . . . . .
~= so .E=
-:-..: , , :"..-~.' - . - :. - - - .. : . -.. . : . . . .. . :. . . . : - - . J
iiiiiii iii
..-;.-.i.--i.'-":'~..-i.-.2
'° . . . . ! . . . . . . . . . . . 0
:
I
:/$,,,';" ..........i ............ F.° :- ;/--.-:,.,.":"---'--.-..:..:D,--"-":'-"'-:---
~3o
10
I
.)-..-"','
'
"~
I | ,
2
4
6
8
10
o0 5:,
n u m b e r ol cadidates
10
i .......
15
i
20
25
total hitting rate
Figure 1. Hitting rate of predicted generalized command-lines,
.... i .... i-..
:, 35i
30 (%)
40
t 45
Figure 2. Single/total hitting rate of predicted command-lines.
according to the EH-rate. The hitting rate of a command-line candidate is the product of the EH-rates of the generalized command-line and all the argument words in it. It is important to improve the total hitting rate, and this is easily realized by increasing the number of candidates to be shown. But too many candidates tend to bother users. Therefore, it is important not to show the candidates whose single hitting rate is low. This is easily accomplished by not showing candidates whose EH-rate is lower than some threshold 7, because the EH-rate is the estimation value of the single hitting rate. We conducted experiments to examine the effect of the generalization, and checked the single and total hitting rate of the command-line prediction with and without generalization. The prediction method without generalization is the same as the method of the generalized command-line prediction. The result for various 7s is summarized in Figure 2. As can be seen, when 7 is large and the total hitting rate is low, the single hitting rate of the generalized method is higher than the non-generalized method by more than 10%. In conclusion, our method of generalization is effective for command-line prediction. 4. E R R O R D E T E C T I O N
AND CORRECTION
Many of traditional UNIX shells only return error messages, when a user enters an erroneous command-line. This can be a source of great frustration for users. t c s h corrects trivial spelling errors using Levenshtein distance (cf. next section). But to correct spelling errors in argument words, the users must describe what type of argument words a command takes, so in many cases the correction is performed only for command names. Bradford[2] introduced a method for detecting and correcting user errors not only syntactically but also semantically. In his method, the command-line sequence entered by the user is converted into a command paragraph, that consists of command-names and predefined symbols. The detection and correction are made by comparing the command paragraph with the correct command paragraphs. The correct command paragraphs are
49 given to the system in advance. But the user has to update them, and it is difficult to deal with the user's private commands and new commands. A generalized command-line represents the syntax of a command, and the system can detect and correct an erroneous command-line using them. Moreover, the generalized command-lines are obtained only from the command-lines entered by the user, so the system designer need not provide the generalized command-lines, and it is easy to deal with the user's private commands and new commands. In the following, we describe the detection and correction methods of the command-lines with typological and syntactic errors using the generalized command-lines. 4.1. L e v e n s h t e i n D i s t a n c e Damerau[3] reported that more than 80o£ of all typological errors are insertion, deletion, substitution or transposition of letters. The distance between two strings can be defined as the Levenshtein distance[4] by assuming each of these four errors as a unit error. The Levenshtein distance is the minimum number of unit errors that transform one string to the other. The distance between two argument symbol sequences can be defined similarly as the Levenshtein distance by assuming insertion, deletion, substitution and transposition of an argument symbol as unit errors. 4.2. D e t e c t i o n of a n E r r o n e o u s C o m m a n d - L i n e There is no way for a shell to detect an erroneous command-line completely, because the detection is performed based on each submitted command. We propose a method for detection of an erroneous command-line using the generalized command-lines. Every generalized command-line has a state. The state is one of 'erroneous', 'correct', 'unknown'. 'correct' is the state that the system infers the generalized command-line to be correct. 'erroneous' is the state that the system inferred the generalized commandline to be erroneous. 'unknown' is the initial state. If the generalized command-line obtained from the command-line entered by the user is 'erroneous', the system detects this command-line as erroneous. Because the command-line corrected by the system is not always what the user wants, the system shows the corrected command-line to the user and he confirms it by entering one of 'y', 'n', 'e'. His confirmation also determines the state of the generalized commandlines. 'y' means 'yes' and the system executes the corrected command-line. The original command-line will be erroneous and the corrected command-line will be correct, so the system determine the state of the original generalized command-line to 'erroneous' and the state of the corrected generalized command-line to 'correct'. 'n' means 'no' and the system executes the original command-line. The original command-line will be correct, so the system determines the state of the original to 'correct'. 'e' means 'edit' and the user edits the original command-line and the system executes the edited command-line. The original command-line will be erroneous and the edited command-line will be correct, so the system determines the state of the original to 'erroneous' and the state of the command-line after the editing to 'correct'. 4.3. C o r r e c t i o n of an E r r o n e o u s C o n n n a n d - l i n e The system corrects the detected an erroneous command-line by applying the following unit-correcting operations each of which corresponds to each of the four unit errors.
50 • For deletion of an argument word, complete argument word in the same method using the argument word prediction described in section 3.2. The cost is 2. • For insertion of an argument word, delete the word. The cost is 4. • For transposition of argument words, transpose these words. The cost is 4. • For substitution of an argument word, substitute the original word with the nearest word. The cost is the distance between those two words. The system selects the command-line with the minimum total cost as the corrected command-line candidate. There are often more than one corrected command-line whose costs equal to the minimum. In such a case, the system selects the one with the lowest EH-rate (described in section 3.1), because the command-line prediction can be assumed as the command-line correction of the null command-line.
4.4. Experiment As an experiment, four users used an extended shell that performed the detection and correction according to the method described above, and entered 11,657 command-lines altogether. 509 of them were detected as erroneous, but 234 of them were actually correct. This is too many to ignore. If the system so frequently detects the correct command-lines as erroneous, the user doubts the system and ignores the detection. Thus, it is important to reduce incorrect detection. This may be done by relaxing the condition for detection. The remaining 275 of 509 were properly detected. 115 of 275 were corrected properly, and 106 were corrected properly in the generalized command-line level, but the corrected argument words were improper. It is necessary to improve the argument word correction. 5. C O N C L U S I O N We have proposed a system that maintains the history of generalized command-lines. Through experiments we showed that by analyzing this history such a system can get information useful to command-line prediction and thus achieve better prediction. A generalized command-line represents the syntax of a command. The system can detect and correct a erroneous command-line using the generalized command-lines which are obtained only from the command-lines entered by the user. However, our detection and correction methods need improvement. REFERENCES 1. S. Greenberg and I. H. Witten: Supporting Command Reuse: Mechanisms for Reuse, Int. J. Man-Machine Studies, 39-3 (1993), 391-425. 2. J.H. Bradford, W. D. Murray, and T. T. Carey: What Kind of Errors Do Unix Users Make?, Proc. IFIP INTERACT'90: Human-Computer Interaction (1990), 43-46. 3. F.J. Damerau: A Technique for Comuter Detection and Correction of Spelling Errors, Commun. A CM, 7-3 (1964), 171-176. 4. V.I. Levenshtein: A Method of Constructing Qua~ilinear Codes Providing Synvhronization in the Presence of Errors, Problems of Information Transmission, 7 (1971), 215-222. 5. J. H. Bradford: Semantic Strings: A New Technique for Detecting and Correcting User Errors, Int. J. Man-Machine Studies, 33-1 (1990), 399-407.
III.4 Interaction Design 2
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
53
FRADS: A System for Facilitating Rapid Prototyping by End Users Irvin R. Katz Division of Cognitive and Instructional Science, Educational Testing Service, Princeton, New Jersey 08541 USA While user interface toolkits and managers facilitate prototyping by programmers, few systems allow nonprogrammers to create their own applications. In this paper, we report some techniques that bring prototyping to nonprogramming domain experts, namely professional test developers at Educational Testing Service. The Free-Response Authoring and Delivery System (FRADS) allows professional test developers to create dynamic, working prototypes of computer-based test questions. FRADS was designed to leverage nonprogrammers' experience with commercial graphics packages. Test developers create questions by importing graphics and other user-interface objects, choosing the tools to provide to students in responding to the question, and delineatingmvia dialog boxes and specially designed graphical objectsmhow the tools and provided interface objects interact. With FRADS, we explore how much "programming power" can be obtained by using direct, graphical specification of applications. 1. INTRODUCTION*
Traditional user interface management systems (UIMSs) rely on a "widgets plus programming" framework to enable prototyping. As a result, few systems allow nonprogrammers to create their own applications. In this paper, we report a system designed to bring prototyping to nonprogramming domain experts--in this case, professional test developers at Educational Testing Service. The Free-Response Authoring and Delivery System (FRADS) allows professional test developers to create dynamic, working prototypes of computer-based test items (such items resemble specially tailored graphical editors). + The authoring component of FRADS makes it possible for a item author with no programming experience to create graphical items on the computer, interact with the items as a test taker (i.e., student) would, and immediately make necessary revisions. A focus of this work is on the set of techniques that allow prototyping without the use of formal programming languages. Free-response items require students to create their own responses rather than selecting a response from a set of altematives (i.e., multiple choice). In a typical test item, a student is presented with background information on the computer screen and uses the mouse or keyboard to respond. Responding to an item might involve selecting a point on a plotted function, moving a curve onto a coordinate system, entering the result of a calculation, etc. In other words, each item may be thought of as a highly-restricted graphics editor, in which information is presented and only certain types of responses (e.g., draw a line, alter a line) are permitted. A sample item is shown in Figure 1. For this item, a student must select a curve and place it onto the axes in a position that corresponds to the equation. * I thank Daniel Zuckerman for his assistance with the implementation of FRADS. I thank Ruby Chan, JungMin Lee, and Kevin Singley for comments on earlier drafts of this report. This research was funded in part by the Graduate Record Examination Board and Educational Testing Service. + A "test item" refers both to the test question (i.e., what is being asked) and the means for responding to that question (e.g., multiple-choice alternatives).
54 1.1. P r o t o t y p i n g f r a m e w o r k s In the "widgets plus C I i c k on an o b j e c t
to sl I ect
i t.
Ho I d d o l n
t h e mouse b u t t o n t o moue the o b j e c t programming" framework, UIMSs provide programmers with typical user-interface Item elements, or "widgets" (e.g., I-..'- .L~ _L-!-:4--., .;i....l-÷-{--i-'--'- s • -,-t--{ .-t-iT1-1 sliders, icons, text fields), and 0 v--f-t-r-i-,..-,-+- :]~LT]-j-.]il.Tj Untried ilili.l'ii: a programming protocol for Fv-t-r-t-N-t-;interaction among the widgets. Although some systems allow -;.' -~-4-4+~..i.-i-.i- '< "-i'-l-'i'-.rr-Fl-l-~ nonprogrammers to arrange i..-~.~-'.+.-..-i....-4. bi--;-f-bi--.:-+:~...-,.-..~....t....~.+.~----:-,. widgets on the screen, the behavior of widgets--how they react to user actions and Produce the graph of the equation Next communicate with each other-Untrkd ( x - 1)~+ ( y - 3)' 4 4 =1 must be specified via a formal by selecting one of the given curves and programming language [2]. positioning it in the coordinate system. Next Item Even systems geared for the c o m m o n user, such as "~ Hypercard and Visual Basic, Figure 1" Move object item follow the widgets plus programming framework, albeit using simplified programming languages. This approach provides a great deal of flexibility in the range of prototypes that can be developed. Unfortunately, it also excludes nonprogrammers from the benefits of prototyping. One method of accelerating the production of domain-specific applications is to enable domain-experts themselves, who lack programming skills, to create their own prototype applications. The challenge is for a UIMS to retain flexibility without relying on a programming language.
Pr£s
U
#
An alternative approach is to provide prototype builders not only with a collection of interface objects, but also with operation "objects" that can act on these widgets. This approach achieves flexibility because prototype authors can combine different widgets with the different operations in a variety of ways. Furthermore, in this framework, authors specify via direct manipulations how the widgets and operations interact. In other UIMSs, operation "objects" and their behavior would be specified through a formal programming language, and thus be inaccessible by nonprogrammers (but see [4]). There are three main elements in the FRADS prototyping framework (Figure 2): widgets, tools, and allowed actions. Like widgets of other UIMSs, those of FRADS are user interface elements that may have associated attributes, such as the font size in a text field. The second element of this end-user prototyping framework are tools. Tools are conceptually the same as the operations provided in commercial graphics packages; tools are the "verbs" that act on the "nouns" of the interface (i.e., widgets, screen). Finally, mediating between widgets and tools are allowed actions, which authors use to specify which tools may act on which widgets. For example, certain graphics may be background information, so should be immutable by the student, while other graphics may be manipulated (e.g., moving a curve onto a set of axes).
FRADS Tools
Widgets • Lines, arrows • Pictures • Text boxes • On-screen keyboards • Static text
• • • • • •
Move object Shade region Rotate object Draw line Enter text Erase object
Allowed Actions • Which objects may be moved, copied, rotated, shaded, etc. • Text entry via on-screen keyboard, hardware keyboard, or both
Figure 2: Sample elements of prototyping framework
55
2. FREE-RESPONSE
AUTHORING AND DELIVERY SYSTEM
A key requirement of any "end-user programming" environment is that it allow nonprogramming domain experts to make use of their own expertise [3]. FRADS capitalizes on item authors' familiarity with commercial graphics applications. Typical graphics applications use a "palette-canvas" interface metaphor. The application provides a set of tools that determine what gets "painted" on the canvas, what changes occur to existing objects on the canvas (e.g., rotation), and what gets removed from the canvas. FRADS generalizes this metaphor into a prototyping system. The "palette" of each test item is editable by the item author, who selects the tools available to students in answering the item. The "canvas" is stocked with background material (e.g., the item's question) and the objects that the student can manipulate using the provided tools. Students manipulate objects on the canvas to create their response. Item authors can further customize students' use of tools by specifying which objects can be affected by which tools. 2.1. Example of authoring
This section presents an example of using FRADS to develop a simple free-response item. The screen layout of FRADS's authoring component is essentially a blank delivery screen (the same screen that a student sees when taking a test, e.g., Figure 1), with an additional set of palettes on the right (Figure 3). One palette contains the tools that an item author can provide to test takers. A second palette contains several user-interface widgets, such as text fields and an on-screen keyboard, which may be added to the "canvas" of test items. Note that the system does not provide sophisticated graphic-drawing facilities; instead, an author may use the graphics application he/she is most familiar with, and then copy-and-paste the graphic onto the item's canvas. Overall, authoring an item consists of (a) choosing the appropriate tools for students, (b) selecting the widgets to include, (c) importing graphics from a graphics application, and (d) specifying restrictions on students' use of provided tools (e.g., only certain imported objects are moveable). In the example below, we assume that any needed graphics already exist and that an appropriate graphics package is running concurrently with FRADS. The item to be created requires students to choose the appropriate curve and move it onto the coordinate system such that the curve satisfies the stated equation (Figure 1). To get the background information for the item, the author switches to the graphics application and copies the coordinate system and problem statement. The author then switches back to FRADS to paste and position the graphic on the canvas. Next, the author specifies the required response tools by clicking on the appropriate button and placing that button into the tool palette. The only tool needed in this case is the "move object" button, although other items may use multiple tools. Now the author must copy separately each of the closed curves into FRADS. Each curve must be copied separately because each must be able to be manipulated by the student. Finally, the author must specify that the closed curves are moveable by the student. By default, FRADS assumes that any imported graphic is part of the background and so is immutable by the student. Double-clicking on a closed curve brings up a dialog box through which the author can specify what actions students may perform on the object (e.g., one or more of: move, erase, rotate). The author specifies that the curve may be moved, then closes the dialog box and repeats this process for the other curves. The author then names the item, saving it onto disk, which completes the authoring of this item. Once an item is saved, the author may tryout the item, bringing up a sample delivery window with the actual, working item. This facility allows authors to review the item and, if necessary, edit the item if it does not perform as expected.
56
Tool valette. Authors add buttons (i.e., response tools)here. ,ak
I.
Response tools, The author choses the tools to be available to students for creating their answer.
File Edit Options Add New Item
I Buttons ~L I
@ Previous Item
Held window. During item delivery, this area contains a short message describing for the ~.~ student how to use the currently selected tool (e.g., Figure 1). The text changes when a new tool is ~rl*~s selected. This area is merely a placeholder in the authoring system.
]
;
"
'
rDZT "rCXT [NTI[RT[XT
" - ...... [ i ~ -,toe,toT
-Ira- sxo~
It0TaTl[
.o~,, ~-~Rtszz[ LZ.E -~
S[L[¢T][ON
(Objects }
m ....
n
K[YBO~ItD
/
Next Untried
Canvas. This area contains the test question and is where students create their response.
Next Item
O_]~ects. gesides imported graphics, the author can add these widgets to an i t e m ' s canvas,
Figure 3" Item authoring environment In summary, authors build interfaces (i.e., test items) that limit students' interactions. That is, authoring consists of effectively programming the actions available to a student when responding to a particular item. The item author specifies what objects the tools can affect, where on the screen the tool can be applied, and in what way the tool is applied (e.g., move an object versus copy an object). For each item, students are given only the tools they need, which have been "tuned" to the particular task. 2.2. Declarative specification of tool use
In the example above, an author specified via a simple dialog box whether objects were moveable by students. Specifying which tools can act on which objects is the simplest way that authors "program" how their items behave. More complex control of student response tools can be achieved through use of special-purpose graphical objects. The way that shading is specified provides an example of using special-purpose objects to "program" a tool's behavior. By default, the shade tool works similarly to such tools in graphics applications: from the point where a user clicks, the system "flood fills" the region with the shade pattern until a boundary (solid line) is encountered. Figure 4 shows a simple shade-region item. When the student clicks on the x-axis, a rectangular area between two adjacent tick-marks is shaded (the five leftmost areas are shaded in the figure). Clicking on a shaded area causes that shading to disappear. If the student clicks elsewhere on the screen (e.g., on the y-axis), no shading occurs. Also, note that even though there are no visible boundaries on the shaded rectangles, shading does not "flood" the entire screen. In a graphics application, shading is bounded only by what's visible on the screen. This item demonstrates that authors can control shading in two ways, specifying (1) the shadeable portions of the visible graphic and (2) the subareas on the screen to be shaded, irrespective of whether those areas are visibly bounded.
57 IClick on a re ion to shade i t ; lick to unshade itg c again
@ Previous
Item
_.~=
Untried
e(x)
Next Untried
S h a d e the l a r g e s t interval of x - v a l u e s f o r which the function y =
f(x)
h a s an i n v e r s e .
Next Item
Figure 4: Sample response to "Shade" item FRADS provides two special-purpose graphical widgets: masks and boundaries. Both of these objects are created and placed by the author, but are not visible to students. Masks specify the allowable areas of a graphic widget that may be shaded. Shading of the visible graphic will occur only if the student clicks in a location that corresponds to an "on" bit in the mask. Boundaries allow the item author to specify invisible (to students) shading boundaries associated with a graphic. Shading becomes bounded by the lines in the boundary object rather than by the lines in the visible graphic. Figure 5 shows the corresponding mask and boundary of the item in Figure 4. The mask shows that shading can occur only in the area around the xaxis; the boundary object indicates that the subareas to be shaded are rectangles between adjacent tick marks (the y-axis and labels are included in each object to aid the author in aligning these graphic objects to the visible graphic). Note that a to-be-shaded graphic might have only an associated mask, only a boundary, or neither, depending on the needs of the item.
I
I
I
I
I
I
I
I
I
p.x
Figure 5: Corresponding shade mask (left) and shade boundary (fight) for Figure 4 These special-purpose objects represent a powerful end-user programming approach. Rather than controlling a tool's action through a programming language, or even through the data-flow representation of many visual programming languages (e.g., [ 1]), the special-purpose objects allow control of tools via simple graphical mechanisms, easily understood by nonprogrammers.
58
3. SYSTEM
EVALUATION
Preliminary evidence suggests that FRADS successfully meets its going of providing a usable rapid prototyping environment to nonprogrammers. FRADS has been used by eight test development staff at ETS, who all volunteered to use the system in their daily work. These authors were trained to use the basics of FRADS within a few hours (2-3 hours). After that time, authors could create items largely on their own. Using FRADS, these item authors have implemented approximately 230 distinct items, which can be classified into approximately 70 different item "types" (e.g., graph a function, arrange words in a sentence, indicate an area on a graph). Anecdotally, authors report that they explore a wider variety of item designs using FRADS compared with creating items on paper (which would be put onto computer by production staff). Similar to programmer-oriented UIMSs, FRADS appears to accelerate the item creation and revision cycle, giving authors more time to refine their items. These items, in turn, have been shown to be usable by their target user population--test taking students. Several items have been delivered in preliminary tests to student volunteers (i.e., usability studies); more than 200 such volunteers have taken a test on FRADS to date. In questionnaires administered after testing, 75% of the students reported that they found the computer test-taking environment easy to use; 62% thought that they would perform equally well on a computer-delivered test or paper-and-pencil test. 4.
CONCLUSIONS
To summarize, FRADS helps nonprogrammers to prototype by providing: • a user interface, similar to that of commercial graphics applications, that capitalizes on users' existing expertise. • a framework for creating restricted palette-canvas graphics editors (i.e., test items). Authors specify the tools available on the palette, the objects that appear in the canvas, and which tools may affect which objects. • a combination of "allowed actions" and special-purpose objects (e.g., masks, boundaries) through which authors can control how particular tools will function for students. Thus, much of the "end-user programming" performed by authors is done via creating and importing graphics as part of an item stimulus. How far can direct manipulation and declarative, graphical specification of applications take us? At what point is it be necessary to rely on some type of programming formalism? Future work will address these questions through (a) extending the functionality of FRADS through, for example, introducing greater flexibility in authors' control of tools and (b) applying the widgets-tools-actions framework to the construction of end-user prototyping systems for other application domains. REFERENCES
1. Haeberli, P. E. (1988). ConMan: A visual programming language for interactive graphics. Computer Graphics, 22(4), 103-111. 2. Myers, B. A. (1989). User-interface tools: Introduction and survey. IEEE Software, 6(1), 15-23. 3. Nardi, B. A. (1993). A Small Matter of Programming. Cambridge: MIT Press. 4. Took, R. (1990). Surface interaction: A paradigm and model for separating application and interface (pp. 35-42). Human Factors in Computing Systems: CHI '90 Proceedings. New York: Addison-Wesley.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
User Interface Development Environment for End Users
59
•
CUIDE
Y. Tokuda ~, E. S. Lee b and N. ShiratorP ~Research Institute of Electrical Communication, Tohoku University, 2-1-1, Katahira, Aoba-ku, Senda.i, 980-77, Japan bDepartment of Infornla.tion Engineering, Faculty of Engineering, Sung-Kyun-Kwai~ Universit.y, 300. (-',hllncl~un-dong, Jangan-ku, Sllwon. Kyllnggi-do 440-746, Korea. To cope with individual requirements of user interface (UI) from various classes of users, it is desirable to have UI development support environment which allows end users to develop UIs without designer's helps. The support of conventional methods and tools, however, focuses on expert of UI development. In this paper, we propose a UI development environment for end users, named CUIDE (Case-based UI Development Environment) in order to support UI development by end users, who have never developed any UI. CUIDE utilizes the case base of design and the case base of parts to make up for their lack of knowledge and experience of UI development. 1. I N T R O D U C T I O N With the spread of computer systems, a great part of the entire system development tends to go for user interface (UI) development, because of strong requirements for userfriendly UI. The conventional methods to develop UI efficiently[I-3], focus on how they could support mainly the expert users. Under the present state of UI development, which requires advanced technical skills and experience, however it is difficult to cope with individual requirements of UI fi'om \ariolls classes of users, and as the result, tile methods have limit of developing UI efficiently. Therefore we need UI develol)ment environment so that even general users can design, modify and expand U I. in this paper, we propose a UI development environment for end user, named CUIDE (Case-based UI Development Environment) in order to support the whole process of UI development by ordinary users, who do not have complete knowledge a,nd experience for UI development. CUIDE includes the knowledge for the techniques of UI design a.s well as parts of UI as case bases, and provides the cases for them visually. 2. D E S I G N
CONCEPT
To allow end users develop a desirable UI satisfying their requirements, it is not enough that they are provided only with parts of UI. They generally have functional requirements about their images of UI based on their experience to use UIs, even if they have never developed UIs. To actually design a. desirable UI satisfying their functional requirements,
60 end user should have the following design knowledge" i)functional knowledge, ii)layout knowledge and iii)a.ttribute knowledge, i) is for determining the parts of UI satisfying the above functional requirements, ii) is for organizing and laying out the above selected UI objects, iii) is for redesigning the attributes of each UI part in detail. Thus, because there is a wide gap between end users' own knowledge and the design knowledge, it is difficult to embody their requirements and to construct UI without the help of UI designers who have the above design knowledge. We propose a. UI development environment for end users, Case-based UI Development Environment (CUIDE). To bridge the above gap, CUIDE makes use of the two design knowledge, stored in i) the case base of design and ii) the case base of parts, i) deals with the cases for the knowledge of design methods with UI parts, ii) deals with the cases for the knowledge of UI parts constructing UI. Since with these case bases, we can retrieve existing design examples similar to desirable ones based on the incomplete and fragmentary requirements, they are suitable for the utilization of the design knowledge of UI. By reusing these case bases effectively, CUIDE a.ims to acquire and formalize users' requirements user-friendly, and to present the UI images according to the user requirements. 3. C U I D E Based on above design concept, CUIDE has the following structure(Fig.i) : three modules, (1) Requirement Management Module (RMM), (2) Case Management Module (CMM) and (3) hnage Generation Module (IGM) and two case bases, (4) Case Base of Design (CBD) and (5) Case Base of Parts (CBP). We'll mention each functional structure of CUIDE in 3.1, and the case bases of design and parts in 3.2.
~;,.;.-~ ~i!~!:.-~:ii~:~i:!:i~:i:~!:i#i-:i~i:!i:i~!~i~:~!i:~i~!~!~i ~'.-.~i*?x .:;~
::.........- ...................... ::::::::::::::::::::::::::::::::::::: i:!:i~:!:i::-"" ::::::::::::::::::::::::::::::::::::::::::::::::::
~ ~i!iiii~i~~iiiiiiiiiiii!ii ~ -:~:i::~:i:?i:i:~:~:i:i-:i:!:?i:i:~:i:!:!:?~:!-:i:i:i:i:!:i:i:?i:: o
f~x~
.;..v...;;;;:....,.....
. . . . .
• ...........;;....:.:
~i'::i:i:!~:i:!:i:~:i:":?N~~i]===================== ....;;;..v.;;.......
v.......:.:.;....;;:.;.
r -*~N : ~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:-:-:~:~:~:~:~:~:~:~:~\~:~:~:~:~:~:~:~:~:~:~:~::::~:~:~:~:~~' :~:~:~:~:~:~*::":~:~" ) , 1 ~ ""~/~i iii::i::i::iR~~~ii::i| ii::iiiii::i::iii~i::iiiiii;::ii~ili::i!M::iil;| IAI ~~.~.~,::~.::;:: ~.~i~i..~i~:-.:.~/t~. ! ~i4::~::::i:~-" i.:,"!iii~....ii:i:'":!~"-":~".'.":~":'-"~:~",'.'." ::.i~:".'.."!~:~.::::!:~i~i~i!i:~:~"-:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::~~m~::~ :~i~!~:!:#:~ ~ ::::::::::: . ..'" :.','.:- ":'.'.'-'.','.'.'.'.:-" :'...',:: End user
,'~,.';?
i:i:!:i:i:i:i:i:i:i:i:" "::" !::':::-: :'?::" :::::::::::::::::::::
~i~..':..'~ii~i~iii~!~iiii!~ii~.*.i~~iiiiii~ iiii~"
:i:i:i:i:!:i:!:i:i:!:!:""':'" 9::!::'" :-::::: "::-::i:i:i:!:!:i:i:
iiiiiiiiiiiiii
~ ! i i~ ..i::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ~ ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ................ ...................... ~ ~......... ................ ............... .Fh" ........................... ~ ~ ..... ......................................... ~. u.~ •~i'.:~'~ ::::' . "~.~:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: "......... .~........ :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::. ~.~6~q~.~~ •••.•••••
~'°'.~" ~.~:t ~ ~ ~ ~
~.
~ ~~
~].~v.' ~..~!~i :.`..`~.~:::~' .`.~!~..~:::..'.~,`:~.~.<~:..' .`..~:.`.~:.`~:::~.' `~:~.!..::!:~,~-.' .:~i~;~i~.'!:!:!:~:~:!:~:!| i`~.`:.~::: ,~!.'
~*3.``.:..`.~J~~...%`;.`...~:~:~.~.~:.~.....;..~?..!.;...!..~......!.;.`..:!..~`...........k..i:g..&.~!.~!.`...`..~..`.`.`..A~!!!::...~.~:~.`.~:*:~.. ..~::.......,..:.~:..!-~
Figure 1. The architecture of CUIDE
61 3.1.
Functional
Modules
(1) RMM RMM acquires, analyzes and formalizes users' requirements of a desirable UI and consists of (i) Requirement ACquisition sub Module (RACM), (ii) Requirement ANalysis sub Module (RANM), (iii) Design State Management sub Module (DSMM) as shown in Fig.2. RACM acquires various end users' requirements such as functional requirements and modification requirements. It gives non-expert end users a easy way using methods such as choice out of menus and direct manil)ulation for the UI images, to embody their requirements. RANM analyzes the requirements acquired in RACM, and formalizes them as case retrieval information (for case retrieval), or case modification informa.tion (for case modification). DSMM manages the cases relating to the designing UI.
i R::~eM:::: ~::~i::::I{i::RANM::.:::::::::::::::::::::::::::::::::::::::: M M ::::::::::::::::::::::::::::::::::
(UI) ......................".
.............ili{~{i!i{i{!iii{i{iii~
Figure 2. The structure of RMM
(2) CMM CMM manages two case bases,the case base of design and the case base of parts, and consists of (i) Case Retrieval sub Mo(1Llle (CRM)and (ii)Case MOdification sub Module (CMOM) as shown in Fig.3. C,RM retrieves cases similar to users' requirements from CBD and CBP by case retrieval information, and restores the cases lllodified in CMOM. CMOM regenerates design examples of UI satisfying their requirements, 1)3, modification or expansion on the cases retrieved in CRM based on case modification information. (3) IGM IGM supplies end users with visual UI images and consists of (i) Ui Automatic Synthesis sub Module (UASM), (ii) Image Presentation sub Module (IPM) as shown in Fig.4. U ASM synthesizes UI objects from a combina.tion of cases to generate more complicated UI images. IPM generates the UI images based on the cases sent fi'om (',MM or synthesized in UASM, displays visual UI image correspondent with their requirements immediately, and provides user interface, on which they can simulate actual behavior with their operation freely.
62
~ ] (RMM) .
(IGM)
I (,,)
I
Figure 3. The structure of CMM
i iiii!iiiiiiiiii!iiiiiiiil I
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Figure 4. The structure of IGM
3.2. Case Bases (1) CBD CBD stores empirical knowledge to select and lay out components of UI based on user's requirements for construction of UI. We classify these requirements into functional ones and modification ones. Functional requirements are for function which wanted UI should have, and modification requirements are for modification to a particular design examples. So, cases of design include part of layout and pa.rt of modification. Part of layout consists of excellent examples designed by UI designers and preferable ones designed by each end user. This part includes a set of hmctional requirements, a set of UI components chosen based on the requirements and design information of layout among those components. Part of modification are design methods to modify a. particular design examples based on user's modification requirements. This part is represented as a pair of the requirements and the methods of modification for a particular layout. (2) CBP CBP stores knowledge to select, a l)articular parts of UI based on users' functional requirements and to modify detailed a,t,tril)lltes of conll)onents of UI based on their detailed
63 requirements. Cases of parts consist of knowledge about general attributes of parts of UI (Cases of image parts) and instances customized according to particular detailed requirements (Cases of function parts). Cases of image parts deal with the case for the attribute knowledge, which is for describing instances of UI parts and for redesigning the a.ttributes of each UI part in detail. Cases of function parts deals with the case for the functional knowledge, which is for determining the parts of UI satisfying their functional requirements.
4. I M P L E M E N T A T I O N
AND
EVALUATION
4.1. I m p l e m e n t a t i o n We have implemented a prot.otype of C[JIDE in C language, with a set of Motif[4] widgets a.s basic parts of UI a.nd ITIL (l!ser Interface Language) of Motif a.s UI design language. CRM and two case bases of ('I;II)E, (-:t~1) and (!BP, have been constructed by our developed Case Sea r011 Sul)l)Orl El l\'i roll ll l¢'lll . ('S.q.'IE[5]. w]lic]-i ('am~Slll)l)Ort t.o construct case bases and to store and retrie\e dcsira.ble ca.ses ea.si]\'.
4.2. E v a l u a t i o n In order to verify the effects of C,UIDE, we have made evaluation experiments for end users, who are not experts in UI design, but are experts in application domain such as communication protocol. In these experiments, subjects used SERA[6](Requirement Acquisition and Ca,ses Based Specification Environment), which have the graphical user interface with Motif widgets, before the construction of their own UI. Then, they constructed a part of SERA's UI(Fig.5) eitl~er b., means of C[!IDE or a. set of Motifs widget. As a result, we have observed that subjects, who ha\;e been provided only with UI parts such as Motif widgets, can not const.ruct the UI close to their inla.ges in their mind, because of the lack of knowledge and experience of UI development. On the other hand, we confirm the following usefulness of CUIDE.
(i) By providing the process which can embody users' requirements and is supported by two kinds of case bases, even end users can represent their requirements, generate the image of UI correspon(lillg to t.h~,ir requirements, and modify' and expand the design. (it) By presenting visual UI images according to their requirements, they can confirm their requirements, and promote enlbodinlent of their new requirements such a.s modification ones. (iii) With providing the design examples of UI by reusing existing cases of design, end users can not only develop UI satisfying their present requirements efficiently, but also create a desirable UI.
64
::i::i::•::i::i::}::!:: ::::::::::!::::::}::i::i::i::i::i!i:: ::::! ::i!i•iii::... i i i i!ii i i i i i!ii i i i i i!ii i i i i!ii i i !i ! i i i!!`:i•:i!ii!ii`:i! i!!.!• :::::::::~:.-~/:~: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ::
i::i!i! : :: :::
!! ::::::::iii::~$! :::::: :: i:i:i:i:~i:i:i
i~NN~i~N~!::ii~ii :~:~:~.~.~.!i~iiii;iii~~~~iiiiii;~iiiii!iii!ii;i;i1iii~iiiiiiiii:i:i ~i!i~!i!~!i!i~!i~!~i~i~!i!~i!i!i!ii~:i~:i~i~;~;:~;~:~;~;~;~;~;~;~i:~;~;~;~;~;~:~;~:~:~:~;~;~:~;~:~;:/~:/~i~;~;~i:~i~i~i:~i ~:::]
liiiiii~!iii!iiiiiiii~!ii!iii!ii:iiiil ,ili,~i;illi:!'i:i!ili!i!i!i!iii!li i!i i ii!:iiii!!!iiii!!ii!i i!!:ii!iiiii li!i~i,] ]iii!iiiiiiiiiill',iiii',ii :: : iiiii i iii:: i',!:ii: ', :i ::i::: ', :ii i : :~ Figure 5. A Example of (:onst, ructed l!l
5. C O N C L U S I O N In this paper, to support UI development by end users efficiently, we proposed the Case-based UI Development Environment, CUIDE. We also inlplemented the prototype of CUIDE and evaluated the usefulness qualitatively. Future work in our research is necessary towards more user-friendly UI devdopment support environment, which has more flexible mechanism for acquisition of various requirements from users.
REFERENCES
1. P. Sukaviriya, J. D. Foley and T. Grifl'ith, A Second Generation User Interface Design Environment : The Model and The R untime Architecture, In Proceedings of the ACM INTERCHI'93, pp.375-382 (1993). 2. P. Szekely, P. Luo and R. Neches, BEYOND INTERFACE BUILDERS : MODELBASED INTERFACE TOOLS, In Proceedings of the ACM INTERC, HI'93, pp383-390 (1993). 3. H. Eriksson, A. R. Puerta and M. A. Musen, Generation of Knowledge-acquisition tools fl'om domain ontologies, Int.J.Human-Conlputer Studies, Vol.41, No.a, pp425-
453 (1994) Open Software Foundation, OSF/Motif Style Guide Release 1.2, Englewood Cluffs, New Jersey : Prentice Hall, Inc., 1993 5. T. Kato, Y.Tokuda, E. S. Lee and N. Shira.tori, A Consideration on Construction of Case Base Management System for Reusing UI Parts, In Proceedings '95, Nat. Conf. of IPSJ (to appear). 6. E.S. Lee, U. Yamamoto and N. Shiratori, Requirement Acquisition and Cases Based Specification Environment,SERA, In Proceedings of The 9th International Conference on Informa.tion Networking(I('OIN-9), pp279-284 (1994).
4.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
Supporting
Computer
Users through
65
Dialogue
Akira Ito, Tadahiko Kumamoto and Tsuyoshi Ebina ~ ~Kansai Advanced Research Center, Communications Research Laboratory, 588-2, Iwaoka, Nishiku, Kobe, 651-24, Japan Abstract The authors have been developing a dialogue-based user support system which assists a novice computer user in performing computer tasks. If a user gets into some trouble while using a computer, the system can provide help through natural language dialogue. The system understands a user's utterance and generates an appropriate response. The e-mail handling program XMH is used as target software. The system records each of XMH operations by the user, and maintains XMH information. To answer user questions, the system consults the current XMH state, and operational history of the user. 1. I N T R O D U C T I O N The best way to learn how to use an application software is simply to begin using it by yourself. Many of the visual interface softwares are advertized with the claim that it can be used without a manual. However, once one is trapped in an error or misconception, it can be hard to escape from it by oneself. This is especially true for a visual interface software, which gives little hints about possible misconceptions. In such circumstances, the best thing to do is to ask someone familiar with the software. Such an advisor can easily point out a solution with a quick glance of your display. Unfortunately, we do not always have a good advisor at hand. Hence we decided to develop a user support system which helps computer users in the same way as a nearby advisor does. A user ordinarily uses application software by means of a keyboard a n d / o r mouse. In our system, when the user experiences difficulty while using a computer, he can ask the system for help using natural language. The system understands a user's utterance and generates an appropriate response. The target software for user support is XMH (Xwindow Based Message Handling System)[1], a visual interface software to read, write, send, store E-mails. The user support system requirements are as follows: 1. To answer a user's question, the system should access the user's operational history and the current XMH state. 2. The system shuold be evaluated by how well it can help the user complete his tasks, not how well the system understands the user's intention. 3. The topic of the user's question can be restricted to the task domain, but the surface form of the user's question should not be restricted to predefined sentence patterns.
55 To design the system, we conducted a user support experiment in which a human expert advised users instead of the system. Users asked their questions either through keyboard or verbally. Answers were presented to the user in a text form. From these experiments the dialogue database was constructed. The database contained not only the user's and advisor's utterances, but also all the user's operations. These dialogue data was then analyzed, and used to develop the system. A prototype was next developed, and then evaluated in a user support experiment. Since the famous UNIX Consultant by Wilensky [2], there has been considerable research on user support or consultant systems. Some were concerned with user modeling [3], others with intention recognition [4] or explanation generation [5]. Our interest was to help the user on the job, and to develop a method that makes effective use of the user's operational history. In section 2 we describe an architecture for a dialogue-based user support system. In section 3 we explain how the system works producing some example dialogues. Section 4 is a summary and discussions. 2. T H E A R C H I T E C T U R E
OF THE XMH USER SUPPORT
SYSTEM
The architecture of the XMH user support system is shown in Fig. 1. The system consists of a dialogue world model, an XMH simulator, a natural language processing module, and a dialogue management module. Note that the XMH itself is external to our user support system. , . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
User' /
~ / ~
:~
//
/
i
i
Naturallanguage processingmodule
~/I 1 infOEVent~~.j XMH XMH User P~Ipr°gramI '~I sim ulator operation(
J Speech
User
Response sentence
synthesizer XMHusersupportsystem
, ~
Dialogueworldmodel ; Information
Dialogue managemant module
An architechture of the XMH user support system 2.1. D i a l o g u e w o r l d m o d e l The dialogue world model is a system's view of the world. It consists of XMH knowledge and a model of the user. XMH knowledge includes knowledge of objects (XMH concepts)
67 and actions (operations possible in XMH). The model of the user is a representation of a user's goals and task plans. These are inferred from the user's operational history and verbal questions. A frame-like representation with a class hierarchy is used to represent the dialogue world model. 2.2. X M H s i m u l a t o r The XMH simulator records every user's operations. These include mouse movements, pushing the mouse button, and keyboard input. It then puts the primitive event data together to construct user actions. From the obtained user action, it simulates XMH behavior, and maintains information on current XMH state. This information is used to update the dialogue world model. The simulator is also used to check the validity of advice plans generated by the dialogue management module. 2.3. N a t u r a l l a n g u a g e p r o c e s s i n g m o d u l e The natural language processing module processes the input sentences, and transforms them into communicative intention (CI) descriptions. A CI description is a dyad of a CItype and a communicative content. It expresses what the user wanted to communicate to the system. A CI-type specifies a sentence type or the user's attitude to the communicative content. The list of CI-types used in our system is given in Table 1. Sample sentences have also been included in the Table. This list was determined by the user support experiment stated aboev. It prescribes the range of user intentions the system is supposed to understand. Table 1. CI-types and sample sentences. CI-types
Sample Sentences
ask-wh:OK ask-wh ask-if:OK ask-if:EQ ask-if:NO ask-if ask-how ask-about have-belief have-goal
/ ~ ~ v ~ t . : b ~,~A~-~'@;6~ (What should I write here?) ~~a~ ~ L tz (I forgot the address.) ~-]" ~ @ t ~ ' ~ A ~ ' ~ ' - C K (Is it OK to push the "Close Window?") •y 4 : - "Yt~ C. ~ ~ ' ~ / . ' ~ ~ (Is this enough for the message?) :_ fL~ ~ t~: < ~ ~ I~EI t~: A~"~3"-75~(Do I have to push this?) :_ f L ' e ~ ' C " $ "T ~ Z~'C'-¢¢~" (Have I moved it successfully?) ~3'-03t~ ~"'5 T ~ ~'~T/)~ (How do Imove it?) 7, ~, t~ - )t, -~ -~/~-~-T 7~ (What is "Scroll" ?) ~ ' ~ " ~ "T t,~,&?c_~t,~ A~'~"C~:t k" (It does not seem to have been moved.)
d-start d-end
(I want to move the cursor to the position after "Roy.") ~ ~ ~-/. (Excuse me.) ~b~ ~) Z L tc (I got it.)
The communicative content is a propositional part of the sentence and is expressed in the object world model framework. The supposition is that the user asks only about the XMH task domain. This insures that the communicative content is expressible by XMH objects and actions. The natural language processing module determines the CI-type and communicative content separately. The former is determined from feature words extracted from the
68 utterance sentence. The rules for CI-type recognition were constructed from the database obtained in the experiment [7]. Voice input is also possible. A few thousands sentences composed from about a hundred Japanese words were registered in the speech recognition module. Due to a low recognition rate, however, this is used only for demonstration. 2.4. D i a l o g u e m a n a g e m e n t m o d u l e
The response strategy consists of a set of heuristic rules human experts are supposed to employ when they help novice users [6]. The expert responses inthe dialogue database are analyzed, and the response generation rules developed. A rule is prepared for each of the CI-types of user questions. An example rule for the ask-how type is given in Table 2. Table 2
Response generation rules for the CI-type "ask-how"
1. If the communicative content (action) is about the action the user executed successfully, explain it. 2. If the communicative content (action) is about the action the user failed to execute, provide the reason and way to execute it. 3. Otherwise, provide a execution plan step by setp. Dialogue is managed using a response process flow (templates) prepared to correspond to a response strategy. These process flows were developed from the analysis of possible dialogue patterns in the dialogue database. The system has a dialogue stack and can respond to an interruptive utterance from the user. Generated responses are presented to the user in both voice and text. 3. E X A M P L E
DIALOGUES
We will explain how our system works by means of some example dialogues between the user and our system. The system works in Japanese, so the English translation below is given just for explanation. In the following, Un stands for a user utterance/operation, S n stands for a system generated response, and [... ] stands for a user action. UI" ~ ,~ 4z- -y"3 ~:~T~: l~ ~" -) Lfc 6 ~ , ~ , A ~ - ~ , ? ( How can I move message 3? ) SI: ~)J~)~: ~ "z q z - ' J 3 ~ : ~ L ' C - F ~ ' . ( First, select message 3. ) U2: [ User selects message 3. ] ( Next, select the destination folder. ) U3: [ User selects the "myfolder" folder. ] S3: ~ : , F~ ,~4z--jJ ~ = : z - ¢ ) F~ , u q z - - ~ c ) ~ J '~~b~Y~,. ( Next, execute "Move Message," in "Message" Menu. ) U4: [ User executes "Move Message." ] ( Lastly, execute "Commit Changes," in "Table of Contents" Menu. ) U5: [User executes "Commit Changes".]
69 A user initiats a dialogue by asking how to move a message. The user's utterance U1 is transformed into a CI-description of: (ask-how #(MoveMessage 042)) where #(MoveMessage 042) is a communicative content, and ask-how is a CI-type. The dialogue management module starts the process flow for the CI-type "ask-how," and constructs a plan to move the message. Then the necessary operations are provided step by step, starting from response S1. In the above dialogue, the user correctly selected message 3 (U2). However, if the user does not know how to do this, he may ask again. This starts an interruptive dialogue. U21" ~ _ o ~ ~°5 ~ ~ ~ 7 ~ / , ~ - ~ b , 9 ( Well, how can I select it? ) $21: , ~ . x ~ - - ~ 3 © ~ © J : ~ : ~ ~ - ' j ) p ~ ~ , -~e]~,~-~,:/~L..-C-F~,. ( Put the mouse cursor on the message 3 line, and then push the mouse button. ) If the user succeeds in selecting the message, the interruptive dialogue is closed and the system restarts the suspended dialogue from $3. The operational window of our system is given in Fig. 2.
(~
O(~ 7'~-f)r~¢~~ /
~
z T~,
, c ~ - - ~ ~a).~ v ~ - - . ~ ,
06/00 ,~
= ,..m,,,....
I~- 9 ~;~W I~ .o. z ~. ~¢~R~ lsaJ04:~ F0t:_~,
lndlxm : 3 Tue, 0 Jun 93 10:23: IS J S I ~ k l r u l t o ~<¢t) ;oge-Id: mark ~305060923./~ 12505e~Jrosokt. korc. cr 1. go. JW"
Read Nessa(je Ito.3
Fig.2 The operation window of the XMH user support system 4. S U M M A R Y
AND DISCUSSIONS
We have developed an on the job user support system, taking up XMH as a target software. User's operational history is used effectively in generating a advice. The system responds to both the user's verbal questions and user's XMH operations. The response strategy adopted resembles that of a human expert. This system, however, has not yet reached a practical usage level.
70 Through a series of user support experiments, we investigated the minimum system requirements for practical usage. A number of conclusions are reached: First, the choice of input method, i.e., either keyboard or voice. Users often prefer to solve a problem by themselves rather than to ask the consultant through the keyboard. This is especially true for Japanese users who are not accustomed to use Kana-kanji conversions from keyboard. An average of only 3.5 questions were asked through the keyboard compared to an average of 26.3 questions by voice. On the other hand, current speech recognition technique is not sophisticated enough to permit spontaneous utterances from the user. A Second point concerns multimodal dialogue. Users often want to use voice utterances and pointing actions cooperatively. Many such actions were observed in the user support experiments.. Human experts are clever enough to identify the referent of a demonstrative such as ~-~L (this), but current natural language understanding systems have difficulty in such identification. If a user's pointing action can be recognized directly (such as through a touch panel), this would be a great help for the natural language processing module. We are currently trying to incorporate such a mechanism for handling a multi-modal dialogue. Third, the cooperativeness of the user is important. Sometimes the user's utterances are ambiguous and no appropriate responses are possible. This is mainly because the system cannot infer the hidden intention of the user, which most users wouldn't explain to the system. Of course, AI technology to infer the hidden user's intention should be developed. In the interim, the user's cooperation is essential for our system to be usable. If the user gives sufficient hints, the system can be more helpful. How can we make users feel like speaking to a machine? This is an important factor for the man-machine interface of tomorrow. REFERENCES 1. J. Peek, Mh and xmh - E-mail for users and programmers, O'Reilly & Associates, Inc., 1991. 2. R. Wilensky, Y. Arens and D. Chin, Talking to UNIX in English: An overview of UC, CACM, Vol. 127, No. 6, pp. 574-592. 3. D.N. Chin, KNOME: modeling what the user knows in UC, User models in dialogue systems, pp. 74-107, Springer-Verlag, 1988. 4. J. Allen, Recognizing intentions from natural language utterances, Computational Models of Discourse, M. Brady and R. C. Berwick eds., The MIT Press, 1983. 5. J . D . Moore, Participating in explanatory dialogues, The MIT Press, 1995. 6. A. Ito, T. Ebina and T. Kumamoto, Dialogue-Based user supprot for a visual interface software, Proc. PRICAI'92, pp. 190-196, 1992. 7. T. Kumamoto, A. Ito and T. Ebina, An analysis of Japanese sentences in spoken dialogue and its application to communicative intention recognition, Proc. ICSLP'94, pp. 943-946, Yokohama, 1994. 8. T. Kumamoto, and A. Ito, An analysis of user-consultant dialogue and its application to dialogue control in a dialogue-based consultant system, Trans. IEICE, vol. J77-DII, no. 8, pp. 1492-1501, 1994, (in Japanese).
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
71
A D e n o t a t i o n a l A p p r o a c h for F o r m a l S p e c i f i c a t i o n of Human-Computer Dialogue K. Matsubayashi, Y. Tsujino and N. Tokura Department of Information and Computer Sciences, Faculty of Engineering Science, Osaka University, 1-3 Machikaneyama, Toyonaka, Osaka 560 Japan E-mail:
[email protected] This paper presents a formal approach to specify the human-computer interaction, especially the information exchanged between a computer and a user, using a notion derived from Denotational Semantics [8], which is originally developed for the formal description of the semantics of programming languages. As a first step, we applied the specification technique to existing application programs, and successfully obtained brief and concise specification of the meaning of user's input symbols to the system.
1. I N T R O D U C T I O N
In the field of HCI, it has been a growing concern to give a formal description of various aspects of interaction between users and computers. By applying the formal approaches to the interaction, it is expected to be helpful for • evaluation of usability and consistency etc. for existing computer systems, • integration into developing system like UIMS to support iterative developing processes, and • precise model for performance prediction and evaluation. Various specification techniques have been proposed to explain the structure of task and user knowledge (e.g. Task-Action Grammar [7]), the user performance (e.g. GOMS model [i]; Cognitive-Complexity Theory [2]), and the behaviour of user interfaces (e.g. CLG [3]). However, many of these models are defined to explain only specific aspects of interaction, which are originally intended to be evaluated and measured by the developers of the models. Instead of these practical approaches to formal analysis, there can be an alternative approach, which is intended to give a fundamental model of the interaction in a formal manner. Our primary concern is to develop a fundamental model of interaction, and to use the model as a theoretical basis for giving interface specifications, prototyping, and any other developing/evaluating environments. Our denotational approach for the specification of interaction is the starting point for our research.
72 2. M O D E L O F H U M A N - C O M P U T E R D I A L O G U E This section presents the formal model of dialogue, as well as the way to give specification of the dialogue.
2.1. Definition of dialogue and interaction There can be a broader model of the interaction among several people and systems, or the interaction with some "agents" in the system. But here we deal with the basic and simple style of the interaction, between one user and one computer system. Given a user and a system, ~), which is a set of dialogue D, is given by
(D={/~oa,/~ a~./~2...a,/~, I a~e z*,/~e/-*, zn/"=~}
(1)
where 2: is a set of user's input symbols to the system, and F a set of system's output symbols to the user. Figure 1 shows the dialogue between a user and a system. Both the user and the system have internal entities, P_~and Es respectively, and they are exchanging symbols each other, changing the p a r t n e r ' s internal entities. f
Interaction
..,~ • System
User
( ':tntertina:~ i!i....~
Dialogue |||||||||
a
(..taskmodels~
( otherenvironments ~
Figure 1. Interaction between user and system. Then, the interaction between the user and the system is defined as a triple of the user, the system and the dialogue between them. Note that in real circumstances some other factors e.g. problem domains or task structures could be involved in the interaction, but here we don't deal with them in the current specification.
2.2. Specifying meanings of the dialogue In our denotational approach, dialogue is treated as a sequence of symbols. S t r u c t u r e of the dialogue sequence are defined using abstract syntax, and the meaning of the dialogue is given for each symbol or sequence. Let a b e a user's input symbol to the system, and fl be a system's correspondent output symbol to the user. The meaning of user's input symbol a is given as a mapping from Es to Es and b. Similarly, let b' be a system's output symbol to the
73 user, and a' be a user's correspondent i n p u t symbol to the user. The meaning of be given as a m a p p i n g from Ewto E v and a'. So the specification of the whole phases of the i n t e r a c t i o n can be given in t e r m s of E U, Es, 2:, F, dialogue structures and the mappings which define m e a n i n g of the dialogue.
system's output symbol fl" can
3. E X A M P L E : A P P L I C A T I O N O F T H E S P E C I F I C A T I O N This section p r e s e n t s a practical example of the denotational specification, applying the t e c h n i q u e to the existing p r o g r a m s . The purpose is to show the expressive ability of this approach to specify the dialogue. Macintosh Finder, t a k e n up here, is a GUI-based file m a n i p u l a t i n g program as well as a kind of login shell, s y s t e m configuration utilities. We specified the s t r u c t u r e of the dialogue between F i n d e r and the user, and the m e a n i n g of the user's input symbols to Finder. Note t h a t we are currently dealing only with the functions relating file manipulating, which involves typical type of operations e.g. handling of m e n u s and windows, dragging icons and even inputs from keyboard. Also note t h a t user's internal entities and the m e a n i n g of system's output symbols to the user are omitted here in the specification this time. 3.1.
Domains
The domains needed for the specification of the meaning of user's input symbols are 2:,/~ and Es. Here we illustrate Es as an example. Es denotes system's internal entities which are needed for the specification of the m e a n i n g of user's input symbols. Here Es is represented as a product domain. Each e l e m e n t d o m a i n is equally defined by more basic domains. Following specification is an extract from the definition of Es. Es = D e s k t o p x M o u s e x C l i p b o a r d x F i n d S t r Desktop Size Disk
• Size x Disk* x WinList x W i n D B x Selection x M e n u b a r • ( Nx N ) /* display size */ • N a m e x Content x Content x C o n t e n t
Content
/* disk name, disk contents, desktop contents and trach contents */ • ( Folder + File + Alias )* /* list of the content of disk (or folder) */
. . . .
--->Window )
WinList Window
• ( N
Rect
• (NxN)x(NxN)
/* window list (which returns the 'Nth' window)*/
• Rect x W C o n t e n t x Path
/* window size, windowcontent and associated path */
The domain Z and F are defined similarly. Note t h a t we prepared two abstraction level of Z-- command level and mouse-key level. C o m m a n d level deals with user's inputs as commands like "copy selected item" or "create new folder under c u r r e n t folder". Mouse-key level deals with inputs as lower inputs like mouse movements or key inputs. F is, in our c u r r e n t specification, the domain which is a set of
74 possible output representations, that is an entire screen of the display.
3.2. Syntax Specification The next step is to specify the structure of the dialogue in a form of abstract syntax. Syntax of the dialogue are given for both command level and mouse-key level. Following specification is the syntax for command level dialogue. Nonterminal symbols are bold-faced, and system's output symbols are underlined. The symbol e denotes 'no input' from the user. Dialoguecor. ::= Init Dco,. Dcor. ::= ~1 ( NewFolder
ResDI OpenSelection ResD I CloseWindow Reso DuplicateSelection ResD I FindFile Resp I FindAgain ResD SelectAII ResDI EmptyTrash Resp I SelectFile ResD RenameFile ResD I MoveSelection <MoveDialog> ResD CopySelection ResD MoveWindow Respl ResizeWindow ResD ) D~or.
Symbols with angle brackets denote involved dialogue, which work as a part of input symbol from the user. For example, the syntax of can be given as"
":= FindDialoQ FindStr ( O K I C a n c e l ) DismissDialoa FindStr ::= ~1 ( Char ResD I CutStr Resp I CopyStr ResD I PasteStr Reso I SelectStr ResD I MoveWindow ResD ) FindStr
The syntax for mouse-key level dialogue are given as follows • DialogUernk ::= Init Dr"k Dr.. ::= ~1 ( MDrag I Click ResD I DblClick ResD I Char ResD I Bs ResD I Esc ResD I UP Resp I Down ResDI Left ResD I Right Resp) Dr.k MDrag ::= MC Drag1 Drag1 ::= PressMB ResD Drag2 Drag2 ::= MC ReleaseMB ResD MC ::= E ( MouseUp Resp I MouseDown ResD I MouseLeft ResDI MouseRight Resp ) MC
3.3. Semantic Specification The final step is to define semantic functions, which are the mapping from input symbols to the meanings, i.e. to specify the meaning of the dialogue with the functions. Following functions are used for specifying the meaning of command level input symbols from the user to Finder. ~cute gives the meaning of user's input in a view of change of system's internal entities; dt~h~ in a view of change of system's output. ~ , u t e • Com ~ ( ( Desktop ~ Mouse -~ Clip ~ FindStr ) --> ( Desktop x Mouse x Clip x FindStr ) ) ~lLqp~ • Corn ~ ( ( Desktop ~ Mouse -~ Clip ~ FindStr ~ Disp ) ~ Disp )
Next specification is a part of the description of the function exe~te, to a user's command Newfolder . The arguments dr, m , c and f are system's current internal entities.
75 execute [ NewFolder ] d t m c f = let dt" = create-new-folder d t in execute [ RenameSelection ] d t ' m c f
Similarly, functions action and edao are defined for specifying m e a n i n g for mousekey level input symbols from the user. action
• MK ---> ( ( Desktop--> Mouse ---> Clip--> FindStr ) --> ( Desktop x Mouse x Clip x FindStr ) • MK ~ ( ( Desktop--> Mouse --> Clip--> FindStr --> Disp ) ---> Disp )
Following is an extract from the specification concerning mouse dragging. This specification shows t h a t the r e s u l t of m e n u selection are h a n d e d over c o m m a n d level, and the result of dragging from inside a certain window. action [ MC MDrag ] d t m c f = let (dt', m', c',f') = aaion [ MC ]dt m c f in oxtion [ MDrag ] dt' m' c ' f action [ PressMB Reso M D r a g l ]dt m c f = case whereMC dt m of onMenuBar : let (dt] m', corn) - s ¢ ~ ~ [ MDragl ] dt m in execute com dt' m' c f
onWindow : let n = whichWindow dt m in let i whichltem dt m in if i= e then let dt'= front-window (dt, n) in .s~et'da~ [ MDragl ]dt m e l s e let dt'= select-file (dt', i) in .urvc,...~etlon [ MDragl ]dt m onDesktop : =
end case
Similarly, the detailed description for the rest of the dialogue are also given in a formal m a n n e r . The whole specification which we have applied to Macintosh F i n d e r are shown in [6].
4. C O N C L U S I O N We have introduced the a l t e r n a t i v e approach to specify the h u m a n - c o m p u t e r dialogue in a formal m a n n e r . With the denotational a p p r o a c h , we can give the d e t a i l e d a n d precise s p e c i f i c a t i o n s u n d e r given d o m a i n s . L i s t e d are t h e charasteristic features and possibilities for the future steps. •
•
Our specification t e c h n i q u e can be shown applicable for c h a r a c t e r - b a s e d text editor [4] and G U I - b a s e d p r o g r a m s [51 [6] which h a s c o m p a r a t i v e l y detailed interaction. This shows the expressive ability of this approach to specify the dialogue. This model can deal with several abstraction levels of input symbols, output symbols and i n t e r n a l e n t i t i e s in a consistent m a n n e r . So modellers can
76 choose any abstraction levels to be discribed which are appropriate for their own purposes. And extending the model so that it can handle task level in a same way should come to the next in our research. • "Meaning" of dialogue sequences can be specified, so the practical application of the model to the interface specifications would be a straightforward extension of this research. • If the user's internal entities are to be given, the meaning of system's output symbols to the user could be defined symmetrically as a mapping from user's internal entities to user's input symbols to the system, and it would bring some possibility for the simulation of the dialogue. However, there is a problem whether an appropriate user model could be given to specify user's knowledge and activities. • Besides a user and a system, other domains like tasks or environments could be introduced to specify the meaning of higher level dialogue in which some other domains are taken into consideration. Finder is a t r a d e m a r k of Apple Computer Inc.; Macintosh are registered trademark of Apple Computer Inc.
REFERENCES
[1] S.K. Card, T.P. Moran and A. Newell : The Psychology of Human-Computer Interaction. Laurence Erlbaum Associates, Hillsdale, NJ. [2] D. Kieras and P.G. Polson : An Approach to the Formal Analysis of User Complexity. International Journal of Man-Machine Studies, 22, pp.365-94 (1986). [3] T.P. Moran : The Command Language Grammer : A Representation for the User Interface of Interactive Computer Systems. International Journal of ManMachine Studies, 15, pp.3-50 [4] K. Matsubayashi, Y. Tsujino and N. Tokura : A Formal Approach To Hierarchical Specification of Human-Computer Dialogue Using Denotational Semantics (in Japanese). Trans. IPS Japan, 94-HI-53-13 (1994), pp.93-100. [5] K. Matsubayashi, Y.Tsujino and N. Tokura • An Application of the Denotational Specification of Human-Computer Dialogue to GUI-based System (in Japanese). Trans. IPS Japan, 95-HI-58-6 (1995), pp.37-44. [6] K. Matsubayashi, Y. Tsujino and N. Tokura • A Denotational Specification of Human-Computer Interaction and Its Application to Macintosh Finder TM (95ICS-3). Department of Information and Computer Sciences, Osaka University, Toyonaka, Osaka, Japan (1995). [7] S.J. Payne and T.R.G. Green : Task-Action Grammers - A Model of the Mental Representation of Task Languages. Human-Computer Interaction, Vol.2, pp. 93-133 (1986). [8] D. Scott and C. Strachey : Towards a Mathematical Semantics for Computer languages. Proceedings of Symposium on Computers and Automata. Polytechnic Institute of Brooklyn Press, New York, U.S.A. pp.19-46 (1971).
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
77
A Case-Based Method To Support Creative Design Incorporating Intention Recognition Takayuki Yamaoka and Shogo Nishida~ a Central Research Lab., Mitsubishi Electric Corporation, 8-1-1, Tsukaguchi-honmachi, Amagasaki, Hyogo, 661, JAPAN. In this paper, we describe a method and a system to support a creative design process. This method is characterized by its ability to infer the designer's intention based on case-based reasoning(CBR) methodology, so that the system can provide more useful and cooperative information on the current design. 1. I N T R O D U C T I O N In order for computer systems to cooperatively support design processes, it is required for systems to understand intentions of the designer and use them to improve the design process[I]. Conventional CAD systems at present, such as graphical drawing tools with simulators or with expert systems, can only support lower level operations of the designer or tend to aim at fully automated systems, but cannot sufficiently support creative aspects of design processes. If a system can grasp intention and share it with the designer, it becomes possible to provide more useful and cooperative information. In spite of the importance of this role, there has been little research on design intention and its use for design support. In this paper, we will provide a method and a system to support a creative design process, incorporating designer's intentions and using CBR methodology. 2. I N T E N T I O N A L 2.1. Design
DESIGN AND SUPPORT
METHOD
intention and means
In many creative design processes, the designer may intentionally generate a result which is satisfactory for his/her preference. In a design task such as furniture layout in a room, for instance, if the room is wide enough to put all given furniture in, the designer may decide their layout along ideas on how or what s/he wants it to be, such as beautiful and/or compact for future extension, and so on. We call these ideas, such as to be beautiful or to be compact in this example, "design intention". Also we call the resulting state to realize design intention, "means". In the furniture example, a means would be a real layout in the room. Normally, an intention causes a designer to choose particular means. We call the structure of this causal relationship "intention structure". Notice that the design intention is not always the design goal or the functional end the final product will perform, but is a kind of mental state of the designer in the design
78 process. If alternatives to achieve the goal exist, designers would make decisions to choose the preferred means to satisfy the intention, Grasping the design intention and sharing it between the designer and the system make it possible for the system to provide more useful and cooperative information, especially in the conceptual design phase. 2.2. C a s e - b a s e d s u p p o r t for i n t e n t i o n a l d e s i g n Figure 1 depicts an image of human-computer cooperative design support. In this image, a designer and a system look at a working field. The designer can perform 3 actions to reflect his/her design intention on the design: 1. operating on objects to change the current design status as s/he like, 2. asking the system to see information associated with the current design, and 3. editing the shared intention structure of the ongoing design. On the other hand, to support the designer, the system performs the following actions: 1. recognizing the designer's intention, 2. showing information which is useful to satisfy the intention, and 3. storing the intention structure and the result to the database for future support. By iteration of interactions using these actions, the designer and the system can share the design intention step by step, so that they can collaborate and make the entire design process more effective.
1. H: Input a problem, 2(a). H: Ask to retrieve means to satisfy the design intention, 2(b). H: Operate on objects to change the status, S: Infer the design intention of the operations or the states. 3. S: Display the intention structure (the result of 2.). 4. H: Take preferred means into the current status, if any. 5. H: Edit the intention structure to reflect the design intention. 6. H: If the result is not sufficient, then goto step 2 and do recursively. 7. S: Store the intention structure in the case-base. Note: H denotes Human designer, and S does System. Figure 1. An Image of Cooperative Design Support
Figure 2. Typical Process of Cooperative Design Support
79 To incorporate capabilities mentioned above with the system, the following functions are a minimum requirement: 1. to presume and confirm the design intention from the ongoing means and/or the current status done by the designer, and 2. to retrieve and display concrete means which are satisfactory for the design intention. However, since design intentions may vary among designers and situations, the most problematic matter is the difficulty to describe fixed rules or knowledge sources about relationships between intentions and means. In order to realize a system with these functions, we employ a framework and an architecture based on CBR methodology. The CBR approach is good for such tasks, mainly because CBR method makes it possible: 1. to avoid preparing fixed rules and knowledge sources in advance, 2. to output flexible and varied information through the modification and adaptation processes, and 3. to extend knowledge sources step by step. Furthermore, since computer systems cannot make creative decisions at present, it is reasonable for computer systems to work as augmented memories of human designers, and to provide information useful for the users to make good decisions[2]. In this framework, a human designer and a system can interactively proceed with a design process as in Figure 2; The input problem includes the goal, objects, constraints, and so on. Step 2(a) may be performed if the designer explicitly has a particular design intention, otherwise 2(b) is performed. In Step 5, the designer can modify any symbol of mental statement in the intention structure to reflect his/her design intention. Each step corresponds to a particular phase in a typical CBR process, 2 to case retrieval, 4 and 5 to adaptation, and 7 to case storage, respectively. A remarkable characteristic of this method is the twofold application of the CBR method for complete design and intention recognition. Details of the intention recognition will be described in the next section. 3. C A S E - B A S E D I N T E N T I O N
RECOGNITION
We represent an intention structure as a labeled graph as shown in Figure 3. It consists of four types of nodes, vocabulary, object, physical and mental, and labeled links between them. An object node consists of a set of attributes, where each attributes name is a link label and value is a vocabulary. A physical node consists of a set of objects(the predicate name and arguments), and generally stands for a physical state, such as "left" or "lower". A mental node consists of a set of any nodes, and generally stands for a mental state, such as "beautiful" or "compact". An intention structure describes a (partial) case realized with an intention which is on the top(root) of the graph. The example in Figure 3 is a intention structure of an electric facilities layout of which the design intention is to be "beautiful". The intention recognition process in a design proceeds incrementally, since the entire design process described in the previous section may recurse. Each step of the intention
80
, h~ut,u-;,
:.>1
(
....
" "
. "Nn,,,...nod.,
Left-1 ) ( ~ght-1 _)
susJ
oa~ s ~
om
( Figure 3. An Example of Data Structure
Figure 4. Synthesis of Intention Structures
recognition is based on a CBR process, in which the input is a list of partial structures, including object, physical and mental nodes (those represents the current state of design). The step can be divided in two main phases: presumption and extension, which correspond to the retrieval and adaptation of CBR, respectively. The presumption phase is a straightforward case retrieval process from the current input. The similarity measurement between two structures(target and source), which is the core of retrieval, is realized by a recursive graph matching method based on a graph unification algorithm. The value of similarity between two structures is the summation of the lower structures. Similarities between vocabularies, which is the lowest level of the intention structure, are predefined in the vocabulary database. The extension phase is done by synthesizing partial intention structures from the casebase, in order to presume the upper intention, which might reflect wider contents of the design. An example of the synthesis is shown in Figure 4. In this example, the system could retrieve a case (rooted by "beautiful") from which is possible to synthesize two input structure ("Sparse" and "Symmetry"), so that the two inputs are extended to a new structure "beautiful". The retrieval in this phase is also based on the similarity, therefore it is possible to extend structures which do not appear in cases of the current case-base. So by this method, it will be possible to generate new structures across several cases in case-bases. 4. S Y S T E M 4.1.
System
architecture
A system architecture based on the framework described above is shown in Figure 5. The user interface is a graphical one in which a user can directly manipulate design materials and edit graph structures (see Figure 6). The input interpretation part interprets operations and states in the user interface and transfers them to the internal data struc-
81
Problem ' User Interface
Dis play
Operation
Operation
________.t_ ___k
I onu
II
Work
Space
] INetworkEditor]
Input interpretation Scoping
Buffer
Edit
Synthesis Case-Base Retrieval =-
Distance
Similarity
Inference Kernel
Memory
BB~
-----"
Input/Output
Process *;,-',=*;,iiJP,~ Data/Knowledge
Figure 5. System Architecture tures (graph), using the objects and statements knowledge bases. The inference kernel is the main part of CBR, including the retrieval and synthesis programs described in the previous section. The buffer in the inference kernel maintains several intention structures which are presumed by the system to be similar to the input, so that the system can show various information, in case the best matched case would not satisfy the user's intention. The memory part maintains cases as intention structures and provides an editing facility to change structures for users to modify and adapt them to reflect the design intention. 4.2. Y A A D : a p r o t o t y p e We implemented a prototype YAAD(Yet Another CAD system) for electric facilities layout design using C and Motif on a UNIX workstation. A screen copy of Y A A D is shown in Figure 6. This figure is a snapshot of GIS(Gas Insulated Sub-station) layout design. Designers can do some layout operations on objects and edit the intention structure in the left hand two windows, while a retrieved layout example and its intention structure are shown in the right hand windows. Although the system is now under evaluation, several features mentioned above have been verified: • Y A A D provides various information associated with design intention robustly, and most information is based on real cases done by other designers, so that it is useful for the conceptual design phase. • Users can interactively refer to information whenever s/he desires, making it possible for the design process, especially in the conceptual phase, to be effective and smooth. • It is an important feature for users to be able to directly see and modify the means and the intention structure through the graphical interface. On the other hand, several shortcomings have been appeared:
82 ~.j
VAAD
I "" z'zl i
~'0~"'
I
Fie
Edit
Window
Tiree Relemnce
Retrieve
Gold .Inlpecl
Trace
q
• oo
t:l Or- ll° ,
.4
IP,
. . . . . . . . . .
I
i
n
II
HI
o o i N
I
GoalTroe
r~----l-~---l~r~---'~ NN 4
I.-
NI
Figure 6. Screen Copy of YAAD
• Since Y A A D is an interactive system, the response time is crucial to real use. Y A A D can respond quickly when there are not too many scoped objects, but it is slow to respond if the number of objects is large. So developing an efficient method to retrieve examples from the case-base, including parallel and/or intelligent search algorithms, is a future goal. • The quality of information shown by the system heavily depends upon the construction of the vocabularies that the system uses. Development of an effective mechanism to obtain and maintain good relationships between vocabularies is also required. 5. C O N C L U S I O N We described a case-based method and system to support a creative design process, incorporating the ability to infer the designer's intention. Grasping the design intention and sharing it with the designer, the system can provide more useful information and can collaborate with the designer. We verified the features of the method using a prototype YAAD. REFERENCES [1] Tomiyama, T., Kiriyama, T. and Yoshikawa, H.: Intelligent CAD Systems: Today and Tomorrow, Journal of JSAI 7(2), !992, (in Japanese). [2] Kolodner, J. L.: Improving Human Decision Making through Case-Based Decision Aiding, A I Magazine, 1991.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
83
D e s i g n i n g Interfaces for C o m p u t e r - b a s e d A s s e s s m e n t s R. M. Kaplan and I. R. Katz Division of Cognitive and Instructional Science, Educational Testing Service, Princeton, New Jersey 08541 USA 1. INTRODUCTION Educational institutions and testing organizations around the world have been moving away from traditional paper-and-pencil tests toward computer-based tests. Computer-administered tests now exist for college entrance exams and professional licensing exams, among other areas. Advances in graphical user interfaces and artificial intelligence make the computer administration of tests on a large scale feasible. For example, administering a test on computer opens the possibility of automatically scoring candidate responses, resulting in potentially large savings over manual scoring. However, whether we simply reproduce a paper-and-pencil, multiple-choice test onto computer or create new kinds of test questions that require more complex responses, significant issues in interface design inevitably arise. The interface for a test question should require minimal computer skill. The end-users of an assessment system--the test takers--are infrequent users of the system, perhaps taking a test a few times a year or once in their lifetime. Such interfaces must be "walk up and use" [ 1]. That is, the interface for a computer-based test question has constraints similar to applications that are publicly available (e.g., cash machines)--the interface must represent a design understandable by the majority of people who will use the system. Furthermore, greater experience with a test question's interface should not translate to better test-taking performance. Thus, facility with the interface's functions and speed of performance are not as important as ease-of-use for an entire test-taking population. A person should reach "expertise" in a relatively short time, and further practice should result in only modest gains in performance speed. For multiple-choice questions, these design constraints do not typically pose difficult challenges. However, testing organizations are beginning to use questions in which students create their own responses, known as "constructed response" questions. In contrast to multiple-choice questions in which a test taker selects an answer from a set of alternatives, constructed-response questions require that each test taker construct his/her own answer, which could be a number, word, essay, or diagram. Constructed-response questions pose significantly more complex problems for user interface design. When designing interfaces for constructed-response questions, it is necessary to meet all of the goals described above, while at the same time making the question realistic. That is,
84 constructed-response questions typically require test takers to perform real-world tasks. Several challenges arise when attempting to accomplish these goals. On the one hand, making an interface easy to learn and simple to use diminishes the question's correspondence to the real-world task. Simplifying a test question's interface might involve constraining the possible range of user-computer interactions. On the other hand, the closer that a testing interface becomes to real-world tasks, the greater the difficulty of automatic scoring of responses. In other words, the data collected in response to that question becomes more difficult to score via computer algorithm. This trade-off is particularly apparent with questions requiring natural language responses. This paper describes two examples of computer-based constructed-response questions that (a) represent real-world tasks and (b) must be automatically scored. We describe the "design specifications" for each question, along with their corresponding interfaces. Particular attention is given to the iterative evolution of each interface as well as the design rationale behind the initial and subsequent designs. The design rationale focuses on compromises made, such as trade-offs between real-world "look and feel" and the constraints necessary to allow automatic scoring. We first describe a question created to assess certain aspects of architectural skill, called the "block diagram" task. In constructing a block diagram, an architect arranges a set of rooms (e.g., the lobby and customer service area of a bank) onto a building site, specifying the spatial arrangement of rooms as well as the building's location on the site. This type of question represents one of the first tasks that architects perform when designing a new building. As in the real-world situation, the test taker (i.e., an aspiring architect) is provided with a general description of the building and each of its rooms. The architect is free to design the building in any way to meet the constraints specified in the problem. The second question is from the writing domain. In this constructed-response question, a test taker is presented with a one to three paragraph passage. The passage contains syntactic and/or semantic errors that are to be located and corrected by the test taker. The interface must allow test takers to correct the errors in the passage, yet must also keep track of the revisions. Single sentences may contain more than ohe error and errors can cross sentence boundaries. Thus, in order to make automatic scoring of responses possible, each revision must be individually tracked and attached to its related error in the passage. 2. B L O C K DIAGRAM When an architect begins to design a building, they are given a set of specifications. These specifications include, among other things, a list of the spaces, their relative dimensions, and the requirements for connectivity between the spaces of the building. Using this information, the architect concretizes this information in the form of a special schematic representation called a block diagram. If we want to incorporate a task like this on an examination for architect we might do so by providing the architect with a generalized drawing tool. Unfortunately, doing so would make the process of scoring the architect's solution next to impossible. For this reason, the item interface gives the architect all of the tools they need to
85 construct a block solution while, at the same time allowing the characteristics of the solution to be recorded in detail. The computer-based interface for the block diagram task is shown in Figure 1. This figure depicts an architect's nearly completed design. Initially, blocks representing the building's rooms are provided at the top of the screen. Down the left side of the screen are buttons that allows subjects to perform various design actions (e.g., move block, rotate entire current design). In the figure, the architect has placed all of the blocks onto the building site and has indicated how people would move from one room to another (i.e., which rooms are connected and which rooms are not immediately accessible from one another).
illNi ::..:........:..:
NN ........
!~il'
........ t
"
.....
......... Figure 1 - Block item interface
II!II .....
i
:1
:i:
Figure 2 - Block item scoring analysis
The characteristics recorded as an architect creates a block diagram include the positions of the specific blocks and the connections between the blocks. These two pieces of information are critical for the scoring process. For example, one of the measurements that is carried out in the scoring program is the measurement of the paths between each space. For a good solution it is necessary for these lengths to be minimum. To calculate the lengths precisely it is important to know exactly which elements of the block diagram are connections between spaces. The interface includes a tool to precisely place connections between rooms. Figure 2 depicts the visualization interface created by the scoring process. Although this display will not be part of the actual scoring process, it is useful to watch the scoring procedure during development to gain an understanding of the operation of the scoring process. In Figure 2, the small black squares are called "occlusions" and represent "doors" that are placed in the diagram to indicate how the spaces are separated. These are an example of how the scoring process has a direct and critical effect on the task interface. In this case, it was determined that occlusions could not be readily inferred from the connections between the spaces. When the design team for this interface realized this was true, a tool for occlusions was added to the interface so that an architect could place the occlusions where necessary.
86
3. I N T E R L I N E A R M u c h o f our research involves creating computer-based versions o f complex examination items that lend themselves to automatic scoring. The latter is particularly important since, such items would not be cost effective to include on an examination if they were not partially or completely scorable by automatic means. This feature of our "problem space" is even more critical with those items whose response will be in the form of a natural language passage. Because no completely effective means of analyzing natural language presently exists, we must s o m e h o w constrain the task so as to (1) mimic to some degree the entirely open-ended task and (2) collect the same or similar information that we would have in the open-ended task. One type of natural language question that demonstrates this trade-off is used to test writing and revising skills. The original, paper-and-pencil version of this question presents to candidates a passage containing syntactic and semantic errors (Figure 3). The candidate makes corrections in the spaces between the lines of the passage--thus the name "Interlinear." The kinds o f corrections that a candidate can make are completely open-ended--syntactic, semantic or some combination of these (even rewriting or combining sentences). Directions: Reprinted below is a poorly written passage. You are to treat it as though it were the
first draft of a composition of your own, and revise it so that it conforms with standard formal English. Wide spaces have been left between the lines so that you may write in the necessary improvements. Do not omit ideas and do not add any ideas not now present. You may, however, change any word which you think is expressing an idea in exactly; and you may omit words, phrases, or sentences that are unnecessary. You are not expected to rewrite the whole passage. Trying to do so will not only waste your time but will also cause you to miss many of the specific errors you are expected correct. Much of the passage is satisfactory as it stands. Leave such parts alone and concentrate on finding weak places that need changing.
In general, corrections should be made by crossing out the word, phrase, or mark of punctuation you wish to change and writing your own version above it. Any clear methods of indicating changes is satisfactory. Simply make sure that what you intended is clear. Edgar Allan Poe is one of the great writers of the pre-twentieth century time period. He was one of the first to develop theories about the characteristics of short stories. In fact, Poe lay down the three most important rules of a short story in his critique of Hawthorne's "Twice Told Tales." His work was called "The Brief Prose Tale." In this critique, he set down three major laws of a short story; one of the rules was that it must be able to be read in one sitting. Another rule is that there must be a dramatic characteristic that must be carried throughout the story. And the third is that a central theme should be contained throughout the story.
Figure 3 - Sample of the original Interlinear item
87 In the sample above, a candidate can do almost anything to make changes to the passage. It turns out that the rubric for such a question (i.e., the specification of the procedure for scoring a response)is quite extensive and amounts to developing all of the possible alternatives for specific corrections to the passage. The complexity of the scoring process led to the eventual elimination of the paper-and-pencil Interlinear question type as a potential examination item. Recent work in natural language processing techniques now open the possibility of more easily scoring these responses automatically. Of course, instead of administering the question using paper and pencil, we would use a computer to administer the item. At first glance it would seem that a computer-based interface for this item would be very simple to construct. For example, we could simply provide an editing box in which the passage is displayed and allow the candidate to edit the passage as they would with any word processing program. Although this interface may be easy for a candidate to use, because of the freedom allowed in changing the passage, the interface is not a good one for automatically scoring responses. It would be the responsibility of the scoring program to determine what changes were made and whether those changes were correct. Scoring could become particularly complex if makes changes to portions of the passage other than those that are supposed to be corrected. Our task was to design an interface that structured candidates' activities without unduly constraining the types of changes candidates could make. At the same time, for automatic scoring, the interface needs to collect information about changes that are made to the passage. The instructions specify that there are specific errors that the candidates should correct, and that candidates should not spend time revising the whole passage. To accomplish the collection of corrections to the passage, we could present the candidate with a list of words and phrases to correct and allow only these words and phrases to be changed. This type of interface constrains what a candidate can do to the passage and also the data that must be analyzed. In this case, the data that must be analyzed is only the changes that a candidate made corresponding to the word or phrase that was to be changed. The resulting interface is shown in Figure 4. The interface consists of a window containing the passage, one containing a list of phrases to correct, and the related windows for editing and maintaining a list of the edits made to the passage. One of the features of this interface is that the passage is updated as the candidate makes changes to it. Just as in the case of the traditional editing window, this interface allows candidates to read the passage with their changes in place. They can, at any time, also have the passage display return to the original version (i.e., before they introduced changes). Although the interface is very different from the standard word processing interface, preliminary reviews suggest that the interface allows the testing of writing and revision skills while collecting proper information to aid automatic scoring. More extensive user testing is currently underway.
88
I
Hi#work story;oneof the rules must be ableto be read that must be carried
Figure 4 - Interlinear interface
4. CONCLUSIONS Over the last several years we have developed many new constructed-response items types. Part of our efforts in this process has been to develop, in conjunction with the item types, automatic or semi-automatic means for scoring these items. The development of scoring processes feeds directly back to the interface design process. Because creating complex intelligent applications to analyze a particular item can be time consuming, costly, and result in a process that cannot be generally applied, this approach to scoring these items is not a viable approach. An alternative to this is to constrain the item interface in such a way as to assist the scoring process while at the same time leaving the task realistic. We have shown two item types for which this type of development process took place. The first is a graphical task for architects. Rather than allow an architect to develop a completely open-ended solution, the item interface collects the same information while constraining the activities of the architect. Similarly, the second item, in the domain of writing, constrains the activities of the writer while collecting information like that which would be produced in a completely open-ended writing task. REFERENCES
1. Lewis, C., & Poison, P. G. (1991). Cognitive walk throughs: A method for theory-based evaluation of user interfaces(Tutorial Notes). ACM Computer-HumanInteraction
Conference (CH1'91).
lII.5 Interaction Design 3
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
91
WMH methodology for HCI design Christian Co~ff~ Sofr~avia / Centre d'Etudes de la Navigation A~rienne 1, rue Champagne 91200 Athis-mons, France
Abstract This paper aims at bringing up a practical methodology for Human Computer Interaction (HCI) design and a formalism for HCI description which have been successfully used in Air Traffic Management (ATM) field, namely for : • the French PHIDIAS programme developing the next generation of Air Traffic Controller Working Position to be installed in 1997 (Ref 1 ; Ref 2 ; Ref 3) ; • the European SWIFT project launched by the European Commission (DGVII/EURET1.5) (Ref 4 ; Ref 5). This so called WMH (What, Means & How) methodology is not tremendously innovative but it works and its usability by both operational and technical people has been proved. Due to a pressing need for better methods and tools in this part of system design, we think that what we see as a successful experience deserves to be made known to a wider circle of HCI design experts. 1. INTRODUCTION 1.1. The place of HCI aspects in the system d e s i g n Nowadays, nobody would make claims about the efficiency of a humanoperated system without having taken into consideration HCI aspects in the global design. Nevertheless, HCI design should play a different role and take a different place in the whole life cycle of the system depending on the type of application to be developed. When the system is entirely new, most of the time its development is technology-driven. In this case, even though HCI aspects are quite important, they are defined late in the design phase and even without interviewing potential operators. On the contrary, whatever its degree of innovation, when the system is to replace an existing one, operational requirements and constraints, the transition phase from one system to the other, operational acceptability by existing operators are major topics to be addressed. From this statement, the benefit of involving end users early on is very high : HCI operational aspects can be addressed early in the design phase so that the result can be considered as a "user requirements document" which goes much beyond stating general operational objectives. For that purpose, HCI specification is not limited to issues such as information presentation or the way to perform some pre-defined functions. On the contrary, identification and description of HCI functions have to be covered during this HCI design phase.
92 Involving end users from the beginning in this kind of exercise sounds very attractive but following this approach can raise several difficulties : • to distinguish the real user requirements from wide-ranging wishful thinking ; • to gather a representative enough sample of future end users to participate in this HCI design phase ; • to satisfy the user requirements by describing first a feasibility framework ; • to ensure that the output of this phase will be directly usable by developers involved in the global system design. When facing this situation, it seems that HCI design lacks both a methodology and a description formalism to tackle these difficulties. 1.2. T h e m a i n f e a t u r e s o f an e f f i c i e n t HCI m e t h o d o l o g y + f o r m a l i s m According to the needs presented above, a methodology for HCI design will not be efficient unless it is associated with a description formalism. Therefore, an efficient methodology + formalism should ensure t h a t : • questions be addressed and answered one at a time ; • all aspects (functional, interaction and presentation) be taken into account ; • the final description of the HCI tend to completeness ; • both users and developers be put in a position to discuss the document content and to use it ; • the methodology + formalism in itself be easy to learn and to use. The WMH methodology was built with these guidelines in mind. 2. WMH M E T H O D O L O G Y 2.1. G e n e r a l i t i e s Three phases illustrate the proposed "top-down" approach to elaborate and describe HCI specifications. These phases are named" "What to do", "Means to do it °' and "How it runs" and have to be used in sequence. One of the advantages of this WMH methodology is to ensure good quality of functional description (what to do) before any definition of basic HCI components (means to do it) and obviously before their use to perform the functions (how it runs). Splitting the elaboration and description of HCI specifications into these three phases allows to clearly separate the functional level (operational needs) from the technical and implementation levels of HCI design (technical solutions). Phase 1 "What to do" is the functional specification of the HCI requirements ; it means that all HCI functions to be developed are described during this phase. This functional description is quite independent on the way these HCI functions can be implemented in the end. Phase 2 "Means to do it '° is the description of HCI components available to perform these HCI functions ; it means that all the HCI components of the interface are described in detail. At the end of phase 2, background information of the interface is described as well as all the components which will be used for performing the HCI functions. But the detailed way to perform these HCI functions is not yet defined. Phase 3 '°How it runs" is the description of the operational use of HCI components to perform the functions. It means that the way to perform each HCI
93 function is presented in detail so that finally, each elementary action o f dialogue a n d its related effect on the interface is described. Applying this methodology implied the use of a specific formalism facilitating the description of HCI specifications. The proposed formalism mainly refers to the use of standard forms or tables with specific headings depending on the phase to be described. Some basic rules of syntax and wording were also defined to ensure consistency through the whole documentation. This methodology was proposed to a multidisciplinary team composed of the project manager, end users, HCI experts and developers. These actors commonly have different objectives to satisfy and use different languages (Ref 6) . The driving force of WMH methodology is to make easier the necessary partnership to perform a constructive HCI design phase. Table 1 gives the expected role of the different actors depending on the methodology phase. Table 1 : Expected roles of the actors Developers End Users HCI Experts What
I Opqra:i°nalts
Means How
Functional Analysis Representation Consistency Fluency of Dialogue
2.2. P r e s e n t a t i o n of t h e m e t h o d o l o g y
Phase 1 "What to do" This question about what we want the system to do is not answered only by stating the system objectives. It deals with the definition of the operational way to satisfy these general objectives, e.g. the definition of the operational functions to be performed. Therefore, this phase aims at a d o p t i n g a f u n c t i o n a l approach to operational requirements.
Answering the question "what to do" from an operational standpoint means that the operational functions have to be abstracted as much as possible from implementation issues. An operational (or HCI) function is defined as a function to be handled by the operator through an HCI procedure or triggered by the system but in this case having some HCI consequence. Both of these functions have to be defined and described. Therefore, each HCI function is described by using a standard form including the following headings : • objective(s) of the function from an operational standpoint ; • constraints or operational conditions on function availability ; • procedure / triggering event presenting the main steps for performing the function- the number of steps will indicate the necessity or not to perform the function very quickly due to its emergency and / or frequency ; • f i n a l effect / consequences presenting what has been changed in the system, having performed this function - final effects on the system lie beyond HCI ; • r e m a r k s allowing to supply any complementary information.
94 During this phase, implementation solutions such as input devices choices, presentation of information or "mouse clicks" description are not to be addressed. Due to this absence of technical issues, this phase was conducted directly by HCI experts with end users. Phase 2 "Means to d o it" Having defined the HCI functions the system must support, this second question focusses on the definition of HCI components to be used to perform these functions. The first step for that purpose consists in defining the different hardware devices available to perform the input/output aspects of the functions as well as the assignment of the functions to these hardware devices. It would not be possible to define HCI components such as, for instance, the content of a Touch Input Device (TID) page or the content of a dialogue box without having defined before the functions we will perform by using the TID or the pointing device. The second step consists in defining HCI components themselves. It requires the description of" • the generic software displays items (shape of a window, a menu, a button, a dialogue b o x , . . . ) ; • the way to use them (close/open a window, choose a menu option, ... ) independently of their final content ; • the detailed information elements specific to the application (for instance in ATM field : tracks, labels, strips, graphical routes, ...) ; • the detailed dialogue elements specific to the application (for instance the content of dialogue boxes, content of menus, ...). This second phase aims at describing in detail the different categories of HCI components presented above by using standard forms including the following headings • • for the generic sot~ware displays items : definition ; content ; figure ; remarks. • for the way to use them : objective(s) ; procedure ; remarks. • for the detailed information elements : functions initiated via this element ; functions implying any consequence on this element; content; figure ; remarks. • for the detailed dialogue elements : running function ; content ; figure ; remarks. Within these forms, the interest of enumerating the different functions related to the described HCI component is : • to propose a cross-reference framework for consistency checkings (e.g. to make sure each function can be triggered via at least one element) ; • to provide the developers with an HCI component-based description. All HCI components are described during this phase so that the global interface of the system is completely defined from a static point of view. While the first phase deals with operational functions, this second phase implies to involve both operational and technical people since the different proposed HCI choices have to be consistent with existing technical constraints. Phase 3 "How it runs" This third phase aims at describing in detail the way to perform the functions defined in the first phase by using HCI components defined in the second phase. The description of dialogues must be complete enough to allow the
95 implementation of the function in the system. Therefore, the different steps, elementary actions and intermediate consequences are described for each function. This last phase provides a "dynamic" description of the functions and allows to focus on dialogue efficiency and consistency. 2.3. P r e s e n t a t i o n of the f o r m a l i s m The methodology presented above provides the project manager with a structured approach for conducting the HCI design phase. Nevertheless, the different specifications produced during the different phases have to be easily usable, comprehensive and as clear and accurate as possible. Therefore, some wording rules and syntax rules are defined in order to ensure that, independently of the affected phase, a global consistency is made traceable across the different documents. w o r d i n g rules A clear understanding of specifications is made much easier when the proposed wording is precisely defined and used in a systematic way. For instance, carfully writing down the operational objective of each HCI function is very important in order to better understand the real justification of this function. Therefore, questions raise when choosing words for the description • "in which extent does this function differ from that one ? Are the objectives different ?" For instance, "to display a view ..." can be used for system functions triggered by a system event and related to an objective such as "to show ...". In other respects, "to activate a view ..." can be used for user functions triggered by a user action and related to an objective such as "to visualise ...". This is a way to take advantage of the different meanings of the words and to clarify what is really behind the description of the function. Definition of wording rules is certainly arbitrary to some extent but, having defined them, the most important point is to use them consistently. syntax rules In parallel with wording rules, syntax rules were also defined to provide a detailed description of the procedures (e.g. the sequence of actions the user may or must perform, the required conditions to perform these actions and the consequences of these actions). While wording rules are project dependent, syntax rules are quite robust and thus more reusable in other projects. Let us illustrate these basic syntax rules by presenting a theoretical example. • condition A • [] condition B 1 [] condition B2 =~ • action 1 • [consequence 1] • IX]action 21 IX]action 22 m [consequence 2] • [consequence 3] m [consequence 4]
96 The way to read this procedure is as follows : indicates the required conditions ; =v indicates the sequence of actions. • stands for a "AND" operator, C:] a "OR" operator and IX] a "XOR" operator. [ ..... ] indicates a consequence of the action(s) These basic syntax rules are completed with the use of IF, THEN, ELSE allowing the description of different options. Therefore these syntax rules can be combined together to allow the description of complex procedures. The balance achieved between readability and accuracy has been found satisfying according to our own experience. 3. C O N C L U S I O N The methodology presented in this paper was built up for the purpose of involving a user group in the definition of operational requirements then on HCI design. It was our first attempt to meet what we felt as pragmatic requirements on an efficient tool for conducting an HCI design phase. Our initial experience with this methodology encourage us to follow on with using it. WMH methodology appears to possess the main properties required from an efficient tool (as listed at the end of section 1). Nevertheless, it is more suitable to design voluminous HCI since offering a very detailed description and a number of consistency checking requires much effort. This methodology has already been successfully used in ATM field and could be used in any other industrial application where operational requirements and constraints are of the utmost importance ans safety is at a premium.
4. R E F E R E N C E S 1. CENA/R93008 : EBO PHIDIAS 1 - les fonctions d'interface contr61eur-syt~me 2. CENA/R93009 : EBO PHIDIAS - les ~l~ments et les r~gles pour rinterface contr61eur-syst~me 3. CENA/R93015 : EBO PHIDIAS - les dialogues d'interface contrSleur-syst~me 4. SWIFT 2 Report WPR10.1 : Specifications of H u m a n Machine Interface, Preliminary Steps, SWT/SOF/WPR/10.1/01, March 1994 5. SWIFT Report WPR10.2 • Specifications of H u m a n Machine Interface, Detailed HMI Specifications, SWT/SOF/WPR/10.2/01, September 1994 6. Beck, A. (1993) : User Participation in Systems Design - Results of a field study - : In : Smith M.J., Salvendy G. (Ed.) : Proc. of the 5th Intern. Conf. on H u m a n - C o m p u t e r Interaction, Orlando :Elsevier, 534-539 1 Position Harmonisant et Int~grant les Dialogues Interactifs, Assistances et Secours (French programme developing the next generation of Air Traffic Control Working Position) 2 Specifications for controller Working positions In Future air Traffic control (European Commission project conducted by Thomson CSF (F)with the collaboration of Siemens Plessey (UK), ESG (D), NLR (NL), Sofr~avia (F), Syseca (F), Roke Manor Research (UK), Aegean University (GR), Captec (IRL) and INESC (PTL)
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
Guided Understanding for P r o b l e m Solving cess Using the Refining Self Explanation
97
Pro-
K. K a n e n i s h i t and Y. Yano$ t D e p a r t m e n t of Children Education, Shikoku University Junior College, 123-1 F u r u k a w a , Ohjin-Cho, Tokushima 771-11, J a p a n $ D e p a r t m e n t of Information Science and Intelligent Systems, Faculty of Engineering, The University of Tokushima, 2-1 Minamijousanjima-Cho, Tokushima 770,Japan 1. I N T R O D U C T I O N
We use explanations to communicate knowledge. However, after the explanation, we find ourselves being able to u n d e r s t a n d the explained subject more deeply. It is considered t h a t explanations have a side effect to help the person who explains to u n d e r s t a n d the subject better, as well as the function to communicate knowledge to others. Here, we decided to use the side effect of explanations for the purpose of education. At present, we are building an intelligent learning environment for geography[i-2] by using self-explanations. In this paper, this learning environment employing self explanation is described. The explanations we is deal with are explanations in the problem-solving process. First, we let a learner solve an exercise in geography. Consequently the l e a r n e r explains her/his solution process to the system. The l e a r n e r deepens on her/his u n d e r s t a n d i n g of geographical problem-solving process by explaining a solution. We believe t h a t learning will be deepened not only by solving the exercise but also by explaining how to solve it. Moreover, it is i m p o r t a n t for the learner to know how to give a self-explanation. It is also i m p o r t a n t to give appropriate advice to the learner r a t h e r t h a n to let him or her explain without any advice. By giving advice, the learning effect will be increased. In the environment of self explanation, the guidance of the learner by the system is critical. To guide the learner, a learning model is necessary. With self explanations, it is obligatory to clarify how learning advances in the mind of the learner. We believe t h a t she/he is carrying out a reflection process due to self explanations. Reflection is also one of the i m p o r t a n t mechanisms in learning. The learner reflects the problem-solving process by explaining it. Reflection can more deep the u n d e r s t a n d i n g of the problem-solving process. To make the l e a r n e r reflect willingly means to make him proceed with learning. It is difficult for the system to control reflection directly. How-
98
ever, the system can control the function of the reflection indirectly by controlling the environment. Recently, studies of intelligent learning environment focusing on reflection have been made explanation based learning[3]. However, it should be pointed that in these studies the control of reflection is insufficient. In this paper, u n d e r s t a n d i n g by self-explanation is described first. Then, the ways to support self-explanation by the system are described. 2. M O D E L O F S E L F - E X P L A N A T I O N
Three cognitive processes are made by the learner's self-explanation is described. Self-explanation is made up of three large recognizing effects. Figure 1 shows the self-explanation model which is composed of several subprocesses. By making the learner explain by himself, learning proceeds within the learner[4-5]. Three of the cognitive processes in self-explanation are reflection, explanation generation and explanation planning. In self-explanation, a purpose is set first. Along with the set purpose, reflection and explanation work in parallel. In reflection, a solution is replayed. Because of the replaying of the solution to the problem, that has already been solved, the learner can observe the problem-solving process subjectively. A problem-solving process obtained as a result of the reflection is sent to the explanation generation. In explanation generation, the explanation is made through division, classification and integration. The explanation planner contents the three stages of explanation generation and manages the exchange between
i~~i~!~!!il i~:~~!
iiiiiiiiii!ii!iiili !iiiiiiil iiiiiiii~~iii~M~M~i~i~i~i~i~ii~i~i~ii~i~ii~iii~i~M!i~iiiiMii!i~iMi~!!~i~
i!ii!ii i!!!ii!il
!iiiii!iiiiiiiiiiiiiiiliillili!iiiiiiiiiiii!iii iii!iii!i!iiiii i!il
iiiiiii!!!i!i!iii!iiiiiii!i~,:!:~' iiii!~!~,:'~i:.iil
:i::i
~ii!iii!iiiiiiiiiiiiiiiii!ii!ii!iiiiiiilili! !iii!ii!!ii~~. ~~~iii~i;i~i~i!ii!!!iiii!i;i!i~!;!!i~!iii!!~i~iiii~!!~i~i~ii!!~ii~i~iiiii!!ii~i~i!i~i~
~'t
i~!~(~,,~= i
macm~
_o
~::iil;iiii
J
iii!i!i!!iii!i: :! i i~i~!i::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::::::::::::::::ii~i!~:i~iiiiiii!ii!i!iii!i :::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::::::::::::::::::::::::::
i!i!iil !iiiii!ii~ii:iiii::i:i:'i~i~i~i:i:!:ii~i~i~i~ii~i:~:~i~i~i:~i~i~!~!~i~!i~!i~i!~i!~!i~i!~!i~..!i~..i~.~ ..!~..!.........................~..i!.~.i.. •
ii
i iiiiiiiiii
i!i
i
i iiiili!i!iii~i~!~i~i:?:i:i:i:~i~i~i:i:i~i~i~!~i:::~i:!~i:!:!:i:!:i~i,'~i~ii,ii~'i!iiiiiiiiii!iiii' ! ,ii',iiii!iiii!i!i!iii!ili~,ii!iii' i ~i',i',i!',i ii!i~,i
:i?ii:i!i•••i::•i!•ii•i•ii:.i••iii:iiigi:i:::••::::•i::?•::i•i:i!ii?ii:f::::i:iiiil;i:: :•i•!i:•i•
~
|
~
Feedbackinformation
Learner ---....-~
Figure 1. Process of the self-explanation.
• Control now
Figure 2. Construction of the self-explanation environment.
99 the explanation generation and reflection. In the division phase, the problem-solving process obtained in reflection is divided into the explanation's components. In the classification phase, each divided component is classified. In the integration phase, classified elements are assembled as an explanation. When assembling an explanation, the planner aims to make an optimal explanation which is made in this way. Self-explanation doesn't always succeed. Sometimes other people can't u n d e r s t a n d an explanation. In this case, explanation generation is made repeatedly. But, when generating an explanation again, the learner wonders why the previous explanation has failed. The explanation corresponding to the reason failure is formed. For repeated explanation, additional information is communicated to the explanation planner. This additional information is also handed to the reflection process. We have already mentioned t h a t the u n d e r s t a n d i n g of a problem-solving process is enhanced by the leaner's reflection. Moreover, in the process of an explanation generation reflection allows progress in the u n d e r s t a n d ing. In the explanation generation, this occurs because is observed once again the problem-solving process t h a t happened in the reflection process. The explanation is asimbolising process. Therefore, the classification and the reorganization of the knowledge is done strictly. Also, by the failure in explanation, the work in each stage of the explanation generation process is completed in detail. As a result, it is considered t h a t the u n d e r s t a n d i n g of the problem solving process. 3. U S E OF E X P L A N A T I O N IN E D U C A T I O N 3.1. T h e r e f i m e n t o f t h e s e l f e x p l a n a t i o n The refiment of the explanation in our learning environment which uses self explanation is described. This refiment of the self explanation consists of five modules, 1)the explanation transformationmodule, 2)the evaluation module, 3)the planning explanation support module, 4)the advice control module 5)the interface module. The support of the self explanation process is shown in figure 2. The learner repeats an explanation through the environment. According to the advice from the system, she/he repeats an explanation. By repeating the explanation, the explanation is gradually polished. If the explanation becomes good enough, the learner deepens his understanding. 3.2. T h e w a y to r e f i n e t h e e x p l a n a t i o n Th explanation units stand basically straight in line sequentially. The contents of the explanation and the structure of the explanation deeply affect its evaluation. When the structure of the explanation is appropriate, it is possible to call the explanation a good explanation. The appropriateness of the structure of the explanation depends on the object of the explanation. We believe t h a t there is a m i n i m u m set of necessary elements which shows an appropriate structure[6]. In the explanation of the problem-solving process, the explanation of the initial state and the explanation of the goal
100 state are surely necessary. In the explanation there are partial explanations of each state, and they are connected according to the sequence of states. The partial explanations which does not have any connection before and after is considered as an abnormal structure. As a way of evaluating the s t r u c t u r e of the explanation, the structure of the optimal explanation has been prepare. The way to refine an explanation is shown in figure 3. In consists of three methods. 1) the insertion of the: partial explanation, 2) the deletion of the partial explanation and 3) the division of the partial explanation. In the insertion of a partial explanation, it is necessary to make the learner to repeat partial explanation when structure is not complete. The system advises t h a t the partial explanation is not compleate. The system advises to delete a r e d u n d a n t partial explanation. When the explanation is redundant, it is necessary to remove it. The system indicates the learner t h a t the explanation is too long. In the division of a partial explanation, in order to give an explanation in detail, the learner inserts some partial explanations. The division of the partial explanation is different from the insertion of the partial explanation. Generally, the learner tends to give only the explanation of the de-
I
~l Unit -- pl'~
Unit p l ~ L )
:iii::iii!iii::i!i::;i~ii!ii~!~::i::~iii;i;::~i~!i::~::~;i!::!ii::;::i .........i..........
"~"" ~
.........
Unit .. , 1 , ~ . : . .
..... U::n:::.............. ~: i:i:i:!:i:i:i:i:i:;:i:i:i:;:i:i:i:i::
Unit
Deletion of explanation unit
.................................
....~3:: .................................~i~ili~:~.......
;
~ i i i i
Unit
Insert Ion of explanation unit
unit p l - " " ~
UnR pl
i:ii!i!!~i!iii!i!ii~...... i~:i
unit pl
i~!~~~i~!~!~i~!~i~::
..% . i!i!i!i!ii!i!i!i!i!i!i!ii~~ .... '
Detail ed explanation
:% ' iiiiii!iiiiii~i~~ ' ,:';:
Division of explanationunit
Roughed explanation
Figure 3. Way of refining an explanation.
101
clarative knowledge. The system require the learner to explain about meta knowledge which is attached to the declarative knowledge. We believe t h a t the division of the partial explanation makes a detailed change to the explanation. The system generates advices such as =why does Poland produce coal?" for partial explanations such as =Poland produces coal". By using the three methods presented above, it is possible to refine the explanation of the learner. Based on the evaluation of the structure of the explanation, the kind of refinement is chosen. In the division of the explanation, the evaluation of both the structure and the contents of the explanation become necessary. 4. S E L F - E X P L A N A T I O N L E A R N I N G E N V I R O N M E N T
In our learning environment we use a graphical interface. Figure 4 shows our system's outlook. The learners can input their explanations by selecting an icon displayed on the screen and writing simple key words. We believe t h a t the input of explanations using natural language becomes a burden for the learners. Learners explanations are symbolized, in order to express their structure visually. Each explanation is expressed on the explanation screen as a picture. We think that the learners notice the problems in the structure of their explanation from the graphical display. Also, Advice window Main w i n d o w
i~
::. :~
~ ~ ~ ' ~ i l i ~ i
~ = ~ "~-'~
I.~ 11
![~ .......................... : " ';5~
~:~'~-~"
fl~J~4 :
,~'t,ll,
B~,~'¢~,~,
Response v~dow
:
ilii];:;i .,
-:
..................
:::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::~;~:~:~:~;~;:~i:i
Explanationexpressionwindow
Figure 4. Overlook of the self explanation environment.
102 the system displays its advice on the advice screen, corresponding to the mistakes of both the declarative knowledge or the meta knowledge of the learner. 5. C O N C L U S I O N In this paper, we presented the recognition model of self explanation. Self explanation consists of the reflection and the explanation production. By giving a self explanation, the understanding of the learner's problemsolving process progresses. We showed the necessity of the environment to make the self explanation lively. Then, according the way of supporting a self explanation, the learner who does agood to explanation is seem to understand an object well. We discribed construction of the self-explanation environment, 1)the explanation transformationmodule, 2)the evaluation module, 3)the planning explanation support module, 4)the advice control module 5)the interface module. And we discribed way to refine an explanation, 1) the insertion of the partial explanation, 2) the deletion of the partial explanation and 3) the division of the partial explanation. We proposed the graphic explanation environment. The graphic environment helps the understanding of the learner. Also, the environment of our system integrates the conventional ITS and the environmental CAI[79]. REFERENCES
1. K. Kanenishi, T. Fujisaki and Y. Yano, Knowledge Structure for Geographical Intelligent Tutoring Systems(ITS), EAST-WEST CONFERENCE on Emerging Computer Technologies in Education, 38, 1992. 2. K. Kanenishi, T. Fujisaki and Y. Yano, Problem Solving Process in Geographical ITS, Proceeding of the IFIP OPEN CONFERENCE, 9-14, 1992. 3. A. Kashihara, K. Matsumura, T. Hirashima and T. Toyota, Load-oriented tutoring to enhance student's explanation understanding, - an explanation planner and self-explanation environment -, IEICET trans., E77-D, 1, 27-38, 1994. 4. C. Bereiter and M. Scardamalia, The psychology of written composition. Lawrence Erlbaum Associates, 1987. 5. J. R. Hayes and L. Flower, Writing research and the writer. American Psychologist, 41, 1106-1113, 1987. 6. K. Kanenishi, and Y. Yano, Construction of an ITS for Geography , Proceeding of the ninth conference on education technology, 344, 1993. 7. D. Sleeman and J. Brown, Intelligent Tutoring Systems, Academic Press, London, 1982. 8. T. Hayashi, and Y. Yano, Kanji Laboratory: An Environmental ICAI System for Kanji Learning, Trans. of IEICE E77-D, 80-88, January 1994. 9. R.Okamoto, and Y. Yano, Development of an Environmental ICAI System forr English Conversation Learning, Trans. of IEICE E77-D , 118-128, J a n u a r y 1994.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
A STRATEGY AND DEVELOPMENT
TECHNOLOGY
103
FOR
FRONT
END
SYSTEM
Linda Candy, Ernest Edmonds, Susan Heggie, Bryan Murray and Nick Rousseau LUTCHI Research Centre, Department of Computer Studies, Loughborough University of Technology, Loughborough, LE 11 3TU. U.K. Abstract
This paper describes an approach to the enhancement of existing software and the development of new applications based upon the premise that advanced software technology is not in itself sufficient to realise high quality usable systems. Development strategies designed to ensure quality must be accompanied by appropriate system architectures and effective implementation tools. We describe a technology and strategy that together enable the efficient development of user and task support systems in a wide variety of contexts. 1.
INTRODUCTION
There exist a wide variety of situations where the complexity of the users' activities and the range of support applications are such that explicit task support must be integrated within the total system. For example, in the field of scientific computation, it has been recognised for some time that the needs of users are not being met by most current software applications [1]. For reasons of economy and the poor availability of such expertise, it is necessary to bridge the gulf between user needs and system solutions and yet to make use of the vast amount of reliable and comprehensive software packages already on the market. Advanced technology such as graphical user interfaces and multi-media may enable developers to construct better users interfaces to existing systems. However, we argue that this must go further than providing mere "facades" [2]. If the system is to be tailored to the needs and task of professional expert users, there is a need for task specific support which is achieved by mapping the users' expertise and the functionality of existing applications. The work described is concerned with the development of complex systems that employ reusability, task support, and the integration of application functionality. An important characteristic of the target users is that they are skilled professionals employed in critical problem-solving roles and, as such they have considerable discretion as to how they complete their tasks. Therefore, they are at liberty to refuse to use a software system that is not suitably tailored to their needs. In consequence they need to be closely involved in a user-centred strategy for system development. The technology discussed in this paper is the Front End System technology (FES) [3]. The FES architecture and tools originated in Interactive Systems development, including and especially, User Interface Management (UIM) systems [4]. The development strategy and associated methods are drawn from a wide spectrum of studies in human-computer interaction and practical applications in the general software engineering field. The strategy and technology were developed in tandem and tested in industrial contexts as part of a large ESPRIT2 project. 2. STRATEGY AND METHODS This section summarises the overall life cycle strategy for FES development. The key concept that underpins the strategy is iterative design within which the role of prototyping and evaluation during the process are key elements as shown in Figure 1 below. The approach is
104 one of user-centred design drawing on existing expertise [5] and a concern to offer costeffective and practical methods for industrial contexts [6].
B test
Sys~[em "
Concept I Requirements
lili
Field
Specification
III I
I I!
I i
I l
|
I I I
,
Evaluation
Formative
Key
:.:~-:.'.'.'.'.'.'-'-'-'.'-'.'"":11
)
~Evaluation /~ Feed:)ack !
Figure 1 The Iterative Development Strategy, Prototyping and Formative Evaluation
2.1
Iterative Development
Strategy
A significant amount of effort must be invested in the analysis of user needs and requirements early prior to any design or implementation work [7]. While the information captured in analysis will be invaluable in supporting the system design process, it will not necessarily be sufficient to enable the design team to "get it right first time". It is often the case that information about the users' requirements will not have been captured fully during the analysis activities• That information will be obtainable only after a prototype is available to evaluate with users. With conventional approaches to development, the effort that is invested in designing and implementing a system prototype is such that it is difficult to make substantial changes and remain within the budget for the project. It is necessary, therefore, to have an early phase of rapid growth of learning that combines analysis and design with evaluations. A second phase where a number of key decisions are made takes place before further development and evaluation activity. During this phase, it is still possible to make changes but they become increasingly superficial as the knowledge is refined.
2.2 Prototyping One of the critical factors in making iterative development a practical possibility is whether or not one can generate prototypes or simulations that embody minimal commitment whilst enabling valid evaluations to be performed. An iterative development process requires the use of evolutionary prototyping as a tool for exploration as well as development. There are three basic types of prototype: Laboratory, Field and Delivery Prototypes. Laboratory prototypes include screen representations and dialogue simulations and may have a set of minimal functions but are not sufficiently robust to be delivered to user sites unsupported. They are used by developers to evaluate technical issues and also to carry out usability tests in experimental task scenarios. Field prototypes can be delivered to the user work environment where they are used to evaluate the support the system provides for real tasks and the impact this has on the general work design. They need to be more robust than the Laboratory prototypes but may not necessarily provide full functionality. They are used to validate the requirements specification in a more realistic situation, run in parallel with existing methods of performance. Delivery prototypes can be delivered to users to support real tasks, but that may not provide full functionality. To be effective, all the prototypes must be quick to build, realistic to users and evolutionary. The technology support for prototype construction is discussed in section 4 below.
105
2.3 Evaluation Evaluation may be classified as formative, measurement or diagnostic. Formative evaluation is particularly important early on and may be used with user interface mock-ups without real functionality. However, as calls to the functional units are developed these can be included in the prototypes and incorporated in the evaluations. Later in the project, evaluations will be more diagnostic or measurement-oriented as attention turns to minimising usability problems and achieving desired levels of performance. At the end of the day, the goal is to support the users' task performance and to provide a system that they experience as usable. It is important, therefore, to focus on the users' experience of their task and of usability [8]. The results of the evaluations may take the form of new or changed requirements which can be immediately fed back into the design process. The presentation of prototypes and design proposals represents a vital mechanism for achieving a full and accurate understanding of the objectives of the system [9]. Methods that support the different types of evaluation are essential for the evolution of the different prototypes. 2.4 Methods Many methods have been developed to support different development strategies. There is a need for methods that can be applied at different levels of complexity, according to the needs of the problem domain and development context. No single method will be appropriate for all development contexts and there is a need for methods that already embody considerable tailoring so that they can be picked up and used by developers, once selected from a "toolbox". The FOCUS methods address the need for 'discount' analysis methods. For that reason, they are quick to learn and easy to use for system developers. They may form the basis for evaluation reports or simply result in changes to a prototype leading to the next version. One example is the User-Software Observation Method (USOM) which involves users performing tasks and developers observing and discussing task performance with the users. It is an evaluation method which uses observational and verbal protocol techniques. 3.
FRONT END SYSTEM A R C H I T E C T U R E AND TOOLS
A Front End System is a separable user interface system that integrates new and existing applications, services and knowledge-based task support in order to provide specific users with tailored solutions. FES Technology is based on an extended Seeheim model, with a distributed modular architecture that employs a client-server level of separation between the modules. It has a specialised Application Interface Module and a Support Module that addresses task support and highly interactive applications. The fundamental concepts were described by Edmonds & McDaid [ 10] and refined in Edmonds et al [3]. The major components and relationships within the architecture are shown in Figure 2 below.
...........................
;-;---......
|
-
App,°.,lo. Inter.orion
l. ..... .'..'.."..'..~.:.'. ..... I
_0.............* _ .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
|c ..... ,c,,,,,, i ._IL_......................... ......... ."..'..7.:.;.'. ........_|
0 Figure 2. -............ The FOCUS Front End System Architecture
............... iiiil
I-A-,i.:~.-Ii
i ""qb- .......... ----'~"" ..............
!
106 The kind of modular architecture shown in the figure above can only work if communication between its components is simple, fast and flexible. FES communication is handled by 'messages', which are routed by the Communications Manager. The following sections briefly describe the concepts involved and the function of each component shown in the architecture.
3.1 User Interaction Manager (UIM) The User Interaction Manager provides user interface presentation and dialogue services to the other modules in the architecture. In addition, it controls access to the user and mediates between modules in conflict for access to the user. Its main function is to create and realise the details of the object oriented dialogue specification it receives via the central message system. The UIM is composed of three major components. The Dynamic Dialogue Manager (DDM) enables the monitoring of all dialogue traffic and where necessary the limiting of access to the user because, for example the screen is already very full and there is a clear risk of overloading the user. The Dynamic Presentation Manager (DPM) is responsible for the maintenance of the logical state of the interface and the interaction objects. It creates, updates and destroys Abstraction Interaction Objects (AIOs), using a library of AIO prototypes. When an event occurs at the end user interface the DPM sends an appropriate return message, using the standard message format. The Physical Presentation Layer (PPL) handles those parts of the presentation function that are specific to a particular windowing system and tool kit. It is responsible for mapping instantiated interaction objects received from the DPM to the presentation system in use.
3.2
Application Interaction Manager (AIM)
If existing software is to be integrated to support a user's task, it cannot use standard messages to communicate. Therefore, these applications must be integrated into the Front End System technology through the Application Interaction Manager (AIM) [11]. The AIM consists of two main components. The Task Manager deals with abstract application independent tasks. It passes a request to the Application Action Manager which has within it all the information necessary to realise that task in an application specific manner. k
3.3 Support Modules A Support Module communicates directly with the rest of the architecture using messages. Support Modules can have a variety of roles, although they may be re-usable in different systems. They range from having general purpose dialogue models to having a task specific dialogue that involves an integration of dialogue model and executable task model. This integration can either involve a logical separation [ 12], or a physical separation [ 14] of these models. In the latter case, individual models 'communicate' using the standard message system. Support Modules provide a range of functionality, such as analysis or complex knowledge base manipulation or visualisation tools, which are not available as Managed Applications. They typically support complex tasks and contains considerable knowledge based task and domain support. They involve task level integration of application functionality to support the user's task(s). They may contain an executable model of the task. There is an integration of the task and dialogue models so that each sub-task, or 'goal', relates to dialogue with the user to obtain data or decision information, internal domain specific processing or a task specified in the AIM's Task Manager. To obtain the data to satisfy its goals, a Support Module may, for example, obtain values for all the necessary parameters by interacting with the end user (via messages and AIOs) and then send a message to the AIM containing an application-independent task description. Alternatively, it may access the AIM to obtain data to support communication with the user. 4. T E C H N O L O G Y SUPPORT TO STRATEGY The strategy and technology have been described in brief. It is, however, the combination that is
107 significant. It is important that this combination is rather an integration and therefore, the integration needs to be of a reasonable depth to produce an effective solution. The section below discusses the integration between the technology and the strategy and methods.
4.1 Support for Iterative Development The Front End technology directly supports the strategy in two main areas: those of Prototyping and Evaluation. In addition it provides considerable support for the project management which is important in making the strategy commercially feasible. 4.2 Support for Prototype Construction The FES System technology provides a range of support to the incremental prototyping approach from laboratory to delivery prototypes The FES toolkit enables the developer to specify user-system dialogues much more rapidly than using conventional programming languages. This is because one can define a dialogue at an abstract level, leaving details of presentation to the User Interaction Manager to determine. Prototypes that simulate calls to underlying functionality that may not yet have been connected can be developed. This enables the FES technology to support rapid user interface prototyping. Also, when building a standard front end (i.e. to existing functionality), one does not have to implement the underlying functionality, but just to call it. In particular, it is much quicker and easier to alter such function calls than to change the actual functional code. These prototypes offering extensive functionality can still be changed with relatively little effort. For certain FESs there will be components that represent new functionality (complex Support Modules). These will have to be developed along more conventional lines but can be integrated with the rest of the system when they are stable. This provides a prototyping tool kit of power and flexibility. The FES AIO provides a rapid specification method with a limited learning curve and the use of defaults to further accelerate the specification process. Using the development tools for prototyping means that software redesign is only needed if fundamental changes are called for. The fidelity, scope and availability of the prototypes developed will also affect the validity of evaluation. Because the tools used are the delivery tools, the prototype will be faithful in 'look and feel' to the proposed system. The functionality and function calls that are implemented will call the Managed Applications. The prototype will increase in breadth as these function calls are developed. However, the functional components will only be integrated when they are stable, and, therefore, the user interface prototypes will be almost continuously executable.
4.3 Support for Evaluation The FES Technology provides a range of support for evaluation. The two main areas are the provision of facilities such as user logging and the ability to generate changes to prototypes rapidly in a structured and well managed manner. Logging facilities provide records of user interactions. This facility is accessed via a simple high level switch. There are facilities to preprocess logs to produce manageable quantities of data. The ability to change prototypes quickly at a range of granularity in response to evaluation is of considerable importance. This can range from changing the detail of a menu item which involves simple text editing to rapidly creating new interface objects using the AIO specification method. The FES technology generates the interface 'on the fly' and therefore there are no detailed representations such as State Transition Networks to edit when changing dialogue details. Finally, the modularity of the system means that a whole new module can be developed in response to evaluation and easily integrated as the module interfaces are defined by the messaging system. 4.4 Support for Project Management The modular construction of the architecture has important implications for the control and management of the software development process. Individual modules can be developed in parallel and various teams can be created to construct the different modules. The development task is, therefore, broken down into manageable development components supported by the underlying architecture. The minimal interfaces between the architectural components eases the
108 management of the development process by limiting the inter-dependencies that have to be managed. The technology, strategy and methods have been used to develop various Front End systems one of which SEPSOL [14], a complex multi-application system that supports chemists using statistical design to identify suitable models for their experiments. It employs considerable domain knowledge in a variety of roles. The support provided by the technology and methods enabled the completion of SEPSOL within the target time of twelve person months. 5.
CONCLUSION
This paper has described an approach to Front End System development that involves the combination of a development strategy and a supporting technology. It has described the main points of the strategy and the key components of the technology. REFERENCES
1. Hague, S. and Reid, I. The changing face of scientific computing. Human Aspects in Computing: Design and Use of Interactive Systems and Information Management. Ed H.-J. Bullinger. Amsterdam: Elsevier. 1991, pp 791-795. 2. Windsor, P. An object-oriented framework for prototyping user interfaces. In Diaper, D., Gilmore, D., Cockton, G. and Shackel, B., Human-Computer Interaction: INTERACT'90. Proceedings of IFIP TC 13 Third International Conference on Human-Computer Interaction, Cambridge, UK, Amsterdam, North-Holland, 1990, pp 309-314. 3. Edmonds, E.A., Murray, B.S., Ghazikhanian, J. and Heggie, S.P. The re-use and integration of existing software: a central role for the intelligent user interface. People and Computers VIII. A. Monk, D. Diaper and M. Harrison (Eds.), Cambridge University Press, Cambridge, 1992, pp 414-427. 4. Pfaff, G.E. User Interface Management Systems, Springer- Verlag:Berlin, 1985. 5. Gould, J.D., Boies, S.J. and Lewis, C. Making usable, useful, productivity-enhancing computer applications. Communications of the ACM, 34, 1, 1991, pp 74-85. 6. Wright, P.C. and Monk, A.F. A cost-effective evaluation method for use by designers. International Journal of Man-Machine Studies. 35, 6, 1992, pp 891-912. 7. DTI/NCC, Starts Guide, 2nd Edition, 1987. 8. Whiteside, J., Bennett, J. and Holtzblatt, K. Usability engineering: our experience and evolution. In Helander, M. Handbook of Human-Computer Interaction. Handbook of HumanComputer Interaction. Amsterdam, Elsevier Science, 1988, pp 791-817. 9. Edmonds, E.A., Candy, L., Lunn, S. and Slatter, P. Issues in the design of expert systems for business, Expert Systems for Information Management, Vol 2, No1,1989, ppl-22. 10. Edmonds, E.A. & McDaid, E. An Architecture for Knowledge-Based Front Ends. Knowledge-Based Systems. 3, 4, 1990, pp 221-224. 11. Prat, A., Lores, J., Fletcher, P. & Catot, J.M. Back-end Manager: an Interface between a Knowledge-Based Front End and its Application Sub-systems. Knowledge-Based Systems. 3, 4, 1990, pp 225-229. 12. Murray, B.S. & Edmonds, E.A. Flexibility in Interface Design. IEE Proceedings-E, Computers and Digital Techniques (Special issue), 1993. 13. Copas, C.V. and Edmonds, E.A., Executable Task Analysis: Integration Issues, People and Computers IX, Cockton, G. Draper, S.W., Weir, G.R.S. Cambridge University Press, Hcr94, August, 1994, pp 339-352. 14. Murray, B.S., Edmonds, E.A. & Govaert, B. SEPSOL: an experimental knowledge-based front end developed using the FOCUS architecture. Proceedings of PRICAI'92, September, Seoul, Sth. Korea 1992, pp 447-455.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) 1995 Elsevier Science B.V.
109
A M e t h o d o l o g y for D e v e l o p i n g N e w Interaction Techniques Deborah Hixl, 2, James N. Templeman 2, Ankush Gosain2, 3, and Kapil Dandekar2, 4 1 Department of Computer Science, Virginia Tech, Blacksburg VA 24061 USA ([email protected]) 2 Navy Center for AI, Naval Research Laboratory, Washington DC 20375 USA ([email protected]) 3 Department of Chemistry, Johns Hopkins University, Baltimore MD 21218 USA ([email protected]) 4 Department of Electrical Engineering, University of Virginia, Charlottesville VA 22904 USA ([email protected]) We present a m e t h o d o l o g y for inventing, implementing, and evaluating new interaction techniques. We illustrate use of this m e t h o d o l o g y using examples of some of the more interesting issues we encountered in developing a new interaction technique for head-coupled panning and zooming, called pre-screen projection. 1. INTRODUCTION An interaction technique is a way in which a human uses a physical input/output device to perform a task in a human-computer dialogue. It abstracts a class of generic interactive tasks, for example, selecting an object on a screen by pointing and clicking with a mouse. A pop-up menu is an interaction technique. Interaction techniques are a useful research topic because they are specific enough to be studied, yet general enough to have practical applicability to a broad variety of interactive systems. But research specifically in interaction techniques often emphasizes technological creativity, while user-based evaluation of techniques is either cursory or non-existent. Thus, over the years, a plethora of interaction techniques have been developed, but to date we know little about their impact on the usability of interactive systems - - we have only scant empirical evidence of whether they improve human performance and satisfaction. In this paper, we motivate the need for a methodology for developing interaction techniques, and present details of a methodology we have evolved for inventing, implementing, and evaluating new interaction techniques. We include examples of some of the more interesting issues we encountered in developing a new interaction technique, called pre-screen projection, for head-coupled panning and zooming.
2. A DEVELOPMENT METHODOLOGY FOR INTERACTION TECHNIQUES On the surface, developing a new interaction technique sounds easy. However, even after the initial idea for a new technique is conceived, designing and implementing the technique can still be difficult. Setting the new technique in the context of meaningful user tasks in a realistic setting, perhaps along with other new or existing interaction techniques, involves a myriad of unanticipated design decisions, often with unobvious consequences. Our recognition of the need for a methodology for developing interaction techniques has evolved through much experience, as we have produced several new techniques [e.g., 1, 2].
110 We found such a development process should include evaluation of the effects of a new interaction technique on human performance, not simply implementation to see if a new technique operates correctly. The major components of this methodology and their relationships are shown in Figure 1. Unfortunately, as mentioned earlier, interaction techniques research often stops after the technique has been coded and tested to make sure that its software works as intended. The remaining three components are ignored. To understand the effectiveness and efficiency of new interaction techniques, it is necessary to co-evolve tasks for a user to perform with a new technique, to set those tasks in an application that can be used in an empirical study, and to perform empirical evaluations of the techniques using tasks within the chosen application. Interaction Technique Application User Tasks . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Empirical Evaluation
J Figure 1. A methodology for interaction technique development. 2.1. Creating a New Interaction Technique Creating a new interaction technique involves many steps, beginning with its conceptualization m for example, the idea, the real world metaphor, or whatever is its underlying motivation. Conceptualization is followed by design of the technique, going from general abstractions to specific details, and prototyping to determine what use of the technique "feels like". Eventually the technique is implemented and continually refined, even before it is set in the context of user tasks or an application.
Pre-screen projection is an interaction technique that allows a user to pan and zoom integrally through a scene simply by moving the head relative to the screen. Its conceptualization is based on real world visual perception [3], namely, the fact that a person's view changes as the head moves. Pre-screen projection tracks a user's head in three dimensions and alters the display on the screen relative to head position, giving a natural perspective effect in response to a user's head movements. Further, projection of a virtual scene is calculated as if that scene were in front of the screen. As a result, the visible scene displayed on the physical screen expands (zooms) dramatically as a user moves closer. This is analogous to the real world, where the nearer an object is, the more rapidly it visually expands as a person moves toward it. Using pre-screen projection, a user wears a lightweight helmet or headband with a 3-D Polhemus tracker mounted on the front, as shown in Figure 2. As the user moves from side to side, the display smoothly pans over the world view. As the user moves closer to or further from the screen, the display smoothly zooms in and out, respectively. The virtual scene is calculated to appear as if it were 20 inches in front of the physical screen. Thus, the scene is displayed on the physical screen, but its dynamic perspective from the user's viewpoint reacts to head movements as if the scene were in front of the screen. This causes the scene to enlarge more rapidly than the screen as a user moves toward it and therefore produces a dramatic zoom. Having developed this concept to underlie pre-screen projection, we began the challenge of instantiating the interaction technique, which includes design, implementation, and
lll iterative refinement. Closely related to these activities is the evaluation plan for an interaction technique. As we proceeded with the design of pre-screen projection, we began to see its large design space of possible attributes. Each attribute could have several (and sometimes many) values, so combinatorics of the numerous design decisions became very large very fast. This meant that early design decisions must be made based on very rapid prototyping and a quick, informal evaluation cycle, rather than on fuller experimental evaluation. Especially interesting design challenges were provided by two attributes: scaling and fade in/out. For both attributes, three types of objects in the scene for the command and control application we chose (described later) could potentially be affected: a map, military icons, and descriptive text. Virtual Scene
\NN
Physica
Figure 2. Configuration of pre-screen projection. Scaling is an attribute that specifies which objects in the scene appear larger or smaller as a user moves the head toward or away from the screen. In all design iterations, the map scaled, because it is the predominant spatial context for icons and text. In early designs, icons did not scale (to remove any implication that their size has meaning, which it does not in our application). However, users found it difficult to zoom in on non-scaling icons, so we changed the design so that icons do scale. We also chose not to scale text associated with the icons, to maximize the amount of readable text presented at any time. Users were comfortable with a final design in which the map and icons scale, but text does not. Further, we found that pre-screen projection is good for zooming only by a factor of about 10 when using real world linear perspective ( l / z , where z is the distance from a user's head to the virtual scene in front of the screen). To overcome this, we added amplified zooming that takes effect when the user is approximately 54 inches from the screen. Fade in~out is an attribute that specifies the gradual appearance and disappearance (progressive disclosure) of objects in a scene as a user moves the head. In all design iterations, the map and icons scaled but did not fade i n / o u t , because of their role in maintaining spatial context. We had text fading i n / o u t at several different levels of disclosure, based on the distance of the user's head from the physical screen, in order to display as much detailed information (in the text) as possible. In early designs, text at one level faded out completely as different text from the next level replaced it. However, users found it hard to determine which level they were in as they looked for specific information. So we changed the design to fade in only additional text, beneath (rather than on top of the location of) text displayed at prior levels. Fade-in/out of text provided some other surprises. At first text was too densely packed, even after a user had zoomed quite far into the scene. Making what we thought was a reasonable adjustment to that problem resulted in the text being too spread out. Text at first was displayed 75% opaque, but this proved to be too low contrast to read easily and also led to incomprehensible clutter when icons were close
112 enough together so that their associated text overlapped. Again, the combinatorics of alternative designs at low levels of detail (e.g., color, font, spacing, icon shape) were huge. Frankly, many of our final decisions (e.g., distance for amplified zooming, distances for levels of disclosure) were based on trial and error, again supporting the need for a fast, highly iterative development methodology.
2.2. Setting User Tasks in an Appropriate Application Our intent was to incorporate pre-screen projection into an application that would provide a realistic situation for its evaluation. This application incorporates interaction technique(s) as used to perform specific tasks, and is the context within which experimentation is performed. The application can be a real world one, or it can be a testbed or simulation developed specifically for purposes of interaction technique development. Panning and zooming are inherently spatial, navigational information-seeking activities; that is, a user navigates through an information space. To design and evaluate pre-screen projection, we wanted to create user tasks that would place heavy demands on a user's ability to pan and zoom. Thus, a critical aspect of our research was the co-evolution of appropriate user tasks along with the interaction technique, as indicated in Figure 1, to produce tasks for evaluating pre-screen projection, rather than for producing the best possible application. This approach re-emphasizes the definition of an interaction technique as a means of performing tasks. Prior research at the Naval Research Laboratory's HumanComputer Interaction Laboratory [1, 4] used rather simplistic, non-military, low-fidelity domains and tasks. Because we wanted our work to be relevant to specific Naval applications, we chose Naval command and control systems (hereafter called C 2 systems) as a rich, realistic application domain for our interaction technique development and evaluation. These systems, simply explained, support the planning, coordination, and execution of a military mission. We incorporated pre-screen projection into a C2-1ike testbed and created task scenarios for evaluation. We are using this testbed and tasks to evaluate and compare many different interaction techniques for panning and zooming, especially in time-critical situations, not just pre-screen projection. We chose to develop our own simplified application, rather than using or modifying an existing C 2 system, because of the complexity of such systems, and because much of the software behind them has nothing to do with the user interface. C 2 is a highly diverse, extremely rich and demanding application area with a breadth and depth of tasks that its users perform. We performed a user task analysis of such systems, to know the design space from which to select possible tasks. Participants to which we have access for evaluation are, in general, civilians with little or no military background. Thus, selecting and designing specific tasks to evaluate pre-screen projection that were rich enough, yet were unbiased and simple enough for users to learn quickly, was a crucial issue. The interaction technique and goals for its evaluation should drive task development, but in reality they are very closely coupled. We often found that simple user tasks worked best for evaluation of our techniques, because they tend not to confound learnability and other issues of the application itself with the issues of evaluation of the interaction technique(s). We chose a set of defensive engagement tasks, in which friendly, enemy, unknown, and neutral military objects are displayed on a map. We developed several scenarios in which groups of friendly ships and planes remain constant from the start of the scenario, but additional enemies, unknowns, and neutrals appear during the course of the scenario. Using the testbed application, the user's goals (tasks) are to acquire and maintain an awareness of an evolving situational picture, and to appropriately allocate weapons from friendly ships to encroaching enemy planes. Specifically, a user monitors the scene, largely by panning, looking for threats. When one is observed, the user then determines the number of planes in
!13 that threat (by zooming in on textual details that are progressively disclosed as the user's head moves closer to the screen) and the number of missiles the friendly has available to fire at the enemy. The user then uses a slider (not pre-screen projection) to enter the number of missiles to shoot at the enemy and fire those missiles.
2.3. Performing Empirical Evaluations As emphasized, much of developing a new interaction technique is evaluation. The first version (prototype) of pre-screen projection simply moved a couple of objects around on a solid background. This was fine for debugging the implementation, but gave us little information about how pre-screen projection affects user performance. Two kinds of evaluation are most appropriate for interaction techniques: formative and summative [5]. Formative evaluation is observational evaluation with users early and continually throughout user interface d e v e l o p m e n t - - in this case, of a specific interaction technique m with both qualitative and quantitative results. Summative evaluation is experimental evaluation with users comparing two or more interaction techniques for performing the same user tasks, with primarily quantitative results. We have performed numerous cycles of formative evaluation m some as short as five minutes, others lasting nearly an hour. Evolution of the design of scaling and fade i n / o u t of text and graphics, described previously, as well as virtually all other decisions about design details, came from many rounds of formative evaluation. The next step in our research is to perform summative evaluation (see summary section). Both formative and summative evaluation need to assess two key aspects of any interaction technique: cognitive/perceptual (mental) issues and articulatory (motor) issues [6]. A user's ability to physically interact with the technique should be evaluated first. If motor demands of a technique are such that a user cannot manipulate it, then whether that technique is useful at a higher, mental level of task performance is meaningless. To determine motor issues surrounding pre-screen projection, we had users perform a series of simple information-seeking tasks, in order to learn to use the technique, before we asked them to perform the more cognitively difficult defensive engagement task. These motor tasks also help a user learn to manipulate a new interaction technique, before moving into the more complicated cognitive tasks.
3. RELATED WORK Most work on interaction techniques typically reports only about the technique itself, with little or no mention of its development process and especially little or no evaluation. Some notable exceptions of works that discuss evaluation of interaction techniques are generally about predominantly summative evaluations, and include fish tank virtual reality [7], the alphaslider [8], the Toolglass [9], marking menus [6], and eye tracking [10]. Few papers discuss formative evaluation of interaction techniques, and we are aware of no papers that present a methodology for developing interaction techniques.
4. S U M M A R Y & F U T U R E In summative evaluations, we are comparing user performance and satisfaction using pre-screen projection to other interaction techniques for panning and zooming, using the defensive engagement task. We intend to provide, by these evaluations, baseline metrics for performing various kinds of tasks for panning and zooming. These baseline metrics can then be used to comparatively evaluate other new interaction techniques for panning and zooming. Without such baselines, the cost of interaction technique evaluations is too great,
114 because every new technique would have to be compared to every existing technique - - a prohibitive exercise. We have discussed a methodology for developing new interaction techniques that focuses not merely on their creation and coding, but also focuses on their effectiveness and efficiency for users. We have found that it is almost impossible to set development (creation and evaluation) of new, novel interaction techniques such as pre-screen projection into a structured software engineering environment (at least at a micro level); developing a new interaction technique is a form of invention and evolution that does not readily conform to current software engineering practices. It is a process of scientific discovery, rather than pure engineering. Our best guesses about the design of pre-screen projection, and other interaction techniques we have developed over the years, were substantiated or refuted by many tight, short cycles of formative evaluation with users performing realistic tasks in a testbed application. The many difficult issues and resulting unexpected design decisions could only have been encountered and resolved using a development methodology, such as we have described, that supports this kind of fast, effective iteration. ACKNOWLEDGMENTS Dr. Rob Jacob and Linda Sibert contributed many excellent ideas to our interaction technique research and methodology. This work is sponsored by the Decision Support Technology block (RL2C) within the ONR Exploratory Development Program, and is managed by Dr. Elizabeth Wald. REFERENCES 1. Jacob, R.J.K. (1991). The Use of Eye Movements in Human-Computer Interaction Techniques: What You Look At Is What You Get, ACM Trans. Info. Sys., 9(3), 152-169. 2. Hix, D., J.N. Templeman, and R.J.K. Jacob. (1995) Pre-Screen Projection: From Concept to Testing of a New Interaction Technique. Proc. CHI'95 Conf. 3. Gibson, J.J. (1986). The Ecological Approach to Visual Perception. Lawrence Erlbaum Associates, Hillsdale, NJ. 4. Jacob, R.J.K., and L.E. Sibert. (1992). The Perceptual Structure of Multidimensional Input Device Selection. Proc. CHI'92 Conf., 211-218. 5. Hix, D., and H.R. Hartson. (1993). Developing User Interfaces: Ensuring Usability through Product & Process. John Wiley & Sons, Inc., New York, NY. 6. Kurtenbach, G., A. Sellen, and W. Buxton. (1993). An Empirical Evaluation of Some Articulatory & Cognitive Aspects of Marking Menus. Human-Computer Interaction, 8, 1-23. 7. Ware, C., K. Arthur, and K.S. Booth. (1993). Fish Tank Virtual Reality. Proc. InterCHI'93 Conf., 37-41. 8. Ahlberg, C., and B. Shneiderman. (1994). The Alphaslider: A Compact and Rapid Selector. Proc. CHI'94 Conf., 365-371. 9. Kabbash, P., W. Buxton, and A. Sellen. (1994). Two-Handed Input in a Compound Task. Proc. CHI'94 Conf., 417-423. 10. Ware, C., and H.T. Mikaelian. (1987). An Evaluation of an Eye Tracker as a Device for Computer Input. Proc. CHI+GI'87 Conf., 183-188.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
115
B a s i c M o d e l s for U s e r I n t e r f a c e Design" Tasks, Users, Data, and I n t e r a c t i o n Devices
Chris Stary TAU-Group at the Department for Information Systems University of Technology Vienna, Paniglgasse 16, A - 1040 Vienna In this paper a representation scheme for the specification of the users of an interactive system and its functionality in a declarative as well as procedural way is introduced. According to this scheme a design methodology is given that does not end with the isolated treatment of interface components but rather provides a sound integration strategy. 1. T H E P R O B L E M
For interface design several factors have to be considered, as given in e.g. [1]" • the users of an interactive system, i.e. all those individuals accomplishing tasks by manipulating interaction devices and problem domain data at the user interface of a computer system. • the task-specific functionality of the operations on problem domain data as well as interaction devices, including a procedural specification. In order to bridge the gap between the control flow on application data and interaction devices several proposals have been developed, such as starting system design with a separated data model of the problem domain, e.g. [2]. Unfortunately, none of these approaches has led to a representation of the control knowledge required for the integration of knowledge about the users, their perception of tasks, problem domain data and interaction devices. The situation will not change, as long as specification techniques will not provide an integrated representation of the knowledge mentioned above at the semantical design level. For instance, this evidence has been shown for object-oriented specification [3]. In order to provide support for task- and user-oriented interface design, designers have not only to be provided with a representation scheme (to know how to represent the results from task- and user-analysis), but also with a methodology (to know what kind of activities to perform in the design process itself). In the following we provide an object-oriented concept for the integrated representation of the results of task- and user-analysis as well as generic steps to follow in order to complete a specification by handling and relating the addressed interface components. In contrast to existing approaches the approach captures all relevant knowledge for interface development, since it takes into account the actual context of user interfaces explicitly - tasks and users. Another difference to most of the existing development paradigms is the top-down strategy that allows for stepwise refinement and decomposition in a natural way.
116 2. THE T A D E U S - A P P R O A C H
In the TADEUS (Task Analysis / Design / End User Systems)-approach we put the following components into mutual context: task models, data models, user models and interaction models. Task models comprise the decomposition of end user tasks according to the economic and the social organization of work. Data models provide the static and dynamic information about the functionality of the system that has to be derived from a task model. Interaction models capture all devices and styles that are required for users in the course of interaction. All of the presentation- and manipulation-issues concerning tasks, functions, and data structures have to be specified and refined in an interaction model. User models detail individual completion of tasks as well as individual features for the manipulation of data structures and interaction devices. User models take into account personal experiences and preferences, personal access modalities to tasks and data, individual task organization, and social conventions at the work place. Since we propose to use a unifying object-oriented scheme and notation, stemming from Object-Oriented Systems Analysis as proposed in [4], the knowledge about what is going on in a system, and how tasks are accomplished using an interactive system can be kept encapsulated throughout analysis, design and implementation. Moreover, since the design is specified in an object-oriented notation, code can automatically be generated according to the models and the integration steps that have to be performed in the design phase. We will use the following sample problem domain: An airline agent has to be supported when h a n d l i n g customer requests for flights, composed of dates, destinations, etc. The airline agent tries to find an appropriate flight for the acquired options: the request data are matched against a flight database containing the available flights. If one or more flights match the request data, the airline agent offers them to the customer. If a flight is accepted by the customer, it is going to be booked - the airline agent issues a ticket for the selected flight. 2.1 T a s k M o d e l i n g
The starting point for design is a proper representation of the knowledge acquired through task- and user-analysis. It leads to the task model depicted in Figure 1. In the ORM (Object-Relationship Model) rectangles are symbols for classes, triangle relationships denote superclass-subclass relationships. Figure 1 shows the static part of a task model for the example. It contains the object classes 'FlightRequest' and 'TicketRequest' as subclasses of 'Request'. Figure 2 shows a part of the dynamic specification by a state net diagram, i.e. the Object-Behaviour Diagram OBM of the task model for the example. States are represented by rounded rectangles with a state name. Each state-net diagram is related to a class by putting the diagram into a tagged rectangle. Transitions are also represented as rectangles, but are always located between states. The top section contains a trigger description. A trigger provides the condition that, if met, causes the transition to fire. The bottom section of the transition triangle contains the actions that is performed when the transition fires. An action may be composed of several operations.
117
I, A O E N T handles]
...... ........... utters
FLIGHT REQUEST Figure 1: ORM of the Task Model
AGENT(idle)
@customer call~ or enters office load flight dbase
(~aflightdbase e x i s t s nd options exi~-] et paper versioO ~ nable~ o load~
fmatch"~ k, options~
Figure 2: A Part of the OBM of the Task Model 2.2 Data Modeling
The data model is a structured set of data (e.g. flight request) and elementary operations (e.g. match) in the work domain (in our case flight reservation) on which user tasks, such as searching for a flight, are operating. Due to space limits the ORMs ('Flight', 'Request' ,'Ticket') and the OBMs have to be omitted.
118
2.3 Interaction Modeling The interaction model is composed of static and dynamic specifications concerning dialogue devices. It also comprises particular interaction styles integrating several interaction devices. For example, direct manipulation can be achieved by combining windows, menus, icons, a mouse, and a keyboard. The control flow is defined through window and icon manipulation by selecting menu items using mouse clicks and/or key strokes. The abstraction from interaction devices to styles allows to specify user- and task-specific interaction.
is assigned to Figure 3: A Part of the ORM for the Assignment of Data of the Problem Domain to the Interaction Domain Besides generic devices and styles in this step tasks and problem domain data have to be assigned to those devices and styles. Figure 3 shows the part of the ORM where the designer specifies that flight requests are presented in forms to airline agents. Flight options are assigned to fields of the corresponding form. The result of this step is a customized interaction model that covers all task- and data-specific devices, such as flight-request form and ticket form.
2.4 User Modeling For the specification of user groups the organization of tasks has to be considered again. For the tasks at hand the agent represents the only user group that has to be considered for refinement. Customers, the second type of persons, do not get in touch with the interactive system directly, they rather deliver the data for acquiring flight options and issuing tickets. The other candidates for user groups, namely managers and clerks, are not involved in handling customer requests.
119
is able to change password
Figure 4: A Part of the ORM of the User Model Figure 4 shows the ORM for the user group 'Agent'. The specification shown comprises the modalities to access problem domain data, e.g. the airline agent is not allowed to change flight data in the database, as well as features to design an individual work space, such as setting the layout of flight information.
2.5 Application Behaviour Modeling When discussing the interaction model and the user model we did not deal with the dynamic integration. We will do that in this section when we discuss the integration of control flows, in order to support the designer in providing an impression about the overall behaviour of the specified system. In general, two issues for dynamical integration have to be taken into account: • the synchronization of operations on the interaction devices with the operations on the problem domain data. • the work space of each user group, i.e. all tasks and operations including the degree of freedom to rearrange tasks and the presentation of problem domain data. In order to demonstrate the second issue for behaviour modeling Figure 5 shows a part of the activity space for agents in an OBM. The specification integrates all task- and user-related features provided for airline agents handling flight- and ticket requests. It subsumes elements from the task-, user-, data- and the user model. After having specified the overall behaviour of the interactive system and the work spaces for all user groups, the functional specification can be started.
120
@customer call~ or enters office
AGENT
oad flight dbase @ idle
set
@manager sick manage flight,,
Figure 5: A Part of the OBM of the Agent's Activity Space 3. C O N C L U S I O N S In becoming aware user interface designers are still struggling for a structured representation that takes into account all knowledge relevant for task- and useroriented user interface development, an integration scheme for the specification of users and its functionality in a declarative as well as procedural way has been introduced. The related design methodology does not end with the isolated treatment of interface components but rather provides a sound integration strategy for global behaviour modeling. These achievements yield in the TADEUSenvironment that is currently being implemented to allow designers to specify the acquired knowledge, comprising tasks, users and their refinements. In addition, it will support the automatic generation of application prototypes.
References
[1] Johnson, P. Human-Computer Interaction, McGraw Hill, London, 1992. [2] Janssen, Ch., Weisbecker, A., Ziegler, J. Generating User Interfaces from Data Models and Dialogue Net Specifications, in Proc. INTERCHI'93, ACM/IFIP, 1993, pp. 418-423. [3] McDaniel, S., Olson, G.M., Olson, J.S. Methods in Search of MethodologyCombining HCI and Object-Orientation: A Case, in Proc. CHI'94, ACM, 1994. [4] Embley, D.W., Kurtz, B.D., Woodfield, S.N. Object-Oriented Systems Analysis. A Model-Driven Approach. Yourdon Press, Englewood Cliffs, 1992.
Ill.6
Screen Design 1
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
123
T h e effects of r e a l i s t i c v e r s u s u n r e a l i s t i c d e s k t o p i n t e r f a c e d e s i g n s on novice a n d e x p e r t u s e r s L. A. Miller ~ and K. M. Stanney b a Department of Psychology, University of Central Florida, PO Box 25000, Orlando, FL 32816 bDepartment of Industrial Engineering and Management Systems, University of Central Florida, PO Box 25000, Orlando, FL 32816
Metaphors are one tool which designers have used to bridge the gap between technology and the novice user. While metaphors work in theory, often the metaphor falls apart or breaks down during seemingly logical procedures. It would stand to reason that a metaphor which approximates the humanenvironment interaction, and remains consistent in the analogy, would be a more valid test of the benefits of a metaphor. In this study, novice and expert subjects were given a set of four computer-based, editing tasks, each of which was composed of six subtasks. The dependent variable was time to complete the task. It was found that novice subjects significantly benefited from the use of the realistic desktop design, while there were no significant differences in the performance of expert users. 1. I N T R O D U C T I O N Technological advances in computer hardware and software have brought faster systems, larger memories, bigger hard drives, and a plethora of different applications in which to use these innovations. Yet despite these great advances in technology, the designer's knowledge and understanding of the computer system has not really changed (Booth, 1989). It is evident that more natural and convenient means for users and computers to exchange information are needed (Hartson & Boehm-Davis, 1993; Jacob, Leggett, Myers, & Pausch, 1993). Recently, many designers have been trying to make computers and their applications "user friendly." "User friendly" is a term which means that the computer can interact with a human much the way humans interact with each other (Eberts, 1994). While this design paradigm provides insight into how the
124 usability of a system could be enhanced, it is limited in its scope. This paradigm should be extended by stating that the computer should interact with a human much the way humans interact with each other and their environment. If today's software programs were similar to human-human communication and human-environment interaction, it is possible that we would not have so many individuals who feel uncomfortable, or even frightened, using the computer. A mental model often helps novice computer users understand the relationships and functions of target systems. A mental model works because it is set up to parallel the model of the system it emulates. According to JohnsonLaird (1980, as cited in Rasmussen, 1990, p.44), "A natural mental model of discourse has a structure that corresponds directly to the structure of the state of affairs that the discourse describes." A common means of aiding the formation of this mental model is to incorporate a metaphor into the computer design. Metaphors aid the user in understanding the capabilities of the computer system and the relationships between its components by showing that the system "acts like" something the user is more familiar with. Although this practice works in theory, often the metaphor 13teaks down" during seemingly logical procedures. The purpose of this study was to compare a realistic desktop design to an unrealistic design, such as the Windows version. A realistic design is defined here as one that preserves the critical relationships that exist for human-human communication and human-environment interaction. It was hypothesized that there would be no significant difference between the desktop interfaces for expert users; however, novice users were expected to perform significantly better using the realistic desktop. The objectives of this paper are (1) determine ff there is a substantial difference between the realistic and unrealistic desktop interfaces as mapped through the GOMS method, and (2) to investigate the effects of the two interface designs on performance times in experts and novices. 3. M E T H O D 3.1 S u b j e c t s Twenty subjects (10 computer experts and 10 novices) were recruited from engineering and psychology courses. They were given course credit for their participation. The subject's computer experience was first determined and then they were tested on both the realistic and unrealistic desktop designs. 3.2 M a t e r i a l s The materials consisted of a nonrealistic desktop interface (a Windows-like environment) and a realistic desktop interface. The realistic design looked much like an office would; it included representations of a desk, phone, file cabinet,
125 Rolodex, printer, trash can, and calendar. The interfaces were displayed on an IBM-compatible 486-DX66. A mouse and keyboard were provided to manipulate the icons on the screen and enter data. The computer recorded the amount of time it took to complete each task.
3.3 GOMS Analysis Before building the prototype, a predictive analysis was conducted on both the realistic desktop and unrealistic desktop interfaces to determine where in the analogy the metaphor broke from the physical world. The four tasks studied were: finding a file, deleting a file, printing a file, and telephoning a client. Applications in both the realistic and unrealistic design interfaces were mapped to real-world tasks by the GOMS method to determine how closely each of the interfaces assimilates the actual procedures an individual would use in real-life (Card, Moran, & Newell, 1983). For example, the physical task of dialing a phone might involve: • looking up the phone n u m b e r in the rolodex by t h u m b i n g through the cards • finding the correct n u m b e r • picking up the receiver • dialing the number. In comparison, the realistic desktop involves: • looking up the phone n u m b e r in the rolodex by clicking the cardtMe and flipping through the cards • finding the correct n u m b e r • pressing a "dial" button on the card. While the unrealistic desktop involves: • opening the accessories window • double-clicking the cardfile icon • clicking on the menu • clicking on in the File menu • double-clicking the correct cardffle entry (you can conceivably have more t h a n one cardfile) • flipping through the rolodex cards • finding the correct phone n u m b e r • opening up the terminal application (also in Accessories) • arranging the windows so t h a t both the cardtMe and terminal applications can be read (or rely on memorization of the phone number) • clicking the <Settings> option • clicking • reading the phone n u m b e r in the cardtMe application
126 •
typing the number into the dial dialogue box in the terminal application • pressing enter. As seen in this example, the realistic desktop interface is better able to preserve the critical relationships of the real-life task. The realistic metaphor maintained the first two procedures of the real-life procedure and then broke from the metaphor on the third. While the original procedures (i.e., picking up the receiver and dialing the number) could have been maintained, a cost-benefit analysis clearly indicated that breaking from the metaphor was beneficial. The break required users to press one button (i.e., the "Dial" button) rather than keying through the entire phone number. The cost (i.e., the break in the metaphor) was out-weighed by the benefit (i.e., fewer keystrokes and an intuitive alternative). The realistic metaphor is more consistent, however, than the unrealistic design which used embedded windows and dialog boxes behind the front end screen. Using the GOMS method it was determined that for every task performed (finding a file, deleting a file, printing a file, and telephoning a client), the unrealistic desktop interface broke from the metaphor at an earlier stage than the realistic design. 3.4 P r o c e d u r e Each subject participated in one two-hour session. Subjects were asked to fill out a demographic page and the Windows Computer Experience Questionnaire (WCEQ). The WCEQ was based on input from a subject matter expert and developed particularly for this study. The results from the WCEQ were used to assign subjects to either the novice or expert subject group. Novice and expert subjects were then given a set of four tasks to accomplish on both the realistic and unrealistic desktops. T h e s e tasks included finding a file, deleting a file, printing a file, and dialing a number. Subjects were instructed to engage in each of these tasks six times (i.e., finding six different files), for a total of 24 trials. To eliminate the effects of interface presentation, half the subjects were first tested on the realistic design, while the other half were first tested on the unrealistic desktop design. The order in which these occurred was random. The dependent variable was time to complete the task. The computer recorded the data. 4. R E S U L T S
An ANOVA revealed a significant interaction between the times to perform on each of the interfaces and expertise level (F (1.18)=10.7, p<.004). As hypothesized, novice users performed significantly faster (F(1,~s) =14.27, p<0.0014) with the
127 n
realistic design (X = 681.0 seconds;S.D. = 217.64) than with the unrealistic design (X = 1395.1 seconds; S . D . - 556.85). Novices were on average 60% faster using the realistic design. As predicted, there were no significant differences detected for expert users (F(1,1a) =2.29, p>. 15) between the realistic design (X=509.4 seconds; S.D. = 182.86)and the unrealistic design (X=663.6 seconds; S.D. - 265.32). In addition, while experts were significantly faster (F(1,~a) =14.06, p<0.002) than novices on the unrealistic design, there were no significant differences detected between novice and expert users on the realistic design (F(~,~a) =3.64, p>0.07). Experts were 52.4% faster than novices on the unrealistic design. 5. D I S C U S S I O N
The results of the GOMS preanalyses showed that the realistic desktop interface design was mapped to selected real-world tasks with greater consistency than the unrealistic desktop interface. Therefore, the model predicted that the realistic desktop would be more usable than the unrealistic desktop because it is more consistent with human-environment relationships. This prediction was supported by the empirical evidence, which showed that novices performed significantly faster on the realistic design as compared to the unrealistic interface. These results suggest that mapping human-environment relationships to the user interface is an effective means by which to accommodate novice computer users. The key to successful accommodation for novices was maintaining the critical relationships between the physical reality and the computer representation of a task. Unlike existing desktop metaphors that immediately break from reality, port users into embedded menu hierarchies, and force users to communicate via dialog boxes, the realistic metaphor design maintains the critical processes and procedures people are accustomed to using in their daily lives. The results of this study indicate that for novice users performance enhancements can be achieved through such mapping. Additionally, the results suggest that such designs are also appropriate for experts since their performance was not hindered when using the realistic design. In the future, it is recommended that researchers consider the importance of mapping interfaces to h u m a n - h u m a n and human-environment relationships. An additional futuristic step may even incorporate a different type of interface altogether. According to Nielson (1993, p83), "The next generation of UIs may move beyond the standard windows, icons, menus, and pointing device (WIMP) paradigm to involve elements such as virtual realities, head-monitored displays, sound and speech, pen and gesture, recognition, animation and multimedia,
128 limited artificial intelligence, and highly portable computers with cellular or other wireless communication capabilities." If designed improperly, such multimodal interfaces may prove to be very challenging for people to utilize. For such complex and involving interfaces, the mapping of critical human-human and human-environment relationships may become essential to maintain sufficient usability. One thing seems clear, however, technology is being developed faster than we can train individuals to use the product. In the future, greater emphasis will thus need to be placed on the environment from whence automated tasks are derived in order to provide an intuitive mapping between the physical reality and the computer representation. REFERENCES
Booth, P.A. (1989). An introduction to human-computer interaction. London" Lawrence Erlbaum Associates. Card, S. K., Moran, T. P., & Newell, A. L. (1983). The psychology of human computer interaction. Hillsdale, NJ: Erlbaum. Eberts, R. E. (1994). User interface design. Englewood Cliffs: NJ: Prentice Hall. Hartson, H. R., & Boehm-Davis, D. (1993). User interface development processes and methodologies. Behaviour and Information Technology, 12, 98114. Jacob, R. J. K., Leggett, J. J., Myers, B. A., & Pausch, R. (1993). Interaction styles and input/output devices. Behaviour and Information Technology, 12, 69-79. Marcus, A. (1993, April). Human communications issues in advanced UIs. The next generation GUIs. Communications of the ACM, 36, 101-109. Morse, A., & Reynolds, G. (1993, April). Overcoming current growth limits in UI development. The next generation GUIs. Communications of the ACM, 36, 74-81. Nielson, J. (1993, April). Noncommand user interfaces. The next generation GUIs. Communications of the ACM, 36, 83-99. Rasmussen, J. (1990). Mental models and the control of action in complex environments. In D. Ackermann & M. J. Tauber (Eds.), Mental models and human-computer interaction I (pp. 41-69). Noah-Holland: Elsevier Science Publishers.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) 1995 Elsevier Science B.V.
129
Rendering Calligraphy Words with 'Kasure' Variations
Qinglian Guo Sony Systems Design Corporation email:[email protected] 1. Introduction Recently, there has been a remarkable increase of interest in computer calligraphy systems, side by side with the publishing of new fonts. Research covers many aspects of DTPS (Desk Top Publishing System), for example, the construction balance of calligraphy word [4] and the representation of the radius of outline fonts[2]. The systems supported by these research provided many benefits to people who wants to create good formation calligraphy words. However, professional people, who has a knowledge about calligraphy, are not satisfied with these systems, which can only fill the inside of outlined word uniformly in the same color, and its impossible to represent the rendering effect of brush. When moving a brush to draw a calligraphy word, a scratchy breakup of ink may occur due to that there is no more ink remanning on the brush, the speed of moving brush is fast, or the pressure applied on the brush is not big enough. The breakup of ink results in some pixels un-occupied by ink, which form a irregular liner-formation texture on the stroke. We call this texture as "kasure". In modem calligraphy, kasure is often used to give strokes live and strength, to make a dynamic and impressive appearance. The fact that the existing systems cannot represent this rendering effect is a big drawback that make their output be un-natural and limited to the most low level. On the other hand, in the area of CG, there has been some reports on simulating the rendering effects of brush and other painting materials. The most notable research is Strassmann's "Hairy Brushes" [3]. His work provided a valuable basis for afterward research on modeling brush strokes. Since the function of brush is analyzed physically, his model results in realistic images that are rather close to what is drawn by using real brushes. Under this situation, we started an approach to apply Strasmmann's method to calligraphy words. The purpose is to include reality and flexibility to computer generated calligraphy words. Our calligraphy word rendering system enhances With more creative and exciting ways of generating your own calligraphy word. With this approach, we managed to sort out several problems, which have not been considered in previous research. At first, a new outline font is needed for providing the necessary information to the brush stroke rendering algorithm. It is also necessary to develop a flexible and efficient method of controlling kasure texture. Furthermore, an functional interface is most desired for supporting the operations of editing and rendering calligraphy word. In the following sections, we will present our methods for solving these problems.
130
2. A new outline font For existing outline fonts, a stroke may cover several elements that are generally called "stroke" in calligraphy. For representing "how a brush is moved when drawing a word", we proposed a new outline font named as "brush-stroke font". As show in fig. 1, the brush stroke font differs from exiting outline font in: (1) one of the outline points is specified as the starting point, which is used to illustrate the moving direction of brush when drawing the stroke; (2) each word could consists of several above strokes, and there is a sequence among these strokes; (3) Each stroke is defined just corresponding to the "stroke" of practical calligraphy. We have developed a font designing software for generating font data. It is possible to create ones' own handwriting font, and also to transform existing outline font to the brushstroke font. The transformation is difficult to be performed automatically, due to the font involves more data and information. Designers' operation is necessary to determining the data for the intersection area of strokes.
/
(a)
(b)
Fig. 1 The difference between "brush-stroke font" (a) and common outline font (b) 3. T h e d a t a used in rendering calligraphy words Following data are used in our rendering calculation: (1) Brush data: it represents the distribution of ink on a brush. It is supposed to have m elements, each corresponding a rendering trajectory on stroke. For each element, the value of ink quantity IQj and ink density IDj (j = 0, 1, . . . m ) are determined by some functions which models the commonly used ink distribution in calligraphy. (2) Stroke data: it is determined from our font data. Usually, a good balance is made when about 1/3 of the strokes composing a word have kasure texture. We used a method to repeat to select the longest one from the strokes that has not be assigned with kasure texture. (3) Kasure area data: as illustrated in fig.2(a), there are m trajectory traces coveting the area of stroke, these trajectories begin at the starting point, pass the stroke area and finish at ending point. The start position KSj and end position KEj (j = 0, 1, . . . m ) of kasure texture on each of the m trajectories are defined as kasure area data. More particularly, a succeeding list of un-rendered pixels on a trajectory is part of the kasure texture on a stroke. If the length of the stroke is 1.0, the kasure area data for this trajectory will change in range of [0.0, 1.0]. When there is no kasure texture on this trajectory, both KSj and KEj will be 1.0. Two buffers are used to record the kasure area data for all the trajectory traces. When KSj and KEj (j =
131 0, 1, ... m ) is known, the pixels un-occupied by ink can be determined during the process of filling the polygons which covering the area defined by the stroke data. (4) Kasure parameters: they are provided for users to control kasure texture through the interface. Seven kasure parameters are used: c 1: the average start position of kasure area for all the trajectories c 2: the average end position of kasure area for all the trajectories c3: the up (left) limit of kasure area along the direction of stroke width c 4: the down (fight) limit of kasure area along the direction of stroke width c 5: the range of a random variation of start positron around c 1 (it is necessary to make the kasure texture have the property of randomness.) c 6: the range of a random variation of end position around c 2 c 7: the density of kasure lines in the range I c4 -c3 I All the parameters vary in range of [0.0, 1.0]. As show in fig.2(b), kasure area data can be determined from these kasure parameters. When c7 is lower than 1.0, n = c7"1c4 e31*m trajectories are selected randomly and assigned to have kasure texture. When c 7 = 1.0, all the trajectories in the range le4-c31*m are supposed to have kasure texture. For each of these kasure trajectories, kasure area data KSj randomly selects a value in the range (c 1, c l +c5), and KEj in the range (c2, c 2 + c 6 ) .
7K
~ . . . - ZK \\
BK
Pe C,"t,
,
KS
\ PS ",,
PS
(a)
(b)
Fig.2 Rendering
trajectory traces and Kasure
area data (a), the relationship
between kasure parameters and kasure area data (b).
4. The methods of controlling kasure parameters An visual menu interface is developed to provide flexible and functional drawing environment to users. The interface is significant in that the rendering operation can be controlled in several ways. As show in fig.3, the most simple way for novices is to use the auto-rendering mode, or to select a rendering texture from the imaged icons. For the people who are used to the system, mouse-adjustment-menu can be used to intentionally control kasure textures. Besides the special rendering menus, it has also standard editing functions. The interface provides an easy, flexible, and creative drawing environment to the users.
Pe
132
: ~ - " . : ~ _ .
Fig.3 The design of the menus of our human interface
The auto-rendering mode is designed to automatically specify kasure parameters based on stroke formation features. Kasure is realized related with stroke formation in great deal. From stroke data, it is easy to calculate the stroke formation feature data, such as stroke length, variation of stroke width, variation of stroke curvature, and the number of positions where the curvature of stroke is lager enough to cause a change in stroke laying direction (we call these positions as turning positions). By using these features for measurement, we classified the strokes of calligraphy words into 20 types based on statistical investigations and studies on calligraphy education books. For each type, its most usually used kasure pattern is designed and the corresponding kasure parameter values provided in the system. In fig.4, some examples are given to show that strokes of different formation can be rendered with different types of kasure texture. 5. T h e process of rendering calligraphy word For rendering a calligraphy word, following process is carried out for each stroke: (1) Specify brush data (2) Input stroke data (3) Specify kasure parameters (4) Calculate kasure area data from kasure parameters (5) Repeat following rendering process for all the polygons which compose a stroke: (a) calculate the distance of ith polygon from stroke starting point Li
133 (b) for each the m trajectories, if KSj < Li < KEj, assign the pixels, which is on this trajectory and within ith polygon, to be ink un-occupied pixels. (c) scan-line search in order to find out all the pixels in this polygon. For a pixel that is occupied by ink, its image intensity I is determined from current brush data IQj and IDj. For a pixel that is assigned ink unoccupied, its intensity is determined as 1.0, referring white color. Here, we made a little extent to Strassmann's stroke rendering model. In stead of assigning the pixel of kasure texture with totally white color, we used an intensity value that varies according to the locally fiber density of painting paper. The painting paper data was constructed for simulating the diffusion of ink in paper [ 1]. (d) change the brush data to represent the decrease of ink quantity. When ink quantity decrease to zero, the afterwards polygons will have kasure texture. For representing more than one kasure textures on a single stroke, it is only need to use several pares of kasure area data buffers and comparing of (b) is also required performed every pare of buffers.
Fig.4 Examples of strokes which have different formation and are rendered with different types of kasure texture
134
6. Conclusions The system is implemented in C and runs at interactive speed on an Sony news workstation, without and additional hardware assistance. The system has proven quite successful at assisting users in easily producing a variety of calligraphy words. An example is presented in fig.5. Although the word draw by the artist may be finer than ours, however, when we consider that our painting were made on a computer by a programmer who has no trained on calligraphy, and in a few minutes for rendering each stroke, we find out results very encouraging. The present interface is weak in input device. We desire to have an more accessible, sensitive, and multi-degree input device. By taking the advantage of input divide, more sophisticated method for control kasure texture is possible. This tendency, taken to an extreme, is virtual reality, where the user abandon s the real world to be completely surrounded by the computer. It is believe the all constraints of the real world can be overcome in VR, and physical tools can be made obsolete by more flexible, virtual alternatives.
Fig.5 An example of computer generated calligraphy word - "dream" References [1] Q. Guo and T.L. Kunii, "Modeling diffuse paintings of 'Nijimi", Proceedings of the IFIP WG 5.10 Working Conference, Tokyo, Japan, 329-338 (1991). [2] M. Kim, E. Park, and S. Lim, "Approximation of variable-radius offset curves and its application to Bezier brush-stroke design", 1993, Butterworth-Heinemann Ltd Computer-Aided Design, Volume 25, Number 11, November 1993. [3] S. Strassmann, "Hairy brushes", Computer Graphics, 20, 4, August 1986, 225-232. [4] T. Yamasaki and H. Kinoda, "A computer formation of brush-written Kanji characters based on the brush-touch handwriting", Japan densi-jyoho-tusin-gakai research reports, Vol. 92, No. 490, pp.l-8, 1993.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
135
D y n a m i c font: its c o n c e p t a n d g e n e r a t i o n m e t h o d K. Takayama ", H. Kano b, Y. Maeda a, K. Misue a, S. Hosogi ~, and K. Sugiyama ~ ~Institute for Social Information Science (ISIS), Fujitsu Laboratories Ltd., 140 Miyamoto, Numazu, Shizuoka 410-03, Japan b Department of Information Sciences, Faculty of Science and Engineering, Tokyo Denki University, Ishizaka, Hatoyama-machi, Hiki-gun, Saitama 350-03, Japan
What is a "dynamic font"? It is a living font. 1. I N T R O D U C T I O N With the desktop publishing (DTP) becoming popular, a variety of fonts of higher quality have been desirable as well as the tools for designing them. Various methods for generating fonts have been proposed and are classified into three categories: (i) dot matrix, (ii) outline vector, and (iii) skeleton vector font schemes. On the other hand, handwritten letters and characters have been generated originally by intersecting some writing implement with a plane and moving the implement continuously in both space and time. The designed patterns of these handwritten letters and characters are the typefaces. Despite the reformations, typefaces still remain their original spatial and temporal characteristics originated from the properties of handwriting movements. "Serives" in letters and "spatters" in Chinese characters are such examples. Under these circumstances, we propose a novel concept of "dynamic font" and its generation method [1]. A dynamic font is generated by intersecting some virtual writing implement with some virtual plane and moving the implement continuously in both space and time subject to the designed writing-motion. 2. D E F I N I T I O N OF M O T I O N A N D O P E R A T I O N S 2.1. Description of w r i t i n g - m o t i o n A writing-motion can be described by the time course of the configuration of the writing implement. Specifically, let pd(t) - [xa(t), ~_d(t)]T be a 6-dimensional vector denoting the writing-motion, where x_d(t) and ed(t) are 3-dimensional sub-vectors denoting respectively the position and orientation trajectories of the writing implement. Besides, let Bk(s) be the normalized uniform B-spline function of degree k (k _> 0) with the single knot points s = 0 , 1 , . . . , k + 1. The writing-motion of a writing implement pd(t) is then generated by m+k-1
p_d(t) -
~ i=l-k+l
diag(p~)lBa(c~(t-ti)).
(1)
136 Herein l and ra are integers satisfying l _< ra, P-/*-k+l"'"' P---=+k-1 are 6-dimensional constant vectors, diag(p~*) denotes the 6 × 6 diagonal constant matrix with the j-th component of p~* in j-th diagonal position, and I is a 6-dimensional constant vector with each component 1. Moreover, a is a positive constant scalar employed for scaling the variable s in Bk(s), and ti for any integers i denote time instants defined from a given tl+i aS t~ = t~+~ +
i-(/+1)
.
C¢
(2)
In (1), the functions 1 B k ( a ( t - t t _ ~ + l ) ) , . . . , 1 B k ( a ( t - - t m + k _ l ) ) are named "unit motions" [2]. Thus, the motions generated by (1) have the smoothness property of spline functions of degree k. Moreover, B-spline functions form a basis having the local and minimal support for the space of spline functions. Such a property may be effectively utilized for adjusting motions locally. 2.2. Definitions of o p e r a t i o n s on m o t i o n s For motions composed of unit motions, various operations on motions can be defined [2]. These include spatial ones such as scale(p_d(/), _c)
C_p_a(t),
-
(3)
_C =[diag(_C)o /0] , translate(p_d(t),b) o
=
pd(t) + p_b,
(4)
=
_O_p_e(t),
(5)
'
rotate(p~(t), 0_)
o = [ n__(0_)0] 0 /_ ' tilt(p_J(t), ~)
[o
=
_.pd(t)+ p__~,
(6)
where _c, _b, 0_, and /3 are 3-dimensional constant vectors; ~ and p~ are 6-dimensional constant vectors; R__(0) is a 3 × 3 constant matrix defined later in (18); and _C and O__are 6 × 6 constant matrices. Structural operations including joining and separating motions can also be defined. However, they can be described more concisely by their formal representations, and will be defined subsequently. 3. F O R M A L R E P R E S E N T A T I O N S
OF M O T I O N AND O P E R A T I O N S
3.1. F o r m a l r e p r e s e n t a t i o n of m o t i o n In the present scheme, any motion can be generated by specifying the weighting vectors for unit motions appropriately. Denote the concatenation of the weighting vectors for the motion defined by (1) as ~*--k+l • " "P* *'" r--J--1 P---I ~-
•
k-1
* Pm+l * " " "P* --re+k-1
P--m
•
k-1
J
=initial
Mfin~t
(7)
137 where M ( = ~ * . . . p ~ ) indicates the main part of the motion, ,nit,~,(= ~p*- k + l ... ~*--1) and finer( = P~+I " " P * + k - 1 ) indicate the supplemental parts of the motion for satisfying the initial and final boundary conditions respectively. Thus, the motion defined in (1) can be represented formally by the concatenation of the weighting vectors as in (7) [2]. 3.2. F o r m a l r e p r e s e n t a t i o n s of o p e r a t i o n s By the formal representation of motion, operations on motions can also be represented formally [2] as
(s)
scale(M, _c)
=
(_C_~*)-.. (Cp__~),
translate(M,b) rotate(M, 0_)
=
(p~ + P-b)''" (P--m+ P--b)' (O_p~*)... (_0_.P_~),
(10)
tilt(/,fl)
=
(p~ + p_Z).-. (_p~ + p_Z).
(11)
(9)
In addition, structural operations can be introduced as concatenate(M1, M2) - MI" M2 = M1M2 he~d(/,
~)
-
p__d*l " " "p__ml!,*2
=
(12) (13)
= p~ • • • p ~ + ~ _ , ,
tail(M, r)
" " "!m2,
Pm_r+l'''Prn,
(14)
where M1 - ~*~ " " "P~I' M2 - q_/*2• • .q~:, and r is an integer satisfying 1 _< r _< r n - l + 1. It is noted that the formal representation of motion (7) can be regarded geometrically as a polygonal line called "control polygon" [2]. Operations on motions can then be regarded as operations on control polygons [2]. We therefore see that the problem of generating motions reduces to the one of specifying appropriate control polygons with operations on control polygons. 4. M O D E L S
OF IMPLEMENT
AND PLANE
4.1. M o d e l of writing implement Let the coordinate system fixed to a writing implement be PO - P X P l / P Z . Denote a coordinate on the coordinate system as Pz__ - [Px,Py,Pz]Y. As an example of a writing implement, we assume here a virtual elliptic cone, which can be described as
+(;)
-(,z) :,
(15)
where ~" and 7} are constant scalars. 4.2. M o d e l of p l a n e for w r i t i n g on Let the coordinate system fixed to a plane for writing on be ° 0 - ° X ° Y ° Z . Denote a coordinate on the coordinate system as °x_ = [ ° x , ° y , ° z ] T . As an example of a plane for writing on, we assume here a simple planar plane, which can be described as °z-O.
(16)
138 4.3. R e l a t i o n b e t w e e n the c o o r d i n a t e s y s t e m s The coordinate represented as Px_ on the coordinate system P O - PX~YPZ, which is fixed to the virtual writing implement, is represented as below on the coordinate system °0- °X°Y°Z, which is fixed to the virtual plane for writing on. Namely, °x = R ( ¢ ) P x + x.
(17)
Herein x_ is a 3-dimensional translational vector, ¢ is a 3-dimensional rotational vector the Euler angles, and R(¢) is a 3 × 3 rotational matrix defined as
denoting
R(_¢) = R(¢, 0, ¢) =
-
rll r12 r13 ] r21 r22 r23 r31 r32 r33 cos ¢ cos 0 cos ¢ - sin ¢ sin ¢ sin ¢ cos 0 cos ¢ + cos ¢ sin ~b sin 0 cos ¢
5. G E N E R A T I O N
- cos ¢ cos 0 sin ¢ - sin ¢ cos ¢ - sin ¢ cos 0 sin ~b + cos ¢ cos ~b sin 0 sin
cos ¢ sin 0 ] sin ¢ sin 0 J . COS0 (18)
OF D Y N A M I C F O N T S
5.1. C o m p u t a t i o n a l a l g o r i t h m Specifically, the equation to get a dynamic font is derived by the following three steps: (i) Combine the model of the writing implement (15), that of the plane for writing on (16), and the relation between the coordinate systems (17) into a set of simultaneous equations; (ii) Solve the equations in terms of (°x,°y, Vz); and (iii) Again combine the resulting equations on (Ox,Oy,Pz) with the model of the writing-motion (1) and the equation on the coordinate rotation (18). Thus, the dynamic font is computed by a set of equations as
[ °x(t) ] [ {(~c°s)~)rii(t)+ (~sinA)ri2(t) + ri3(t)}pz(t) + x(t) ] °f(t) = °y(t) = {(~cos)~)r21(t) + (~sinA)r22(t) + r23(t)} Pz(t) T y(t) , °z(t)
Vz(t) =
(19)
o
1 - ( ( c o s A)r31(t) + (r/sin A)r32(t) + r33(t)
z(t) >_O,
(20)
together with (18) and (1). Herein ~ is a scalar. Notice that °f(t) is the 3-dimensional vector representing the shape of the desirable dynamic font. Consequently, the dynamic font is designed by the following three steps: (i) Specify a control polygon corresponding to the desirable font with the parameters related to the shape of the writing implement, i. e., ( and 77appeared in (19) and (20) (or originally in (15)); (ii) Derive the dynamic font by computing (1),(18),(20), and (19); and (iii) If you would like to modify the resulting font, change the control polygon by using some suitable operations on control polygons, and/or the parameters, then go back to the step (ii). 5.2. S i m u l a t i o n e x a m p l e s We then show some examples of the dynamic fonts. In each figure below, (a) shows the generated font, (b) the font and the corresponding control polygon shown in °O-°X°Y°Z
139 space, and (c) those shown on °0 - ° X ° Y plane. Figure 1 is an example of a cursive letter "a". Figure 2 shows concatenated fonts "ac", of which motion M~c is generated by using the operation as M~c = concatenate(M~, M~). Thus, in the present framework, various complex fonts including concatenated fonts like continuously handwritten letters or characters can be generated in an easy way. Figure 3 is an example of cursive Chinese character ,,ql,,. Figure 4 shows a rounded Gothic font "q~". Note that the control polygon shown in the ° 0 - °X°Y plane in the Figure 4 (c)is the same as that in Figure 3 (c). The control polygon of Figure 3 was designed referring to that of Figure 4. Thus, the framework presents a possibility to design various fonts of different typefaces in more easy manner. 6. C O N C L U S I O N S A new concept of "dynamic font" and its generation method [1] were proposed. The dynamic font was generated by intersecting some virtual writing implement with some virtual plane and moving the implement continuously in both space and time subject to the designed writing-motion. The writing-motion was defined by using the concept of "unit motions" [2] and this made possible a local and dynamic generation of motions and the fonts as if human wrote such fonts in real time [3]. In order to build various motions and the fonts, a notion of operations on motions [2] was also contained. They included spatial operations such as scaling, translating, rotating, and tilting a motion. Structural operations of joining two motions and separating a motion into two enabled to generate any sequence of continuously connected cursive fonts. The writing-motion was represented formally as a sequence of the weighting coefficients for unit motions [2]. The sequence formed a "control polygon" geometrically and was used effectively to design the motion as well as the dynamic font. Several simulation examples were demonstrated by using an elliptic cone and a simple planar plane respectively as examples of the virtual writing implement and the plane for writing on. The figures showed fonts of really different typefaces were generated from similar control polygons except the rates and timings of putting the implements up and/or down and the shapes of the implements. REFERENCES
1. K. Takayama, H. Kano, Y. Maeda, K. Misue, S. Hosogi, and K. Sugiyama, "Dynamic font: its concept and generation method," (in Japanese), in Proc. Graphics and CAD Syrup., IPSJ (Tokyo, Japan, Sep. 1994), pp. 181-190, also to be published in ISIS Research Report, Fujitsu Laboratories Ltd., 1995. 2. K. Takayama and H. Kano, "A new approach to synthesizing free motions of robotic manipulators based on a concept of unit motions," to be appeared in IEEE Trans. Syst., Man, Cybern., vol. 25, no. 3, 1995. 3. K. Takayama and H. Kano, "A mathematical interpretation on the organization of free motions of arm," (in Japanese), in Proc. 4th SICE Syrup. Biological and Physiological Engineering (Tokyo, Japan, Nov. 1989), pp. 291-294.
140
(~)
(~)
(b)
(b)
(c)
(c)
Figure 1. A cursive letter "a".
Figure 2. Concatenated fonts "ac".
(~)
(~)
(b)
(b)
(c)
(c)
Figure 3. A cursive Chinese character " ~ " .
Figure 4. A rounded Gothic font " ~ " .
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) 1995 Elsevier Science B.V.
141
A Spatial D a t a Structure for the 3D Graphical Facility M a n a g e m e n t S y s t e m Takashi Tamada, Teruhiko Teraoka, Minoru Maruyama and Shogo Nishida Information L: System Science Department, Central Research Laboratory, Mitsubishi Electric Corporation 1-1, Tsukaguchi-honmachi 8 chome, Amagasaki, Hyogo 661 JAPAN
ABSTRACT Geographical information systems (GISs) are now used extensively in the analysis of environmental data. due to their capability to manage, manipulate, and display spatial data. However, most of the GIS applications (such as facility management system) possess data management structures that deal with only 2 dimentional space. The 3D system that enables users to explore the 3D space interactively and examine 3D spatial views of the environments would be very useful. In this paper, we propose a spatial data management method for a 3D graphical facility management system. Using this method, we have developed prototype 3D graphical management system that offers interacive operations with the 3D virtual city environments.
1
INTRODUCTION
Geographical Information Systems have been used for land and facility management [1,2,3]. In conventional GIS applications, there are several significant problems. For example, GISs handle only two-dimensional data and the capability to support user interaction with the data is insufficient [2]. Recently much work has been done using 3D graphics to visualize geometric objects in the areas of virtual reality, scientific visualization and so on. The emerging technologies of 3D and interactive animation can be effectively exploited to improve management of and access to large information space [4]. And it seems plausible that 3D space can be used to maximize effective use of screen space. From these observations, we have attempted to develop a graphical facility management system utilizing 3D virtual environment instead of 2D representations. Although the increasing power of graphics hardware and software technology has made it possible to create, explore and manipulate 3D virtual environment, if the virtual environment contains too many polygons, it is still hard to render them interactively. For example, the complex scene such as very large city may contain millions of polygons. This is far more than currently available workstations can render interactively. It implies that some techniques are required to reduce computational cost for object rendering [5].
142 This paper proposes a method of 3D virtual city management for interactive operations based on the 2D hierarchical data structure. Using this method, we have developed 3D graphical facility management system. Even if this system manages the virtual city consists of ten thousand objects, the user can perform various interactions with the virtual city environments such as rapid walkthrough, ray intersection picking for object selection, the change of spatial views and so on. 2
THE
MANAGEMENT
OF 3D VIRTUAL
CITY
ENVIRON-
MENTS In geographical information system, the user must interact with the data as solutions to many problems. Therefore, interactive performance is very important. Interactive performance particularly depends of the frame update rate. To realize interactive performance, the frame update rate around 10 frames/sec is required. In this section, we propose a spatial data management method for a 3D graphical facility management system that offers the user interactive operations with the 3D virtual city environments. Our methods target to manage the virtual city which consists of not only objects on the ground but also under the ground. In our method, each 3D object in the virtual city is managed by using its 2D projection onto the ground. The projected 2D shape is managed by the 2D hierarchical data structure (such as quad-tree [6], MD-tree [7]) which is specialized to manage geometric objects efficiently. Given a view volume, the objects to render are searched rapidly using the data structure. The search area is also a 2D projection of the view volume. Potentially visible objects whose 2D projections intersect with the area are retrieved by the range search. When the user is walking on the ground, the objects under the ground are not visible. For the efficient display control, the objects on the ground and the objects under the ground are managed separetely by distinct 2D trees (layers) as shown in Fig.1. When the user is exploring the space under the ground, all the layers(2D trees) are used. On the other hand, when the user is walking on the gound, usually, the layer that manages the space under the ground is not used. Moreover, the space on the ground is further sliced into multiple layers depending on the height (Fig.l). As Fig.1 shows, all the objects on the ground are managed by the tree (layer) To. If the object is higher than the given threshold, it is also managed by another tree T1. In the city area, if an object is far from the viewpoint, it is likely that the object is occluded by the other objects which are closer to the viewer. This implies that an object far from the viewer is visible only if it is tall enough. Based on the heuristics, the search area (i.e. the 2D projection of the view volume) is derided into two regions. During the walkthrough operation on the ground, in the region which is close to the viewpoint, the potentially visible objects are searched using the layer To. In the other region, which is far from the viewpoint, visible objects are searched using the higher layer T1. With this technique, we can expect to get a natural scene interactively without increasing the objects to render.
143 ~
//..' ...........................
2D ~ (layer T,)
?,
~ T1
OayerTo)
~
::/
/ (layer Ta)
To
Fig.l: Object management by multiple 2D-trees based on the height
3
ADAPTIVE
CONTROL
OF D E T A I L S
By the object management method described above, we can get the efficient visibility culling function. However, sometimes, too many polygons are visible from certain viewpoints to render interactively. And also, the number and complexity of the potentially visible objects changes suddenly in case the user may turn around a corner. In order to guarantee an intera.ctive frame rate and frame to frame coherence even in those cases, an adaptive algorithm for level of detail is employed. In our method, every potentially visible object is rendered according to its value of reality parameter [8]. The value of reality parameter for each object is calculated regarding total amount of detail in a scene, distance from the viewpoint to each object, and the size of polygons in the screen image at each frame. For example, when an object is far away from the viewpoint, simpler representation of the object is displayed to reduce the number of polygons rendered without noticeably reducing image quality. The shape of each object with various number of polygons is prepared to approximate the original shape of object. Using the algorithm, the desirable frame rates (10frames/second) can be realized without causing significant image quality down even if the total number of objects in the virtual city is more than millions. 4
3D GRAPHICAL
FACILITY
MANAGEMENT
SYSTEM
We have developed prototype 3D Graphical Facility Management System using the methods described above. To test whether our system can provide an interactive performance independent of the scale of the virtual city environments and the scene complexity, we ran our system on the virtual cities whose number of objects varied from thousand to hundred thousand. The virtual cities include many kinds of 3D objects on the ground
144
....i:!:!:~:~:~:~:~:~:~:~:~:!:~:~:~N ~$.~ ~ . - : . . , : ~ ~ i ....~;:~:~i~:~:s:~i~isi~.'..,x.'~N~g'."$ ,~. "~'.;~. •~;_~"~" .... •..'.
i
~
"
!! •
.:::.,::~.,'~::s:-.::..-.-.-::...-...'::.e..-...~.! ~ i ~ : - . . : ,
.....
~:,......,..
/u/ .
. . . . . . . . . . . .
.. . . . . . . . . .
• .....~..,.~:~...... ~i~.
Fig.2: (a)Entire region to be managed (b)A scene fi'om the viewpoint under the ground
such as buildings, houses, roads, and the electric facilities installed under the ground such as pipes, cables, man-holes. Each object is constructed from about 10 ,-~ 100 polygons. Fig.2 shows example views of the created city and electric facilities. In each test, the average fi'ame rate was around 10frames/second and the frame to frame coherence was guaranteed. Therefore, the user can explore 3D virtual city without losing the sense of interaction.
4.1
I n t e r a c t i o n w i t h t h e 3D O b j e c t s
In 3D graphics systems, Human Computer Interaction (HCI) is a very important issue [4][9]. As for the ItCI, in addition to interactive walkthrough mentioned above, our system supplies a rapid interaction method with the 3D virtual city environments using only the three-button mouse, keyboard, and pop-up windows without using special 3D devices. Many interactive operations in virtual environments are based on the task of moving a viewpoint. In our system, the viewpoint movement is controlled by simply a three-button mouse. Translations and rotations are selected with the mouse buttons and the mouse motion is transformed into an appropriate viewpoint movement. Based on the techniques of viewpoint control described above, interactive operation is performed as follows. Access to the detail information of the object of interest is the essential operation for GIS. In our system , when the user picks the object of interest directly, the detail information about the object is displayed immediately through the popup window (Fig.3 (a)). Moreover the user can move 3D object freely. That is performed by placing the mouse cursor over the desired object and input the transform matrix by the keyborad. The main advantage of tile user interface based on 2D input devices is that these devices
145 are ubiquitous and the user do not have to wear special equipments (such as gloves and helmets). 4.2
Visual
Interaction
with
3D Environments
Our system also provides the user the capabilities to visually interact with the data using a variety of visualization techniques. It may be important that the user can easily change the visualization process in order to try different ways to view data. For example, making the objects on the ground semitransparent using transparency blending, the user can observe the underground facilities beneath the objects on the ground. The interactive change of the transparency level is very u~eful to understand the spatial relationships between them (Fig.3 (b)). In our system, texture mapping is applied to some objects. Using texture to reduce polygonal complexity, we can often get both an improved picture and improved performance. The user is allowed to change texture pattern on an object surface so that the user can experiment with the different ways of observing the virtual city without spending a lot of time experimenting. The view of the city may be either perspective or orthographic. The former is corresponding to the 3D spatial view and the latter is to the 2D. The user can switch the view of the city respectively at any time. This assists the user to assimilate current statement immidiately during some operations (such as object positioning and navigating through the large space).
(b) Fig.3: (a)The result of 3D picking (b)Transparency blending
146 5
CONCLUSIONS
We have presented a new 3D object management and rendering method for the 3D graphical facility management system. In this method, the virtual city is managed by multiple 2D-trees and each object is rendered at appropriate level of detail to guarantee an interactive frame rate without reducing image quality. We have developed prototype 3D graphical management system using this method to test whether this method produces interactive performance. With this method, our system provides the user the interactive operations such as smooth walkthrough and the rapid information retrieval of object of interest. Moreover, in our system, the user can visually interact with the data using a variety of visualization techniques. Future work is to find what kinds of interaction are most useful for 3D graphical facility management system, and to develop systems which can support them. Also of interest is integrating a method which allows to link visualization environmen~ to the GIS database.
REFERENCES 1. Brodlie K.W. et al.,Scientific Visualization : Techniques and Application, SpringerVerlag, (1992) 2. Rhyne T.M., Ivey W., Knapp L., Kochevar P. and Mace T.,Visualizatin and Geographic Information System Integration : What are the needs and the requirements, if any?, Proc. Visualization'94, pp.400-403 (1994) 3. Nakamura Y., Abe S., and Ikeda K., Interactive Graphics and Spatial Data Management for GIS using the Hierarchical Data Structure, FEGIS '93, pp.106-120, (1993) 4. Robertson G.G.,Card S.K.,and Mackinlay J.D.,Information Visualization Using 3D Interactive Animation, Comm.ACM, Vol.36, No.4,pp.57-71(1993) 5. Funkhouser, T.A. and S6quin C.H.,Adaptive display alogorithm for interactive frame rates during visualization of complex virtual environments, Proc. SIGGRAPH'93, pp. 247-253 (1993) 6. Samet H., Webber R.E., Hierarchical data structures and algorithms for computer graphics, Part I:Fundamentals, IEEE CG & A, May 1988, pp.48-68 7. Nakamura Y., Abe S., Ohsawa Y. and-Sakauchi M., A Balanced Hierarchical Data Structure for Multidimentional Data with Efficient Dynamic Characteristics, IEEE Trans. KDE, Vol.5, No.4, pp.682-694 (1993) 8. Tamada T., Nakamura Y. and Takeda S.,An efficient 3D object management and interacive walkthrough for the 3D facility management system, Proc. IECON'94, pp.19371941 (1994) 9. Houde S.,Iterative design of an interface for easy 3D direct manipulation, Proc. CHI'92, pp.135-142 (1992)
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
Cryptographic
Human
Identification
147
Extended Abstract
Tsutomu Matsumoto Division of Electrical and Computer Engineering, Yokohama National University 156 Tokiwadai, Hodogaya, Yokohama 240, Japan. email: tsutomu©mlab, d n j . ynu. ac. jp Conventional password checking schemes have a disadvantage that an attacker who has correctly observed an input password can perfectly impersonate the corresponding user. To overcome it there have been proposed schemes such that a human prover knowing a secret key is asked a question by a machine verifier, who then checks if an answer from the prover matches the question with respect to the key. This paper presents practical interactive human identification schemes with novel ideas of displaying questions. Keywords: authentication, hulnan-computer interaction, passwords, information security. 1. I N T R O D U C T I O N Human identification is a necessary item for access control mechanisms. Figure 1 illustrates the above mentioned difference between widely-used conventional password schemes and the interactive human identification schemes firstly examined in [1]. The same reference contains a brief description on the feature and significance of the latter class by contrast with schemes requiring auxiliary devices for human provers. The resistance of such schemes can be evaluated by the probability P(n) of which an attacker can correctly answer a given question after obtaining n (_> 0) pairs of questions and correct answers. A simple interactive identification scheme [2] is easy to understand and requiring no randomness to make an answer, but its resistance has been developed only by computer experiments [3]. This paper presents practical interactive human identification schemes with novel ideas of displaying questions. Linear algebra supports theoretical basis of these schemes and clarifies their rigorous profiles of P(n). 2. I D E N T I F I C A T I O N
SCHEME
BASED ON LINEAR ALGEBRA
Let s be a prime power, u and v positive integers. Also let F~ and F~ *' respectively represent the finite field of order s and the vector space {[Xl,... , X v ] l X l , . . . , x v e Fs} consisting of all v-dimensional row vectors over F~. Now we define a basic protocol to be conducted by two players: Prover P who claims that P is a person P; Verifier V who acts as a machine V and communicate with P. P r o t o c o l 0.
Preparation Phase O. Key Sharing
P and V agree on a key Kp - [ k l , . . . , k~] where kiT, ... , k~ T E Fs v, and keep the key secret.
148
Conventional
password
Interactive human identification schemes
schemes
Verifier
Verifier
~ Prover**secret ~ ~ 1 **
Secret leaks out easily
to protect
Prover'
eeret
Figure 1" Conventional Password Schemes & Interactive Human Identification Schemes Interaction Phase 1. R e q u e s t
P requests V to decide " P -
P".
Serially or concurrently for i - 1 , . . . , u, entities P and V execute steps 2 and 3. V generates a question qi E F s t' - {0}, and sends it to P. When V - V, we assume that qi is selected randomly and uniformly from F ~ " - {0}.
2. C h a l l e n g e
3. R e s p o n s e
P sends an a n s w e r ai E F~ to V. When P = P, we assume that
ai = qiki E F~
(1)
If V = V, then using/(p, Verifier V checkswhether equation (1) holds or not for each i - 1,...,..u. V judges "P = P"if and only if all of them hold. Then V informs P whether V has judged "P = P" or not.
4. A c k n o w l e d g m e n t
We can observe the following features. P r o p o s i t i o n 1. For Protocol 0, the key, answers, and questions are respectively described by u . v . log 2 s [bit], u. log 2 s [bit], and Do(s, v, u) = u . v . log 2 s
3. S E C U R I T Y
[bit]
(2)
OF BASIC PROTOCOL
3.1. T a x o n o m y of A t t a c k s An attack is an action by an entity A, attacker, who is different from P and V, to aim at letting V decide "A - P". An attack succeeds if and only if V judges "A - P". Depending on the knowledge the attacker can utilize we can distinguish three types. B l i n d A t t a c k An attack by A who has been given no pair of a question to P and the corresponding answer from P. Let P(s, v, u; 0) denote the least upper bound (LUB) of the success probability of any blind attack to Protocol 0.
149
Buccess Probability
,'
/ /'", ."'" / // /
'
i
[' t,
:
i #
....
:
•
/
/
.
I / i
/
o,
.
"
: i
.
F;l
"
: I/
....
i
/ I / ..... if# t, / i / /1 // / / _. . . . . ~,~
:
i,#
, t
l'ii * * * t /
o...... o
l
/
/
./ i" ...../ .-/" ....,~,,..'~ .,, s"
~
~
i
....... "
. . . '
:, .
~
~
""
;
i i
' ' ' ' ' '#
i i i i i
[,,, , . :I
#
i
'
/
l I : I : I : I : I I : / /
l
l"
# i
/
/!
/
,
i / #
i !
o
;
.
oool
l'!,,'/
o...... o
/
:
~
! "
/
-
.. . .. ... ... .
:
2
~
,
=
-
. . . . .
~
,
'
/ " " "
. . . . . I .l l ; ./ . .
- . - . . ~ ..... v -
:
.
" I
i ! / ..... f / / / / / / 7 :I , i ,
.
r,s
o.oool
8uccess Probability
-- ~ = ~..7----;--~.~--~~---- -~---.,...... "
~
....
v
=
.
5
;
TI A
Figure 3" Examples of P(s, v, u; n)
Figure 2: Examples of P(s, v, u; n)
K n o w n Q &: A A t t a c k An attack by A who has observed n (> 1) pairs of given questions and corresponding answers. Let P(s, v, u; n) denote the LUB of the success probability of any known Q & A attack to Protocol 0. C h o s e n Q &: A A t t a c k An attack by A who has acted as V - A and observed n ( _ 1) pairs of chosen questions and corresponding answers. Let P(s, v, u; n) denote the L UB of the success probability of any chosen Q &: A attack to Protocol 0. A
3.2. S u c c e s s P r o b a b i l i t i e s We can prove the following profiles. See Figures 2 and 3. Proposition
2.
We have 1)~,
P ( ~ , ~, u; 0) -
Proposition
3.
(3)
(7
For n >_ 1, we have
1 P(s , v.u;n) - (1- + ( 1 - - ) . ' s s
s~'
1
£ ( s t - 1). R(s, v; n, l)) ~ l=l
-1
(4)
where for 0 _< l _< v the quantity R(s,v;n,l) is the probability that the vector space spanned by n vectors selected uniformly and randomly from F".; - {0} has dimension of l. Proposition
4.
-. P(s v~ u; n)
For 0 _ n _< v we have 1
- (-s- +(1-
1
s ' " - 1 ~,
-)'s"s
1)
(5)
150 4. I D E N T I F I C A T I O N
PROTOCOLS
USING HUMAN
CHARACTERISTIC
Computing the scalar multiplication in equation (t) seems hard for ordinary persons. To avoid this difficulty we direct our attention to a characteristic of human vision. Assume a screen where a lot of points lie at determined location and that each point is labeled by a symbol selected from a finite set. Assume a keyboard where each symbol in the set can be input. We use the fact that ordinary persons can quickly focus on a predetermined point and input the corresponding label into the keyboard. 4.1. P r o t o c o l 1 Let f~ be a set of s ~ elements. In Protocol 1, each question qi in Protocol 0 is assigned a question-expression, Q,-
{(w, q~(w)) I~a e f2},
q , ( w ) = q , . ¢,(w) w e F~
(6)
where ¢i is a bijection from f2 onto F,". Prover remembers u-tuple [kx, k2,... ,k~] over as a key. For each i = 1 , . . . , u, Prover answers qi(ki) to given question-expression Qi. Note that ¢i(ki) T = ki and qi(ki) = ai. Protocol 1 has the following properties. P r o p o s i t i o n 5. The LUB of success probability of blind, known, and chosen Q & A attack to Protocol 1 is upper bounded by that of the corresponding attack to Protocol 0. P r o p o s i t i o n 6. In Protocol 1, the question-expressions are described by D l ( s , v , u ) = u . s". log 2s
[bit]
(7)
E x a m p l e 1: M a p S c h e m e . Figure 4 depicts an example where ( s , v , u ) = (3,4,9), Fs = {1,2,3}, and fl is the set of 81 sampled names of railway stations. The questionexpression is the set of pairs of station names and figures appearing in small circles put on the location of the stations. This example displays u - 9 question-expressions serially. The route map helps Prover to remember a key and to quickly look for the location of a remembered station and recognize the figure. From this example we can see that such a relation among the elements of fl can greatly reduce load of human provers. 4.2. P r o t o c o l 2 Propositions 2, 3, 4, and 5 imply that to increase resistance against attacks we should make s, v, and u grow. However, human memory capacity limits u. And capacity of a screen to display question-expressions or human visual ability of discriminating points limit s and v according to Proposition 6. Now we point out a middle approach between Protocol 0 and Protocol 1 to reduce the amount of d a t a for displaying a question-expression. Let m (___ v) be a positive integer. Recall Protocol 0. We divide v into v - ~ "f=l vf, vf > 0. Correspondingly for i 1 , . . . , u, we divide qi and k w as qi - [ q i l , ' " , qi,n.] and k w - [kW, . . . , kiT] so that by letting air - qifkij E Fs for f - 1 , . . . , m, we have ai = ~ = 1 air. Thus we can derive a scheme, called Protocol 2, that 1) for each f = 1 , - . . , m Verifier challenges a sub-question-expression like in Protocol 1; 2) Prover makes in mind an subanswer for each sub-question-expression; 3) Prover mentally sums up all of the sub-answers
151 .,
.." ............ •"
•."
........ .:
~
~"
¢3
".. J ...... Nlshi-Urllwa ...". . . . .... A r e a ' E "
........
1".....
...•" / " '-~- ,....... ~z~ .-:p~ H,+m," ...... ~ - 4 " ~ C ~ ..... i r'.: ~. ............." ~ . 1 . . . ~ ~ . . : [ : :
m
~a..............
K,*,oo,~. " n" •
~ #r ~
.:':"-...
~r,,,y,
Migasni-urawa
"li ~
r~ ~
]_Klsuklbe ....... Klshiwl .............. ( ~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ~r~mmmmz ! Shin-Matsudo ~ r ~
~ t.............. ,y, O ........ .... °o
.................. ......~\._ . / _ . ~
. . . . . . . . . . ".
/" ............~.,.-.
~..........i''. . : .
. . . .
:
............."+" ... ...................~..... K ami-Shakujii
~#~
...................~
~ I
K
:"
: ,
e
O
' ~Kita"~"
.
.
X
,
, , ..,
°"I~" S * ehJblma'* ""n " " ~
. ~
Abiko k
"%" "\"" ~
-+
'":-::',,
.
.
"....".; Shln-Yahashira (
"
"-...
"
mC
K
........... " " /
Nobor to
+ .:
i . + ! + +
hin-Kamagaya
c
........................................................ 1
i
.-.."" " .....J I!
.....( •
,,
.:...~.......................... ..-;
-,'..
"T ~
7 . . . . .j ...... ~
i
!
i
".:÷............................. ~ ) .................................
.." ........................... "~.
..." .." O',
,,
,
.........
t , . . ~ . \
!
1
|
z
.........
:
``...
J Kawasaki""
""
Figure 4: An Implementation of Protocol 1: Map Scheme ((s, v, u ) = (3, 4, 9)) to have an answer; 4) then Prover inputs the answer to Verifier. following properties.
Protocol 2 has the
P r o p o s i t i o n 7. The LUB of success probability of blind, known, and chosen Q & A attack to Protocol 2 is upper bounded by that of the corresponding attack to Protocol 0. Proposition
8.
In Protocol 2, the question-expressions are described by 27/.
D2(s, v, u) - u . (~-~ s~'S) • log 2 s I=l
[bit]
(8)
I , Protocol 2 P~ov+vshouid make addition(s)over F , but the value D2(s, v, u) can be far smaller than D1 (s, v, u). E x a m p l e 2: G a m e S c h e m e . Reducing load of Prover we translate the F~ addition into some binary operation that can be readily done by ordinary persons. Here we utilize the J a n k e n game of 'paper', 'stone' and 'scissors'. Let s = 3 and m = 2. See Figure 5. This example displays u = 9 question-expressions serially. In the figure, the left and right screen respectively displays a question-expression of (s,v) = (3, 3). In the left screen, element 0, 1, and 2 is respectively m a p p e d to hand 'stone', 'scissors', and 'paper'. In the right, element 0, 1, and 2 is respectively mapped to 'stone', 'paper', and 'scissors' Prover selects a hand from the left screen with respect to a key and one from the right screen in a similar way. Prover mentally matches both the hands and answers the winner. Namely, if the left wins the answer is to touch symbol L, if the right wins it is to touch symbol R, and if the game is drawn it is to touch symbol D. Note that the rule of the
152
j+~'.....A"'"':B......C'""D'.....E .....':F'"'"G'"'"~ Left
i+++++++++i i
. R ight I'~"'~'B'"C"'D' .....E .....~4.....'~'"",'.~
,,+~4+~ i+++++~++~i
i _+2qo,,y,..+/+ !
iI J K L M N 0 P Q K ~ / ~ , ~ / ~ L / ~ ~ I
i+ + + + + + + + + i ~ i +.a + ~ . v ,
x + ~.+i
i
a X L M N 0 P Qi
+ + + + + + + + +i
ita + + , ,, ,+ X + ~.,i
"<~
t..+,..,+,,..f.,._+,,,,,+,,,,,+,,,,,+,,,,+.,,,,+if_++.,+ J .+, . ., .,+,. . ., +, ., .,.+.,_,+. +.J ~ ° I+++++++++++I
~ ===-Ii~ o
L
D
R
~ ===,==I~ 0
Figure 5: An Implementation of Protocol 2: Game Scheme ((s, v, u) = (3, 6, 9)) game is exactly the same as the F3 addition if we interpret L, D, and R respectively as 2, 0, and 1. We can inlplement the case where m > 2 by a similar way: We translate symbol L, D, and R respectively into hand 'paper', 'stone', 'scissors' and conduct an elimination tournament by matching one of L, D, and R with a hand selected from another questionexpression newly displayed in the right screen. 5. C O N C L U S I O N We have examined interactive human identification schemes that can resist Q & A attacks in some extent. Applied Protocols require human provers only fairly simple manipulations. An important subject is to further investigate whether we can develop practically appealing human identification protocols along with the suggested line. REFERENCES 1. T. Matsumoto and H. Imai, "Human identification through insecure channel," Advances in C r y p t o l o g y - EUROCRYPT'91, Lecture Notes in Computer Science No.547, pp.409-421, Springer-Verlag, 1991. 2. H. Ijima and T. Matsumoto, "A simple scheme for challenge-response type human identification," Proc. of 1994 Symp. on Cryptography and Information Security, SCIS9413C, Jan. 1994. 3. R. Mizutani and T. Matsumoto, "Evaluating security of a simple interactive human identification scheme," IEICE Trans. Fundamentals, Vol. E78-A, No.5, May 1995.
III.7 Screen Design 2
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
155
Adjustment Mechanism for a Drawing System with Stationery Metaphors Naoki KATe, Natsuko FUKUDA and Masaki NAKAGAWA Nakagawa Lab., Department of Computer Science, Tokyo University of Agriculture and Technology, 2-24-16 Naka-Cho, Koganei-Shi, Tokyo, 184 JAPAN, e-marl: [email protected] I. INTRODUCTION We have been carrying out research aiming for a system which will support our creative activities. As a part of this research we have realized a penbased drawing system for the creation of diagrams on a computer [1,2]. Up until now many drawing systems have been researched and developed, but the majority of current systems are dedicated to the stage of making a neat copy. As opposed to this approach, the drawing system we are developing is aimmg to support work from the stage of creating a rough copy, and makes use of a handwriting (pen) interface using a display-integrated tablet as a humancomputer interface. As handwriting interfaces do not obstruct the user's thought process with their operations, they make it possible to realize a system which can be used from the rough copy stage where the user's thinking is of utmost importance. With this drawing system, we have provided stationery metaphors as a new drawing interface. The stationery metaphors are virtual stationery displayed on a display-integrated tablet, and are metaphors which replicate the abilities and operation methods of real stationery. As they are manipulated in the same manner as actual stationery they can be used easily, even by people unaccustomed to computers. As with real stationery, however the positional adjustment of drawings that are about to be drawn is still a troublesome task. Although the difficulty of adjustment has been a problem with many mouse-based drawing systems, it is more serious since operations requiring precision are difficult with a pen on a display-integrated tablet due to parallax and unstability of hand movement [3]. In this paper we present an automatic adjustment mechanism designed for a drawing system with stationery metaphors to solve this problem. 2. NECESSITY OF ADJUSTMENT FUNCTIONS With this drawing system it is possible to draw rough copies in freehand with no restrictions. For freehand drawing, a pen is much mightier than a
156 mouse. When geometrical shapes such as lines, circles, and so on are required stationery metaphors are used. The stationery metaphors that have been developed so far consist of a ruler metaphor for drawing straight lines, a compass metaphor for drawing circles and arc lines, and template metaphors [4] for drawing objects such as squares, triangles, and so on. When drawing with stationery metaphors it is sometimes difficult to align the position or size of the drawing. For example, positioning a triangle template metaphor so that the point of the triangle matches to the end-point of an already drawn line (figure 1) is more difficult than it at first seem. In an experimental trial of this drawing system, five of the ten students from our laboratory who participated expressed the opinion that minute operations were difficult with the template metaphors. This phenomenon also occurs when drawing with pen and paper, or using a mouse-based drawing system. If the computer can support such precise adjustments it then becomes possible for the user to draw neat drawings in a short period of time. Here we propose an automatic adjustment mechanism for a drawhag system with stationery metaphors where the computer supports these adjustments. This automatic adjustment mechanism allows adjustments such as of position or size exactly as the user intends, without placing an unnecessary burden on the user. The employment of computing power for this type of adjustment is a natural automation of our physical alignment of real stationery. Therefore, its effect could be different from automatic adjustment in the mouse-based drawing systems. A template metaphor of a triangle. Moving
An already drawn line. ,
User wants to match these. Figure 1. An example of positioning a template metaphor.
3. CLASS OF ADJUSTMENT FUNCTIONS Adjustment operations for drawing with a computer can be divided into two classes depending on whether the adjustment is carried out when the object is drawn, or after it has been completed. The adjustment mechanism we present is the former. When manipulating a stationery metaphor the computer carries out
157
adjustments as it sees necessary. The types of adjustment required for drawing with stationery can be further divided into two main types. One type supplements movement, size changes and rotation of stationery metaphors to fit them precisely to already drawn objects. For example, ff you want to use the compass metaphor to draw a circle touching a straight line it is necessary to precisely line up the drawing point of the compass with the line (figure 2). When you move the pencil ~ m to a point near the line, the automatic adjustment adjusts the arm precisely tO the line. The other type is adjusting the start or end point of the object to be drawn when using a stationery metaphor. For example, when using the ruler metaphor it is difficult to align the start point with the position intended (figure 3). An adjustment can be carried out to adjust this alignment. We shall refer to the former case as an adjustment to the stationery metaphor, and to the latter case as an adjustment to the object
being drawn.
~
djusted. Start point is adjusted so as to fit a drawn line. A compass m e t a p h o r A ruler metaphor
An a l r e a d y d r a w n lme
An already drawn line
Figure 2. An example of positioning a compass metaphor.
Figure.3. An example of adjusting the start or end point of the drawing object.
4. DESIGN OF ADJUSTMENT FUNCTIONS 4.1. Design 1" Automatic adjustment and cancellation This adjustment mechanism supports adjustments made when the user is drawing with a stationery metaphor. Here the computer anticipates the user's intentions and automatically carries out adjustments. However, there are some cases where the user would not want adjustments carried out automatically. To cope with these situations the user can choose whether or not adjustments will be carried out automatically. 4.2. Design 2: The Operation the user last carried out is chosen as the method of adjusting The adjustment method carries on the operation that the user was performing. For example, an adjustment carried out after the user had performed a
158 movement operation would adjust the object by moving it. Revolving the object to adjust it, even though the user had been moving it, would produce an unexpected result. 4.3. Design 3: Objects that have already been drawn are not altered When adjusting a stationery metaphor, objects that have already been completed are not moved or distorted. Adjustments of position or size only apply to the stationery metaphor or object being drawn, and thus confusing the user is avoided.
5. IMPLEMENTATION This adjustment mechanism has been implemented in the drawing system with stationery metaphors. This system supports freehand strokes, straight lines and arc lines as drawing objects, and provides a compass metaphor, ruler metaphor and template metaphors. There are several types of adjustment mechanism that should be provided, but we currently have only the following. (i) The ruler metaphor is adjusted so as to fit a start or end point of a straight or arc line (figure 4(a)) (ii) The ruler metaphor is adjusted so as to touch an arc line (figure 4(b)) ('all)The pencil arm or needle arm of the compass metaphor is adjusted so as to fit a start or end point of a straight or arc line (figure 4(c))
(a)
Co)
(c)
Straight line
ight line Arc line
line
/
Arc line
/
h
Fit
I I A ruler metaphor
A ruler metaphor
A compass metaphor
Figure 4. Type of adjustment functions.
6. EVALUATION We have carried out a simple experiment to evaluate the effectiveness of adjustment functions. Twenty students from our laboratory were asked to reproduce a drawing, like in figure 5, in two different environments, one with adjustment functions and one without. In order to remove any learning effect the
159 subjects were divided into two groups of ten each, with group A using the environment without adjustment features first, while group B used the environment with adjustment features first. The results of the experiment were as follows. (A t-test validated that there was no difference between the results for the two groups, and so they have been combined.) The average number of operations performed with stationery metaphors is given m table 1, while the average time taken to complete the drawing task is given in figure 6. A t-test verifies that there is a difference, in both the number of operations performed and the time taken for the task, between the cases where the adjustment functions are or are not used. From the above results we believe that adjustment functions are effective in reducing the labor required m drawing tasks. Table 1. The average number of (moves) operations With adjustment functions
Without adjustment functions
Ruler
Ruler
5.45
Compass Pivotal
Pencil
3.10
3.50
13.35
Compass Pivotal
Pencil
10.20
8.85
7. SUMMARY This paper described an automatic adjustment mechanism for a drawing system employing a handwriting (pen) interface with stationery metaphors. With this mechanism the computer automatically carries out adjustments of the position or size of stationery metaphors at the stage of drawing. By including this adjustment mechanism in the drawing system it is possible to support the timeconsuming task of making adjustments, and thus improve efficiency of the users. REFERENCES 1. Nakagawa, M., Kazama, S., Satou, T. and Fukuda, N.: Pen-based Interfaces for Dra wing Figures with "Stationery Metaphors: Human-Computer Interaction: Software and Hardware Interfaces (Salvendy, G. and Smith, M.J. ed.), Elsevier Science Publishers B. V., Amsterdam, (1993) 1046-1051. 2. Kazama, S., Kato, N. and Nakagawa, M.: A Hand-Drawing System with 'Stationery Metaphors'On Japanese), Trans. IPS Japan, 35, 7 (July 1994) 14571468. 3. Kato, N., Fukuda, N. and Nakagawa, M.: An Experimental Study of Interfaces Exploiting a Pen's Me~its, Proc. HCI '95, to appear. 4. Fukuda, N., Masaki, N.: Prototyping of Pen-based Drawing Interfaces with Template Metaphors (in Japanese), SIGHI IPSJ 48-5 (May. 1993) 33-40.
160
~) The center of the circle is the left point of a drawn line. the radius of the circle is the ength of a drawn line.
Po t of contact F i g u r e 5. F i g u r e for e v a l u a t i o n e x p e r i m e n t .
W i t h o u t a d j u s t m e n t functions.
With a d j u s t m e n t functions.
Task time (sec) .
.
.
.
.
.
160-
.
! 140-149 130-139 120-129 ....... 110-119 . . . . . . . . . . . . . .
~
100-109
-
90-99
m
80-89 70-79
~
60-69 50-59
i~:::[ ~ ::
40-49
~
_
_
. . . . . . . . .
~
~
.
-39 I
8
6
4
2
0
0
2
4
6
F i g u r e 6. N u m b e r of s u b j e c t s b y t i m e t a k e n for t h e d r a w i n g t a s k s .
8
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
161
Analysis Tool for Skill Acquisition with Graphical User Interfaces Based on Operation Logging Nobuko Kishi ~ ~Department of Mathematics, Tsuda College, Kodaira-shi, Tokyo 187, Japan Observing users working with a system is essential for better user interface design. One of popular observation methods is logging users' operations, i.e., having the system collect the users' operations automatically. However, it is known that operation logs, particularly the ones collected in the systems with graphical user interfaces(GUI), can be very large and difficult to extract meaningful information. We developed a set of tools, collectively named SimUI, to log and analyze users' operations on applications with GUI. SimUI's analysis tool uses a technique, named multi-step matching, to detect differences between two sets of operation logs. This paper describes the use of multi-step matching in a experiment, in which the subjects are asked to learn to use a bitmap editor. In the experiment, we observed that the 'matching rate,' measured by multi-step matching, can indicate various type of skill acquisition, compared to statistics about the task completion time and the frequency of command use. 1. I N T R O D U C T I O N Observing users working with a computer is known to be an effective approach for better user interface design and better user support. Several observation methods can be used such as video-camera recording, audio-tape recording, and interviewing. However, the effectiveness of these methods depend on the skills and knowledge of human observers. To overcome these drawbacks, logging, i.e. having the system collect users' operations is often used to record the users' behavior automatically. Several statistical techniques have been used analyze operation log[2]. However, the log data collected in the system with graphical user interfaces can be huge and requires more powerful analysis techniques[l]. We developed a set of tools, collectively named SimUI (SIMulated User Interaction evaluation tools), to collect operation logs and to extract meaningful information for user interface designers. The central idea of SimUI is to record all user input and system output, and to generate the difference between the operation logs of novices and skilled users automatically. Although the amount of recorded data is very large, SimUI's uses two technique to automate the analysis' data gathering in playback mode and multi-step matching of recorded data. The usefulness of SimUI's analysis technique was studied by conducting an experiment. Four subjects were asked to learn to use a bitmap editor on X Window Systems. We found that the difference detected by SimUI can suggest various aspects of skill acquisition with applications with graphical user interfaces.
162
I
Modified X ~ Display Server ~ • ~"
-~ -]
~.
X Protocol Analyzer
.,
.- ~-Modified X - lToolkit
F
Application
k
".,,, ,/
Playback Tool
v~Tool[Data Collection
~ Operati Log on
} Recorded Data
Figure 1. Data collection during playback
2. S i m U I - T O O L S
FOR GRAPHICAL
USER
INTERFACE
EVALUATIONS
SimUI is implemented for X Window Release 5 applications. User operation logging capability of SimUI is provided by: • modified X Display server, • recording tool, • playback tool, • X protocol analyzer, • data collection tool, • modified X toolkit library. These tools are used in the following steps. 1. The recording tool sends a 'record' request to the modified X display server. The display server start sending all the input events to the recording tool. 2. After recording, an application, which is to be the main focus of observation, is relinked with the modified version of X toolkit library. 3. The X protocol analyzer is run as a pseudo display server. It behaves as a display server for the application program, while it behaves as a client of the modified X display server. 4. The playback tool sends a 'playback' request to the modified display server. The server switches the source of input from the user to the playback tool, then the playback tool starts sending the input events recorded by the recording tool. 5. The application and the display server runs as if they are in playback mode. In addition, the application program, the protocol analyzer and the playback tool send the information on their status to the data collection tool.
163 s>LOC_Y_DELTA: 12388714251 mili value -27 s>LOC_X_DELTA:I2388714661 mili value 3 s>LOC_Y_DELTA: 12388714661 mill value -2 sc>EVENT: FocusOut detail: Nonlinear event: WIN2000038 mode: Normal sc>EVENT: EnterNotify detail: NonlinearVirtual time: TIM49d7adaa root:WIN2b sc>EVENT: FocusIn detail: Pointer event: WIN2000038 mode: Normal sc>EVENT: FocusOut detail: Pointer event: WIN2000038 mode: Normal ap>handler WID27728 index 0 proc ADRf76fOeac eventtype 7 ap>handler WID27728 index 0 proc ADRf76fOeac eventtype 9 ap>handler WID27728 index 0 proc ADRf76fOeac eventtype I0 ap>handler WID27728 index 0 proc ADRf76fOeac eventtype 9 s>LOC_X_DELTA:I2388715081 mill value -I s>LOC_Y_DELTA: 12388715081 mill value 5 sc>EVENT: FocusIn detail: Nonlinear event: WIN2000038 mode: Normal s>LOC_X_DELTA: 12388715501 mill value 1 sc>EVENT: LeaveNotify detail:Virtual time: TIM49d7add4 root: WIN2b event:
Figure 2. Example of data: collected during playback
The relationship of the tools at the last step is shown in Figure 1. The example of data collected by the data collection tool is shown in Figure 2. Each line contains one record sent to the data collection tool from the other tools. The prefixes indicate which tool has sent the record as follows. s> Sent from the playback tool when it sends input events to the display server. sc> Sent from the protocol analysis tool when the server sends an event or reply to the client. cs> Sent from the protocol analysis tool when the client sends an event or reply to the server. xt>,ap> Sent from the application program (a X client program) when its X widgets changes their status.
3. L O G A N A L Y S I S B Y M U L T I - S T E P M A T C H I N G
The analysis tool in SimUI is to analyze the data recorded by the data collection tool. Although the analysis tool can generate various statistics, its main technique is the multistep matching. The multi-step matching is to compare two sets of records and generate .the difference between them. When one of the records are obtained from a skilled user, and the other from a novice user, the difference generated are closely related to the differences noticed by human observers. The multi-step matching is performed by repeating: 1. filtering out less important records.
164
ap>A :
--ap>A
lap>A:
:
lap>A
lap>A:
:
lap>A
S>X
•
•
S>X
•
•
S>X
S>X
~
S>X
sc>a
sc>a
•x
• s>x
S>X
•
• s>x
ap>B ~ s>x .
s>x
s>x sc>a
s>x ap>B
s>x
•
•
sc>a
First Matching
2ap>B,k~. s>x . \ , s>x • \.
s>x sc>
"~
2sc>a %
• s>x
3ap>B,k, ~ , _ \- s>x :\:
sc>a
s>x L,,,.~ 2ap>B
s>x " 4sc>a ~.....~ 3ap>B
s>x
•
s>x
-',
sc>a
Second Matching
•
--
4sc>a
Third Matching
Figure 3. Example of multi-step matching
2. generating a difference between two sets of records, by using a tool such as a 'diff' command in UNIX. 3. numbering the matched records so that they lnatch again at the later repetition. By doing so, we are more likely to match meaningful records (e.g. a command is executed) rather than somewhat meaningless records (e.g. a mouse pointer is moved by one pixel.) Figure 3 shows an example of the result obtained by the multi-step matching. The multi-step matching generates a matching rate, a rate of the matched records in all of the records. Because filtering and matching are repeated three times based on the prefixes of records, there are three kinds of matching rates. 1. Application-Level Matching Rate The matching rate of records prefixed with xt>,ap>. This rate is assumed to become large when the same callback functions and event handlers are called, which often occurs when the same commands are entered in the same context. 2. Protocol-Level Matching Rate The matching rate of records prefixed with sc>,cs>. This rate is assumed to become large when the same requests and replies are exchanged between the application and the X display server, which often occurs when the same windows or widgets are selected. 3. Device-Level Matching Rate The matching rate of records prefixed with s>. This rate is assumed to become large when the same keys are hit and when the mouse is moved the same distance in the same direction, which often occurs when the mouse are moved in the similar way.
165 4. E X P E R I M E N T -
LEARNING
TO USE
A 'BITMAP'
EDITOR
The assumptions about multi-step matching were studied by analyzing the operations on 'bitmap.' 'Bitmap' is a bitmap editor under X window systems (X11R5), included in the standard source distribution and implemented with X toolkit. The 'bitmap' used in the system was linked with SimUI's X toolkit library, so that it can produce an application level records. Four participants were asked to draw a shape of star with a 'bitmap' as shown in Figure 4. The reason for choosing the task of drawing a star is that it involves various types of skills. Figure 4: bitmap': a bitmap editor under X window systems • Motion skill.
For example, a user can move the mouse faster and more precisely. • Knowledge about an application program. For example, a user can learn the meaning of commands such as 'flood fill' and 'Invert', instead of painting each rectangle. • Knowledge about an application area, geometry. For example, a user can draw a star with only 5 lines although rhe shape of star has 10 edges. The four participants are asked to perform the task, drawing a star, twice. First they were asked to draw a star without any advance notices, although they are familiar with X window applications and have used the 'bitmap' editor for several times. Then they were asked to draw a star again after practicing as many times as they want. Figure 5 and Figure 6 show the results of the two participants, User C and D, respectively. Both participants improved the task completion time, i.e. they draw faster at the second performance. However, one participants, User D, didn't increased the applicationlevel matching rate at the second performance. This indicates that User D used a different set of commands to perform the task. That means that User C improved the motion skill, while User D found a different strategy to draw star between the two performances. 5. D I S C U S S I O N S A N D C O N C L U S I O N S Although we omitted other findings in the experiment, we found that the matching rate can detect the differences in the type of skills the participants improved with the bitmap editor. These differences would have been overlooked by using statistics such as the task completion time the frequency of command use. By implementing SimUI, we found:
166
• T h e o p e r a t i o n logging with X w i n d o w applications are feasible. In paticular, d a t a g a t h e r i n g d u r i n g p l a y b a c k m a k e s the collection of b o t h user i n p u t and s y s t e m o u t p u t feasible. • Because m o s t SimUI tools run as servers, it is possible to u n o b s t r u v e l y collect log d a t a from a large n u m b e r of users working in their e n v r i o n m e n t .
REFERENCES I. A.C. Siochi and R.W.Ehrich, Computer Analysis of User Interfaces Based on Repetition in Transcripts of User Sessions. ACM Transactions on Information Systems Vol.
9, No. 4 (1991) 309-335. 2. S.J. Hanson, R.E. Kraut and J.M. Farber, Interface Design and Multivariate Analysis of UNIX Command Use. ACM Transactions on Office Information Systems Vol. 2, No. 1 ( 1 9 8 4 ) 4 2 - 5 7 . Percent 100.00
I
-
_'Application Level
I
" P ~ & - 6 i ~ v 6 i ......................
- "g6~7~4%i . . . . . . . . . . . . .
80.00 60.00 --
m
40.00 -
...........'..,~.... ...................
° °-~t
20.00 0.00--
I
I
First
Second
Figure 5. User C's M a t c h i n g Rate.
Percent 100.00 80.00
I
I_ "Application Level ~i~8i8~'8i"E~'~;~i.................. - "~4~7L/~%i ........... ~I'ime (Seconds)
I
I
I--
First
Second
I
_l
I
-
60.00 40.00 20.00 0.00
-I
F i g u r e 6. User D's M a t c h i n g Rate.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
167
The role of screen p a r a m e t e r s in visual communication Masaaki K u r o s u a, H i t o s h i Y a m a d e r a a a n d Itaru M i m u r a b aDesign Center, Hitachi Ltd. 5-2-1 M i n a m i - a o y a m a , M i n a t o - k u , T o k y o 107, J A P A N Phone: +81-3-5485-1451, Email" [email protected] b c e n t r a l Research Laboratory, Hitachi Ltd. 1-280 H i g a s h i - k o i g a k u b o , K o k u b u n j i shi, Tokyo 185, J A P A N Phone: +81-423-23-1111
1. I n t r o d u c t i o n
One reason why the video conference system has not yet widely used is the lack of the evidence that will convince us its cost-performance compared to the face-to-face meeting. Reality is a key concept to describe the performance of the system. Previous researches on the video conference system had a tendency to focus on the hardware and software aspects of the system. But few of them studied the human side of the system, i.e. how the reality can be measured or how physical parameters may affect the degree of the reality. In this study, we tried to fix the independent and dependent variables relating to the reality and performed a psychological experiment on some of these variables. 2. Variables relating to the reality of the c o m m u n i c a t i o n
Dependent variables that represent the degree of the reality may vary according to the type of the meeting.
For one-way communication, the understandability of the
information and the resulting change of the attitude of the attendants are important, and the degree of sharing the topics, the depth of the discussion and the satisfaction of each participant are important factors for two-way communication. Independent variables include the coference setting, i.e. the type of the meeting, the size of the meeting, the layout in the meeting place, or the distance among each participant. Other type o f independent variable is the device parameters, i.e. the resolution on the screen, the size of the screen, or the configuration of the sound field. As a first step, we decided to investigate the one-way communication by way of the attitude change as a dependent variable and the screen size and the resolution as
168 independent variables. 3. C o m m u n i c a t i o n e x p e r i m e n t
3.1 Experimental method The basic paradigm of the experiment was the before-after design that measures the degree of the attitude change of the subjects before and after the presentation of persuasive information. As a topic, we picked up the discriminative expression including the sexual harassment and the discrimination to the minorities. Based on the assumption that most of the subject may initially think that to avoid the use of such discriminative expression is enough, the persuasive information given at the presentation period paradoxically emphasizes that it does not necessarily matter whether to use such expressions or not and that what really matters is the way of thniking at the back of such expression.
The
presentation lasted for about 7 minutes. The presentation was given both via the screen and by the direct speech. There were two screen conditions, the screen size and the resolution. Screen size conditon included 110 inches and 55 inches. The image of the speaker appears on the screen in the actual size in 110 inches condition and in relatively minimized size in 55 inches condition.
Resolution
condition included HDTV level and NTSC level. 10 subjects participated in each condition who looked a t the screen in a row with the distance of about 3 meters from the screen. In the screen conditions, the presentation was pre-recorded in a HDTV video and the resolution level was controlled at the time of
the
presentation.
In the
direct
condition, the presentation was given by the same speaker directly in the same room.
Because there were four screen
conditions and one direct condition, total of
50
subjects
experiment.
participated
in
the
All the subjects were the
employees of the research laboratory. The image of the experimental situation is
Figure 1. Experimental Situation.
shown in Figure 1. Three types of data were collected. One is the attitude scale that included 9 subscales on 1. whether the government should control the expression, 2.1 whether the mass communication corporates should define the list of prohibited sexual expressions, 2.2 whether they should define the list of prohibited discriminative expressions against the minorities, 2.3 whether they should rely on the individual judgment of their employee, 2.4
169 whether they should ask the specialist for the adequacy of the expression, 3.1 whether the company to which all the experimental subjects work for should define the list of prohibited sexual expressions, 3.2 whether the company should define the list of prohibited discriminative expressions against the minorities, 2.3 whether the company should rely on the individual judgment, 2.4 whether the company should ask the specialist for the adequacy of the expression. Those were rated on the 7-points scales. Because the presentation given in between the attitude measurements were focusing on how the company to which the subjects belong should behave for the problem of the discriminative expression especially for the minorities, it was hypothesized that there will be a big attitude change at least for the scale #3.2. Other data were additional. One is the subjective evaluation d a t a on the personality of the speaker.
Fifteen
7-points scales were used after the presentation to describe the
personality of the presenter such as social ingelligent, nervous etc. Another is the evaluation data on the quality of the presentation.
Three 7-points scales were used including the
clearness or the visibility of the image on the screen, the naturalness of the atmosphere and the reality of the total
,0
..........................................................................................................................................................................................................
impression.
55 / HDTV Direct\ ~:i~.!i~
..............................................................................
3.2
Result
on
the
personality
60%
evaluation
. . . .
Figure
...~. ...............................................................................................................
i
; ..................
. . . . . .
;
..........................................
2
shows the mean of
each
of
conditions 15
"~40
~I
_..
,
for
personality
traits. was
5
There not
¢1,~
e, o / .................................
\
...................................
_.,.
a
significant difference in the
20
rating data, i.e. it can not
be
10
attributed to the personality
m
of
--~ ~-~
co
E
~
-~
==
=; ~ ~
~
8
~
._
~
the presenter, if there may be a
Figure 2, Subjective evaluation on the personality of the
difference in the
presenter.
170 attitude scales. As a supplement,
the
principal
component
analysis
was applied
to the
rating data.
Results
oisti
~ ewous ject :iabl
are shown in Figure 3. Dimension 1 and 2
ntuiti
~lli
showed about 40% of
;itiv
the total variance and the
configuration
of
the
scales
is
reasonable,
ive
which
means the personality of the speaker was adequately recognized by
the
without
opera
subjects
any
special
biases. X,Ca~m 3.3
Result
on
the
attitude change First of all, the
Figure 3. Result of the principal component analysis of the evaluation scales. The figure shows the configuration for dimension 1 and 2.
analysis was done to see if there is any previous differences among the attitude of the groups of subjects. The onefactor analysis of variance was applied to the data of each of 9 attitude scales obtained before the presentation.
The results are shown in Table 1. The probability ranges from
0.2805 to 0.9310 which means it can be said that there were not any differences among 5 subject groups for any of 9 scales before the experiment. For the purpose of analyzing the attitude change after the presentation, two-factors analysis of variance was applied to the difference of the data before and after the presentation.
The results are shown in Table 2.
Most scales revealed no significant
differences but the scale 3.1 showed the p=0.0716 (<0.1000) for the size factor and the scale 3.2 showed p=0.0098 (<0.0100) for the resolution factor.
These scales seemed to show
significant change of the attitude. But the inspection of the original data suggested that for the scale 3.1 there were just two exceptional subjects among ten subjects who showed quite large amount of differences. This suggests the possibility that the significance for the scale 3.1 can be attributed to the artifact and may nor reflect the true tendency among the
171
Table 1 Analysis of Variance" Pre-experiment Attitutde S c a l e 1 2.1 2.2 2.3 2.4 3.1 3.2 3.3 3.4
Nparm 4 4 4 4 4 4 4 4 4
DF 4 4 4 4 4 4 4 4 4
Sum of Squares 4.3700000 8.9300000 11.570000 8.7300000 16.130000 12.330000 8.3500000 3.8300000 1.7700000
F Ratio 0.5405 0.6998 0.8352 0.6330 1.3107 1.1431 0.7818 0.2918 0.2110
Prob>F 0.7067 0.5961 0.5101 0.6415 0.2805 0.3486 0.5430 0.8818 0.9310
Table 2 Analysis of Variance" Difference of Attitude Sc a l e s 1 Size Resolution Size*Resolution 2.1 Size Resolution Size*Resolution 2.2 Size Resolution Size*Resolution 2.3 Size Resolution Size*Resolution 2.4 Size Resolution Size*Resolution 3.1 Size Resolution Size*Resolution 3.2 Size Resolution Size*Resolution 3.3 Size Resolution Size*Resolution 3.4 Size Resolution Size *Resolution
Nparm 1 1 1 Nparm 1 1 1 Nparm 1 1 1 Nparm 1 1 1 Nparm 1 1 1 Nparm 1 1 1 Nparm 1 1 1 Nparm 1 1 1 Nparm 1 1 1
DF 1 1 1 DF 1 1 1 DF 1 1 1 DF 1 1 1 DF 1 1 1 DF 1 1 1 DF 1 1 1 DF 1 1 1 DF 1 1 1
Sum of Squares 8.1000000 1.6000000 0.9000000 Sum of Squares 0.10000000 0.40000000 0.10000000 Sum of Squares 0.9000000 2.5000000 2.5000000 Sum of Squares 0.02500000 0.02500000 0.62500000 Sum of Squares 1.6000000 2.5000000 0.4000000 Sum of Squares 3.6000000 1.6000000 1.6000000 Sum of Squares 1.225000 18.225000 0.225000 Sum of Squares 1.2250000 0.0250000 1.2250000 Sum of Squares 0.22500000 0.62500000 0.02500000
FRatio 2.7561 0.5444 0.3062 FRatio 0.1241 0.4966 0.1241 FRatio 0.6045 1.6791 1.6791 FRatio 0.0084 0.0084 0.2109 F Ratio 0.6590 1.0297 0.1648 F Ratio 3.4468 1.531 9 1.5319 F Ratio 0.4994 7.4304 0.0917 F Ratio 0.8596 0.0175 0.8596 F Ratio 0.1419 0.3940 0.0158
Prob>F 0.1056 0.4654 0.5834 Prob>F 0.7266 0.4856 0.7266 Prob>F 0.4420 0.2033 0.2033 Prob>F 0.9273 0.92 73 0.6488 Prob>F 0.4222 0.3170 0.6872 Prob>F 0.0716 0.2238 0.2238 Prob>F 0.4843 0.0098 0.7637 Prob>F 0.3600 0.8954 0.3600 Prob>F 0.7087 0.5341 0.9008
172 subjects.
The
scale 3.2 had no
problem
with regard to this
aspect
1.6 1.4 1.2 1.0 -~ 08 t~ o
and this result coincides with the hypothesis
0.6 0.4 0.2
:~ 0.0 ,~ -0.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ,~ -o.4
that the infor-
-0.8
mation in the
- 1.0
110/HDTV
presentation
110/NTS C
....................................
55/NTSC
Direct
Figure 4. Amount of the attitude change for item 3.2.
has affected to the
55/HDTV
~
same
aspect of the
B
attitude.
B Natural
The
5.0
Realistic
result for the scale
3.2
in
Figure 4.
4.0
is
shown
This
result suggests
Clear
o
>t~
3.0
O m
t~
@
2.0
that higher the resolution, the more
1.0
people
are affected in their attitude.
0.0
.:-x-:.:
,~.',~-:-:
4:;:.:;:
::::~::::: 1
ll0/HDTV 55/I-IDTV ll0/NTSC 55/NTSC
Direct
Figure 5. Rating values on the presentation quality.
3.4 Result on the quality of presentation Two rating scale values, clear and realistic, showed significant differences (p=0.0579 and p=0.0748) for the screen size.
This means that the screen size affects to the
impressional aspects of the presentation. The result is shown in Figure 5. 4. C o n c l u s i o n The effect of the video conference system was studied experimentally using the paradigm of the attitude change. Based on the result, it was found that the resolution has a strong effect over the attitude change whereas the screen size is influential on the impressional aspects of the audience. This type of psychological study was revealed to be effective in determining the relationship between the physical parameters and their mental effects of the user.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
173
R e - s a m p l i n g of 3 - D o b j e c t r a n g e d a t a b y c u b e - b a s e d s e g m e n t a t i o n S.J. WANG, Y. CAI, M. SATO and H. KAWARADA Precision and Intelligence Laboratory, Tokyo Institute of Technology, 4259 Nagatsuta-cho, Midori-ku,Yokohama, 227 Japan This paper proposes an efficient method for scattered range data re-sampling using uniform segmentation in 3-D space. The method combines the techniques of uniform cubebased segmentation, patch formation, and lattice points generation. After analyzing the scattered range data which formed by optical measuring machine, a preliminary processing is carried out by making the cross section in 3-D object space to form a group of cubes. Based on the cubes from preliminary processing, we generate lattice points in each cube. So using fewer re-sampling points, it is possible to reconstruct a smooth and complete 3-D object with parametric surface equations. 1. I n t r o d u c t i o n 3-D computer graphics(CG) applications and user interfaces offer a new qualitative step in man-machine interfaces by relating to the human's natural familiarity to live in a 3-D world. It provides us with information that makes it possible to locate and recognize objects and their interrelationships without direct physical contact. One of the main problems in virtual reality research is how to generate 3-D objects approaching to a real object. However, modeling of 3-D object in CG is too complex to be done by manpower. In another words, the scattered range data obtained by optical measuring machine are too enormous and discrete to be used directly to reconstruct 3-D objects and display it in real-time. Although a lot of methods of 3-D reconstruction from scattered range data were proposed, almost of them are the methods of plane interpolation. The methods of triangle patches [1]- [2] and the bilinear plane patches [3] proposed up to now have the problem that it is difficult to obtain the natural and high quality 3-D images if the number of extracted points are not enormous enough. Nagashima et al.[4] employed a method by inputting the control points to B~zier surface equations to reconstruct 3-D models, it is inconvenience since the control points have to be selected by manpower. In this paper, an efficient method for scattered range data re-sampling using uniform segmentation in 3-D space is proposed. A preliminary processing is carried out first by making the cross section in 3-D object space to form a group of cubes, and the scattered range data were segmented into each cube. By re-sampling the data in each cube, the lattice points are generated. Employing this technique, it is possible to reconstruct a smooth and complete 3-D object with parametric surface equations using fewer re-sampling points. The experimental results proved that the proposed method is efficient.
174 2. U n i f o r m cube s e g m e n t a t i o n 2.1. C u b e g e n e r a t i o n From the measurement scattered range data expressed as {Di(xi, Yi, z~)li = 1, ..., g } , the position and volume of 3-D object outline can be obtained. Then by making cross section along X, Y, Z axis in 3-D object space, the object is segmented, and a group of cubes are generated. As a result, the scattered range data are separated and located into each cube as shown in Figure 1. The segmented data in each cube are called patches. The sizes of the cubes are determined by
A:= W:In: } A,, - W,,In,, , A:= W:I~:
(1)
where Wx, Wv, and Wz are the sizes of the outline volume and nx, nv, nz are segmentation thresholds in X, ]I, Z directions, respectively.
Z
I
/w[:
w,
X
Figure 1 The cube-based segmentation of 3-D object.
iiiiiiiiii!iiiiiiiiiiiii ,,, (d)
(e)
(t)
Figure 2 The forms of patches in cubes.
2.2. F o r m of patches in cube The patches segmented in cubes are polygonal surface patches. Scattered range data re-sampling means dealing with these surface patches. Assume that a 3-D object is segmented fine enough, and the patches are restricted to six kinds shown in Figure 2. They are triangle ((a)), quadrangle ((b), (c)), pentagon ((d)), and hexagon ((e), (f)). 2.3. R e - s e g m e n t a t i o n of patches As the re-sampling results, the quadrangle or triangle patches will be formed since it is required by Spline or B~zier patches. That means the re-sampling can be carried out directly for triangle and quadrangle patches, and the re-segmentation has to be done if patches are pentagon or hexagon ones. The pentagon shown in Figure 3(a) is re-segmented into a triangle and a quadrangle patches. There are five possible re-segmentation forms. As a rule, the distant between
175 two nonneighbor points is calculated first, then the new boundary is decided by selecting two points with the shortest distance. The solid line B D re-segments the pentagon into two patches. The same rule can be used for the case of hexagon as that shown in Figure 3(b). The solid line B E re-segments the hexagon into two quadrangle patches. After re-segmentation, the obtained new patches can be re-sampled just as that done in the case of triangle or quadrangle patches. 3. G e n e r a t i o n of r e - s a m p l i n g p o i n t s 3.1. Basic a l g o r i t h m of r e - s a m p l i n g The distribution of scattered range data in a patch is shown in Figure 4. n presents the unit normal vector of the surface patch, and O is a point placed in the outside of the surface patch. A line L can be determined by n and O, P(t)
=
(2)
O + n . t,
where t is a parameter. Selecting three points which are the nearest to the L among scattered range data, a plane can be made as
(3)
( P ( t ) - D 1 ) . [(D2 - D1) × (D3 - D1)] = 0,
where D1, D2, D3 are the selected range d a t a . Substituting eq.(2) to eq.(3), (O + n . t - D 1 ) . [(D2 - D1) x ( D 3 - D1)] = 0
(4)
is deducted. Solving equation (4), parameter t can be found. Substituting t into equation (2), an intersection point P which is called re-sampling point is obtained. Further more, changing the point O in the orthogonal direction of n, the new intersection point can be generated. It is shown that once n and O are determined, the re-sampling points can be obtained. The following discussion is how to determinate n and O for six kinds of patches shown in Figure 2.
A
A n
Bi\~'7-~\"//
B
C
D
(a)
(b)
Figure 3 Re-segmentation of pentagon and hexagon patches.
Figure 4 The computation of a re-sampling point.
176
3.2. Generation of re-sampling points on boundary of patch In order to join the adjacent surface patches continuously, common re-sampling points on boundaries between adjacent patches are necessary. The points P1, ...,P6 are the apexes of the two adjacent patches in Figure 5. They are obtained re-sampling points on the edges of cubes. Assuming that M x M re-sampling points are required in the patch, so M re-sampling points are needed on boundary. At first,using P1 and P4, the M points can be determined on the line segment by
oi=el+(i-1)
×
P4 - P1
M-1
where Oi are the points on b are defined as
n~
=
nb
=
'
(5)
PIP4, i =
1, 2, ..., M. The normal vectors of the patches a and
(P3 - P1) × (P2 - P4) If(P3 - P1) × ( P 2 - P4)[I' (P4 - P6) X (P1 - P5) i [ ( p 4 _ P6) x (P1 Ps)II" -
(6) (7)
-
Using obtained normal vectors, the unit normal vector about the re-sampling points on the boundary is n a -k- nb
n -
IIn. + nbll"
(S)
With the obtained Oi and n, the M re-sampling points can be generated according to the equations (2)-(4). In the case of triangle patch, the patch is considered as a quadrangle patch which degenerates a boundary to a apex called degenerative point. The degenerative point is decided as the apex whose opposite side is the shortest one in the triangle. Here the degenerative point is considered as M common re-sampling points.
P1
PII
P ~ n ~ ~ ~ - ~ ~ P2
~
PIM p~j-'~
P,
P,
Figure 5 The generation of re-sampling points on boundary.
PMM
Figure 6 The generation of re-sampling points inside patch.
177 3.3. G e n e r a t i o n of r e - s a m p l i n g p o i n t s inside p a t c h Based on the re-sampling points on boundary, the generation of re-sampling points inside patch is carried out. For a re-sampling point P # in Figure 6, M points are obtained on the line segment of PilPiM and P l j P M j , respectively, given by ,
O#=Pil+(j-1)
x
,,
O#=Plj+(i-1)
x
PiM -- Pix M-1
PMj -- Pli M-1
(9)
' '
(10)
where P i l , PiM, Plj and PMj are the re-sampling points on boundaries of the patch, i, j - 1, 2, ..., M. Using O# and O#, the point O# can be determined as t
I
it
It
Oij = O# + 0 # . 2
(11)
The unit normal vector nij about the Pij is decided by the equation of
(PiM -- Pil) x (Pli - PMi)
(12)
n,j = [[(P,M - Pil) x (Pxi - PMi)][" Using O# and n#, the M x M re-sampling points P # can be generated according to the equations (2)-(4) by c h a n g i n g / a n d j from 1 to M. In the case of triangle patch, the method mentioned above also can be used by considering the triangle as a quadrangle patch with a degenerative side, just as that discussed in 3.2. 4. E x p e r i m e n t a l r e s u l t s For quantitative evaluation, an experiment has been done using scattered range data of a car model. The number of original scattered range data of the car is 70421. Figure 7 shows the 3-D range data of the car, whose size is 43.4 x 16.8 x 6.8(inch3). The outline volume enclosing car is 49.6 x 22.4 x 8.0(inch3), the number of cubes is 32, and the size of one cube is 6.2 x 5.6 x 8.0(inch3). The wireframe car model, which is the re-sampling result, is shown in Figure 8. The intersections of mesh act as the re-sampling points. The number of patch is 32, and there are 4 x 4 re-sampling points in one patch. The total re-sampling points are about 400. The shading representation of the car model, which is reconstructed with bicubic B~zier surface equation from re-sampling points [5], is shown in Figure 9. The evaluation of root mean square error(RMSE) of the reconstructed car is shown in Figure 10. 5. C o n c l u s i o n
This paper presents an novel approach in which scattered range data of 3-D objects are re-sampled by uniform cube-based segmentation. The proposed technique for generating lattice points can be considered as an interface between optical measuring machine and 3-D object reconstruction or modeling. Starting from scattered range data, a group of cubes are generated, and re-sampling for the data located in each cube is carried out. As a result, the patches with lattice re-sampling points are formed, and the 3-D object can
178 be represented by these points. Also the enormous scattered range data are compressed greatly. It is very useful for high speed display in the application of virtual reality.
Figure 7 The scattered range data of a car model.
Figure 8 The re-sampling result of the car model. '
"~-mod~'--
0.4 m
0.3
0.2
00
Figure 9 The shading of reconstructed car.
40
80
120
160
200
Figure 10 The RMSE evaluation of reconstructed car.
REFERENCES 1. W . T . Zheng and H. Harajima, Surface Representation Based on Invariant Characteristics, Technical Report of IEICE, IE93-123, pp. 31-38. (1994) 2. H. Nisino, K. Akiyama and Y. Kobayasi, Acquisition of 3-D Object Shape Using SlitRay Projection and Reconstruction of Surface Model, Trans. of IEICE, vol.J72-D-II, no. 11, pp. 1778-1787, (1989) 3. A. Amano, Y. Sakagukchi, M. Minoh and K. Ikeda, 3D Model Generation from A Range Image by Regularization, Technical Report of IEICE, PRU92-58, pp. 1-8. (1992) 4. S. Nagashima, K. Suzuki and S. Nagano, A System for Human Bodies Measurements with Free-Formed Surfaces, IPSJ SIG NoteS, Graphics & CAD 38-3, (May 1989) 5. S. J. Wang, Y. Cai and M. Sato, Modeling of 3-D Objects by Re-sampling Range Data, IPSJ SIG Notes, Graphics & CAD 73-2, (Feb. 1995)
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
179
Harmonic Curve Design Dr John R Rankin LaTrobe University Australia ABSTRACT
Many aesthetically appealing curves can be generated by using epicycloid and spirographic parametric curve equations. The difficulty in obtaining an interesting and appealing curve comes from having to choose good values for the coefficients in these equations. If a poor choice of coefficients is made the curve generated will have little regularity and a high amount of confusion. Since these curve types involve up to ten real coefficients the set of aesthetic curves is a set of measure zero on the infinite coefficient space. Rather than making random searches through coefficient space another approach which proved to be far more productive in discovering curves of aesthetic appeal was investigated. In this approach we looked at the Fourier Transform of the curves and performed editing on the coefficients in Fourier space. This approach is especially suited to the epicycloid and spirographic curve types which turn out to be simply low-order harmonic functions after suitable filtering and a rotation. As a result, curves can now be defined by free-form input and then cleaned up by this method to yield the nearest epicycle or spirograph. 1. I N T R O D U C T I O N Aesthetic curves involving cycles and spirals have long been used in motifs, logos and elementary graphics artwork. Celtic artwork, in particular weaved knotwork, has considerable appeal and is used widely (see Figure 1 taken from [4]). An analysis of this sort of artwork is given in [4] however no naathematical prescription is provided for generating and generalizing these patterns. The families of mathematical curves we concentrated on for this research were the spirographs [3] and the epicycles [1] which are 3 and 10 parameter families respectively. Many of the Celtic knotwork patterns of interest have strong similarities to spirographs and epicycles which are mathematically based. A number of books have described how to draw spirographs and epicycles [1,3] and catalogued some of their variety I1,21. When one begins to experiment with generating new designs of this sort one quickly discovers that the desirable or good shapes require very careful selection of the coefficients in the equations. A randomly selected set of coefficients yields drawings that contain very little repetition and far too much detail for the eye to be content with or else don't match other target criteria. One way of gaining an overview of patterns possible within these families is to build graphics animations that show sequences of curves in a family. In order for this to be achievable the curve parameters have to be taken as integers and only a finite range of values can be considered. Nested loops allow the rounded parameters to assume each value over their allotted ranges and a graph is displayed for each parameter combination. Parameter increments of +1 may suggest non-smooth transitions between animation frames however on a fast machine the human eye manages to interpolate well and a good understanding of smooth real parameter variations is obtained. Animation programs were written for the spirograph and the epicycle families with facilities to freeze the animation, view current parameter values and reverse the animation direction. However the possible paths through parameter space for each of these curve families is infinite and
180 these animation programs follow p~u'ticular parameter-space paths (which are cycles). Animations using different paths produced considerably different sequences and effects. Consequently, even with the fastest animation these programs, as interesting to watch as they are, can show only a negligible or measure zero fraction of the possibilities. Additionally the chances of finding a particular desired pattern are still very low. Another approach to the problem of locating desirable patterns is to start by precisely define an aestheticity measure [6]. The criterion can include non-fuzzy and fuzzy features such as the desire for symmetry, no sharp corners, uniform space-filling appearance and no triple or higher self-intersections. Once one has such a measure a hunt program can be implemented to optimize the objective function. This is a program that searches through all of parameter space (again only finite subranges of integer values for the parameters are chosen) selecting parameter combinations that meet the criterion and saving them in a file. Since the search is a long process the hunt program should be interruptible such that it can be resumed from the same place at a later time. Such a program requires special design and a hunt program was implemented by the author in a similar context (to solve a multi-parameter Diophantine equation for the design of a graphic character). A companion program reads through the file of solutions found and displays them one after the other without stopping but allowing the user to pause and query any solution. However, generally such a measure is difficult to express mathematically. Again the chances of finding a particular desired pattern are very low, it could take a long time to find and in any case the integer subset of parameter values might not even include the desired solution. The third approach investigated is to allow the user to describe to the computer roughly what the required pattern should look like by a free-form user-input drawing and then have the program analyse this drawing for closest match to the desired curve type. This is the approach taken for the research presented here. A program is provided that allows the user to sketch the target curve using the mouse input device. Mouse drawn curves are typically inaccurate in expressing the user's desired curve. The input data is then Fourier analysed by a fast fourier transform. After this a filter is applied to the results in Fourier-space and then transformed back into the original space. After a coordinate transformation the required spirograph or epicycle is obtained. This method does not restrict the curve parameters to integer values over a finite range and does not have uncertainty over finding a solution or cause long delays in finding the solution unlike the previous two approaches. It is therefore far more powerful than the other two approaches and worthy of a full investigation. Once the parameters for spirograph or epicycle curves are obtained, further operations can be performed on these curves such as banding and interlacing as described in section 2. 2. T A R G E T C U R V E F A M I L I E S The curves to be rendered are the spirograph and epicycles. The parametric formula for spirographs is [3]:
181 /?. x(O) = (R R + R w )cos(0)+ Rp cos(-z-c-0) Rw
(la&b)
y(O) = ( R R + Rw) sin (0) + Re sin(RR 0) Rw
This is a three-parameter family of curves each one forming a loop over the range 0 = 0 to 2~Nwwhere Rw= otN w, R R = o~NR and N R and N w are relatively prime integers. All curve parameters are real but the ratio RR/R w should be rational. The parametric formula for epicycles is [1 ]: x ( 0 ) = R, cos(F~0) + e2 cos(F~0) + R~ cos(F30) + R, cos(F,0) + R5 cos(Fs0)
(2a&2b)
y ( 0 ) = e I sin (El0) + e z sin (F20) + R3 sin (F30) + R 4 sin (F40) + R 5 sin (Fs0) This is a ten-parameter family of curves each one tbrming a loop over the range 0 = 0 to 2~. The curve parameters R i are real and F~ are integers (i = 1 to 5). Spirographs are a special case of epicycles. A spirograph or epicycle can be converted to look like a piece of Celtic knotwork by banding and interlacing. (The banding and interlacing algorithms will be presented elsewhere.) These operations are shown in Figure 2. However, not all spirographs or epicycles are suitable for this - some cannot match with Celtic art as for example the epicycle in Figure 3. Likewise not all Celtic weaving knotwork can be matched to spirographs and epicycles. The Fourier analysis described in the next section enables us to find the closest available spirograph or epicycle.
3. FOURIER ANALYSIS OF CURVES The Fast Fourier algorithm is described in [5]. It enables one to break down any periodic input curve into its frequency components. It was suitable here to use the complex discrete Fourier transform with z(0) = x(0) + i ' y ( 0 ) where i = {(-1). Taking N samples z k, for k = 1 to N, these are transformed into frequency amplitudes z'j by the Fast Fourier Transform. The original curve can then be recovered by the Fourier expansion: 1 Zk
"-
-1
ZZ'j+I+ N - N j=-N/2
e 2~ijk / N
1 N/2 "+"- - Z Z' j+l N j=0
e 2~djk/ N
(3)
where k = 1 to N. Tests were performed initially whereby spirographs were Fourier transformed and then regenerated from the Fourier expansion, equation 3 above. These tests all showed the regenerated points as visually very close to the original input set.
4 FREE-FORM INPUT A program was written to allow a user to design a curve on the screen using the mouse. The points digitized along the free-form curve (xk,yk) for k - 1 to N, were then stored in a file. These points were then read in to the Fast Fourier transform program and transformed to frequency amplitudes z'j for j - 1 to N. The user-curve was then regenerated from these coefficients (by equation 3) and displayed. Next the negative
182 frequencies were removed, i.e z'j set to zero for j = N/2+l to N giving the filtered amplitudes z"j" When the curve was regenerated from z"j the non-smooth free-form nature of the curve was still evident and the regenerated curve was not a spirograph or epicycle. The next operation in Fourier space was to remove the DC zero frequency component and to select out only the D dominant frequencies with D = 2 for spirographs and D = 5 for epicycles. Next the amplitudes were replaced by the root-mean-squared amplitude sign adjusted to retain the original signs of the real and imaginery parts to yield the new amplitudes z'"j. Interpreting the real and imaginery components of z"'j as the cartesian coordinates still does not result in the equations for spirographs or epicycles (equations 1 and 2 above) and the following scaled rotation transformation of coordinates is required:
x=(x'"+y'")/2 y = (y'"-x'" )/ 2
(4)
where x'" and y"' are the real and imaginery parts of z'".
5 RESULTS Figure 4 shows a sample free-form input consisting of N = 100 points. When these were transformed using the Fast Fourier transform and then reconstructed using equation 3 above a virtually identical set of points was obtained. When negative and zero frequencies were removed and components replaced by signed rms values the regenerated curve appeared as in Figure 5. This is clearly close to the desired spirograph. After the rotation of coordinates (equation 4 above) the desired sprographic curve was obtained (Figure 6). This was then banded and interlaced as shown in Figures 2b and 2c. Further work is progressing with the hand input of more complex curves. 6. CONCLUSIONS Fourier analysis of free-form mouse input can provide an intelligent interface for user input. Using suitable frequency filtering and transformations in Fourier space described above the nearest available smooth curves to the spirograph or epicycle curve families can be obtained. This is a much more efficient way of obtaining particular curve shapes than searching through parameter space. This research suggests that Fourier analysis of free-form user input can be used to smooth user-designed graphics in a pleasing way which can assist in curve design and user-interface friendliness overcoming the drawing limitations of input devices such as the mouse and even couter-balancing for the usual lack of drawing skills in the user.
REFERENCES 1. Edwards, R "Microcomputer Art", Prentice-Hall of Australia, 1985. 2. Guest, Julius "New Shapes, A Collection of Computer-Generated Designs", published by Robin A Vowels, RMIT, Melbourne, 1979. 3. Rankin, J "Computer Graphics Software Construction", Prentice Hall Australia, 1989, pp 79-82. 4. Bain, George, "Celtic Art, The Methods of Construction", Constable London, Harvey Menzies Johnston, 1977, pp 28, 36.
183 5. Conte, S and de Boor, C, "Elementary Numerical Analysis An Algorithmic •Approach", McGraw-Hill International, 1981, pp 268-283. 6. Stiny, George and Gips, James "Algorithmic Aesthetics, Computer Models for Criticism and Design in the Arts", University of California Press, 1978.
Figure 1. Celtic weaving patterns with similarities to epicycles.
Figure 2. (a) A simple spirograph. (b) The result of applying the banding algorithm. (c) The result of applying the interlace algorithm.
Figure 3. An epicycle unsuitable for representing Celtic art or banding.
184
Figure 4. Free-form input (diamonds) with the FFr regenerated points (+ markers).
~~.~~0
O0
Figure 5. The curve generated from the positive rms FFT frequencies on the data.
.L ...............
_, . . . . . . .
,
.
,
....
, . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
,...,....,
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
, . , . ,
. . . . . . .
|,,,,,
~.,
Figure 6. Rotated and rescaled spirograph derived from the free-form input of Fig. 4.
III.8 Screen Design 3
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Moil (Editors) © 1995 Elsevier Science B.V. All rights reserved.
187
GUIs and SUIs: more of the same or s o m e t h i n g different? Alison Black a and Jacob Buur b aIDEO Product Development, 7/8 Jeffreys Place, London NW1 9PP, UK bDanfoss A/S, DK-6430 Nordborg, Denmark
1 INTRODUCTION Solid user interface (SUI) is a term coined in Japan to distinguish the user interface of products with embedded microprocessors like video cassette recorders, photocopiers and cellular phones from the Graphical User Interfaces (GUIs) of computer applications. SUIs are a neglected area in HCI research despite the substantial revenue earned from sales in both industrial and consumer applications. The great potential of SUIs that are easy to use is evident from the success of products such as the VideoPlus (VCRPIus) programmer, which has reduced the mystery of video programming in households world-wide. Fortunately interest in the usability of SUIs is growing [3,14]. Many factors may have contributed to this shift: • HCI has evolved from large-scale systems, through personal cornputing, to more recent (and yet to be consolidated) personalised SUI devices, such as Personal Digital Assistants. • Industrial products with SUIs, which were once stand-alone devices used in remote workplaces (and often by blue-collar rather than white-collar workers), are now being integrated into organisational data handling systems. For example, stand-alone refrigeration case controllers in supermarkets are now linked, allowing computer control of a complete store, and even chain-wide monitoring from a central headquarters. So problems of SUI use are now set in the wider context of essential computing systems. • Design for usability has not kept up with the dramatic increase in functionality made possible by increasing sophistication and miniaturisation of electronic components, to the point where there is a crisis in usability that simply cannot be ignored [12]. • While feature promotion and price wars still dominate industrial and consurner markets for products with SUIs, usability is becoming a rneans of securing competitive advantage [9]. For example, in developing a new TV control interface Nokia Consumer Electronics adapted their product development process, bringing in expert usability consultants, in order to a c h i e v e a highly usable interface that would distinguish them from their competitors [1]. Despite the trend towards integrating products with SUIs and computer systems, we believe the distinction between GUIs and SUIs is worth preserving because user interaction with SUIs differs significantly from interaction with GUIs. And for many industrial and commercial applications SUIs offer distinct advantages over GUIs. The objectives of this paper are to encourage increased attention to SUIs in the research community. We will identify characteristics that differentiate SUIs from GUIs and highlight the different methods for analysis, design and testing that SUIs require.
188 2 SUI CHARACTERISTICS: CONSTRAINTS AND OPPORTUNITIES While user interaction with GUIs is supported by a mouse supplemented by a keyboard for input, and a high resolution display for feedback, SUIs are typically limited to a restricted set of buttons and small, low-resolution display. To understand the opportunities these offer, and constraints they impose we need to investigate SUI characteristics systematically. 2.1 W h a t u s e r s s e e
One button one function When buttons are dedicated to single functions the SUI is likely to be relatively transparent to the user as long as there are not too many buttons, their functions are clearly marked or can be learned and remembered, and the buttons are differentiated adequately in position, appearance and feel. However with the high number of functions offered by many microprocessor products, the one b u t t o n / o n e function solution is not viable. It may take up more surface space than is available on the product (both for the buttons and their identifying labels); it is likely to be expensive to manufacture; it will at best be off-putting to users; at worst it will lead to errors in use if the physical design does not allow for logical arrangement and differentiation of buttons with different functions. Hidden buttons One of the solutions most frequently adopted to solve the problem of too many buttons is to hide infrequently-used buttons under flaps in order to present an apparently simple interface (corresponding to unopened windows on a GUI display). Some users, however, may never find these hidden buttons; or even if they know they are there, be unable to find or open the flap to access them because the industrial design of the SUI does not cue interaction effectively. Unlike windows there is no equivalent of using a search routine to find out where the buttons are, nor any convention of clicking to open them.
Hidden functions An alternative way of reducing button count is to attach several functions to each button: either by double (or triple) sets of labels with a means to shift among them; or, alternatively, by soft keys - buttons tagged by variable labels that either change automatically, according to the stage in an interaction (the buttons are modal), or which users can scroll through to find the function they need. Modal interaction underlies usability problems in both GUIs and SUIs, particularly for products or functions that are used relatively infrequently. If it cannot be avoided then users need to be prompted that the status of function keys has changed. But available cues are likely to be limited given that in many SUIs variable information is presented on graphically-limited liquid crystal displays.
Limited alphabetic input SUIs are severely restricted in their capacity for alphabetic input. Neither of the standard options - cursor input or modal 'international' keypads - suit entry of more than a few characters; and solutions such as soft keyboards on SUI displays, or miniaturised attachment keyboards have some enthusiasts, but tend to the 'swiss armyknife' approach of providing partially effective solutions to interaction problems [16]. There is potential for voice input, especially as this increases users' freedom to use products while on the move, or in environments where space is constrained. But voice activation and control is fraught with difficulties for users [13] and speech analysis of any level of sophistication is likely to require processing and memory capacity well beyond the typical SUI product.
Limited display feedback SUI displays tend to be small with either segmented digits or only a few lines of dot matrix characters. If full graphics are used they are likely to be constrained in size and resolution. So displaying overviews (a common feature of large, high resolution
189 GUI displays which helps users find functions, or understand how presently displayed information relates to the recent sequence of activity with the product) is simply not possible. Use of additional cues to change of status, such as colour, sound or illumination is often limited by the need to conserve memory capacity or power and, of course, cost. 2.2 What users don't see
Although to users the most obvious differences between GUIs and SUIs lie in their external interface components, limited processing capacity and power supply have an impact on how those components can be deployed in user interactions.
Limited processing power Products with embedded processors tend to be more limited in power and program size than those driven by central processing units, so considerable ingenuity is required to make the most of opportunities for feedback and responsiveness to user interactions. For example, when there is insufficient processing capacity to drive a fully flexible display, specially designed icon characters may be used to give graphic visual feedback, with considerably less demand oi1 memory than full graphics.
Limited power supply Although not all products with SUIs need to be battery powered, those products which have to be portable or pocketable need small batteries. Minimising battery weight can be critical for ease of use if a product must be held single-handedly, and must be balanced and stable in the user's hand. Compactness also often helps to sell these products, and, up to a point, makes them more usable. At the same time small batteries restrict user feedback through illuminated displays and keys, and sound cues. Battery technology has not kept pace with the advances in electronics that have significantly reduced product size and cost at the same time as increasing functionality. 2.3 D e s i g n opportunities
Despite both internal and external constraints the conlbined soft/hard SUI interface presents opportunities for promoting interactions that are easy to learn and carry out.
More than just push buttons If input components are custom-designed or dedicated to particular functions, SUIs can extend beyond standard push buttons to exploit fully the hands' and fingers' capabilities. Turning knobs, rollers, sliders, pull rods etc. allow precise and intuitive adjustment, often signalling a position or state more efficiently than display graphics can. Research is underway to incorporate tactile and force feedback into input components, and the consumer and automotive industries have already seen a shift back from 'digital' button input to more responsive knobs and dials.
Supporting ergonomics of operation Since SUIs can be custom-designed, there is freedom to shape the physical form of the interface to fit users' hand and body postures. This opportunity is being exploited for hand-held measuring equipment, ticketing devices etc. Supporting functional understanding Custom design also means the layout of the interface can be planned to communicate a sequence of operation through fixed functional grouping. Combined with labelling and graphics, this can promote understanding of use in a way that is sometimes not as effective with more flexible GUI displays. Flexible buttons A novel approach, yet to be developed fully, but which could simplify modal interaction, is to change the buttons, rather than the labels, according to interaction mode.
190 With a flexible casing material, buttons required at any point of interaction can be shaped and raised from the surface when needed by means of active, computer-controlled elements underneath; then removed from the perceptible interface when they are not required [7].
Product identity Corporations have used (and defended) graphic identity to reinforce the 'look and feel' of their software products. Their motive is to boost sales by differentiating their products from competitors', but also to facilitate transfer of learning to new products, thereby increasing customer experience of ease of use. SUIs provide additional opportunities to build product identity by exploiting both three-dimensional and graphic qualities in user interactions. Carried across a range of products SUIs can literally have an identifiable 'feel' that communicates product features and functionality very directly to new users.
3 SUI DEVELOPMENT: DESIGNING FOR MAXIMUM USABILITY
There are many products with SUIs that frustrate users through poor design (video recorders are often cited). Poorly designed SUIs may lose revenue for their developers (if self-service machines are difficult to use potential customers may reject the service). And in safetycritical tasks and environments (transport, medical care, industrial systems maintenance), poorly designed SUIs can be positively dangerous. So organisations developing SUI products need to focus their development processes on achieving maximum usability. 3.1 User observation and testing In many cases software engineers are not good designers of interfaces for other users who do not share their specialist experience of the product [4]. So organisations developing products with either GUIs or SUIs need to research their users' needs and to test prototype products with users as early and as often as they call [5]. Organisations developing both industrial and commercial products with SUIs have tended to lag behind GUI developers in fostering contact with end-users. They may have a tradition of close communication with their customers via their sales force who have built relationships with clients over many years. But this dialogue often focuses on product price and features (compared to those offered by competitors), unless end-users have had difficulties or made complaints. So these organisations still have to make specific efforts to research products beyond the purchasing decision, in their environment of use [6]. Specialists in human computer interaction have developed innovative methods for observing and evaluating users' interactions with GUIs, and these days many software development houses have 'usability labs'. But lab methods are not always easily transferred to products with SUIs. Typical GUI applications (such as spread-sheets) tend to be the focus of work in relatively static office environments, which can be partially simulated in lab conditions. SUIs, however, are often incidental tools for mobile work (such as industrial control devices) or leisure activities (electronic games) where user interactions cannot be replicated well outside the real use environment. Designers need to make sure they engage with end-users in their own environments (albeit sometimes inhospitable or private) in order to understand user requirements from the earliest stages of product development. Similarly as designers develop prototypes they must test them in the environment in which the products will be used. If there are criticisms that models of user behaviour fail to capture the local contingencies that trigger users' actions [15] this is likely to be particularly so for
191 products and users whose natural domain is far removed from the usability lab. New methods are required to ensure a good fit between innovative products and their proposed environment; for example, 'Inforrnance' where designers act out future user scenarios in realistic settings, to understand and communicate the impact of new designs [2]. 3.2 Product prototyping There are many tools for rapidly prototyping GUIs to yield a close resemblance to the final products, and there are well-established techniques for recording from these tools in usability labs in order to develop detailed profiles of product performance [11]. In contrast, the combined hard and soft interaction with SUIs is more difficult to prototype. SUIs can be partially represented on touch screens but at some point prototypes representing the dimensions and weight of tile solid interface must be made because the real arrangement of the components influences the smoothness of user interactions. Three-dimensional modelling of products is more expensive and less flexible than screenbased modelling, so econornising by producing a series of approximations to the finished product is generally accepted. It can also be reassuring to practitioners working with SUIs to find that much of the data captured by direct recording from GUI prototypes in usability labs is never analysed [11]. So the difficulty of preparing three-dimensional prototypes need not always be compounded by the requirement to record directly from them. 3.30rganisational processes for product development Usability practitioners often complain that they lack opportunity to contribute to product development or that the opportunity comes too late in the development process [10]. If this is the case in GUI development it is even more so for SUIs, where commitment to tooling schedules mean the hard interface is fixed early on in the development process. For hard and soft interfaces to work together well, design of the two must be coordinated, so typical product development cycles may need adjustment to bring the two into alignment. There are many examples of failures to coordinate the hard and the soft. But successes, such as the Mercury one-2-one phone, which links a hard function key to soft, modal cues on the display, by means of a moulded 'lens' detail, testify to the power of coordination [8]. Interactive rendering tools, such as MacroMedia Director and Alias can help link hard and soft before commitment to solid prototyping. But these tools will only be influential if used within the context of an organisational approach to design that brings hardware and software together with the common goal of serving user needs. In typical engineering company structures hardware and software products are, to some extent, independent, but in SUI users' minds, naturally enough, they are not.
4 FOCUS FOR FUTURE DEVELOPMENT IN SUI DESIGN As touch-screen technology develops, and GUI design aims for increased realism in portraying three-dimensional objects on two-dimensional screens, screen-based solutions may be regarded as a universal answer to information handling problems. They have the advantages of flexibility and relatively low manufacturing costs. But thereare many environments in which SUIs are the most appropriate technology, and a growing
192 understanding of both products and environments of use is necessary to ensure that solutions are applied appropriately. Research is necessary to establish the best organisational structures, development tools and techniques to bring about increased SUI usability. Manufacturers must become more oriented to end-users than at present, and their efforts must be supported by prototyping tools and testing that allows close replication of real use conditions. The tools must help bridge hardware and software, combining the skills of human scientists, industrial designers, graphic designers and hardware and software engineers to deliver user-centred solutions. REFERENCES 1 Black, A., Bayley, O., Burns, C. Kuuluvainen, I., and Stoddard, J. Keeping viewers in the picture: real-world usability procedures in the development of a television control interface. CHI '94 Conference Companion (1994) 243-244. New York: ACM. 2 Bums, C., Dishman, E., Verplank, W. and Lassiter, B. Actors, hairdos and videotapeInformance Design. CHI'94 Conference Companion (1994) 119-120. New York: ACM. 3 Buur, J. and Windum, J. MMI Design - Man Machine Interface. (1994) Copenhagen: Danish Design Centre. 4 Gentner, D.R. and Grudin, J. Why good engineers (sometirnes) create bad interfaces. CHI'90 Conference proceedings (1990) 277-282. New York: ACM. 5 Gould, J.D. and Lewis, C. Designing for usability: Key principles and what designers think. Communications of the ACM, 28 (1985) 300-311. 6 Grudin, J. Systematic sources of suboptimal interface design in large product development organisations. Human-Computer Interaction, 6 (1991) 147-196. 7 Harada, A. and Tamon, H. Simulating mental images through user operation. Industrial Design, 157 (1992) Tokyo: JIDA. 8 London Business School Case study of handset design & development: Mercury one2one (1994) 9 March, A. Usability: the new dimension. Harvard Business Review. Sept/Oct (1994) 144-9. 10 Mulligan, R.M., Altom, M.W. and Simkin, D.K. User interface design in the trenches: some tips on shooting from the hip. In CHI "91 Proceedings (1991) 232-236. New York: ACM 11 Nayak, N.P., Mrazek, D. and Smith, D.R. Analyzing and communicating usability data. SigCHI Bulletin 27/1 (1995) 22-30. 12 Nussbaum, B. and Neff, R. I can't work this thing! Business Week. April 29 (1991) 58-66. 13 Oviatt, S. Interface techniques for minimizing disfluent input to spoken language systems. CHI'94 Proceedings (1994) 205-210. 14 Sato, K. User interface design theory. Special feature: cutting edge on interface design. Industrial Design, 157 (1992) Tokyo: JIDA. 15 Suchman, L. Plans and situated actions (1989) Cambridge: Cambridge University Press. 16 Verplank, W. (1991) Sketching Metaphors: graphic invention and user-interface design. Friend 21: International Symposium on Next Generation Human Interface (1991) Japan.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
193
Vision-Based Human Interface System with World-Fixed and Human-Centered Frames Kang-Hyun JO, Yoshinori Kuno, and Yoshiaki Shirai Dept. Mech. Eng., Faculty of Eng., Osaka Univ. 2-1, Yamada-oka, Suita, Osaka 565, Japan E-mail: [email protected] p This paper presents a vision-based human interface system that enables a user to give 3D positioning commands by moving his hand. The user can choose either the world-fixed frame or the human-centered frame for the reference frame with which the system interprets his motion. If he uses the latter, he can move the object forward by moving his hand forward even if he changes his body position. The key for the realization of the system with this capability is vision algorithms based on the multiple view affine invariance theory. We demonstrate an experimental system as well as the vision algorithms. Appropriate uses for both frame cases are discussed. 1. I N T R O D U C T I O N Since hand gesture is one of the important means of communication for humans, research interests have been increasing to recognize hand gestures and to use them for computer-human interface. Hand gestures are used in various ways. An important one is sign language expressing some meaning by a particular hand shape and/or motion. Darrell and Pentland[1] have demonstrated a vision system to recognize such kind of hand gestures. Indicating the position and orientation of an object in 3D space is also one of the major usages of hand gestures. We concentrate on this type of gestures in this paper. Fukumoto et a/.[2] have proposed a stereo vision system where a user can point a place on the computer display screen by his hand and give some commands by hand gestures. We have also presented a system where a user can move an object in the 3D CG world by moving his hand [3]. Such conventional vision systems measure hand positions and motions with respect to the camera-centered coordinate system, that is, they interpret hand gestures in the world-fixed frame. This is appropriate in such cases where the user gives commands by hand gestures in the world where the results of the commands take effect. For example, suppose that the user asks a robot beside him to bring something by pointing it with his finger tip. The recognition system should interpret the pointed direction in the world-fixed frame without considering the position of the user.
194 There are, however, other cases that gesture recognition should take a user's position into account, such as, moving 3D CG objects by hand gestures while watching a headmount display, or controlling a tele-operation mobile robot while watching the images sent from the cameras on the robot. In these cases, it is natural and easy-to-use if the user can move the object in the 3D CG world or the robot forward by moving his hand forward. The user can do this with conventional systems if he does not move his body from the initial position. However, he cannot expect that the system can work as well when he moves. Recognition systems should interpret hand gestures considering the user position in such cases. In other words, they should obtain 3D information in the human-centered frame. In this paper, we present a vision-based human interface system that can interpret hand gestures both in the world-fixed frame and in the human-centered frame. It adopts vision algorithms based on the multiple view affine invariance theory[4,6]. We demonstrate an experimental human interface system and propose appropriate usages for the two frame cases. 2. A F F I N E I N V A R I A N T S F R O M M U L T I P L E V I E W S This section briefly describes the multiple view affine invariance on which our system is based [6,7]. We adopt the weak perspective projection as our camera model. Suppose we have a set of five 3-D points Xi, i E { 0 , . . . , 4}. We use four of these points to establish a basis vector Ei with origin X0 (see Figure 1): Ei = X i - X0,
i E {1,2,3}.
(1)
In this basis, the fifth point, and any other point, is given by X4 = X0 + aE1 +/3E2 + "yE3
(2)
for some a,/3, and ~/. The coefficients a,/3, and "y do not change under any 3D affine transformations, thus called affine invariants. They can be viewed as invariant coordinates of point Xa in the basis and are related to the 3D structure of the object.
,/X3 X2 X0
~
.......................
F i g u r e . 1. Affine basis. Since weak perspective projection is a linear transformation, it is possible to find the coefficients a,/3, and "r in terms of the projected points. For a particular view, we obtain
195
(3)
x 4 - Xo = c~el + fie2 + "/e3
where x 0 . . . x 4 are the projected points and e l . . . ea are the projections of the basis vectors. This means that we can derive two equations with three unknowns from the five point locations in a single 2D image. The problem is underdetermined. However, if a second view with known correspondece to the first view is available, we can obtain the coefficients a,/3, and -y. Hand position and orientation can be calculated based on this method. Details are found in [5]. 3. E X P E R I M E N T A L
HUMAN INTERFACE
SYSTEM
We have developed an experimental human interface system using the algorithms described in Section 2. Figure 2 shows the system configuration. The system consists of a personal computer (IBM PS/V) with an image processing board (Sharp Flex Vision), and two cameras on a computer-controlled pan-tilter. The image processing board captures images and the transputer on the board carries out feature extraction and invariant calculation. The personal computer moves 3D CG objects according to the 3D information sent from the image processing board. A CRT display or a headmount display can be chosen as a display device.
Pan-filter CRT display Cameral Headmount display
Video Signal
FlexVision Transput~"
~
Host Computer(IBM-PC PS[V)
F i g u r e . 2. System configuration. The user of this system can move an object in the 3D computer graphics world by his hand motion. He can control the object position by the tip of his index finger and change the object attitude by the direction of the finger. He can rotate the object around the index finger orientation by rotating his hand with his thumb up. Although our final goal is to develop a human interface system without any user attachments, we use special markers for reference points and a glove with three marks to make image processing fast and reliable in the current implementation.
196
!!!~i~ii!:i'i:i:i:i~i:ii!i!:~~iil!i:iiii!i :::::::::::::::::::::::::::::
(a) Reference object for the world-fixed frame case.
:: ::: ~::~!!::iii~::! ~
~! z
~
........
r::
(b) System operation in the human-centered frame case. Four marks are attached to the user for reference.
F i g u r e . 3. Reference point settings and overview of the system operation(b). The system can interpret hand gestures both in the world-fixed frame and in the human-centered frame, although the user should select either of them in advance in the current implementation. When we use the world-fixed frame, we put an object as shown in Figure 3(a) in the camera field of view. When we use the human-centered frame, we attach three point marks on the upper body and one on a knee as shown in Figure 3(b). We consider the situation in which the user is sitting on a swivel chair. He gives commands to the computer, watching the display. The user can use the system even if he turns the chair. Since we use special marks as features, the system can do with only simple image processing techniques. First, two synchronized images are taken. They are thresholded to extract the marks. The centers of gravity of these marks including the line mark on the index finger are used as the feature positions in the images in 3D information calculation. The areas around the current mark positions are examined in the next frame computation to track features. 4. E X P E R I M E N T S
We have examined whether we can control an object in 3D CG by hand gestures with the proposed system. The experimental results show that the system can track features and calculate 3D information at 10 Hz and that the user can move a 3D CG object in both frame cases. Figures 3(b) and 4 show an overview of the system and an example of the system operation, respectively. In principle, we need the human-centered frame when we use a headmount display as a display device. We made an experiment on this point. A subject sat on a swivel chair. He operated the system in both frame cases while turning the chair. The humancentered case had no problem when the rotation angle was within + 45 degrees. When
197 ii.~ i !!): i ¸¸
::
(a) Directing leftward
j"
......
"~,
(b) Heading left
:
t
(c) Downward
(d) Heading down
F i g u r e . 4. System operation examples. The hand gestures as shown in (a) and (c) move the object in CG as ( b ) a n d (d), repectively. the angle was larger than that, the reference marks were occluded by the body in the current implementation. This problem can be solved if we use multiple cameras. The operation in the world-fixed frame was uncomfortable when the angle became larger than -4- 10 degrees. 5. D I S C U S S I O N We have proposed a vision-based human interface system by which we can operate both in the world-fixed and human-centered frames. We need to select either of them depending on our applications and/or situations. Preliminary experiments with the developed system have led us to consider that there are three cases in terms of the frame selection. C a s e 1. T h e case w h e r e t h e i m a g e on t h e d i s p l a y s h o w n to a u s e r d o e s n o t m o v e w h e n he m o v e s . This is the case that we have mainly experimented. The user picks up a 3D CG object and moves it by his hand motion. The experimental results in Section 4 show that the human-centered frame should be used in this case. However, both frame cases can be used when we use a conventional CRT display. This is because we have a real existance, the CRT display, on which we can easily establish the world-fixed frame. Although we can use both frames, we have to operate considering which frame we are using, because the same hand motion might be interpreted differently depending on the body position. C a s e 2. T h e case w h e r e t h e i m a g e on t h e d i s p l a y s h o w n t o a u s e r m o v e s as he m o v e s . The user's position should be measured by some means in this case. The proposed method can be used for this purpose. This is the case for conventional virtual reality systems with a headmount display. When the user moves, the image on the headmount display changes according to his motion. The world-fixed frame should be used in this case. For example, when the user points at something by his hand, the hand direction should be interpreted in the world-fixed frame. The case where we see the world directly instead of through a display can be included in this case. However, the human-centered frame operation is useful in some cases. An example is the case of
198 controlling a remote operation robot. The user watches the image sent from the camera on the robot and controls the robot's arm or its body motion. To control the actual robot, the human-centered information is more convenient. Since the user's position is known in this case, the hand's position in the world-fixed frame can be translated into that in the human-centered frame. However, this translation process might increase measurement errors. C a s e 3. C o m b i n a t i o n c a s e . An example is the following case. We have many monitors of surveillance cameras whose pans and tilts can be controlled. The user chooses one of them by pointing it by his hand. In this case, the world-fixed frame should be used. Then, the user controls the pan and tilt of the selected camera by his hand. The humancentered frame case is more appropriate in this situation. Since there are many monitors and distance between the user and the cameras is long, it is not comfortable to establish the fixed-frame on the monitor screen each time a new camera is chosen. Our current theory for the selection is as follows. If we can easily consider a target object as a part of our body, the human-centered frame is preferred. Otherwise, the worldfixed frame should be chosen. It can also be used if we can easily establish the reference frame on some place even in the first case. We are planning to make experiments on this theory using the system presented in this paper. 6. C O N C L U S I O N We have proposed that a vision-based human interface system needs to interprets hand motions both in the world-fixed frame and in the human-centered frame. We have developed a system with this capability using multiple view aifine invariants. Experimental results confirms the usefulness of both frame cases. REFERENCES
1. T. Darrell and A. Pentland, "Space-Time Gestures", IEEE Computer Vision and Pattern Recognition, pp.335-340, 1993. 2. M. Fukumoto, K. Mase, and Y. Suenaga, "Real-Time Detection of Pointing Actions for a Glove-Free Interface", IAPR Workshop on Machine Vision Applications, pp.473476, 1992. 3. R. Cipolla, Y. Okamoto, and Y. Kuno, "Robust Structure from Motion using Motion Parallax", IEEE Fourth International Conference on Computer Vision, pp. 374-382, 1993. 4. J.J. Koenderink and A.J. Van Doom, "Affine Structure from Motion", Opt. Soc. Am. A, Vol.8(2), pp.377-385, 1991. 5. Y. Kuno, M. Sakamoto, K. Sakata, and Y. Shirai, " Vision-Based Human Interface with User-Centered Frame", IROS'94, pp.2023-2029, 1994. 6. J.L. Mundy and A. Zisserman, editors, Geometric Invariance in Computer Vision, Chapters 1 and 23, pp.1-39, pp.463-519, MIT Press, 1992. 7. S. Vinther and R. Cipolla, "Towards 3D Object Model Acquisition and Recognition using 3D Affine Invariants", Technical Report CUED/F-INFENG/TR136, University of Cambridge, 1993.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
199
FUZZY REASONING APPROACH TO ICONIC INTERFACE DESIGN Rungtai Lin
Industrial Design Department, Chang Gung College of Medicine and Technology 259 Wen-Hua 1st Road, Kwei-Shan, Tao-yuan 33333, Taiwan Abstract Understanding how users recognize an icon is as complex as understanding visual perception itself. Because the cognitive factors that affect the evaluation of icons have not been properly analyzed, the comprehension of pictorial symbols is typically ill-defined. The fuzzy approach seems to be a better way to provide a possible solution to this methodological problem. To remedy the disadvantage of the conventional approach, and to use the full potential of the computer as an aid to icon design, the purpose of this study is to propose a fuzzy graphic rating as a technique for evaluating the icon design. Although a fuzzy rating can be conducted using a pencil and paper technique, analysis is a difficult part through obtaining the rating in a pencil and paper method. In addition, iconic interface design is a highly graphic-oriented, so the graphic is playing an important role in design process. Therefore, based on the CAD technology and the fuzzy graphic rating as an approach to iconic interface design, this paper is intended to propose a more systematic treatment of iconic interface design than has hitherto been made. In this paper, a FUZRID (FUZZY Reasoning in Icon Design) system was implemented to provide designers with the predictive information using the fuzzy reasoning approach. Key Words: fuzzy set theory, icon design, human factors. 1. I N T R O D U C T I O N Along with the increasing availability of high quality graphic displays, the use of multiwindows and iconic interfaces in complex system environments is becoming much more common and accessible to computer users. While icons are playing an increasingly important role in iconic interface, many cognitive factors of icon design are not well understood [5,14,18]. For icons to be effective in an iconic interface, icons need to be properly designed to be meaningful, associated, leamable, memorable and consistent. There are no specific set of rules or criteria that can be followed by the designers during the design stage. Although the Intemational Organization for Standardization (ISO TC 145/SC 1) provides the selection criteria and original reference principles, whether an icon has met the criteria can be known only after the test is completed. During the design stage, the difficulty designers might confront is that there is not enough information available for modifying the proposed design. All designers can do is to develop many potential designs, then select the best of the proposed icons by a preference test. Whether these selections have met the criteria as a standard symbol is determined by a comprehension test; If not, a redesign has to be deployed. Even after following the ISO procedure and finding that one version of a symbol is more recognizable than other, the question of why the symbol is better remains [1,5,26]. This is because the ranking and comprehension test are subjective evaluation of the possible candidate symbols, and the ability of the viewer to understand what a symbol means will depend on many factors. These cognitive factors that affect the evaluation of pictorial symbols have not been properly explored. Because the comprehension of pictorial symbol is typically ill-defined, the fuzzy approach seems to be a better way to this methodological problem.
200 Fuzzy logic defmes concepts and techniques that provide a mathematical method able to deal with thought processes that are too imprecise to be dealt with by classical mathematical techniques [22,23,24]. The application of fuzzy approach is widely spread in various fields, from the behavioral and social science to product design and human factors [10,11,12,16,25]. Fuzzy set theory has been used in the psychology, for example, Hesketh [6, 7] proposed a computerized fuzzy graphic rating scale to the psychological measurement. The scale that is an extension of a semantic differential allows respondents to provide an imprecise rating and lends itself to analysis using fuzzy set theory. It takes account into the reality of imprecision of human thoughts by allowing ranges of score to be measured and translated into a single score. The availability of relative low cost personal computers and PC-CAD systems began to revolutionize the fields of design. In addition, icon design is a highly graphic-oriented, so the graphic is playing an important role in design process. Instead of using paper, pencil, and drafting board, the designer can execute design on the computer by using interactive graphical devices. No similar approach has been introduced for evaluating the icons. In order to remedy the disadvantage of the conventional approach, and to use the full potential of the computer as an aid to icon design, we must provide designers with a computer-based tool for designing icons. Therefore, based on the earlier studies [2,6,7,8,9,10,19,20, 21 ], a FUZzy Reasoning in Icon Design system (FUZRID) was implemented to provide designers with the predictive information using the fuzzy reasoning approach. 2. M E T H O D 2.1 Derivalion of Cognitive Factors To help the designer properly evaluate pictorial symbols, Lin et al. [13,14,15] derived six important factors for evaluating icons. They are "Associable," "Identifiable," "Meaningful," "Concise," "Eyecatching," and "Symbolic." Through a factor analysis, these items were reduced to the three cognitive factors, namely, "Communicativeness," "Design Quality," and "Image Function." These three cognitive factors can explain 87% of the total variance in terms of subjective rating on the icon design. It is suggested that these three cognitive factors can be used as the basis for evaluating icon design during the design stage. Moreover, based on the percentage of variance each factor can explain, the ratings can be weighted to provide a simple overall rating as follows [13]: Overall rating = 0.41 * rating on "Communicativeness" + 0.33 * rating on "Design Quality" + 0.26 * rating on "Image Function" ......... (1) 2.2 Derivation of Membership Functions The first step of applying fuzzy approach in evaluating icon is to derive the membership functions. On the basis of the studies [8,9,10,19,20,21], Lm [ 16,17] used the three cognitive factors to derive the membership functions. Using direct rating to determine membership functions have been reported [ 19,20, 21]. The semantic differential rating was designed to obtained membership functions of the three cognitive factors and the total performance. The subjects rate each cognitive factor on a scale of 0-6; 0 being no membership and 6 being full membership. Two techniques of probability distribution and normalized rating scores were used to derive membership functions. The frequency distribution of rating scales can be applied equally to the probability distribution; then we can derive membership functions based on the cumulative distribution function. According to the membership functions from probability distribution, we can define the grade of membership of averaged rating scores of total performance as: Averaged Rating of Total Performance = {0.0/0, 0.0/1, 0.0/2, 0.06/3, 0.59/4, 0.96/5, 1.0/6} ..... (2)
Another method to derive the membership function is to normalize the rating scores [20]. The overall
201 rating were normalized regarding the lowest rating score as zero ( no membership) and the highest rating score as one (full membership). Similarly, according to the membership functions from normalized means, we can define the grade of membership of normalized means of total performance as: Normalized Mean of Total Performance = {0.0/0, 0.0/1, 0.0/2, 0.19/3, 0.60/4, 1.0/5, 1.0/6} ..... (3)
With these membership functions, we can transfer the averaged rating score to membership of averaged rating of total performance. Then, the relationship between the membership and averaged rating score can be computed. 2.3 Calculation of Expected Value of a Fuzzy Variable
On the basis of the membership functions and the link between fuzzy set theory and probability theory [8,9], the "average weighting" procedure introduced by Baas and Kwakemaak [2] was used to calculate an expected value. Then, the expected value of a fuzzy variable can be calculated as follows [6,71:
X-~~X~ U
fx(X,) ~,u f~(X,)
Whero
(4) ordina o oe oach point Xi, and
o or umma ion of
all Xi multiplied their respective probabilities gives the expected value of the distribution. The procedure used to calculate the expected value was illustrated graphically with an example in [7]. 3. I M P L E M E N T A T I O N To improve the conventional design procedure, we propose the FUZRID system that will benefit icon designers. The system has been developed and implemented on PC/486. AutoCAD has been selected as base system for supporting the implementation because of its popularity, programmability, and flexibility. AutoLISP has been used as a programming language to develop many utility functions and subroutines. The brief concept of fuzzy reasoning approach to icon evaluation is shown in Figure 1. The fuzzy graphic rating subsystem is first asked respondents to rate an icon with its intended meaning using the fuzzy rating. Three values were collected automatically for the further analysis. Then, the fuzzy computing subsystem was provided to handle such imprecise rating data to compute rating scores, comprehension rates, and generate a comparison graph. Finally, the fuzzy reasoning evaluation subsystem provides the information of the current icon design to designers for modifying the proposed icon during the design stage. The FUZRID environment consists of three major subsystems: fuzzy graphic rating subsystem, fuzzy computing subsystem, and fuzzy reasoning evaluation subsystem. These subsystems are logically connected to a main function as shown in Figure 2. Three subsystems are described as follows Fuzzy eraphie rating subsystem. Based on the concept of fuzzy graphic rating proposed by Hesketh [6, 7], the fuzzy graphic rating subsystem provides fuzzy graphic rating scales with three pointers. The subsystem is composed of three functions" (iconlist), (frating), and (getdata). The (iconlist) function creates a list of icons for evaluating. The (frating) function allows respondents to provide imprecise rating by moving the pointers freely along the x-axis with a mouse. The (getdata) function is used to collect three values of most preferred point, left extension, and right extension. All the functions were implemented by using AutoLISP programming language. Fuzzy comoutin~ subsystem. On the basis of these values, the "average weighting" procedure introduced by Bass and Kwakemaak [2] was used to calculate an expected value. The fuzzy computing
202
FUZRID I I I I FuzzyCOMPU~CI_~l~uzzvR~SONINC FuzzyoRAP.,c RATING SUBSYSTEM1~ st~sYsrSM I",'ISUBSYSrE. [
.• ] ._• (FRATINC) FUNCTION[
_~ (READDATA)[
(CETDATA[)
..~ (EVALUE)] FUNCTION
OCONUST) ] FUNCTION
FUNCTIONI
~UN:ION I
.~ (MGRADE)[ FUNCTION
[
I (CerTES) [
~-~ FUNCTIONI I ] [ (FRULES)] ['---I FUNCTION[ I [ [ (DGUIDES)] ~ FUNCTION[
~c~?~o"~] Figure 1 Block diagram of F U R I D system
Figure 2 System structure of FUZRID
subsystem, including (readdata), (mgrade), (evalue), and (cgraph) function, was designed to compute the expected value based on the three values got from the fuzzy graphic rating subsystem. First, the (readdata) function reads the values from the fuzzy graphic rating subsystem. Then, the (mgrade) function was designed to convert the rating score to a grade of membership using equations (1,2,3). The (evalue) function was used to compute an expected value using the grade of membership according to the equation (4). Finally, the (cgraph) illustrates a comparison graph to show the results of icon evaluation. Fuzzy reasoning evaluation subsystem. The unique aspect of the FUZRID system is the fuzzy reasoning evaluation subsystem which was designed to provide a predictive information to designers for modifying the proposed icon during the design stage. The subsystem contains (crates) function, (frules) function, and (dguides) function. The (crates) function calculates total performance and estimates comprehension rate of the proposed icon. The (frules) provides the inference rules. The (dguide) provides designers with the information for modifying the proposed icon.
4. S A M P L E R U N The FUZRID system is organized into three subsystems which connected to a main function. This main function provides a function for learning how to use the system and errors handling. The explanation of using the FUZRID system was designed as a CAI format which were a part of main function of the FUZRID system. A sample run is shown as follows: 4.1. Startup. After booting the computer by entering the FUZRID system disk into driver A:, a batch file will be automatically executed and the system presents on the screen a briefing of the F U ~ system (Figure 3). 4.2. The procedures. The fuzzy rating procedure was presented on the screen as shown in Figure 4. The procedures are: 1) move the central pointer to a preferred point which best represents the rating. 2) indicate how far the rating could possible go by moving the fight-pointer and left-pointer, and 3) adjust the rating if necessary. 4.3. How to move the pointers. The pointers are moved by a mouse, and the explanation of moving pointers and errors handling is shown in Figure 5. The steps are: 1) moving the mouse to a pointer and press the fight button, 2) drag the pointer to the desired position press button again, 3) repeat the same steps until you are satisfied. The three graphic scales are provided for practicing. While respondents familiar with moving the pointers pushing the button will bring them to the formal rating test. 4.4. Rating the icons. After preliminary instructions and tuition in the use of the system, the user is asked to enter the icon's name for evaluating. Then the fuzzy rating scales with the proposed icon and its intended meaning is presented on the screen to the respondents (Figure 6). Respondents were asked to rate a proposed icon with its intended meaning using the semantic differential scales of the three
203
cognitive factors. After rating, respondents are asked to press the button at left comer for rating the next icon. 4.5. The fuzzy ¢omlmting. After the subsystem was running, an instruction (Figure 7) was presented on the screen to teach users how to use the subsystem. The subsystem required the user to give the icon to be reported, then the grade of membership of each rating score was computed, and an expected value was transferred 4.6. The comparison graph. After expected values were calculated by the fuzzy computing subsystem, a comparison graph was displayed (Figure 8). The most preferred point, left extension, right extension, and the expected value were drawn on a three-axis graph representing the three factors communicativeness, design quality, and image function, respectively. 4.7. The fuzzy reasoning. After the fuzzy reasoning evaluation subsystem running, the rating scores of communicativeness, design quality, image function, and total performance displayed on the screen. These rating scores provide designers an idea about current icon design. Then, the individual suggestions for each factor were given for modifying the current icon (Figure 9). 5. C O N C L U S I O N Although fuzzy graphic rating can be conducted by using a pencil and paper technique, analysis can be a difficult part through obtaining the rating in a pencil and paper method. In addition, the icon design is highly graphic-oriented that the graphic is playing an important role in the design process. The designers need a computer-based tool as an aid for icon design, therefore, this study explored the feasibility of fuzzy reasoning approach rating for evaluating the icon design. The FUZRID system is not meant to replace the designer or the respondent, but to augment both designers and respondents by providing a computerized tool for icon design and icon evaluation. The technique of fuzzy graphic rating may be advantageous by having fewer subjects to be tested, and using the computer for analyzing [17]. It is suggested that the FUZRID system will be validated in more testing and evaluating of icon design in the further study. 6. R E F E R E N C E S 1. Barthelemy, K. K., Mazur, and Reising, J. M. (1990). Color Coding and Size Enhancements of Switch Critical Features. Proceedings of the Human Factors Society 34th Annual Meeting, 99-103. 2. Bass, S. M., and Kwakernaak, H. (1977). Rating and ranking multiple-aspect alternatives using fuzzy sets. Automatics, 1977, 13, 47-58. 3. Brugger, C. (1990). Advances in the International Standardization of Public Information Symbols. Information Design Journal, Vol. 6/1, 79-88. 4. Collins, B. L. (1982). The Development and Evaluation of Effective Symbol Signs. Washington DC: US Department of Commerce, National Bureau of Standards NRS Building, Science series 141. 5. Gittins, D. (1986). Icon-based human-computer interaction. International Journal of Man-Machine Studies, 24, 519-543. 6. Hesketh, B., Pryor, g., Gleitzman, M., and Hesketh, T. (1987). Practical applications and psychometric evaluation of a computerized fuzzy graphic rating scale. In Zetenyi (editor), Fuzzy Sets in Psychology, 425-455.
7. Hesketh, T., Pryor, R., and Hesketh, B. (1988). An application of a computerized fuzzy graphic rating scale to the psychological measurement of individual difference. Int. J. of Man-Machine Studies, 29,21-35. 8. Hisdal, E. (1986a). Infinite-valued logic based on twovalued logic and probability. Part 1.1. Difficulties with present-day fuzzy-set theory and their resolution in the TEE model. International Journal of Man-Machine Studies, 25, 89-111. 9. Hisdal, E. (1986b).Infinite-valued logic based on twovalued logic and probability. Part 1.2. Different sources offitzziness. International Journal of ManMachine Studies, 25, 113-138. 10.Karwowski, W., Evans, G. W., and Ragade, R. R. (1984). Fuzzy Modeling Techniques in Human Factors Research. Proceedings of the Human Factors Society - 28th Annual Meeting, 403-407. 11.Kreffeldt, J. G., and Rao, K.V.N. (1986a). Fuzzy Sets in Consumer Product Design: Applications to Instructions and Warnings. Proceedings of the Annual Conference of the Human Factors Association of Canada-19th Annual Conference, Vancouver, BC, 99102.
204
FUZRID SYSTEM
FUZZY
GRAPHIC
RATING
SUBSYSTEM
g ' d 2 t y R e u o . n i m l in Icon D.mlill,n r o m e - All m , t m ' ~
m~ n , m ~ m c
~
HOW TO MOVE THE POINTERS? SIMPLY. J u s t llovz TaZ ttous~ T o THZ P o t m l m
PROCEDURE:
~zac~.
THEN P R g S S THE BUTI'ON. NOW TI.H~ P O ~ WILL BE III~GHT~'D. THF_,N DRAG; THZ POINTER TO ?HI~ DIL.qlR~} POSITION P R E ~ B U ' I ' ~ N . RF.,PFAT ' r l m sAm~ PROCr.DURZS m e r l L Y o u am~ S A T I ~ .
t~ a m y oR,a,mc ~,lW~ e u m ~ l m . .,~ i m ' z ~ t z a s m m ~ zv,,am.'nol~ s ~ m ' v s ' m t
|¢ sol~t-rml~wRol~a,v r ~ crm. c To s'ro~. m z l,lzo~ma. ~. u e t w - . ~
N
]
NNN -II-=- -I
4. t m ~ t
~ t o mmc,n • ~
tr~r ~mtt
AND T Y P E [ F I S O ] T O R E S T A R T .
~rr.
~ m nmnm ,rot ~ . t w c r o m z
I.
tur.
- = - -I Figure 5
Figure 4
Figure 3
-I
f
mt rtnm ~
m m
,rr~
at~e
com*tm~.,
m a~mD ~
70 ~ u o u t ~ t
WILL ~
~
.
mma
TABLET
....I
-
-
-
I
1:5 Figure
-i
6
Figure 8
Figure 7
I lM~et FUetC'rlON:
B2~.
Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9
A briefing of fuzzy graphic rating system The procedure of fuzzy graphic rating Practice of moving pointers Fuzzy graphic rating screen Instructions of fuzzy computing subsystem An example of Comparison graph Fuzzy reasoning evaluation subsystem
Figure 9
12.Kreifeldt, J. G., and Ran, K.V.N. (1986b). Fuzzy Sets: An Application to Warnings and Instructions. Proceedings of the Human Factors Society - 30th Annual Meeting, 1192-1196. 13.Lin, R. (1992). An Application of the Semantic Differential to Icon Design. Proceedings of the Human Factors Society 36th Annual Meeting, 336-340. 14.Lin, 1~ and Kreifeldt, J. G. (1992). Understanding the Image Functions for Icon Design. Proceedings of the Human Factors Society 36th Annual Meeting, 341345. 15.Lin, R., Kreifeldt, J. G., and Chi, C. F. (1992). A Study of Evaluation Design Sufficiency for Iconic Interface Design: The Design Perspective. Proceedings of the Ergonomics Society Annual Conference 1992, 376-384. 16.Lin, R. (1994). Fuzzy Approach to Standardizing Public Information Symbol. Journal of the Chinese Institute of I. E., Vol. 1I, No. 1, 33-39. 17.Lin, R. (1994). An Application of Fuzzy Graphic Rating in Icon Design. Mingcui Institute of Technology Journal, Vol.26, 201-207.
18.Lodding, K.N. (1982). Iconic interfacing. IEEE Computer Graphics and its Applications, 3, 11- 20. 19.Oden, G.C. (1977). Integration of Fuzzy Logical Information. Journal of Experimental Psychology: Human Perception and Performance, 3, 565-575. 20.Turksen, I. B. and Norwich, ,*~M. (1981). Measurement of fuzziness, Proceedings of the International Conference on Policy Analysis and Information System, Taipei, Taiwan, 745-754. 21.Turksen, I. B. (1986). Measurement of Membership Functions. In Applications of Fuzzy Set Theory in Human Factors, edited by W. Karwowski and A. Mital, 55-67. 22.Zadeh, L. A. (1965). Fuzzy Sets. Information and Control, 8, 338-353. 23.Zadeh, L. A. (1968). Fuzzy Algorithms. Information and Control, 12, 94-102. 24.Zadeh, L. A. (1973). Outline of A New Approach to the Analysis of Complex Systems and Decision Processes. IEEE Transactions on Systems, Man, and Cybernetics, 3, 28-44. 25.Wang, M. J., Sharit, J., and Drury, C. G. (1991). Fuzzy Set Evaluation of Inspection Performance. International J. of Man-Machine Studies, 35,587-596. 26.Zwaga, 14_ J. G. (1989). Comprehensibility Estimates of Public Information Symbols: Their Validity and Use. Proceedings of the Human Factors Society 33rd Annual Meeting, 979-983.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
205
Inferring Graphical Constraints from Users' Modification Takeharu Tanimura, Tsukasa Noma, and Naoyuki Okada a Department of Artificial Intelligence, Kyushu Institute of Technology, 680-4, Kawazu, Iizuka, Fukuoka 820, JAPAN This paper presents a new approach to inferring graphical constraints in graphical editors. In our approach, users modify graphical objects interactively so that the objects have approximate geometric relations, and relevant graphical constraints are automatically inferred by comparing the states before and after the modification. This approach can grasp users' intention more accurately than existing approaches that infer constraints only from a single state of drawings. 1. I N T R O D U C T I O N Graphical constraints have been used for maintaining graphical/geometric relations between objects while users edit drawings in constraint-based graphical editors[l-3]. But for many years, most users have preferred non-constraint-based editors such as MacDraw T M since it is often troublesome for users to manually specify a large number of constraints required for drawings. To lessen the load of constraint specification, several approaches have been proposed for inferring constraints from unfinished drawings[4-7]. Recent and typical approaches are found in Rockit[6] and Grace[7]. In both systems, constraints are inferred if their relevant objects are positioned in gravity fields associated with each constraint. Then the drawings are beautified so that the inferred (and chosen) constraints are satisfied. But the existing gravity field approach has difficulty in inferring proper constraints. The larger the gravity fields are, the more candidate constraints are inferred. On the other hand, the smaller the fields are, the more accuracy is required in users' operation. In fact, Grace forces users to position objects at only 4-3 pixels accuracy. In spite of the difficulty in inferring constraints mechanically, we humans can easily recognize users' intention, that is, intended graphical constraints from users' editing process. This gap arises from the fact that, in the existing gravity field approach, graphical constraints are inferred from only a single state of drawings after modification. This paper proposes a new approach to inferring graphical constraints in graphical editors. In our approach, users modify objects interactively so that the objects have approximate geometric relations, and relevant constraints are automatically inferred by comparing the states before and after the manipulation. Our relative criteria for deciding intended constraints work well in both rough positioning after big motion and precise positioning after tiny motion, and thus solve the problem of the current "fixed" gravity fields.
206
ART
B
.%
Delete VRT constraint
G
O
(2)
(1)
B
A
Move Point D
A
(Gravity Field Approach) Infer HRT and VRT constraints and redraw ¢~:~:y:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~t :::::::::::::::::::::::::::::::::::::::::::::::: |~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:!:~:~:~:} ::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::::::::::::::::::::::::
(Our Approach) Infer HRT constraint and redraw
(3)
(4)
I
~iiiii~iiiiiiiiiiiiii: [ ......~.. j • VRT gravity field of Point A
~ ~~"
(4)' "HRT gravity field of Point C Overlapping gravity field of VRT and HRT
Figure i. Interactions in the existing gravity field approach and in our approach. (VRT and HRT are verticality- and horizontality-constraints, respectively.)
2. I N F E R R I N G CATION
GRAPHICAL CONSTRAINTS FROM USERS' MODIFI-
2.1. P r o b l e m s in inferring graphical constraints with gravity fields In this subsection, we discuss problems in inferring graphical constraints with gravity fields. Figure 1 shows a sample sequence of interactions for illustrating the problems in the existing gravity field approach (and later for comparing our approach with the existing one): (1) A user is to modify a drawing with a polyline A B C D and a verticality(VRT)constraint that forces points A and D to be vertically aligned; (2) the user deletes the VRT-constraint between A and D; (3) the user moves D so that C and D are horizontally aligned. Supposing that point D is in A's gravity field of VRT-constraint as well as C's gravity field of horizontality(HRT)-constraint, (4) then the system infers both VRT-
207 constraint between A and D and HRT-constraint between C and D. From the step (2) to (3), the verticality-relation between A and D is weakened. In the existing approach, however, the VRT-constraint between A and D is inferred if point D remains in the VRT gravity field of A after the modification. This problem is solved to some extent by much narrower gravity fields. But they would force users to position objects with hairbreadth accuracy.
2.2. Inferring graphical constraints from users' modification The above-mentioned problem in the existing gravity field approach is caused by its inference mechanism where constraints are selected if and only if the objects are in the corresponding gravity fields after the modification. To recognize users' intention more properly, we need to pay attention to how users edit drawings. We thus propose a heuristics given below: "If a user changes a drawing where a graphical relation is not fulfilled into another where the relation is almost satisfied, then the constraint corresponding to the relation is intended by the user." The above heuristics is realized as follows. To solve the problem of the current "fixed" gravity fields, we adopt relative criteria for deciding whether each constraint is intended or not so that our approach works well in both rough positioning after big motion and precise positioning after tiny motion. We numerically express the degree of satisfaction with graphical constraints. If the satisfaction degree of a particular constraint increases more than a predefined threshold after a modification operation, then the constraint is treated as "intended." Formally, let x be a vector of positions and directions of objects, and c be a constraint. A function e maps from a pair (x, c) into a real number which represents the degree of satisfaction of x with c. e is maximized when x satisfies c completely. Let xl and x2 be the vector x before and after a modification, respectively. The constraint c is inferred if and only if e(x2, c) - e(xl, c) > threshold.
(1)
Let us suppose that our new approach is applied to the case in Figure 1. Then the system infers only a HRT-constraint between C and D (See (4)'). The VRT-constraint between A and D is not inferred since its satisfaction degree does not increase considerably. In fact it decreases. As shown in the above example, our approach can securely pick up graphical relations intended by users even if the users' positioning is rough. On the other hand, it filters out most of the relations unexpectedly established.
3. I M P L E M E N T A T I O N 3.1. System overview We implemented a prototype graphical editor in C on a IBM PC/AT compatible running Linux with X l l R 5 and Motif. Our system consists of four modules: a user interface module, a drawing module, a constraint inference module, and a relaxation-based constraint solver (Figure 2). The user interface module lets users to create, modify, and
208
iili !iijiiiiiiiiiiii~~i~
ii~~iiiiiiii~~
2. Send object modification data
~~iii iiiiiiiiiiii!!!i!iiiill
3,6. Redraw the window
ii~;ii!~ii!i!::::~:::~:~:iii;~i!!!~i ii!i!!!!:i~/::::/:~:~::/:i:~:::i~i:~:::i:i:~i:i:!i:!::i:!i::i:!;::i::i:::::::::::::::::::::::::::::::i::,::::::iill ::::::::!::::i:: ::::::::::::::
iiiiiiiii ii iii~~~~
iiiiii!iiiliiiiiiiliiiiiiii iiiiii i iililiiiiiiiili i!iiii!iili!iiiiii
...................................................................................................................................................
4. Send constraint/object modification data l .....
i~iii~~i~ii iii i iiiiiiiiiii l
i: :: =:!:.!:::
::i.
Con~mt :: se:: : 5. Send object ill modification data :: : - :
j
!!i::!~i:ii~i~ii~!iiii~:!!(iiiiiii!!!!i!iii!ii~iiiiiii%ii!iil)~!ii!iii~ii:~:i :¸
Figure 2. System overview.
delete graphical objects in a direct manipulation fashion (See Table 1). The user interface module sends object modification data to the constraint inference module as well as the drawing module. The constraint inference module infers graphical constraints from the received data and sends newly established constraints to the constraint solver. The constraint solver solves all the current graphical constraints and sends the solution to the drawing module. The drawing module receives the object modification data from the constraint solver and redraws the window.
3.2. Graphical objects and constraints Graphical objects have their associated points and line segments depending on the object type. Constraints in our system are imposed on these points and line segments. Table 1 shows object types and their associated points/line segments in our system, and
209 Table 1 Graphical objects object type marker polyline (n-folded) box polygon ellipse string
associated points and line segments position (n + 1) vertices(points), n component line segments and their midpoints 4 corners(points), center point, 4 sides(line segments) and their midpoints n vertices(points), center of gravity(point), n sides(line segments) and their midpoints center/top/bottom/leftmost/rightmost points baseline(line segment) and its endpoints, 4 corners of bounding box
Table 2 Graphical constraints constraint type equivalence-of-points equivalence-of-length horizontality verticality parallelism perpendicularity
meaning Two points have the same position. Two line segments have the same length. Two points have the same y-coordinate value. Two points have the same x-coordinate value. Two line segments are parallel with each other. Two intersected line segments are perpendicular to each other.
graphical constraints in our system are listed in Table 2. 3.3. The degree of satisfaction with graphical constraints As discussed in Section 2.2, a function e maps from a pair of vector x and constraint c into the satisfaction degree of x with c. For example, in our current implementation, the function e for the constraint of equivalence-of-points is given below:
e(x, equivalence-of-points) =
log k b - l o g k a logk b - logk d
0
(0_
(2)
d)
The d in the above function is the distance between two objective points of the equivalenceof-points constraint. It is calculated from x depending on the candidate constraint. Let us suppose that the value of the above d decreases from dl to d2 by a modification operation, both dl and d2 are in [a, b], and the threshold is 1. The function e defined above guarantees that the candidate constraint is inferred if and only if d2 < ~dl. This means that the inference of constraints is independent of dl or d2, and dependent only on the ratio of d~ and d2. e is defined as constant in [0, a] since an infinity value of e should be avoided and users cannot move a point properly in close proximity to the other point.
210 3.4. I n f e r e n c e w i n d o w
To enhance users' productivity, the system is also furnished with some other useful heuristics and a mechanism of inference windowing. The inference window, which is interactively specified by users, determines the scope of inferring graphical constraints. Only the objects in the window are considered as objects for inferring constraints. This inference windowing also improves the efficiency of constraint inference. After new constraints are inferred and then appended to the system, the positions and directions of objects are numerically solved, and then the beautified drawings are redisplayed. 4. C O N C L U S I O N Our prototype editor lets users manipulate objects interactively so that the objects have approximate graphical relations, and then relevant constraints are automatically inferred by comparing the states before and after the modification. Our inference approach can grasp users' intention more accurately than existing methods that infer constraints only from a single state of drawings. In addition, our approach is free from snapping in snap-dragging-based systems (e.g. Briar[D]) and taking multiple snapshots in Chimera[8]. The experiments with our system also give us a suggestion that more advanced intent recognition mechanism (e.g. constraint inference from a series of modification operations and/or inference of constraint deletion) can further improve the efficiency of graphical editing work. REFERENCES
1. 2. 3. 4. 5. 6.
I.E. Sutherland, Proc. Spring Joint Comput. Conf. (1963) 329-346. A. Borning, ACM Trans. Program. Lang. Syst., 3 (1981) 353-387. G. Nelson, Comput. Graph., 19(3) (1985) 235-243. T. Pavlidis and C.J. Van Wyk, Comput. Graph., 19(3) (1985) 225-234. M. Gleicher, Proc. 1992 Symposium on Interactive 3D Graphics (1992) 171-174. S. Karsenty, J.A. Landay, and C. Weikart, In: A. Monk, D. Diaper, and M.D. Harrison (eds.), People and Computers VII: Proc. HCI '92 Conf., Cambridge Univ. Press (1992) 137-153. 7. S.R. Alpert, IEEE Comput. Graph. Appl., 13(2) (1993) 82-91. 8. D. Kurlander and S. Feiner, ACM Trans. Graph., 12 (1993) 277-304.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
211
Enhancing Fold Manipulation Techniques Ying K. Leung and Richard J. King Swinburne Computer Human Interaction Laboratory, School of Computer Science & Software Engineering, Swinburne University of Technology, P.O. Box 218, Hawthorn, Victoria 3122, Australia. Abstract Folding is an effective means to overcome the inherent problem associated with displaying a large information space on a small computer screen. This technique has been successfully deployed in a variety of application domains, ranging from text editors and spreadsheets to information spaces with a tree structure hierarchy. However, this paper argues that the folding facilities currently available in many commercial products could be improved. It further proposes a number of features which may be used to enhance folding to enrich the human-computer interaction. 1.
INTRODUCTION
Despite rapid advances in display technology resulting in the advent of large high-resolution monitors in recent years, the information space which the user wishes to view is invariably too large to be presented on the computer screen. Over the years various techniques have been developed to address this fundamental visualisation problem. These techniques range from the use of a simple windowing and scrolling mechanism, which inevitably brings with it the problem of screen clutter and/or severe navigation difficulty, to the more sophisticated distortion-oriented display techniques (Leung & Apperley 1994) such as the Bifocal Display (Spence & Apperley, 1982; Leung 1989), the Fisheye View (Furnas 1986; Sarkar & Brown, 1992) and the Perspective Wall (Mackinlay, Robertson & Card, 1991), sometimes at great expense to the system's computational resources. One simple yet powerful technique to address this visualisation problem is folding. This paper argues that the folding facilities available in many commercial products could be improved and proposes some additional features to enhance folding.
2.
FOLDS AND ADVANTAGES OF FOLDS
The Japanese have long applied the concept of folds in the art form of origami. In the context of visualisation, folding is an effective and powerful metaphor which has been used to view
212 large information spaces, especially those with an inherent hierarchical structure. Many word processors and spreadsheets provide a folding mechanism to enable the user to manage and interact with hierarchical documents. The Macintosh Finder has also applied this technique to the display of file directory information, which can be conveniently represented as a linear tree. Folding provides an elegant way to present the structure of the information as well as finer levels of detail. It also supports many of the desirable properties of good software engineering: encapsulation, data abstraction, information hiding and top-down design (King & Leung, 1994). In the context of displaying large information spaces, it satisfies Misue & Sugiyama's five requirements for display methods which they termed: detailedness, wholeness, simultaneity, image-singleness and appropriateness (Misue & Sugiyama, 1991.).
File
Edit
_View Fold
Window
Help
m q declarations etc THyApplication m
methods
~] 2 THy~/indow E] 9 methods 2 constructor I~l d e s t r u c t o r CetWindowClass [~] 4 SetUp¥indow m 1 YNqueryOpen [~ WHDrag I con [~ YHSysCommand [T'I q,,/HTi mer Paint [~
main program
Figure 1" A numeric indicator shows the number of immediate sub-folds beneath the fold mark 3.
ENHANCING TECHNIQUES
Despite the advantages of folding described in the previous section, additional facilities are needed to support navigation and fold manipulation tasks for documents which have a wide
213 and deep structure. The following three techniques, which consist of specific visual cues and manipulation tools, are particularly useful for these tasks.
3.1 Fold Indicator/Structure Map In a folded document where only the first layer of headings is displayed, much information is hidden from the user. Space-effective visual cues can be used to indicate the physical size or the structure which lies beneath them. Such information allows the viewer to establish a richer mental model of the document he/she is viewing. Figure 1 shows a numeric indicator alongside each fold mark of the number of sub-folds that would be exposed if that fold was opened. This technique could be further extended to indicate the depth of the tree below the closed fold. This would require two numeric indicators displayed as a superscript and subscript on the fold heading. The superscript would indicate the number of levels of fold beneath the heading and the subscript would indicate the number of immediate sub-folds beneath the heading. These numbers could be represented graphically (for example, i~), but it may be hard for the user to see any appreciable difference between these miniature images with large structures; there is a conflicting requirement that these miniatures have to be small to conserve space.
~int"old "-]
II
c:~devellltpw~winfold~test~iconinfo.pas
program IconlnFo;
{$R ICONIHFO.RES} d e c l a r a t i o n s etc THpApplication methods
main program
i line 22 of 40
~-J methods I~ constructor I~ destructor GetWindowClass SetUpWindow I~ WHQuerpOpen HHDragicon WHSysCommand I~ WHTimer ~] Paint
col I |
Figure 2: A navigation map showing the descendants of a fold.
214 Beard and Walker (1990) suggest that map windows are a significant advantage when moving in a large two-dimensional information space. A navigation map in a separate window (Figure 2)showing the descendants of a fold, which is being worked on by the user, would be very useful to enable the user to navigate around the document and to manipulate folds quickly. These navigation maps can have the same functionality of the originating window; a series of these maps may be generated as the user works up and down the document.
3.2 Sliding View of Folds Current folding mechanisms are restrictive in that a section of text may only be in one of two possible states - folded or unfolded. Although some systems do allow the first line of a folded section to be displayed, this is not adequate in many situations. Furnas's idea of the fisheye view (Furnas, 1986) may be adapted here to provide varying degrees of openness in the unfolded section. Figure 3 shows a handle marker at the bottom Of the fold marker. The user may use the cursor to drag this handle to reveal the desired amount of information beneath the current fold. This operation is similar to moving the top sheet of two overlapping sheets vertically thereby exposing more of the information on the bottom sheet.
~inFold _File _Edit V__iew F_old Window
4 declarations E~
Iml
Help
etc
THpAppZication methods
[~2 THyWindow 9 methods 2 constructor destructor [~ GetWindowClass ~
~ p f l l n W ~ ndni,l
main program
Figure 3: A handle marker at the bottom of the fold marker allows the user to reveal the desired amount of information beneath the current fold.
215 3.3 Animating Fold Operation
In Hollands et al's (1990) experiments with the fisheye view system for a fictitious subway map, the subjects occasionally reported that in the course of navigating around the map, the shifts in view were somewhat jarring. During the manipulation of a folded document, sections of text are expanded and collapsed very quickly producing similar jarring effects. This problem can be overcome by using animated sequences to simulating expanding and collapsing text. The dynamics of folding may also be improved greatly by adopting techniques such as those used by the Macintosh Finders which shows a trail of the window frame when opening and closing a window.
~
] SetUpWindou
1~ WHQueryOpen SetUpWindow bmgdLn ~ n w n d s ~ l r ~
~ ~ ~ P
n m n u
d :L ! p "m .N.SJM:IL n SEI P r ~
N ~ s ~ Q r ~ m l
•
I-~ WMQueryOpen SetUpWindow procedure T ][ c o n W t n d o w . beQtn r;I amena sMstem menu !~1 start ot1=tl= d t s p l a j j l n j ; ] Get Module Handles Ell::a r t 11:::lL m m i ~ grid ;
SetUpWtndow
Pree
;
Flesou~ces;
M WHqueryOpen
SetUpWindou procedure TleonWindow.SetUpWindou; begin F41 amend system menu ~l s t a r t oFF displaying Free Resources; Get Module Handles s t a r t timer end; [~] WMQueryOpen
Figure 4: A sequence of four snapshots from an animated fold open. A simple but nevertheless effective technique for animating folds is to programmatically slide the fold open or closed mimicing what the user can do manually as described in the previous section. A more sophisticated solution is to progressively expand the vertical dimension of the fold being opened using distortion techniques (Figure 4). As has been mentioned previously, these techniques are computationally expensive, although for text, the opening (closing) fold could be rendered using a progressively larger (smaller) size of font.
216 4.
CONCLUSION
This paper has highlighted the advantages of folds in providing a powerful means to view and manipulate large documents. Various visual techniques have been proposed to further enhance folding: visual cues to indicate the structure beneath folded headings; the provision of a mechanism to support varying degrees of openness of folds; and the application of animated sequences to enrich the dynamics of folding. These techniques have been designed to enable users to develop a more accurate representation and a richer mental model of hierarchically structured documents so that users can navigate around them effectively. Whilst these concepts have been illustrated using a text document, they can be applied in a similar way to other documents such as spreadsheets and diagrams. REFERENCES
Beard, D. and Walker II, J.Q. (1990), Navigational techniques to improve the display of large two-dimensional spaces. Behaviour and Information Technology, 9, 6, pp.451-466. Furnas, G. (1986), Generalised fisheye views. Proceedings of CHI'86, pp.16-23. Hollands, J.G., Carey, T.T., Matthews, M.L. and McCann, C.A. (1989). Presenting a graphical network: a comparison of performance using fisheye and scrolling views. In G. Salvendy and M. Smith (Eds), Designing and Using Human-Computer Interfaces and Knowledge Based Systems, Elsevier, Amsterdam, pp.313-320. King, R.J. and Leung, Y.K. (1994), Designing a user interface for folding editors to support collaborative work. People and Computer IX, Cambridge Unversity Press, pp.369-381. Leung, Y.K. (1989), Human-computer interaction techniques for map-based diagrams. In Designing and Using Human-Computer Interfaces and Knowledge Based Systems,. (ed. Salvendy, G. and Smith, M.), Elsevier, Amsterdam, pp. 361-368. Leung, Y.K. and ~Appefley, M.D. (1994), A review and taxonomy of distortion-oriented presentation techniques. ACM Transactions on Computer-Human Interaction, 1, 2, pp.126-160. Mackinlay, J.D., Robertson, G.G. and Card, S.K. (1991), The Perspective Wall: detail and context smoothly integrated. Proceedings of CHI '91, pp. 173-179. Misue, K. and Sugiyama, K. (1991), Multi-viewpoint perspective diplay methods: formulation and application to compound graphs. In Human Aspects in Computing: Design and Use of Interactive Systems and Work with Terminals (Ed. BuMngers, H-J) Elsevier, Amsterdam, pp.834-838. Sarkar, M. and Brown, M.H. (1992), Graphical fisheye views of graphs. Proceedings of CHI'92, pp.83-91. Spence, R. and Apperley, M.D. (1982), Database navigation: an office environment for the professional. Behaviour and Information Technology, 1, pp.43-54.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
217
P r o v i d i n g d i a g r a m user interfaces for i n t e r a c t i v e t h e o r e m p r o v i n g Jun Han ~ and Tao Linb ~Peninsula School of Computing and Information Technology, Monash University, McMahons Road, Frankston, Vic 3199, Australia. [email protected] bCSIRO Division of Information Technology, Canberra Laboratory, GPO Box 664, Canberra, ACT 2601, Australia. t [email protected] User friendly interfaces are essential for interactive theorem proving to be practical. We have argued that proofs can be naturally modelled as having graph structures. As such, diagrammatic presentation is seen as a natural medium for communicating proofs between the user and the theorem proving engine. In this paper, we present an approach to providing diagram user interfaces for interactive theorem proving. 1. I N T R O D U C T I O N Theorem proving is one of the major tasks involved in using formal methods to develop reliable (hardware/software) systems. As the demand for reliable systems increases, it is the system developers who will ultimately shoulder such task, rather than mathematicians or logicians. It has been widely accepted that a cooperation between user and computer is most effective in proving theorems, even though some subtasks may be carried out fully automatically [81. This cooperation is called interactive theorem proving (ITP). Over the last three decades or so, a number of systems have been developed to investigate the feasibility of interactive theorem proving. Most of these systems have focused on the logical reasoning capability and achieved remarkable progress. However, little attention was given to a user friendly interface in such systems. They usually provide a very primitive textual user interface. A visual presentation in such traditional textual form often hinders the understanding and construction of proofs because of the large amount of information involved. For interactive theorem proving to be practical, a better user interface combining textual and other suitable forms of presentation and communication is necessary. Diagrammatic presentation represents such an alternative communication medium, which allows the user to view the proofs easily and manipulate them directly through the screen. Diagrammatic presentations are normally used to visualise relational information which can be modelled as "graphs" or "networks". Proof objects are such information and can be naturally represented using graph structures [2]. Therefore, we choose diagrammatic presentation as an additional communication medium for interactive theorem
218 proving. In an interactive theorem proving system, we separate the visual front end for handling the diagrams from the theorem proving engine. We call the visual front end a diagram user interface (DUI) [51. In this paper, we discuss the issues of providing diagram user interfaces for interactive theorem proving systems. In section 2, we present a brief overview of a structural model for interactive theorem proving [2], which provides the basis for diagrammatic presentation of proof objects. In section 3, we propose a framework for providing diagram user interfaces for interactive theorem proving and demonstrate the results achieved through some examples. Finally, we conclude in section 4. 2. A S T R U C T U R A L M O D E L F O R I N T E R A C T I V E T H E O R E M P R O V I N G Each interactive theorem proving system follows a specific theorem proving methodology and supports a variety of inference styles. For example, Nuprl [1], B [9] and Demo2 [7] support only "decomposition inference", while Mural [3] supports all three inference styles, i.e., "decomposition inference", "composition inference" and "connection inference". To provide support for the engineering aspect of interactive theorem proving across different theorem proving methodologies, we have developed a structural model for interactive theorem proving [2], where the general structures and operations of proof objects are defined. The support required by a specific theorem proving methodology is a specialisation of this general model. The general structural model addresses the representation and presentation issues of a full range of proof objects, including proofs, theory elements, proof theories and proof theory systems. Due to space limitation, we will focus on the proof structures in the following discussion. Proofs are the focus of the theorem proving task. Constructing a proof involves an inference process aimed at establishing the validity of an assertion which expresses a theorem proving problem. This process is usually composed of a number of proof steps. Each of these steps relates an assertion to other assertions according to an inference rule, where establishment of these latter assertions guarantees establishment of the former assertion. The former assertion is regarded as using the latter assertions. This inference process continues until all the assertions involved either have immediate proof or have been related to other assertions. To define the representation structure for proofs, we distinguish three cases. First, we consider proofs whose proof steps are all straightforward applications of simple inference rules and do not require further justification. We call such proofs simple proofs. A directed acyclic graph (DAG) can be used to represent such a simple proof, with vertices denoting the assertions involved in the proof and edges denoting the use relationships between the assertions. The detailed information of the assertions and proof steps are regarded as attached to the relevant vertices and edge groups. Second, a large simple proof may be structured by using or enclosing proof-related lemmas. A proof-related lemma is a relatively independent unit in the proof. It has a name, and encapsulates a statement and a proof of its own. The lemma proof may not use or refer to assertions outside itself. Within its enclosing proof, a lemma has a role similar to a theorem and may be used in some of the proof steps of the enclosing
219 proof, but it is not accessible from outside this enclosing proof. As in the above DAGbased representation of simple proofs, a simple proof with proof-related lemmas can be represented as a partitioned DAG with each sub-graph containing all and only the vertices and edges of a proof-related lemma or the primary proof. Therefore, we call a proof with proof-related lemmas a (structurally) partitioned proof. Finally, some inference steps in certain theorem proving systems require justification by subordinate proofs, depending on the inference rules applied. Although being part of the overall proof, these subordinate proofs are structurally regarded as belonging to the individual proof steps concerned and at a lower level than the enclosing proof. Therefore, we call a proof with justification proofs a nested proof. All the component proofs at the different levels of an overall proof may be a simple or partitioned proof and can be represented by a DAG or partitioned DAG. In addition, a justification proof may use assertions of its enclosing proofs. Such use of assertions is semantically the same as the use relations between assertions. But, it is structurally treated as references to the assertions used, similar to references to theory elements such as theorems. This is due to the fact that the assertions concerned are at different proof levels. In general, a proof can be represented using a partitioned, nested DAG structure with possible references between sub-graphs and from nested graphs to enclosing graphs. The proof structure required by a specific theorem proving methodology is a specialisation of this general structure. For example, proofs in Demo2 have a nested tree structure, proofs in Mural have a nested DAG structure, and proofs in B have a partitioned tree structure. 3. A F R A M E W O R K
FOR PROVIDING DIAGRAM USER INTERFACES
The fundamental functions of a DUI for interactive theorem proving can be divided into two categories: visualisation, through which a user can view the proofs, and manipulation, through which a user can directly operate on the proofs. A DUI for theorem proving only supports a specific theorem proving methodology; for example, the methodology of Demo2. A DUI is a user centred system which depends on specific application context. To reduce the effort of the design and implementation of each individual DUI, a generic framework for DUI has been developed [5], where we have adopted an object-oriented development methodology. Figure 1 illustrates an extension to the information visualisation pipeline proposed by Kamada [4] to translate the proof objects in the theorem proving engine to the graphical entities displayed on the screen (visualisation), and to transform the user's interaction on graphical entities to the theorem proving engine (manipulation). Four entity layers are involved:
• Proof entity. This layer is located in a theorem proving engine, where a proof entity wraps around (or inherits from) a proof object. While the proof objects in the theorem proving engine support inference operations, the proof entities support the proof structure discussed in the previous section and are responsible for communication with the DUI.
220
Diagram User Interface
~, Upa''s''k'' I ',, I
Theorem Provln8 Engine
~'"'PP'q'~"l
~"P'~"a*"m"l
Visual Entity
.... O r a p h l t d Entity
-r---'~Llnk to iollical entity LProot sunbuws
, , i
y
~ ~
Link to visual entity [ Lojical mucture
Texture
I
r,-eIOblee~ ] An objKt is a ~ o ~ pmvinE eeEi~
0 I
I strum type Tmnspmncy type J Colour type ~Dmw style
I
I
~Ltalt to vismi rarity
j
0
Trsnspsmncy Coiout DrswStyb Viewiq anglo Shading Shadowing
.J
Figure 1. A visualisation and manipulation pipeline.
• Logical entity. The components of this layer are abstracted from proof entities in the theorem proving engine. Depending on the specific theorem proving methodology to be supported in a DUI, certain syntactic and semantic consistency checking is carried out in this layer. In general, it is the proof objects in the theorem proving engine that provide consistency checking. However, we believe that some pre-processing in the DUI without involving the theorem proving engine can reduce the response time of the system, and is essential to support the generic proof entity structures across different theorem proving methodologies. • Visual entity. This is an intermediate layer, where the attributes in the logical entities are mapped into visual attributes, including both geometric attributes, such as shape and size, and appearance attributes, such as colour and texture. However, the format for the attributes in this layer differs from that supported in the graphical facility. This layer makes the pipeline more flexible. The most important function of visual entities is for handling layout (creation and adjustment) [61. • Graphical entity. The entities of this layer directly use the graphical objects provided by a specific graphics system, such as the graphics library provided by Smalltalk VisualWorks which has been used for prototyping the D UIs described in this paper.
Bi-directional links are used in this pipeline. A user can perceive the proof information by looking at the graphical attributes displayed on the screen. While there are many features in the proof objects of a theorem proving system, there are only a few graphical attributes that can be used, such as colour, transparency, and texture. Therefore, different mapping rules are required to convert logical attributes to visual attributes in different presentation situations. For instance, a user can use colour to visualise different logical attributes, while the layout remains the same. Following the links of the pipeline, the colour-type value for a particular presentation situation is first assigned to the visual entities, and then the colour values are assigned to the graphical entities.
221 :..~......,.~..........,...........................,....- ....:.. .......... .......:...-.. "...:....~.:.....~.~.~....~.~....~.~.~.;.:...~.:~!.;.:.:.:.~...:.:~.:.~.:.:.;.;~:~ ~ ~-~.i;i;~" :";"~:''!:!::~.~::" ..... "~::~.::f.::.'.~ " " ~ "
~ .¢4
"~ ~"
"~" ":*~ ~ ' ' ~ : : ~ :
.........
:::::::::::::::::::::::::::::::
:i:~:~:~:~:~:i:~:::.~i:;:i:!:~
Figure 2. Diagrammatic presentation for Demo2.
A user can directly manipulate the diagram on the screen, and the modification is interpreted into changes on the logical entity. The logical entity conducts certain consistency checking and also sends, along the link, messages to the proof entity for inference and further consistency checking. If the action violates the consistency, the associated logical entities inform the relevant visual and graphical entities to notify the user or reject the modification. Figures 2 and 3 show the diagrammatic and textual presentations of a proof, respectively, as in the diagram user interface developed for the Demo2 theorem proving system. The diagrammatic presentation illustrates the structure of the proof while the textural presentation provides more details. In fact, our framework allows the definition of different view types, which are seen as appropriate for the theorem proving methodology concerned. Some of the typical view types for a proof include a proof step, a sub-proof, a justification proof and the entire proof. These views can be presented in textual and/or diagrammatic form as their nature dictates. From a particular view, other related views can be invoked. For example, a view focusing on a justification proof can be invoked from the entire-proof view shown in Figure 2. 4. C O N C L U S I O N S We have reviewed a structural model for interactive theorem proving, where proofs are naturally represented using graph structures. Based on such representation, we have proposed a framework for providing diagram user interfaces for interactive theorem proving. As a natural means to visualising graph-structured information, diagrammatic presentations complement textual presentations and prove to be a useful additional communication medium between user and system in the interactive theorem proving process. A particular
222
~.....~.:..:.......:~.:.:..`::.:.~.:.:.:..~:.:.:.:.:':...~:..'.:.:.:.~.:.:.:.~.:.::`:.~.:.:..'.:.:.:.:.~.~.:.~:~.:.:.:.~:::::.:::::::~:?.::::~:.~.::~::~::!.<.:.~:::::::g::::.~...::::..::::.~.~:::::.:::.~:::::::::.~:::::
: :,,i .~.:.~.~.i.i.i.i.i.~.:.i.:.~.~.i.i.i.i...-.i .~.~.i.i.i.i.i.i.i .i.i.i.i ,i.i.i.i.i.i.i..:.i.i.i..:.i.i.i.i.i.i.i.i.i.i.i.i.~.i.i.i.i.i,i.i.i.i.i.i.i..~.~.~.i.~.i.i.~.i.i...i.i.i.i.i.i.i.i .i.i.i.i.~.i.i.i.~.i..:.i.~.i.i.i.i.i.i.i.i.i.i:i:i.i:i:i:i:i:i:i:i:i:i:i:i:i:i:i:i:i:i:i:i:i:i:i:i:!:!.'.'-:! :!.'.-":!.:""..'.i" i!~-I-1: !i~Z-I- 1: !3-1-1. !3-1-Z: 12-I - ~.. i::.4-1-1:
Enlidls (({NOT A -~. B) -:~ NOT {C -~. A) OR NOT A -~. (NOT A -~. e .,. O) .,. o ) . , . , . ~r.e) NOT A ,,~ (g -~ O). NOT (C -~ A) OR NOT A, NOT A -~ O Enlldlt (D ¢.~ line) NOT A o:~ (g -~. O), NOT (C -:. A) OR NOT g, NOT A -~ B EnlJs (NOT A 4.~. In~) NOT A -~ (E -~ D), NOT (C ' ~ A) OR NOT A, N O T A . ~ B E r 4 1 ~ ( ( ( N O T ( C , , ~ A ) . ~ NOTA)AND(NOTAoNOTA))¢-~.lme) NO1"A -~. (B ,,:. 0), NOT (C~-:. A) OR NOT Ao NOT A -:. B. le~. NOT A EIMailS (O <.:, M ) NOT A, NOT (C: -~ A) ,,:~NOT A, NOT A ,,~ (B .,:~D), NOT (C ,,~ A) OR NOT A, NOT A ,.) O EnlMIt (NOT A ,(,,:~Irue)
::::[s-z-~: .OT e . . o r • - - ( . - - D). ~ T (C -~ ~) O~.OT ~ . . O ~ X -~ e. ~ o r ~ ~ . t ~ , 0~or ~ , . ~ M ) iiiS-l-S: NOT ^ - , t~ - , O), NOT (C - - ^) O~ NOT ^. NOT ^ . , e E n ~ ((NOT (C - - ^ ) . , NOT .,Q. . , We) ~4-1-Z: iZ-l-3: !3-Z-Z: li-Z-l: ~i~3-3-1: i3-1-4: ,:4-Z- 2: iZ-1-4: :::3-3-;t: ii13-1-4: i~-1-5: i3-4-Z:
NOT A. NOT ~C -~' A) -:~ NOT A, NOT A -:. ~i -:, O), NOT (C -:. A) OR NOT Ao NOT A .:, O Enlails (Irue <.~ ~ ) NOT A - ) (E - ) D), NOT (C -~. A) OR NOT A, NOT A - ) B, Bo le~ NOT A El~llt (D ,~.) 11~) NOT B, NOT A -~ (El -> D). NOT {C -:. A) OR NOT A, NOT A -:, B. NOT A Entails ~nJe <-:~ Into) C. NOT Ao NOT A - ) (9 o) O)o NOT (C - ) A) OR NOT A, NOT A , ) B Enlalls (NOT A ( . ) Ir~) B. NOT O, NOT A -:, (B -=. O), (C -:. A) OR NOT A. 8. lira: NOT A Enta~ (I~K)TA .~.:. line) NOT A ,,, (8 - - D), NOT (C - - A) OR NOT A, NOT A o , B Enla~ Orue <,,) Irue) C, NOT A, NOT A -:. (B -). D). NOT (~ -=. A) OR NOT A, NOT A -:. B EnlJlS Qme <.). Irue) B -~ D, NOT (C -~ A) OFI NOT A, NOT B, lea: NOT A E ~ l l t ~O ~.~ tree) B, NOT O, NOT A .> ~ .> O}, NOT (C .~ A) OR NOT A. B, W~: NOT A EnII~ ~ m <.~. tIlm) NOTA o - (9 - ) O), NOT (C - - A) OR NOT A. NOT A - ) 9 Ertldlt ~ ¢ - , line) O. NOT (C -:. A) OR NOT A. a. kin: NOT A Enta/Is (O 4.:, 1me) NOT O, NOT(C . ) A) OR NOT^, 8, l i t NOT A Entdt ~ - . ) IIVII)
Figure 3. Textural presentation for Demo2.
feature of this framework is that diagrammatic user interfaces can be provided for existing interactive theorem proving systems with only minor changes to the existing systems. Acknowledgements. We would like to thank Professors Peter Eades and Jim Welsh for their advice and suggestions. REFERENCES 1. R.L. Constable, S.F. Allen, et al. Implementing Mathematics with the Nupd Proof Development System. Prentice-Hall, Englewood Cliffs, New Jersey, 1986. 2. J. Han. A Structural Model for Methodology-based Interactive Rigorous Software Development. PhD thesis, The University of Queensland, Brisbane, Australia, 1992. 3. C.B. Jones, K.D. Jones, P.A. Lindsay, and R. Moore. mural: A Formal Development Support System. Springer-Verlag, London, 1991. 4. T. Kamada. Visualizing Abstract Objects and Relations. World Scientific, Singapore, New Jersey, London, Hong Kong, 1989. 5. T. Lin. A General Schema for Diagramatic User Interfaces. PhD thesis, The University of Newcastle, Newcastle, Australia, 1993. 6. Tao Lin and Peter Eades. Integration of declarative and algorithmic approaches for layout creation. In Proceedings of Graph Drawing '9~ (An International Workshop), pages 376-387. Springer-Verlag, October 1994. LNCS 894. 7. T.G. Tang, P.J. Robinson, and J. Staples. The demonstration proof editor Demo2. Technical Report 175, Department of Computer Science, The University of Queensland, Brisbane, Australia, 1991. 8. L. Th~ry, Y. Bertot, and G. Kahn. Real theorem provers deserve real user interfaces. In Proceedings of 5th A CM/SIGSOFT Symposium on Software Development Environments, pages 120-129, Tyson's Corner, Virginia, December 1992. 9. T. Vickers. An overview of a refinement editor. In Proceedings of 5th Australian Software Engineering Conference, pages 39-44, Sydney, Australia, May 1990.
1II.9 Active Interface
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
225
Active Interfaces for Useful Software Tools Amedeo Cesta ~, Daniela D'Aloisi b and Vittorio Giannini b IP-CNR, National Research Council of Italy Viale Marx 15, 1-00137 Rome, Italy [email protected] b Fondazione Ugo Bordoni, Information Systems Group Via B. Castiglione 59, 1-00142, Rome, Italy {dany, gvittori}@fub.it A b s t r a c t - The majority of public domain software tools is endowed with programmeroriented interfaces and, as a consequence, they are mainly devoted to "hackers" or specialized programmers. This paper concerns the development of an active interface used to make the utilization of a free software for filtering e-mail messages easy. A multi-agent implementation of an interface is proposed that from one side allows the user to easily specify his needs and from another actively reasons on the user's choices to improve the performance of the filtering process. 1. I N T R O D U C T I O N With the development and diffusion of the UNIX operating system and the Internet worldwide connection, a large amount of software tools is available in public domain repositories. These tools accomplish the more disparate tasks and solve many problems: it is possible to find a piece of code, a software, a report, an address, a reference, an answer to technical questions, and so on. Almost anything can be retrieved if a user is able to navigate in this un-structured world: potentially these tools could make some jobs easier if well utilized. Some of the tools involve a generalized usage and justify a great attention to their interfaces (see for example the interface for the World Wide Web browsers, such as Mosaic and Netscape). But most utilities can be thought of as programmer-oriented interfaces and, as a consequence, they are mainly used by "hackers" or specialized programmers. We are investigating the possibility of developing specialized interfaces .... based on the metaphor of the software agents [1] - for these tools to enable their use to non-programmers. Our investigation has concerned an interface for a tool that filters e-mail messages, Procmail. Our system not only minimizes the user knowledge of the tool but also actively proposes improvements on the filtering process to obtain a better performance of the whole process. Our Mail Agent, as we name the particular tool, represents a limited contribution in the development of an environment for collaborative man-machine interaction. The paper is developed as follows: in Section 2 the general ideas of our research are given, then the problem of mail filtering is introduced and the inadequacy of a first solution
226 is described in Section 3, while Section 4 shows the details of the present solution. A concluding Section closes the paper. 2. G E N E R A L
FRAMEWORK
As several authors claimed [3], a full utilization of software tools disregards their potentiality and sophistication, while it is relevant their autonomy and the auto-comprehension of the tools. We have focused our attention on building automatic tools to support human work in office environments, following some general issues: • Minimizing the user's acquisition of specific competence in using the tools. • Developing active interfaces able to reason autonomously and to supply performance not explicitly required by the user. • Designing distributed and agent-oriented architectures. The environment we developed offers a software Personal Assistant to connect the user with an intelligent distributed architecture that supports and assists him in dealing with disparate tasks, tackling different network utilities and utilizing office services. The architecture exploits the metaphor of the software agents [4]: a software agent is an intelligent entity designed to solve specific problems on the behalf of the user and according to his directions. The software agent technique allows an incremental development of complex architectures and offers a uniform language to interface different modules [4]. Our architecture involves the utilization of several specialized agents in order to increase the efficiency of the user's daily activities: the agents communicate with each other and with the user, and can cooperate among them to accomplish particular goals. The architecture of the agent filtering the e-mail messages, i.e., the Mail Agent, is in turn distributed and implies some interesting ideas concerning how to build interfaces for software tools. In particular, it satisfies the requirements abovementioned: it hides completely to the user the software utilized for filtering the messages, i.e., Procmail, it plays an active role since it tries to automatically improve the performances of the filtering process, and it is wholly implemented into the agent-based paradigm. 3. M A I L A G E N T : A N I N I T I A L S O L U T I O N In the present technological scenario, human beings are continuously overloaded with information while working. Electronic mail represents a standard tool in office environments but the growing amount of messages from bulletin-boards, mailing-lists, interest groups, etc. makes quite impossible to read all of them. To cope with these problems specific tools exist, e.g., packages like Procmail or MailAgent, that allow a user to filter the incoming messages: such tools are powerful but of difficult usage for non-expert programmers. Our Mail Agent is based on Procmail. Procmail is a software that checks each incoming e-mail message according to a number of user-defined filters. Filters have to be specified using the Procmail command language consisting of a set of operators, as in the following example:
227
MAILD IR=/us r/users / amede o/mai 1 :OH * ^ From.,dai-list :0 c $MA ILD IR/bbo ards / dai :0 ! danyOfub, it This c o m m a n d states that any message coming from dai-list should be stored in the user's s u b d i r e c t o r y / b b o a r d s / d a i , and then forwarded to dany@fub, i t . The first version of a Mail Agent concerned the implementation of a Graphic User Interface to Procmail: the aim was to avoid the user to master the tool language. The solution required a serious amount of work in order to write an interpreter for graphical specification into Procmail commands, but turned out being inadequate because of its rigidity. The user was able to specify some of his needs without being aware of the complex syntax of the language. While our solution represented a considerable effort in addressing the problem of the knowledge about the tool language and its use, it completely neglected how to acquire competence in an effective use of the tools. This issue is important and involves the role of the programmer's skill and experience: in fact, in order to effectively use Procmail, the user should reach a good level of expertise and gather a certain amount of knowledge about the possibilities offered by the tool. And that should have been the task of our interface: to minimize the user's knowledge of the tool. The second version collects the results from the previous system, and adds a functionality of autonomous observation of non-accepted messages to see if some of them is similar to some of the messages accepted. The similar messages are gathered in a database and presented to the user in a dialogue box: if the user accepts some of them, the system automatically modifies the filters in order to consider that kind of messages in the future. The similarity is checked with an information retrieval technique made available by the public domain software WAIS, that can effectively index a database of texts, updates its contents, and answers queries about the similarity between texts.
4. T H E C U R R E N T
SYSTEM
In our implementation each user calls his personal assistant, named the Interface Agent, that helps him to choose the agent able to satisfy his current needs. At present, its implementation is quite simple since it allows the user to select the kind of agent he needs from a menu. We are developing a more sophisticated version endowed with the ability of establishing a restricted dialogue with the user. The Mail Agent is in charge of managing the email messages: in order to avoid the overflow of information, it helps the user to cut down the quantity of messages in his mail box. According to specified filters, only pertinent messages are put in the user's box, but all the incoming messages are manipulated for the active behavior of the interface. The Interface Agent connects the user directly with the Mail Agent so that its architecture is transparent to the user. Actually, the Mail Agent consists of four agents (Figure 1), the Preference Specificator. the Refinement Proposer, the ProcMail Agent, and the
228
j
Mail
ProcMail
WAIS
DB Relevant I Refiniment
Preference "~ Agent
/ Dialog A
Dialog B
Dialog C
User
Figure 1. The system architecture
Message Manager. Each of them has precise tasks, utilizes software functionality to accomplish them, e.g., Procmail and WAIS: they cooperate among them basically using the KQLM language [2] to accomplish the final task of filtering the email messages of the user. Besides, they can also suggest changes in the filter setting by searching in the rejected messages similarity with the accepted ones. The user is not aware of the agent he is interacting with. The choice of implementing more entities is due to flexibility and efficiency criteria. Moreover, each agent should be though of as an instance of an agent type that could be active in different tasks. The Preference Specificator is the agent that allows the user to specify his filtering needs. A dialogue window (Figure 2 ) - - t h a t corresponds to Dialogue B in Figure 1 - - is shown to the user that can select the type of constraints to be applied on the incoming mail. The requirements are specified in the Select area by inserting restrictions on the body of the message and/or on the fields From and Subject as a list of keywords: a condition can be also negated by selecting the box Not. In the Action area the user can specify actions to be performed on the messages passed through the selection: in particular, he can set the destination folder, run an external command, e.g., produce a beep whenever a filtered message arrives, or select onto a list of pre-defined actions, e.g., forward the message, automatically reply or send a message. Since the user can specify one filter or more, he can create a chain: if he fills in the box Activate Next Filter, the messages are passed through a number of contiguous filters, i.e., an AND of conditions, otherwise there are several non-connected filters, i.e., an OR condition. It is possible to pass from a filter to another by moving the arrows in the box indicating the number of the current filter
229 with respect to the active ones. The Preference Specificator translates the descriptions into an intermediate format that becomes the body of a KQLM message to be sent to the
ProcMail agent. The ProcMail Agent is in charge of generating Procmail filters. It translates the message from the Preference Specificator Se2 ect : No~ Ac~ i on~ : and asks to the ProcMail program, that is part of '~=1 I I~ r°ld"': I I its body, to synthesize the $Xdaject:: I ] ~ Ctmm~d: I I filter. It is one of the problem solving module in the ..~Forward to ] ~ody: I I ~i ot~er,: l-IA~to~tio R~plyI system. At present, there ~ S e n d Messaqe I is not a dialogue phase between the agent and the Filt=,,= llof~ ~=ti.~t, . , = t f i l t , ~ Q user who is left free to fill in the slots of the menu: the system does not interFigure 2: The filter specification window pret or understand the user's specifications but just translates them in Procmail commands. The filter is then applied on the incoming mail. Each message is marked as accepted or rejected, and then passed to the Message Manager. The accepted messages are also put in the mail box of the user. The ProcMail Agent is also in charge of modifying a preexisting filter when the user, following a DA I-Li.~t# 188 suggestion from the Refinement Proposer, all ports buxymxg decides to update the constraints. all ports busy m-~g The Message Manager is the agent in SUMMARY:How to detect networkactivatm charge of handling the sets of the accepted Summmy mac filetramfe~ and rejected messages. The accepted messages are stored in the DB Mail and indexed by the WAIS program. Each re- Figure 3: The window showing the relevant jected message is translated into a query messages for WAIS against the DB Mail: it is cornpared with each of the indexed messages to verify if there exist similarities between them in order to extract a set of messages that could be relevant for the user. The retrieved messages are sent to the Refinement Proposer. The Message Manager can be seen as an instance of a more general kind of agent able to manage different sources of information in order to find similarity. The Refinement Proposer stores the relevant messages in the DB Relevant. This agent proposes to the user the window of Figure 3 that lists the messages that the Message Manager selected for him in descendant order. For each message, the Refinement Proposer creates a new filter--that can be seen by pushing the bottom Whg--deducing it from the
~
230 features of the message most similar to it: also the message is shown with enlightened the relevant keywords. The user can choose to accept or to modify or to reject the suggested filter. If the user accepts some of the modification proposed, this monitoring module sends a request to the ProcMail Agent of revising the current set of filters. The idea is that, given a standard generation of filters by the user, the system gets a first idea of the user interests and starts a work of observation and revision. Some of the rejected messages may be "similar" to one or more of the accepted ones. Chances are that the messages were deleted because of a "rigid" use of Procmail by the user. A negotiation phase with the user is started to understand if he acknowledges the acceptance of the retrieved similar messages. The system is implemented on a Sun workstation and is currently used by several people in our office environment. 5. C O N C L U S I O N S In this work, we have described an interface to a shareware software that collects contributions from the field of distributed artificial intelligence and software agents. As a consequence, our architecture is flexible and allows the designer to distribute and decentralize the functionalities. On the other side, we do not have proposed "the solution", but simply an interface for a specific problem. The main issue of our approach is the notion of active interfaces that can be enlarged and generalized. We are currently working on this aspect.
6. Acknowledgments Daniela D'Aloisi and Vittorio Giannini carried out their work in the framework of the agreement between the FUB and the Italian PT Administration. Amedeo Cesta's work is realized in the framework of the ESPRIT III Working Group No.8319 "A Common Formal Model of Cooperating Intelligent Agents (ModelAge)".
REFERENCES 1. Etzioni, O., Weld, D., A Softbot-Based Interface to the Internet. Communication of the ACM, Special Number on Intelligent Agents, July 1994, pp.72-76. 2. Finin, T., Weber, J., et al., Specification of the KQLM Agent-Communication Language. DRAFT, February 1994. 3. Fisher, G., Lemke, A. and Schwab, T., 1985. Knowledge-Based Help Systems. Human Factors in Computing Systems, CHI '85 Conference Proceedings, San Francisco, CA, ACM, New York, April 1985, pp.161-167. 4. Genesereth, M.R., Ketchpel, S.F., Software Agents. Communication of ACM, Special Number on Software Agents, July 1994, pp.48-53.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
231
Interacting with Real Objects: Real Object Interface and Transferred Object Interface Soichiro Iga and Michiaki Yasumura a aKeio University, Graduate School of Media and Governance 5322 Endo, Fujisawa, Kanagawa, 252 JAPAN This paper describes the concept of "Real Object Interface(ROI)" and "Transferred Object Interface(TOI)". This idea is to create a virtual world that corresponds to a real world, and manage a computer by a real object by making one-to-one correspondence between the real object in the real world and the virtual object in the virtual world. Then we would test these concepts by making two application examples on a simple prototype system called " O u t e r / ~ " and describe the result. Lastly, we discuss on its impact to new computer application areas. 1. I n t r o d u c t i o n We used to interact with the everyday things like pens, erasers, and paper documents in the real world, and accomplish our daily work using these objects[I,2]. But, since the appearance of high performance computers, this situation has been gradually changing. In other words, we interact with some objects inside computers, like icons, windows, scroll bars, so called GCI(Graphical User Interface) [3,4]. We have to control them indirectly by the devices like a keyboard and mouse. However, there may be such a problem in the utilization of GUI that many of these tasks are always distributed on or hidden behind the display. Users should pay too much attention to look for them. Especially, disabled people have a difficulty to use these user interfaces, since they must manipulate both hands and multiple fingers simultaneously. The recent researches, including "real world computing", are aiming at the realization of more friendly relationship between the real world and the computer-generated virtual world[5]. These efforts may provide a new way of using computers, where some special displays like head mounted displays(HMD) and large screen projectors are commonly used. Any of the objects in the real world are designed as a tool to carry out some particular function which should be performed mostly by controlling by hands. The user interface metaphor to control computers will become much easier, if we could operate as if we were manipulating a real tool. In this research, we propose a novel system to realize this kind of concept, which consists of "Real Object Interface(ROI)" and "Transferred Object Interface(TOI)", where even a display, keyboard and mouse are not necessary. This method is based on the control of computers by using a real object in the real world. Real objects in the real world and
232 virtual objects in the computer world are functionally connected, and users could control the computer by controlling the real objects. A displayless and friendly user interface can then be produced. 2. " R e a l Object Interface" and "Transferred Object Interface"
A: Real Object Interface In the first concept "Real Object Interface(ROI)", a virtual world should be made as to correspond to the real world. The function of the objects in the real world is mapped to the virtual objects. It can be said that each virtual object has the knowledge of the object in the real world, and behaves like the real object. Users can interact with computers by grabbing and handling the real object. For example, the virtual world which corresponds to the real world is created and the user is translated into human like object as shown in Fig.l(a). When the user grabs a pen in the real world, a virtual pen object will follow it as shown in Fig.l(b). In the case that the user draws a picture on a sheet of paper, the virtual pen starts to draw on a virtual paper in the virtual world as shown in Fig.l(c).
(a)
(b)
(c)
Real World
...%i . iiiii~,iiii,'i~ ' ~i..... il ::,:~ii,'~i,'ii,'ii~ ' ii
Virtual World Human Object
Pen Object
Figure 1. Real Object Interface
In this one-to-one correspondence interface, users are not requested to be aware of the computer interface like a display, mouse, keyboard, and so on, and could control computers directly by using the knowledge of the physical real object.
B" Transferred Object Interface However, in the first concept Real Object Interface(ROI), it is rather difficult to control exactly the virtual object having some particular functions which may not exist in the real world. The second concept "Transferred Object Interface(TOI)" fills this gap in ROI. In this concept, we could use any objects to control virtual objects in the computer world, if the virtual object has the characteristic function of the computer, and there is no real object
233 that has the related function. By transferring the functions of the real objects, virtual objects can be fully controlled. For example, assuming that the user is going to write some musical notes on a sheet of paper, the same kind of virtual world is created as in the ROI as shown in Fig.2(a). In the ROI, a virtual pen and paper are made in the virtual world and musical notes should be drawn on a virtual paper as the user writes on a sheet of real paper. In the ROI, however, the goal of final task is not necessarily fully accomplished. In the TOI, a musical instrument object is created that corresponds to the real pen object as shown in Fig.2(b) and as the user draws musical notes, that virtual instrument starts to play music as shown in Fig.2(c). In the first concept ROI, a virtual sheet of music paper would be made in the virtual world and musical notes data would be written on it. However in this concept, the goal of the task is not accomplished enough. In the TOI, the musical notes data would transferred into sound data, which is performed by digital musical instruments. It could be used as a creative use.
(a)
(b)
v,.°o, wo.d
(c)
-i:O Human Object
Ins t r u m e n tO'~'ect
Figure 2. Transferred Object Interface
In this concept, disabled users especially blind users who could not see displays would be able to use computers largely, and could get more benefits from them. In addition to this, it could be used for more creative areas, such as composing music, drawing pictures, and so on. 3.
Outer~
Prototype
3.1. O u t e r ~ D e f i n i t i o n To test Real Object Interface and Transferred Object Interface, we have designed a simple prototype system called " Outer/~sk ", which has an ability of creating a virtual object which corresponds to a real object, and enables users to operate computer from the real world. In this prototype system, users can operate a computer only by manipulating the real
234 object such as a pen and canvas. The goals of the Outer/~sk work are to explore the following areas: 1. 2. 3. 4.
An use of real objects for interaction with computers An use of mental model in the real world for interaction with computers An interface for disabled users whose screen or keyboard are unusable A new concept for creative use of computers
3.2. System Architecture The O u t e r ~ prototype consists of a simple pattern recognition system, voice recognition system, voice synthesizer, MIDI musical instruments, and printers. The prototype is implemented in C and currently runs on a Sun Sparcl0 and SGI Indy. The system has four components: 1. 2. 3. 4.
Image analysis part Knowledge-base part Virtual space part Feedback part
In the image analysis part, the user and real objects are captured by a video camera that is connected to the video capture board on a work station. The shape, movement and pixel value of the object in the captured image are analyzed by the pattern recognition system. The only thing which the user must teach the machine is to lock the real object by pointing the object on the display and with the voice recognition system, and give a name to the object by voice. In the knowledge-base part, the system obtains the knowledge of the real object and functions. Knowledge is structured by a simple form which contains objects, rules, and tasks. These knowledge is described in a C interpreter like form. In the virtual space part, the virtual object is made in the virtual space related to the real object and it performs like the real one. The user also can get the sound reaction by the sound synthesizer and the MIDI musical instruments, when we touch or manipulate the object in the feedback part. Then we can operate a virtual object which has been stored in the hidden computer.
3.3. Examples of Application We considered two applications to demonstrate the advantages of Outer~sk prototype system. In the first example, it enables a very simple drawing which is associated with Real Object Interface. User can draw a picture by using a pen and a sheet of paper in the real world, and the activities in the real world will be reflected in the virtual world. Movements of real objects are recognized and these are translated into action related to the real world. In this example, the user is writing pictures on a sheet of paper using pen, and the related activities are done in the virtual part as shown in Fig.3. The second example reflects Transferred Object Interface which could control windows in a computer display by manipulating real objects in the real world. The location of real object is mapped to a location of window in the display and the size of the object is mapped to a width and height of the window. When the user moves the real object, the
235
/' Figure 3. Drawing by the real objects
corresponding window would move in proportion to the move of the real object. In this example, xclock and xload applications are controlled by real colored balls as shown in Fig.4.
Figure 4. Controlling windows by the real objects
These examples show a new direction of using computers where explicit computer interfaces does not exist, but the only interfaces for users are laid in the real world. We could consider a number of potential applications, including CSCW, manufacturing, art, and so on. 4. Discussions The works like VIDEOPLACE[6], KARMA and DigitalDesk[5] are well-known and successful systems which concern the real world and the virtual world. In the several
236 works in CSCW also related to the real and the virtual world[7,8]. In these systems, real and virtual activities are well-blended and made seamless by using output devices like HMD, projectors or screens. Our intention is to focus on the separation of the virtual and the real, in order to define the position of the virtual world where it should be. We believe that this could make novice users to get more benefits from computer technologies and could provide yet another way of using computers than the preceding approaches which blend the real and the virtual. 5. Conclusion We have proposed the two concepts, (A)Real Object Interface(ROI) and (B)Transferred Object Interface(TOI) which yield a simple and easy way to user interfaces only by interacting with the real objects. And we have demonstrated " O u t e r ~ " prototype system to test these concepts. Our preliminary prototype suggests that these concepts could be used in large area, such as office work, aid for disabled users, art, and so on by extensive studies.
Acknowledgments We wish to thank Makoto Imachi, Makiko Igusa, Kei Okano, Hiroyuki Sato, Koji Tanaka and Motohiro Sakamaki of Multi Modal Interface Project in Keio University for having fruitful comments and discussions. REFERENCES 1. D. A. Norman, The Psychology of Everyday Things, Basic Books Inc., New York, 1988. 2. D. A. Norman, Turn Signals are the Facial Expressions of Automobiles, AddisonWesley, 1992. 3. B. Shneiderman, Designing the User Interface, 2nd ed., Addison-Wesley, 1992. 4. J. Preece, Human-Computer Interaction, Addison-Wesley, 1994. 5. S. Feiner, B. Macintyre, D. Seligmann, M. Resnick, M. Weiser and P. Wellner, "Computer-Augmented Environments: Back to the Real World", Comm. ACM, Vol. 36, No.7, pp.52-97, 1993. 6. M.W. Krueger, Artificial Reality II. Reading, MA: Addison-Wesley, 1991. 7. H. Ishii, M. Kobayashi, J. Grudin, Integration of Inter-Personal Space and Shared Workspace: ClearBoard Design and Experiments, Proceedings of CSCW'92, pp.3342, ACM, New York, 1992. 8. H. Takemura, F. Kishino, Cooperative Work Environment Using Virtual Workspace, Proceedings of CSCW'92, pp.226-232, ACM, New York, 1992. 9. Meera M. Blattner, Roger B. Dannenberg, Multimedia Interface Design, Addison Wesley, 1992. 10. B. Laurel, The Art of Human-Computer Interface Design, Addison-Wesley, 1990. 11. B. Laurel, Computers as Theatre, Addison-Wesley, 1991.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
237
User Identification in Human Robot Interaction Using Identification Pendant Kaoru Hiramatsu
Yuichiro Anzai
Department of Computer Science, Keio University 3-14-1, Hiyoshi, Kohoku-ku, Yokohama, 223 JAPAN E-mail: {hiramatu,anzai} @aa.cs.keio.ac.jp
Abstract In this paper, we propose a user identification system in human-robot interaction, and design and implement the system. The identification system consists of transmitters called identification pendants and receivers equipped on personal robots. For signaling between pendants and receivers, we use infrared signals. With the system, a robot can identify users who are near the robot. Considering the issue of a user identification, we can design more convenient applications.
1.
Introduction
We believe that compact-sized general-purpose autonomous mobile robots will be used in home and office. A personal robot is an autonomous robot which is designed to use for daily use. When we use personal robots, we have to communicate with them. However, most of the current work on robotics lack attention to computer science, particularly to human-computer interaction[I]. Most of the knowledge which is acquired from humancomputer interaction can directly be applied onto human-robot interaction. For instance, voice recognition system offer a preferable mode of interaction with mobile robots as it facilitates users to issue an order without approaching a moving robot. In this paper, we propose a user identification system for personal robots, which is needed for human-robot interaction. We design an identification system which consists of transmitters of user ID attached to human users and a receiver module on personal robots. We have made a prototype of transmitters called Identification Pendants which is put in a small box. A receiver module mounted on the personal robot receives a signal including user ID. On receiving the user ID, the personal robot identifies the user who is near to it. In the following, we describe some user identification systems and design the proposed identification system for personal robots.
238
2.
Human-Robot Interaction
A personal robot is an autonomous mobile robot which is designed to work in home and office. To use personal robots, we have to communicate with them by certain methods. Still more, personal robots have to identify users in many situations. In this section, we describe some human-robot communication styles and some user identification systems.
2.1.
R e q u i r e m e n t s of U s e r Identification
In our laboratory, we have developed a variety of personal robot applications including an interface system called EM which expresses the present state of a robot by colored lighting[2], a human-robot-computer interactive system called RT/Michele[3]. These applications have aimed at robot-mediated cooperative work. There are many situations that human need robot's cooperation. Robots work for many users amidst a crowded place, for example, during room cleaning. In this case, robots are used publicly and a user identification is not always required. When robots are used by restrict users, such as personal mail delivery, user identification is needed. Besides mail delivery, user identification should be considered in many applications involving personal robots. On the other hand, ATM ( Automated Teller Machine ) which identifies its user by the bank card and personal code number has been very popular. In order to identify users, there are biometric systems which are automated methods of verifying or recognizing the identification of a person on the basis of some physiological characteristic, like fingerprint or some other behavior[4]. Furthermore, an active badge location system is designed for managing a positional information[5]. Staffs wear badges that transmit signals proving positional information of staffs. The information is able to use by P B X for call forwarding. Personal robots are general purpose autonomous robots so that we can use it for various tasks. As personal robots become familiar to us like personal computers, the issue of identifying users would be more significant. For example, if we design a porter robot like a human porter working at a hotel, robots would not only carry baggage but also identify users because of security of its keeping baggage. In some other situation, we may require more severe security for robots. So we have to consider identification systems for personal robots for using in our daily life.
2.2.
L i m i t a t i o n s of P e r s o n a l R o b o t Interface S y s t e m s
The design of human-robot interface must be an essential factor to make personal robots more familiar. In general, autonomous mobile robots have various input devices and output devices, such as cameras and ultra sonic sensors. However there are few devices for identifying a person, so we design a user identification system for personal robots. As described in previous section, many systems which are aiming at identifying a person have already existed. However, all of the existing systems are not appropriate for personal robots because of the limitations of their capability and requirements from human-robot interaction. On the design of the identification system, we consider the following two criteria.
239
)
Infrared Signals
)))) )))) Human user
Transmitter of user ID
Personal Robots
Figure 1: An overview of identification system • Input to robots in motion: Personal robots move to complete their tasks, so that it would be necessary to access them while they are moving. If we issue an order using a keyboard, we can make a complex order to the robot. Though we have to approach the moving a robot, it is dangerous and difficult to access it. Such devices as keyboard which requires direct access to robots is unsuitable. It is better for us to be able to access away from the robot. • Device size limitation: Personal robots should need to be equipped with many i n p u t / o u t p u t devices. However, each device has a size limit and a weight limit to be equipped on robots. If a convenient device which offers very accurate information is too large or too heavy, it is not able to be equipped on the robot. These items would be typical factors that we must consider for designing of personal robot's devices. So, we design a user identification system with consideration of them. 3.
Design
and
Implementation
Based on various conditions which are described in previous section, we design a user identification system for personal robots. The system consists of transmitters of user ID and a receiver module on robots. In this paper, we design a prototype of transmitters of user ID, identification pendants. Figure 1 shows an overview of this identification system. Human users put on pendants which transmit their user ID to personal robots every second. While personal robots get the user ID, human users who are near the robot are identified. Infrared signals are used for signaling between the pendant and a receiver module on robots. • Robots can get indirect signals that are reflected by walls or human when used indoor.
240
IR
Signal
~~Cl I!IIIgslrilD i UIR iii~
ili iiiii!iii iiiiiiiiiiiii:ii
i O°ner,,or NOonor,tor NOr,ver
Figure 2: Identification Pendant structure
. . . .i!i. . i ii:~~i~~iiilliiiiiiii~,~,',','~"~'~'"~',~~,',"'""',',','~~!,i !ii: :~ii:~iiiiii~. . . . . . . .!..... . . . . ~[~
~:~' .~:~:~:~:~:~:~:~:~:~:~:~:~:~
iiiiiilili
~i~ii
iiii!i!ii
!iii
iiiiiiiiiiiiiiiiiiiiiiiii~iiiiiiiiiiiii!iiii{iiiiiiiill
!i•!••i!••i•!•i!••i!•i•!iU•i!i•!i••!i!i•i!•!•!•!•!•i!•!i•!•!i•!!•!•!•!•i!•!i!•i! Figure 3: Identification Pendant
• The signals will not travel through the obstacle such as walls and human so that only visible users from robots are identified. • The signals reach only a 5 meters range so that users who are far from robots would be ignored. So, we think the user identification of this system would be natural like human relation. Besides infrared signals, other signals such as radio signals will use for identifying users, we can design another system that have a long identifiable range.
3.1.
Identification
Pendant
An identification pendant is a prototype of transmitters of user ID. The pendant is a small box which is capable to send a user ID. A user of personal robots puts on the pendant on one's chest. The pendant sends their user ID to personal robots using infrared signals every second, which is received by a receiver module on the robot side. The structure of the pendant is shown in Figure 2 and the prototype of the identification pendant is shown in Figure 3. Clock Generator includes a oscillator which makes a pulse stream of the user ID. The oscillator have a thermistor which is a thermal sensor of a negative temperature coefficient so that the sending cycle of the user ID varies with temperature of its environment. As varying cycles by the temperature, signal collisions are avoided among pendants. ID generator consists of an infrared remote control transmitter IC. This generator makes signals which include each user ID. When IR driver gets user ID from ID generator, the driver amplify it and transmit it to personal robots by an infrared photo diode. Figure 3 is a prototype of identification pendant. The pendant size of this version is 74 x 50 × 20,,,,.
3.2.
Receiver Module
A receiver module gets the user ID from the pendant. The module is mounted on a personal robot and consists of a IR receiver and a ID decoder. IR receiver, infrared remote control receiver module, gets user ID from the pendant. Each receiver has directivity, so we use four receiver modules to extend receiving range. The decoder, infrared remote control receiver IC, is placed on each IR receiver. While the receiver gets infrared signals, the ID decoder decodes the user ID contained infrared signals. This IC consists of inner shift registers and a ID checker.
241
IR Receiver Module ~jiiii~!~:i:~iii~i!~iiii~!~ii~i!~!i!ii~i~!~i~i!!~!!~!~!~i~i~i~`i~!~`~`~.~!!~i~
~/.::;ii%i::::~ ::i~: !: : :::::::: ~::: ....... :::
iii!iiiii!iiiiiiiiiiiiiiiiil Figure 5: Receiver Module
Figure 4: A topview of a receiver module
After that, the receiver module connects to the main module of the robot. The receiver module will keep the user ID information which is decoded by decoder together with directional information. Figure 4 presents a topview of the receiver module and Figure 5 is a photo of a receiver module.
3.3. Identification Pendants and Personal Robots The user information which is received by the receiver module is dealt with by a main board of the personal robot. We have designed a personal robot architecture which is called ASPIRE and consists of some modules ( Figure 6 ). Figure 7 shows the structure of the ASPIRE architecture with the receiving module. In personal robots, #-PULSER[6] has been implemented for a basis of applications. The #-PULSER is an operating system which employs multi-task and multi-thread model. A user ID thread is polling the user ID information every one-tenth of a second. While the information of the user ID is dealt with combined into other sensing data, more convenient applications for users will be able to design. ..........................i~i............".. iiiiiiiiiiiii Decoder ~!!{i!ii~i~iii
ii!iiiiiiiFii............
(((( (((1 User Code
........
!ii i i i i i
Motor
:-::-::::
~::::::: i~i~i~i~i~i~i s. . . . Bo~ i~i~i~!i!i! ....... Figure 6: Personal Robot ASPIRE II
~-
--~
--m
m
>>> >>>
Ultra Sonic Sensor
Figure 7: ASPIRE architecture
242
3.4.
Evaluation
Our preliminary investigation on the identification system has confirmed its effectiveness. Equipped with the system, a personal robot is able to determine whether users are within its sensing range or not and identify who is near them. The prototype uses infrared signals which communicate between transmitter modules and receiver modules with an effective range of 5 meters. When used indoor, infrared signals will also reach indirectly because of reflection.
4.
Concluding Remarks
We have designed and implemented a user identification system for personal robots. With the facilities provided by the system, it is expected more convenient application for personal robots can be achieved. However, a size of the pendant, collisions of infrared signals and a. range of this system are some problems to be solved. These problems are much concerned with the implementation style, and the environment in which the system is operated. To solve these problems, we need to acquire more experience on some specific problem domains through experiments.
REFERENCES [1] Y.Anzai. "Human-robot-computer interaction: a new paradigm of research in robotics". Advanced Robotics, 8(4):357-369, 1994. [2] C.Ono, Y.Yamamoto, and Y.Anzai. "A Model of Expressive Machine and Its Application to Human-Robot Interaction". In Proc. of the 5th International Conference on Human-Computer Interaction, pages 19A-231-236, 1993. [3] Y.Nakauchi, K.Kawasugi, T.Okada, N.Yamasaki, and Y.Anzai. "Human-Robot Interface Architecture for Distributed Environments". In Proc. IEEE Int. Workshop on Robot and Human Communication, pages 413-418. IEEE, 1992. [4] B.Miller. "Vital signs of identity". IEEE SPECTRUM, pages 22-30, Ferbuary 1994. [5] R.Want, A.Hopper, V.Falcao, and J.Gibbsons. "The Active Badge Location System". A CM Transactions on Information Systems, 10(1):91-102, January 1992. [6] T.Yakoh, T.Sugawara, T.Akiba, T.Iwasawa, and Y.Anzai. "PULSER: A Sensitive Operating System for Open and Distributed Human-Robot-Computer Interactive Systems". In Proc. IEEE Int. Workshop on Robot and Human Communication, pages 404-409, Tokyo, 9 1992.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
243
Applying Personal Robots and Active Interface to Video Conference Systems Nobuyuki Yamasaki
YuichiroAnzai
Department of Computer Science, Keio University 3-14-1, Hiyoshi, Kohoku-ku, Yokohama 223, JAPAN
E-mail: {yamasaki, anzai}@aa.cs.keio.ac.jp Abstract Robots for personal use like current personal computers will appear in an office or at home in the near future. We call these robots personal robots. As of today, a personal robot can be thought as a small general-purpose autonomous mobile robot. For a personal robot interface, we have already proposed a new user interface concept: Active Interface. Active Interface does not only wait for users' explicit input but also tries to get information from users' implicit input and external environment. Based on the gathered information, it acts spontaneously and keeps the system in an advantageous condition for users. In this paper, we apply personal robots and Active Interface concept to a video conference system. The system should be made flexible and user-friendly. In brief, we design and implement a self-movable, flexible and user-friendly video conference system.
1.
Introduction
Today, personal computers are widely used for business and hobbies. Moreover many people use personal computers to communicate with others through computer networks which are being enlarged on a world scale. But personal computers can only exchange electrical information in the form of data files. We think that physical objects such as documents or baggage are also important information which should be exchanged during cooperative work. A personal robot can handle both the electrical world and the physical world. In other words, it can compute something and communicate with others as a computer, while it can also move physical objects. Using this ability, we try to expand the object handled by computer science to the physical world [1]. In this paper, "personal" means personal use. We think a personal robot as a small general-purpose autonomous mobile robot, such as an office work support robot, a housework support robot, an amusement robot, a welfare robot and so on. We believe that personal robots will eventually become members of our society in the near future, and co-exist with us in an office and at home as well. For example, there
244 will be some personal robots in an office, and a user will ask a personal robot to help a user, moreover a personal robot will ask a user to help it and work together. In this situation, we have proposed a new user interface called Active Interface to interact with human beings easily. On the other hand, using many conventional video conference systems, users must be in front of the systems in a video conference. Even if users want to move slightly in the video conference, users can not move at all. So the users may feel constrained, using it. In this paper, we apply personal robots and Active Interface concept to a video conference system. The system should be made flexible and user-friendly. In brief, we implement a self-movable, flexible and user-friendly video conference system. And we evaluate the video conference system.
2.
Active Interface
Active Interface has been defined as follows. Active Interface does not only wait for users' explicit input but also tries to get information from users' implicit input and external environment. Based on the gathered information, it acts spontaneously and keeps the system in an advantageous condition for users (Fig.2)[2]. The i n p u t / o u t p u t of Active Interface have been classified as follows. i • • •
Explicit input: (key typing, mouse movement, voice, etc.) Implicit input: (facial expression[3], volume and direction of voice, etc.) Information from external environment: (location, noise level, temperature, etc.) Output: (action, display expression, utterance, etc.)
These are classified from an interface designer's view. These are important for a designer in order to determine what kinds of input users should be aware of, and what kinds of input users can safely ignore. external environment external environment
! User
Input
..................
t
:::::::..-.f..x..........J
,t).
IL ~I ~ '"'~~"~: ~ l~J' (~
Computer
•
lost Information
Figure 1: A conceptual model of a conventional interface
Figure 2: A conceptual model of Active
Inter/ace
In comparison between a conventional interface (Fig.l) and Active Interface (Fig.2), the former only uses explicit input. So user's implicit information, which has certain amount of information, is lost. And the information from external environment has often been neglected in a conventional man-machine interface (especially human-computer interface). On the other hand, the latter actively handles and uses the information, which
245 is discarded in conventional interfaces, and tries to keep the system in an advantageous condition for users. 3.
Implementation
3.1. The ASPIRE H robot We have already proposed a new personal robot architecture ASPIRE (ASynchronous, Parallel, Interrupt-based and REsponsive architecture) [4], which is designed so as to move quickly in non-artificial and unknown environment (ex. a crowd). And based on ASPIRE, we have implemented the ASPIRE H robot which is the second generation of the ASPIRE personal robot families [5]. The ASPIRE II robot is implemented as a functional classified parallel computer, which is connected by the VME bus and has the distributed shared memory (Fig.3,Fig.4). It is divided into functional modules, including a main module, a motor control module, a sensor control module, etc. Each module has its own processing unit, which is a RISC processor (#-SPARC or SPARClite), to process information quickly and to decide its behavior for itself reactively. And we design and implement the video conference system on the ASPIRE II robot (Fig.5).
~PARClite
t
Im.nupt
Sh.bus I F -
E
VM~ ua.~ler ~ , ~
~,'
Comrdl,r
[
LocalbuslV
Main M ~ I ~ I
...... I VME bus
"-'-"=
Shared bus
............... Sbus :
=
Interrupt lines
Figure 3: The architecture of the #-SPARC board
3.2.
I/O
I]..
~
Functional Modules
I
I VME bus
==~=
Shared bus
..................... Local bus =
:
Interrupt lines
Figure 4: The architecture of the SPARClite board
Figure 5: The ASPIRE H robot
Devices and Sensors On the video conference system, the following I/O devices and various sensors are equipped on the ASPIRE II robot to interact with human users. As I/O devices, a CCD camera, a directional microphone, a color LCD ( T F T type), a speaker, a wireless modem, and a transmitter for moving pictures are equipped on the ASPIRE II robot to interact with users, other robots, and base stations. As sensors, a speech recognition module (Panasonic DSP MN1901611BAT1), a speech synthesis module (NTT VCSllA-CIF), a noise level monitor, a sound source sensor,
246 a eight-directional heat source sensor, a eight-directional ultra sonic distance sensor, a eight-directional infrared distance sensor, a eight-directional touch sensor, an optical fiber gyroscope (Hitachi HOFG-3), and two odometers (for each right and left wheels) are equipped on the ASPIRE II robot to get information from user's implicit inputs and external environments. 4.
Video
Conference
System:
Typical
Behavior
Users communicate with each other through the CCD camera, the microphone, and the LCD on the ASPIRE H robot. A user can look at a specific user through the LCD and listen to his voice through the speaker. And the user's movements can be sent to the other user through the CCD camera and the user's voice can also be sent to the other user through the microphone. The CCD camera and the microphone are connected to a transmitter, which sends the moving pictures and voice to the other system. Every user has access to the video conference system with similar settings. The robot communicates with the base station and the other robots by wireless modems. The base station connects to a computer network, so users can order personal robots through all computers and terminals that are connected to the computer network.
Figure 6: Communication with the base station
Figure 7: Approach the user
A typical behavior of communication through the video conference system is presented as follows. 1. If a user wants to communicate with the other person using the video conference system, the user orders the ASPIRE H robot using a computer (especially on the user's desk) (Fig.6). The order is sent to the base station via computer network, and the base station send the order to two robots which are ready to work. If there is an ASPIRE H robot near a user who wants to use the system, the user can directly
247 issue an order to the ASPIRE H robot by voice. The ASPIRE H robot sends the message to the base station, and so forth. 2. Using the internal map and various sensors, one robot approaches the user and the other robot approaches the specific communication partner. 3. Based on Active Interface, the robot trys to turn the partner's attention to the video conference system by informing the partner through the speech synthesis system. At this time, based on Active Interface, the robot measures noise level around it by using the noise level monitor, and produces a speech at an appropriate volume. 4. The user communicates with the partner using the system. While the video conference goes on, if the person moves, the ASPIRE H robot spontaneously trys to follow the person, based on Active Interface (Fig.7). The ASPIRE II robot measures the direction of the person using the sound source sensor and the heat source sensor, and also measures the distance using the ultra sonic distance sensor and the infrared distance sensor. The robot autonomously approaches the person employing the sensor inputs. So the person can interact with the system flexibly and easily. 5. If the user wants to finish the communication, the user orders the robot using the computer or by the user's voice.
5.
Related Work
Up to the present, many video conference systems have been proposed and implemented. COGENT [6] is an integrated design model for a computer supproted meeting environmeat to support intellectual collaboration using computers and INTERNET for information access and sharing, and voice and audio communication over TCP/IP. The integrated design model of COGENT includes work space and ergonomics design, such as lightning, heating, air conditioning, acoustics design, and design of operation and maintenance. A distributed multiparty desktop conferencing system: MERMAID [7] is designed based on group collaboration system architecture, and provides an environment for widely distributed participants, seated at their desks, to hold real-time conferences by interchanging information through video, voice, and multimedia documents. This system is implemented by using narrow-band ISDN, high-speed data network, and UNIX-based EWSs with electronic writing pads, image scanners, video cameras, microphone-installed loudspeakers, etc. TeamWorkStation (TWS) [8] is not a pure video conference system but a desktop real-time shared workspace characterized by reduced cognitive seams. TWS integrates two existing kinds of individual workspaces, computers and desktop, to create a virtual shared workspace. The key ideas are the overlay of individual workspace images in a virtual shared workspace and the creation of shared drawing surface. These systems are collaboration systems, and support networks (INTERNET or ISDN) and multimedia. These systems are sophisticated and convenient to experienced users. In contrast, the current implementation of our system is still in the preliminary stage, and networks and multimedia capabilities have not been fully supported yet. However, the design of our system is based on a concept which is radically different from the conventional systems.
248 With a conventional video conference system, a user must become experienced in the system. In brief, a user adapts to the user interface. On the other hand, with our video conference system, Active Interface adapts to its user. In this way, user's load is considerably lightened. Whether it is the user who should adapt to the user interface or the user interface that should adapt to the user is a very important issue to be considerd in designing a user interface.
6.
Concluding Remarks
We applied personal robots and Active Interface concept to the video conference system, and we designed and implemented the self-movable, flexible, and user-friendly video conference system. A user can use the video conference system at various places where the user wants to use it in a room, because personal robot can autonomously move there and follow the user. And the user can order personal robots using any robots, computers, or terminals which the user wants to use. We think that human users can interact with our video conference system more conveniently, because the user interface of our system adapts to users by applying personal robots and Active Interface concept to our system.
REFERENCES 1 Yuichiro Anzai. Towards a new paradigm of human-robot-computer interaction. Proceedings of IEEE International Workshop on Robot and Human Communication, pages 11-17, 1992. 2 Nobuyuki Yamasaki and Yuichiro Anzai. Active Interface for Human-Robot Interaction. In Proceedings of IEEE International Conference on Robotics and Automation, Nagoya, Japan, April 1995 (to appear). 3 H.Kobayashi and F.Hara. Recognition of six basic facial expressions and their strength by neural network. In Proceedings of IEEE International Workshop on Robot and Human Communication, pages 381-386, 1992. 4 Nobuyuki Yamasaki and Yuichiro Anzai. The design and implementaion of the personal robot hardware architecture ASPIRE. In Porceedings of the 48th Annual Conference of Information Processing Society of Japan, volume Vol.6, pages 91-92, 1994. 5 Nobuyuki Yamasaki, Syoji Sawamura, Jun Yamamoto, and Yuichiro Anzai. A design and implelnentaion of personal robot architecture : ASPIRE using risc processor. In Porceedings of the 12th Annual Conference of Robotics Society of Japan, volume No.l, pages 303-304, 1994. 6 Eiji Kuwana, Tsuyoshi Masuo, Yuzo Nakamura, Yasuhisa Sakamoto, and Eiji Yana. Computer supported meeting environment for intellectual collaboration: COGENT. NTT ReiD, Vol.43, No.9:21-30, 1994. 7 Kazuo Watabe, Shiro Sakata, Kazutoshi Maeno, Hideyuki Fukuda, and Toyoko Ohmori. Distributed multiparty desktop conferencing system: MARMAID. In Proceedings of the Conference on Computer-Supported Cooperative Work '90, pages 27-38, October 1990. 8 Hiroshi Ishii. TeamWorkStation: Towards a seamless shared workspace. In Proceedings of the Conference on Computer-Supported Cooperative Work '90, pages 13-26, October 1990.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
249
An Object-Oriented GUI for the Specification of Robotic Systems Raymond K. Wong Department of Computer Science, University of Science and Technology, Clear Water Bay, Hong Kong
To facilitate the specification of robotic systems in a natural, incremental and systematic way, a graphical user interface (GUI) for the extended object-oriented model which supports the visual specification and modeling of the robotic systems as well as the computation of robot kinematics is described in this paper. This GUI provides an interactive environment from robot kinematics (low level) to robot activities (high level)in more user-friendly way when compared with most of the existing modeling tools. 1.
INTRODUCTION
Advanced manufacturing systems, which normally consists of one or several robots, have become larger in size and more complicated in their operations. In order to investigate the underlying principles of such systems, an appropriate specification and modeling tool is essential. Most of the current available modeling tools are, however, based on the principles of operations research and control theory (e.g., see Desrochers [1] for detailed discussions of these tools), and they can only facilitate the quantitative but not the qualitative analysis of the system. Moreover, they fall short of accommodating the growth of the system: when more data and complex interactions/relationships are encountered, the overall picture of the system becomes very difficult (if not impossible) to specify (and hence to model) using these tools. For reasons as fundamental as such, object-oriented modeling approaches have been actively exploited with growing popularity for describing these and other types of engineering application systems [2, 3], due to their significant characteristics such as expressive modeling power, flexible modifiability (extensibility), definition/code reusability, and implementation independence (via information hiding), etc. While the advantages of using an object-oriented approach to describe and model a robotic system are numerous, current object-oriented models are still primitive in capturingthe more dynamic semantics of the applications. Moreover, the existing objectoriented models may capture the notion that an assembly robot is a robot, they do not support the notion that a given robot may act as an assembly robot. To facilitate the specification and modeling of very large robotic systems which typically involve such multi-faceted entities, necessary extensions to current object-orientedmodels need to be made. In [8], the concept of roles has been employed to serve as another level of ab-
250 stractions to the objects in robotic systems, so that the behaviour of the objects can be naturally partitioned and hence decomposed (in top-down manner). With these features, an GUI which is user-friendly and supports rapid prototyping can be easily constructed. This paper focuses on this interface for the role modeling tool, and there is no doubt that the interaction between users and the GUI will be much better than that between the users and the mathematical equations which have been used in most of the existing modeling tools.
2.
BACKGROUND
In a conventional object-oriented model, the basic unit of interest is object, and objects are instances of some abstract data types (called "classes" in conventional object-oriented languages and systems). When defined, classes are organized into an inheritance (subclass) hierarchy, and objects are created within classes that prescribe both structural and behavioral properties (attributes and methods) for the objects. Such a "classificationbased" approach, however, falls short of supporting those applications (such as robotic systems) involving objects and inter-object relationships/interactions that by nature are dynamic, concurrent, and of multi-faceted nature. In the context of an object-oriented database model, roles can be used gracefully to partition messages for objects so that objects can receive and send different messages at different stages of their evolution/life-cycle [5, 6]. The partitioning of messages I for an object according to different roles has the advantage of allowing the designer (and possibly the implementer of the application) to concentrate on the life-cycle of an object in one role at a time. By adopting the ideas of roles to model a complex robotic system, we can similarly partition the objects into different roles such that objects can perform different actions at different stages of performing a complex task (which may involve cooperation with the other robots). Similarly we can specify the interactions between activities in terms of the dependencies among the roles of objects. For example, a robot can undertake numerous ephemeral roles at different times or even at the same time: an assembly robot to assemble the product components, an "intelligent" worker who retrieve the right parts from the conveyor, and even a sensor to identify the defects of the products. It is important to note that this phenomenon is not unique for manufacturing systems, but a general characteristic of real world objects: many kinds of entities other than robots also have roles that change over time. For instance, an employee in an organization environment can play different roles at different times (or even at the same time): host or audience of a departmental seminar, car-owner of some car(s) parked inside the company space, and organizer of some social/entertainment event (see figure 1). In each of these roles the person can perform, quite often orthogonal, actions prescribed by that role. 1The different ways of implementing the mechanism of roles have been discussed in [7].
251 host of seminar club-organizer ....--
person
tl
s
%,
s
s
.. t2
t4
employee
""-k car-owner
Figure 1" A person playing different roles at different times 3.
THE SPECIFICATION INTERFACE
To increase the user-friendliness of the role model, an GUI has been specially designed on top of the role specification language mentioned in [7, 8]. It supports: • Visual definition of the robot manipulator parameters; • Both top-down and bottom-up design methodologies; • Reuse and sharing resources of the defined robot components by applying the appropriate class inheritance and instances; • Computation of kinematics (e.g. forward kinematics, reachable space, etc.) immediately after the robot is visually defined; • Trace of message passing routines to assist the analysis of robot activities; and • Visual simulation of the defined robotic system (ongoing work). These features will be further described in the following subsections.
3.1.
Visual Specification of Robots (Joint Level)
With the object-oriented specification, a robot can be specified and modeled incrementally with the base classes defining the primitive joints and links of the different types of manipulators (Readers can refer to [4] for the introduction of manipulator kinematics). The manipulators with higher degree of freedom can be formed by inheriting the properties from the classes which define the manipulators (or joints) with lower degree of freedom. As illustrated in figure 2, the reference frame is .defined locally with respect to each manipulator class. Therefore, the transformation of reference frame(s) is(are) necessary after the joining of manipulators (or joints) during inheritance. Furthermore, all the visually defined classes will be stored in the libraries for future reuse. For example, the coordinates corresponding to the base frames S' and S" for the links need to be transformed such that they will make reference to the base frame S after the links being
252
/
Resizable bi tmapped button A r r o w used to show the inheritance
iii~i~!!:;~.~:~i~:ii~i~i~ii~i~i~i~!i~i~!i!!iii!~i~i~!i;~ii;i~ii~
~!~ i~ i@ i ~~ i~ i~ i~ i~ if~!i~~ i~ i~ i!~!~~ i!~~ i!~~ i!~~ i~ i~ i~ i~ i~ i
~Bitmap to show the change of some parameters during inheritance
Figure 2: Using the GUI: Constructing SCARA manipulator with the classes of different links inherited and integrated together to form the SCARA manipulator as shown in figure 2. This transformation is done automatically by our prototype system. The configuration here (embedded in the class hierarchies) is useful in solving inverse kinematics as well, but the computation will be more expensive. Apart from the benefits gained from class inheritance, the role hierarchies define the set of tasks which can be performed by a robot in a particular situation. This is the idea behind the definition of roles. For example, consider the roles played by the assembly robot. A screwing role can specify a set of screwing tasks as well as constraints which can be performed by the role player(s). These tasks and constraints may depend on some instance variables such as ~'s and 0% shown in figure 2. This is also useful for the robot task scheduling and task learning by partitioning the task space into different roles.
3.2.
Visual Specification of Robot Activities (Robotic System Level)
The inheritance capability of object-oriented model support the sharing of the properties of robots among a group of robotic systems which ease the design and redesign process and provide a systematic way for maintenance. For instance, the different kinds of assembly stations can be easily defined by inheriting the properties from the base class of assembly station. Moreover, the role extension give one more abstraction level to model the system, in which the systems can be designed in top-down or bottom-up manner, as shown in figure 3. In the top-down manner, users will identify the objects in the entire system first, and then refine each object into role-level. Finally the states of each role will be developed. Alternatively, a system can be defined in bottom-up manner.
253
Top-down
Object Level o ..
Role Level State Level
I
Bottom-up Figure 3: Top-down and bottom-up design through the role model 3.3.
I n f o r m a t i o n A b o u t R o b o t Activities
The general information of the design can be obtained through some static analysis with the role model. In order to ease the task of design and analysis, the GUI is prototyped in a way that users can specify the system according to the way shown in figure 3 and then the tool will automatically generate some static information for analysis. To illustrate this further, consider two robots with different number of roles they can act as and each role can have different number of states. After the robots being visually specified, we can have the information about the number of roles in each object and the clear picture of the inter-role dependencies (by tracing the message passing routines). Furthermore, if we focus on the inter-role dependencies of the model, we can easily find out which role(s)is (are) the most active, and which one(s) is (are) the most passive by the definition that Role A is said to be more active than Role B if more roles are dependent on Role A. In fact, the GUI will display the inter-role dependencies as a directed graph auch that the most active role is the source of the graph and the most passive one is the sink of it. Since a role of an object denote a set of functions which can be performed by that object, the activity of roles can assist us to find out where the bottleneck(s) of the manufacturing system or robotic system is (are). 4.
CONCLUSIONS
Robotic systems consist of enormous volumes of dynamic data and interactions. The deficiencies of using traditional or existing models to specify and model these systems have been described. In order to facilitate the specification of such complex systems in a natural, incremental and systematic way, an GUI for an extended object-oriented model is proposed in this paper. The model equipped with the GUI supports the object-oriented specification and modeling of the robotic systems, as well as the computation of robot kinematics. More importantly, it provides the information from robot kinematics (low
254 level) to robot activities (high level) of the robotic systems in more user-friendly way when compared with most of the existing tools. Finally, the issues on the static analysis of robotic systems based on roles have been discussed along with the visualization of the results via the GUI. This analysis will give us some useful information about the activities of the robots. REFERENCES
[1] Desrochers, A.A., Modeling and control of automated manufacturing systems, IEEE Computer Society Press, 1990. [2] Gupta, R. and E. Horowitz, Object-oriented databases with applications to CASE, networks, and VLSI CAD. Prentice-Hall, 1991. [3] Kim, W. and F.H. Lochovsky (Editors). applications. ACM Press, 1989.
Object-oriented concepts, databases, and
[4] Murray, R.M., Z. Li, S.S. Sastry, A mathematical introduction to robotic manipulation, CRC Press, 1994. [5] Pernici, B., Objects with roles, Conf. OIS, 1990. [6] Richardson, J. and P. Schwartz, Aspects: Extending objects to support multiple, independent roles, Proc. ACM SIGMOD Conf., pp. 298-307, 1991. [7] Wong, R.K., Design and analysis of automated manufacturing systems with roles: An object-oriented approach, Second Australian and New Zealand Conference on Intelligent Information Systems, November 1994. [8] Wong, R.K., Advanced object-oriented techniques for modeling robotic systems, IEEE Int'l Conf. on Robotics and Automation, May 1995.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
255
Augmented Interaction: Interacting with the real world through a computer J u n Rekimoto *~ Sony Computer Science Laboratory Inc., 3-14-13 Higashi-Gotanda, Shinagawa-ku, Tokyo 141, J a p a n This paper discusses why traditional GUI is not adequate to support highly portable computers, and proposes a new HCI style called Augmented Interaction, which is concentraining on the user's real world activities. Situation awareness and implicit interaction are the two key ideas of this concept. We also report on the prototype system called NaviCam, which is based on the idea of Augmented Interaction. 1. INTRODUCTION: F r o m GUI to post-GUI As the term human computer interaction implies, we have addressed interfaces between humans and computers for a long time. However, regardless of the fact that we are living in the real world, we have paid less attention to the gap between the computer and the real world. As a result, the interface between h u m a n s and computers and the interface between humans and the real world are not well integrated. Recent progress in hardware technology has produced computers that are small enough to carry or even wear. However, these new computers, often referred as PDAs, preclude the use of traditional user-interface techniques such as graphical user interfaces (GUIs) or the desk-top metaphor. The fundamental limitations of GUIs can be summarized as follows: • GUIs can reduce the cognitive overload of computer operations, but do not reduce operations themselves. This will be a problem for portable computers. When users bring their computers into their daily lives, they do not pay much attention to them. Instead, they prefer interacting with each other, and interacting with the real world. Consequently, how to reduce computer manipulation itself becomes an issue as well as how to make manipulation easier and more understandable. • Portability means that computers will be used in various situations in the real world. Thus, the functionality required for mobile computers is dynamically changing. Traditional GUIs are not designed for such a dynamic environment. Although context sensitive interaction is available on GUIs, such as context sensitive help, GUIs do not deal with real world contexts. GUIs assume desk-top computers and users at a desk, where the real world situation is less important. *[email protected], http://www.csl.sony.co.jp/person/rekimoto.html
256
....:::::::::::::::::::::::::::::::::::::::::::::: .,v,v,v,v,v,.:,: :: ::
(a) GUI
R
@
(b) Virtual Reality
@
R
oz/
v i
(c) Ubiquitous Computers Computer World ~Real
World
(d) Augmented Interaction Human - Computer Interaction
--~:~: Human - Real World Interaction Real World - Computer Interaction
Figure 1. A comparison of HCI styles
2. A U G M E N T E D I N T E R A C T I O N To address the problems described in the previous section, we propose a new concept of human computer interaction called Augmented Interaction. Augmented Interaction is a style of human-computer interaction that tries to make computers as transparent as possible. Using this style, a user will be able to interact with the real world augmented by the computer's synthetic information. The user's situation will be automatically recognized by applying a range of recognition methods, allowing the computer to assist the user without having to be directly instructed to do so. The user's focus is not on the computer, but on the real world. The computer's role is to assist and enhance interactions between humans and the real world. A wide variety of recognition methods can be used for this concept. Time, location (by using GPS), and object recognition by computer vision are possible examples. Also, we can make the real world more understandable to computers, by putting some IDs (bar-codes, for example) on the environment. Figure 1 is a comparison of HCI styles from the viewpoint of humans and the real world interaction. (a) In a desk-top computer (with a GUI as its interaction style), interaction between the user and the computer is isolated from the interaction between the user and the real world. There is a gap between the two interactions. (b) In a virtual reality system, a computer surrounds a user completely and interaction between the user and the real world vanishes. (c) In the ubiquitous computers
257 LCD Display
I
'T i!ii!ii~::::
CCD Cameras Head-up configuration
Palmtop configuration
Figure 2. NaviCam: outer view
environment, a user is interacting with the real world but can also interact with computers throughout the real world. (d) Augmented Interaction supports a user who is interacting with the real world, by computer augmented information. The difference between (c) and (d) will be discussed in Section 4. 3. THE P R O T O T Y P E SYSTEM: N a v i C a m
As an initial a t t e m p t to realize the idea of Augmented Interaction, we are currently developing a prototype system called the "real world navigator" or NaviCam. NaviCam is a wearable computer t h a t has a small video camera to detect real-world situations. This system allows a user to see the real world together with context sensitive information generated by the computer. NaviCam has two hardware configurations. One is a palmtop computer with a small CCD camera, and the other is a head-up display with a head-mounted camera (Figure 2). Both configurations share the same software. The palmtop configuration extends the idea of position sensitive PDAs proposed by Fitzmaurice [3]. The head-up configuration is a kind of video see-through HMD [1], but does not shield the user's real sight. In both configurations, the user can interact directly with the real world, but can also see the computer augmented view of the real world. The system uses color-codes to recognize real world situations. A color-code is a sequence of color stripes (red or blue) printed on paper that encodes the ID of a real world object. For example, a color-code on the door of the office identifies the owner of the office. By detecting a specific color-code, NaviCam can recognize where the user is located in the real world. The information flow of the system is shown in Figure 3. First, the system recognizes a color-code in front of the camera. Entire image processing is performed by software at a rate of 10 Hz. Next, NaviCam generates a message based on the real- world situation. Currently, this is done simply by retrieving a database record by using a color-coded ID as a key. Finally,
258
Head-up Display (VirtualVision)
.....i ~ i ~
Real World Environments
a
CCD Camera .,
l lllll
.ll
l lll.iH
Figure 3. NaviCam: the system architecture
the system displays a message on the captured video image (i.e., superimposition). Figure 4 shows a snapshot of the screen image. We have built two applications using NaviCam. One is a museum navigator that recognizes which picture the user is looking at and generates explanations about that picture. The other example is an intelligent door. In this application, a door with a color-code can act as an intelligent and active object. When the user is standing in front of the room entrance, the system recognizes an ID on the door and gives information regarding the room. For example, a meeting schedule of the conference room is automatically retrieved from the database and displayed. Though these applications are very simple, they demonstrate the great potential of situated interaction. The user can get highly context sensitive information without issuing any command. By regarding real world situations as implicit inputs from the environment, computer manipulations can be greatly simplified. These prototypes are developed to demonstrate how we can interact with the real world without being bothered by computer operations. However, it is also true that such operations are still required for more complicated (and probably more practical) applications. In such a case, we can combine several kinds of traditional interaction techniques with NaviCam to enhance communications between the user and the system. A touch-sensitive panel on the NaviCam screen is a possible choice. Another possibility is voice. Nagao and the author are currently developing a spoken language dialogue system called Ubiquitous Talker, on top of NaviCam [4]. Using situation information from the real world, Ubiquitous Talker allows a user to have situated conversations with the system.
4. RELATED WORK 4.1. Ubiquitous computers Augmented Interaction looks similar to ubiquitous computers [7]. Both aim to create a computer augmented real environment instead of building a virtual envi-
259
A message superimq on a real image
color code
Figure 4. A screen image of NaviCam
ronment in a computer. The main difference between the two is in their approaches. Augmented Interaction tries to achieve its goal by introducing a portable or wearable computer that takes real world situations as implicit commands, while ubiquitous computers realizes the same goal by spreading a large number of computers over the environment. These two approaches are complementary and can support each other. The main problem with ubiquitous computers is their reliability. In a ubiquitous computers world, each computer has different functionality and requires different software. It is essential that they collaborate with each other. However, if our everyday life is filled with a massive number of computers, we must always expect some of them to not work correctly, because of hardware / software troubles, or simply because of their dead batteries. It is very difficult to detect such problem computers and fix them. Another problem is the cost. Although the cost of computers is getting cheaper and cheaper, it is still costly to embed a computer in every document in an office, for example. In contrast to ubiquitous computers, Augmented Interaction's ID awareness is a low cost and reliable alternative to making everything a computer. Suppose that every page in a book has a unique ID (e.g., bar-code). When the user opens a page, its page ID is detected by the computer, so the system can supply specific information regarding the page. When the user has some comments or ideas while reading that page, the user can simply read them out; the system records the voice information with the page ID tagged for later retrieval. This scenario is almost equivalent to having a computer in every page of the book but with very little cost. ID-awareness is better than ubiquitous computers from the viewpoint of reliability, because it does not require batteries, does not consume energy, and never breaks down. 4.2. C h a m e l e o n Chameleon [3] is a spatially-aware palmtop computer. Using locational informa-
260 tion, Chameleon allows a user to navigate through a virtual 3D space by changing the location and orientation of a palmtop in his hand. Locational information is also used to display context sensitive information in the real world. For example, by moving Chameleon toward a specific area on a wall map, information regarding that area appears on the screen. Using locational information to detect the user's situation, although a very good idea, has some limitations. First, location is not always enough to detect situations. When real world objects (e.g., books) move, the system can no longer follow the situation. Second, detecting the palmtop's position itself is a difficult problem. The Polhemus sensor used with Chameleon has a very limited sensing range (typically 1-2 meters) and is sensitive to the other magnetic devices. Relying on this technology forces the user's activity to remain within a very limited area. Obviously, combining several information sources (such as location, real world IDs, and time) should increase the reliability and accuracy of situation detection, although the problems are not trivial. This will be the future direction of our research. 5. SUMMARY In this paper, we proposed a new human-computer interaction style called Augmented Interaction. Augmented Interaction is designed for the highly portable computers of the future, and concentrates on reducing computer operation by accepting real world situations as implicit input. We also reported on our prototype system that recognizes real-world situations by using computer vision. REFERENCES
1. Michael Bajura, Henry Fuchs, and Ryutarou Ohbuchi, Merging virtual objects with the real world: Seeing ultrasound imagery within the patient, Computer Graphics, volume 26, pp. 203-210, July 1992. 2. Steven Feiner, Blair Macintyre, and Doree Seligmann, Knowledge-based augmented reality, Communication of the ACM, Vol. 36, No. 7 (1993), pp. 52-62. 3. George W. Fitzmaurice, Situated information spaces and spatially aware palmtop computers, Communication of the ACM, Vol. 36, No. 7 (1993), pp. 38-49. 4. Katashi Nagao and Jun Rekimoto, Ubiquitous Talker, Spoken language interaction with real world objects. Technical Report SCSL-TR-95-003, Sony Computer Science Laboratory Inc., Tokyo, Japan, 1995. 5. Jakob Nielsen, Noncommand user interfaces, Communication of the ACM, Vol. 36, No. 4 (1993), pp. 83-99. 6. Jun Rekimoto, The World through the Computer: A new human- computer interaction style based on wearable computers, Technical Report SCSL- TR-94013, Sony Computer Science Laboratory Inc., April 1994. 7. Mark Weiser: The computer for the twenty-first century, Scientific American, (1991).
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
261
InfoBinder: A Pointing Device for a Virtual Desktop System Itiro Siio Tokyo Research Laboratory, IBM Japan, Ltd. 1623-14, Shimotsuruma, Yamato-shi, Kanagawa-ken 242, Japan e-mail: siio @trl.ibm.co.jp
InfoBinder is a new wireless pointing device that provides an information-binding function in a virtual reality environment. Each device has a unique ID number and is mapped to an object such as a telephone directory in the computer system.
1. Virtual desktop This paper discusses the application of InfoBinder to a virtual desktop system, shown in Figure 1, where a computer-generated image is projected onto a work table. This type of desktop display was proposed by P. Wellner [ 1]. The projected image used in my system is a common desktop metaphor consisting of icons and windows. In this metaphor, icons and windows represent computer objects such as documents, folders, trash cans, printers, and database objects. An icon represents an object that has been shrunk to make it easier to handle. A window represents an object that has been opened to allow the information it contains to be browsed and manipulated. In addition to these computer objects (virtual objects), real objects such as sheets of paper, telephones, and calculators can also coexist on the same work table.
Projector
Video camera
.........;;;;¢
:::::::~i....... i
f jr"". :// .,
.,..
""%o'%..;.-...
•
Trash~ ' ) - - - 7 .::
....
~
.,
..
f;~ im ~ ...........\ (~, ~\ ~
/ Document Folder I ~
InfoBinder
Telephone Figure 1. Virtual desktop system
Printer
262
2. Issues in virtual desktop InfoBinder is intended to resolve the following issues in virtual desktop systems. 1. Projected objects are difficult to handle because they have no solid bodies that can react to manipulation by the user. 2. Real and virtual objects on the desktop could work cooperatively, that is, the functions of real objects could be expanded by means of a virtual objects. For example, projecting a list of phone numbers could give a telephone an additional function as an online directory. It should be easy for users to add the new functions to real objects and to understand them.
3. InfoBinder hardware
Topview ~
LED Pushbutton
Sideview Figure 2. Right: InfoBinder device Left: It can be attached to a real object such as a telephone by velcro Figure 2 (right) shows an InfoBinder, a small pointing device less than 2 inches long, with a push button and an LED. When a user pushes the button, the LED is lit by a battery inside the device. The position of the device is detected by means of a video camera positioned over the table that senses light from the LED, as shown in Figure 1. The light from the LED is modulated to carry a unique ID number for each InfoBinder. This mechanism allows each InfoBinder to be identified. More than one InfoBinder, each with a unique ID number, can be used at the same time on the desktop.
4. I n f o B i n d e r m o d e InfoBinder works in two modes:
1. Pointing device mode This is the initial mode. An InfoBinder works like a conventional pointing device such as a mouse. It can manipulate open objects in window form. When a window is closed, the InfoBinder goes into the information binding mode.
263
2. Information binding mode Each InfoBinder can hold an object if the object's windows is closed. This follows the analogy of a paper binder that holds information written on sheets of paper. In this mode, the icon of the object follows the InfoBinder when the user drags the device. Since each InfoBinder is identified by a unique ID number, each can hold a different object. When the device is double-clicked, the bound object is released and opened as a window. The device then goes into the pointing device mode.
•\ \,
Figure 3. The icon follows when an object is bound to an InfoBinder.
iiiiiiiii' ~-~
°@
I
®
II ~l] I, = ,I
loooll I a a a[i}J
Figure 4. An example of online phone number objects.
5. Example of a telephone directory To understand how the device is used, let us take an example in which it is holding an online telephone directory. A user can handle the device as if it were a real binder holding a bundle of papers on which phone numbers are written, and can arrange them, store in a drawer, or even move them to another virtual desktop connected by a network.
264 The InfoBinder can be attached to a telephone with velcro, as shown in the left side of Figure 2. Although the metaphor is a simple one of attaching a memo containing a phone number to a telephone, it gives an ordinary telephone a powerful directory function. This is an example of cooperative work between a real object (a telephone) and a virtual object (an on-line directory). To use the phone number object, the InfoBinder should be removed when it is attached to a telephone. If the device is dragged on the desktop, the icon of the object will follow the device (Figure 3). If the icon is double-clicked, a phone number window will be opened on the desktop (Figure 4). At this moment, the InfoBinder will go into the pointing-device mode, and the user can search for a number and dial it. When the window is closed by means of the device, it becomes an icon and is bound to the device again.
6. Merits of InfoBinder The InfoBinder device resolves the issues described in Section 3. First, it provides a concrete representation of a virtual object. It gives solid bodies to icons. Users can handle the objects represented by projected icons easily, because they can feel a reaction, and could arrange, store, and carry such objects as if they were handling, say, a real paper binder holding information. Second, the functions of real objects could be expanded by attaching the InfoBinder to them. In the above example, a simple telephone acquires an on-line directory and dialing function in cooperation with the projected virtual object. The InfoBinder shows this mechanism in an intuitive manner. Users can easily understand how to add and use the new functions.
7. Implementation A first-stage prototype was built, using a Proxima projector and a video camera system. The projector and the camera were installed 140 cm above a work table with dimensions of 150 cm x 80 cm, and a computer image was projected on the table (Figure 1). The prototype InfoBinder device has a red LED and a push-button switch. No ID function has been implemented yet. The position of the device is detected by the video camera system, and transferred to the mouse port of the computer. An object-holding mechanism and a phone directory object have been implemented and tested in the HyperCard environment. A demonstration video tape will be shown at the conference.
Reference 1. Pierre Wellner, The DigitaIDesk Calculator: Tangible Manipulation on a Desk Top Display, UIST '91, November 11-13, 1991. Hilton Head, South Carolina.
III. 10 Evaluation and Analysis 1
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
267
An Object Oriented Methodology for Man-Machine Systems Analysis and Design A. Mahfoudhi, M. Abed, J-C. Angu6 Laboratoire d'Automatique et de Mrcanique Industrielles et Humaines URA CNRS 1775 Universit6 de Valenciennes et du Hainaut-Cambrrsis B.P. n ° 311 59304 VALENCIENNES CEDEX-FRANCE
Despite the recent progress in the domain of Man-Machine Interface engineering, several problems concerning the incompatibility between the information presentation to the user and his cognitive representation remain. This paper presents a new Task Object Oriented Description methodology (TOOD), especially adapted to the taking into account of the human factors for the specification of the Man-Machine Interfaces (MMI). A concrete application of this methodology was presented in the air traffic control context.
1. INTRODUCTION The interactive systems development always presents ergonomic problems, essentially, bound to the man-machine communication. These problems can be generally related to the taking of human factors into consideration and the presentation of the information to the human operator (what, when and how). So, it's very important to have models and methods that permit, on the one hand to make more accessible the users' knowledge, and, mainly, more formal and more detailed their description. On the other hand to permit the specification of the (Man-Machine Interface : MMI). There are two main approaches aiming at appropriate solutions for these problems. The first one concerns the users' tasks analysis and description methods. These methods, such as MAD (Scapin and Pierret-Golbreith, 1989), the SADT/Petri method (Abed, 1990), DIANE (Barthet, 1988), try to describe the task as a set of operations and, at the same time, they integrate implicit information about human factors. However, most of these methods present the same disadvantage; the managed treatments are not detailed, and the MMI specifications are not completely integrated. The second approach, aiming at offsetting the disadvantages of task analysis and description methods, concerns the object oriented methods. An example of these methods is the OOA method (Coad and Yourdon, 1991) or the interactive cooperative objects formalism (Palanque, 1992). These methods have several advantages; they are easily usable, clear and favor the reusability and the modularity. However, they also present some disadvantages. Account of their partial use (most often in the end, just before the programming), the object oriented methods take into account only the software engineering aspects in which case the considerations about users and human factors are completely missing. It' s only natural that our decision enjoys advantages of those two approaches. So, we propose a global Task Object Oriented Description (TOOD) methodology beginning from the task analysis and description to the MMI specification, based on the object oriented techniques and Object Petri Nets (OPN). Our goal was to test this methodology in a real complex environment (air traffic control). So, this paper provides a formal description of the air traffic controllers' task and the exploitation of this description for an ergonomic and complete specification for the future air traffic control interface PHIDIAS (Position Harmonisant et Integrant les Dialogues Interactifs, Assistances et Secours).
268 2. DESCRIPTION OF THE AIR TRAFFIC CONTROLLERS' TASK The interface design to be implemented in a new system begins by analyzing the existing or similar system and the operator's current tasks, and raising both negative and positive interface aspects. These aspects are determined by taking into consideration the factors and relationships between the user, the tasks and the interface (Reiter and Oppermann, 1993). The relationship accomplish tasks describes how a user can carry out the tasks. The relationship usability describes how difficult it is for the user to use the interface. The relationship functionality or utility describes how well the interface supports the tasks and allows the user to reach the task goals. So, it's very important to have a method that permits, to make more accessible the users' knowledge, and more formal and more detailed their description. For this, the TOOD methodology was found to be suitable not only for a simple task but also for a complex tasks such as the air traffic controllers' task.
2. 1. TOOD : Task Object Oriented Description TOOD is a new ergonomic methodology which tries to relate the characteristics of the user's task with those of the Man-Machine Interface. It uses two complementary approaches : the first one for the users' tasks analyzing and describing and the second for the MMI specification (cf. 3). For that TOOD uses the genetic term "Task-Object" which indicates each task. Indeed, the task-object is defined as an independent entity and responsible for a treatment, whatever his complexity level, to reply to a goal to be carried out with given conditions. The task-object has a graphic form, inspired from the HOOD formalism (Michel, 1991) and Extended SADT method (Feller and Rucker, 1990), presented in Fig. 1.
E
o'~. R
'°'u' ~.N/i
I I ~~":
'° 4:-
'i i
.......
......1
LJ
ou,,
i iff~..t.: LJ',.il
o
..........
Resources
Figure 1. Task-object control structure A task-object is also defined by a set of attributes, called "descriptors" : a name which identifies the tasks, a set of events (E) which defines the necessary events for the task release, a set of control/command data which defines the constraints to be respected, a set of Input data (I) which defines the list of data and information transformed by the task performance, a set of Reactions :(R) which defines the reports of the task performance, a set of Output data (O), a set of Resources (R) which defines the necessary human and material entities for the performance of the task, and a body which identifies the actions or sub-tasks to be carried out.
2.2. Identification and specification of the air traffic controllers' task The first stage of the TOOD methodology is the identification of the tasks of the future system. Then by a hierarchical decomposition, it organizes the identified tasks-objects in a hierarchical tree form. It starts from the global task-object (the hierarchical tree's root) passing through the least abstract task-objets (the knots) and finishes with the terminal task-objects (the leaves). Let us consider the air traffic control, "to configure the flight entry" can be regarded as a task-object. In order to reduce its abstraction, this task-object can be decomposed into three children task-objects : "T111 : take knowledge of a new flight" (terminal task-object), "T112 : take decision about flight" and "T113
269
: Check the position on radar screen" (terminal task-object). It is to be noticed that the events which activate the same task-object are shared out among the children task-objects. As shown on figure 2, the task-object "T11 : configure the flight entry" can be activated by two events "El 1-1 : Arrival of a new flight" and "E 11-2 : Proposition of an entry level (EFL)". Yet those events activate two different children task-objects. Which means that both events ask for two different processing of the task-object T 11. Thus the event E11-1 activate the task-object "T111 : take knowledge of a new flight" to read information about the new flight while the event E11-2 suppose that the flight information have been read and activate the task-object "T112 : take decision about flight". Once all future system tasks are identified, the second stage of TOOD concerns the task specification. It consists in listing and identifying all the descriptors or attributes of each task-object. The resulting document of this specification includes two kinds of description : a graphic description for a clean, legible and exploitable representation, (figure 2) and a textual one for a complete description of the descriptors of each task-object (Mahfoudhi and Abed, 1994).
/ 1 I
[
ITII"~
Configure the flight entry
/"
!...................7:E e_=, ...........LL 9_'_I ,'~ £ E11.1
.. c111.1 .1~'i
~ ~"~"~''G .... 2"'~P[
~
~-~1
I/
"%L~..--I ~~
{
............ ~ ~ ~"~:":"'1111-2 "''~ Ik /T-: lv'
"J
I
t
~ii
~ ~ i
~
I
~.
I
\_L.,..C~,~.j"
I i ~1;
I "1
~
i
I ~ I °'1~1 {
i
I
0111-2
...... ,L,,._.,~
c11~.1
~(:~:t .... i
1113-1
®
.~ .i
Figure 2. a graphic specification of the task-object "T11 : to configure the flight entry" As for the rest of this paper, we devote our study to the terminal task-object "T111 : to take knowledge of a new flight". So we give the formal description and the simulation of its performance. 2.2. Task Control Structure (TCS) : a simulation tool
Most methods present the disadvantage that their descriptors are not exploited and their treatments are not detailed. TOOD has found a solution to this problem. Indeed, it adds a TCS (Task Control Structure) to each task-object (a gray net of figure 1). The TCS is modelized by a Colored Petri Net (Jensen, 1987) which we add three functions : (f) the input distribution function which selects the necessary input and control/command data to activate the task-object with a given event, (g) the output distribution function which selects the produced output data with a given reaction, and (5) a priority function which arranges the events of the task according to their importance (alarms, interruption, temporal constraints, etc...). At any moment, the TCS must determine, by the equation (1), the task-object state; (treatment authorized, waiting for an event, waiting for a resource, producing a reaction, etc...).
270
M'=Mo +W.S • M' • M0 • W
(1)
: current marking of the TCS (vector with 7 dimensional). : initial marking of the TCS (vector with 7 dimensional). : incidence matrix of the TCS (matrix with a 7 , 2 dimensional) (2). The (i, j) element of this matrix is equal to the difference between the Post (Pi, tj) and Pre (Pi, tj) functions tl
w
*S
=
t2 0 0 0 l -q
-fe -fc -fl -k h 0 0
gR go
P1 P2 P3 P4 P5 P6 P7
(2)
• A firing sequence of the task-object Tk (3). It's a vector with 2 dimensional indicating with which event and reaction to fir the input transition tl and the output transition t2 <Ek,j> such as ~5(Ek,j) = sup (~5(Ek,1),fi(Ek,2)....) s
=
(3)
In the e x a m p l e of the task-object "T111 : take knowledge of a new flight", the Organic controllercan open and read the new flight information only after receiving the event "El 1-1 : Arrival of a new flight" convoying the display of the Strip,corresponding to the new flight, in the new stripstable.Once he decides to read information then to execute the task-object T111, the organic controller must read the input data I111-1, I111-2 and I111-3 not only to refer to the control/command data "C 111-2 : flight direction and type" which influences his way to read input data, but also to respect the control/command data "C 111-1 : Wait time".
In the TCS, the task-object performance finds expression in the presence of a token <.> in the place P6 (Figure 3b), given by the firing of the transition tl. The marking M ' I of the figure (3b) can be also defined by the equation (1).
P3~p~~P1
P3(~p~T~ (~P1
I /.
I
I
k
P4~ l I
fE
N~N~ ~C/t 1
g
k
P5
~iiii~iii~ :::::::::::::::::::::::: q
/
:o \gRt2
,,66,6 Imo(P1)[ <EIII-I> mo(P2)I I <12111-1> + <(2111-2> mo(P3)l + + M0= too(P4)[ = <M-I> + <M-3> mo(P5)I 0 mo(P6)I o mo(~) I o
/a/Before the performance
P5
~ii~4~ l
iii~m q
q
~go\ g . t2
/
I
P766 m'1(P1) m'l(P2) m°l(P3) M' 1 = m'l(P4) m'l (P5) m'l(Pr) m'l(P7)
l
I
P6
0 0 0 =
fE
I k
0 <~> 0
/b/ During the performance
m~(P2)~ m~(P3)~ M' 2 = m~(p4)]rn~(P5)~ m~(P6)I
m~(PT)I
/~o \agR 6 oo
<M-l> + <M-3> 0 + +
°
/c/ After the performance
Figure 3. Performance and simulation of the task-object "T111 "to take knowledge of a new flight"
I
271
3. USER INTERFACE SPECIFICATION The aim of this stage is the automatic passage from the users' tasks description to the MMI specification. It allows to define all the necessary action plans and manipulated objects for the task performance. So, the resources of each terminal task-object become its component-objects which include MMI objects to be implemented in the future Interface, and Operator objects. All the component-objects co-operate in a precisely manner in order to fulfill the aim of the terminal task-object. A component-object shall be defined from its class (Interface or Operator) and provided with a set of states and a set of operations (or actions) which allow to change these states. Graphically, the component-object is presented in an identical structure that the one of a taskobject. However its internal control structure called Object Control Structure "ObCS", inspired by the cooperative interactive objects formalism (Palanque, 1992), is modelized by an Object Petri Net "OPN" (Sibertin, 1985).The OPNs are characterized by the fact that the tokens which constitute the place markings are not atomic nor similar entities, but they can be distinguished from each other and take values allowing to describe the characteristics of the MMI and operator. The terminal task-object "T111 : take knowledge the new flight" needs using two componentobjects : MMI object "a New Strips Table : NST" and operator object " Organic Controller : OC" (figure 4). The comportment of the MMI object "a New Strips Table" is defined by four states P1, P2, P3 and P4. From each state the Organic Controller can carry out a group of actions (transitions). From the P3 state (strip selected), for example, he has the possibility to achieve two actions : t3 ( open a road-zoom) or t5 (temporize the new strip).
ITl11
Take knowledge of a new flight ~,,
A new strips table
f
Organic Controller OC
'-"iii. . . . . . j ~ f
t...~
t
Fig. 4. A graphic Specification of the component-objects "New Strips Table" and "Organic Controller"
"
272
For the component-object "Organic Controller", the set of his states and his operations represents the different possible procedures to perform the terminal task "T111 : take knowledge of a new flight". So, the display of a New Strip NS in the MMI object "new strips table" invokes, by the event E2,1, the operation service "Consult the NS" of the operator object "Organic Controller OC". According to his selection "Ch=", the organic controller make a first reading of the NS information ("Consult the road" or "Consult the level"). After this reading, he changes his state into cognition in order to evaluate his information level. Then he decides to "read again the basic information" or "to ask for additional information". The asking for additional information expresses itself by a change of his state into "Action" in order to "select the NS" and to "open the Road-Zoom". Both actions transmit R2,2 and R2,3 reactions to the MMI object "new strips table". It is to be noticed that the organic controller carries out the action "open a road-zoom" only after receiving the event E2,2 confirming that the action "Select the NS" has been carried out. Once the Road-Zoom has been opened, the Organic Controller changes his state into "information reading" in order to read the additional information and then into the "situation evaluation" state to decide either to read again the information, or "to temporize the NS" or to invoke, by the reaction R2,1, the terminal task-object "T112 : Take decision about flight".
4. CONCLUSION The TOOD methodology enjoys the contributions of methods and concepts taken from cognitive sciences and ergonomic domains together with those of the software engineering domain. It provides a framework of efficient collaboration between various users and between ergonomists and computer specialists. Its formalism allows to define in a formal, coherent and structured model the task knowledge and to specify an adapted interface to the users' characteristics.
5. REFERENCES
Abed, M., 1990, Contribution ~ la modrlisation de la tfiche par des outils de sprcification exploitant les mouvements oculaires: application ~ la conception et l'rvaluation des Interfaces Homme-Machine, Doctorate thesis, University of Valenciennes France. Barthet, M-F., 1988, Logiciels interactifs et ergonomie : modules et mrthodes de conception, DUNOD Pads Coad, P. and Yourdon, E, 1991, Object Oriented Analysis, 2nd Ed., Yourdon Press Computing Series. Feller, A. and Rucker, R., 1990, Extending Structured Analysis Modelling with A.I.: An Application to MRPII Profiles and SFC Data Communications Requirements Specifications, IFIPS Conference. Jensen, K., 1987, Colored Petri Nets, In Petri Nets : Central models and their properties, LNCS Nb. 254, Spring Verlag. Mahfoudhi, A. and Abed, M., 1994, Description orientre objet des tfiches du contrrleur pour la sprcification et la conception des interfaces, Contract research report CENA 94, University of Valenciennes France. Michel, L., 1991, Conception orientre objet : Pratique de la mrthode HOOD, Dunod Press Pads. Palanque, P., 1992, Modrlisation par objets Cooprratifs Interactifs d'interfaces homme machine didgges par l'utilisateur, Doctorate thesis, University of Toulouse I France. Reiter, H. and Oppermann, R., 1993, Evaluation of user interfaces, Behaviour et Information Technology 12 (1993), 3, 137-148. Scapin, D.L. and Pierret-Golbreith, C., 1989, MAD : Une Mrthode Analytique de Description de Tfiches, Actes du Colloque sur l'ingrnierie des Interfaces Homme-Machine, France. Sibertin, B.C, 1985, High-level Petri nets with Data Structure. 6th European Workshop on Petri Net and application, Espoo, Finland.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
273
An Analysis of Relationship Between Human and Information System by Quantification Theory 11I Tsuneki Mukahi, Ken Murasugi and Tetsuo Ui Department of Industrial Management, Osaka Institute of Technology, 5-16-10miya, Asahi-ku, Osaka, 535 Japan The purpose of this study is to clarify the framework for understanding the relationship between human and advanced, large-scale information systems by analyzing the empirical data gathered using questionnaire. As the result, three axes of 'goodness of relationship between human and information system', 'attitude of organization toward the information system', 'stress felt by individuals in the information system' were found out. A new image of 'an advanced information-based organization' was obtained from these axes. It was also observed that such 'an advanced information-based organization' has been already realized in some organizations.
1. INTRODUCTION Advanced, large-scale information systems have been introduced in many organizations. To make full use of the ability of workers within such organizations, we should make it clear the relationship between human and information systems. Though the effects given on human by a computer have been well investigated in the field of psychology and ergonomics (e.g. Brod 1984), the number of studies on the impact to individuals by a networked information system is much smaller. However, we can find some studies concerning the relationship between the networked information system and the organization or job (e.g. Whisler 1970). Among these studies, it has been pointed out, for instance, that the information system brought fiat organization or job autonomy. Now that we can see the close relationship between the networked information system and the organization or job, we have to take into consideration the effects given on the individual human by the system through the organization and job, in order to grasp the relationship between the system and the individual human. The purpose of this study is to clarify the framework for understanding the relationship between human and advanced, large-scale information systems.
274 2. METHODOLOGY We gathered 584 sample data from five organizations using advanced information systems. Questionnaire consisted of items concerning the introduction of computer systems, how workers relate with computer systems, how they think of them, and some demographics. Quantification theory 1II was employed to clarify the potential axes which explain the relationship between human and information systems. Ten items were analyzed: 'instruction given when the system was introduced', 'time to spare caused by using the system', 'opportunity for self judgment', 'effect on ability development', 'stress', 'active use of information', 'changes of the way to use the system', 'information shared', 'prevention of mistakes on input' and 'satisfaction on using the system'. Each item included three choices, which sums into 30 choices altogether (throughout 10 items). 3. RESULTS Three axes were clarified by quantification theory 111. Category score of each axis is presented in Table 1. Figure 1 combines the first and the second axis, and Figure 2 combines the first and the third axis. In the first axis, scores of the categories such as 'J1. very satisfied', 'A1. sufficient instruction', 'B 1. time to spare obtained', 'F1. active use of information', 'H3. information well shared', 'I1. mistakes well prevented', became large. On the contrary, scores of the categories such as 'J3. unsatisfied', 'A3. insufficient instruction', 'B3. time to spare lost', 'F3. inactive use of information', 'HI. information shared limited', 'I3. mistakes not prevented', became small. Thus, the direction of plus in the first axis expresses the good relationship between human and information system, while the direction of minus expresses the bad relationship. So, the first axis is considered as 'goodness of relationship between human and information system'. In the second axis, scores of the categories such as 'C3. opportunity for self judgment not increased', 'E3. stress not felt', 'F3. inactive use of information', 'G3. the way to use it not changed' became large. On the contrary, scores of the categories such as 'C1. opportunity for self judgment increased', 'El. stress felt', 'F1. active use of information', 'G1. the way to use it changed', became small. Thus, the direction of plus in the second axis expresses active use of the information systems by organizations, while the direction of minus expresses inactive use of the information systems. So, the second axis is considered as 'attitude of organization toward the information system'. In the third axis, scores of the categories such as 'El. stress felt', 'F3. inactive use of information', '13. mistakes not prevented', 'B 1. time to spare obtained', 'B3. time to spare lost', 'C1. opportunity for self judgment increased', 'C3. opportunity for self judgment not
275 Table 1 Category scores by quantification theory Ill 1st axis
2nd axis
3rd axis
A1. Sufficient instruction
0.0410
-0.0321
-0.0204
A2. Fair instruction
0.0078
0.0096
0.0120
-0.0281
-0.0079
-0.0163
B1. Time to spare obtained
0.0203
-0.0021
0.0152
B2. Unchanged
0.0112
0.0096
-0.0180
-0.0428
-0.0469
0.0265
Items
Categories
A. Instruction
A3. Insufficient instruction B. Time to spare
B3. Time to spare lost C. Self judgment
C1. Increased C2. Unchanged C3. Not increased
D. Ability development
E. Stress
D1. No negative effect
G. Changes of the way to use
J. Satisfaction
-0.0154
0.0044
0.0137
0.0308
-0.0336
0.0126
0.0020
0.0159
0.0179
D3. Negative effect
0.0066
-0.0049
-0.0267
El. Stress felt
-0.0220
-0.0536
0.0381
E2. Stress a little felt
-0.0069
-0.0007
0.0073
0.0112
0.0106
-0.0145
0.0197
-0.0205
0.0008
F2. Sometimes use
F1. Active use
-0.0101
0.0085
-0.0163
F3. Inactive use
-0.0165
0.0236
0.0487
G1. Changed
-0.0144
-0.0440
0.0168
0.0042
-0.0039
-0.0092
G3. Not changed
-0.0069
0.0360
0.0228
H1. Shared limited
-0.0262
-0.0233
0.0092
H2. A little shared
-0.0078
0.0181
-0.0025
0.0243
-0.0082
-0.0023
H3. Well shared I. Prevention of mistakes
0.0143
0.0118
-0.0215
G2. Sometimes changed H. Information shared
-0.0250
D2. Not conscious
E3. Stress not felt F. Active use of information
0.0182 -0.0118
I1. Well prevented
0.0259
0.0032
-0.0041
I2. A little prevented
-0.0037
-0.0006
-0.0062
I3. Not prevented
-0.0232
-0.0027
0.0207
J1. Very satisfied
0.0556
-0.0345
-0.0283
J2. Satisfied
0.0065
0.0082
0.0081
-0.0261
-0.0182
-0.0188
J3. Unsatisfied
increased', became large. On the contrary, scores of categories such as 'E3. stress not felt', 'D 1. no negative effect on ability development', 'A1. sufficient instruction', 'A3. insufficient instruction', 'J 1. very satisfied', 'J3. unsatisfied', became small. In this axis, category whose
276
v'~3 mmF3
mH2 ~3
oo !---4
-0 05
ml~--'I" 1D~t
mI3 mA3
lI1 1B1 IH3
mi$3
mJ3 ~I1
or)
0.36
mF1 mAl
mnl
mj
mG1
IB3
1El V, VV
FIRST AXIS Figure 1. A distribution chart of the 1sff2nd axes combination.
NF3'. w
mE1
mC3
IB3
(.o
lI3
I---I
ID1 mill
1G3
roD2
m~l
mpJ~l
m~
mE2
=J2
I--I
-0, 05
mH2 lI
~2 mE3 mAl
0.03
~3
m j:
FIRST AXIS Figure 2. A distribution chart of the 1st/2nd axes combination. scores marked both plus and minus values was 'stress'. Here, the direction of plus expresses the existence of the stress felt in the information system, while the direction of minus expresses the non-existence of the stress felt in it. So, the third axis is considered as 'stress felt by individuals in the information system'.
277 4. DISCUSSION 4.1. An advanced information-based organization
The quadrant made by the combination of the first axis-plus and the second axis-minus expresses the good relationship between human and information system, and active use of information system. In other words, this quadrant shows that the information system is employed as successfully as the organization expected. So, we call this type of organization 'an advanced information-based organization'. In the concrete, the quadrant includes items which are 'J1. very satisfied', 'A1. sufficient instruction', 'C1 opportunity for self judgment increased', 'F1. active use of information', and 'H3. information well shared'. Furthermore, we calculated the average of sample scores of each organization and applied t-test in comparison of the average of each organization and the total average (Table 2). As a result, a tendency is observed that an organization A belongs to 'the advanced informationTable 2 Calculation of the average of sample scores of some items Items
Categories
Organization
A B
Sex
Age
Section
0.0057*** 0.0009
2nd axis -0.0033*** 0.0040*** -0.0007
3rd axis 0.0000 0.0019"*
C
-0.0059***
D
-0.0006
0.0002
-0.0018"*
E
-0.0008
-0.0008
-0.0019"
Men
-0.0008
-0.0010*
-0.0008*
-0.0001
Women
0.0019"*
0.0027**
0.0017"*
-24
0.0007
0.0007
0.0011
25-34
-0.0008
-0.0005
-0.0004
35-44
-0.0003
0.0003
-0.0003
0.0004
0.0006
45-
0.0024**
System development
0.0006
System use Class
1st axis
-0.0023**
-0.0008
-0.0002
0.0007
0.0002
Manager
0.0011
-0.0012"
-0.0009
Common
-0.0009
0.0004
0.0002
* "p<0.05, ** "p<0.01, *** "p<0.001
278 based organization'. An organization A is famous for CIM. Thus 'the advanced informationbased organization' can be considered as an ideal condition of organization which is to employ the advanced, large-scale information system. 4.2. Calculation of the average of sample scores of some items
We also calculated the average of sample scores of some items in order to clarify further characteristics of the advanced information-based organization and the three axes (Table 2). In sex, the women have good relation to the systems and belong to inactive organization. On the contrary, the men belong to active organization. Thus, we suppose from this result that women are engaged in monotonous work. In age, the middle-age workers have good relation to the systems. Usually, it is considered that young workers adapt to the information systems because of their physical strength or capacity for adopting of new technology. But this result become opposite to such consideration. So, we can guess that middle-age workers can adapt to information systems in a certain. In section, system development section belongs to active organization. It is no wonder that such a result come out. So, this result supports that the axes are correct. In class, managers belong to active organization. This result supports previous studies that managers receive better information after computerization (e.g. Atiyyah 1988). 5. CONCLUSION We clarified three axes of 'goodness of relationship between human and information system', 'attitude of organization toward the information system', 'stress felt by individuals in the information system' by analyzing the empirical data. And a new image of 'an advanced information-based organization' was obtained from these axes. Based on the results, the analyses must be made between these axes and the other items in the future studies. Then the image of 'an advanced information-based organization' will emerge more clearly. REFERENCES
Atiyyah, H.S. 1988 Computer impacts on Saudi Arabian public bureaucracy.
Organization
Studies, 9(4), 511-528. Technostress. Reading, MA : Addison-Wesley. Whisler, T.L. 1970 Information Technology and Organizational Change. Wadsworth. Brod, C. 1984
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
279
TOWARDS AN EFFECTIVE SUBJECTIVE MEASUREMENT METHOD BASED ON FUZZY SET THEORY* Hiromi Terashita, Mieko Ohsuga, Futomi Shimono, and Mamiko Toda Mitsubishi Electric Corp. Central Research Laboratory 8-1-1, Amagasaki, Hyogo, 661 Japan
1. INTRODUCTION For quantitative evaluation of human behavior like an operator's mental workload, we have been studying the relationship between physiological responses and subjective measures under several experimental stressful conditions. For this purpose, it seems useful to develop a system as a research tool which collects behavioral and environmental indices including psychological assessment data triggered by physiological response patterns in order to analyze the correlations among these parameters effectively. To develop such a system, it is necessary to examine the methods for subjective evaluation that allow subjects to express their feelings easily and consistently. We think that the constraints of the measurement procedures often cause inconsistency in subjective rating scores partly due to lack of the consideration of the imprecision of subjective judgment processes. Hence, the purpose of this study is to examine the most suitable methods for subjective evaluation that can be applied to the system under development. From the perspective of psychometric theory, it is suggested that fuzzy set theory provides a possible solution to such methodological problems. It takes into account the reality of the imprecision of human thoughts by allowing ranges of scores to be measured and translated into a single score [1]. It also provides a means for humans to express judgments qualitatively but precisely, formalizing the use of verbal judgments [2]. So, it is appropriate to introduce fuzzy set theory in measuring subjective feelings. Psychometric methods including the ones provided by fuzzy set theory were compared by conducting two experiments. The scaling methods investigated in this study are as follows :Graphic Rating Scale, Fuzzy Categorical Rating Scale, and Fuzzy Graphic Rating Scale. Graphic Rating Scale (GRS) : A respondent checks the one point which represents his rating on a continuous graphical scale usually with two anchors. *The present study is supported by MITI's Project on "Human Sensory Measurement Application Technology".
280
Fuzzy Graphic Rating Scale : A respondent is presented with the option of indicating a preferred point and extending the rating directly on a continuous graphical scale. Here we used three ways to express a subjective extent : checking two points (FGRS2), checking three points according to Hesketh et.al. (FGRS3)[1], and checking four points according to Yoshikawa & Nishimura (FGRS4)[3]. Fuzzy Categorical Rating Scale (FCRS) : The subjective extent of an attribute measured with rating categories such as "very tall" is quantified by providing a set of scale values for the categories with linguistic truth values which were pre-experimentally obtained with the fuzzy graphic rating scale. It improves the quality of the scale values while the categorical rating scale neglects the vagueness [3]. Figure 1 shows the scoring methods.
l
! i'~:
:
'•
l
i; , ' ~,i -v |, ' '
,
,
,
i•
I
B, |
|
i
I,
,
i
I;i . . .i
: i',_:'! • |
I
b i
I I n
. I
a. C a t e g o r i c a l b. Fuzzy Categorical c. Graphic d. Fuzzy G r a p h i c e. Fuzzy G r a p h i c f. Fuzzy Graphic Rating Scale R a t i n g Scale Rating Scale Rating Rating Rating Scale (FCRS) Scale ( GRS ) (2 points: FGRS2) (8 points: FGRS3) (4 points: FGRS4) Scale
Figure 1. The scoring methods.
2. EXPERIMENT 1 • PSYCHOLOGICAL ASSESSMENT WITH THE PENCIL & PAPER TECHNIQUE 2.1. M e t h o d
In the first experiment, twenty subjects (seventeen male and three female employees of our company, aged from twenty-two to thirty-eight) participated in a psychological assessment task with a set of questionnaires written in Japanese. All subjects performed two assessment sessions, a physical property assessment and an emotional state assessment under experimental stress tasks using four kinds of evaluation methods : Fuzzy Categorical Rating Scale (FCRS), Graphic Rating Scale (GRS), and two types of Fuzzy Graphic Rating Scale (FGRS2, FGRS4). The experiment was controlled by a Macintosh IIfx computer. In the physical property assessment session, subjects were asked to estimate the number of dots randomly arranged on a 13 X 13 dot matrix. Twenty kinds of patterns ranging from 1 to 169 dots in number were presented on the CRT display for 3 sec each. The subjects estimated the number of dots using a graphic rating scale with two anchors ranging from "a few" to "many." They used an expression of quantity rather than numbers, i.e. "a good deal of ", "a great number of, "a few", "many" etc., when using categorical ratings. Twenty trials for each method were carried out in random order. In the emotional state assessment session, subjects performed one mental arithmetic task and one written addition task, and were asked to rate the
281 feelings of tension, discomfort and fatigue after each task for each method. This was repeated four times for each task. Scaling procedures were carried out for fuzzy categorical rating scale values. Subjective measurement of linguistic variables such as "slightly, " "fairy," "extremely" and so on, were recorded using graphic rating scales with two anchors of "none" and "completely" in order to identify the fuzzy membership functions and to obtain psychological scale values of linguistic variables for fuzzy category ratings. Thirteen successive categories and nine successive categories were used for physical property assessments and emotional state assessments respectively. This scaling phase was conducted both before and after each assessment session. 2.2.
Results
The psychological scale values for the fuzzy category ratings were obtained to give the trapezoidal fuzzy sets for the linguistic truth values matching the best area to 1 level set, and the maximum area to the support set respectively, and these were interpolated between their ends linearly. The centroid method was used for defuzzification. The linguistic truth values were the estimates of scale values for the primary categories with the same hedges. Figure 2 shows some examples of the results of the physical property assessments, and Figure 3 shows some examples of the results of the emotional state assessments. The test - retest reliability of linguistic variables for FCRS was examined by correlatingf scale values obtained at two scaling phases and was confirmed with correlation coefficients of 0.95 to 0.99 in all the subjects for physical property assessments (Figure 2a). For emotional state assessments, the ratio of cardinality of intersection set to union set of a fuzzy membership function was examined. Table 1 shows the number of people who demonstrated the overlap of membership function for each linguistic variable. oq Sub.16
100 m
~
m
.
50
0
oo
o
o
oo
50 100 scores at scaling 1
a. An example of test-retest reliability of linguistic variable scores in two scaling phases in subject 16.
Sub.16
100
-'-
.
~5
~
,
0
1
2
No. of dot(log) b. A comparison of rating scores calculated with four scaling methods. + :fuzzy categorical rating score (FCRS) * :graphic rating score (GRS) O:fuzzy graphic rating score(FGRS2) × :fuzzy graphic rating score(FGRS4)
Figure 2. Example results of physical property assessments.
282
Sub.9 1
H ~ ' 0.5
Sub.9
100
I
0
8 50 x ~!!i~ 0
x 50
x 100
score
a. An example of fuzzy membership functions and defuzzied scores (X) for linguistic variables at scaling phase 1 in subject 9.
0
O
¢
~, c
mental arithmetic
addition
b. An example of subjective rating scores with 4 rating methods in emotional state assessments. a.tension, b.fatigue and c.disgust + :fuzzy category rating score (FCRS) * :rgraphic rating score (GRS) O:fuzzy graphic rating score (FGRS2) × :fuzzy graphic rating score (FGRS4)
Figure 3. Example results of emotional state assessments.
The results indicated t h a t the scores obtained from these rating methods correlated to each other with correlation coefficients of above 0.90. It showed improvement in accuracy of expressing the range of category values in spite of the limitation of using discrete categories when introducing FCRS values obtained in the scaling procedures. However, some subjects showed inconsistent of j u d g m e n t s for several categories and the problem of such inconsistency remained. The results also indicated t h a t subjects preferred both graphic and fuzzy graphic rating scales for physical property assessments, while they preferred verbal expressions for emotional state assessments, though the most suitable rating method differed from person to person (Table 2).
Table i The overlap of membership function on linguistic variables. (The n u m b e r of subjects, the ratios are a: >0% and b: >50%. *The Japanese verbal hedges used in the study and their provisional translations are shown for reference.) Linguistic Variables ~ l~ k A~U ~b ~ 9 ~ ~ k ~- b ~ t~ 9 ~ o ~- ") :2" < 9 ~ bz
: not at all* : hardly,scarcely : slightly : alittle : not much, not very : fairly " quite : very : extremely
a.
b.
19 16 14 18 14 19 17 19 18
12 8 6 8 12 5 7 9 9
283 Table 2 S u m m a r y of the evaluation of rating methods. (The number of subjects, a" Categorical Rating Scale, b" Graphic Rating Scale, c • Fuzzy Graphic Rating Scale 2 (FGRS2), d" Fuzzy Graphic Rating Scale 4(FGRS4) ) Physical Property Emotional State Assessments Assessments a. b. c. d. a. b. c. d. easy to respond easy to judge uneasy to respond uneasy to judge highly loaded
7 6 4 5 3
7 7 1 2 0
5 1 9 7 0 12 1 14 3 1 12 3 1 12 2
8 5 2 3 1
2 3 1 1 1
1 0 14 13 12
3. I M P L E M E N T A T I O N OF A S S E S S M E N T M E T H O D S
According to these results, it seems to be most appropriate to apply various kinds of methods suitable to each individual. Therefore, five methods of subjective m e a s u r e m e n t were chosen to implement the system. A program written in HyperCard2.1.J for a Macintosh Computer interactively elicits these rating methods by presenting a HyperCard Stack card with one rating item on the screen. A subject manipulates the indicators on the card and presses the "OK" button when the card's display best represents the rating of the card's item. A mouse and a trackball are used as input devices. Examples of the assessment displays used in the study are shown in Figure 4. 4. E X P E R I M E N T 2 • P S Y C H O L O G I C A L A S S E S S M E N T COMPUTERIZED TECHNIQUE
WITH THE
In the second experiment, twelve out of twenty subjects in the first experiment participated in the psychological assessment task with computerized rating scales. Subjects perform an emotional state assessment session consisting of ten mental arithmetic tasks with ten experimental conditions combined with five evaluation methods and two input devices ( mouse / trackball ). After the experiment, seven subjects chose a mouse and five subjects chose a track ball as the suitable input device. Referring to evaluation methods, five subjects selected categories(FCRS), two selected GRS, one selected FGRS2, two selected FGRS, and two selected FGRS4 as the best method. The results of the second experiment indicated that, again, the most suitable rating method and input device differs from person to person. 5. C O N C L U S I O N S Though further study is still needed to examine the score values obtained from different methods, the consistency of subjective judgments, and the system usability, it is possible to conclude from the above results t h a t these computerized fuzzy rating methods can be quite effective for subjective
284
8
g
t$
"9
tz
c. Fuzzy Graphi c R a t i n g Scale (2 points : FGRS2)
item to be evaluated linguistic variables
M~t~(tension) t$
g
#-,: (didn't feel)
(felt)
~
~
c-z
a. C a t e g o r i c a l R a t i n g Scale
d. Fuzzy G r a p h i c R a t i n g Scale (3 points : FGRS3)
~$yr$ i~dieator
=
/j, [ tz
ii •
m
=
t; Jr.-
fz t:
)uttons Reset & O K but
b. G r a p h i c R a t i n g Scale (GRS)
e. Fuzzy G r a p h i c R a t i n g Scale (4 points : FGRS4)
Figure 4. Examples of the assessment displays. assessments. Because of the ability of FCRS to use ordinary verbal expressions at the assessment stage, we are now trying to add voice input capability to the system which is expected to improve the constraints on judgments under working situations. REFERENCES
I. B. Hesketh, et.al., in T. Zetenyi(ed.)Fuzzy sets in psychology, North-Holland, (1988), 425-454. 2. L. Zadeh, .et.al.,Fuzzy sets and their applicationsto cognitive and decision process, Academic Press,N.Y., (1975). 3. A. Yosikawa & T. Nishimura, Journal of Japan Society for Fuzzy Theory and Systems, vol.5,no.4 (1993), 719-731.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
The Design and Experiment
285
of a n E v a l u a t i o n F u n c t i o n for U s e r
I n t e r a c t i o n C o s t in t h e I n t e r a c t i v e S e m a n t i c D i s a m b i g u a t i o n Masaya YAMAGUCHP, Nobuo INUI, Yoshiyuki KOTANI, Hirohiko NISIMURA Department of Computer Science, Tokyo University of Agriculture and Technology, 2-24-16 Naka-cho, Koganei, Tokyo, 184, JAPAN E-mail: {masaya [nobu I kotani}@cc.tuat.ac.jp In this paper, we will design a function which evaluates the cost of user interaction that resolves the semantic ambiguity in natural language processing system. Through user interaction, the system shows some example sentences that express each meaning to let the user select one from them. We define the evaluation function based on semantic and syntactic features of verbs. 1. Introduction Natural Languages have semantic and syntactic ambiguity. To resolve them is one of the main problems in natural language processing. Especially, for practical NLP systems(for example, the machine translation system), it is a serious one. In this paper, we will handle the disambiguation of semantically ambiguous words. We have proposed a method of disambiguation by user interaction[l]. In this method, if a system can not automatically resolve semantic ambiguity in a sentence, it shows some example sentences that express each meanings of an ambiguous word to let a user select one from them. Such an interactive system needs to hold down the user interaction cost as little as it can. One of solutions is to consider the easiness of disambiguation among words and the order of user interaction, since the result of the disambiguation of one ambiguous word may constrain the meanings of other ambiguous words. To hold down the user interaction cost, our system in [1] searches all paths and minimizes the cost of user interaction. After this process, it interacts with a user actually; We used the following strategy: (1) calculate each user interaction cost, (2) get the order in which the system can interact with a user i n t h e minimum user interaction cost. We think that "each user interaction cost" depends on the semantic similarity. Therefore we define "user interaction cost" as the probability that a user may interact correctly. This is based on the assumption that the probability is small if it is difficult for a user to select one example sentence. The whole of user interaction cost in which the system identifies the single semantic structure of input sentence is the probability that a user can select example sentence correctly in all user interactions. We will obtain here an evaluation function which calculates user interaction cost. We will focus on the evaluation function for verbs.
286 2. I n t e r a c t i v e D i s a m b i g u a t i o n 2.1. A M e t h o d of I n t e r a c t i v e S e m a n t i c D i s a m b i g u a t i o n In this section, we will illustrate our method of the interactive semantic disambiguation [1] briefly. We assume in the method that a user user the system under the following situation. First, the system finishes to resolve the syntactic ambiguity before user interaction process. Second, the system can identify the semantic structure of all example sentences which are shown to a user. Followings are the process of interaction. 1. the system constrains the meanings of words in an input sentence semantically, and evaluates the semantic structures of the input sentence by means of some preference and gets the most plausible semantic structure. 2. it searches all possible orders of user interaction, and gets the order with the minim u m user interaction cost. 3. it interacts with a user according to the order of interaction to get the semantic structure. It shows some example sentences for an ambiguous word to let a user select one from them. We show an example of user interaction in Figure1. The parenthesized word is the ambiguous word.
f Input S e n t e n c e : Watashi wa Kare ga Katta Tama wo [ Keru ] (I kick the b a l l t h a t he bought) (1) Kono Urea ha Sugu Hito wo Keru (This horse often kicks a man) (2) Kaisya ha Kate no Youkyuu wo Keru (The company refuses h i s demand) Please select a number
==> i
Figure 1. Example of User Interaction
The function to evaluate user interaction cost and evaluation elements are needed when the system searches the minimum user interaction cost. We will define each evaluation element in the next section at first, and get an evaluation function experimentally. 2.2. R e l a t e d W o r k s
So far, some interactive natural language systems have been proposed. Maruyama et al.[2] proposed a method to resolve the ambiguity of modifier-modifiee relationships between phrases. He reported the relation between the length of the input sentence and the number of interaction, and the improvement of the percentage of correct answers.
287 Raft et al.[3] showed a method to resolve the ambiguity of the slot filler in their semantic structure, and a user interface for user interaction using a multi window system. Since their system shows more than one targets of user interaction to each window at the same time, a user decides which window he begins to interact. Kameda et al.[4] showed the system that resolves the syntactic and semantic ambiguity interactively. It can interact with two ways: system-directed and user-directed interactions. In the former way, it uses the strategy that it asks a user in order of the amount of information for the user. Since these method have a lack of knowledge about the control of the order of user interaction, more efficient ways are considered by means'of introducing control strategies. Our method would be applied to such a problem.
3. Evaluation Function 3.1. A M o d e l of Selecting an Example Sentence As described in Section 2, our system shows example sentences for an ambiguous word to let a user select one from them. We define the model of selecting an example sentence as follows. At first, a user understands the meaning of an input sentence, and determine the meaning of the target verb(that the system is going to resolve) in it. Next, he interprets each example sentence, and checks whether it's case elements match to the input sentence's case dependency structure, and also checks the similarity of verbs He does it for all example sentences. Therefore, we assume that he would do such a process for all example sentences, if he find an example sentence halfway that he intends to select. Finally, he selects the most similar example sentence with respect to the case dependency structure and the meaning.
3.2. Definition of Evaluation Elements We define the following evaluation elements as ones that have influence to user interaction cost: (a) semantic similarity, (b) syntactic characteristic, (c) the number of example sentences. These relations are calculated between an input sentence and example sentences that the system shows to the user.
Semantic S i m i l a r i t y 'Semantic similarity' Sem(mk, mi) is defined as the similarity between the meaning mk of the verb in the input sentence that the system is going to disambiguate and the meaning mi of the verb in an example sentence. We assume that a value of Sem(mk, mi) is proportional to the difficulty of selecting an example sentence for users. A following expression therefore denotes Sem(mk, mi) as the relation between mk and mi on the thesaurus, provided that dp is the depth from the root concept to the lowest super concept that they share:
Sem(mk, mi) = dp + 1 Syntactic Characteristic "Syntactic characteristic"(SC) is characterized, comparing the case dependency structure ck of mk with an example sentence. "Impossible to recognize SC" is the situation
288 that all case elements of an example sentence ei match with the ck in the system lexicon, while "Possible to recognize SC" is otherwise. A following expression Syn(ck, ej) denotes recognizeability between ck and ei.
Syn(ck, ei) =
1 possible to recognize S C 0 impossible to recognize SC
T h e N u m b e r of E x a m p l e S e n t e n c e s "the number of example sentences" is defined as the number of the meanings that the system could not disambiguate. We assume that it has influence to user interaction cost because the number of user's judgment increase. This element is described as an parameter in the evaluation function. 3.3. Design of E v a l u a t i o n F u n c t i o n We define a function to evaluate the user interaction cost, based on the model of selecting an example sentence. At first, we think about the user interaction that the system shows two example sentences to a user. We define the probability Pt that a user can select an example sentence correctly as follows, provided that ml is the meaning that the user intends to select in his mind, Pt(a, ~) = 1, t is a threshold, and P1, P2, P3, P4 are probabilities that a user can select an example sentence correctly in each condition. We will find their value in Section 4 experimentally.
I P-, Pt(Sem(ml, m2),Syn(c~,e2)) =
P2 P3 P4
(Sem(ml,m2) > t, Syn(c~,e2) = 1) (Sem(m,, m2) > t, Syn(c,, e2) = O)
(Sem(m~,m2) < t, Syn(c~,e2)= 1) (Sem(m,, m2) < t, Syn(c,, e2) = O)
As above, we classified the relation between the input sentence and each example sentence by two evaluation elements for four categories. Next, we think about the user interaction that the system shows n(n >_ 2) example sentences to a user. We define the probability Pm,(di) that a user can select an example sentence correctly as follows, provided that dj is a j th user interaction in which the system shows n example sentences to a user. Pm,(dj) expresses the process that the system repeats to evaluate the input sentence and each example sentence using Pt, and finally decides the example sentence with ink.
Pm,(di) = Pm,_l(di) x Pt(Sem(mk, m,_a), Syn(ck, e,_l))) Finally, we define the total user interaction cost DC for the interaction order dl, d 2 , . . . , d,~ in which the system can identify the single semantic structure of the input sentence. The number of example sentences in user interaction of dx, d2, . . . , d,~ is nl, n 2 , . . . , nm respectively.
DC = Pm,,(dl) x Pm,2(d2ldl) x . . . x Pm,m(dmld~_,,...,dx) 4. E x p e r i m e n t s of User Interaction 4.1. Conditions of E x p e r i m e n t s Here, we will describe common conditions for the following experiments.
289 T h e Classification of Verb's M e a n i n g We use the classification of verb's meaning defined in [5]. In [5], the meaning of the verb is classified by the difference of semantic and syntactic structure. Each lexical item stores its case dependency structure, its semantic marker of the meaning classification and its example sentence that expresses its meaning. The case dependency structure consists of the pairs of a case marker and a set of semantic feature that can be placed in the case. The marker of the semantic classification shows a place on a thesaurus[6]. The maximum depth of hierarchy is 4. E x a m p l e Sentences, I n p u t Sentences a n d S u b j e c t Example sentences are described in [5] which the system shows to a user. Each input sentence can uniquely be recognized, but some words in them are ambiguous. We make all input sentences so that all users can identify their meanings. We select the verbs from [5] at random. Subjects are three students. They belong to our laboratory, however, are not linguistics experts. 4.2. P r e l i m i n a r y E x p e r i m e n t The objective of this experiment is to find the probabilities P1,..., P4 and the average time of user interactions. P1,..., P4 is determined by the result of Tablel and Syn(ck, ei). Therefore, subjects, in this experiment, select a correct one from two example sentences against an input sentence. The Tablel shows the percentage of correct selections for each degree of the similarity and the average time of user interactions for 36 verbs with two meanings.
Table 1 The relation between semantic similarity and the rate of correct selection interaction time/rate of correct selection) Sem(mi, mi) Frequency Subject A Subject B Subject C 1 11 11.7/90.9 7.26/90.9 6.30/81.8 2 6 7.51/83.3 7.17/83.3 5.39/83.3 3 0 m 4 1 4.79/100 5.64/100 3.38/100 5 18 22.4/66.7 14.9/83.3 8.20/94.4
(%: average Average 8.42/87.8 6.69/83.3 4.60/100 15.2/81.5
4.3. Discussion In Tablel, the percentages of correct selection of the subject A and B decrease(except Sem(mi, mj) = 3, 4), as the degree of the similarity increases. This result proves our assumption about "Semantic similarity". That of the subject C, however, did not. We found, in this experiment, the average time of incorrect user interactions is longer than that of correct user interactions. The later is 1.69 times as long as the former in the preliminary experiment. Therefore, to consideration user interaction cost may make the interaction time short.
290 5. E v a l u a t i o n
At first, We do an experiment of user interaction using verbs with more than two meanings in order to evaluate the evaluation function, and get wrt and Wrg. W,~ is the verb that a user interacts correctly, while w,g is the verb that a user interacts incorrectly. The method of evaluation is as follows: (1) calculate Pm(w,t) and Pm(w,g). (2) judge that the result is correct, if Pm(wrt) < Pm(w,g). (3) do (1), (2) for all combination of Wrt, Wrg.
Table2 shows the rate of correct results and undecided result for each t. The average of the maximum rate for each subject is 64.0%. Therefore the system can avoid the incorrect interaction in the rate by using the evaluation function, when two ambiguous word would exist in an input sentence. Table 2 The rate of correct result for each t (%: correct/undecided) t=l t=2 t=3 t=4 Subject A 68.1/1.72 70.7/1.72 70.7/1.72 71.6/1.72 Subject B 52.6/1.72 62.1/1.72 62.1/1.72 61.2/1.72 Subject C 57.5/1.15 58.2/2.68 58.2/2.68 58.2/2.68
6. Conclusion In this paper, we designed a function to evaluate user interaction cost in selecting one example sentence, and experimentally determined parameters of it. By using the evaluation function, we estimated user could select an example sentence with correct usages of verbs in the rate of 64.0% by using the evaluation function. REFERENCES 1. Masaya Yamaguchi, Nobuo Takiguchi, Yoshiyuki Kotani, Hirohiko Nisimura: An Interactive Method of Semantic Disambiguation in Sentences by Selecting Examples, NLPRS'93, pp.51-58, 1993 2. Hiroshi Maruyama, Hideo Watanabe, An Interactive Japanese Parser for Machine Translation, COLING-90, pp.257-262, 1990 3. Ralf D.Brawn, Sergei Nirenburg, Human-Computer Interaction for Semantic Disambiguation, COLING-90, pp.42-47, 1990 4. Masayuki Kameda, Shin Ishii, Hideo Itoh, Interactive Disambiguation in a Japanese Analysis System, Technical Report of IPS:I. NL-84-2,1991 5. IPA Technical center, IPAL(Basic Verbs) dictionary volume, IPA,1987 6. Susumu Ohno, Masando Hamanishi, Ruigo Kokugo :liten, Kadokawa-syoten, 1990
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
291
An Analysis of the Human-Computer Interfaces to High-Energy Physics Control Systems at CERN J. F. Meech a, P. Huuskonen b , E. Wagner c, M. Meri b, J.-M. Le Goffb aUniversity of the West of England, Bristol, UK bCERN, ECP Division, CH-1211 Geneva 23, Switzerland cUser Interface Design AB, Sweden
1. INTRODUCTION This paper describes an investigation of the user-interfaces to high-energy physics experiments at CERN, Geneva. As part of this project a study of existing interfaces was carried out together with an analysis of operator characteristics and responsibilities. The results of this study are presented together with recommendations for the design of future user-interfaces for High-Energy Physics experiments.
1.1 CERN: The European Laboratory For Particle Physics CERN (the European Organisation for Nuclear Research) is an international scientific organisation carrying out research related to nuclear particle physics. Researchers from various member and subscriber states conduct high-energy physics research at this facility in Geneva, Switzerland, by contributing in various ways to the experiments conducted.
1.2 High-Energy Physics Experiments A number of variously sized particle-accelerators are located at CERN, with experiments using both fixed targets (where particles are collided into a stationary target) and colliders (where two particle beams are collided into one-another). The largest particle beam is LEP (Large Electron Positron collider). LEP is a circular accelerator in which e ÷ (positrons) are circulated in one direction and e- (electrons) in the other. The beam is housed in an underground tunnel around which are placed several experiments (e.g. ALEPH, OPAL, DELPHI and L3, see Figure 1). These experiments consist of a series of detectors which are used to detect, identify and track the sub-atomic particles which result from electron and positron collisions. The experiments carried out at CERN are intended to discover more about the fundamental nature of matter. LEP will be superseded at the turn of the millennium by LHC (the Large Hadron Collider) in which particles of higher energy will be used. L3 (See Figure 2) is a representative current HEP (High-Energy Physics) experiment that consists of a detector system built around an interaction point at which two particle beams collide together. The particles generated at this interaction point are influenced by large superconducting magnets and pass through several detectors intended to establish different aspects of each particle in order to correctly identify each particle and its path. There are multiple detectors for each experiment, e.g. gas ionisation systems (a time projection chamber that allows particle path to be identified), calorimeters (which allow the measurement of particle energy), etc. The next series of experiments proposed at CERN represents a significant
292 challenge due to the scale and complexity of the systems proposed which will exceed current experiments by an order of magnitude. e~i N FILEPH
! 8I,GenisL~
'
i
'~
/r
"~ GENEVA Figure 1. The circular LEP particle beam path, showing experiment positions. 1.3 The CICERO Project CICERO [ 1] (Control Information systems Concepts based on Real-time Objects)is a largescale collaborative project that aims to build a genetic control and information system for future experiments at CERN. The project's motivation is the need to improve the integration of complex control systems both at CERN and, ultimately, in industrial systems. The CICERO project will produce an integrating object-oriented software framework (into which distributed user control objects may be plugged) and a supporting control information system for the configuration and management of that system.
Figure 2. A schematic of the L3 HEP experiment (note scale of person at bottom left). The CICERO project aims to build a control structure for the "slow-control" (the automatic and supervisory systems that control the operation of an experiment) systems of future experiments, and part of this project is concerned with analysing the requirements for the enduser (operator) interface for future systems. The design of the controlling interface for such systems is of critical importance, both for safety reasons and also to ensure that valid data is captured in an experiment. This paper outlines some of the findings of the first phase of examining the user-control systems interfaces of existing experiments at CERN. 2. EXISTING C O N T R O L SYSTEMS As part of the analysis phase of HEP Human-Computer Interfaces, six existing interfaces and the associated systems were examined. In addition, the work and organisational aspects of the
293 environment were studied to identify the characteristics of the users or operators of the systems.
2.1 Operators As part of the initial phase of the project the characteristics of slow control systems operators were studied to identify operator tasks and responsibilities. Currently operators come from the institutes that participate in collaborations doing the research in an experiment. They are essentially 'free labour', as they handle the slow control tasks as a side activity whilst they collect physics data. The operators rarely have time to fully familiarise themselves with all the experiment's subsystems, as the personnel turnover at the experiment is high. During active data taking, operators are physicists who are more concerned with the actual experiment than with the detector's equipment. They know the experiment's subsystems only superficially, since typically they only spend a few days or weeks in this occupation. Therefore, they have to be considered 'naive' users in the sense of not knowing the slow control systems. Slow control documentation is somewhat unfamiliar to them and some operational situations can be quite problematic. In addition to these day-to-day operators there are expert physicists who understand the experimental equipment in detail. The experts are called in for problem situations. They know the subsystems well, but often have little knowledge about computer or software technology. Arriving on the scene they have little time to understand the situation, to gather more information and then make informed decisions on what to do. An additional problem is that the original situation can no longer be reconstructed when the expert arrives at the experiment. Finally, some of the subsystems are constantly changing and new systems are being constructed, adding to the problems. As the operators are often unsure of what to do, they must call in the experts. Therefore, the experts get urgent calls even in cases where the problem could have easily been handled by the operator. It seems that the operators have enough information at hand, but, being inexperienced in the environment, are not able to digest it all and act accordingly.
2.2 User Control Tasks Three investigative techniques were used to analyse the user-interfaces to current systems. Firstly a task analysis was performed on several systems to identify the tasks and their relationships with one another. Videotape was used to record users walking-through experimental control, whilst describing what they were doing and why they were doing it. A variant of Hierarchical Task Analysis was used to analyse the resulting video and verbal protocols [2]. Secondly a display audit of existing systems was carried out to identify the number and content of each display involved in each experiment. Thirdly the models used to describe the 'process' involved in each experiment were analysed. From an experimental point of view, the structure of each experiment is quite similar (at least to the non-physicist!) consisting of a collision chamber surrounded by magnets (to influence the particle tracks) and several different detectors (to identify and track the particles). As a representative example we will examine ARGUS [3], a Macintosh-based browser interface to the L3 experiment (See Figure 1). It enables data associated with the L3 slowcontrol systems to be examined and also displays alarm status of the various sub-systems (Figure 3). ARGUS identifies a series of navigational requirements for display traversal. In ARGUS navigation is possible as: 1. Physical co-locality (it is possible to examine two pieces of equipment that are physically close to one another, e.g. on a schematic of the experiment, but which are not directly linked in terms of process). 2. Functional/Task connectivity (e.g. two physical systems connected on a schematic diagram) 3. Functionally separate systems (e.g. telephone systems, environmental control etc.)
294 4. All navigational systems use hierarchical decomposition to allow progressively more detail to be displayed as the menu tree is navigated. Terminal display-tree choices usually display textual information. The display trees are rarely more than 4 levels deep.
Figure 3. Two screens from the ARGUS interface to the L3 experiment. The leftmost picture shows the initial high-level display. The picture on the right shows a more detailed process schematic. The generic task structure of HEP experiments appears to conform to classical ' supervisory process control task structure [4]. This provides for a period of system initialisation (process start-up) and shut-down with the main task being one of monitoring the system to ensure it stays within acceptable limits (i.e. limits which enable valid data to be collected). A generic representation of this task structure (based on several task analyses of experiments and accelerators) is shown in Figure 4. Current displays do not always make this task structure clear, and some navigation between screens is usually necessary to set-up and monitor an experiment. The process modelling phase examines the physical structure of the experiment (or process plant) and identifies the major functional elements which play a part in the process. The material, secondary input/outputs, sub equipment, process variables, information input/output and deviation from normal operation are then identified for each functional part. This allows the identification of display requirements to be made. This information may then be used to produce process displays for the user interface. In the control systems community there seems to be a tendency to place the emphasis in the data processing and control software, and communications and processing hardware. Often the processes to be controlled are explained briefly, if at all. There are few clearly identifiable process models (at the user-interface level). This is the opposite of industrial control, where the process descriptions sometimes dominate controls. Part of the reason for the lack of process modelling is the time overhead involved in the analysis and design of a process model when the experiment for which the process is described may only be short-lived. However for some systems a process model was observed to provide a valuable representative aid to operators. Of particular interest was attention to the state behaviour of the processes. Some control systems had gone to considerable lengths to identify the possible states that the processes could exhibit and then to design the control system on the state model. In one case this state modelling and state transition formed the basis of the operator interface with great success. Such models could be integrated into the graphical user interface and support systems to provide contextsensitive interaction. An audit of existing displays showed a wide variety of display designs, from monochrome textual displays to colour windows and detailed graphical representations. Display designs often varied within a single control-room environment, splitting functionality across displays. In some cases the separation of interfaces for control and data acquisition was not clear. From
295 this diversity it would seem that common display design guidelines for future experiments would benefit operators greatly and allow shorter operational learning times. Control slow-control systems
Start control systems 1
For Example, navigate through system-'~ in correct sequence, activating equipment
Maintain systems within operating paramete~rs
Shut-down systems
For Example, For Ex e, Activate data acquisition eqpt. navigate through system Monitor all systems for alarms in correct sequence, deactivating equipment
There is an additional task which may occur off line as well as on-line in response to a fault: that of analysing current and historical data Figure 4. The generic task structure of High-Energy Physics Slow Control Systems 3. I M P L I C A T I O N S F O R FUTURE E X P E R I M E N T U S E R - I N T E R F A C E S
3.1 Interfaces and Interaction The navigation required in operating the slow control systems requires that displays can be traversed using both physical co-locality (moving between information on equipment that is located in the same place) and on functional and task connectivity (i.e. moving through information structured according to the process model or task sequence used). The accuracy and explicitness of process models should be used at the interface to enable operators to understand the relationships between displayed data. Physical co-locality may be represented by digitising images of equipment and overlaying these with displays of the current and historic values of parameters (a domain user-interface). The use of process models and taskbased displays will require some investment at the design and implementation stage of the experiment to gather the knowledge and embody it diagramaticaly. The user interface must also encompass aiding, fault diagnosis and alarm monitoring and on-line documentation. Several operators and experts voiced a desire for prototype interfaces based on emergent technology such as virtual reality, multi-media and personal digital assistants (PDA's) for remote and roaming access to process behaviour and control. We hope to explore these options in a later phase of the project
3.2 Support Systems HEP operators differ from traditional process operators. They usually have an extensive education in physics and mathematics which allows them to quickly grasp abstract information. Operators rarely posses such abilities in industrial process operation. The extent to which the design of control systems should take this into account is an important subject for further study, including how well the existing body of research pertaining to process control is
296 applicable in an HEP setting. It may be that the levels of abstraction used in HEP control interfaces can be considerably raised as compared with traditional industrial systems. For the day-to-day operators of HEP controls, general training will not be able to offer explicit help for specific situations, so the operators must rely on their conception of the systems given by training, their general problem solving skills, and the technical data about the systems. Operator support systems would seem very beneficial here. The user's general knowledge about the system can be augmented by automatic reasoning capabilities (diagnosis, explanation, on- line documentation) to solve problems [5]. For instance, understanding alarm messages could be eased by offering explanation on the alarm's purpose, how it was generated, what events are related to the alarm, and how the operator should act. The burden on training could be lessened by offering on-the-spot help based on encapsulating expert knowledge. In this scheme, initial training could concentrate on giving general ideas about the systems, and more specific cases would be helped by on-line support. Part of the CICERO project is aimed at supplying such diagnoses, alarm-handling and on-line help facilities. 4. CONCLUSIONS This paper has provided a brief introduction to the challenges faced in the design of operator interfaces for high-energy physics experiments. Whilst this domain of study has some similarities with industrial process control, the operator characteristics and complexity of the physical systems and environment provide a multitude of challenges for interface designers. The results of an initial survey of existing user-interfaces has been described. This survey was based on user task analyses, process modelling and display audits. The results of this survey suggest that there are many possibilities for improving the design of operator user interfaces, with the penalty of an initial up-front investment of effort and time. It is hoped that with the advent of an integrated control framework for the experiments, it will be possible to provide design guidelines and new implementations of user-interfaces that will be generic to a wide range of future HEP experiments.
BIBLIOGRAPHY
1. For further details see Project RD38 at http://www.cern.ch/CERN/Research.html 2. Diaper, D. (Ed)(1989) Task analysis for Human-Computer Interaction, Ellis Horwood. 3. Barillere, R., Cabel, H., Diez-Hedo, F., Le Goff, J-M., Milcent,H., Pothier, J., Stampfli, R (1991) "ARGUS: A Graphical User Interface Package to Monitor the Slowly Changing Parameters of a Physics experiment on a Macintosh II" Paper presented at the CHEP'91 Computing in High-Energy Physics Conference, 11-15 March 1991, Tsukuba, Japan. 4. Meech, J. F. (1994) The Intelligent Management of Information for Human Interaction with Complex, Real-Time Systems. Paper presented at AAAI Spring Symposium on Intelligent Multi-Media, Multi-Modal Systems, Stanford University, USA March 21,22,23 1994. 5. Huuskonen, P., Kaarela, K., Meri, M., Le Goff, J.-M. (1994) "On Knowledge Representation for High Energy Physics Control Systems", Paper presented at the Workshop on Current Trends in Data Acquisition & Control of Accelerators (CTDCA '94) Calcutta, INDIA, December 6-8, 1994.
III.11 Evaluation and Analysis 2
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
299
P D S A n a l y s i s for E v a l u a t i n g P r o c e d u r a l U s a b i l i t y o n Conversational Systems Akinori KOMATSUBARA a and Masayuki KOBAYASHI b aKanazawa Institute of Technology, Ishikawa 921, J a p a n bcorporate Design Division, NEC, Tokyo 108-01, J a p a n This study discusses whether we can evaluate procedural usability by checking only guidance. Four experimental systems were developed: The systems consist of same sequence but the guidance is designed differently. H u m a n errors are compared among the systems. Based on the results of the experiments, the effects of guidance on procedural usability are to be discussed. 1 Introduction
Recently, various types of conversational type systems, such as personal facsimiles, video tape recorders with timers, multifunctional telephones, etc., have been appearing, not only in business offices, but also in our casual home life. In such systems, users can not arrive at their goal without following the predetermined operational sequence of the system. Consequently, novice users often complain about the difficulty of understanding the operational sequences, and sometimes fail in their use of the systems. This means that the procedural usability of these systems is poorly designed. Lack of procedural usability seems to be attributed to discord between the system model of operational sequences and the user model. The degree of this discord appears in operational complaints or errors. Therefore, to increase the operational usability, it is important to analyze and describe the user model in full detail, and to design the user model. Another approach that is considered is that interfaces should be designed to clearly show the system model that is designed by the system designers. This approach is expected to be suitable for such systems as home electronics facilities and public system terminals, because such systems are used by so many different kinds of users that user models seem to differ from each other or are vague because the users' previous experiences are different from each other. If we take the latter approach, only clearness of the system model, that is, clearness of the operational sequences, must be checked. Accordingly, in this study, a method for evaluating the clearness of the system model to increase operational usability is discussed. The effectiveness and limits of the method are also experimentally checked.
300
2 PDS Analysis and Guidance Evaluation Referring to D.A.Norman(1988), it is supposed that users of some systems will proceed through a "Plan","Do", and "See" cycle, and that they willprogress through a series of system operations while repeating these cycles. (1) "Plan" or developing intention means making the decision about what to do at the present step in the sequence. For Planning, the system must indicate the present situation and provide guidance for what should be done by the users. Otherwise, the users must previously have specific knowledge about what should be done at each particular system state. In other words, any information for planning must be offered by the system, or must be possessed by the users. Experienced users are expected to have the knowledge, whereas the system m u s t offer information for planning, that is guidance, for novice users because they usually have no knowledge that would be useful for planning. (2) "Do" or executing means activities for developing the planning. Doing is related to usability of hardware rather than procedural usability. (3) "See" or evaluating means evaluation of whether the "Do" is properly accepted by system. For seeing, any feedback must be offered by system. Feedback willact as guidance for planning at the next step of sequence. For novice users, guidance is expected to play an important role in procedural usability. In a system where planningis easy to do owingto clear guidance, users can go through the appropriate sequence. However in a system where improper guidance is given, the users will do incorrect planning, and will go into inappropriate sequence. In a system where guidance is not offered by the system, or users cannot do planning because the offered guidance is not understandable, users cannot progress in the procedure, or users may perform a scrambled "Do". Therefore, in order to evaluate procedural usability, it is important to demonstrate that suitable guidance is given. Feedback will be guidance for the following step. When feedback is not offered, users will have the misunderstanding that the system has not yet processed the prior step, and continue to wait to receive the feedback. In this case, users willnot be able to move to the planning of the next step. Otherwise, users will re-"Do" the prior step, because they believe that the "Do" was not accepted by the system. When the feedback is not understandable, users will do incorrect planning because they will believe that the right process was done whereas the wrong process was done, or they will re-"Do" or do an error recovery action, because they are convinced t h a t the wrong process was done, whereas the right process was actually accomplished by system.
301 Based on the above discussion, so as to make the system better in the area of procedural usability, especially for novice users, appropriate guidance must be offered at every step of the system sequence. Therefore, to evaluate the procedural usability of the system, we analyze and express the sequence into PDS cycles, and particularly check the "Plan" to see whether appropriate guidance is offered by the system. 3 Experiment 3.1 E x p e r i m e n t a l System In this study, four kinds of model e-mail sending systems were developed on a personal computer. They consist of the same operational sequence of eleven steps shown in table 1, but the conditions of guidance are different, as shown in table 2" Messages indicate what to do at that step. Colored buttons indicate the workable buttons at that step. Figure i shows an example of the screen of the system. The systems were developed on a Fujitsu FM-Towns personal computer with Towns Gear software. The systems are operated by pointing at the button icons on the screen with the mouse. In the systems of type 3 and 4, subjects receive feedback when button entry is accepted as blinking of the button, but no messages that indicate what to "Do" at the step are indicatedi Therefore a lot of errors related to procedural usability were expected to occur. On the contrary, in types 1 and 2, messages that clearly indicate what to "Do" at the step are displayed on the screen. No procedural errors were expected to occur because subjects can arrive at the task goal by just following the messages. Table 1 Operational Sequence of the Experimental System 1 e-mail 2 e-mail Kind 3 Character & Set 4 OK(Confirm) 5 ConnectingLine 6 External Line 7 Dial
8 Set
9 Start
10 Exit
11 Stop
Table 2 Condition of Guidance of Experimental Systems Messages Colored Buttons
~e
1
Yes
Yes
Type 2
Yes
No
2~rpe 3
No
Yes
Type 4
No
No
302
94/12/10 Connect
LINE
START
I liE 211 31 4 5101 171 s 9
I 11 01
Please
STOP
Connect LINE EXIT e-m a il KIND
SET
Figure i An Example of the Screen of the Experimental System (type 2) Twenty subjects were employed and five each were assigned to each type of experimental system. No instruction about how to use the system was given before the experiments. Subjects were required to operate the systems and to achieve the task goal, that is sending a document by e-mail, by interacting only with the system. 3.2 R e s u l t s of the E x p e r i m e n t s The behavior and protocol of each subject was analyzed using a video tape recorder. The mean operational time to achieve the final task goal of sending e-mail is shown in figure 2. Types 3 and 4, where no messages were given, needed more operational time than types 1 and 2, where messages as guidance were given. The systems where workable buttons were colored showed about 30 sec shorter time t h a n those without colored buttons. The mean operational number of entering buttons per subject is shown in figure 3. Types 3 and 4, without messages, needed two times more operations than types I and 2, with messages. This means that a lot of mis-entries were made in types 3 and 4, and that messages as guidance prevent the occurrence of mis-entries. No significant differences in the number of operations, however, observed between the systems with colored buttons and those without colored buttons. Coloring the buttons can not decrease mis-entries but can shorten operational times. If we consider the cognitive process of subjects behavior, subjects seem to decide on or "Plan" the button to operate at a step first, before searching for the button. Perhaps coloring buttons cannot help decision making but can help searching.
303 These experimental systems are designed with eleven steps to achieve the task goal. In type 1 and 2, in which clear messages are given, the number of operational entries is expected to be eleven to reach the goal if subjects just follow the messages. However, in types 1 and 2, the mean number of operations is more than eleven. This means that some errors occurred in types 1 and 2. Therefore, we analyzed the error types. It turned out that all errors were errors of entry order between the connecting line button, and dialingand start buttons. Several subjects operated dialing or start buttons, which should be entered at a later stage of the sequence, before entry of the connecting line button. These experimental systems are designed so t h a t the dialing and start buttons would not work unless the connecting line button is entered. Therefore, subjects who enter the dialingor start button before the connecting line button must redo entry of these buttons. Hence the total number of operations came to more than eleven. At the stage where the connecting line button should be operated, a message such as "Connect Line Please" is displayed on the screen. This message does not mention the dialingor start button. Therefore, it is seen t h a t subjects neglected the messages as guidance and made a "Plan" on their own. An interview was conducted with the subjects who made this kind of error. It turned out, as a result of the interviews, that they had thought vaguely without any specific reason that "Dialing must be done first because this system must be connected to the telephone line" or "It is very natural that a system will not work before the start button is operated". These opinions mean that the subject would establish an opinion or mental model that any steps that play an important role in reaching the task goal or whose operation is held by common sense to be needed in such a system must be processed ahead of other steps, if the process neglects the provided guidance.
0
50
O p e r a t i o n a l T i m e (sec) 100 150
i
i
200
i
Type 1
i
176.8
Type 2 Type 3
250
i
1140.6 r][ .........
........ :--:":= ~
~
~
~
~
~
~
.............................
Type 4
Figure 2 Mean operational Time
~
s ......
248.4 210.2
304
5
OperationalNumber 10 15 20
25
3o
==1"-
--1
Type 1 Type 2
i iii ili {ii
Type 3
ili ]11.8 25.8
Type 4
Figure 3 Mean Operational Number of Entering Buttons 4 Discussion and Conclusion
At first, this study considered that clear guidance could prevent the occurrence of any procedural errors. However, the results of the experiments showed that several subjects ignored clear guidance and processed some steps out of sequence. It was noticed that steps that were incorrect processed at an earlier stage of the sequence were through to be of importance to reaching the task goal or to be required by common sense. This means that, probably except for absolute novice users with very passive attitudes, even novice users do not or can not follow only guidance but tend to establish some kind of users' mental model of the operational steps that should be processed at first, even if the model is vague, and they tend to follow this model. In this experimental system, subjects probably establish more importance to the dialing or start button than the connecting line, and, therefore, they operated these buttons earlier. Therefore clear guidance is not enough to make users progress through the designed procedure. The user model has strong influence on procedural usability, however vague the model is. When we develop systems that provide procedural usability for novice users, we must also evaluate the order of steps in the sequence from the viewpoint of the users' expectations relating the meaning of each step to the task goal and their common sense about the operation of such kinds of systems. Guidances can only confirm procedural usability. Reference
1) D.A.Norman, The Psychology of Everyday Things, Basic Books Inc.,(1988) 2) A.Komatsubara and M.Kobayashi, Evaluating Operational Sequences with "Plan","Do" and "See" Analysis, The Japanese Journal of Ergonomics (in printing)
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
305
Quantitative Evaluation of Media Quality by Method of Competitive Priority
Hiroshi TAMURA, Jun WU Kyoto Institute of Technology, Department of Information Technology Matsugasaki, Sakyoku, Kyoto, 606 JAPAN fax: +8175-701-7211, email: [email protected] abstract: Method of competitive priority was proposed to evaluate quality of speech and images. Speech coded by ADPCM showed some priority to the one by PCM undercompetitive situations. The image quality was evaluated by presenting two speech words associated with talking head image of one of the two speeches. The effect of image presentation was compared with normal, ISDN(64 kbps) and the stop motion image. The effect was low with ISDN image. Some people were found insensible to ISDN or the stop motion image. keywords: speech image interaction, image quality, speech quality, talking head video, image compression
1,
Introduction
In the traditional technology of message presentation, single message was essentially chosen for the clear transmission. However in the real world events, more than one things happen at the same time. And people have the liberty of choosing one message out of others. Advanced information systems have to support such freedom of choice. The evaluation of message quality in the multi-media age should be done under influences of other coexisting presentations. Thus the method of competitive priority is developed for that purpose. This paper first applies the method to evaluate two speeches of different quality. Then the method is applied to compare the image quality of normal and ISDN image. Here ISDN image is the image transferred by 64kbps rate.
2. Competitive Priority Suppose speeches encoded at a low and a high sampling frequency would be equally recognized and understood when presented one at once. But when two speeches, one sampled at a low (say 5kHz) and the other sampled at a high frequency ( 2 2 k H z ) , are presented simultaneously, the latter is recognized at a higher rate. A message of a higher quality is recognized and understood better than that of the lower quality. The quality of the media presentation can be evaluated by the comparison of the correct ratio of the words recognized by each presentation. Two methods of competitive speech presentation are derived. The one is to present two speeches, one from the fight and the other from the left ear simultaneously. Such presentation is
306 called dichotic by Kimura(19613). In the dichotic presentation, the listener can use the lateral cues in differentiating the two. The second method of presenting two speeches is to mix the two electronically and present them from the both ears. The method is called mixed presentation. The mixed presentation is harder to recognize than the dichotic for the listeners, so the mixed presentation is mainly applied to the experienced listeners who show high correct ratio, and the dichotic for the novice listeners. Normal listening in the free space is supposed to be in between dichotic and mixed listening, since listener can make use of lateral cue to some extent, but certain part of sounds are mixed together in the free space. Interesting results concerning the dichotic presentation have been reported by (Chin et a1.19921,19942) , and (Tamura et al. 19938), this paper is mainly concerned with the mixed presentation.
3. Experimental Systems Fig. 1 is showing the outline of the experimental systems. First the speech and talking head image of talker is recorded in the optical video recorder. The recorded speech sounds are then converted into digital data, normalized to have the equal maximal amplitude and stored in the memory. The speech data can be reproduced in an arbitrary pair of speech words. In case the speeches encoded at different sampling frequencies are competitively evaluated, each presentation pair consist of one word sampled at one frequency and the other at the other frequency. In case the image quality is to be evaluated, one word in a pair is associated with talking head image. Then the correct ratio of the word with image are enhanced, while the word without image are depressed. The difference of the correct ratio can be the index for the effect of image presentation.
I mage+speech
Optical
Imaae RS-232C Control
speech board
Speech "headphone
subjects Fig. 1 Experimental System
307 32 words are chosen to prepare the video tape for presentation. All words had the similar form of ANBN type. Here A or B is a various combination of a consonant and a vowel. The number of words of this type are rich in Japanese and there are large freedom of choice in selecting the words for presentation.
4. Speech Quality Evaluation Table 1 shows the correct ratio of subjects when mixed speech words are presented. The result shows correct ratio is significantly different when the sampling frequencies are 5 and 22 kHz. The difference is not significant when the frequencies are 11 and 22 kHz. Table 1 Correct Ratio of Speech Recognition Sampled at High and Low Frequency using Mixed Presentation Correct Ratio of Recognition
Sampling frequency. Exp.A 22 vs. 5.3kHz Exp.B 22 vs. 11 kHz
No. of subjects High freq.
Low freq. Difference
5 experienced
52%
34%
18%
4 novice
29
9
20
6 experienced
45
42
3
6 novice
23
19
4
The speech data for the experiment are sampled at 22kHz which is high enough with regard to the above results. In the recent ISDN communication devices, two types of data coding:i.e. ADPCM and PCM are installed for the speech signal. Normally the quality of coding methods are equivalent. A test has been done to competitively compare the quality of the speech transferred by the two coding methods. The test was done within a series of presentations which included competitive sounds encoded by ADPCM and PCM. Average correct ratio of recognition by two coding methods are equal. But it is interesting to see the ratio in the different combinations. When two speeches are coded both by ADPCM ( column shown by A/A) or by PCM ( column shown by P/P), the correct ratio is higher for the PCM.
Table 2 Correct Ratio of Recognition for the speech coded by ADPCM and PCM. By mixed presentation
Average Correct Ratio of Recognition
Correct Ratioof Recognition in different combination
AlP No. of subjects ADPCM Exp.C 6 experienced
43 %
PCM
A/A
P/P
43 %
38 %
45 %
A
P
49%
42%
308 This may suggest that details of two speech words are coded properly by PCM coding so that they could be differentiated better. When ADPCM and PCM speeches are presented together ( column A/P ), correct ratio of ADPCM coding is 49, and that of PCM coding is 42. The result is impressive. ADPCM has the competitive priority to PCM coding. Recognition priority is different in the homogeneous and in heterogeneous coding environment. When unique coding method is used consistently in an environment, PCM coding has merits. But in situations where different coding methods are jointly used, ADPCM has the merits.
1
3
2
stop motion image 1
I
4
3
5
6
IS
2
G
A
N
M
E
N
ISDN image
1
2
3
4
5
6
Fig.2 Image Transferred by ISDN 64kbps and by Stop Motion
5. Quality Evaluation of ISDN image The method to evaluate the effect of talking head image in speech recognition has been proposed previously by Chin 2, Ostberg4 and Tamura8, and the method was applied to the normal video image. The purpose of this chapter is to evaluate quantitatively the effect of the ISDN image presentation. There are various uses of ISDN image exchanges, and various pessimistic and optimistic expectations for its contributions to communication. The purpose of this study is limited to the effect of the image presentation to the speech understanding. There are two main problems in the use of ISDN image presentation. The one is the time delay and the other is the image distortion. The time for encoding and decoding a talking head image
309 takes more than 500ms. This processing delay fatally removed the effect of image presentation. So the ISDN image and the speech were synchronized in our presentation. The six talking head image in the bottom of Fig.2 are the ISDN image actually received through the network. The wave form in the middle is speech (GANMEN, face mask in English). The maker 1 to 6 below the wave form is corresponding to the moment each image are captured. In the ISDN network less moving images are transmitted more frequently. So in the talking head images, the frame transmission is more frequent when the talker is silent, and less frequent when the talker is talking. The transmission rate is contradictory to the actual need. Also image at the critical moment in time are not assured to be sent. Table 3 shows the effect of image presentations. For the normal images, the effect of image presentation to the mixed speeches reach up to 13% after previous report by (Ito et al.9). But for the ISDN image it is as low as 6%, in spite of synchronization of the speech and image. Table 3 Effect of Image Presentation for Various Images (mixed presentation) No. of
word with image
subjects normal image
Total labial non labial
49 %
27
ISDN image
Total labial non labial
52
11
Total labial non labial
64
STOP motion image
word without effect of image image presentation 36 %
13% 22 49
36 62 46
6
51 41
56 49
5 8
10
54 71 57
14 13
55 53
16 4
The table indicates the effect of image presentation is low for the ISDN image, compared to the normal image, which was 13% after (Ito et al. 19939). The effect is low equally for the labial and the non labial consonants. The reason the effect of image presentation is small might be partly due to small number of frames which is transferred, and partly to inappropriate selection of frames. Since encoding of image starts soon after the previous processing is over, the frame to be encoded next is automatically determined. But there might be certain frames that are informative for the speech recognition. So it was assumed that the low effect of image presentation is due to ill selection of the frames that is transferred, but not to the low number of frames nor to image distortion in image processing. To prove this assumption the stop motion presentation was introduced. Since the number of the frames transmitted in the ISDN image are 6 in average, the number of frames in the stop motion presentation was also chosen to be 6. Experimenter selected visually 6 frames from talking head video whose length is 1.2 to 1.5 sec., so to say 36 to 45 frames. The same fame are presented repeatedly presented from one to the next chosen frame.The frames chosen for the stop motion presentation is shown in the top row of the Fig.2.
310 The result of experiments using the stop motion image is also shown in Table 3. The result shows that the effect of image presentation using stop motion image is apparently enhanced near to the level of normal video image
6. Personal d i f f e r e n c e It is seldom to find a subject who has normal sight but shows an extremely low effect of image presentation to normal image. But for the ISDN and STOP motion images, the effect of image presentation is not observed to some subjects. About 2 to 3 peoples in 10 are not sensitive to ISDN or stop motion images. It seems that people use different cues to enhance the recognition of the speech with the video. In the ISDN image, some sort of cue is lacking, and the people who use the cue seem to fail to enhance the recognition ratio. The fact has a serious meaning, and has to be traced further carefully. 7. C o n c l u s i o n The use of images and speeches in digital communication are becoming common in the network and multimedia systems. The proper evaluation methods of use of images are not provided yet. The method of this paper provides the evaluation with high precision and reliability than those provided earlier6,7. The results of this paper have to be considered in developing digital networks and multimedia. It is by no means smart to improve image communication simply by increasing the transmission speed. The data rate has to be flexibly adaptive to the actual need. For the reproduction of speech and image in multimedia, informative frames should be properly chosen in order to enhance the speech understanding.
Reference 1. Y.Chin, H.Tamura(1992), Effects of ImagePresentation to the Cognition of Plural Speech, Human Interface, News & Report, Vol.7, pp.167-172 2. Y.Chin, H.Tamura,Y.Shibuya, A. Ito (1994), Analysis of presenting Talking HeadVideo by the Method of Recognizing Plural Speech Words, Trans. IEICE J-77-D-II, No.8, pp. 1484-1491 3. D.Kimura (1961), CerebralDominanceand the Perception of Verbal Stimuli, Canad. J. Physiology, Vol.15, pp.156-165 4. J.Macdonald, H.Mcgurk (1978), Visual Influence on Speech Perception Process, Perception & Psychophysics, Vol.24, pp.253-257 5. O. Ostberg, Y. Horie, M. Warren (1992), Contribution of Visual Images to Speech Intelligibility, SPIE, Vol.1666, 526-533 6. K.Sekiyama,Y.Tohkura (1991), Cultural Differencein Dependenceon Visual Cues in Speech Perception, Technical Report of Acoustic Society of Japan, H91-56 7. K.Sekiyama, Y.Tohkura (1991), McGurk effect in non-English Listeners: Few Visual Effect in Japanese Subjects hearing Japanese Syllables of High Auditory Intelligibility, J. Acoustic Society of America, Vol.90, pp. 1797-1805 8. H.Tamura, Y.Chin,Y.Shibuya (1993), Effect of image presentation to the cognition of plural speech, Advances in Human Factors/Ergonomics,VoI.19B, pp. 62-67 9. A. Ito, Y.Chin, H.Tamura, Y.Shibuya (1993) Effect of Image Presentation to the Cognition of Mixed soeech, Human Interface,Vol.9, pp.43-48.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
311
E v a l u a t i o n of control strategies in a c o m p l e x space-vehicle control task: Effects of training type Ravindra S. Goonetilleke a, Colin G. Drury b, and Joseph Sharit b Department of Industrial Engineering, Hong Kong University of Science & Technology, Clear Water Bay, Kowloon, Hong Kong a
bDepartment of Industrial Engineering, State University of New York at Buffalo, Buffalo, New York 14260, U.S.A.
ABSTRACT The fundamental differences in operator control strategies in a complex task were evaluated in two training scenarios: in-the-loop training and out-ofthe-loop training. Verbal protocols and performance measures revealed four types of complex control mechanisms dependent upon these two training approaches. The four types were display based control, open loop input control, closed loop input control, and an input-display control mix. Performance differences favored in-the-loop training, and led to the development of an open loop input control strategy. The overall results indicate that performance improvements may be achieved with operator training on the system dynamics and optimization aspects rather than operator training directed only at the optimization aspects. A "sitting by Nellie" approach such as watching an expert or watching an algorithm perform a task may be disastrous if the system dynamics are poorly understood. This study also suggests how operator strategies can be effectively used to design user-friendly aids which improve operator performance in complex control tasks.
1. INTRODUCTION Supervisory control has shifted the activities of the operator from an inthe-loop controller towards an out-of-the-loop supervisor or monitor. This shift in activities poses two important questions with regard to operator training. If the operator is to be primarily a monitor rather than a controller, would not training be more effective if also performed out-of-the-loop? Secondly, if this apparently logical approach of training is adopted, what possible problems may exist?
312 The issue of manual controllers versus monitoring controllers has been researched in the past (Hopkin, 1992; Kessel and Wickens, 1982; Brigham and Laios, 1975), but there are a number of issues related to training that are still largely unresolved. Knaeuper and Rouse (1984), using a rule-based system as an on-line "coach" providing advice to operators, found no significant differences in the primary performance measures even though significant differences did exist in the secondary measures. The differences in the secondary measures could have resulted from differing strategies. This paper addresses both performance issues and strategy differences in an "ill-structured" control task (i.e., the operator is unaware of the system dynamics) as a function of two training methods: in-the-loop (or hands-on) and out-of-the-loop (or observation).
2. M E T H O D 2.1. O v e r v i e w The experiment involved a low-medium fidelity simulation of the relocation of a geosynchronous satellite from a known location in a given orbit into a desired geosynchronous orbit while optimizing a three component objective function, J (Goonetilleke, 1990). Optimality (i.e., minimization of J) was defined as being the minimization of fuel (Jfuel, related to thrust usage), minimization of the deviation from desired trajectory at final time (Jpos), and minimization of the deviation from desired velocity at final time (Jvel').
Objective function J = Jfuel + Jpos + Jvel A second order differential equation governed the two-dimensional system dynamics. Since the system was simulating an ill-structured process, the system dynamics knowledge was not given to the operators. The necessary orbital adjustments were carried out by firing directional rockets positioned around the body of the satellite in the x- and y- directions. The objective of the experiment was to find a 51-second 2-dimensional thruster burn which optimally guided the satellite into closer alignment with the geosynchronous orbit. The simulation was coded in "C" and run on a SUN 4/280 workstation. The input to the system was via the three-buttoned mouse and the keyboard. 2.2 S u b j e c t s A 2-factor factorial experiment with a 2-level between-subjects factor (type of training; in-the-loop and out-of-the-loop) and a 7-level within-subjects factor (repetitions or trials) was used with 5 subjects under each betweensubjects factor. Each subject received US$4/hour for participation. 2.3 P r o c e d u r e The five operators in each group underwent a pretraining session (reading a manual) and four training trials prior to actual experimentation.
313 In-the-loop training involved hands-on performance on the task. During out-ofthe-loop training, each subject watched an optimal control algorithm perform the task. Subjects received approximately 5-7 hours of training on the satellite maneuvering task. After the training trials, all subjects performed the actual task across seven trials. Verbal protocols were taken at the end of each trial and analysed to provide information on intended strategy.
3. RESULTS The performance measures used were J-ratio (= J/Joptimal = Objective function / Optimal algorithm objective function), logl0(Jfuel), l°gl0(Jpos), logl0(Jvel), % x-distance-to-go (calculated based on initial and final x-positions), % y-distance-to-go (calculated based on initial and final y-positions) and % orbit traveled (measured using radii). Results from the ANOVA revealed that type of training was significantly (p < 0.05) different for the three measures logl0(Jvel) , % orbit traveled and % y-distance-to-go. A significant (training x repetition) interaction was seen with % orbit traveled and % y-distance-to-go measures. Finally, repetition or trials were significantly (p < 0.05) different for the measures logl0(Jpos) , logl0(Jve l) and % x-distance-to-go.
4. D I S C U S S I O N
A significant difference in the performance measure, logl0(Jvel) , indicates that the type of training influenced the subject's strategy in achieving the desired final velocity. Thrust and velocity were related by a first-order differential equation, and the thrust-position relationship was second order. In-the-loop trained subjects controlled the system based on the velocity display and hence had a lower velocity error (Jvel), whereas out-of-the-loop trained subjects had difficulty understanding any thrust, velocity and position relationships. The performance measures and the verbal protocols showed a clear difference in the cognitive architecture. In-the-loop trained subjects identified the dynamics first and then attempted the optimization. The out-ofthe-loop trained subjects attempted to perform both these tasks simultaneously using input patterns they observed during training. Subjects trained in-the-loop started with "Display Based Control" and then with experience adopted "Open Loop Input Control" (see Figure 1). However one subject who was unable to form a "good" strategy, also demonstrated "Display Based Control" initially, but this strategy gradually changed to "Closed Loop" rather than "Open Loop" input control. Humans are good at pattern recognition (Fleishman and Quaintance, 1984), and when identifiable patterns seem to exist, operators tend to use these patterns even when they lack a good mental model of the system. It is hypothesized that the input based control bias shown by the out-of-the-loop trained group was primarily due to watching the algorithm perform the task during the training trials with different parameters. The out-of-the-loop trained
314 subjects started with an "Input-Display Control Mix" strategy to accommodate all the variables involved and to overcome the deficiencies of a good mental model (see Figure 2). One subject did, however, display "Open Loop Input Control" from the start of the experiment. It is well known from the process control literature (e.g., Edwards and Lees, 1974) that control strategy changes from feedback control to open-loop control with practice. These results show that the transition from feedback to open-loop occurs after complex and critical changes that eventually determine task performance. Furthermore, the verbal protocols and knowledge of the task strategies can be effectively used to design user-friendly performance aids to improve this transition based on the trial or repetition effect seen in the three phases of control: start-up, search for optimum and "shut-down". Out-of-the-loop training may become attractive due to resource constraints that may be imposed by hands-on or in-the-loop training. This research, which indicated significant performance improvements when combining operator training on the system dynamics with knowledge of the optimization aspects rather than providing knowledge of the optimization aspects alone, cautions against such an approach. Watching an expert or watching an algorithm perform a complex task as a basis for training may be disastrous if the system dynamics are poorly understood. Trainee controllers need to first understand the system dynamics well enough to form an appropriate mental model. Only then should these trainees be given the optimization aspects of the task. Thus merely watching the actions of an expert may be a singularly ineffective method of training for complex tasks.
REFERENCES 1. V. D. Hopkin, Human factors issues in air traffic control. Human Factors Society Bulletin, 35(6) (1992) 1-4. 2. F. Brigham and L. Laios, Operator performance in the control of a laboratory process plant. Ergonomics, 18 (1975) 53-66. 3. C.J. Kessel and C. D. Wickens, The transfer of failure detection skills between monitoring and controlling dynamic systems. Human Factors, 24(1) (1982) 49-60. 4. A. Knaeuper and W. B. Rouse, A rule-based model of human problem solving behavior in dynamic environments. IEEE Transactions on Systems, Man and Cybernetics, SMC-15(6) (1985) 708-719. 5. R. S. Goonetilleke, Humans in complex control tasks: Mental model development and use. Unpublished doctoral dissertation, State University of New York at Buffalo, Buffalo, NY, USA, 1990. 6. E.A. Fleishman and M. K. Quaintance, Taxonomies of human performance. Academic Press, Florida, 1984. 7. E. Edwards and F. Lees, The human operator in process control. Taylor and Francis, London, 1974
315
II
I I
II
DISPLAY-BASED CONTROL / I
Short term
¢
I
I
nit
I II
i
FEEDBACK CONTROL
-- uman]L
to match display display set I point ,vf set point input OLSysteml display_~@÷ error .~
r
I
LOSED LOOP INPUT CONTROL
previous ~ input , ,differenc " [Huma~ -,~,~)-----~lSystem..] A
Long term (experience)
I
OPEN LOOP INPUT CONTROL
~ input [Human I
Subject 04
m] display _ error J Syste--' - +~---~_--0) Idisplay set point
~ubject 0 I, 0 2 , 0 ~ Figure 1. Effects of in-the-loop training
display
316 i
i
CONTROL I
Short term
CLOSEDLOOP INPUT CONTROL
OPEN LOOP INPUT CONTROL
previous / input [Huianiifferen::~-~System ~display
Long term (experience)
OPEN-CLOSEDINPUT CONTROLMIX
previous |input iHui andifference¼+ I -~)~System
input display error display iHuman~_~System I +
wheneverdisplaysare stable Figure 2. Effectsofout-of-the-looptraining
=~+(_--0) display set point
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
317
Development of the Analysis Support System for Incidents and Troubles; "ASSIST" Yuriko YOSHIZAWA and Keiko MUTOH E-mail: [email protected] H u m a n Factors D e p a r t m e n t , Nuclear Power R&D Center, Tokyo Electric Power Company, 4-1, Egasaki-cho, Tsurumi-ku, Yokohama-city, K a n a g a w a , J a p a n
1. I n t r o d u c t i o n For safe operations and works, it is i m p o r t a n t t h a t h u m a n error not be made.
So we s t r e n g t h e n e q u i p m e n t a n d educate workers for safety.
It is
necessary to provide a methodical education s y s t e m and to take trouble data. So we need to m a k e an i n t e g r a t e d information system on h u m a n error and e q u i p m e n t troubles for supporting h u m a n error reduction activities in the field.
2. P r o p o s e d m e t h o d s for u s e i n a d a t a a n a l y s i s s y s t e m s It is i m p o r t a n t to analyze the troubles and incidents to avoid repetition of the similar troubles.
Of course workers in the fields also try to collect d a t a on
accidents a n d incidents and use t h e m to avoid repetition of the similar troubles. They are not m a k i n g use of the data, because they do not have a good tool. Therefore a t the first step to m a k e the integrated information system, we had a plan to develop a d a t a analysis tool.
The tool m u s t be simple and
convenience to analyze an information for workers.
It also m u s t assist to
u n d e r s t a n d the s t r u c t u r e of the event. FTA ( F a u l t Tree Analysis) which h a s already been developed is a method t h a t clearly d r a w s flow of events t h a t m a y lead to accidents. It is r e p r e s e n t the flow of events using block diagram, so it is easy to u n d e r s t a n d the events. We can m a k e the events' tree by using a combination of AND and OR junction.
This
318 method is, however, designed for use in searching for potential accident and it is not good at d r a w events of the accident in time sequence. There
is
also
the
J - H P E S 1 (namely
the
Japanese
version
Human
Performance E n h a n c e m e n t System) which is being developed CRIEPI.
This
s y s t e m is a modification of the H P E S (namely H u m a n Performance E v a l u a t i o n System) t h a t was developed by I n s t i t u t e of Nuclear Power Operations in USA This is the s y s t e m for
analyzing and evaluating the causes of the accidents.
This s y s t e m has the guideline of the method for investigating and e v a l u a t i n g accidents and t a k i n g the c o u n t e r m e a s u r e s to them. accidents following the guideline.
So worker can analyze the
On the other hand, these s y s t e m needs
detailed information, so it takes m u c h time for using this system. It needs m u c h time to m a s t e r this system, so workers in the field can not easily to use it. The variation tree method is, however the method for describing a tree of an accident in time sequence. So we can u n d e r s t a n d easily outline of the accident. It is also easy to describe a tree of an accident w i t h o u t difficult rules. It is said in an Accident Prevention M a n u a l 2 t h a t was m a d e by the I n t e r n a t i o n a l Civil Aviation Organization t h a t an accident can not generally be traced to a single cause, b u t r a t h e r together to lead to the occurrence.
t h a t a variety of different factors i n t e r a c t To prevent an accident, it is therefore
necessary to cut the chain of the events leading up to it. In a fact after a n a l y z i n g an accident, we can discover t h a t there are ways of preventing the accident at a n u m b e r of different stages of the process leading up to the occurrence of the accident. It is for these reasons t h a t we reached the conclusion t h a t it would be easier to clearly identify the chains of events leading to accidents if we could use the variation tree method. 3. T h e v a r i a t i o n t r e e m e t h o d The variation tree method is a method first proposed by Jacques Leplat and J e n s R a s m u s s e n '~ in 1987. Using this method, we investigate the factors why an accident has happened, and then these events are d r a w n like a tree in time sequence. The following is a description of the procedure to the d r a w n of a variation tree:
319 1. Organize the
nodes t h a t
are performance
of irregular
operations,
incorrect j u d g m e n t s in the relation between the causes and the results in the form of a tree in time sequence. 2. This method is based on the fault tree analyses.
However, it is the
method for the analysis of an accident t h a t has already occurred, so we connect the nodes of the tree using AND junction alone. 3. Search the tree t h u s created to find w h a t led up to the accident and w h a t would h a v e prevented the accident. Then consider to remove nodes and to cut the links t h a t led up to the accident. 4. After that, take the c o u n t e r m e a s u r e to the picked out nodes and links t h a t led up to the accident using the step ladder model by R a s m u s s e n 4. Following this procedure, it represents the chain of events leading up to the occurrence of an accident and a clear picture of the flow of events leading up to the accident.
4. T h e A S S I S T s y s t e m The idea of the variation tree s y s t e m is clear, simple, and easy to u n d e r s t a n d and it would be useful as a tool for workers in the field. So we proposed the method for actual use. However it takes time to d r a w up a tree using pen and paper. It is also difficult to store the results of such analysis. We instead provided the tool t h a t would m a k e it possible to describe such trees using personal computers.
We
adopted the "m-SHEL model" instead of the "Rasmussen's step ladder model" for the guideline of the countermeasure, because it is difficult to u n d e r s t a n d the concept of the information processing model for workers. We n a m e d this s y s t e m the Analysis Support S y s t e m for Incidents and Troubles; "ASSIST" The screens of this s y s t e m are composed of four modes:
4.1 F i l e d a t a m o d e This mode is the screen for the entry and display of basic file d a t a such as the title, date, time, place and a s u m m a r y of the accident. Workers enter d a t a on personal information related to the accident such as their n u m b e r of years of experience, their physical condition on the day, and any other background data.
320 4.2 O u t l i n e m o d e We provided a less structured form similar to a word processor, because we thought that it was difficult to describe an accident using a tree form from the start (figure 1). Equipments' names and workers' names are arranged horizontally on the top of the display. It is written each event of every behavior vertically. We describe and grasp the accident story at this mode.
=l ~l.l([)
~lil[I
.....
IIiiI,,~.~) c!ll~(~l
I-I"
",,,,;,.I-'~A_~i
i'J
~A~:'J@_~ : ~lUi °
A~a~.~:~',)i'*:.:~b
'j
=,l
"
....
°]
I-V-
....... ~.... :..............
t-]-I
.................................
=7
::::!!:
i~:3~........
-hi ........... r [!!!~:!:!!_~ i~i~~
r~~:~~:: ~~
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
iA - S
'~:47 ....
.......F
3 i:l"it~-~,f
'i ............
• 'i.!~!@.~"J{:S,s
{
-
~!.
I:::
•
•
.
•
.!i: :
I~ l!IA I7.t~,~, ~.50 ........
~57,-
i
........
t~! k :: :: :;:ii,:~i:,:i:
;
..:
';: ................ '7 IIi~t~ ~.~C,~"= ss:
!~i:::i~:!,~:++i::~:~iiiiiii:ii~ 7::7;i~77~i:77:7!i::ii~,::i;:~!!~::7~;:i;i;i+ ~!i~:;:~;:,::;:iii;~, <;I +
Figure 1. Sample of outline mode
~,,..~-
Figure 2. Sample of tree mode
4.3 T r e e m o d e In this mode, the data entered outline mode is automatically converted into the tree form
possible (Figure 2). W e can connect the relations between
horizontal boxes (namely nodes) in which individual actions are represented by clicking the two related nodes using mouse. The system also has the command to add and delete nodes easily. We draw a complete variation tree of the accident at the tree mode. After that, we pick out nodes that are irregular action or controversial behaviors and put the symbol on them. 4.4 C o u n t e r m e a s u r e
mode
In this mode, we take countermeasures against the nodes that are picked out at the tree mode (Figure 3). We added the guide line to propose the countermeasures based "SHEL
321 mode" which was proposed by F. H. H a w k i n s 5.
SHEL model can explain
relationship between h u m a n and factors a r o u n d the h u m a n . Software, H a r d w a r e , Environment, and Liveware. SHEL model, which was new concept.
It consists of
We added m a n a g e m e n t to
Because we t h i n k m a n a g e m e n t is very
i m p o r t a n t from the viewpoint of the h u m a n factors.
It called m-SHEL model
(Figure 4). So we can take the countermeasures against the problem points from the view point of the m-SHEL model.
i!i!i!i!i~:i:~;iBi!ii!iiii!~' ~ i'ii!ili:,;i'~,~ii'i,iiiiii!iiilli!i~~ i~ i:~ i!i~~ i................................ ~ i~ i~ ;~ :~ :i.!~ :i...~;~:~;~;~;!!~i~. ..........~...:~!!~i~:~ii!:~:iiii~iii~iii~:iiiii~!iii~ii~:::~iii~i!~:!:~:~..~.~:i::i~!i~i~:::::::::::::::::::::::~iii~::~iiii~iiii~i!~i~i~i~i~i~iii~ ]
!
'~!ilili!i i i!i !i i i i !il
Figure 13. Sample of c o u n t e r m e a s u r e mode
Figure 4. m-SHEL model
5. M a i n f e a t u r e s of t h e A S S I S T s y s t e m We developed the ASSIST t h a t is a s u p p o r t system to draw variation trees on windows s y s t e m of a personal computer. This s y s t e m supports to u n d e r s t a n d an accident flow easily because the events of the accident are d r a w n node of tree form in time sequence. Since this system was made on a personal computer, it is easy to d r a w a tree, to add or delete the nodes easily and speedy. We provided the icon of some command, so it was easier to draw trees. We introduced the m-SHEL model guideline to take the countermeasures. After continuing to analyze a lot of accidents and store the analyzed results, this system proposed the past trees' c o u n t e r m e a s u r e s and so on. Workers used this system to try, with the result t h a t it is easy to u n d e r s t a n d the content of an accident because the accident tree was d r a w n visually and in time sequence. This system is on windows system on a personal computer, so they also said t h a t it was easy to d r a w a tree. Using the m-SHEL model, they could take countermeasures from m a n y points of view.
322
6. F u t u r e plan This system is currently testing on field of nuclear power plants and branch offices and we will reflect the idea expressed by users in these tests. The variation tree method is originally developed as a tool for analyzing accidents after they occurred.
We believe t h a t it can also support as an
important tool for use in analyzing incident information. It is important to prevent an accident before it occurs, so it is necessary to analyze the incident information.
Incident
information,
however,
is incomplete
information
in
comparison with accident information. In future we will apply the ASSIST to the system for analyzing the incident information. We have a plan to make the large information system t h a t is integrated analyzed accidents' and incidents' data and research results.
Data is analyzed
statistically and fed back to the field. "ASSIST" is a part of the database system t h a t constitutes the "integrated service system on h u m a n factors" t h a t we construct in future. From now we will store research and study results and variety information into the database. Moreover, by linking such data comprehensively, we plan to construct the integrated h u m a n factors information system to support activities at power stations from a viewpoint of h u m a n factors. Through such efforts, we expect to establish comfortable work environment.
REFERENCES 1.
K. Takano, CRIEPI, Development of J a p a n e s e H P E S - an elaboration, HPES Coordinator Conference, 1989.
2.
International Civil Aviation Organization, Accident Prevention Manual, 1984.
3.
J. Leplat and J. Rasmussen, Analysis of H u m a n Errors in Industrial Incidents and Accidents for Improvement of Work Safety, New Technology and H u m a n Error 15, (1987) 157.
4.
J. Rasmussen, Outlines of hybrid model of the process operator, Monitoring Behavior and Supervisory Control, 1976.
5.
F . H . Hawkins, H u m a n Factors in Flight, 1987.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) 1995 Elsevier Science B.V.
323
Discount Video Analysis for Usability Engineering Mark H. Chignell a , Tetsuro Motoyamab and Venicio Meloa aDepartment of Industrial Engineering, University of Toronto 4 Taddle Creek Road, Toronto, Ontario, Canada M5S 1A4 bRicoh Corporation, 3001 Orchard Parkway, San Jose, CA 95134-2088 Usability analysis has to be cost-effective. In this paper we discuss the role of video analysis in usability engineering, outlining an approach for making it easier and less time consuming. After reviewing the various approaches to usability analysis, the role of video in iterative design and evaluation is discussed. A method is proposed for simplifying video analysis by automating clip segmentation based on sound analysis and other techniques. This is followed by a brief description of a direct manipulation video editor that we are developing for usability engineers. I. I N T R O D U C T I O N There are many possible approaches to usability engineering. Designer-user communication looks at the compatibility between the designer's model of the device as embodied in the device and the user's model of the device. This approach has been championed by Don Norman (e.g., Norman, 1988) and leads naturally to an analysis of critical incidents and conceptual bugs. Scientific analysis of usability looks at the underlying factors or independent variables that affect usability. This approach uses formal experiments as its main methodology. This method has fallen out of favor in industrial usability labs in recent years because it is perceived to be too costly and insufficiently product- and design-oriented. Heuristic evaluation (e.g., Nielsen and Molich, 1990) is a low cost means of usability analysis where interfaces are judged (rated) in terms of heuristics such as "the interface should be consistent." Rapid prototyping (iterative) design typically uses minimalist experiments (usability tests) in working through a cycle of design, test, and evaluation repeatedly, detecting the most important usability problems in each cycle, fixing them in the next design, and continuing until a satisfactory level of usability is reached. Usability testing and rapid prototyping are the predominant methods of usability analysis used currently. In this paper we consider how video analysis can be adapted to the specific needs of usability engineering. 2. I T E R A T I V E DESIGN AND EVALUATION Figure 1 shows how the evaluation phase in rapid prototyping design can be modeled as a cycle of usability testing, analysis, communication, and interpretation. The thoroughness of an evaluation will depend upon the results of prior evaluation iterations, and adaptation to the specifics of the product and the situation. Typically, one would expect more iterations in the evaluation loop when evaluating the first prototype of a new product. Fewer iterations during evaluation should be required as usability engineers as the design matures. Interpretation is a particularly important step in evaluation. The goal of the evaluation step in iterative design is not just to assess how usable the current prototype is, but also to figure out what design changes might make it more usable. Video analysis is an important tool in this process of interpreting what usability problems occur, and why.
324
Prototyping
[Redesign]
I~ [Evaluation] Interpretation
Figure 1. The Evaluation Loop within the Iterative Design Process. 3. VIDEO ANALYSIS Video analysis is an emerging technique for analysing not only data in usability engineering, but also in a wide range of research and applied settings, including group and collaborative work in particular. In this section we review some of the video analysis tools that have been developed for research, before considering how video analysis can be used in usability engineering.
3.1 Video Analysis Tools for Research Many video analysis tools have been developed for research. For instance, the GroupAnalyzer is a video analysis system that represents group dynamics over time (Losada & Markovitch, 1990) using the Bales SYMLOG dimensions (Bales, 1983) to describe mood, cooperativeness, and degree of task orientation. Another set of video data collection and analysis tools was developed for research in group collaboration (Olson and Olson, 1991). Some tools emphasize annotation of video, including VideoNoter (Roschelle, Pea, and Trigg, 1990), and VANNA (Video Annotation and Analysis) system (Harrison, 1991; Harrison and Baecker, 1992). VANNA allowed users to define index markers as buttons which could be pressed to create index labels and link them to corresponding locations (times) on video. Other video analysis systems include Timelines (Harrison, Owen, and Baecker, 1994), Eva (McKay, 1989), and MacShapa (Sanderson et al., in press). The video analysis tools mentioned above were developed specifically for studying computer supported collaborative work, and group interaction (e.g., in meetings) in a research setting. They have considerable functionality, including indexing, annotation, and visualization of overall patterns in the data. However, they tend to be too labour intensive to be effective in usability labs. 3.2 Video Analysis in Usability Engineering Video analysis is a tool for assisting usability engineers in evaluating prototypes, and for enhancing communication between usability engineers and the people responsible for designing and selling the end product. In many usability labs, video is routinely collected in all usability tests. In some labs, more than one camera may be used (e.g., one camera on the screen, another camera on the user or the keyboard).
325 Current practice in usability testing emphasizes efficiency through the use of small subject samples. This approach is supported by research findings on the relationship between the number of evaluators or subjects used and the proportion of usability problems found (e.g., Virzi, 1992; Landauer and Nielsen, 1993). The available studies suggest that roughly three to five subjects should be sufficient to find a majority of the usability problems for a given prototype and set of tasks. Although sample sizes are small, multiple hours of video may be collected per subject, particularly if more than one camera is used. Thus ten or more hours of video are often collected in a single usability testing cycle, and fifty or more hours of video might be collected during development of a typical product or application. Video captures a rich set of data about user interactions with products. Often two or three 30 second video segments can provide the designer with clear examples of the experiences and problems that end users encounter in a particular design. Visual examples (real or videotaped) of actual usability problems often have more impact than charts or statistical summaries. Numerous anecdotes have been told about developers stating through the one-way mirror at actual users and reacting with shock and disbelief to the usability problems that they observe the users trying to deal with. While this reality check should be part of the education of every designer or developer, it is too difficult and expensive to bring every developer or designer down to the usability laboratory to spend long hours watching how their products are used. Thus video presentation or browsing tools are needed that allow designers to review usability problems quickly, without having to leave their offices or laboratories. In many situations a short "highlights" video would motivate developers to find better solutions and designs. Thus a simple tool which could support the development of short video highlights of usability tests would be extremely useful.
4. Simplified Video Analysis While large amounts of video are collected in usability tests, relatively little of this video is actually analyzed. The extra time required to analyze videos is often judged to be better spent in carrying out usability tests to identify more usability problems. Thus methods of simplified video analysis need to be developed that will stimulate more use of video in usability engineering (e.g., using the video editing process shown in Figure 2). In this approach, a variety of tools can be used to assist in segmentation and clip selection. In addition, clips may be indexed and stored. Digital video is a useful part of this approach since it simplifies the task of storing, indexing, and accessing video. In some situations there may be a need to accompany selected video clips with a textual report summarizing the main usability problems. In such cases it is natural to create a multimedia document that combines the video clips with accompanying annotations and text. The selection and organization of clips can be accompanied with annotation. For instance, clips from a usability test could be annotated with a description of the usability problems that they exemplify. Further modifications can lead to a comprehensive multimedia document on the usability test, including creation of an overview or table of contents, and creation of links between different clips to provide hypermedia navigation capability. We have identified the following activities in creating multimedia presentations of usability test highlights using digital video: 1. Pre-processing (This step includes selection of video to be analyzed and digitization of this video); 2. Segmentation (into potentially interesting video clips); 3. Indexing; 4. Exploration; 5. Organization; 6. Annotation; 7. Production. Our method of creating highlights videos is based on a model of video as a document. In creating the video document, one first carries out research, by collecting and exploring relevant information (video clips) and then organizes that information into a presentation. Annotation can then be used to interpret and highlight the clips in the resulting presentation. The final product might then be a videotape of highlights and usability problems that is passed to designers/ developers, or a multimedia document that can be played on a computer.
326
Usability Video
ISegmentation l - - n n n n n n n n m n n n n n n n m u
I
I.J..I.I..L.IJ
Clip Selection
Organization
l n l n n l n n m m n n n n n n l u m
Video Clip I..]..l,.J..l..l.J Library
-
Figure 2. A Simplified View of Discount Video Analysis. 4.1 Segmentation and Indexing In recent years there has been considerable interest in automatic video indexing. In one approach, large changes in color are used to detect scene changes (Ueda et al, 1993). Existing image analysis approaches to video indexing are unlikely to work well in the important application areas of usability testing, lecturing, and group meetings. In these applications there is relatively little camera movement, no scene changes, and the visual information is often relatively sparse. In contrast, these application areas tend to be rich in sound content. Even when sound is not the dominant channel of communication (e.g., in l~ctures when visual aids are used extensively, or in meetings where the focus is on whiteboards or flip charts) it still is highly correlated with the organization of content. In meetings, lectures, or usability tests, we can expect that important segments of video are likely to have sound associated with them. While speech recognition is notoriously difficult, considerable progress has been made on the analysis of the physical characteristics of speech and other sounds. This is leading to an important opportunity for exploiting the sound track of videos. One recent example of how relatively straightforward sound analysis techniques can be used in segmenting and indexing speech is a speech skimming application (Arons, 1994). The speech skimming method proposed a number of methods for compressing and skimming speech (Arons, 1994, p. 99). Pause-based skimming is based on the idea that exceptionally long pauses tended to indicate a new topic, some content words, or a new speaker. Thus pause-based skimming assumes that speech following a long pause is more likely to be interesting or important. Pitch-based skimming also focuses on interesting or important segments of speech. Support for pitch-based skimming has come from research findings that there tends to be an increase in pitch range when a speaker introduces a new topic (e.g., Hirschberg and Grosz, 1992). Pause-based skimming can be adapted to segmentation of speech (and associated video). Similarly, pitch-based skimming can be used to detect emphasis, indexing segments according to their importance. For digital video, the analysis task may be conceptualized as segmenting and indexing the video, essentially creating a multimedia database of video clips. Thus pausebased segmentation would be a useful first step in this paradigm. Similarly, identification of important segments through pitch analysis would allow the observer or coder to skip from one important point to another, saving a great deal of time and effort over current methods of serially scanning the video.
327
4.2 A Digital Video Editing Tool We are developing a digital video editing tool to put some of these ideas into practice. This tool is being developed specifically for usability engineers. The tool simplifies the video editing task by allowing video clips to be edited and organized using a drag and drop interface. Figures 3 and 4 show two screen shots of the current version of the tool. Figure 3 shows the application immediately after a clip has been dropped into the movie window. This clip was collected using a Connectix QuickCam on a PowerMac 6100/60. The highlighted selection (black bar below the image) of the QuickTime controller indicates the portion of the clip that was selected.
....................................................................................
............................................~.!~.~!~.".~2!L~.~.~.~.V........................................... .~/. ::i:~'~
''""
"
[
l~lovi* Clipping l 5
vie C "ppin916 Maeh.moov
- ~ . ? ~ - ~ . . . . . . . :~:-~:
:: ! Lots.moor
4
Lien.Moor
~i[i ~i i i i i ~i i~i ~i~i i~i i~i i i i ~i i i ~i~i~i~i~i i i i i i ~i i~i~i~i i i i i ~i i i ~i ~i i i~i i~i~i~i i~i i~i i~i i i ~i ~i~i i~i~i i i i i i i~i~i~i~i~i i i~i~i i i ~i~Figure 3. Dragging a Clip from a Video Window
Figure 4. Adding in Stored Clips to a Movie.
Figure 4 shows the application after three previously stored clips have been added to the movie. The three movies (shown as highlighted files in the Macintosh finder) are simply dragged and dropped over the movie window. The digital video editing tool greatly simplifies the process of editing video. Future versions will allow hierarchical organization of clips, also using a simple direct manipulation style of interaction. However, discussions with usability engineers have indicated to us that the main problem is not with editing the video, but with preprocessing and segmenting it. Thus the editing tool is currently being modified to include sound-based segmentation methods.
5. SUMMARY This paper described the rationale for discount video analysis in usability engineering. Segments of usability video that are accompanied by speech will tend to be of more interest. This can be exploited in discount video analysis by speech-based segmentation of the video, resulting in a significant data reduction that can be carried out automatically. The remainder of the video analysis then consists of organizing and indexing the video. The final product is a multimedia document that describes the results of the usability testing in a comprehensive way. Sound-based segmentation can be supplemented by direct manipulation editing techniques using a drag-and-drop interaction style that greatly simplify the video editing task. Another approach (e.g., used in the VANNA and Timelines systems) is to index the video manually as it
328 is being collected during the usability test. This indexing can then be used as a basis for segmentation, or for later querying and retrieval of clips. Our goal using these and other techniques is to make video analysis a more effective component of usability engineering. This should lead to better awareness of usability problems, and ultimately, to better products. ACKNOWLEDGMENTS
This research was funded by Ricoh Corporation. We would like to thank John Itoh for his support. Ed Patrick and Nick Kooij assisted with programming. We would also like to thank Beverly Harrison, Sarah Zuberec, Alan Lahosky, and Gene Golvchinsky, for sharing their views on the role of video analysis and usability engineering with us. REFERENCES
Arons, B.M. Interactively skimming recorded speech. Ph.D Dissertation. Program in Media Arts and Sciences, Massachussetts Institute of Technology, Cambridge, MA, 1994. Bales, R.F. (1983). SYMLOG: A practical approach to the study of groups. In Small Groups and Social Interaction. H.H. Blumberg, A.P. Hare, V. Kent and M. Davies (Eds). John Wiley and Sons. 499-523. Harrison, B.L. (1991). The Annotation and Analysis of Video Documents. Unpublished Masters Thesis. Department of Industrial Engineering, University of Toronto, Toronto, Ontario, Canada. Harrison, B.L. and Baecker, R.M. (1992). Designing video annotation and analysis systems. Proccedings of Graphics Interface '92 Conference, 157-166. Vancouver, B.C. Harrison, B.L., Owen, R., and Baecker, R.M. (1994). Timelines: An interactive system for the collection and visualization of temporal data. Proceedings of Graphics Interface '94. Canadian Information Processing Society. Hirschberg, J. and Grosz, B. Intonational features of local and global discourse. In Proc. of the Speech and Natural Language Workshop (Harriman, NY, Feb 23-26). San Mateo, CA: Morgan Kaufmann, 1992. Losada, M. and Markovitch, S., (1990). GroupAnalyzer: A System for Dynamic Analysis of Group Interaction, Proceedings of the 23rd Annual Hawaii International Conference on System Sciences, IEEE Computer Society, 101-110. McKay, W. E. (1989). EVA: An experimental video annotator for symbolic analysis of video data. SIGCHI Bulletin, 21 (2), 68-71. Nielsen, J. and Landauer, T.K. (1993). A mathematical model of the finding of usability problems. Proceedings of INTERCHI '93, 206-213. N.Y.: ACM Press. Nielsen, J. and Molich, R. (1990) Heuristic evaluation of user interfaces. CHI '90 Proceedings, 249-256. Norman, D.A. (1988). The Psychology of Everyday Things. N.Y.: Basic Books. Olson, G.M. and Olson, J.S. (1991). User-centered design of collaboration technology. Journal of Organizational Computing, 1, 61-83. Roschelle, J., Pea, R. and Trigg, R. (1990). VideoNoter: A Tool for Exploratory Video Analysis. IRL Technical Report No IRL90-0021, March 1990. Sanderson, P.M., Scott, J.J.P., Mainzer, J., Johnston, T., and James, J.M. (in press). MacSHAPA and the enterprise of exploratory sequential data analysis. International Journal of Human-Computer Studies. Ueda, H., Miyatake, T., Sumino, S., and Nagasaka, A. (1993). Automatic structure visualization for video editing. Proceedings of INTERCHI '93, 137-141. NY: ACM Press. Virzi, R.A. (1992). Refining the test phase of usability evaluation: How many subjects are enough? Human Factors, 34(4), 547-468.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
329
User Interface Evaluation" is it Ever Usable? Christelle Farenc a, Philippe Palanquea, bl , Jean Vanderdonckt c aL.I.S., Universit~ Toulouse I, Place Anatole France, F-31042 Toulouse Cedex, France, Tel: +33- 61.63.35.{62,88}, E-mail:{farenc, palanque}@cict.fr bCentre d'Etudes de la Navigation A~rienne, 7 av. E. Belin, F-31055 Toulouse Cedex, France. c Institut d'Informatique, FUNDP Namur, rue Grandgagnage, 21, B-5000 Namur, Belgium, Tel: +32 - (0)81/72.49.75, E-mail:[email protected] 1. INTRODUCTION
According to h u m a n behavior studies, several disciplines (e.g., cognitive psychology, software ergonomics, visual design) have brought substantive results to improve the user friendliness of user interface (UI). One possible output of these disciplines come as recommendations that could be translated into ergonomic rules (or guidelines). Guideline knowledge is often contained in five sources : recommendation papers [1], design standards (e.g., ISO 9241 [2]), style guides which are specific to a particular environment (e.g., IBM Common User Access [3]), design guides (e.g., Scapin's guide [4], Vanderdonckt's guide [5]) and algorithms for ergonomic design (e.g., automatic selection of interaction objects [6]). Studies carried out with designers show that these guidelines are difficult to apply at design time: • average search time for a guideline in a design guide lasts 15 minutes [ 1]; • about 58% of designers succeed to find guidelines relevant to their problem [ 1]; • designers do not respect about 11% of guidelines [7]; • designers experienced interpretation problems for 30% of guidelines [7]. Guidelines are intended to various people involved in a team which is responsible for developing an interactive application, but it is not clear how they best fit into their needs: • a task analyst hopes that guidelines are organized according to a taxonomy of tasks characterized by attributes, and not according to interaction styles; • a project leader only pays attention to high-level guidelines (e.g., selecting an appropriate metaphor) because they are the most likely to influence positively or negatively the success story of UI implementation; • a h u m a n factors expert assumes that guidelines are sorted by cognitive principles t h a t initiated the guidelines because they form the basis of work; • a designer expects to see guidelines organized in such a way that s/he could instantly isolate specific guidelines and solve conflicts between them by ranking them into ordered sequences; • a programmer thinks that guidelines should be organized with respect to the interaction objects (IO) or widgets of the interactive task because his/her responsibility only covers this part; • a UI evaluator produces a UI evaluation report including scores, measures qualifying the degree to which guidelines have been respected, errors,...
330 2. UI DESIGN AND EVALUATIONWITH ERGONOMIC RULES
Such a UI evaluation is performed not only in order to evaluate the UI quality, but also to improve the user friendliness of the product. Unfortunately, evaluation reports are not taken into account as expected because: 1. reports are not standardized : there is no unique evaluation method that could report all usability aspects and that could be teached as a whole. Instead, UI evaluators are forced to use multiple approaches which are not necessarily complementary and which are often grounded on human expertise and knowledge. The report structure depends therefore on both approaches and evaluators; 2. reports are not easily reused by designers and programmers • we just saw how the lack of standardization hinders a thorough reading and understanding of report by designers and programmers. Moreover, the report contents are not always presented to easily highlight usability problems in terms that programmers could manage. "Immediate feedback is not satisfied in 10% of cases", "Average task time is 3 min. 45 sec.", "Mean of error rate is one for fifteen transactions", "heart rate is 70 pulsations/min when achieving data retrieval, 90 when validating data" are examples of non-directly reusable conclusions. 3. reports are not prescriptive : if reports lists problems, the degree to which they are described may vary from one problem to another, from one evaluator to another, from one evaluation method to another. Moreover, it is the responsibility of designer to solve problems by finding relevant guidelines, successfull interaction techniques, to correct misleading dialogues,... And it is the responsibility of programmers to interpret these guidelines into practical IO programming. 2.1 C o n c e p t s We suggest a knowledge organization that accomodates these differences among actors and that supports integrated use by both designers, programmers and evaluators. This organization consists of the concepts depicted in fig. 1. Utility concerns the adequacy that should exist between functions provided by a UI and the user's task. Usability concerns the adequacy between the way an interactive task is carried out by a particular user and the cognitive profile of this user. When UI evaluation is performed, these two aspects should be handled, though usability is often emphasized. Utility and usability goals can be expressed as factors. A factor is a statement of a general evaluation dimension which is expressed symptomatically by intrinsic qualities and drawbacks and which could be measured and/or estimated. Several authors have their own factors : • Shneiderman [8] enumerates five evaluation criteria: time to learn specific functions, speed of task performance, rate of errors, subjective user satisfaction, and human retention of commands over time; • ISO 9241/10 standard [2] highlights several principles such as suitability for the task, self-descriptiveness, controllability, conformity,... • Marshall, Nelson & Gardiner [9] call them "key categories, sensitive dimensions"; • Nielsen [10] argues that ten heuristics are sufficient such as: simple and natural dialogue, speak the user's language, minimize the user's memory load,...
331 A factor could be related to utility and/or usability (hence, the "0-2" connectivity in fig. 1), 1-r ~ 0 - 2 L . I ~ 1-n but, at least, one of them. Factors match user expectations for a usable UI and are related to Factors ,) one or many ergonomic criteria (hence, the "l-n" connectivity). 1-n Ergonomic Criteria are widely recognized and (~rgonomicCritem% accepted criteria that lead to an elaborated, efficient, sophisticated, user friendly UI [11]. They 1 1-n include for instance compatibility, consistency, ~ 1 work load, adaptability, dialog control, represen(ErgonomicRules % tativeness, guidance and error m a n a g e m e n t as in [5]. Ergonomic criteria are primarily consid1 1-1 ered as design criteria because they can serve at C RealValues~ design time, but they can also serve as evaluation criteria at evaluation time. For instance, it I r O-n is interesting to see the impact of ergonomic criteria (e.g., consistency) on factors (e.g., rate of ~nteractionObjects'~ errors). If all ergonomic criteria might be considered for each UI, all criteria are not equally im1 0-n portant: they should be weighted so that UINorm priviledged factors could be identified by examining important criteria respected by a UI. The Figure 1. S t r u c t u r i n g usability satisfaction percentage of particular ergonomic and utility into concepts. criteria are a function of n u m b e r and rate of ergonomic rules. An ergonomic rule [5] consists of a design and/or evaluation principle to be observed to get and/or guarantee an ergonomic UI. This definition highlights that following guidelines is a necessary but insufficient condition to reach the goal. One can imagine that a same ergonomic rule can step in several ergonomic criteria (this is consequently a multi-criteria approach). But, if each recommendation from the literature is translated into a clean ergonomic rule with premisses and only one obvious conclusion, then each ergonomic rule realizes only one ergonomic criteria (hence, the "1-1" connectivity). However, it seems obvious that applying a particular ergonomic rule may influence the applicability and compliance of other. The multiplicity of sources listed in the introduction inevitably mix empirical and conventional considerations. Premisses of some ergonomic rules may also depend on user stereotype parameters. We suppose that a particular ergonomic rule can be verified by comparing the real value of attributes coming from the IO and the ergonomic value praised by the ergonomic rule. For example, the "colour" attribute of a push button should be set to "grey" when this push button is inactive. The real values are actually the observed values of all the IO composing a UI. The Look & Feel (i.e. the presentation and the behaviour) of common IO are partially regulated by norms (e.g., standards or style guides).
332 UI seen by EVALUATOR
Usability/Utility Dimensions
/ ~ ~ F a c t ° r
Ergonomie criteria
Ergo o
/F~°r
Ergonomie criteria
Ergonornic rule
Favor Ergonomie criteria
Ergonomicr u l e /
Real Value Real Value Value
\/ Interaction
Value
I
"'"
. .. 7 ~
.
Ergonomicrule . . . ~,z FactualUI description User Interface (UI)
0 '~'-~"
UI I0 values
Object Interaction Object Interaction Object
UI norm (standard, style guide)
UI seen by PROGRAMMER
UI seen by USER ~
i,
.
.4
Figure 2. Multiple UI views.
2.2. I n t e g r a t i n g UI views Using these concepts aids integrating the multiple UI views possessed by three important actors (i.e. evaluator, programmer, and user) as depicted in fig. 2. E v a l u a t o r s typically examine the UI of concern for finding usability problems. The results of this examination usually consist of strengths and weaknesses which are related to factors, which are themselves decomposed into ergonomic criteria which are in turn more related to UI properties and features. Partial or full respect of these criteria heavily depends on the ergonomic rules that are satisfied or not. Rather than expressing usability problems in terms of factors only, an evaluator could explicitly link them to ergonomic rules through ergonomic criteria. Prog r a m m e r s see a UI in terms of IO: whatever the development methodology is, UI design subsumes selecting and creating IO. If the evaluation report is expressed in terms of ergonomic rules working on effective IO, programmers are more likely to solve usability, utility problems. F i n a l users perceive a UI by manipulating IO inducing easiness or difficulty in order to reach the task's goal. Highlighting the real values of current IO is a practical entry point for evaluating a UI with users. Linking usability and utility to IO as introduced in fig. 1 allows us to use this approach for both UI design (dotted arrows in fig. 2) and repair (plain arrows in fig. 2). In the first case, one specifies which factors should be priviledged according to the UI context and user's demands. This enables us to compute appropriate values to be further satisfied for each criteria and to compare them with the real values. In the second case, we start from an existing UI ; we evaluate the compliance of ergonomic rules related to ergonomic criteria by computing the value for each criteria to be respected. This strategy embodies external parameters coming from context analysis (e.g., user stereotypes, socio-organizational context, work place). They are fundamental since we observe their real impact on ergonomic criteria, whether positively or negatively, allowing recognition of relevant factors.
333 3. IMPLEMENTATION ISSUES
In order to prove the feasibility of the above outlined integration of views, we examplify this by detailing two implementation issues. Tools for helping designer of interactive software have been implemented according to the above discussion. These tools try to embody as explictly as possible the organization defined in the previous section. These tools are grounded on a uniform knowledge structure which provides to the designer information compatible with knowledge domain ERGOVAL is a tool which provides help for evaluating the UI usability. The system includes three sub-systems [12]: • a rule base, containing ergonomic rules independent of the task and structured according to an interface object typology; ° a structural decomposition of the objects of the graphic interface; ° an interface object typology. Graphic objects concerned by the same recommendations were grouped into object classes. These classes formed a typology of objects with multiple levels of abstraction. Rules concerning an object class were implemented once, at the highest possible level, and are inherited by the graphic objects, which are the instances of object class. According to this knowledge base, various organisations of ergonomic recommendations are allowed: design criteria-based (e.g., guidance, consistency), dialogue mode-based (e.g., command, action), graphic objet-based [3]. Moreover, the system reaches the following goals: usability by computer experts, time cost reduction when searching violated recommendations, completeness, cohesion, and maintainability of the knowledge base. TRIDENT automatically generates a usable UI from context analysis (i.e. task analysis, definition of user stereotypes, and description of work place) with explicit use of ergonomic rules. In this system, a task-based approach is followed in order to conduct the presentation design. Ergonomic rules are not only imbedded and visible within the system, but are also documented and accessible on-line on request depending on the context. These rules could also be used to perform a UI evaluation by linguistic ergonomic criteria, though manual [13]. We do not claim that the knowledge organization outlined in fig. 2 still works for every evaluation method, but experience lead us to be convinced that it is practical for both surface evaluation (as in ERGOVAL) and by linguistic ergonomic criteria (as in TRIDENT). Not all UI aspects can be evaluated in terms of a comparison between a real value and an ergonomic value. For instance, it seems hard to evaluate if a particular metaphor is better than another since, even it is concretized as IO, the true reference values for appropriate IO materializing the metaphor remain unknown. Coupling automated UI generation and evaluation is considered as a key future work : it is sound to evaluate a particular UI that have been automatically generated with the same ergonomic criteria and guidelines that have been used to produce it. Moreover, ergonomic rules do not necessarily lead to an optimal UI because of conflicts and contradictions. CONCLUSION This paper has shown that the availability of guideline knowledge is not enough in order to provide helpful and efficient information to the agents involved
334 in the design process of an interactive application. By showing the different points of view of those agents towards the UI we have explained why it is so difficult to provide relevant and easy to use information. Two different projects addressing this question are presented. The first one aims at evaluating and proposing solutions for the improvement of the interactive applications previously developped, while the second one aims at providing a set of tools for the automated design of interactive applications taking into account guidelines for user interface design. This paper has tried to give an answer at the question included in the title: without appropriate tools supporting both design and evaluation, it is no more allowed to follow the numerous guidelines currently available. Those tools are a necessary condition but of course not a sufficient one as the craft knowledge of user interface designers and ergonomists is far to be not worthy. User Interface evaluation: is it ever usable? REFERENCES
1.
2. 3. 4. 5. 6. 7.
8. 9.
10. 11. 12.
13.
S.L. Smith, Standards Versus Guidelines for Designing User Interface Software, in Handbook of Human-Computer Interaction, M. Helander (ed.), North-Holland, Amsterdam, 1988, pp. 877-889. ISO/WD 9241, Ergonomic requirements for Office Work with Visual Displays Units, International Standard Organization, 1992. IBM Common User Access Guidelines, Object-Oriented Interface Design, Document SC34-4399, IBM Corp. Publisher, 1993. D.L. Scapin, Guide ergonomique de conception des interfaces homme-ordinateur, Research report INRIA N°77, INRIA, Le Chesnay, October 1986. J. Vanderdonckt, Guide ergonomique des interfaces homme-machine, Presses Universitaires de Namur, Namur, 1994, ISBN 2-87037-189-6. F. Bodart and J. Vanderdonckt, On the Problem of Selecting Interaction Objects, Proc. of HCI'94 (Glasgow, 23-26 August 1994), Cambridge, 1994, pp. 163-178. F. de Souza and N. Bevan, The Use of Guidelines in Menu Interface Design : Evaluation of a Draft Standard, Proc. of INTERACT'90 (Cambridge, 27-31 August 1990), Elsevier Science Publishers, Amsterdam, 1990, pp. 435-440. B. Shneiderman, Designing the User Interface: Strategies for Effective HumanComputer Interaction, Addison-Wesley, Reading, 1987 C. Marshall, C. Nelson and M.M. Gardiner, Design Guidelines, in Applying Cognitive Psychology to User-Interface Design, M.M. Gardiner and B. Christie (Eds.), John Wiley, Chichester, pp. 221-278. J. Nielsen, Enhancing the Explanatory Power of Usability Heuristics, in Proc. of CHI'94 (Boston, 24-28 April 1994), ACM Press, New York, 1994., pp.152-158. D.L. Scapin, Guidelines for User Interface Design: Knowledge Collection and Organization, Technical report ITHACA.INRIA.89.D12.03, INRIA, 30 Dec. 1989. M.-F. Barthet, V. Liberati, M. Ponamale, ERGOVAL - A Software User Interface Evaluation Tool, in Proc. of the 12th Triennial Congress of the International Ergonomics Association (IEA) (Toronto, 15-19 August 1994) pp.429-431 F. Bodart and J. Vanderdonckt, Using Ergonomic Rules for User Interface Evaluation by Linguistic Ergonomic Criteria, in this proceedings.
1ph. Palanque would like to thank the HCI Group of the Department of Computer Science of the University of York (U.K.)where he was a visiting researcher during the developmentof this paper.
III.12 HCI Evaluation Methodologies
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
337
S o f t w a r e T o o l s for E v a l u a t i n g the Usability of U s e r Interfaces Sandrine Balbo Human-Computer Communications Centre School of Information Technology Bond University 4229 QLD - AUSTRALIA Phone: Fax: Email: WWW:
+61 75 953331 +61 75 953320 sbalbo @surf.Bond.edu.au www.dstc.Bond.edu.au:8008/staff/sandrine.html
Abstract In this article we propose a review of some different techniques and methods to evaluate the usability of user interfaces (UI). So far, the evaluation process has been mostly based on "craft techniques" [Long 89], but as we will demonstrate, formalisation is possible, and a few software packages in this domain will be presented. The models and techniques we will consider are: • general guidelines such as those proposed by [Smith 86, Nielsen 90, Bastien 9311, • the Cognitive Walkthrough [Lewis 90], • metrics [Whiteside 85, Bevan 94], • usability labs [Hammontree 92, Weiler 93], • predictive models [Young 90, Barnard 87], • automatic monitoring systems [Siochi 91, Balbo 94] and • critics [Lrwgren 90, Kolsky 89]. We will present these methods and techniques around a taxonomy developed by Jo~,lle Coutaz in [Coutaz 94], taxonomy designed to help in the choice of a method to evaluate UI. As well, we will highlight the role played by software tools for evaluating the usability of UI.
1. Introduction The development of interactive systems is not an exact science. It relies on an iterative four-step process that includes requirement analysis, design, implementation, and evaluation. A number of tools are now available to support the first three phases. In particular, for the requirements and design phases, task models such as GOMS [Card 83] and MAD [PierretGoldreich 89], provide useful ways to structure the task space. Implementation is supported by an even wider variety of tools ranging from toolboxes such as Motif [Motif 90], to application skeletons such as MacApp [Schmucker 86], and UI management systems such as SIROCCO [Normand 92] or Interface Builder [Webster 89]. On the other hand, the software engineering community has placed little emphasis on the evaluation of UI. For the purpose of this article, we will organise the evaluation methods around the taxonomy developed by JoElle Coutaz in [Coutaz 94]. This taxonomy is designed to help the person in charge of the evaluation (not always an expert) to formulate questions that will help in the choice of the right tool to conduct the evaluation. It proposes a set of five preoccupations, where each preoccupation is decomposed into a set of axes. These five preoccupations concern knowledge resources, hardware resources, environment resources, human resources, and outcomes. Let us now examine each preoccupation in turn.
338
2. K n o w l e d g e
resources
Ergonomic expertise
The first preoccupation with which ~._... Descriptions this taxonomy is concerned is the I-- mgn knowledge needed to conduct the l ~ u s e r model evaluation. We consider two types of knowledge required in conducting an | j ~ - external specifications evaluation: the description needed as an F non~ scenarios input for the evaluation and the level of none expertise required from the evaluator in order to perform the evaluation. • The Smith & Mosier guidelines result from empirical knowledge (inferred by observation and experimentation) [Smith 86]. They can be used by the human factor specialist just by "looking at the interface". These guidelines propose a set of 944 recommendations concerning very specific points to be examined. These recommendations are quite difficult to manage for a non-ergonomist expert. Then, in the case of the Smith & Mosier guidelines, if we consider the knowledge resources preoccupation, no description is necessary but the ergonomic expertise of the evaluator must be high. By providing tools and software packages to the evaluator, we try to reduce the amount of ergonomic expertise needed to conduct the evaluation.
Flow
3. E n v i r o n m e n t
resources
The next preoccupation concerns the environment resources which define the context of the evaluation. This dimension is expressed using a set of five axes: the location where the evaluation takes place, the structure of the dialogue provided by the interface, the phase of the SE development life cycle in which the evaluation may be conducted, the type of interface being evaluated and the financial or temporal constraint on the evaluation.
/z
Software development phase
Dialog ~'~equirementanalysis structure L
•
User Interface type
~t)esign I¢ l j~" mono/ flexible..~ t.lmplementation/ multi user •~ / ~ multimedia/ mzxt:e,~ ~-Tests / muiiimc~xtal rigid"~ I ~ directmanipulation Location ~1 ./" .... -" I I I [ v temporal in the in the money calendar constraints zoo wild limit limit • [Nielsen 90] or [Bastien 93] propose a set of ergonomic criteria derived from existing guidelines, similar to the Smith & Mosier guidelines, but the structure is different. They supply a "way of improving the completeness and explicitness of the diagnosis, of standardising the format of the evaluation, and of better documenting the evaluation" [Bastien 93, p.1]. This is provided by giving some recommendations, of a higher level than the one proposed by Smith & Mosier. For example, [Bastien 93] will speak about consistency, when Smith & Mosier will introduce it at different stages, like, for example, in the guideline "2.7.1/4 Consistent format for display labels", or the guideline "4.4/9 Consistent format for prompts". We note that in the Smith & Mosier guidelines, the word extract "consisten" (cf. consistent~consistency~etc.) appears in 142 guidelines. • The Cognitive Walkthrough supplies another type of support for evaluation in the form of a questionnaire [Lewis 90]. This questionnaire focuses primarily "on ease of use for first-time users, using a theoretical analysis based on the CE+ theory of Poison and Lewis" [Rieman 91, p. 427]. Regarding methods such as those proposed by [Bastien 93], [Nielsen 90] or the Cognitive Walkthrough, the only restriction concerning the environment resources refers to the financial and temporal constraint. A study conducted by Pollier in [Pollier 91] shows, that to be effective, an evaluation needs to be conducted by at least 3 evaluators. An evaluator alone will find an average of only 42% of the problems in the interface. Therefore, if the budget is tight, such a method will not be suitable.
339 Origin of subjects
4. H u m a n
resources
~ external The human resources concern the !1 persons involved in the evaluation / Subject types process. This may refer to the evaluators £-N,%. l-in ou o . 4 as well as to the subjects. For the evaluators, the taxonomy takes into a, "/ . levels ofexpertise account their number and their level of °~0~ ~ l .,~representative/effective expertise, which is directly linked to the o
340 Instruments for
5. H a r d w a r e
resources
data capture
The hardware resources cover the computer what is evaluated physical components for the evaluation. -- usability lab tversion product They include the object of the evaluation (i.e. what is evaluated), and the paper& ~ prototype instruments used to capture the data. " pencil . ~ . mockup This second concept emphasises again • n o ~ a - task model the importance of the observation of the end user manipulating the UI while ~ a" user model conducting the evaluation. As a further step towards the automation of UI usability and learnability, we will review tools that do observe the end user in action. These actions are recorded during task execution with the real system on video tape, or on the computer itself through event capture. We will first introduce usability labs and the tools developed to analyse their results. • Usability labs are generally used to record sessions of the user manipulating a UI without being intrusive [Weiler 93]. These sessions can be evaluated on the fly, and later on as well. The tools to help the evaluator later on cover a vast range going from a simple replay of the sessions, to filters or multimedia data analysers that allow the evaluator to go through the sessions in an effective way [Hammontree 92]. The capture of events on the computer opens the perspective of an automatic analysis of the information recorded. Going in that direction, we present two software packages: Maximal Repeating Pattern (MRP) and EMA, an automatic analysis mechanism. • MRP detects repeating patterns of behaviour by analysing the log of the end user's actions [Siochi 91]. These actions are recorded during task execution with the real system. Detected patterns can then be used by the human factor specialist as driving cues for the evaluation. • The automatic analysis provided by EMA is not only based on the acquisition of the end user's actions, but uses as well a data-flow oriented dialogue model of the interface being tested to detect patterns of behaviour modelled by a set of heuristic rules. EMA automatically constructs an analysis that will help the evaluator to perform the evaluation [Balbo 94]. In the case of MRP or EMA, a prototype or a version of the product is needed, and an automatic analysis is provided. As for usability labs, they may apply to task models or user models as well, but do not provide automatic analysis. A
t
"
Rendering support Information types
6. O u t c o m e s computer
~~objectived
The outcomes of an evaluation technique or method are characterised by video I ....... subjective the rendering support, as well as the type paper& ~ quantitative/ -• qualitative of information provided. This second edictive/ qualitative axis allows objective information, explicative/ quantitative results, or corrective outputs corrective to be distinguished. We present here two systems, Knowledge-based Review of user Interfaces (KRI) and SYNOP that provide corrective output automatically. • KRI provides the designer with an automatic critic [L/3wgren 90]. The tool uses a collection of ergonomic rules to evaluate a formal description of the interface. The evaluation produces a set of comments and suggestions. The improvements, however, are concerned with lexical issues only such as the minimum distance between two icons or menu items ordering, and only form filling interfaces are concerned by this technique. • The expert system SYNOP is similar to KRI in regards to the services it provides [Kolsky 89]. However, its organisation is different in addition to the interfaces it concerns. SYNOP is applicable only for control system applications.
341 With KRI or SYNOP the level of automation in the evaluation is very high. For example SYNOP is able to modify the interface by itself. However these systems exclude the user from the evaluation and the constructive information their observations can provide.
7. Conclusion This discussion has resulted in a comparative analysis of some of the software packages available to help in evaluating the usability of UI. We note four differences in the use of software tools during the evaluation of the usability of UI: help to conduct the analysis (,Cognitive Walkthrough, usability labs, MUSIC), computerised capture (usability labs, MPP, EMA), automatic analysis (MRP, EMA), and automatic critic (KRI, SYNOP). The outcome preoccupation is concerned by the rendering support and the type of information provided. We should consider to include as well in that preoccupation a new axis referring to the use of software during the evaluation.
Acknowledgments Many thanks to Nadine Ozkan, Michael Rees and Dominique Scapin for their review of this paper.
REFERENCES [Balbo 94] S. Balbo, "Evaluation ergonomique des interfaces utilisateur • un pas vers l'automatisation", PhD thesis, University of Grenoble I, France, September 1994 [Barnard 87] P.J. Barnard, "Cognitive Resources and the Learning of Human-Computer Dialogs", in Interfacing Thought, edited by J.M. Carroll, The MIT Press, pp. 112-158, 1987 [Bastien 93] J-M. C. Bastien & D. L. Scapin & "Ergonomic Criteria for the Evaluation of Human-Computer Interfaces", Rapport Technique INRIA no 156, Juin 1993 [Bevan 94] N. Bevan & M. Macleod, "Usability measurement in context", Behaviour and Information Technology, Vol. 13, Nos 1 and 2, pp 132-145, 1994 [Card 83] S.K. Card, T.P. Moran & A. Newell, "The psychology of Human Computer Interaction", Lawrence Erlbaum Associates, 1983 [Coutaz 94] J. Coutaz & S. Balbo, "Evaluation des interfaces utilisateur: taxonomie et recommandations", IHM'94, Human-Computer Interaction Conference, Lilies (France), December 1994 [Hammontree 92] M.L. Hammontree, J.J. Hendrickson & B.W. Hensley, "Integrated data capture and analysis tools for research and testing on graphical user interfaces", in Proceedings of the CHI'92 Conference, pp. 431-432, ACM Press, Monterey, 3-7 May 1992 [Kolsky 89] C. Kolski, "Contribution a l'ergonomie de conception des interfaces graphiques homme-machine dans les procEd6s industriels : application au syst6me expert SYNOP", Th~se prEsent6e ?t l'Universit6 de Valenciennes et du Hainaut-Cambr6sis, Janvier 1989 [Lewis 90] C. Lewis, P. Poison, C. Wharton & J. Rieman, "Testing a Walkthrough Methodology for Theory-Based Design of Walk-Up-and-Use Interfaces", Proceedings of the CHI'91 Conference. April 1990, Seattle. ACM New York, pp. 235-242
342 [Long 89] J. Long & J. Dowell, "Conceptions of the Discipline of HCI: Craft, Applied Science, and Engineering", in Proceedings of the Fifth Conference of the BCS HCI SIG, A. Sutcliffe and L. Macaulay (Eds), Cambridge University Press, 1-989 [Ltiwgren 90] J. Ltiwgren & T. Nordqvist, "A Knowledge-Based Tool for User Interface Evaluation and its Integration in a UIMS", Human-Computer Interaction-INTERACT'90, pp. 395-400 [Motif 90] 1990
OSF/Motif, Programmer's Guide, Open Software Foundation, Prentice Hall,
[Nielsen 90] J. Nielsen & R. Molich, "Heuristic evaluation of user interfaces", Proceedings of the CHI'90. 1990, Seatle. ACM New York, pp. 349-256 [Normand 92] V. Normand, "Le modEle Sirocco : de la specification conceptuelle des interfaces utilisateur h leur rEalisation",PhD thesis, University of Grenoble I, France, 1992 [Pierret-Goldreich 89] C. Pierret-Goldreich, I. Delouis & D.L. Scapin, "Un Outil d'Acquisition et de ReprEsentation des Taches OrientE-Objet", rapport de recherche INRIA no 1063, Programme 8 : Communication Homme-Machine, aofit 1989 [Pollier 91] A. Pollier, "Evaluation d'une interface par des ergonomes : diagnostics et strategies", Rapport de recherche INRIA no 1391, FEvrier 1991 [Rieman 91] J. Rieman, S. Davis, D.C. Hair, M. Esemplare, P. Polson & C. Lewis, "An Automated Cognitive Walkthrough", CHI'91 Conference Proceedings, New-Orleans (USA), ACM Press, Addison-Weslay, April 27-May 2, 1991 [Schmucker 86] K. Schmucker, "MacApp: an application framework", Byte 11(8), 1986, pp. 189-193 [Siochi91] A.C. Siochi & D. Hix, "A study of computer-supported user interface evaluation using maximal repeating pattern analysis", CHI'91 Conference Proceedings, New-Orleans (USA), ACM Press, Addison-Weslay, April 27-May 2, 1991 [Smith 86] S.L. Smith & J.N. Mosier, "Guidelines for designing user interface software", Report MTR-10090 ESD-TR-86-278, The MITRE Corporation, Bedford, MA, August 1986 [Webster 89] B.F. Webster, "The NeXT Book", Addison-Wesley, Reading, Mass., 1989 [Weiler93] P. Weiler, "Software for the Usability Lab: a Sampling of Current Tools", INTERCHI'93 Conference Proceedings, Amsterdam (Holland), ACM Press, AddisonWeslay, April 24-29, 1993 [Whiteside 85] J. Whiteside, S. Jones, P.S. Levy & D.Wixon,"User Performance with Command, Menu, and Iconic Interfaces", in Proceedings of the CHI'85 Conference, ACM Press, Addison-Weslay, April 1985 [Young 90] R.M. Young & J. Whittington, "A knowledge Analysis of interactivity", Proceedings of INTERACT'90, edited by D. Diaper, G. Cockton & B. Shackel, Elsevier Scientific Publishers B.V. pp. 207-212, 27-31st August 1990, Cambridge (United Kingdom)
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
343
How usable are usability principles, criteria and standards ? J. M. C. Bastien and D. L. Scapin INRIA, Projet de Psychologie Ergonomique pour l'Informatique, Domaine de Voluceau, Rocquencourt B.P. 105, 78153 Le Chesnay Cedex, France. E-mail: [email protected] ; [email protected] One evaluation method for human-computer interfaces, usually called expertbased evaluation, is defined as an informal method of usability analysis consisting of an analytic examination of a specified, prototyped or existing interface, with the goal of identifying ergonomic design flaws. It relies either on the evaluators' expertise (be they human factors specialists, system designers, software engineers, etc.) and/or on some human factors knowledge as available in documents such as general design guides [1], sets of guidelines [2, 3], checklists [4], standards (e.g., AFNOR, ISO, etc.), and heuristics [5, 6] or criteria [7, 8]. All of these documents have been developed for the purpose of good humancomputer interface design. Paradoxically, only a few of these documents have been evaluated in terms of their validity, thoroughness, reliability, effectiveness, and their ease of use by their potential users. The paper presents available data on these issues and research work focusing on the assessment of ergonomic criteria. The aforementioned issues are then discussed together with the research needed to develop a set of ergonomic criteria [8] into a full evaluation method.
1. T H E U S A B I L I T Y OF G U I D E L I N E S
A survey conducted on the use of Smith and Aucella's [9] compilation of design guidelines showed that it was difficult to use: the guidelines were not easily located, choosing which guidelines would actually be used was difficult as well as establishing priorities among the selected guidelines, and translating guidelines into specific design rules [10]. It was reported that only 41% of the respondents used the compilation to evaluate a proposed design; 25% to evaluate a completed design, and 18% to evaluate an operational system. Smith and Mosier's [3] new and improved set of guidelines, probably the most cited one, has not been further tested with users. A study [11] concerning a draft s t a n d a r d with menu interface design guidelines showed that when using the standard, designers made errors, i.e., made design decisions which contravened some of the guidelines themselves. The errors appeared to result from a lack of information about: the design goals and benefits of using the guidelines; the conditions under which they should be
344 applied; the precise nature of the proposed solution; and the procedures to be followed to apply them. The experience of designers with real, live systems has been shown to have more influence on the design of an interface t h a n a s t a n d a r d s document consisting of only two pages of guidelines related to the use of function keys, and to the partitioning of the screen into fixed fields [12]. In another study, participants had difficulty in interpreting guidelines and relied on pictorial examples, often to the exclusion of the accompanying text [13]. To alleviate some of the previous difficulties in the consultation of large compilations of design guidelines, tools to help designers working with guidelines have been developed (see [14] for a survey of these tools). However, only a few of these tools have been subjected to usability tests [15].
2. USABILITY P R I N C I P L E S , CRITERIA, HEURISTICS, AND STANDARDS To help the design and evaluation of human-computer interfaces, various kinds of dimensions have been defined (heuristics, principles, criteria, etc.). However, the sets of dimensions currently available vary from one author to another in terms of number of dimensions, and level of precision. Some of these differences seem to stem from different design strategies. One strategy has been based on an examination of knowledge derived from research on cognitive processes from which recommendations were extracted and organised into highlevel dimensions [16]. It is such an approach, augmented by a set of expert meetings and other standardisation efforts, which has produced some principles for standards (e.g., ISO 9241-Part 10, [17]). Another design strategy has been based on the review of currently published dimensions with the goal of organising them into a common structure [4, 18, 19]. Sometimes, personal experience has been coupled with existing principles [5], and several published sets of usability heuristics have been compared with a database of usability problems in order to determine which heuristics best explained them [6]. The set of heuristics proposed by Molich and Nielsen [20] has been used, in several experiments, by usability specialists as well as non-specialists to evaluate different systems. Individual evaluators only found between 20% and 51% of the usability problems the interfaces contained [20]. On the other hand, when the reports of several evaluators were aggregated into a single evaluation, such aggregates did r a t h e r well even with only three to five people [20]. It also appeared t h a t major usability problems had a higher probability of being detected than minor problems in a heuristic evaluation, but that about twice as many minor problems were found in absolute numbers; in addition, certain problems (e.g., lack of clearly marked exits) appeared to be more difficult to uncover than others [21]. The set of principles of IS0-9241-Part 10 has been tested in terms of acceptance through a questionnaire survey with usability experts from different countries and was said to provide "a framework for the design and evaluation of dialogue systems" [22]. That set of principles remains to be tested in terms of usability with the population of its real future users which may not be experts. Particularly, the questions of independence and distinctness, if not an issue for general guidance on good interface ergonomics, may be a major issue when
345 planning on using the standard as an evaluation method or as a way of indexing individual recommendations (such as those contained in other parts of the 9241 standard).
3. T H E P R O B L E M
The existence of different sets of principles, heuristics and standards raises indeed several important questions: - intrinsically, how do such dimensions relate to individual guidelines and how comprehensive are they? How independent are they? - from a usage point of view: When used by different people for the evaluation of an interface, does the use of dimensions lead to the same performances? Are the evaluations conducted with the help of these dimensions complete? Do all the dimensions apply in all contexts? It is only by answering such types of questions on internal validity, independence, explicitness, consistency, and external validity that the usability and impact of these dimensions can be established. In that way, dimensions corresponding to these requirements could be used extensively for organising human factors knowledge and transferring knowledge to the designers [7], for training in human factors, for the design of evaluation grids, for structuring evaluation reports, for the design of metrics, for indexing guidelines databases, and for the design of computer-based evaluation tools.
4. T H E C R I T E R I A
APPROACH
To address some of the previous issues we have adopted an empirical approach. To design valid dimensions, available experimental data and a large set of individual guidelines were first translated into rules, then iteratively grouped into sets that were characterised by specific dimensions ("Ergonomic Criteria") which were supposed to best describe the rationale for using such guidelines (initially as a way of accessing a data base). After a few iterations, using also decisions based on group agreement, a set of criteria was established [7]. In this sense the criteria may be said to be valid and complete: they may be said to synthesise most if not all the guidelines available at the time in the field of human-computer interface (A version of this set has been used to index a large compilation of guidelines [2]). Although the set of criteria was already found to allow an accurate description and classification of the usability problems found by experts in an evaluation task [23], further work was aimed at assessing the reliability and effectiveness of the criteria: first by determining the consistency with which the criteria can be used in a classification of design flaws; secondly by determining the role of the criteria as an aid for the evaluation. Reliability of the criteria was assessed in a classification task [24], where participants (human factors specialists and nonspecialists) were asked to identify which criterion was violated for each usability problem (which included explanations about the interaction context and copies of display screens) they were presented with. Agreement statistics (Cohen's Kappa) calculated from the
346 confusion matrices were .61 for the h u m a n factors specialists and .51 for the nonspecialists. An analysis of confusion matrices led to improvements in the set of criteria through: the inclusion of new examples, the addition of comments allowing a better distinction between criteria, and the refinement of some definitions to insure good independence and distinctness. Effectiveness of ergonomic criteria as an aid for the evaluation of user interfaces was assessed in another experiment [25]. Two groups of h u m a n factors specialists evaluated the interface of an application designed purposely to include in its different components a large set of ergonomics design flaws (about 500). After an exploration-diagnosis phase in which all the evaluators' actions and comments were recorded along with the corresponding states of the application, the participants re-evaluated the same interface states through the replay of their first exploration. This second evaluation was conducted with or without ergonomic criteria depending on the group. Results showed t h a t in the first phase, the n u m b e r of usability problems uncovered and the proportions of usability problems found with respect to the size of the aggregates of evaluations were similar for both groups. In the second phase of the experiment, the use of criteria increased both the evaluation diagnosis and the proportions of problems with respect to the size of the aggregates. The criteria were thus helpful for the diagnosis of design flaws, in organising the evaluation, in increasing its completeness and in decreasing the number of experts necessary to diagnose a given proportion of design flaws. O t h e r issues of importance concern the variations in the use of such dimensions by different types of populations (experts/non-experts in: h u m a n factors; computer science; etc.) and the comparison of the relative efficiency of the different dimensions currently available. In those directions, an experiment was conducted to compare the criteria to the ISO 9241- Part 10 principles in terms of number and nature of design flaws diagnosed. The same study also assessed the influence of the criteria on the evaluation diagnosis by nonspecialists; analyses are under way.
5. D I S C U S S I O N AND F U T U R E W O R K The set of criteria in their current form appears to be valid, reliable, and usable. Most importantly, it increases the evaluation performance. The set has been used as a grid for the diagnosis and the reporting of interface evaluations. However, it is not yet a fully developed evaluation method: for example, it does not specify ways to explore systematically the interface nor does it operationalise each criterion in detail. In order to reach t h a t goal, i.e., to develop a more detailed evaluation method which could also increase evaluation performances further, and to design tools (hypertext, expert-system, etc.) supporting such a method, future work should be carried out in the following directions: • extending the content of each criterion; completing the level of detail of the criteria i.e., including a full set of specific "rules" for each of the criteria; identifying ways of supporting each of the criteria; • defining a set of priorities for the evaluation (which criterion first, which next, etc.). Such priorities should be defined as a function of the evaluation goals (e.g., Guidance should have priority over Flexibility for
347 inexperienced users; if high performance is requested, then Work load should have a high priority, etc.); • defining the pre-requisites to the evaluation, i.e., all the task and user related characteristics that are needed to apply the criteria. At this point, links should be identified with formal task analysis methods; • defining ways to examine systematically all interface states and elements (screen, windows, sequences of inputs, etc.). This requires the determination of interaction contexts and the definition of appropriate models of the interface (components and behaviour) on which to apply the criteria. Of course, the reliability of any of these developments of the method will have to be assessed with real users (designers and end-users). The use of the method and its influence on the software produced in terms of cost/benefits will also need to be evaluated. The criteria approach should also be compared with other evaluation methods to assess their relative efficiency, and to identify their relative contributions to the interface evaluation process. However, it should be remembered that a criteria-based evaluation is not intended to replace other evaluation methods (e.g., "model-based" methods, interviews, user testing, etc.). The criteria should rather be viewed as a means for ensuring compliance with interface design guidelines, for uncovering existing design flaws prior to user testing which will remain necessary, especially for complex or new problems for which no guidelines yet exist.
REFERENCES
Mayhew, D. J. (1992). Principles and guidelines in software user interface design. Englewood Cliffs: Prentice Hall. 2. Vanderdonckt, J. (1995). Guide ergonomique des interfaces homme-machine. 1.
3. 4. 5. 6.
7. 8. 9.
Namur, Belgique: Presses Universitaires de Namur. Smith, S. L., & Mosier, J. N. (1986). Guidelines for designing user interface software (Report No. ESD-TR-86-278). Mitre Corporation. Ravden, S. J., & Johnson, G. I. (1989). Evaluating usability of humancomputer interfaces: A practical method. Chichester, England: John Wiley & Sons. Molich, R., & Nielsen, J. (1990, March). Improving a human-computer dialogue. Communications of the ACM, pp. 338-348. Nielsen, J. (1994). Enhancing the explanatory power of usability heuristics. In B. Adelson, S. Dumais, & J. Olson (Eds.), Proceedings of ACM CHI'94 Conference on Human Factors in Computing Systems (pp. 152-158). Boston, Massachusetts: ACM. Scapin, D. L. (1990). Organizing human factors knowledge for the evaluation and design of interfaces. International Journal of Human-Computer Interaction, 2, 203-229. Bastien, J. M. C., & Scapin, D. L. (1993). Ergonomic criteria for the evaluation of human-computer interfaces (Report No. 156). Rocquencourt, France: Institut National de Recherche en Informatique et en Automatique. Smith, S. L., & Aucella, A. F. (1983). Design guidelines for the user interface to computer-based information systems (Report No. ESD-TR-83-122). Hanscom Air Force Base, MA: U.S.A.F. Electronic Systems Division.
348 10. Mosier, J. N., & Smith, S. L. (1986). Application of guidelines for designing user interface software. Behaviour and Information Technology, 5, 39-46. 11. de Souza, F., & Bevan, N. (1990). The use of guidelines in menu interface design: Evaluation of a draft standard. In D. Diaper, D. Gilmore, G. Cockton, & B. Shackel (Eds.), Proceedings of the IFIP TC 13 Third International Conference on Human-Computer Interaction: INTERACT'90 (pp. 435-440). Cambridge, U.K.: Elsevier Science Publishers. 12. Thovtrup, H., & Nielsen, J. (1991). Assessing the usability of a user interface standard. In S. P. Robertson, G. M. Olson, & J. S. Olson (Eds.), Proceedings of ACM CHI'91 Conference on Human Factors in Computing Systems (pp. 335-341). New Orleans, Louisiana: ACM. 13. Tetzlaff, L., & Schwartz, D. R. (1991). The use of guidelines in interface design. In S. P. Robertson, G. M. Olson, & J. S. Olson (Eds.), Proceedings of ACM CHI'91 Conference on Human Factors in Computing Systems (pp. 329333). New Orleans, Louisiana: ACM. 14. Vanderdonckt, J. (1993, December). Hypermedia on human-computer interaction principles and guidelines: A survey. ACM SIGLINK Newsletter. 15. Ogawa, K., & Yonemura, S.-I. (1992). Usability analysis of design guideline database in human-computer interface design. In Proceedings of the Human Factors Society 36th Annual Meeting (pp. 433-437). 16. Marshall, C., Nelson, C., & Gardiner, M. M. (1987). Design guidelines. In M. M. Gardiner & B. Christie (Eds.), Applying cognitive psychology to userinterface design (pp. 221-278). Chichester: John Wiley & Sons. 17. International Standards Organisation (1994). ISO 9241 Part 10. Ergonomic requirements for office work with visual display terminals - Part 10 Dialogue Principles; Draft International Standard (Report No. 18. Johnson, G. I., Clegg, C. W., & Ravden, S. J. (1989). Towards a practical method of user interface evaluation. Applied Ergonomics, 20, 255-260. 19. Ravden, S. J. (1988). Ergonomic criteria for design of the software interface between human and computer in CIM. International Journal of Computer Applications in Technology, 1(1-2), 35-42. 20. Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. In J. Carrasco & J. Whiteside (Eds.), Empowering people: CHI'90 Conference Proceedings (pp. 249-256). Seattle, Washington: ACM. 21. Nielsen, J. (1992). Finding usability problems through heuristic evaluation. In P. Bauersfeld, J. Bennett, & G. Lynch (Eds.), Proceedings ofACM CHI'92 Conference on Human Factors in Computing Systems (pp. 373-380). Monterey, California: ACM. 22. Beimel, J., Schindler, R., & Wandke, H. (1994). Do human factors experts accept the ISO 9241 Part 10 - Dialogue principle - standard ? Behaviour & Information Technology, 13, 299-308. 23. Pollier, A. (1992). Evaluation d'une Interface par des Ergonomes: Diagnostics et Strategies [User Interface Evaluation by Human Factors Specialists: Diagnoses and Strategies]. Le Travail Humain, 55, 71-96. 24. Bastien, J. M. C., & Scapin, D. L. (1992). A validation of ergonomic criteria for the evaluation of human-computer interfaces. International Journal of Human-Computer Interaction, 4, 183-196. 25. Bastien, J. M. C., & Scapin, D. L. (in press). Evaluating a user interface with ergonomic criteria. International Journal of Human-Computer Interaction.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) 1995 Elsevier Science B.V.
349
Usability is Quality of Use Nigel Bevan NPL Usability Services, National Physical Laboratory, Teddington, Middx, TWl 1 0LW, UK [email protected] INTRODUCTION In a paper at HCI Intemational 1991, Bevan et al (1991) asked "What is usability?", and distinguished between broad and narrow approaches to usability. This paper builds on that distinction, identifying the broad approach to usability with the higher level quality objective of"quality of use" (Bevan, 1995a). Quality of use should be the major design objective for an interactive product: does the product enable the intended users to achieve the intended tasks? This relates usability to business objectives and elevates usability from an optional extra to the prime design goal. The narrow approach is complementary and is concerned with the design of features of the product which are a pre-requisite for quality of use. The two different interpretations of usability lead to two approaches to the specification and evaluation of usability. DIFFERENT INTERPRETATIONS OF USABILITY Different people use the word usability in different ways. Even Nielsen (1993) gives two different incompatible classifications. Usability had its academic origins in psychology, human factors and ergonomics. What makes usability different from the rest of design is that it focuses on the human issues. As a contribution to the design process it is most often interpreted by software engineers as relating to skills in interface design which complement other design objectives such as functionality, efficiency (ie execution speed) and reliability. This is a narrow product-oriented view of usability which suggests that usability can be designed into a product. In this sense usability is closely related to ease of use, which is probably the most common way the term is used. It is for instance consistent with Figure 1 in Nielsen (1993) which nests usability within usefulness, within practical acceptability, within system acceptability. It is also consistent with the limited responsibility of usability specialists in many organisations. social acceptability system acceptability
cost
practical acceptability
[
~
[
'
t
reliability
compatibility [I ~
r
]
utility
usefulness
usability Figure 1" Usability as ease of use
350 In this sense one can talk about a system (with a well-designed user interface) which is usable but not useful (ie has no utility). However, for this very reason, it is not a very good way to conceptualise usability. A system which is easy to use but useless will not sell (the product, or the reputation of the usability engineer, except in narrow domains such as computer games!). What really counts is whether a user can achieve their intended goal when they use the product. This is also a "human" question: it immediately raises the issues of what users in what situations carrying out what tasks (not typical software engineering concerns!). Unfortunately the answer depends not only on usability as ease of use, but also utility (is the fight functionality provided?), reliability (does the software crash and can you recover?) and computer efficiency (response time). In designing to enable a user to achieve their goals one needs to make a trade-off between these properties. Usability was defined in this broad sense long ago by Whiteside, Bennett, and Holzblatt (1988). It has the advantages that: • it is a business-oriented view which focuses on the real objectives of design; • it is relatively easy to measure. It leads to the definition of usability used in ISO 9241-11 and the MUSIC project (Bevan and Macleod, 1994): The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. This broad definition of usability turns out to be synonymous with "quality of use" (Bevan 1995a), ie the higher level quality objective that not only does the product meet its specification, but also works in the real world! In software engineering, the conventional objective for quality is to build a software product which meets the specification. However, this alone is rarely sufficient to ensure quality of use - that the product can be used for its intended purpose in the real world. The quality of a product in use can be measured by the extent to which the product can be used with effectiveness, efficiency and satisfaction in a particular context. The European MUSIC project has developed tools and techniques which enable usability to be specified and measured in this way, implementing the principles of ISO 9241-11 (Bevan and Macleod, 1994). USABILITY AND QUALITY The purpose of designing an interactive system is to meet the needs of users: to provide quality of use (see Figure 2, adapted from the working draft of ISO/IEC 14598-1: Evaluation of Software Products). The users' needs can be expressed as a set of requirements for the behaviour of the product in use (for a software product, the behaviour of the software when it is executed). These requirements will depend on the characteristics of each part of the overall system including hardware, software and users. The requirements should be expressed as metrics which can be measured when the system is used in its intended context, for instance by measures of effectiveness, efficiency and satisfaction. At this level, the required system characteristics could be minimum values for the effectiveness, efficiency and satisfaction with which specified users can achieve specified goals in specified environments (for more information, see ISO DIS 9241-11, Guidance on Usability, and Bevan 1995b). The required values of these external metrics provide goals for design. To achieve these goals the internal attributes of the system can be specified as internal requirements. At this level usability requirements may be in terms of general principles (eg provide consistency, support the user's task), specific interface details (eg icons and menu design), or use of style guides. These attributes of the software can be evaluated to produce internal metrics verifying how closely the internal requirements have been met. Although these attributes contribute to achieving quality of use, users and tasks vary so much that no set of interface guidelines alone can ensure that a product will be usable.
351
OPERATIO N
reeds
quality in use
EXTERNAL
system characteristics
external J quality ' re quire me nt s
=,,-. alidation
|tttl|iilllt
evaluation i! . _ (so ft ware u s e ~ - w -
external
met~-s
IL
|tllitltilillltittlllliillttllllt
INTERNAL
.•
sottware characteristics
A I
int( rnal quality re qdre ment
e v aluat ion (at t ribut es)
internal metrics
verification
Figure 2 Quality requirements in design EVALUATION OF USABILITY ATTRIBUTES The usability attributes which contribute to quality of use will include the style and properties of the user interface, the dialogue structure, and the nature of the functionality. Measures of quality of use provide the criteria which determine whether the design of the attributes is successful in achieving usability. There are a number of ways of evaluating the usability attributes of a product. • Style guides such as IBM CUA (IBM 1991a, 1991b) or Windows (Microsoft, 1992) can be used. These provide the raw material for an interface, although quality of use is dependent on the extent to which a dialogue implemented in a particular style is successful in supporting the user's task. • Detailed attributes of the user interface can be evaluated, for instance using ISO standards (Bevan, 1995b) such as ISO 9241-14 (Menu Dialogues). • Individual features can be assessed, such as the presence of a help system or the use of a graphical interface. These are examples of functionality which generally contribute to usability, although particular aspects may not be required in every case. • General usability principles can be used such as the need for consistency, to be selfexplanatory and to meet user expectations, such as those in ISO 9241-10 (Dialogue Principles). These are examples of useful guidelines for design, but they are difficult to use for evaluation as guidelines are imprecise, not universally applicable and may
352 conflict, and there is no way to weight the relative importance of the individual items for usability in any particular conditions. There have been several attempts to use checklists as a basis for evaluating usability (eg McGinley and Hunter, 1992; Ravden and Johnson, 1989; and Reiterer, 1992). Usability guidelines and checklists are useful aids for design, and can be used to make quick expert assessments of user interface design, but they cannot provide a reliable means of assessing whether a product is usable. THE IMPORTANCE OF CONTEXT The quality of use is determined not only by the product, but also by the context in which it is used: the particular users, tasks and environments. The quality of use (measured as effectiveness, efficiency and satisfaction) is a result of the interaction between the user and product while carrying out a task in a technical, physical, social and organisational environment (see figure 3, from Bevan, 1995a). This means that there is no such thing as a "usable product" or "unusable product". For instance a product which is unusable by inexperienced users may be quite usable by trained users.
Context
/ ~ ~ _ ~
social and organisational environment
physical environment environment
interaction ~
product
It
It It |
:
\
J
~ | |
,'
It
It It
°
Quality of use measures
|
- ~ effectiveness| ~ & efficiency J
Figure 3 Quality of Use Measures Determined by the Context of Use It is therefore essential to identify the intended context of use before carrying out any usability evaluation. In many cases it will be necessary to evaluate a product separately for different user groups can?ring out different tasks. This applies both for evaluation of usability attributes and for evaluation of quality of use. For instance it may be necessary to consider different user groups when evaluating the appropriateness of the design and content of a help
353 system. Similarly, when evaluating quality of use by user testing, it may be necessary to decide which combinations of user and task should be selected for evaluation.
USER-CENTRED DESIGN Although the influence of usability professionals during design is often restricted to user interface issues, usability is frequently evaluated by testing a prototype of the product with users. This leads to a number of problems. This type of user-based evaluation is often left till late in design when there is only scope to make minor changes to the user interface. If evaluation reveals deeper problems with the functionality or basic dialogue design, this may be outside the responsibility of the usability professional. The best solution to these problems is to adopt a user-centred approach to design with a continual cycle of user-based evaluation. This is recommended in the first draft of ISO 13407 (Human Centred Design) produced by ISO TC159/SC4/WG6:
concept
prototype
release
understand context specify usability build solution evaluation by users At each stage of design it is important to understand the intended context of use, to specify usability requirements (preferably in terms of user performance and satisfaction), and then construct design solutions which can be evaluated by users. If usability evaluation is left until just before release, there will be no chance to make any significant changes in design to correct deficiencies. In order to achieve a usable product it is important to begin the cycle of understanding, specifying and evaluating usability by using simple mock-ups at the earliest stages of design. For most cost-effective design feedback, repeated evaluation with 3-5 users is recommended (Nielsen, 1993) rather than less frequent evaluation with more users. However, to be confident that usability objectives have been achieved, a final evaluation with 10 or more users will be required. EVALUATION: DESIGN FEEDBACK OR MEASUREMENT Most current usability evaluation practices focus on providing design feedback to improve usability. The most common methods for testing users involve some form of co-operative evaluation, requiting active intervention from an observer to probe usability problems with the user (eg Monk et al, 1993). The potential disadvantages of active intervention are: • You never find out what would really have happened if the user had been left to their own devices. What appear to be minor problems may prevent the whole task being achieved. With active intervention it is difficult to estimate the extent to which task goals would be achieved with real use of the product. • You cannot get comparable measures of task time. • Measures of satisfaction are dependent on the amount of help or clues given in the interventions. The inability to obtain reliable measures means that this type of evaluation cannot be used to test criteria which can form part of a statement of requirements. Specifying formal usability requirements is an important part of the design process. One of the major reasons for the lack of resources allocated to usability in design is that the acceptance criteria do not include specific usability requirements. To obtain reliable measures, the context of evaluation must closely match the intended context of usage. This means that: • it essential to have a complete understanding of the exact context in which the product will be used; • it is essential to replicate the important aspects of this context for the evaluation;
354 • the user should work in realistic conditions without interruption from an observer, in order to accurately replicate the intended context of use. One of the outputs of the MUSIC project is a documented procedure for identifying the context of use and the context of evaluation: the Usability Context Analysis guide (Thomas et al, 1995). When there is no active intervention, design feedback can be obtained by closely observing the interaction (usually with the help of video) and debriefing the user after the session. The Performance Measurement Method (Macleod et al, 1995) developed by NPL as part of the MUSIC project implements this approach. It is used in conjunction with SUMI to measure user satisfaction (Kirakowski, 1995). CONCLUSIONS The objective of usability is to achieve quality of use. Usability requirements should be stated in terms of the effectiveness, efficiency and satisfaction required in different contexts. User-based evaluation can be used to validate achievement of these requirements. Usability attributes provide a contribution to achieving quality of use. The presence or absence of these attributes can be verified early in design. In addition, frequent user-based evaluation of early mock-ups and prototypes is required to give feedback on the quality of use of potential solutions.
REFERENCES Bevan N, Kimkowski J and Maissel J (1991) What is usability?. In: Bullinger HJ (ed): Proceedings of the 4th International Conference on Human Computer Interaction, Stuttgart, September 1991. Elsevier. Bevan (1995a) Measuring usability as quality of use. Journal of software quality (in press). Bevan (1995b) Human-Computer Interaction standards. Proceedings of the 6th International Conference on Human Computer Interaction, Tokyo, July 1995. Elsevier. Bevan N and Macleod M (1994) Usability measurement in context. Behaviour and Information Technology, 13, 132-145. IBM (1991a) SAA CUA Guide to user interface design. IBM Document SC34-4289-00. IBM (1991b) SAA CUA Advanced interface design. IBM Document SC34-4290-00. Kirakowski J (1995) The software usability measurement inventory: background and usage. In: P Jordan et al, Usability Evaluation in Industry. Taylor & Frances, UK (in press). Macleod M, Bowden R and Bevan N (1995) The MUSIC performance measurement method. In: Usability measurement - The MUSIC approach, BOsser T (ed) (in press). McGinley J and Hunter G (1992) SCOPE catalogue of software quality assessment procedures, 3: Usability section. Verilog, 150 Rue Nicolas-Vauquelin, Toulouse, France. Microsoft (1992) The Windows interface - An application design guide. Microsoft Press, Redmond, USA. Monk A, Wright P, Haber J and Davenport L (1993) Improving your human-computer interface. Prentice-Hall. Nielsen J (1993) Usability Engineering. Academic Press. Ravden and Johnson (1989) Evaluating the usability of human-computer interfaces. Ellis Horwood, Chichester. Reiterer H (1992) EVADIS II: A new method to evaluate user interfaces. In People and Computers VII, Monk (ed), Cambridge University Press. Thomas C, et al. (1995) Context guidelines handbook, Version 3. National Physical Laboratory, Teddington, UK. Whiteside J, Bennett J, Holzblatt K (1988) Usability engineering: our experience and evolution. In: Handbook of Human-Computer Interaction, Helander M (ed). Elsevier.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
355
Usability Evaluation: How Does It Relate to Software Engineering? D e b o r a h Hix Department of Computer Science, Virginia Tech, Blacksburg VA 24061 USA ([email protected]) We present an integrated set of activities for development of both user interface and non-interface components in an interactive system. Within the context of these activities for both software and user interface engineering, we present several types of techniques for evaluation of usability. For each technique, we give examples, strengths and weaknesses, and results of its use at appropriate stages in the software engineering process.
1. INTRODUCTION Developers of interactive systems m systems with user interfaces ~ often confuse the boundaries between the software engineering process and the usability evaluation process. This is due at least in part to a lack of understanding of techniques for usability evaluation, as well as which of these techniques are appropriate for use at which stages in the development process. Software engineering has as a goal to improve software quality, but this goal, in and of itself, has little impact on usability of the resulting interactive system. For example, wellestablished "v&v" (validation and verification) techniques focus on software correctness, robustness, and so on, from a software developer's view, with little or no consideration of whether that software serves its users' needs. Thus, quality of the user interface ~ the usability m of an interactive system is largely independent of quality of the software for that system. On the surface, this may seem a controversial statement. However, as shown in Figure 1, there are distinct two domains of interactive system development m the behavioral domain and the constructional domain [1]. The behavioral domain represents the view of the user, while the constructional domain represents the view of the software developer. In the behavioral domain, the user interaction is developed the look and feel and behavior of an interface as a user interacts with it. The user interaction component includes all icons, text, graphics, audio, video, and devices t h r o u g h which a user communicates with an interactive system. In the constructional domain, all software, including that for both the interface and the rest of the application, is developed. Roles that support each of these domains require different training, skills, and attitudes. And while these roles are relatively welldefined and well-trained for software development in the constructional domain,
356 they are much less well-defined and have far fewer well-trained practitioners for user interaction development in the behavioral domain. In the behavioral domain, interaction designers and evaluators do their work, while software engineers and related roles do their work in the constructional domain. Well-known techniques from software engineering are appropriate for developing and evaluating the user interface software component. This kind of software evaluation can have many objectives, such as determining fidelity of a design to its implementation, reliability, reusability, and so on. Usability, however, is not one of these objectives. It is not the user interface software component that is evaluated for usability, but rather the user interaction component (that happens to be instantiated in software). USER INTERFACE DEVELOPMENT
J
Behavioral Domain: User interaction development
Constructional Domain: User interface software development
Figure 1. Domains in user interface development.
Thus, both the behavioral and the constructional domains are necessary for producing an interactive system, but the domain that ensures usability is the behavioral domain. Developers of interactive software now typically recognize the importance of usability, but the lack of knowledge of appropriate techniques for usability evaluation and how they relate to various activities in the software engineering process preclude effective incorporation of these techniques into the process. 2. RELATIONSHIP BETWEEN USABILITY EVALUATION AND SOFTWARE ENGINEERING Figure 2 shows a set of activities for interactive system development, including both non-interface and interface development. The top portion shows development of (non-interface) application software and the bottom portion shows development of the user interface and its software. Arrows indicate communication paths among these various activities. Note that the top portion and the bottom portion indicate analogous activities between interface and non-interface development. The loop at the lower left of Figure 2 is the key set of activities that supports usability evaluation in a timely and effective manner during development; this evaluation is firmly in the behavioral domain. Other activities are related to software development and therefore are in the constructional domain; other kinds of evaluation are related to such characteristics as the robustness, correctness, and performance of the software. For example, the large box on the far right of Figure 2, user-based testing and evaluation of interface and non-interface software, is not evaluation of the usability of the system. Rather, it is user-based evaluation of such considerations as whether there are bugs in the system, whether modules are properly integrated, and so on.
357 Test plan, criteria
straints and problems
Interface design req'ts, usability
t
straints I straints bugs and and problems problems Main feedback: design flaws, errors, modifications Major reconsiderations Software design reouire-
Software implementation
strants strantSan, 1 strants I problems Usability spec'ns
problems
n,
bu0s
problems ain feedback is due to low usability: design flaws, errors, modifications
Figure 2. Activities in the process of developing an interactive system, including both user interface and non-interface components ([reprinted with permission from [1]). 3. TECHNIQUES FOR USABILITY EVALUATION
Although there are numerous common techniques for usability evaluation, at a high level, as shown in Figure 3, these techniques fall into at least two main classifications: empirical and analytical. Empirical evaluation, in this context, is experimentation in which observations of user task performance with an interface are collected and assessed. Analytical evaluation is analysis of a user interface based on some kind of formal or semi-formal representation of that interface, typically without observation of users performing tasks. Both empirical techniques and analytical techniques can be further classified as formative or summative [2]. Formative evaluation is early and continuous evaluation during the user interface development process, with the purpose of improving the usability of a user interaction design. Summative evaluation is comparative evaluation after an interface is implemented, with the purpose of comparing two or more interactive systems to each other. All these broad classifications of usability evaluation can be set in the
358 context of the interactive system development activities shown in Figure 2. Although there are other classifications of techniques for usability evaluation, we will focus on the classification shown in Figure 3, due to space limitations. Within this classification, we will briefly give a broader definition and some examples of each technique and present some strengths and weaknesses. Further, we will discuss for which stage in Figure 2 each technique is most appropriate, and indicate expected outcomes of using that technique at the suggested stage. The examples we present are intended to be representative only, and not exhaustive; there are numerous other techniques for usability evaluation that we do not include here. USABILITY EVALUATION
Empirical
Analytical
Formative
Summative
Figure 3. A broad classification of techniques for usability evaluation. 4. EXAMPLES OF USABILITY EVALUATION TECHNIQUES Most techniques for evaluation, in the behavioral domain, could be classified under the broad heading of usability engineering [3, 4]. These techniques are those that help ensure usability in an interactive system.
4.1. Empirical Formative Techniques Empirically-based formative techniques for usability evaluation are perhaps bestknown as a rapid prototyping-based, iterative design approach to interactive system development. They include establishing usability specifications (quantifiable goals), rapid prototyping of designs, careful observations of representative users interacting with the prototype, and structured cost~benefit and impact analyses of results of the observations [1, 3, 5]. While these techniques are well-established in practice in a wide variety of organizations, there are still many organizations developing interactive systems that do not know these techniques, or do not have the expertise to put them effectively into practice. Such techniques are, in general, those that best ensure usability in an interactive system, especially when used early and continually throughout the development process. They are represented in the cycle in the lower left of Figure 2, and when used effectively, they result in diagnostic lists of usability problems from observations with users, and suggestions for changes to alleviate these problems in a way that is most effective in terms of cost and effect on usability. Further, they allow developers to monitor convergence of the usability of an interaction design toward usability specifications, and thus provide a way to determine when to stop iterating and r e d e s i g n i n g - namely, when usability specifications are met in observed performance and satisfaction from users. 4.2. Analytical Formative Techniques Analytically-based formative techniques for usability evaluation are based on examining some abstract representation or specification of an interaction design,
359 possibly even before any of it has been prototyped or implemented. This necessitates, in most cases, a detailed design of at least some parts of the interaction design, captured in a particular notation, but removes the need for observing users interacting with the system. GOMS [6], command language grammar (CLG) [7], taskaction grammar (TAG) [8], and the keystroke-level model [9] are all examples of analytical formative evaluation techniques. Like the empirical formative techniques, they are typically performed during the user interface interaction design activity shown in Figure 2, after that design had been captured in the appropriate design representation. Usability inspection is another technique for analytical formative evaluation. Heuristic evaluation [10] and walkthrough techniques [11] are well-known examples of this type of evaluation. While they rely less on a formal specification of the interaction design, they nevertheless require experts in usability to assess (inspect) core user tasks for usability. Results of analytical formative evaluation can be a predictive forecast of what performance would be expected if usability were empirically measured and analyzed with users. Results can also be diagnostic, for example, in the case where analytical evaluation early in development uncovers inconsistencies in an interaction design, or, with inspection techniques, where violations of user interface design guidelines are found.
4.3. Empirical Summative Techniques Empirically-based summative techniques for usability evaluation are a type of typical human factors experimentation in which System A is compared to System B, to determine which is "better", where better might be defined as, for example, greatest user productivity, lowest user error rate, highest user satisfaction, and so on. These types of techniques do little to ensure usability of an interaction design; rather, they result in comparison of two (or more) alternative designs, or different versions of the same design. This type of technique generally falls just outside the activities shown in Figure 2, in that summative evaluation is typically performed at the end of a development life cycle for each system.
4.4. Analytical Summative Techniques Analytically-based summative techniques for usability evaluation do not differ markedly from analytically-based formative techniques; the main distinction is in the completeness of the design and of the design representation being analyzed. Thus, these techniques can be used to comparatively analyze two or more different interaction designs, when their design representations are complete. One issue with these techniques is determining what can reasonably be compared across several different design representations. Like empirical summative techniques, analytical summative techniques tend to fall outside activities in Figure 2, for the same reason.
4.5. Hybrid Techniques At least one type of usability evaluation technique is a hybrid of empirical and analytical approaches, specifically, those in which user session transcript data are collected (i.e., empirical observations) and are then analyzed (i.e., analytical evaluation), usually automatically, to uncover particular information in the transcript data. One example of this type of hybrid technique is called maximal repeating patterns (MRPs) [12], in which recurring user action patterns are extracted from a user session transcript. They are analyzed on the hypothesis that repeated patterns of
360 actions contain interesting information about usability of an interface. Such techniques could be either formative or summative in nature, depending on how early in the development life cycle of an interactive system they are used. 4. S U M M A R Y Numerous techniques for usability evaluation exist. Most, in one way or another, fall under the purview of usability engineering. However, they all fall within the behavioral domain in which the user interaction component of an interactive system is developed. Usability evaluation techniques are not applicable in the constructional d o m a i n in w h i c h the user interface software c o m p o n e n t is d e v e l o p e d . Understanding of this dichotomy, and knowledge of the various types of usability evaluation and w h e n to appropriately use them during the overall software engineering process, should help ensure usability of interactive computer systems. ACKNOWLEDGMENTS Dr. H. Rex Hartson has contributed to the exploration of new techniques for usability evaluation, some results of which are reflected herein. Human-Computer Interaction research at Virginia Tech is funded in part by the National Science Foundation, Dr. John Cherniavsky, monitor; and by the Naval Research Laboratory, Dr. Alan Myerovitz, monitor. REFERENCES 1. D. Hix and H.R. Hartson. (1993). Developing User Interfaces: Ensuring Usability through Product & Process, John Wiley and Sons, Inc. 2. R.C. Williges. (1984). Evaluating Human-Computer Software Interfaces. Proceedings of
International Conference on Occupational Ergonomics. 3. J. Whiteside, J. Bennett, and K. Holtzblatt. (1988). Usability Engineering: Our Experience and Evolution. In M. Helander (ed.), Handbook of Human-Computer Interaction, Elsevier Science Publishers. 4. J. Nielsen. (1989). Usability Engineering at a Discount. In G. Salvendy and M.J. Smith (eds), Designing and Using Human-Computer Interfaces and Knowledge-Based Systems, Elsevier Science Publishers, 394 - 401. 5. M. Good, T. Spine, J. Whiteside, and P. George. (1986). User-Derived Impact Analysis as a Tool for Usability Engineering. Proceedings of CHI Conference on Human Factors in Computing Systems, 241 - 246. 6. D. Kieras. (1988). Towards a Practical GOMS Methodology for User Interface Design. In M. Helander (ed.), Handbook of Human-Computer Interaction, Elsevier Science Publishers. 7. T.P. Moran. (1981). Command Language Grammar: A Representation for the User Interface of Interactive Computer Systems. Intern. Journal of Man-Machine Studies, 15, 3 - 51. 8. S.J. Payne and T.R.G. Green. (1986). Task-Action Grammars: A Model of the Mental Representation of Task Languages. Human-Computer Interaction, 2, 93 - 133. 9. S.K. Card and T.P. Moran. (1980). The Keystroke-Level Model for User Performance Time with Interactive Systems. Communications of the ACM, 23, 396 - 410. 10. J. Nielsen and R. Molich. (1990). Heuristic Evaluation of User Interfaces. Proceedings of CHI Conference on Human Factors in Computing Systems, 249 - 256. 11. C. Lewis, P. Poison, C. Wharton, and J. Rieman. (1990). Testing a Walkthrough Methodology for Theory-Based Design of Walk-Up-and-Use Interfaces. Proceedings of CHI Conference on Human Factors in Computing Systems, 235 - 242. 12. A.C. Siochi and R.W. Ehrich. (1991). Computer Analysis of User Interfaces Based on Repetition in Transcripts of User Sessions. Trans. on Information Systems.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
361
Standards and software-ergonomic evaluation Harald Reiterer a+b, Reinhard Oppermann a aGerman National Research Centre for Computer Science (GMD) Institute for Applied Information Technology P.O. Box 1316, D-53731 Sankt Augustin bUniversity of Vienna Institute for Applied Computer Science and Information Systems Liebiggasse 4/3-4, A- 1010 Vienna 1. INTRODUCTION Software-ergonomic evaluation is aimed at assessing a system's quality for particular users, for particular tasks, and in a particular environment. Functional, task oriented and user oriented issues are to be covered by an evaluation, where the user oriented perspective is the main focus of the consideration. An evaluation assesses a system against criteria. Criteria can be theory defined and measured in experiments where particular relationships between independent and dependent variables are constituted by the theory and tested by empirical data. Criteria can also be established by a political, industrial and scientific community for practical reasons, e.g., ending in standards. Standards are based on empirical evidence in HCI-research and practical and economical considerations. Not all desiderata from HCIresearch are transferred into standards - e.g., there may be objections due to technical or financial constraints. The main advantage of standards are the contribution to an increased attention of the ergonomic subject in the community of developers and users of technology and the chance to refer to the described demands of the standards by the users of the technology. In some cases demand of standards are enforceable by law, e.g., to establish common working conditions for visual display terminals (VDT) users the European Union (EU) published a directive concerning the minimum safety and health requirements for VDT workers [EEC 1990]. The national governments of the EU members have to transform this directive into national law. The international standardisation activities in ISO 9241 concerning ergonomic requirements for VDTs are the basis to define the relevant requirements for the technology. An important consequence of these standardisation activities is that software developers and software buyers must take ergonomic requirements and principles into consideration. To assure the conformance of products with the established standards practicable software evaluation methods are needed. 2. STANDARDS ISO 9241 "Ergonomic requirements for office work with visual display terminals (VDTs)" consists of the following parts to be considered during a software-ergonomic evaluation (the other parts are dedicated to task and hardware issues): • Part 10: Dialogue principles • Part 11: Guidance on specifying and measuring usability
362 • Part 12: Ergonomic requirements for presentation of information • Part 13: User guidance • Part 14: Menu dialogues • Part 15: Command dialogues • Part 16: Direct manipulation dialogues • Part 17: Form-filling dialogues ISO 9241 is far from being a pure technical standard that can be assured by quantitative measures. Rather it requires interpretation and tailoring to be useful in user interface evaluation and reflection of the state-of-the-art in research and development. It is subject to an ongoing process of discussion and negotiation. Different stages of consensus and ballots have to be achieved for the multipart standard. Different expertise and interest are influencing the results. They establish a "minimum level of user oriented quality" [Dzida 1995]. Although common terminology has been arranged for the multipart standard no general level of concretness of the definition of the standard can be found. Divers styles have been adopted in several parts: divers structures and divers levels of exhaustiveness. The concept of usability is defined in part 11 by effectiveness, efficiency and satisfaction of the user. The information presentation, i.e., the "look" of the interface, is described in part 12. The dialogue principles, i.e., the "feel" of the interface, is described in part 10. Requirements for user guidance, i.e. prompts, feedback, status, error support and help facilities, are described in part 13. The requirements for several dialogue techniques are described in part 14 to 17. Due to this structure there is a considerable amount of systematic overlap between parts of the standards, in particular between part 10 and 12 with part 14 to 17. Part 10 establishes a systematic framework of ergonomic principles for the dialogue techniques with high-level definitions but with only illustrative applications and examples for the principles. All principles appear in the parts 13 - 17 of the standard but without explicit reference. Part 13 to part 17 of the standard define more or less low-level and fairly exhaustive requirements for the user interface. The structure follows a technological systematic of interface techniques and systems components. In many cases task-, user-, and technical environment aspects are considered as conditions of the applicability or relative relevance of the specified requirements. These aspects constitute the context of use defined in part 11 to be considered in applying the standard for a given work system (see the next section). Part 11 gives the following definition of usability: "Usability is measured by the extent to which the intended goals of use of the overall system are achieved (effectiveness); the resources that have to be expended to achieve the intended goals (efficiency); and the extent to which the user finds the overall system acceptable (satisfaction)." For a more detailed discussion of the term usability see [Bevan 1995]. Effectiveness, efficiency and satisfaction can be seen as quality factors of usability. To evaluate these factors, they need to be decomposed into sub-factors, and finally, into usability measures. [Dzida 1995] presents a usability quality model that refines the term usability. This model introduces a stepwise operationalisation of the factors effectiveness, efficiency and satisfaction, which ends up with specific measures called criteria 1. Each criterion is defined as a required level of measure of a usability factor, the achievement of which can be verified. Example: The usability factor learnability can be measured with the help of the criterion: "the use of an application program has to be learned 1
Note that in the author's notation the term criterion is used different from our terminology: we perceive a criterion as an abstraction of different measures for an ergonomic quality, e.g. flexibility, what Dzida calls sub-factor, where he uses a criterion for the measurable operationalisation of a sub-factor.
363 within 30 minutes". The criteria define measurable requirements and therefore the evaluation of usability implies a comparison between software product attributes and the measurable requirements. Another model to refine usability factors is the linguistic decomposition approach [Vanderdonckt 1995]. 3. CONTEXT OF USE The software-ergonomic evaluation of usability has to be placed in the context of use consisting of the users, their jobs and tasks, the hardware and software, and the organisational, technical, and physical environment. Although usability is a property of the overall system, the focus of attention is usually on a specific element within the overall system- in our case the software product. It is possible to address the usability of the user interface, but only if the particular context of use has been identified. The investigation of the elements of the context of use is done by considering the following characteristics [ISO 9241 Part 11 ]: • The user: User types (e.g., user populations) based on aspects about users skills and knowledge (e.g., software experience, hardware experience, task experience, organisational experience, education, training), personal attributes (e.g., age, gender, physical capabilities, disabilities), cognitive attributes (e.g. intellectual abilities, motivation). • The software: Descriptions of the functionality and main application areas of the software, available instructional items (e.g. handbooks). • The j o b and tasks: Details about the job of the user as a whole, and the tasks for which the software will be used as an aid (e.g., task goal, task name, task frequency, task breakdown, task duration, task flexibility, task output, task dependencies). • Organisational environment: Aspects of the structure of the organisation (e.g. hours of work, group working, job function, work practices, management structure, communication structure, interruptions) the attitudes and culture (e.g., policy on use of computer, organisational aims, industrial relations), and the job design (e.g., job flexibility, performance monitoring, performance feedback, pacing, autonomy, discretion). • Technical environment: Hardware and basic software (e.g. operating system) which is necessary to use the software, reference material. • Physical environment: Workplace conditions (e.g., humidity, temperature, noise), design of the workplace (e.g., space and furniture, user posture, location), workplace safety (e.g. health hazards, protective clothing and equipment). - EVALUATION BASED ON STANDARD REQUIREMENTS The GMD is developing in close co-operation with scientific (University of Vienna) and industrial partners (TI]V Rheinland, Cologne; Priimper & Partner, Munich; Ernst & Young, Vienna) an evaluation approach called Evaluation of Dialogue Systems (EVADIS III). The EVADIS III approach is based on EVADIS II [Oppermann et al. 1992]. The aim of the development of this version of EVADIS is to consider the requirements of the ISO 9241 standard in much more detail. The application of ISO 9241, part 8 and 10 to 17 is supported by the EVADIS procedure. To facilitate the application of the multipart standard explanations for the understanding of the requirements and explanations for the completion of conformance testing are provided. To provide these facilities, i.e., a systematic, explanations, and test instructions, are the aim of EVADIS III. Other examples of tools for evaluating the usability of user interfaces are described in [Balbo 1995].
4. EVADIS
364
4.1 Expert based Evaluation Approach The EVADIS evaluation approach is a comprehensive one consisting of a combination of methods forming a method based expert judgement: • a guideline-oriented checklist of test items to evaluate the user interface of the software, • a questionnaire to explore user characteristics, and • a simplified work place analysis to explore the context of use. The evaluator is a human factors expert using the methods to evaluated the conformance with ISO 9241. The expert approach is based less on a task to be performed by the tested system than on questions asked by software ergonomics. This method is subjective since the expert examines and answers questions according to his personal assessment. It is objective since the ergonomic requirements are operationalized and precisely formulated to an extent enabling the evaluator to answer questions based on clear test rules and traceable conditions. Advantages of an expert based evaluation method are: relatively fast, uses few resources, provides an integrated view, and can be addressed to a wide range of behaviour. 4.2 Evaluation Procedure Detailed test instructions in the EVADIS evaluation guide help to reduce the subjectivity of this method. The evaluation guide consists of five steps and includes a detailed evaluation process description with a clear notation and structure. The first three steps can be executed simultaneously. The results of these three steps are a list of attributes describing the user, the task, and the system context. Step 4 is the central step of the evaluation process. Here all selected test items have to be answered considering the context of use. The result of these activities is a test record that forms the basis for the interpretation of the results and the writing of the test report in step 5. The evaluation procedure can be used for summative or formative evaluation purposes. Therefore it has to be embedded in the software engineering process at different stages [Reiterer and Oppermann 1993]. [Hix 1995] discusses in more detail the incorporation of usability evaluation techniques into the software lifecycle. 4.3 Components of the Evaluation Procedure The simplified work place analysis and the questionnaire to explore user characteristics are still under development. Their development will be based on the necessary context of use information (e.g., tasks attributes, user attributes, system attributes) which are derived during the development of the test items. This retrograde development process guaranties that only necessary information will be gathered with these methods and helps to develop a highly practicable evaluation procedure. The guideline-oriented checklist of test items is at present based on the requirements of ISO 9241 part 8, 10, and 14 that have reached the Draft International Standard (DIS) status. Parts 12, 13, 15, 16, 17 will be included when they have reached the DIS status. The current work is concentrated on the development of the test items. They are embedded in a two-dimensional framework. The first dimension forms the technical system components - inspired by the IFIP model for user interfaces. The second dimension forms the software-ergonomic principles- inspired by the dialogue principles of ISO 9241 part 10. For a discussion of how usable usability principles are see [Bastien and Scapin 1995]. Figure 1 shows a sample item in more detail: how each item is typically structured and how the context of use is considered. To test a software product for conformance the standard requirements have to be interpreted regarding the criteria of usability that are derived from the
365 product's context of use [Dzida 1995]. In the example item the "if-clause" refers to the condition of application, the remaining sentence provides the guideline, i.e., recommendation according to the involved "should". Therefore the conformance test is a two-stage process. First the application of the "if-clause" is to be proven. If the condition is not applicable this will be marked under the test situation; otherwise a comparison of the product attributes with the criteria of usability will be done. In this comparison the context of use has to be considered. The necessary context information has been gathered using the simplified work place analysis and the questionnaire to explore user characteristics. The test instruction gives the evaluator useful information how to test the specific attribute of the software product systematically. The comment contains desirable ergonomic requirements. The evaluator can use this in formation during the rating process. Normally a test item will be answered multiple times in different test situations i=l ....n. Each test situation will be shortly characterised by the evaluator (e.g., menu structure of the main menu; menu structure of the customer window). For each test situation a new rating will be done. Answering a test item in different test situations is necessary for a systematic evaluation of a software product. Different answers in different situations can also be an indication for an inconsistent user interface. Using the ratings of each item a final judgement can be made whether or not the software fulfils the ergonomic requirements of ISO 9241.
Component: 2.1.5.1 Menu structure Source: ISO 9241 Part 14
Criterion: Conformity with user expectation Item No.: 2151.04.10
Standard requirement: If options can be arranged into conventional or natural groups known to users, options should be organised into levels and menus consistent with that order.
Context of use: Task context: Examine task specific conventions
User context: Examine user specific conventions System context: Test instruction: Examine all menu items concerning a consistent order of the structure.
Comment: Menu structures should reflect user expectations and facilitate the user's ability to find and to select menu options relevant for the task and should support the user's flow of work. Task specific or user specific conventions are better then logical or arbitrary ones.
Test situationi=l .....n" Criteria of usability: o o o
consistent order no consistent order notapplicable
Explanation of the rating: Figure 1. Example of a test item
366
4.4 Evaluation Software An evaluation software available under Microsoft Windows supports the evaluator during the whole evaluation process and provides an assessment summary. The software presents all test items on the screen. The evaluator selects the criteria of usability and writes an explanation of the rating. The software calculates an average mark for each ergonomic criterion and each technical component and sorts the results by components or ergonomic principles. Therefore the evaluator is freed of routine work and can concentrate on the evaluation. 5. S U M M A R Y There is an increasing need for practi cal and comprehensive evaluation methods and tools for conformance testing with standards. Practical means that the amount of time and resources must be manageable in software projects. Comprehensive means that the context of use has to be considered during the evaluation of user interfaces. The evaluation approach EVADIS III is such a practical and comprehensive one. In particular, it takes the context o f use into consideration and provides computer support for the use of the evaluation procedure. It supports the evaluator during the evaluation process with detailed instructions. The first version of EVADIS III should be available at the end of 1995. REFERENCES Balbo, S. 1995, Software Tools for Evaluating the Usability of Human-Computer Interfaces. In this volume. Bastien, Ch., Scapin, D. 1995, How usable are usability principles, criteria and standards? In this volume Bevan, N. 1995, What is usability? In this volume EEC, 1990, The minimum safety and health requirements for work with display screen equipment, Directive of the European Economic Community, 90/270/EEC Hix, D. 1995, Usability Evaluation: How Does It Relate to Software Engineering? In this volume Dzida, W. 1995, Standards for user-interfaces, in: Computer Standards & Interfaces 17 (1995), 89-97 ISO 9241 Ergonomic Requirements for Office Work with Visual Display Terminals, Part 8, Requirements for displayed colours, Draft International Standard, June 1994. ISO 9241 Ergonomic Requirements for Office Work with Visual Display Terminals, Part 10, Dialogue Principles, International Standard, September 1994. ISO 9241 Ergonomic Requirements for Office Work with Visual Display Terminals, Part 11, Guidance on specifying and measuring usability, Committee Draft, May 1993. ISO 9241 Ergonomic Requirements for Office Work with Visual Display Terminals, Part 12, Ergonomic require ments for presentation of information, Committee Draft, October 1994. ISO 9241 Ergonomic Requirements for Office Work with Visual Display Terminals, Part 13, User guidance, Draft International Standard, May 1994. ISO 9241 Ergonomic Requirements for Office Work with Visual Display Terminals, Part 14, Menu dialogues, Draft International Standard, May 1994, ISO 9241 Ergonomic Requirements for Office Work with Visual Display Terminals, Part 15, Command dialogues, Proposed Draft International Standard, April 1994. ISO 9241 Ergonomic Requirements for Office Work with Visual Display Terminals, Part 16, Direct manipulation dialogues, Committee Draft, October 1994. ISO 9241 Ergonomic Requirements for Office Work with Visual Display Terminals, Part 17, Form filling dialogues, Committee Draft, December 1994. Oppermann, R., Murchner, B., Reiterer, H., Koch, M. 1992, Software-ergonomische Evaluation - Der Leitfaden EVADIS II. Walter de Gruyter, Berlin, 1992. Reiterer, H., Oppermann, R. 1993, Evaluation of User Interfaces, EVADIS II - A comprehensive Evaluation Approach. Behaviour & Information Technology 12 (1993), 3, 137-148. Vanderdonckt, J. 1995, Using Recommendations and Data Base Tools for Evaluation by Linguistic Ergonomic Criteria. In this volume.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawaand H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
367
Using Ergonomic Rules for Evaluation by Linguistic Ergonomic Criteria Francois Bodart and J e a n Vanderdonckt Institut d'Informatique, Facult~s Universitaires Notre-Dame de la Paix, rue Grandgagnage, 21, B-5000 NAMUR (Belgium) - Tel. : +32 (0)81-72.49.75 Fax. • +32 (0)81-72.49.67 - Email • {fbodart, jvanderdonckt}@info.fundp.ac.be 1. I N T R O D U C T I O N Heuristic Evaluation is a well established method for evaluating a user interface (UI) of an interactive application [1]. This method belongs to the class of informal evaluation methods since the evaluation is performed on the basis of evaluator expertise and knowledge. One characteristic of heuristic evaluation is to guide evaluators by providing them a small set of predefined heuristics rather t h a n a long list of general guidelines. Evaluators are given a widely applicable description of these heuristics so t h a t they are general and largely applicable [2]. In evaluation by ergonomic criteria [3], heuristics that have been judged too general are replaced by a taxonomy of eight main ergonomic criteria (i.e., guidance, workload, explicit control, adaptability, error management, consistency, significance of code, compatibility). These criteria could be decomposed into subcriteria leading to eighteen elementary (sub-)criteria. These criteria present at least three advantages : 1. they are precisely defined, especially when distinguishing between concurrent criteria; 2. they have been experimentally tested and validated; 3. they could be directly linked to useful ergonomic rules (or guidelines). In this paper, we introduce the evaluation by linguistic ergonomic criteria which is an extension of the latter method. First, the reasons that motivated this extension are presented by showing their definition. Second, first steps toward a complete evaluation method are outlined. Third, we exemplify this approach by detailing a UI evaluation report. Finally, we discuss our experience with this work in progress. 2. E V A L U A T I O N B Y L I N G U I S T I C E R G O N O M I C C R I T E R I A 2.1. DEFINITION OF LINGUISTIC ERGONOMIC CRITERIA
An ergonomic criteria is a well recognized usability dimension in human-computer interaction whose reliability effectiveness and usability have been assessed [3,4]. Linguistic ergonomic criteria are criteria that have been linguistically decomposed into sub-criteria according to the seven layers of Nielsen's linguistic model of interaction [5]. They consist of eight main criteria : compatibility, con-
368 sistency, work load, adaptability, dialog control, representativeness, guidance, and error management. The idiosyncratic logic that has been followed to produce these criteria involves the steps indicated below. The eight main criteria were retained and a general wording was adopted for each to allow a full linguistic decomposition. For example, "dialog control" replaced "explicit control" since dialog control does not necessarily have to be restricted to explicit. The main criteria were sorted by decreasing rank of importance to set an explicit ordering between criteria. This absolute ordering provides a first way to a relative ordering when weighting criteria for a particular interactive task. For instance, compatibility is supposed to be more important than consistency : a compatible, but inconsistent, UI is preferable to a consistent, but incompatible, UI since a consistent UI is unuseful if not compatible with the user's task. Similarly, reducing the work load of a UI is inefficient if it is not at least consistent (consistency may reduce work load). Other criteria are ranked in the same way. The eight main criteria were subdivided into sub-criteria according to the seven layers of Nielsen's model [5] : goal, pragmatic, semantic, syntactic, lexical, alphabetic, and physic. These seven layers are highlighted for each criteria after separating important aspects : consistency is divided into consistency between applications (inter-application consistency) and consistency within a same application (intra-application consistency). Intra-application is further refined into linguistic sub-criteria as depicted in table 1. Table 1. Linguistic sub-criteria of consistency 2. Consistency 2.1 Inter-application Consistency 2.2 Intra-application Consistency 2.2.1 Pragmatic Consistency 2.2.2 Semantic Consistency 2.2.3 Syntactic Consistency 2.2.3.1 Operational Consistency 2.2.3.2 Homogeneousness 2.2.4 Lexical Consistency 2.2.4.1 Spatial Consistency 2.2.4.2 Grammatical Consistency 2.2.4.3 Linguistic Consistency 2.2.5 Alphabetical Consistency 2.2.6 Physical Consistency Compatibility, work load and adaptability have been directly refined in the same way, whereas dialog control, representativeness and guidance include a first separation between conversation (dynamic behavior) and presentation (static appearance). Error management escapes from this rule since it follows a chronological decomposition. Criteria are consequently decomposed into sub-criteria which are sorted by
369 decreasing linguistic level. Within a same criteria, higher-level sub-criteria are therefore considered as more important than lower-level sub-criteria. Elementary criteria which have been proved successfully assessed (i.e. sub-criteria leading to a high recognition rate in the reading matrix) have been extracted from [4] and merged into this decomposition. Finally, special criteria have been extracted from Coutaz's design criteria [6] and Ravden & Johnson's evaluation criteria [7] because practical experience showed that it was useful to identify them as a whole in recognition procedures. Moreover, managing and classifying ergonomic rules (as in [8]) has been greatly facilitated by the introduction of these fine sub-criteria They include : homogeneousness, concision, action reversibility (especially), action structure, visual clarity, explicitness,... This approach resulted in a set of eighty two elementary (sub-)criteria including the eight main criteria. A complete list of ergonomic criteria with definition, aims, decomposition and examples is electronically available through anonymous F T P at arzach.info.fundp.ac.be/~jvd/Criteria.ps http: //www. info. fundp, ac.be/-jvd/Criteria.http.
and via Mosaic at
2.2. T o w a r d an e v a l u a t i o n m e t h o d by linguistic e r g o n o m i c criteria We successfully used the following approach both in TRIDENT project [9] and Human-Computer Interaction course in a Master of Computer Science degree for four years. This approach consists in performing the following steps : 1. conduct a context analysis (i.e. a task analysis, an identification of user stereotypes, a description of work place) resulting in a series of parameters for task, user and work place (e.g. pre-requisite, objective task environment, task structure, task experience,...) 2. select appropriate ergonomic (sub-)criteria with respect to parameters listed in context analysis. Thus, specific ergonomic criteria matching the user's task are raised. This selection could be achieved manually or in a computer-aided fashion with production rules. We are currently developing a knowledge base containing such production rules (Figure 1). t a s k profile = closed ^ u s e r level = i n e x p e r i e n c e d ~ criteria (Explicit Actions) = very i m p o r t a n t t a s k profile = closed ^ u s e r level = i n t e r m e d i a t e ~ criteria (Mixed Actions) = very i m p o r t a n t t a s k profile = closed ^ u s e r level = experienced ~ criteria (Implicit Actions) = i m p o r t a n t task structure = high ^ objective environment = existing criteria (Support Compatibility) = very important
Figure 1. Some production rules for deriving (sub-)criteria from context analysis. 3. sort all specific (sub-)criteria by rank of importance. This step is helped by existing orderings implied by criteria and by linguistic levels. 4. gather ergonomic rules for each (sub-)criteria into a list. This activity involves accessing ergonomic rules in an ergonomic knowledge base. This could be done manually by writing a checklist of ergonomic rule titles to evaluate by extracting them from style guide, standards or design guides. [9] is an example of
370 multi-purpose design guide containing 3,700 ergonomic rules sorted by division (e.g. input, display, dialog, visual design, interaction media, interaction styles,...) by criteria and by linguistic level. We are currently implementing a hypermedia tool allowing automatic rule gathering by criteria, linguistic level, interaction style, and interaction objects. 5. define one or two task scenarios for each interactive task of the UI. for windows-based applications, this usually consists of a series of sub-tasks matching all menu item active in pull-down menus of the menu bar. For direct manipulation UI, this activity typically represent a sequence of direct actions of interaction objects to reach a task's goal. 6. experience the UI through defined task scenarios one after another ; identify, explain and report usability problems in the light of the checklist of specific ergonomic rules. Of course, problems without linkage to these rules could also be reported. 7. experience the whole UI globally and consolidate usability problems into an UI evaluation report (in particular, for intra-application consistency).
2.3. UI Evaluation report Such a report often contains steps involved in task scenarios, screen snapshots at a given time during execution and pointers of usability problems. Each problem is fully explained and argumented by one or many related ergonomic rules, themselves sorted by criteria. Figure 2 highlights a UI hard copy augmented by numbered arrows. Each number denotes a usability problem reported below. 2.4. List of usability problems (excerpt) 1. "Base line Data", "Actuals Data", "Project Trace",... are windows related to the current activity in the hierarchical project decomposition. The user always has to display these windows for each new activity. Ergonomic rule : coordinated windows should be used for dependent tasks Reference : 6.13.2.1"12, p. 433 in [8] Ergonomic criteria : minimal actions Linguistic level : lexical Possible solution : coordinated windows so that their contents automatically changes when current selection changes. 2. Four windows are displayed separately, whereas their information are all related to current selection. Ergonomic rule : multi-windowing should be used for independent tasks Reference : 6.13.2.1"1, p. 430 in [8] Ergonomic criteria:operational compatibility Linguistic level • syntactic 6. Too much windows at the same time leading to unnecessary cluttering Ergonomic rule : simultaneously displayed windows should not exceed 6 Reference :6.13.2.2"4, p. 435 in [8] Ergonomic criteria : cognitive respect Linguistic level:semantic
371
--File
Create
Edit
Hierarchy
Process
Options
Format
--View 88K
2.2 - Identify Major Hodules 2.3 - Identify Hardware Requirements . . . . . Z q - Uoolng " 241-CodeHodulel _'_" . . . . . . . . Z - q - Z - u o a e r, l o a u l e Z 2.4.3Code Hodule 3 2.4.4 - Code H
4-
5 6 7
10 . . 'I I ' / Act 1d-3 _ '' "
ues¢~:
Develop
ES: /./ .
.
.
.
.
D_._,--.._
.
.
.
.
.
3 . 2 - Develop Test D~ 3 . 3 - I n s t a l l i n Test Environment \ 3.4 -Install~Test Production Envir. .8 i:l Develop Training Plan / I __. T._:_;_.. I~al / I Project u.=....,,,,,,y / I I . . . . Project F i n a l R e v l e w ~ R e V. l s l O n S l A c t • I d - 3" " 1l P r o ..1" e c t System Operational ftl Oesc= Oeve~ D , - n 4 . . .
.
.
Ins:
.
/ .
1
l
5
/
.
~ .
.
.
.
12
==
.
Project Trace _Name_: I DEII O Start • I q~r 1 1 8 9 " Finish: | 2/ 2t90 Data Date ] 12131188 enlmnl-~tP1-
n
6a
....... 1
II |
Plan
5 .
I I " I Our'5 ' " I
i I: I:
_P:ojeo¢
_ 9 /
Test
E F :"
9Ie9 .
r~-4-
I--IP O 4b q~ l I l I q~ i . ~ a t a
1
"
Deadline
Date:
\
|
OFF
/ 2,3,4,5,6,11,13,15
Figure 2. A screen snapshot from a project management UI.
7. "Toolbox" window provide tools for drawing Gantt charts. This is helpful for defining project structure only, not for project information display. Ergonomic rule • windows that are not related to current interactive task should be undisplayed. Reference • 6.13.2.2"5, p. 435 in [8] Ergonomic criteria • syntactic compatibility Linguistic level" syntactic . . .
12."Project Tree" window obscures the contents of another window which is placed just behind. Even the window title is no longer visible. It is therefore quite difficult to understand the aim of this window, forcing the user to click on it in order to see the information. Ergonomic rule • overlapping windows should not obscure definitively the contents of other windows which are used in the current interactive task. Reference • 6.13.2.4"4, p. 440 in [8] Ergonomic criteria" Visual guidance Linguistic level • lexical
3. C O N C L U S I O N The approach mentioned in 2.2 should be interpreted only as a first step toward an empirically validated method for UI evaluation. Nevertheless, our four year experience emphasizes several advantages t h a t were part of the main goals •
372 • the double ordering (i.e., by criteria, by linguistic level) is a feasible method for organizing ergonomic rules; • the reliance of evaluation on context analysis (especially on task analysis) is greatly improved since specific criteria are derived from it. This strategy clearly establishes a continuum ranging from context analysis to ergonomic rules through specific ergonomic criteria. This feature is particularly interesting because general guidelines only have limited impact on usability (28% of cases) whereas specific guidelines are more influential (about 54% of cases); • the idea of giving evaluators a checklist of specific ergonomic rules provide them more guidance : having only ten heuristics sometimes seems too little, but having a ton of general rules is definitively impracticable. Conversely, a reasonable list of specific ergonomic rules may reduce the high level of expertise required by experts in Heuristic Evaluation; • the above approach does not preclude that evaluation should be carried out without users. Evaluation could be achieved with or without users. In the case of evaluating with users, experience shows that evaluators detect usability problems more quickly by referencing them with respect to ergonomic rules; • the checklist of specific ergonomic rules is intrinsically normalized so that it can be reused at both design and evaluation time. REFERENCES 1. J. Nielsen and R. Molich, Heuristic Evaluation of User Interfaces, Proc. of CHI'90 (Seattle, 1-5 April 1990), ACM Press, 1990, pp. 249-256. 2. J. Nielsen, Enhancing the Explanatory Power of Usability Heuristics, Proc. of CHI'94 (Boston, 24-28 April 1994), ACM Press, 1994, pp. 152-158. 3. J.M.C. Bastien and D.L. Scapin, Ergonomic Criteria for the Evaluation of Human-Computer Interfaces, technical Report No. 15, INRIA, Rocquencourt, May 1993. 4. J.M.C. Bastien and D.L. Scapin, Preliminary Findings on the Effectiveness of Ergonomic Criteria for the Evaluation of Human-Computer Interfaces, Adjunct Proc. of InterCHI'93 (Amsterdam, April 24-29, 1993), pp. 187-188. 5. J. Nielsen, A Virtual Protocol Model for Computer-Human Interaction, International Journal of Man-Machine Studies, V24N3, 1986, pp. 301-312. 6. J. Coutaz, Interfaces homme-ordinateur - Conception et r~alisation, coll. Dunod Informatique, Bordas, Paris, 1990. 7. S. Ravden and G. Johnson, Evaluating Usability of Human-Computer Interfaces, A Practical Method, Series in Information Technology, Ellis Horwood, Chichester, 1989. 8. J. Vanderdonckt, Guide ergonomique des interfaces homme-machine, Presses Universitaires de Namur, Namur, 1994, ISBN 2-87037-189-6. 9. F. Bodart, A.-M. Hennebert, J.-M. Leheureux, I. Provot, J. Vanderdonckt, G. Zucchinetti, Key Activities for a Development Methodology of Interactive Applications, Chapter 4 in Critical Issues in User Interface Systems Engineering, D. Benyon and Ph. Palanque (Eds.), Springer-Verlag, 1995.
III.13 Usability Engineering
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
375
A T e a c h i n g M e t h o d as a n A l t e r n a t i v e to the C o n c u r r e n t T h i n k - A l o u d M e t h o d for U s a b i l i t y T e s t i n g P. R. Vora a and M. G. Helander b a U S WEST Technologies, 1475 Lawrence St., Suite 304, Denver, CO 80202, USA Dept. of Mechanical Engineering, Div. of Industrial Ergonomics, Linkoping Institute of Technology, 58133 Linkoping, Sweden
b
In this paper, we propose a teaching method as an alternative to the concurrent think-aloud (CTA) method for usability evaluation. In the teaching method, the test participant, after becoming familiar with the system, demonstrates it to a seemingly naive user (a confederate) and describes how to accomplish certain tasks. In a study that compared the teaching and the CTA methods for evaluating usability of human-computer interactive tasks, the results indicated that the number of verbalizations elicited using the teaching method far exceeded those elicited using the CTA method. Also, the concurrent verbalizations were dominated by the participants' interactive behavior and provided little insight into the participants' thought processes or search strategies, which were easily captured using the teaching method. 1. INTRODUCTION The goal of usability evaluation is to quickly identify and rectify usability deficiencies in human-computer interfaces. The Concurrent Think Aloud (CTA) method, where the participants verbalize their thoughts while interacting with the system, is considered to be one of the most useful methods for practical usability evaluation (Nielsen, 1993a). The strength of the CTA method is "to show what the users are doing and why they are doing it while they are doing it in order to avoid later rationalizations" (Nielsen, 1993b; p. 195). Thinking aloud, however, is not a normal practice for most people using a computer, and is often intrusive and distracting to the users of an unfamiliar software (Holleran, 1991; Nielsen, 1993b). The following methods have been suggested to overcome the "unnaturalness" of the concurrent think-aloud method:
1. Constructive interaction or Codiscovery learning (O'Malley, Draper, and Riley, 1984; Kennedy, 1989), where two test users work together to solve a problem (see Bainbridge, 1990). However, the disadvantage is that the users
376 may have different strategies for learning and using computers (Nielsen, 1993b). 2. Question-answer method or coaching method (Kato, 1986; Mack and Burdett, 1991), where participants are asked to complete a given task by obtaining information from a tutor (a coach), whose responses are based on a policy of limited assistance. This method is aimed at discovering the information needs of users to improve training, documentation, and interface design. We propose a teaching method, as alternative to the concurrent think-aloud method as a non-intrusive and a more natural method to elicit verbalizations from test participants. In the teaching method, the participants interact with the system first, so that they are familiar with it and have acquired some expertise in accomplishing tasks using the system. At the end of the task, the test participant is introduced to a naive user (a confederate); the confederate is briefed by the experimenter to limit his/her active participation and not to become an active problem-solver. The test participant describes to the confederate how the system works and demonstrates to him/her a set of tasks pre-determined by the experimenter. Since the confederate is introduced as a novice, the test participants can be expected to feel more comfortable in describing and explaining how to accomplish tasks using the system. Therefore, they may freely reveal useful information such as their understanding of the system components, organization of the information, their strategies for accomplishing tasks, and their mental model of the system. Whereas, if they are asked to describe the system to the experimenter, who is assumed to be an expert, they may feel uncomfortable and may withhold their verbalizations. This approach of introducing a "non-threatening" participant in the study can be compared with Gelman and Gallistel's (1978) Counting Study, where 2- to 5year old children were introduced to a puppet and were asked to answer or do things for the puppet. The presence of "non-threatening" puppets helped the experimenters both to elicit children's responses and to increase their attention span and tolerance for a lengthy testing procedure. Though not used for usability testing, we think that introducing a puppet may be useful for evaluating systems designed for young children. In this paper, we describe a study that compared the teaching method and the CTA method for human-computer interactive tasks. We were particularly interested in identifying the differences in the content of the verbalizations using both these methods and to determine if the teaching method was a useful alternative for evaluating usability.
377 2. METHOD 2.1. Test-bed To compare the two methods we used an experimental database on nutrition referred to as NutriText. NutriText was designed as a hypertext system, where the related pieces of information were connected by "hot buttons" or links.
2.2. Participants Nine participants (1 female and 9 males), graduate and undergraduate students at the State University of New York at Buffalo participated in the study. Their ages ranged from 20 years to 32 years (average = 23.4 years). On a pre-experiment questionnaire, all the participants categorized themselves as computer users and indicated that they were familiar with using a mouse to interact with the computers. 2.3. Procedure After receiving training on NutriText, the participants answered 12 search questions and thought-aloud concurrently for the 1st, 6th, and 12th questions. At the end of the search task, they were introduced to a confederate to whom they described NutriText and demonstrated how to access information to answer 3 search questions given by the experimenter.
3. RESULTS & DISCUSSION
The participants' concurrent and teaching verbalizations were transcribed and analyzed for their content; see Table 1. The teaching method seemed more natural to the users as the number of verbalizations elicited from the participants using the teaching method far exceeded those elicited during the concurrent think-aloud method m 72.8 vs. 59.4 verbalizations per participant over 3 questions. The differences were not only in the total number of verbalizations, but also i n their content. The concurrent verbalizations were dominated by the participants' interactive behavior (26.4% verbalizations were Actions) and provided little insight into their thought processes or search strategies (10.8% were Explanations for Actions and 3.1% were Search Strategies). For example, $1: "...that means that I have to find which vitamin corresponds to wheat germ [State Goal]... and I'll start with fat-soluble vitamins [Action]... I'll start with vitamin A [Action]... backtrack [Action]... vitamin D [Action]... backtrack [Action]... vitamin E [Action]... here I found wheat germ [Match Goal]... so I think the person is lacking vitamin E [Answer]..." In contrast, in the teaching method, the participants' thought processes and search strategies were more pronounced in their verbalizations (only 15.0% of the
378 verbalizations were Actions, whereas 18.0% were Explanations for Actions and 11.2% were Search Strategies). For example, $3: "... so the key word here probably is biotin [State Goal]... you have to look under all the various categories of various vitamins A, B, C, D, and find if we can find the word biotin.., and then work backwards [Strategy]..."
Table 1 Content analysis of Concurrent and Teaching protocols CTA method Protocol Type Answering Questions (3) Read Question State Goal Assumptions Info. from Past Knowledge Strategy Information Organization Explanation for Action Action Analyze Action Reading Screen Match Goal Answer Interaction with Experimenter Others Subtotal: Describing NutriText Purpose of NutriText Identify Features Explain use of Features Information Organization Explanation by Example Difficult Characteristic Interaction with Confederate Interaction With Experimenter Others Subtotal: Total:
Teaching Method
Avg. # Avg. # verbalizations % of Subtotal verbalizations / par ticip ant / participant
3.71 3.71 0.71 0.43 1.86 0.00 6.43 15.71 0.57 5.14 9.14 3.86 6.29 1.86 59.43 n/a n/a n/a n/a n/a n/a n/a n/a n/a 59.43
6.3 6.3 1.2 0.7 3.1 0.0 10.8 26.4 1.0 8.7 15.4 6.5 10.6 3.1
% of Subtotal
3.13 3.25 3.50 1.25 8.13 1.50 13.13 10.88 3.25 1.75 15.13 4.25 1.88 1.75 72.75
4.3 4.5 4.8 1.7 11.2 2.1 18.0 15.0 4.5 2.4 20.8 5.8 2.6 2.4
1.00 5.13 8.00 6.13 6.13 0.50 0.63 2.13 2.25 31.88 104.63
3.1 16.1 25.1 19.2 19.2 1.6 2.0 6.7 7.1
379 Further, when the participants were describing NutriText, their mental model of the information organization and the difficulties they experienced while interacting with it were evident in their verbalizations. The participants also used their experiences to give hints and suggestions to facilitate the confederate's future interaction with NutriText. For example, $5: "This basically gives the information about vitamins [Purpose of NutriText]... and it is arranged in a sort of hierarchical way [Information Organization]..." $2: "... and when you click on vitamin B, you get a whole list of words, and unless you are careful, you don't realize these are vitamins [Difficult
Characteristic]..." The participants may have experienced difficulty in verbalizing concurrently while trying to search for information due to their lack of familiarity with both the domain and hypertext technology. Verbalizing, therefore, posed an additional workload on them causing them to elicit their actions and not their thoughts. Since participants' actions and navigation data can be collected in the background by the system, the CTA method did not provide additional useful information.
4. CONCLUSION In sum, the data suggests that with the teaching method it's easier both to elicit verbalizations and to capture the thought processes of the participants (including search strategies and interaction difficulties). We believe that the teaching method is particularly suitable for the interactive tasks requiring extensive navigation (moving between screens or windows), where using think-aloud method may reveal action history rather than their thoughts. Other benefits of the teaching method are: 1. A more natural approach. The teaching method avoids situations where the experimenter unwittingly constraints the participants' behavior (Kirakowski and Corbett, 1990). 2. No effect on task performance. The teaching method does not facilitate initial stages of learning, which is a problem with the CTA method (Ahlum-Heath and Di Vesta, 1986). Berry and Broadbent (1990) have also shown that instructions to report justifications for actions verbally can improve performance of a task. 3. No effect on the time to complete the task. Since teaching method is used at the end of the experimental task, time-based interaction data can be collected during the use of the system and they will not be confounded by the concurrent verbalizations. Ericsson and Simon (1984) have summarized evidence indicating that verbalizations during performance may have a significant effect on the time taken to complete a task.
380 REFERENCES
1. Nielsen, J. (1993a). Evaluating the thinking-aloud technique for use by computer scientists. In H. Rex Hartson and D. Hix (Eds.), Advances in Human-Computer Interaction, Vol. 3. Norwood, New Jersey: Ablex. 2. Nielsen, J. (1993b). Usability Engineering. New York: Academic Press. 3. Holleran, P. A. (1991). A methodological note on pitfalls in usability testing. Behavior & Information Technology, 10, 345-357. 4. O'Malley, C. E., Draper, S. W., and Riley, M. S. (1984). Constructive interaction: A method for studying human-computer-human interaction.
Proc. IFIP INTERACT "84 First Intl. Conf. Human-Computer Interaction (London, U. K., 4-7 September), 269-274. 5. Kennedy, S. (1989). Using video in the BNR usability lab. ACM SIGCHI Bulletin, 21, 92-95. 6. Bainbridge, L. (1990). Verbal protocol analysis. In J. R. Wilson and E. N. Corlett (Eds.), Evaluation of Human Work: A Practical Ergonomics Methodology. New York: Taylor and Francis. 7. Kato, T. (1986). What "question-asking protocols" can say about user interface. International Journal of Man-Machine Studies, 25, 659-673. 8. Mack, R. L. and Burdett, J. M. (1991). When novices elicit knowledge: Question-asking in designing, evaluating, and learning to use software. In R. Hoffman (Ed.), The Cognition of Experts: Empirical Approaches to Knowledge Elicitation. New York: Springer-Verlag. 9. Gelman, R. and Gallistel, C. R. (1978). The Child's Understanding of Number. Cambridge, MA: Harvard University Press. 10. Kirakowski, J. and Corbett, M. (1990). Effective Methodology for the Study of HCI. New York: North-Holland. 11. Ahlum-Heath, M. E. and Di Vesta, F. J. (1986). The effect of consciously controlled verbalization of a cognitive strategy on transfer in problem solving. Memory and Cognition, 14, 281-185. 12. Berry, D. C. and Broadbent, D. E. (1990). The role of instruction and verbalization in improving performance on complex search tasks. Behavior & Information Technology, 9, 175-190. 13. Ericsson, K. A. and Simon, H. A. (1984). Protocol Analysis: Verbal Reports as Data. Cambridge, MA: The MIT Press.
Symbiosis of Human and Artifact
Y. Anzai, K. Ogawaand H. Mori (Editors) © 1995Elsevier ScienceB.V. All rights reserved.
381
Tools for Iterative User Interface Design: UI-tester and OST Toshiyuki Asahi, Hidehiko Okada, Osamu Iseki, Ryoichi Matsuda* Kansai C&C Research Laboratories, NEC Corporation, 4-24, Shiromi 1-Chome, Chuo-ku, Osaka 540, Japan asahi@obp, cl. nec. co.jp *Second Personal C&C Operations Unit, NEC Corporation Abstract A computer-aided iterative design environment is proposed. A usability testing tool "UI-tester" and a user interface design tool "OST" are integrated so that iterative design programs can be effectively implemented in the early stage of product development without usability expertise. A feasibility study being conducted on an ongoing facsimile product development project in this environment gives data showing that common erroneous pattern extraction functions of UI-tester are applicable for identifying user interface problems, and that user interface redesigns are completed without additional delays in the product development cycle. 1. I n t r o d u c t i o n Iterative design is a practical way of assuring a product's usability. In this case, usability testing and design revisions should be closely related and applied repeatedly before the product is shipped to reduce usability problems to a certain level. However, traditional usability testing methods, such as monitoring or thinking aloud, have had their disadvantages: 1) an overly time-consuming data analysis preventing the testing of a number of participants, 2) possibly distorted evaluation results from individual differences among participants, and, 3) a tendency for testing to be executed in the late product development stages because of the need for operable prototypes for user behavior observation. Usability inspection methods such as heuristic evaluation[l], usability walkthrough[2], and cognitive walkthrough[3] have been proposed for nonempirical usability evaluation. These methods are applicable early in the design stage and are cost-effective[4], but they assume evaluating teams including some human factors experts; this assumption can be a hurdle to their implementation in companies. The objective of this research is to develop a ~ computer-aided iterative design environment that enables efficient usability testing in the early design stage with no usability specialist requirement.
382 2. M e t h o d Consistency between operational procedures, which users and designers have come to expect, is an important factor influencing product usability[5]. These user and designer expectations are called the user mental model and design model, respectively. Automatic evaluation could be achieved if both models were represented in a common form and one could compare their consistency, but this is difficult because of the impossibility of exactly representing the mental model in a computational form. The fundamental concept of computer-aided usability testing is having usermachine dialogue data including erroneous operations reflect the user mental model to some extent, and having a standard operation sequence represent the design model. Therefore, usability evaluation can be done by analyzing the difference between the user interaction log and the standard operation sequence. UI-tester was developed to test the validity of this concept. Dialogue visualization technique UI-tester has functions to test relatively simple user interfaces, that is, those whose dialogue sequence for a certain task can be represented in a single route. Facsimiles and automatic teller machines are good examples of devices having this kind of user interface. But even these user interfaces need diagramming techniques that allow testers to grasp how users interact with the machines, since the actual user dialogue data tends to be quite complex. These diagrams should: 1) clarify deviations from the standard operation sequence; and 2) appropriately reduce the complexity. A "dialogue structure diagram" is newly proposed to enable visualization of the user-machine dialogue data. Figure 1 shows a n example of the dialogue structure diagram. Nodes and arcs represent states of the machine and user operations, respectively. A standard interaction sequence is represented as repetitive trial by " R E S E T " I 'Initial State
I ~ _ ~
I Function Menu ]............ i ..... IItemSelecti°n ! INumerical Input I ICharacter Input I
_~
(a)sequential incorrect operations Q
omission,of_necessary__ope~tion i ~ ~
.....
:~ rejected operations
Figure 1. Interaction log diagramming with Dialogue Structure Diagram
383 nodes tied with broad arcs, and is input by rehearsal before the diagram is displayed. Each node is located on one of five levels, i.e., dialogue modes, such as "initial state", "menu selecting", "numerical input", and so on. This makes it easier to understand the diagram. All erroneous user operations are drawn with fine arcs, enabling testers to easily identify where and how they deviate from the standard sequence. To reduce the complexity of the diagram, sequential error operations are represented as a single node like in Figure l(a). Testers can check the detailed interaction sequence simply by clicking the node. E x t r a c t i n g Common Erroneous P a t t e r n s If a number of participants make similar errors, the usability problems causing these errors should be considered serious. In order to find these problems, UItester has functions to extract common erroneous patterns from a number of dialogue logs for a certain task. These functions are also effective in eliminating "noise", i.e., factors not necessarily related to user interface problems, from the extracted patterns. To support the effective spotting of critical patterns, one of the four following viewpoints of pattern matching can be selected by the testers: (1) Deviation occurrence." From what states and by what operations do the participants deviate from the designed dialogue sequence commonly? (2) Typical operations in error sequences." What operations are commonly made by the participants in the error sequences? (3) R e t u r n from error sequences: With what states and by what operations do the participants return from their deviation to the designed sequence? (4) Types o f j u m p i n g patterns." Do the participants' dialogue sequence jump forwards or backwards on the designed sequence commonly? If yes, from and to what states does their dialogue sequence jump? Each viewpoint has four or five conditions that determine the degree of similarity. For example, the conditions for the viewpoint (1) are given in Table 1. Erroneous dialogues are grouped into a locus only if they deviate from the same node of a standard dialogue sequence by the same operation, when condition 1 is applied. The criteria become less strict as the condition count increases. The degree of commonness is also adjustable, and is given as the percentage of participants who have shown the prescribed error patterns. All extracted common error patterns are displayed on the dialogue structure diagram. Testers Table 1. Conditions determining similarity for the pattern matching viewpoint "deviation occurrence". starting node of deviation I the first deviant operation condition 1
the same node
the same operation
condition 2
the same mode
the same operation
condition 3
the same node
and
don't care
condition 4
don't care
the same operation
condition 5
the same mode
don't care
384 can find the critical error patterns while adjusting these pattern matching viewpoints, conditions and the degree of commonness interactively. Integrating With A User Interface Design Tool OST is a user interface CAD tool for designing and simulating facsimile user interfaces, and can also generate program codes for product manufacturing. Figure 2 shows an OST-designed operation panel with numeric keys on its right side and an LCD on the other side. In the simulation mode, users can actually operate displayed objects (menu items, numeric keys, etc.) by means of touching the panel or using a mouse, and OST changes the LCD's display contents just as the facsimile will do. This means that usability testing can be done in the early design stage without any physical prototype. The proposed iterative design environment was developed by mutually integrating OST and UI-tester (Figure 3). The panel layout includes operational objects and LCD display contents and is graphically prototyped on OST. Interaction logs are collected while participants try to complete given tasks in UI simulation with OST. The log files are analyzed with UI-tester, and extracted erroneous interaction patterns are revealed to the testers. After the designers and testers have finished examining how to improve the usability based on the testing results, the user interface is redesigned on OST. When the UI problems are thought to have been decreased enough, OST outputs source codes thus enabling manufacture of the physical facsimile prototype. wm
m
no
w
m
m
m
w
~
man
m
w
•
an
w
w
m
~,
simulatca. I: files .~ analyzer ~ - , t ~ •
~ :::::::::::::::::::::::::::::::::::::::::::::::::::::::: :- • • : - :: ":0~:: 'IL.........L......~ :~!IJ=================================== :: ::::::: : : " " : :"":;l~] .............. i ! !iiiiiii~i~!ii!si~i~ii~i!ii~iii!iiii!!~U~!i!~.~..~!i~i~i~ii~i~i!i~i!i~i~!~!~ ::i~'=:'[[
'i~ ~
9
[
~
~ii;~ ~i~:~
N:[UI •
~:~ FAX
COPY
REG
REPOR
~i
i~
~:]
Figure 2. Example of an operation panel designed with OST.
[i
:[ ~ l a y m m
(~ e l ~ "
m
m
n
m
w
m
Ji
:
w
w
.tes.'ti~ r~.s.~ . ,
."
Figure 3. Iterative design with OST and UI-tester.
3. Feasibility Study OST and UI-tester were applied to facsimile product development. After the UI prototype was designed with OST, usability testing was conducted for eighteen participants. The tasks were somewhat more complex than just sending copies; a) setting the current time and date; b) registering the sender's name and telephone number; and c) registering the receiver's data (on the sender's side) for sending with a single key. Nine usability problems were found by the common error pattern extraction functions. Two examples of them are explained below, both of which were observed in task c).
385 (1) When a participant tried to input Kanji to register the sender's name, there was a chance he/she used the wrong keys causing rejection by the system. In fact, this error pattern was found in the data of 14 of 18 participants. This common error was determined to be caused by two different functions being mapped into the same key. Figure 4(a) shows a pattern indicating this problem. (2) When a participant tried to input a phone number to register the sender's number, there was a chance he/she omitted a specific operation and jumped forward. In fact, this error pattern was found in the data of 12 of 18 participants. This common error was determined to be caused by the participants lack of being able to find out how to input "pause(-)" at specific positions in the phone number. Figure 4(b) indicates this problem. In order to eliminate these problems, UI redesigns such as key label modification, additional message display, etc., were done. A person not an expert on human factors was able to complete the data analysis task in a day. In addition, user interface revisions on OST were completed in a few days thereby preventing delays in the product development schedule. .(4)
~
," i~Ii) 2)
Figure 4. Common error patterns extracted from 18 participant data.
4. Discussion A computer-aided iterative design environment has been achieved with the integration of UI-tester and OST. A feasibility study has shown that this environment enables practical usability improvement before shipment in actual product development. Even though it is not clear just how the testing technique is effective especially when compared with other traditional methods, this kind of environment should be helpful for companies lacking expertise in implementing usability programs into their product development cycles. It should be noted that most Japanese companies fall in this category. We have considered that errors are "critical" when many participants make them, and have developed a common erroneous pattern extraction function.
386 However, what patterns should be extracted for usability inspection still remains a mystery because usability contains so many aspects. Dialogue pattern analysis techniques should be explored further for usability problem detection[6]. Ongoing research activities include the following: (1) Establishing theoretical models defining what extent of usability can be evaluated just from user interaction logs. This includes a comparative consideration of the testing method with other practical testing methods (e.g. protocol analysis, heuristic evaluation, cognitive walkthrough, and so on). (2) Proposing new methodologies and techniques to expand the UI-tester concept to other user interfaces. We have already started developing testing tools, e.g., "GUI-tester" for Windows TM applications. (3) Providing testing user interface functions capable of being done without user participation. Layout Appropriateness[7] is a good example of such functions where a GUI screen widget layout is evaluated automatically from the viewpoint of operational cost (e.g. mouse movement distance). (4) Providing a usability program framework for the computer-aided iterative design environment. The framework should include technical, organizational, beneficial and human resource considerations. References [1] J. Nielsen: Usability Engineering, Academic Press, 1993 [2] R. Bias: Walkthroughs: Efficient Collaborative Testing, IEEE Software, September, pp. 94-95, 1991. [3] C. Lewis, P. Polson, C. Wharton, J. Rieman: Testing a Walkthrough Methodology for theory-based design of Walk-up-and-Use Interfaces, CHI '90 Conference Proceedings, pp. 235-242, 1990. [4] R. Jeffries, H. Desurvire: Usability Testing vs. Heuristic Evaluation: Was There a Contestg SIGCHI Bulletin, Vol.24, No.4, pp.39-41, 1992. [5] D. A. Norman: Cognitive Engineering, in D. A. Norman, S. W. Draper (Eds.): User Centered System Design, Lawrence Erlbaum Associates, pp. 31-61, 1986. [6] A. Sioehi, R. Ehrich: Computer Analysis of User Interface Based on Repetition in Transcripts of User Sessions, ACM Transactions on Information Systems, Vol. 9, No. 4, pp. 309-335, 1991. [7] A. Sears: Layout Appropriateness: A metric for widget-level user interface layout evaluation, Technical Report CAR-TR-603, CS-TR-2838, University of Maryland Computer Science Department, 1992.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Moil (Editors) © 1995 Elsevier Science B.V. All rights reserved.
387
A composite measure of usability for human-computer interface designs Kay Stanney and Mansooreh Mollaghasemi a aIndustrial Engineering and Management Systems Department, University of Central Florida, 4000 Central Florida Blvd., Orlando, FL 32816, USA A methodology for formulating a composite measure of interface usability is provided. The measure integrates multiple usability criteria into a single measure by which designs can be directly compared. The primary advantages of the proposed approach are the ability to consider multiple criteria and to weight the importance of these criteria according to a particular company's priorities and requirements. 1. INTRODUCTION Usability is a term often used to described the efficacy of an interface. It has been defined as '~he extent to which (an interface) affords an effective and satisfying interaction to the intended users, performing the intended tasks within the intended environment at an acceptable cost" (Sweeney, Maguire, and Shackel, 1993). There are 3 primary manners of assessing such usability: user-based, theory-based, and expert opinion-based assessment. This study focuses on user-based assessments, with the level of user effectiveness and satisfaction forming the basis of this assessment. User-based usability assessments of human-computer interface designs generally involve the evaluation of multiple system and user criteria. Multiple criteria are used because it is difficult to derive a single measure that effectively characterizes the overall usability of a complex system (Kantowitz, 1992; Karat, 1988). Thus, since no single indicator is pervasive, a multitude of measures are used in system assessments. These criteria generally include performance time, errors, use of help, learning time, mental workload, and satisfaction (Eberts, 1994; Shneiderman, 1992). These measures can be quantified through direct measurement (i.e., performance time, frequency of errors, use of help), analysis (i.e., learning time can be determined via an NGOMSL analysis (Kieras, 1988) or production rules (Kieras and Poison, 1985)), or through questionnaires (i.e., human mental workload (Hart and Staveland, 1988) and satisfaction (Slmeiderman, 1992)). When a combination of usability indicators are used, often times evaluators report the results of each indicator separately without providing a comprehensive assessment of the system This makes it difficult to compare different designs. A technique is needed that integrates usability testing results, thus providing a composite measure of usability to facilitate direct comparisons between interface design options. The present study proposes the use of
388 the analytic hierarchy process to rank-order computer interface designs based on multiple evaluative criteria. The importance of each evaluative criterion can be weighted according to a particular company's priorities and requirements. Using these weights and the objective or subjective value of each criterion, a rank-ordering of the different interface design options can be achieved. Alternatively, Kantowitz (1992) suggests that a statistical combination of multiple usability indicators be derived. This is the approach that Paas and Van Merrienboer (1993) used when they combined mental workload assessments with performance efficiency (i.e., accuracy) by transforming the measures into z-scores, which were then displayed and related in a cross of axes. This approach, however, is limited because it assumes a linear relationship between mental workload and performance scores, which may not always hold. MiRa (1993) has suggested using the analytic hierarchy process (AHP) to rank-order computer interfaces based on multiple evaluative criteria. Using the AHP, MiRa (1993) evaluated and compared different computer interfaces in terms of three criteria: usability, leamability, and ease of use once mastered. This approach is one of the few in which computer interfaces have been evaluated in terms of an integrated assessment of several interface characteristics. It has several advantages over traditional approaches such as predictive modeling, expert evaluations, observational studies, surveys and experimental studies. First, the approach allows simultaneous consideration of multiple criteria. Secondly, by using the AHP, one is able to provide a measure for the consistency of the decision maker's judgments. In fact, no other approach used in solving multiattribute decision problems provides the ability for such consistency checks. Mitta's use of the AHP (1993), however, has some drawbacks. While she used multiple criteria in her analysis, the evaluations of the criteria were based solely on subjective assessments. Multicriteria decision making inherently has subjectivity built into the process, therefore it is not advisable to needlessly add to this subjectivity. The major disadvantage of Mitta's (1993) approach is that the implementation involves interpretation of users' abilities in judging interface characteristics. With this approach experimenters make pain~se comparisons of users with respect to their abilities to make satisfactory judgments regarding usability and learnability. The repeatability of this assessment procedure is questionable, since any two evaluators assessing the same set of users using this technique could come up with quite different results. This paper presents an alternative to using the subjective opinions of experimenter's by evaluating some of the interface attributes in terms of objective measures. 2. THE PROPOSED APPROACH
The proposed approach uses the AHP to assess the relative importance of each software usability attribute with the objective of selecting the best interface. The AHP was originally introduced by Thomas Saaty in the mid 70's (Saaty, 1980, 1994). Since its development, AHP has been applied in a wide variety of practical applications including those related to economic planning, energy policy, health, conflict resolution, project selection, and budget allocation. In fact, the AHP is one of the most popular multicriteria decision making methodologies available
389 today. The popularity of this method is primarily due to its flexibility, ease of use, and the ability to provide a measure of the consistency of the decision maker's judgment. The use of the AHP in assessing interface usability is a natural extension. The AHP is appropriate for situations, like usability assessment, where both objective and subjective criteria must be considered. The AHP also imposes more structure into the usability assessment process by requiring each decision-making criteria to be organized into a hierarchy. Each level of the hierarchy consists of a few, critical criteria that influence the quality of the interface (e.g., user effectiveness, user satisfaction, design intuitiveness, cognitive workload). Each criterion is, in turn, decomposed into sub criteria. For example, user effectiveness could be decomposed into number of errors and performance time; whereas intuitiveness could be broken down into percent of task correctly completed and number of references to help. The process of decomposing the various criteria continues until the decision-maker reaches the most specific elements of the problem, the alternatives (e.g., interface design options) which are listed at the bottom of the hierarchy. Once the hierarchy has been constructed, the next step is to determine the relative importance or priority of each of the elements at each level of the hierarchy. While the relative importance of the criteria for any given system will depend on the objectives being met by that system, Raskin (1994) provided some insight into possible priorities. Based on the results of recent surveys of a number usability labs, three general criteria were assessed in terms of their relative importance. These factors included user satisfaction, productivity, and intuitiveness, with the assigned relative importance of 50%, 30%, and 20%, respectively (Raskin, 1994). In general, relative importance is determined through a pairwise comparison of each pair of elements with respect to the element directly above. In general, this comparison takes the form of "how i ~ o r t a n t is element 1 when compared to element 2, with respect to the dement above?" The decision maker then provides one of the following responses in either numeric or linguistic fashion: Importance Numerical Rating Equally Important (preferred, likely) 1 Weakly Important 3 Strongly Important 5 Very Strongly Important 7 Absolutely Important 9 2, 4, 6, 8, are intermediate values
The responses of the decision-maker are put in the form of a comparison matrix t~om which the local priorities are determined using the eigenvalue method. The local priorities at each level of the hierarchy are then synthesized to determine the cardinal ranking of the alternatives. The selection and structuring of attributes is one of the most difficult as well as critical steps in solving multiattribute problems. For interface design, designers must identify those attributes that are most important to usability. There is really no ideal number of attributes for consideration. One must, however, be aware that too few attributes may mean that some
390 important design attributes have not been included while too many attributes may mean that too much detail has been included in the design decision. Perhaps the most important restriction in the selection of the attributes is that they must be independent of one another. This condition is sometimes difficult to achieve. Therefore, one may assume that important attributes are independent originally, but be willing to reject the outcome if he or she feels that this assumption significantly affects the results. Once the attributes have been selected, it may be helpful to structure them into a hierarchy. This is particularly useful when the number of attributes becomes large. 3. METHOD In order to determine the validity of the proposed approach, a set of interfaces was assessed. The objective of this assessment was to determine the relative usability of these interfaces using a number of evaluative criteria. 3.1 Interface Designs The purpose of this analysis was to perform a user-based usability assessment of a realistic and an unrealistic desktop interface design. The realistic design looked much ~like an office would, with a picture of a desk, phone, file cabinet, Rolodex, printer, trash can, and calendar. The unrealistic design looked like a typical Windows design with windows, icons, and menus. 3.2 Procedure The two interface designs were assessed based on user productivity, design intuitiveness, and user satisfaction. This data was obtained from a study by Miller and Stanney (1995). The relative weights of these criteria were derived from Raskin (1994), with user satisfaction, user productivity, and intuitiveness being assigned the relative importance of 50%, 30%, and 20%, respectively. Productivity was quantified in terms of performance time. Intuitiveness has been defined as a function of the percentage of tasks correctly completed and the number of help references made (Raskin, 1994). This analysis used the percentage of tasks correctly completed as a measure of design intuitiveness. User satisfaction was obtained by the users' pairwise comparison of the interfaces. Each of the users were asked the question of '~n terms of satisfaction, how much better is interface 1 than interface 2?". The answer to this question led to the realization of how well each interface performs with respect to user satisfaction. 4. RESULTS The two interfaces were tested by 30 users. Each user performed a set of 4 tasks on both interface designs (see Miller et. al., 1995 for details on the tasks). A relative measure of user satisfaction was obtained for each interface by the subjects' painmse comparison of the two interfaces. The user's performance time was traced by the computer as a measure of user productivity. The number of tasks completed was determined from a computer trace from which an Intuitiveness score was obtained. The mean performance times for the 30 subjects were calculated to be 926 seconds and 1177 seconds for the realistic and the unrealistic
391 interfaces, respectively. The mean number of tasks correctly completed for the 30 subjects were determined to be 90% and 88.4% for the realistic and the unrealistic interfaces, respectively. The relative importance weights for the two objective measures (i.e., productivity, design intuitiveness) were then computed by (1) and (2), respectively. Note that equation (1) is used when the lower values of the measures are preferred (e.g., cost, performance time) while equation (2) is used when higher values are favored (e.g., profit, percent of tasks correctly completed). 1 OM~ = ~ Ck × S
(1)
where r 1 S= )-'--, k=l
Ck
and O M k = objective measure for alternative k, and
c k = the actual measure obtained from alternative k.
OMk
Pk
=
r
(2)
k=l
where P k = the actual measure obtained from alternative k. Using equation (1), the relative importance of the interfaces with respect to user productivity was determined to be 0.56 and 0.44 for the realistic and the unrealistic interfaces, respectively. The relative importance of the interfaces with respect to design intuitiveness was computed to be 0.51 and 0.49 for the realistic and the unrealistic interfaces, respectively. The users' painvise comparison of the two interfaces with respect to user satisfaction resulted in the relative weight of 0.55 for the realistic interface and 0.45 for the unrealistic interface. These relative weights along with Raskin's (1994) relative importance measures of the criteria were synthesized to achieve a composite usability measure for each interface. The overall usability measure for the realistic interface was computed to be 0.55 while that of the unrealistic interface was calculated to be 0.45. 5. DISCUSSION
The results from the AHP analysis indicated that the overall usability of the realistic interface was better than the unrealistic interface. This was based on a composite measure of user
392 productivity, user satisfaction, and design intuitiveness. This composite indicator provided a common basis of comparison by integrating information from a diverse set of usability criteria. This overall measure of usability allowed for a straightforward comparison of the two interface design options and clearly indicated that the realistic design was preferable in terms of usability.
REFERENCES Eberts, 1~ (1994). User interface design. Englewood Cliffs, NJ: Prentice Hall. Gibson, J. (1994). How to do systems analysis. Englewood Cliffs, NJ: Prentice Hall. Hart, S.G. and Staveland, L.E. (1988). Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In P.A. Hancock and N. Meshkati (Eds.), Human mental workload. Am~erdam, Netherlands: North-Holland. Kantowitz, B.H. (1992). Selecting measures for human factors research. Human Factors, 34(4), 387-398. Karat, J. (1988). Software evaluation methodologies. In M. Helander (Ed.), Handbook of human-computer interaction (pp. 891-903). Amsterdam, Netherlands: North-Holland. Kieras, D.E. (1988). Towards a practical GOMS model methodology for user interface design. In M. Helander (Ed.), Handbook of human-computer interaction (pp. 67-85). Amsterdam, Netherlands: North-Holland. Kieras, D.E. and Polson, P. (1985). An approach to formal analysis of user complexity. International Journal of Man Machine Studies, 22, 365-394. Miller, L. and Stanney, I~M. (1995). The effects of realistic versus unrealistic desktop interface designs on novice and expert users. Proceedings of the 6th International Conference on Human-Computer Interaction, Yokohama, Japan, July 9-14. MiRa, D.A. (1993). An application of the analytical hierarchy process: a rank-ordering of computer interfaces. Human Factors, 35(1), 141-157. Paas, F.G.W.C. and Van Merrienboer, J.J.G. (1993). The efficiency of instructional conditions: an approach to combine mental effort and performance measures. Human Factors, 35(4), 737-743. Raskin, J. (1994). Intuitive equals familiar. Communications of the ACM, 37(9), 17-18. Saaty, T.L. (1980). The analytic hierarchy process. New York: McGraw-Hill. Saaty, T.L. (1994). How to Make a Decision: The Analytic Hierarchy Process, Interfaces, 24(6), 19-43. Shueiderman, B. (1992). Designing the user interface (2nd ed.). Reading, MA: AddisonWesley. Sweeney, M. Maguire, M. and Shackel, B. (1993). Evaluating user-computer interaction: a framework. International Journal of Man-Maehine Studies, 38, 689-711.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) 1995 Elsevier Science B.V.
393
W h y choose? A process a p p r o a c h to u s a b i l i t y t e s t i n g T. Kelley a n d L. A l l e n d e r Army Research Laboratory, AMSRL-HR-MB, Aberdeen Proving Ground, Aberdeen, Maryland, 21005-5425 1. ABSTRACT Two sets of prototype screens for a complex, computerized analysis tool were evaluated using a series of usability analysis techniques. The empirical, or experimental usability method identified more interface design problems of a severe nature than the other methods and gave a clear indication of which prototype design to choose for the final development process. While the individual walkthrough evaluation identified the most design problems overall, many of the problems tended to be of a less severe nature than were identified by the experimental method. The implications for selecting appropriate usability techniques and using them collectively, as a process, are discussed. 2. INTRODUCTION Several comparisons of usability methodologies have recently appeared in the literature (e.g., Karat et al., 1992; Virzi et al., 1993; Jeffries et al., 1991). Questions addressed by this research include: How effective is one particular usability technique over another? How many and what type of problems are uncovered with each technique? How much does one technique cost in comparison to other usability analysis techniques? Are the benefits of cost savings of a given method reduced by lack of problem identification? Comparisons of usability techniques are, however, sometimes difficult to interpret. Each method is still only loosely defined and overlap across techniques is common. In general, though, it can be agreed that the techniques vary according to certain factors: whether the technique is empirical and task oriented or a subjective walkthrough; and for the walkthrough techniques, whether they are individual or group, what the subjects' level of user interface (UI) design expertise is, whether the group is all end users or pluralistic (a mix of users, UI experts, programmers, etc.), whether the walkthrough is "self-guided" or guided by a set of guidelines, and the method of data collection (written, "think-aloud"; time periods ranging from hours to weeks). The desire to use one technique over the other is driven by cost and effectiveness concerns. But, the literature is unclear which variant of which technique: empirical, individual or group walkthroughs is the "best." Karat et al. (1992), compared three techniques, empirical, individual walkthrough, and group walkthrough and found that the empirical method identified the largest number of problems and identified problems missed by the
394 individual and group walkthroughs. Cost analysis also showed that the empirical usability technique used the same or less time to identify each problem. Contrary to the findings of Karat et al. (1992), Jeffries et al. (1991) reported heuristic evaluations found the most problems with the lowest cost. However, Jeffries used usability specialists, whereas, Karat used a mix of mostly end users and developers of graphic user interface systems, along with a few usability specialists and software support staff. Also the subjects in the Jeffries et al. study reported problems found over a two-week period; those in the Karat et al. study, a three-hour period. One common element across the reviewed research is that the software being evaluated using the various techniques was all relatively simple. When developing analytical tools for the scientific community, the software can be quite complex, both in terms of overall conceptual organization and in terms of individual screen designs. The work reported here is an evaluation of a complex task analysis modeling tool being developed for the U.S. Army and helps to answer the questions about the effectiveness of the various usability techniques for larger scale software products. To reiterate an earlier point, there is no clear answer on which usability technique is the best. Karat et al. (1992) point out that each of the techniques serves to uncover different types of problems. In the work reported here, the natural developmental cycle of the software to be evaluated was used as the deciding factor: Different techniques were used at different times. 2. O B J E C T I V E S The usability evaluations reported here were conducted with two goals. The first goal was to user the output of the usability evaluations to select one of two different interface design prototypes for a complex task analysis tool for the scientific and analysis communities and to subsequently refine the selected design. The two prototypes differed in their conceptual and organizational structure, and, therefore, in most of the screen designs. However, a number of the screen designs, particularly "lower level" data input screens were identical. The second goal was to use and compare a variety of usability analysis techniques as a part of the developmental process in order to gain insight into the strengths and weaknesses of each and to guide future technique selection. Five techniques were used: The first and fifth techniques were employed without strict experimental controls; the second, third, and fourth techniques were employed and evaluated in an experimental setting. The five techniques were: (1) individual walkthrough evaluation; (2)empirical evaluation (experimental); (3)individual heuristic walkthrough (experimental); (4) group walkthrough (experimental); and (5) group pluralistic evaluation. 3. PARTIC IPANTS Three different groups of participants used the five different techniques: One group used the individual walkthrough; one group used the three techniques that were evaluated experimentally and one group used the pluralistic evaluation technique. The group who participated in the individual walkthrough evaluation consisted of six expected end-users of the software under development who were currently active users of the predecessor software tool.
395 The second group participated in the three experimental evaluations and comprised 20 subjects. All 20 subjects were employees of the Army Research Laboratory and had various educational and professional backgrounds. All of these subjects were equal in experience in that they had all received a three day training course on the predecessor software, but had not used the software since the course. All 20 participated in the empirical evaluation. Half of the subjects then participated in the individual heuristic evaluation and the other half participated in the group walkthrough evaluation. Finally, the third group included 18 participants who employed the pluralistic evaluation technique. By definition, this group included a mix of end-users, designers, programmers, and human factors practitioners. 4. PROCEDURE 4.1. I n d i v i d u a l W a l k t h r o u g h Evaluation For the individual walkthrough evaluation, a hard copy packet was mailed to the participants after initial prototyping of the two prototypes had been completed. This was the first evaluation point in the developmental process. The packet included printouts of every screen in each of the two prototypes and a set of instructions. The instructions provided information about how to interpret the printouts and how to map the prototypes onto the functionality of the predecessor software. Rather than the human interface guidelines which might be given to subjects in a standard heuristic evaluation, the evaluators were given a set of questions to guide and prompt their responses. For example, "What do you like or dislike about the prototypes? .... What changes should be made to the layout and organization of the prototype?" They were also encouraged to mark directly on the printouts and to make any other comments they felt appropriate. They had three weeks to evaluate both prototypes. 4.2. Empirical Evaluation The second evaluation technique was the empirical method. It was used at the same time as the other two experimental techniques which was during the development of software specifications but before the coding had begun. Twenty subjects were each tested individually using the same Gateway 2000 33 megahertz computer with a color VGA monitor. The interactive screen prototypes, which were created using the ToolBook T M development environment, were presented in a counterbalanced scheme so the time and errors for each could be compared. All subjects received refresher training session on the predecessor software immediately prior to the experiment. Subjects then had to successfully complete five training tasks before proceeding with the experiment. The experiment consisted of carrying out 10 goal-oriented tasks that would actually be performed using the software. Although prototypes were used in the experiment, there was sufficient functionality to permit performance of all ten tasks. The same set of ten tasks was presented in different random orders for each of the two prototypes. (Of note, 10 subjects received one set of 10 tasks; 10 subjects received a different set of ten tasks. This change was to maximize the amount of information available to guide actual interface revisions.) Data were collected by use of a video camera and also directly by the computer the subjects were using during the experiment. Subjects were not given any special instructions about how fast or accurately to work.
396
4.3 Heuristic Evaluation The third technique, the heuristic evaluation, was conducted immediately after the empirical evaluation session. Ten of the twenty subjects who participated in the empirical evaluation were randomly selected. They were given the set of usability guidelines (Nielsen and Molich, 1990) which included: simple and natural dialog, speak the user's language, minimize user memory load, be consistent, provide feedback, provide clearly marked exits, provide short cuts, good error messages, prevent errors. The subjects were then instructed to use the guidelines to identify usability problems with each interface. They could choose to use the computer on-line versions of the prototypes or be given a printout of each screen to work from.
4.4. Group Walkthrough For the fourth technique, the group walkthrough, the subjects were the remaining ten from the empirical evaluation. Subjects met in one room facing a large screen monitor displaying the prototype. One experimenter served as the moderator for the session. Task lists which were used for the empirical evaluations were given to each of the subjects and then each task was presented for evaluation with the interface. Subjects vocalized any concerns they had with the interface while each task was being walked through. Data were collected by using a video camera and by a second experimenter taking notes.
4.5. Group Pluralistic W a l k t h r o u g h The final usability technique was the group pluralistic evaluation. It served as the final review before actual software coding began on the task analysis tool. Eighteen people participated. The prototype was displayed onto an overhead projector and one moderator, the program developer, took the group through as much of the interface as possible in the time that was allotted, which was approximately eight hours. The pluralistic walkthrough is distinguished by the wide range of experience from its participants (Bias, 1991). In this case, the pluralistic walkthrough included end users, human factors experts, developers, and programmers. 5. R E S U L T S As Figure I illustrates, the individual walkthrough evaluation identified more problems t h a n any of the other techniques. The individual walkthrough evaluation technique identified a total 39 unique problems compared to 21 for the pluralistic, 15 for the empirical, 12 for the heuristic, and 9 for the group walkthrough. Severity ratings of each problem identified were calculated using a three-point scale. Two human factors experts conducted the severity ratings. Each human factors expert did his own rating independently, then the ratings were compared for differences. If there were any disagreements, discussion ensued until a consensus was reached. Figure 2 shows the severity rating scores for the problems found with each usability technique. As Figure 2 indicates the empirical method identified the highest number of high severity problems, a total of six. The individual walkthrough identified the highest number of low severity problems, a total of 29.
397
40 35 30 25 20 15 10 5 0
Figure 1. Number of problems identified.
30 25
2o
I I I" °w !
15
10 5 0
to
m
._~_~
Ca
._
"~
o
i..
Q.c-4.,
Figure 2. Error severity identification.
The individual walkthrough and heuristic evaluations were further compared in order to address issues of low priority problem identification. The heuristic evaluation yielded a total of 84 individual comments. (Note that many individual comments all reported the same problem so there are more comments t han problems identified). All of the 84 comments were classified in order to give an indication of the type of comments received. The individual walkthrough evaluation yielded a total of 356 individual comments. Of the 356 individual comments, 84 comments were randomly selected and classified in order to give some indication of the type of comments received and compare to the heuristic evaluation. The categories used for the classification of user comments were as follows: fidelity problem with the prototype, question about the prototype, a compliment of the prototype, a suggestion to change prototype, a problem with the interface identified, a syntax or wording problem, and a meaningless comment that could not be interpreted. Results indicated, the individual walkthrough evaluation, which was given without the "standard" usability guidelines, yielded a smaller percentage of problem identification (16%) than did the heuristic evaluation, which was given with guidelines (33%). Results also indicated that evaluators had more questions during the individual walkthroughs (22%) than during the heuristic evaluation (2%). This is not surprising given that those in the individual walkthroughs were seeing the prototypes for the first time, whereas, those in the heuristic evaluation had previously participated in the empirical evaluation. During the experimental evaluation, the two prototypes were evaluated on the time and errors obtained on one set of ten tasks. A 2 (prototypes) X 10 (tasks) repeated measures ANOVA was conducted on the data for 10 subjects. Results indicated a significant main effect of prototype, F(1,9) = 14.39, p< .01, as well as task, F(9,81) = 14.85, p < .01. The effect of prototype by task interaction was also significant F(9,81) = 5.15 p < .05. 6. CONCLUSIONS The usability analysis process should be a combination of usability analysis techniques. Each usability analysis technique has its own advantages and
398 disadvantages, but together, each technique can compliment the other methods and can collectively be more powerful than if used separately, in other words, a
Gestalt analysis. First, for this evaluation process, we chose to use the individual walkthrough which was a paper and pencil exercise without usability guidelines characteristic of an heuristic evaluation. These two factors most likely led to the feedback including many comments or questions and not problem identification per se, and of the problems that were identified, many were of low severity. Next we used the empirical method which was task-oriented and used the interactive prototypes. With this method, the largest number of severe problems were identified by noting where subjects made navigation and menu selection errors and which steps took the most time. The heuristic and the group walkthrough evaluations came next with the expectation that subjects would draw from their intensive experience in the empirical evaluation and be more likely to identify additional, severe usability problems. Both techniques did identify additional problems: the heuristic evaluation identified slightly more problems; this was a tradeoff consideration if selecting one method over the other. The last evaluation technique was the pluralistic evaluation since at this point in the development cycle, the most severe problems should have been identified already, allowing the discussion of detailed design changes from the user and programmer points of view. Additional investigation needs to be done to help clarify the process, as well as to identify the best order in which to use each methodology in an overall usability process. 7. ACKNOWLEDGMENTS The authors would like to thank CPT Jim Nagel, Diane Mitchell, Linda Fatkin, Jock Grynovicki and Debbie Rice for their assistance with the statistical analysis. 8. R E F E R E N C E S
1. Bias, R., (1991) Walkthroughs: Efficient Collaborative Testing IEEE Software, 8, 5, 1991, pp. 94-95. 2. Jeffries, R., Miller, J.R.,Wharton, C., and Uyeda, K.M., (1991). User interface evaluation in the real world: a comparison of four techniques. In Proc. of ACM CHI'91 Conference on H u m a n Factors in Computing Systems, pp. 119-124, ACM, New York. 3. Karat, C., Campbell, R., and Fiegel, T. (1992) Comparison of empirical testing and walkthrough methods in user interface evaluation. In Proc of ACM CHI'92 Conference on H u m a n Factors in Computing Systems pp. 397-404, ACM, New York. 4. Nielsen, J. and Molich, R. (1990) Heuristic evaluation of user interfaces. In Proc. of ACM CHI'90 Conference on Human Factors in Computing Systems, pp. 249-256, ACM, New York. 5. Virzi, R.A., Sorce, J.F., and Herbert, L.B. (1993) A Comparison of Three Usability Evaluation Methods: Heuristic, Think-Aloud, and Performance Testing. In Proc. of Human Factors and Ergonomics Society 37th Annual Meeting - 1993, pp. 309- 313.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
399
Usability and Quality Control of H u m a n - M a c h i n e Interaction Elena A. Averbukh Laboratory for Man-Machine Systems (IMAT-MMS) University of Kassel (GhK), D-34109 Kassel, Germany
EXTENDED ABSTRACT
(Technical paper, verbal presentation) High quality of Human-Computer Interaction (HCI) in terms of usability, users' acceptance and job satisfaction becomes more and more critical along with the increasing complexity of automation, requirements to its safety, quality, ecological friendliness and technology transfer(Johannsen, 1994, Shneiderman, 1992). This in its turn demands more sophisticated and systematic approaches to the quality control of HCI during all phases of systems' interfaces life cycle, i.e., from conceptual design to the development, evaluation, operation and possible redesign/reuse (Averbukh and Johannsen, 1994, Katai et al, 1991). This paper presents an integrated approach to the quality control of HCI and focuses mainly on the problems of so-called "in-process inspection" of the quality of human behaviour during interaction with the computer. From the integrative point of view both interface development tools and interface software systems themselves are persistent subjects of quality control and adaptation, as it is schematically shown in Fig. 1. Modem Human-Machine Interfaces (HMI) are designed as distributed knowledge-based systems which contain •
knowledge about the users, i.e. User Model,
•
knowledge about application domain, that is, e.g., in industrial control applications Technical System (TES) Model,
•
knowledge about their interaction (Averbukh et al, 1994). Different strategies for increasing the usability of interfaces by both on-line and off-line
adaptation and further management of these knowledge structures which consider specific users' needs and expectations in concrete task situations (Averbukh, 1994). The functionalities embedded into the HMI which support this adaptation are also depicted in Fig. 1. Several advanced architectural design paradigms for effective implementation of such "inprocess inspection" functionalities are discussed. For this purpose, the appropriate criteria of
400 their effectiveness are formulated. The multidimensionality and the dominant role of the User Modelling functionality in the frame of usability and quality control is analysed and discussed. Concrete options and interface quality control strategies based on User and Situation Modelling are specified. The application examples are given for supervisory control of technical systems, particularly for the chemical industry.
REFERENCES
E.A.Averbukh: Task-Orientation of Human-Machine Interaction in Distributed Systems, Proceedings of 3rd IEEE International Workshop on Robot and Human Communication, Nagoya, Japan, July 18-20, 1994. G. Johannsen: Design of Intelligent Human-Machine Interface, Proceedings of 3rd IEEE International Workshop on Robot and Human Communication, Nagoya, Japan, July 18-20, 1994. G. Johannsen and E.A. Averbukh: Human Interaction for Process Supervision Based on EndUser Knowledge and Participation, Proceedings IFAC/IFIP/IMACS Symposium on Artificial Intelligence in Real Time Control, Valencia, Spain, October 3-5, 1994. E.A. Averbukh, G. Johannsen, F.Mletzko et al: Toolkit Architectural Design, Issue 2, BRITE/EURAM, AMICA Project 6126, Internal Report D4-3, Laboratory for ManMachine Systems (IMAT-MMS), GhK- University of Kassel, Kassel, September 1994. O.Katai, H.Kawakami, T.Sawaragi and S.Iwai: A Knowledge Acquisition System for Conceptual Design Based on Functional and Rational Explanations of Designed Objects, from :Artificial Intelligence in Design, John Giro (Ed.), Oxford, Butterworth-Heinemann Ltd., 1991, pp. 281- 300. B. Shneiderman: Designing the User Interface. Strategies for Effective Human-Computer Interaction, Addison-Wesley Publishing Company, 1992, 573 pp.
The presented work is partly supported by the Commission of the European Union under the BRITE/EURAM II programme in Project 6126 (AMICA: Advanced Man-Machine Interfaces for Process Control Applications). The consortium parmers are: CISE (Italy), ENEL (Italy), FLS Automation A/S (Denmark), University of Kassel (Germany), Marconi Simulation and Training (Scotland).
HMI
-
life-cycle phases
Verification & Validation
HMI Components for Adaptation
401
Fig. 1 HMI Quality Control through "in-processinspection"of human behaviour
This Page Intentionally Left Blank
III. 14 Cognitive Engineering
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
Color Coordinate Supporting System with Navigating State of User's Mind Y a s u s h i Yagi, T o m o h i k o Y a g y u , Y o s h i h i k o H i s a m o r i & M a s a h i k o Y a c h i d a D e p a r t m e n t of S y s t e m s E n g i n e e r i n g Faculty of E n g i n e e r i n g Science Osaka University 1-3 M a c h i k a n e y a m a - c h o , T o y o n a k a , O s a k a 560, J A P A N P h o n e : 0 6 - 8 5 0 - 6 3 6 2 , Fax : 06-850-6341 E-mail • [email protected]
Abstract In this paper, we propose a retrieval system for detecting color coordination that matches with a current state of user's mind. First, the user selects a few keywords such as wild and vivid, and the system extracts the initial candidates of color coordination from design database which correspond to keywords. Next, these initial candidates are narrowed down to candidates adapting to the state of user's mind. Remained candidates usually suit user's color preference. Thus it is not easy for the user to select one from these candidates. Therefore, finally, by changing the balance of color coordination, the user can find the favorite design of the color coordination with great satisfaction.
1. Introduction There has been much work on a retrieval system which finds a desired design from the database[I-3]. In the retrieval process, the user usually selects a few keywords such as wild and vivid, and the system extracts candidates of design from the database which correspond to the selected keywords. However, the usually the visual impression of human beings may be different for different persons. Therefore, the system is not comfortable for user becasue the database does not adapt to the visual impression of each user. Kurita proposed the algorithm for learning personal visual impression on visual object. Their method was based on multivariate data analysis methods and could provide a model on visual impression of user from a small set of training examples [4]. However, the visual impression of user also varies according to such environmental factors as weather, season,, time and etc. For instance, red and blue images are usually felt warm and cool, respectively. However, blue image is felt not so cold in a warm room. Therefore, a retrieval system, which can adapt to the current state of user's mind and can select the most suitable design for the user, is required for human-machine interaction. In this paper, we propose a navigating method for detecting color coordination that can match with the current state of user's mind. First, the user selects a few keywords such as wild and vivid, and the system extracts the initial candidates of color coordination from design database which correspond to keywords. The system can learn the color preference of the user while the user select favorite candidates from these initial candidates. After selection, remained candidates usually suit user's color preference. Thus the user's mind is
405
406 torn among these candidates. Therefore, finally, by changing the balance of color coordination, the user can find the favorite design of the color coordination with great satisfaction: ~
/'--S--
System & Monitor
System User
Memory & Files . . . . . . . . . . . . . . . .
_-_-_-_-_
_ _
Extraction of design which 41__.__1 Personal [,q correspond with selected [~__Database __..~ keywords from database
Keywords
Generation of feature histogram of color preference
V etreival of initial candidates color coordination
Input "I like it" and "I don't like it"
I
--,
=1 Narrow dow candidates v j adaptive to thiuser
Modify Candidates as User Preference
Make Feature Histograms with User's lnput.~
l%,d,.
Histogram 2 )
Help the user to select the best candidate
~&inal Candidate is selected! Modify Database ) Fig. 1 Outline of Color Coordinate Supporting System
2. Color Coordinate Supporting System 2.1 Retrieval of Initial Candidates of Color Coordination Fig. 1 shows the outline of the color coordinate supporting system. First, the user selects a few keywords such as wild and vivid, and the system extracts designs of color coordination from the design database which correspond to selected keywords. The initial pasonal database does not adapt to the current color preference of each user. Therefore we make the feature histogram for representing color prefernce of the user. Then we retrieve the initial candidates of color coordination from the database. Detail of the feature
407 histogram is described in the section 2.3. Currently, the user select three keywords and the system extracts approximately 30 initial candidates from the database. 180 keywords and 992 color coordinated patterns are stored in the initial database. As shown in Fig.3, degree of user's preference against each color coordinated pattern is represented by a numerical score such as +100, and -20. A plus and a minus signs mean likeness and dislikeness, respectively. u~~¢~.:~2i~i~i~i~i~i~.:i~i~#::ii~i~ii~:ii~:~i~i~i::i.:i~i!iiii?i~i~ii!ii;i!ii;i~i i!i;)i;i;i~i~i~i!i;i~i~i~i~i~ii~i~ii~i~ii~i~i;~:i~i!i~i~ii~ii;i~i;i~~i;~!~i~!~i~i~ii~i=~i~i~i;~.i~:~;i~i~i!ii~i;i1 ii{i:-:i:i:~.i:~i ~
4-X
J
- ->'~ ~
"': ....
iii:i:~?
t.
~ -3--~
j
. . . . . :"" ":':':':":':':':':':" +:':':':':" ================================ ::::::::::: ::::::: ::::::::::::::::::::::::::::::::::::::::::::::::::::i:!:i:!:i:i:i:!i:.:.:.:.:.:.:.:.:.:.:.:.:.:.:.:.:.:.:.:.:.:~
9 ~3~
!!!!!ii!!i!i!ii!i!iii!iii!!i!!ii!!!!ii!i!!!iiii!ii~ii!~i~iiii!iiiii!i!iii!~!:i:i:!:!:i:i:!:i:i:!:i:i:i:!:i:i:i:!:i:!:!:3:!:!:!...:.~:. ....... ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
~iii::.:::-.:.~.!::iii!iii::i::i::i!:::::~iiii::iiiiiiiii::! ~:i!i ili!i~::~:~:~ii!::iiiiiiiiiiiiiii~:~:~:~ii i::ii i i i i !ili ::i:::::i~:~:~:::~iiii ::::ii::ii::!::ii::!ii~:~:~:~iill i ill i iil i i ::i ::ii~:~:~:~i:: i!iiii::i::::iil if::i ::ii::iii:::¢:~:~:~i::iii::iii!i::iii! i!ii!::~::~i~i~:~:~:~!~::::!::ii::::::iii::!iii::i~::!~i~::ii~:~:~:~:~i:?:!::::ii::!::::i~!~:~i::i~i::ii~ii~i~:~:~:~i~:i~::~!~i::ii ~:~
!i !i !i !i i iii i ililii!Iil ~
iii!iiiii!ii!iiii iiiiiiii!!i!i iii!i)ii iiiiiiiii}ii i iiiiiii i!i ii
.......................................................................................................................... - : _ - : - - - : ......................................................... ............... ~_________~
__
_..
Fig.2 An Example of Initial Candidates of Color Coordination IKeyWord 1IlKeyWord2 IKeyWord3]
DATA
........................................................................................ i~iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii!iiiiiiiiiiiiiiiiiiiiiii!iiiiiiii~iiiiiiiiiiiiiiiiiii
Active
BASE
+ 10
~rea,_ iiiiiiiiiiiiiiiii~'~iiiiiiiiiiiiiiiil
(Jacket) iiiii Area2 ~!~:~..~.-,~iiiiiiiii (shi.~ --!iiiiii ii!i? Area3~iii!ii!i (Trousers)!i~[i: ,iiiiiii!iiii! "rs~~iiiiii! I iiiiiiiiilrill..............
::iiiiii~!ii~
Calm
-20
Bold
+5
Mild
0
Fig.3 An Example of Men's Clothes Design
Fig.4 Representation of Color Preference
408
2.2 Representation of Color Coordinate Space As shown in Fig. 3, each pattern consists of three color areas such as a jacket, a shirt and a pair of trousers. Each area is represented by three parameters such as hue, saturation and lightness. The color coordinate space is represented by following 18 features. A color coordinated pattern in the database is corresponded to a point on this 18 dimensional color coordinate space each other. Area 1 (Jacket) : Hue, Saturation, Lightness Area 2 (Shirt) : Hue, Saturation, Lightness Area 3(Slacks) : Hue, Saturation, Lightness Defference of Hue between Area 1 and Area 2 Defference of Hue between Area 2 and Area 3 Defference of Hue between Area 3 and Area 1 Defference of Saturation between Area 1 and Area 2 Defference of Saturation between Area 2 and Area 3 Defference of Saturation between Area 3 and Area 1 Defference of Lightness between Area 1 and Area 2 Defference of Lightness between Area 2 and Area 3 Defference of Lightness between Area 3 and Area 1
2.3 Representing and Learning User's Color Preference This initial database corresponded to three keywords, does not always adapt to the current color preference of each user. Therefore, the system provides a method for learning the subjective color preference of the user. Generally, the subjective color preference of each user and the features of the color area such as hue and saturation are represented by a 1-dimensional result space and a high dimensional parameter space. The mapping relation between them is discontinuous and nonlinear. Therefore, it is difficult to model the color preference of human beings and estimate parameters of the model. We use the memory-based like model such as the table lookup with interpolation. The method can map from the result space to the parameter space directly. The user checks both desirable and undesirable designs from candidates. Then the score of the design with favorite ones is increased. Therfore, this method can learn the user's color preference effectively by using results of a sequence while the user narrows candidates of color coordination and finally decides the suitable one. On the othre hand, we consider an user has two types of color preference; a fundamental and a temporal ones. Temporal color preference varies according to such environmental factors as weather, season, time and etc. Using only an above method, disirable designs are not always mapped with selected keywords. Therefore, as shown in Fig.4, we make 18 feature histograms H*(x) for representing both color prefernce of the user. Against each feature, we accumulate the plus and the minus scores H(x) of color prefernce of every design respectively. Each feature histograms H*(x) is calculated by following equations.
1
H(x) = - ~ H(x) n .,=0
cr(H(x))=l ff ,-=0 H* (x) = 10 * H(x) - H(x) cr(n(x))
(1)
409 Using this 18 feature histograms H*(x), the system can generate initial candidates of color coordination with not only direct memory mapped ones but also similar ones.
m:-;:::::~ ...........~:':':::5::i:............................................ :..~!~!~i~z~!.i~.:~:.~!~~:*~i~!~.~i~!~.~!~!~.~.~!~i~.~!~!~!~B
.-:..ilia,, Ill i )i!ll ii:.iiiii/ i i ,...
" :~:~:E~E~::S q. =============================== :I~:~i~:E~: :~:: :::i~:~:l :"1 ":':: ::~:: :E:~:~EE:iE:::" :i'~$i':':~::~EE:~!:~:E:~:::::: : .... ::"i::E::: E:~,m::[
)i~iE" .~i~
)!:.i!
:::E)~i::E! ~I~!i~:
i~!~))~i~'): :i~!~i~....
:', i E::E::))~.~i~i~ :~!~i
"~i~))i~i~ii ~-::.~!~-:'.-
"~::Ei~> "~::
Ni))ii .... ~))i)i..... !l)i)!',i)'',,)~'....,i',)ii!i)ii))ii~i)L )i)~ Ni:~ ))) ))))))) )))))))~)))))))
~":':':::~"'Ii" !!ii ~))~!! ~))~i'" "[i:"::!..... )~~L:-:ii[.)))i i~~)i!~).... ~i~i!i~i .)'!!)~.~!~:::~"[~!i)~i!l:: i~: !i [~iii)..
Fig.5 An Example of Ten Candidates of Color Coordination at the Final Step
Fig.6 Final Step
2.4 Navigation to the Final Candidate of Color Coordination In the second step, these candidates are narrowed down to candidates adaptive to the user by an interaction to computer. The user checks both desirable and undesirable designs from candidates. Furthermore, if the user can not find a desirable design and want to change color coordination, the system shows several new designs which the system change a color pattern of area pointed by the user, then the user can chooses his favorite design. The score of the design with favorite ones is increased. The new choosed design is added in the personal database. Then, the candidates are sorted by these scores and narrowed down to ten as shown in Fig.5. In the final step, the system navigates the user to decide color coordination that matches with the current state of user's mind. To navigate the user, any two candidates in remaining ten candidates are shown to the user repeatedly (See Fig.6). However, it is often the case that most remaining candidates suit the user's preference. Thus it is not easy for the user to narrow the candidates of color coordination down to a single one. Therefore, by changing the balance of color coordinate, we make it possible for the user to find the design of color coordination with great satisfaction. Here, every remained candidate suit the user's preference and denote a stable condition for the user. When two candidate designs both satisfy the stable condition, it seems to be difficult for the user to differentiate between two. Therefore, we change the balance of color coordinate of one candidate to decrease likeness of the feature histogram of the color prefernce.
3. Experimental Results For evaluating the system, 15 subjects gave this system more than 13 trials. The persons were between 22 and 35 years of age. Fig.7 and 8 show results of the average time of retrieval and the average of feature histogram H(x), respectively. The average l,t is calculated by following equations.
H(x) /1 = Zx {x* p(x)}
P(x) = Z H(x) x
(2)
410 As the retrieval time is decreasing and the average of feature histogram becomes constant, it seems that the system learn the color preference of the user. In case of 11 subjects, the selectibity of design is increased by changing the balance of color coordinate. A number of this evaluated data is more than 200. Therefore we coonsider that the change of color coordinate is usuful for this kind of design system. Moon & Spencer proposed the principle of color harmony. In their principle, two areas in color harmony denote a stable condition. When two candidate designs both satisfy the stable condition, it seems to be difficult for the user to differentiate between two. By making unstable condition, we make it possible for the user to find the design of color coordination with great satisfaction. We analyized experimental data considering from Moon & Spencer's principle of color harmony. In case of the unstable condition, the design of color coordination can be easily selected with great satisfaction. This means that Moon & Spencer's principle of color harmony is effective rule to change the balance of color coordinate of one candidate. 400
it \
A V
-
" ; ~
TotalTimefor Retrieval RetrievalTimeat Final Stet r s t Step
300
~14 o~12 ~IO
200
=
8
6 100
4
Trial l
0
2
a
4
i
6
i
I
8
I 0
Fig.7 Time for Retrieval
I 2
<
2"
: .
o o
.
.
.
2 4
.
.
6
.
.
8
.
.
1
.
.
J
Triali |
.
.
.
.
1
1
1
1
0 2
4
6
8
Fig.8 Average of Feature Histogram
4. C o n c l u s i o n s In this paper, we proposed a retrieval system for detecting color coordination that matches with a current state of user's mind. By learning the color preference of the user, retrieval time decreased. The selectivity of the use was increased by changing the balance of color coordinate. As a result, both of rules are effective for finding the design of color coordination with great satisfaction. Moon & Spencer's principle of color harmony was effective role to change the balance of color coordinate of one candidate.
References [ 1] Y. Itoh and H. Nakatani, Image Retrieval by Natural Language, Proc. 8th Scandinavian Conf. Image Analysis, vol.2, pp. 1221 - 1229 (1993) [2] S. Senda, M. Minoh and K. Ikeda, Document Image Retrieval System Using Character Candidates Generated by Character Recognition Process, Proc. 2nd Int. Conf. Document Analysis and Recognition, pp.541-546 (1993) [3] T. Kurita and T Kato, Learning of Personal Visual Impression for Image Database Systems, Proc. 2nd Int. Conf. Document Analysis and Recognition, pp.547-552 (1993) [4] M. Oda An Image Retrieval System that uses Human Cognitive Properties, Technical Report of IEICE SP92-68, HC92-45 pp.55-62 (1992)
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
COMPARISON BETWEEN THREE HUMAN-INTERFACES INFORMATION SYSTEM
411
IN H O S P I T A L
Kotaro MINATO and Akira ENDOH Department of Medical Informatics, Kyoto University Hospital, Sakyo-ku, Kyoto, 606-01 JAPAN In this report, we describe a quantitative method to evaluate human-computer interaction (HCI) for the direct prescription order entry system in the hospitals. This method is based on a GOMS-like cognitive model for the interaction and the model is represented by a tree structure of five layers on goal-task hierarchies. Three different interfaces at university hospitals were compared by this method and the differences (similarity) among them were measured. 1. INTRODUCTION It has been over ten years since we started using the hospital information system (HIS) in Japan, where medical doctors typed their prescriptions into a computer terminal directly by themselves. However those data entry systems have wide variety of human-computer interfaces for different hospitals. The purpose of this study is to develop a quantitative method to compare the difference between these interfaces in the hospital information system. We will describe a tree model for each individual interaction based on the concept of the goal hierarchy and the "grain" in GOMS model. Then, we will define a similarity distance between these tree representations in order to measure the mutual difference between interfaces for the direct prescription order entry system. 2. METHOD
At first, we define an idea of "reproductive" human-computer interaction (HCI). A combination of a system, a task and a user determines the reproductive interaction. There are a lot of influential elements for human-computer interaction, such as the type of pointing device, the performance of computer, the application system, the variety of tasks on the system and the individual user characteristics. We focus our attention to the three elements that are the "system", the "task" and the "user". These three elements can define a unique reproductive HCI. The user here means the skilled and trained person who can achieve the task steadily and repeat the task exactly the same manner without slips and mistakes. Once the skilled user performs the same task on the same system, the unique sequence of interactive operations is reproduced repeatedly. An reproductive HCI can be represented by a tree structure based on the grain of analysis. The tree structure has five grains (layers) such as, the root, the unit task, the function, the argument and the key stroke level, respectively.
412 We can treat these interfaces abstractly as shapes of the tree representation. That is, if the shapes of tree are equal, we could regard these interfaces as the same HCI in spite of the type of operators in the key stroke level or the semantic difference among tasks, systems, and users. Therefore, once the tree representation of reproductive HCI is given, we can compare the difference of these interactions by using the difference of the shape of tree. In order to measure the similarity of reproductive HCI quantitatively, we introduce the concept of distance between trees. At first, a quadruplet is defined as four values, which are the four average numbers of branches per a node for the unit task, the function, the argument and the key stroke level layer respectively. The similarity distance is defined by Euclid metric in the four dimensional space constructed by those quadruplets of coordinate values. We call the distance as the HCID (Human-Computer Interaction Distance). As the distance becomes smaller, these two HCIs look alike more similar. Using the HCID, we are able to compare the difference of any two HCIs quantitatively. 3. THE TREE REPRESENTATION In order to obtain a tree representation, we start to record the interaction by a video camera and make the timing chart. From the chart, we construct the key stroke level model, which is the sequence of physically measurable operations such as keystroking, pointing, homing and mentally preparation time (mental operator). Then, we group up the smallest set of key stroke level nodes separated by the mental operator as an argument level node. A meaningful group of argument level nodes makes a function level node. A set of function level nodes that the user can carry out together at a stroke is defined as a unit task level node. The whole task (the root node) is represented by the sequence of these unit tasks. This hierarchical relation is expressed by the tree model. For an example, Figure 1 shows a part of tree representation of a direct order entry interface, where the doctor types a prescription into a terminal in his office. In the figure, the unit task of a recipe prescription (RP) is subdivided into three branches that are entering the drug name, the dosage and the usage in the function level. The dosage function is divided into two branches of the argument level such as entering "2 pills" and pushing down "accept" button to confirm. In the keystroke level, the former node is composed by two operations that are moving the hand from the mouse to the keyboard [H] and typing down the "2" key [K]. Root
Unit T a s k
~ Pre-prossicing Prescription entry / / R~i~l~...~.. (N
p~,l
lpUU~t~
Eunction
Argument
Key Stroke
Drug name ~
Confirming of the dosage on screen 2 pills---------------M°ving to keyboard -~'--"--"-"-~yping "2"
D°sage~ N ~ Usage
~ Acceptance ~
Confirming the "2 pills" Movingto mouse Pointing Clicking 'accept' button Comfirming
M H 1K M H P 1K M
Figure 1. An example of the tree structure for a part of actual direct prescription order entry.
413 The latter node is separated three operations, which are moving the hand back to the mouse [H], the pointing the "accept" button with cursor [P] and clicking the mouse [K]. 4. EXPERIMENTAL STUDY Applying the above method, we compared the actual difference of HCIs for a same task of prescription entry in three independent systems, which were the HIS of Kyoto university hospital (KUH), Shiga medical school hospital (SMSH) and Yamaguchi university hospital (YUH) respectively. In the system of KUH, the multi-windows and mouse control was used for the user interface (GUI/WlMP: windows, icons, menus and pointers) on the OS/2 terminal. The SMSH system introduced the interface using cursor operations on MS-DOS (menu driven without windows), and the YUH system used a command line form-filling interface. In the experiment, the typical prescription (Table 1) for a patient of chronic hypertension was used as the experimental "task". Two types of user were prepared as the skilled user. One was an exemplary user who followed the operational manual and divided the prescription into two RPs depend on the difference of the usage. The other was a somewhat eccentric user who divided it into three separate RPs. Then we analyzed four systems, which were the three systems mentioned above and the short cut key version of the KUH system. Then, tree representations were constructed for five combinations within the elements. Table 1 Experimental task: Prescription Drug name
Nitorol R
Adalat
Kolantyl
Dosage Usage
2 pills 2/day, (BID)
3 pills 3/day, (TID)
3g 3/day, (TID)
(The start date and the term to give medicine (one week) were the default values) 5. RESULT and DISCUSSION
The parameters of the five tree representations are shown in Table 2 and three examples of the tree shape are shown in Fig. 2. Table 2 shows the number of branches of each layer for the five experiments including the short cut key version with the exemplary user and the eccentric user with the mouse version in the KUH system. In any combinations, the final prescriptions are the same substantially. The total number of operations such as keystroking and homing are 94 in the KUH, 54 in the SMSH and 47 in the YUH system, thus the number of operations of the KUH is as much as twice of the YUH system. Table 3 shows the mutual HCI distance (HCID) among five different tree representations. Applying the proposed method, the distance between KUH and SMSH is HCID=I.02, while YUH-KUH is 3.24 and YUH-SMSH is 3.33. The distances between YUH and other systems are grater than the distance between KUH and SMSH. It suggests that the YUH system would be quite different from other systems. Because the YUH system was developed early in the HIS history than others, it was designed for combining three functions such as the drug name, the dosage and the usage into the one integrated function in order to decrease the number of keystroking as small as possible.
414
c c.~.~
c.d.ml
~d
M
c..~ ,,.-t,.,Jm d,q,,,t ~ . ~ I c l ~ .l,lrro. c I ~ .,,¢,,a.
|K
c ~
~,il(m,,~
cmm'...,~
,',-s
,.,."
c . , ~ . , , ~ "~'ro"
.~--.
~ ' '
C a d ~
1 p,lb
U~Ip col,q
e ~ . l , ~ ,.-7 ~ , , a , ~
c ~
,.....,w.t
u.p
M ~ e q ~ o ~
c . , , ~ , , ~ ~.,.~ R ~ " "
C ~ ' ~ " tit
C~..I, c~,~ms.,---Oq,~*-.l~ T~*.?
•
.,,.~
c~ .,,,~,,.. ~,at. t,,xq,li..2
"
C.,~.,~S '*'- - . ~ , ( ~ t . , i ) Cwh,mC ~ h . ~ . . . p ~
'
C * ~ m i ~ "-- " A ~ " C ~ , ~ I ",,,M,,e
M IMK
c~.,.~.
.K
~,o.. • ~,a.qv ..
c ~ ~- ~0~,~ d~,,t ~..~ c~a~l-r c ~
~.-~
~'mm"
c ~
u . ~ a . , , t =t m'l
c -~~' r. .-o~~. ~" - ~ = v ~
~
ead~c,~
c.,~ll,
n.l.lr~
~... c~.~,~,o~
C~-ml~-
,.l.a ~ .l~.~t.
~
/
I~ M
c~
, .
~
,
~ , ~
c ~ c~,r
la.
~ ~
d .~
M
~
lip
I>q
M*.,.*,IIa.,M. V o ,.....ae
u~nr.I, t,.~m k~ c * d , , . n l .t,e ~ q
u
c~,.I,,..
~ , . ~ ~t ,q,~ m p
M
C~,,al"-
~t,,--,
M
r," ..-. oa,U m•(
P'.J.a,~ crldms,,,t0t,~ dl,~ X:)m,,N~
~,~i
or
,,... ~
•
M
~ .,,me ~, • ~,~.-- ~,~,~.
m
~ m,=..Imn.~ =.,.m,,= Mo,,ml ~ ,~,~=aml.llr
cur~i,,l~' I~m,,,m,
tK " "
,~.~
•
-n. c~ ' ,~,.a'..~ Mo,,l,~ ~ I,.9~.=d Tg~..,a'UCOa~'~"
~
M x
~mm |K
(b) Shiga university hospital
w.."
: ~---__
U*,~t*m. ~ " "
• i. ct~ma . . - ~ . , , " .
-
-
.,=m,..
T~7 M . , ~ S t. ,,.,,,.
c~,,..~
•
IK .c
,~..~
.,..,,..
,,,. ,.-*--, ~ r l m . p =
"
c~d,,~ .,,,q c..a, m s ~ . m . , . l c.,~ml'~-'.'-"-~--"
u . , ~ l ~. Cl,d ~ l " - - " I~/ c~ '
~,-.,-.-t..,m~
c.~.=~
- , - ,-,,d ~=~ x , . . .
•
H
.,=.~,.
~om * m 'l
,.~,~r c ~ , ~ , , ~ , , - -.., ~t ~ , . ~ , ~
c ~ "-mp--.'.,.~,~ V*a.~*¢.,ml ra. ~.r~,, • Mm,,l*l ~ "t'P k,../
/
.
/
(a) Yamaguchi university hospital (c) Kyoto university hospital (mouse) Figure 2. Examples of the tree representation for the prescription order entry interface
415
T a b l e 2 T h e n u m b e r o f b r a n c h e s for five tree r e p r e s e n t a t i o n s Level Total branch Averal~e Tree structure ( Number of branches for each level )
Kyoto (short cut) Kyoto (Eccent. usr) iYama~chi Shi~a Kyoto (mouse) UT F A KS UT F A KS UT F A KS LIT F A KS UT F A KS 5 17 43 100 4 15 40 56 4 15 40 94 4 15 25 54 4 7 10 47 4.00 1.75 1.43 4.70 4.'00 3.75 1.67 2.16 4.00 3.75 2.67 2.35 4.00 3.75 2.67 1.40 5.00 3.40 2.53 2.33 2 2 1 3 1 2 2 51 1
6
1
2
2 2
6 5
1 1
6 5
5 2 5
2 2 5
3 6 2
1
3
2 5 2
2 2 2
1
3
2 5 2
2 2 2
1
2
3 3
2 2 2 2 4 3 2 2 2 3 2 2 2 2 5 3 2 2 2 3 2 2 2 2 2 2 2
UT: Unit Task Level F: Function Level A: Argument Level KS: Keystroke Level
Table 3 Similarity distance ( H C I D ) b e t w e e n five interfaces a: Yamaguchi a: Yamaguchi system b: Shiga system c: Kyoto system (mouse version) d: Kyoto system (short cut key version) e: Kyoto system (Eccentric user)
b: Shiga 3.24
c: Kyoto (mouse) 3.33 1.02
d: Kyoto (key) 4.05 1.22 0.95
e: Kyoto (eccentric user) 3.25 1.38 1.07 1.42
416 Using the short cut key version in the KUH system, the total number of operations is 56. This is about half of the mouse version and almost the same as the SMSH system. While using the HCI distance, the distance between two versions of KUH system is HCID=0.95 and it is shorter than the distance between the mouse version and the SMSH system (HCID=I.02). That is, two brother interfaces based on the same mother system are similar, even if the difference of the number of operations is very large. This result shows that both of the way of dividing a unit task and the way of making a sequence of functions have rather larger effect on the similarity distance than the number of operations alone. The distance between the exemplary and the eccentric user in the KUH system is HCID= 1.07. It suggests that the difference between users could have lager influence than the difference of systems in some cases. 6. CONCLUSION In this report, we proposed the concept of the reproductive human-computer interaction as the composition of three elements, which:were the system, the task and the skilled user, for the direct prescription entry system. The reproductive HCI had a tree representation with five layers. We defined the similarity distance between those reproductive HCIs by introducing Euclid metric for the four dimensional feature space characterized with the four coordinate values, which were the average numbers of branches per a node for each layer of the tree. We made experimental studies for five reproductive interfaces of the direct prescription order entry in three different university hospitals and measured the mutual differences among them. The results showed that the similarity distance agreed with the user's impression. The proposed method could not specify the degree of "usability" or "good and bad" to a HCI directly. However, it might give a reasonable similarity distance between any two HCIs from the compromising point of view among the system, the skilled user and the task. This method could offer a common platform for designing and standardizing the human interface of hospital information system and for discussing the possibility of replacing old systems. REFERENCES
1. S.K.Card and T.P.Moran, The Keystroke-Level Model for User Performance Time with Interactive Systems, Comm. ACM vol.23 No.7:396-410 (1980) 2. S.K.Card, et al, The Psychology of Human-Computer Interaction, Lawrence Erlbaum Associates (1983) 3. A.Endoh, K.Minato, et al, Quantitative Comparison of Human-Computer Interaction for Prescription Order Entry Systems (by Japanese), Japan J. of Medical Informatics, Vol. 14 No.2:45-55 (1994)
ACKNOWLEDGMENT The authors wish to thank Dr. M.Komori, Dr. Y.Inoue, Dr. S.Nagata and Prof. T.Takahashi for their help and advice with this work.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
417
Explaining Plant Design Knowledge Through Means-End Modelling Pertti Huuskonen, Kari Kaarela VTT Electronics, P.O.Box 1100, FIN-90571 Oulu, Finland Tel. +358-81-5512111, Fax. +358-81-5512320 Email [email protected] ABSTRACT We apply the multilevel means-end model of Rasmussen to explain the intentions behind design choices of industrial systems. We have extended a design tool, the Design++ environment, to capture higher level knowledge about the artefact under design and to explain this knowledge to the users. Means-end knowledge is encoded through relations added on top of the existing part-of and subclass hierarchies in the tool. Explanations are encoded as structured text in the objects and design rules. We have tested the explanations with a power plant application. Although still very primitive, this research prototype has confirmed that meansend information can be effectively used both to capture and explain design knowledge. We adopt a simplified view to design processes and decisions, placing our main emphasis in artefact modelling and explanation techniques. 1. INTRODUCTION Industrial plant design is a complex activity. The designers have to understand a lot of design information provided by other design disciplines to be able to successfully extend the designs. A major part of the designer's work consists of reusing designs. Unfortunately design documents represent only the end result, not the underlying intentions. This implies reverse engineering the purposes behind designs. Ways to clarify this teleological knowledge are needed.
1.1 Capturing design knowledge The purpose of an item is well documented only in the case of exceptional design choices, where the designers want to emphasise the deviation from standard practice. Otherwise, the purpose of the item is expected to be understood via experience. This may not be the case: memorised knowledge deteriorates with time, and seemingly similar solutions may be fundamentally different. Even between experienced designers there may be misunderstandings due to their different backgrounds. Moreover, the end users could interpret the purpose incorrectly, which may lead to non-optimal performance at the plant. We believe that some of this design knowledge could be captured with the design tools. The tools should include clear models of the design knowledge and be able to communicate this knowledge to the end users. Even if the tools do not fully support modelling, they should support knowledge acquisition from the designers. In addition to enhancing the communication between designers, operators would be helped as well. 1.2 Related work In this paper, we concentrate to capturing and explaining knowledge in design tools, somewhat overlooking the user's end of the problem. We adopt a simplified view to design proc-
418 esses and decisions, placing our main J Sub~oal ~ S u ~ o a l I "~ =realize/ emphasis in artefact modelling and ~' ~" ~ / achieve explanation techniques. Other reI /,~°" ~ / / ~ S~bqoa~l j "~ SubQo~ ~oall = suopan searchers have studied extensively the ~ ~u~o~, topic of modelling design t ~-~Sbbfun~.tionl [Chandrasekaran 93, Gruber 92, Ke~.Subtunetion~~~~_~ uneke 91, Klein 93, Lee 91, Stephar F. ion ~-------~Subfunct r~ io..n]~,^~.------~Subtu~'~lonl"l~bfuncti°nl~" nopoulos 90]. We share their views, ~ ~ , y | ....... r--.~.~Subfundliorl]| adding ideas from means-end modelEna, \ / I Suboa~ ] \ ~ Subpart/ I ling [Kaarela 93a, 93b, 94, Larsson 92, / ~ • Part ~ Subo~rt] . Lind 90, Rasmussen 86, Sassen 93] [Device ~ Part ~ .qnhn~£~ and explanation research [Franke 91, ~ ,//{Subpart I " "'q:=~ Part r'-------.t Subpart I Huuskonen 92, Swartout 91]. Our main contribution has been in formuWhole ~ ~. ~ Part lating an algorithm for deriving explanations and in embedding required Figure1: Dependencies between levels capture means-ends representations in a design tool. Here information we target only two aspects of design knowledge: purposes and justifications. 2. EXPLANATION THROUGH MEANS-END MODELS 2.1 Means-end modelling
Means-end modelling, pioneered by Rasmussen, is a sound way to structure knowledge about a plant [Rasmussen 86]. It makes the function and goal levels of the systems explicit (Figure 1). The means-ends dimension is divided into levels of abstraction, with more abstract concepts (goals) at the top and the very concrete concepts (devices) at the bottom. Each device is given a purpose through the functions it performs. In turn, the purpose of these functions is to achieve a set of goals. Trough these many-to-many relations between the model items one can explain the purpose behind a design choice [Rasmussen 86]. To see why an item was included in the representation, the mapping to the next higher level has to be investigated. This gives the purpose for this item. Correspondingly, to find out how an item is implemented, the relation to the next lower level has to be examined. [Rasmussen 86] 2.2 Explanation as means-end navigation
We view explanation as navigation in the means-end model. Explanations can be found either explicitly, as structured text in the item under consideration, or indirectly, through relations to other items. We define two ways of explaining designs: explicit explanations, referring to general design knowledge, and derived explanations, referring to related design choices. This is a straightforward approach to modelling design knowledge that does not allow for any deeper automatic understanding of the design, but will suffice for the purpose of explaining the knowledge. Explicit explanations are structured text that can be attached to objects, their attributes, or to design rules. They contain a number of predefined keywords giving rationale for individual items, and freeform text for less formal definitions. Possible criteria that the designer might offer if asked to justify a design choice include: authority, textbooks, similar case, standards, law, safety regulations, cost, tradition, and personal preferences. These criteria are represented
419 as keywords. This semiformal approach allows for both formal and informal knowledge to PREssuRE-25~) - - ~t h;~i~e,-'~ be captured. Realizes:I J I Part-of: Part-of: /[~1 ........ A justification can be derived by traversing the network / Pump_system / formed by the means-end relaType: assembly / Pump.1 tions. Indirect explanations are Purpose:/ ~ I Type'."device derived from other items I Realizes: ~ , Purpose'r ~ Realizes:through relations (Figure 2). tgj\ I Part-of: [~] Explicit justifications \' // An item may possess explicit I~ Derived via means-ends-relations justifications of design knowl[~1 Derived via structural relations edge, in which case the justifications are simply shown to Figure 2: Explanations derived through relations the user. If an item does not have an explicit purpose, its neighbouring objects are searched recursively until an explicit purpose is found. If means-end links ('realises' relation in the figure) are not defined for an item, the purposes can be derived through structural decomposition (part/subpart relations). The search continues until an explicit justification is found. Means-end relations are given the priority and, if they are not explicitly defined, an implicit relation is derived through part-of hierarchies. The number of potential relations is decreased, since several items can inherit a common purpose from their superiors. ..Raise-pressure , /ype •• goal Purpose:(INCREASE [~1
I Fw-pressure-elevation Type • function
~.nc.t
3. RESULTS The proposed ideas have been embedded in a research prototype. It is built as an extension to the Design++ tool, increasing its representative capabilities with features supporting meansend modelling and justifications. 3.1 Research prototype The prototype can answer some specific types of questions about purposes and justifications, as well as export the knowledge as C++ and Prolog structures for future embedding in automation systems. Allowed questions currently include "What is the purpose of this item?", "What is the purpose of this attribute?", "How is this item implemented?", and "How are the values for this attribute obtained?". We have tested the explanations with a power plant application (Figure 3). The research prototype, although very primitive, has shown that means-end knowledge can effectively explain design information. Currently, the user interface of the prototype is quite limited. It uses the facilities available in KEE and Design++: windows, mouse and menus. The queries for justification can be made either via object menus or Lisp commands. The justifications are displayed as narrative text. The means-end relations are drafted in AutoCad drawings or directly in the object hierarchy [Kaarela 94].
3.2 Design++ tool Design++ is a general engineering tool marketed by Design Power Inc. It is suited for maintenance of product and design knowledge [Katajam~¢i 91 ]. The tool encodes knowledge about the plant under design in KEE and Oracle objects, organised in class hierarchies and part-of
420
proj ect#
11bra~e=
ledel=
ut tL~ties
Component FEEBWATER_PUMP $627 in Model HI=W31 i[ H Class: FEEl)WATER_PUMP in Library HFW3 -ii Assembly: FEEDWATER_PUMP_SYSTEM Parts: AD3USTING_DEVICE ENGINE_DEVICE INSTRUMERr PURPOSE Comment: Vhy is this thing necessary in the process? Oefau]t: <(not specified>> Design Rule: <<not specified>> Valuec]ass: <<not specified>> V a l u e : <<not specified>>
iii til ~- " i. i i riri[' iin,',i i i ~i[l' t.,i~!i ~ii=!i ~'iLE /hp_desuperheat i n g _ s p r a y _ s y s t e m - -
storage_deaeration--...
,piping_system--... ~llfeeduater system /pump.1 ... ~iilli ~ p L ~ p _ s y s t e m ~ - - - ~ - p u r n p "2~ ~'.il ~ ~-~discharge_line.1 ~.discharge_]ine.2 h _feeduater_heatin s s t e m - - . . .
~un~tmsg "feeduater_Pump.s627 'shou-purpose!) No e x p l i c i t purpose for Part/Feeduater_pump.s627, but i t is part o f : Assemb]y/Feeduater_pump_system.s491. No e x p l i c i t purpose for Assembly/Feeduater_pump_system.s491, but i t has links to upper levels: Function/Fu-pressure-e]evation. No e x p l i c i t purpose for Function/Fu-pressure-elevation, but i t has links to upper levels: Goal/Produce-flou Goa]/Raise-pressure. The purpose of Goal/Produce-flou is: (Produce a uater flou of 125 kg/s). The purpose of Goal/Raise-pressure is: (Raise pressure from 7 to 245 bars). NIL
>|
Figure 3: The research prototype built on top of Design++. The windows at top right and centre show the meansend and part-of hierarchies of a feedwater system. The bottom window shows the trace of a generated explanation.
hierarchies. Each attribute of an object may have an associated design rule written in Lisp, that can be used to infer the value of the attribute. Figure 4 shows a rule that calculates the hydraulic power of the feedwater pump. The rules may retrieve knowledge from other objects' attributes. As a test bed, we used the feedwater system of a peat-fired power plant, modelled by Tampella Power, a Finnish boiler manufacturer. We extended these models to capture means-end information about the plant and to explicitly record justifications of design decisions. 3.3 Justifiable Objects We have extended the Design++ tool with the notion of justifiable objects. They are objects, classes, or instances that can reside at any level of abstraction or decomposition, and can offer justifications of themselves. This extends authors' previous work with explainable objects [Huuskonen 92]. Explicit justifications are encoded in Lisp keyword lists PURPOSE and RATIONALE. The lists may also contain free form English explanations. They can reside in an object's attributes, in an attribute's facets, or in a Lisp rule's code. The lists are generic structures in the sense that their representation is the same regardless of the type of the item they refer to. However, their meaning changes depending on the place, referring to either the object, the attribute, or to the rule. 4. DISCUSSION The prototype, built on top of a design tool, helps to capture knowledge from the design and to structure the knowledge in a meaningful way. The suggested representations capture semantic knowledge of the domain and the designed artefact.
4.1 Improvements to design The suggested ideas raise the abstraction level of the design documentation, formalise work habits, and force the designer to think about the reasons for design decisions. These develop-
421 ments should bring increased quality, reliability, and safety in the use of the plant. Also, communication among designers would be aided. However, design knowledge may sometimes be very fuzzy, hard to model with the framework described here, and therefore hard to justify. The designers or the companies may be unwilling to justify their proprietary designs. The extra work imposed by modelling will not be easily accepted by the designers, even if the benefits would be clear. Our approach would save some of the work by propagating the purposes through relations, though some manual knowledge entry is still necessary. Moreover, different power plants are rather similar. Most designs are reused at succeeding plants. This suggests that the justifying work could be reused as well. We chose the semiformal approach to modelling design knowledge because of its simplicity, especially since our tool did not support modelling the design process itself. Our main aim was to test the explanatory capabilities of FEEl)WATER_PUMP.S627 in Model HFWS1 the means-end model. More elaborate de- Component Class: FEEDWATER_PUNP in Library HFWS Assembly: FEEDWATER_PUMP_SYSTEM sign representations were not seen essential Parts: ADJUSTING_DEVICE ENGINE_DEVICE INSTRUMENT for this purpose. HYDRAULIC_POWER Hydraulic pouer efficiency [kY]. Note that the prototype has only been Comment: Default: <<not specified>> Design Rule: embedded into a design tool. It demon- (! SELF HYDRAULIC_POWER (DOCS strates supporting designer's work through (RATIONALE (TEXT explanations. In the future, similar explana"The hydraulic pouer of the pump is the pressure differ ence over the pump divided by the operation flou rate volume.") tory facilities should be included in operator (TEXTBOOK "Fundamentals of hydraulic engineering, p.999") support systems as well. (PURPOSE (TEXT "To calculate the hydraulic power of the pum 4.2 Efficiency issues
."))) (, (/ (:? MY PRESSURE_DIFFERENCE) (:? MY OPERATION_FLO~_RATE_VDLUME_NORMAL)) 1888)) Valueclass: (NUMBER)
Initial analysis suggests that even though means-end models form networks with a V a l u e : <<not specified>> large number of links, they do not pose any major computational problems. The physiFigure 4: Explicit explanations in a design rule cal level of the feedwater system model consists of some 1500 components, with numerous potential many-to-many relations. Fortunately, the complexity is reduced by the relations shared through the part/subpart- or class/subclass relations. Several subparts' purpose is defined by a single superpart. The number of items decreases on the upper levels. There are some 100 items of the functional level, and about 25 items on the goal level. For justification, the search space narrows upwards, avoiding complexity problems. Furthermore, the system can be made to act in a consultative mode. The user can interactively guide the search to interesting parts of the system, helping the complexity problem. 4.3 Future work Ways to manage several altemative paths through the models need more study. Further work is also needed in reduction of explanatory detail and in customisation to individual users. Now the prototype is built inside the design tool, but in the future the explanatory mechanisms will be embedded into automation systems. The prototype needs better interfaces for the design of the means-ends hierarchy. Improved graphical browsers and relational analysers would be needed before the tool could be introduced to the designers. More elaborate design representations would have to be included in further development.
422 ACKNOWLEDGEMENTS This research has been funded mainly by the Technology Development Centre (TEKES), Finland. Financial support has also been provided by the Technical Research Centre of Finland (VTT), the University of Oulu, Imatran Voima Oy, Valmet Automation Inc., and Tampella Power Inc. We would like to thank our industrial partners for their support and feedback. Jaakko Oksanen of VTT has contributed substantially to the implementation of the features supporting means-end modelling in the design tools. REFERENCES [Chandrasekaran 93] Chandrasekaran, B. et al: "Functional Representation as Design Rationale" IEEE Computer, Vol. 26, No. 1, 1993, pp. 48-56. [Franke 91] Franke, D.W.: "Deriving and Using Descriptions of Purpose", IEEE Expert, April 1991, pp. 4147. [Gruber 92] Gruber, T., Russel, D.M.: "Beyond the Record and Replay Paradigm for Design Rationale Support", in Working Notes of AAAI'92 Workshop on Design Rationale Capture and Reuse, San Jose, July 15, 1992, pp. 111-118. [Huuskonen 92] Huuskonen, P., Korteniemi, A.: "Explanation Based on Contexts", Proc. 8th IEEE Conference on Artifical Intelligence for Applications, Monterey, Califomia, March 2-6, 1992, pp. 179 185. [Kaarela 93a] Kaarela, K., Huuskonen, P., LeiviskiS, K., "The Role of Design Knowledge in Industrial Plant Projects", Proceedings of the International Conference on Cognitive and Computer Sciences for Organizations, Montreal, Canada, May 4-7, 1993, pp. 173-183. [Kaarela 93b] Kaarela, K., Huuskonen, P., Jaako, J.: "Providing Plant Design Knowledge to the Operators" Proc. Fifth Intl. Conf. on Human-Computer Interaction, August 8-13, 1993, Orlando, FL, USA, Vol. 1, pp. 546-551. [Kaarela 94] Kaarela, K., Oksanen, J., "Structuring and recording plant design knowledge", Proceedings of the IFIP international conference on Feature Modelling and Recognition in Advanced CAD/CAM Systems, Valenciennes, France, May 24-26, 1994, Vol. 2, pp. 853-866. [ K a t a j ~ 91] K a t a j ~ , M., "Knowledge-Based CAD" Expert Systems with Applications, Vol. 3, 1991, pp. 277-287. [Ketmeke 91] Keuneke, A.: "Device Representation, The Significance of Functional Knowledge", IEEE Expert, April 1991, pp. 22-25. [Klein 93] Klein, M.: "Capturing Design Rationale on Concurrent Engineering Teams", IEEE Computer January 1993, pp. 39-47. [Larsson 92] Larsson, J.-E.: "Knowledge-based Methods for Control Systems", Doctoral thesis, Lund Institute of Technology, Lund, Sweden, 1992, 236 p. [Lee 91] Lee, J., Lai, K.-Y.: "What's in Design Rationale?", in: Human-Computer Interaction, Vol. 6, No. 3&4, 1991, pp. 251-280. [Lind 90] Lind, M.: "Representing Goals and Functions of Complex Systems: An Introduction to Multilevel Flow Modelling", Institute of Automatic Control Systems report No. 90-D-381, Technical University of Denmark, November 1990. [Rasmussen 86] Rasmussen, J.: "Information Processing and Human-Machine Interaction", North-Holland, Amsterdam, 1986, 215 pages. [Sassen 93] Sassen, J.A.M.: "Design Issues of Human Operator Support Systems", Doctoral thesis, Delft University of Technology, Delft, The Netherlands, 1993, 226 p. [Stephanopoulos 90] Stephanopoulos, G.: "ArtificialIntelligence in Process Engineering -- Current State and Future Trends", Computers in Chemical Engineering, Vol. 14, No.11, 1990, pp. 1259-1270. [Swartout 91] Swartout, W.R., Paris, C.L., Moore, J.: "Design for Explainable Expert Systems", ~EE Expert, Vol. 6, No. 3, 1991, pp. 58-64.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
423
M e t h o d of E c o l o g i c a l I n t e r f a c e D e s i g n A p p l i e d to I n t e r a c t i v e Diagnosis Support System Yoko Asano, Shun-ichi Yonemura, Hiroshi Hamada, and Katsuhiko Ogawa Human Interface Laboratories, NTT, 1-2356 Take, Yokosuka-shi, Kanagawa, 238-03 JAPAN This paper proposes a procedure for designing ecological interfaces suitable for interactive diagnosis support systems based on the perspectives of work analysis and interface design proposed by Rasmussen [1, 2]. Several of the perspectives proposed by Rasmussen are chosen and concrete transformation paths are created for them. The effectiveness of the proposed procedure is confirmed by designing and testing a support system for interactive diagnosis.
1. INTRODUCTION Human recognition of work situations is necessary for ensuring flexible work. Consequently, interfaces that support human cognitive processing in a given work situation are needed. Instead of giving advice as to what should be done next in any situation, it is more effective to represent the whole work situation by providing alternatives for the following task and the information necessary for cognitive processing, that is, to design an ecological interface. An ecological interface is an interface through which people can directly perceive the meaning and value of the information available. This paper investigates a method of ecological interface design adapted to interactive diagnosis support system based on the perspectives proposed by Rasmussen [1, 2]. First, perspectives of work analysis and the design process are introduced to the design ecological interface proposed by Rasmussen. Next, a procedure for designing interactive diagnosis support systems is proposed. Dialogues of interactive diagnoses are analyzed and an interactive diagnosis support system is designed based on the procedure. An experiment using the designed system is conducted to evaluate the effectiveness of the design procedure.
2. PERSPECTIVES OF ECOLOGICAL INTERFACE DESIGN Rasmussen proposed several different perspectives of work analysis and design concept to design ecological information systems. Five perspectives were proposed for work analysis: work domain, task situation, decision making situation, mental strategies and information needs. The work domain is represented using a two-dimensional matrix: means-ends abstraction
424 level and whole-part decomposition level. Means-ends levels are divided into five levels: purposes, abstract functions, general functions, physical functions, and physical form. Whole-part levels are divided into four levels: total system, subsystem, unit function, and component Task situation analysis represents information flow and causal relations of the tasks. The work organization and user characteristics are understood from the information needs analysis. Five frameworks were proposed to characterized the design territory: work domain characteristics, knowledge base organization, navigation representation, knowledge representation, and display composition. The work domain characteristics are characterized in terms of the necessity of human information processing to carry out the work. Navigation is represented by a flow chart of the task transitions. The display composition is represented based on the means-ends and whole-part structure. The knowledge representation gives us a framework of the transformation from the perspectives of work analysis and user characteristics to the interface design.
3. APPLYING THE PERSPECTIVES FOR INTERACTIVE DIAGNOSIS When customers experience trouble with a piece of equipment, they often query an expert operator over the phone. The operator must recognize the customer's trouble situation and diagnose the trouble. If the operator can visit the equipment site, he/she can inspect the trouble situation directly and diagnosis is easier. However, when the operator makes the diagnosis over the phone, he/she can get information about the trouble only through the customer's reports. The operator always has to be sensitive to variations in the dialogue situation and change his/her diagnosis strategies flexibly. Therefore, a system that supports the operator's cognitive processing during an interactive diagnosis session is needed. This section investigates a procedure of interactive diagnosis support system design based on the perspectives proposed by Rasmussen.
3.1. Adaptation of the Perspectives for Interactive Diagnosis Since the perspectives proposed by Rasmussen provide only a framework of the design process, we must choose and adapt the perspectives to the interactive diagnosis being performed. We must also investigate how to reflect the results of work analyses to the design. Functions, information flow, and display compositions must be decided to design an interactive diagnosis support system. The frameworks of navigation and display composition are needed for this. According to the knowledge representation framework, the representation of the display information depends on the necessity of human judgment to carry out the work. Consequently, the framework of the work domain characteristics is necessary to categorize the relationships between activity and human information processing. The perspectives of work domain, task situation, and mental strategies for work analysis are needed to use these design frameworks. 3.2. Procedure of Interactive Diagnosis Support System Design Some of the perspectives proposed by Rasmussen were selected and their transformation paths were created to design an ecological interface for the interactive diagnosis support system, as shown with arrows in Figure 1. First,
425
Work Analyses
Classification of the Work Characteristics
~Are
work domain task situation mental strategy
[ h~ii: ~ ~d In~ ~ ~t~
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
there any rules or~no const~ .v I yes :ili:iii ~ ilililill ii:iiiiie~~e~i:i:i: [
.. . . . . . . . . . . . . . . . . . . . . . . . . .
prototyping
experiment
analysis
I
Figure 1. Procedure of interactive diagnosis support system design. work analyses are conducted with three perspectives: work domain analysis, task situation analysis, and mental strategy analysis. Next, the work characteristic is categorized based on the task transition pattern. The concrete decision rule is newly determined to categorize the work characteristic using the framework of work domain characteristics. The interfaces of the support system are then designed using the frameworks of navigation and display composition based on the results of the work analyses. 4. INTERACTIVE DIAGNOSIS ANALYSES
Twenty-eight cases of interactive diagnosis of telephone trouble over the phone were analyzed according to the proposed procedure. Each entire dialogue was analyzed in terms of three perspectives of work analysis. 4.1. Work D o m a i n Analysis Keywords of the dialogues were analyzed based on means-ends and whole-part abstraction levels. Five means-ends abstraction levels and four whole-part decomposition levels were determined for the dialogue of interactive diagnosis. All sets of subjective and predicative keywords of the dialogues were categorized into the two-dimensional levels. The result indicated that dialogue flow often stagnated when one's keyword abstraction level was extremely different from that of the other. 4.2. Task situation Analysis Operators' information processes of interactive diagnosis were analyzed and decomposed into several task units in terms of decision making function. They include interpreting expressions, supposing causes, asking questions, testing,
426 verification, diagnosis, and so on. These tasks were performed with many different processes. Possible patterns of transition between each task unit were extracted using a flow chart, although the transition rules could not be clearly determined. The total time for each kind of task was calculated. The interpreting expression task accounted for 45 percent of the total time for the dialogues of interactive diagnosis. The asking question task accounted for 28 percent of the total time.
4.3. Mental Strategy Analysis The operators' mental strategies were analyzed based on the relationships of dialogue flow between the customer and the operator. Two operators' diagnosis strategies were found: an operator leading strategy and a user leading strategy. The operator leading strategy has the operator's questions mainly taking the initiative. The user leading strategy has the customer's reporting predominate. The operator frequently shifted between these two strategies. When the operator led the dialogue and did not change his strategies, his dialogues had nearly twice as many words than when the operator shifted his strategy. 4.4. Classification of the Work Characteristics Task flow is not regular with the interactive diagnosis. The following task is selected from many alternatives by operator's judgment. Consequently, interactive diagnosis is not categorized as highly structured work. The work is inevitably categorized as autonomous work or constrained work. On the other hand, each task is considered while being aware of the constraints of physical and functional structures of telephones, because the flow chart could be made to describe the alternatives for the following tasks based on the task situation analysis. Therefore, interactive diagnosis is finally categorized as constrained work. 5. INTERACTIVE DIAGNOSIS SUPPORT SYSTEM The result of the task situation analysis indicated that interpreting user's expression task and asking question task, in particular, take longer than the other tasks. Consequently, functions that support these two tasks are required. Moreover, the result of the mental strategy analysis indicates that the operator's strategy involves frequent shifts. Therefore, all the functions should always be active in the support system. Since the interactive diagnosis was categorized as constrained work, we can indicate the alternatives for the following task based on the list of confirmed conditions according to the flow chart of the tasks. Illustrations organized by elements of similar abstraction levels would prevent dialogue flow stagnation based on the work domain analysis. However, many representations can be supposed. We cannot determine the best representation based only on the work domain analysis. In this system, the most primitive representation is adopted to develop keyword menus.
5.1. The Interactive Diagnosis Support System Design An interactive diagnosis support system is proposed based on the results of the work environment analyses. Figure 2 shows one of the displays of the proposed system. The system supports two operator tasks: interpreting expressions and
427 asking questions. The t a s k of Objects Functions interpreting the customers' troubles i bell H I receive H~ through their reports is supported by ~ ] bottom i] ] recondition ] 1 | I butt°n II I rec°nfirm II J a function with which operators can I busy t°ne II i rec°rd II enter the reported conditions by I call II I rec°ver il selecting a p p r o p r i a t e keywords I I cord H I reduce H matching the customers' reports from Question the alphabetical keyword menus. The I ] Can Y°Ureceive the Ph°ne? ] keyword menus are divided into the ( yes )(sometimes)( no ) menu of objects and the menu of functions according to the two dimensions of the work domain analysis. The task of asking question Figure 2. An example of the interactive diagnosis support system to diagnose the trouble is supported (Original written in Japanese). by suggesting the question that the operator should ask next. When the operator requests the result of a diagnosis, the alternative causes are presented. Following questions and causes are decided based on the flow chart. Conventional diagnosis support systems mainly support the asking question task by showing the next question in fixed order. In the proposed support system, conditions reported by a customer can be entered at any time and be reflected in the following questions.
5.2. The Interactive Diagnosis Experiment An interactive diagnosis experiment was conducted to evaluate the designed system. Participants: Eight subjects participated in the experiment. They were not experts in telephone servicing. Four were asked to acts as operators to diagnose customer's telephone troubles using two interactive diagnosis support systems. The other four were asked to act as customers reporting their trouble to the operator over the phone. Each operator and customer were paired. Systems: The proposed system and a conventional support system, which did not respond to entry of the customers' reports were used. The operators used both systems to diagnose two kinds of troubles. The customers used a telephone that had a cordless extension telephone. Procedure: Two telephone trouble situations were assumed. They were caused by the experimenter purposely. The operator diagnosed them with one of two systems through the customer's reports. The problems, the systems, and the order were randomly assigned. 5.3. Results and Discussions All operators could solve the troubles created. The total time of each dialogue was measured. There was no significant difference between both systems in terms of time. However, it is not essential to evaluate the efficiency of the systems based on time, because the time taken mainly depends on the participants' pace and dialogue expressions. Contents of the conversations and operators' entering operations were analyzed to evaluate their diagnosis processes. The total number of operator's questions and customer's reports was counted as the number of interactions. When using the conventional system, the average number of interactions was 14.25, while the average number of interactions was
428 12.0 with the proposed system. When using the proposed system, the operators never asked questions that overlapped the previous customer reports, because the customers' reports affected the diagnosis offered by the proposed system. The results indicate that interpreting the customers' reports leads to efficient interaction. Therefore, the task situation analysis and the mental strategy analysis are effective in determining which tasks should be supported and in designing the relationships among the functions. In the proposed system, the function of entering the customers' reports is performed by selecting appropriate keywords from the alphabetical keyword menus. However, three keywords out of eleven were selected as wrong meaning for entering the customers' reports on the proposed system. The results indicate that many keyword expressions might be supposed for one condition, and it is difficult for operators to search appropriate keywords among the whole keywords' list. Another operation for the entering function is expected. Graphic choice and hierarchical menus would reduce the wrong selections instead of keyword choice from alphabetical menus. 6. CONCLUSION A new procedure for interactive diagnosis support system design was proposed based on the concept of the ecological interface proposed by Rasmussen. Interactive telephone trouble diagnoses were analyzed according to the proposed procedure to design an interactive diagnosis support system. An interactive diagnosis experiment using the designed system was conducted to evaluate the method. The results indicate that the task situation analysis and the mental strategy analysis are effective to design functions and information flow of interactive diagnosis support systems. However, the representation of the display cannot be determined logically. Since there can be many alternatives, it is necessary to repeat design and test process at the final stage of the interface design. ACKNOWLEDGMENTS The authors would like to acknowledge Dr. J. Rasmussen for his helpful discussions and comments in earlier stage of this research. They also wish to express their appreciation for T. Kishimoto of NTT Human Interface Laboratories. REFERENCES 1. J. Rasmussen, and A. M. Pejtersen, Mohawc Taxonomy Implications for Design and Evaluation, RISO-R-673(EN), RISO National Laboratory, 1993. 2. J. Rasmussen, A. M. Pejtersen, and L. P. Goodstein, Cognitive Systems Engineering, Wiley-Interscience, 1994. 3. Y. Asano and K. Ogawa, Interactive Diagnosis Support Systems based on Operators' Mental Strategies, The Japanese Journal of Ergonomics, No. 30 (1994) 136. 4. Y. Asano, S. Yonemura, and H. Hamada, The Effect of Keyword Choice on an Interactive Diagnosis Support System, Proceedings of the 49th Annual Meeting of Information Processing Society of Japan, (1994) 4-347.
III. 15 Computer Modeling of Mental Processes
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
Computer
analysis of characteristics
431
of creative thinking
and self-
esteem level A.E. Kiva, V.A. Molyako b, Stephen T. McHale c, V.G. Orishchenko a and I.A. Polozovskaya a aSouth-Ukrainian Pedagogical University, 26 Staroportofrankovskaya, 270020 Odessa, Ukraine blnstitute of Psychology, 2 Pankovskaya, 252033 Kiev, Ukraine CUniversity of Maryland, Art-Sociology, College Park, 20742, USA Abstract The present paper consists of a description of correlation between the level of creative abilities of a person and his self-esteem. The relationship is demonstrated between creative thinking level parameters and the self-esteem level. These dependencies may be used for the different mental groups comparison. 1. D E T E R M I N A T I O N OF CREATIVE T H I N K I N G LEVEL It is believed that creative abilities influence many other psychological features of a person. It may consider the creative abilities level as a main character of a person which his behavior and self-esteem are based on. A high level of creative thinking promotes to an adequate perception of external conditions. It may be explained by certain correlations between the creative thinking level and the intellectual level of a person. That is why it is important to have effective methods for these characters measuring. There are well known tests for intellectual level (IQ) measuring [ 1]. But we need today the new testing methods for the creative thinking level determination [2, 3].
1.1. Computer testing of creative thinking In accordance to proposed model [3] of creative thinking processes the creative problems solving can be considered as a consistent series of thinking steps. To get a solution of a creative problem some critical number of effective thinking steps must be fulfilled. Each thinking step to the solution is considered as an element of the thinking space. There are a number of local regions in the thinking space that contain the different kinds of thinking elements. We give account of the creative thinking process on the basis of the kinetic model of thinking elements accumulation in some regions of a thinking space. The interactions between thinking elements are also considered. Typical equations for processes of creative thinking describing when choice problems solved are:
432 dml
(1)
dt - 11 + al N1n - bl N2,
dt =/2 + a2 where N1 and N2 are numbers of solution elements accumulated in two alternative directions corresponding to the right and the wrong choices, 11 and/2 are coefficients of the intuitive component, al and a2 are coefficients of an accumulated information mastering, k and n the order of "psychological reaction", bl and b2 are coefficients of a mutual influence of the alternative solution ways. The structure of these equations always corresponds to the particular type of test problems that have computer realization. The computer testing problems do not require any preknowledge and look like games. The developed software can give experimental longitudinal dependencies N1 (t) and N2 (t) that are stored in the data file. On the next stage the theoretical functions N1 (t) and Nz(t) obtained as solutions of the above written equations are compared with the experimental data. As a result of this comparison the coefficients of thinking process Ii,z, al,2, bl,2, n, k are calculated by the software. It is necessary to notice that the proposed testing programs do not check knowledge but bring out the objective information about the structure of the personality creative thinking. We use the simple logical computer games, such as: Hanoi towers, River crossing etc to collect data. When the children are done playing, we work out the psychological parameters of the thinking process by using our special program, which contains the approximation procedure and the procedure to determine psychological parameters. 1.2. Comparison with other testing methods In this study the general indexes of the children abilities by Torrence method were measured [6]. Results are represented in Table 1.
Table 1 Comparision of our computer and Torrence's methods Age
Mean Parameters
A -- 6.9 SD = 0.5
Subjects 1
2
3
4
5
6
7
8
9
10
0.6 0.4 0.7 0.7 0.5 0.6 0.2 0.5 0.2 0.2 FT
0.4 0.4 0.5 0.6 0.6 0.4 0.4 0.4 0.1 0.2
m
A = 10.1 SD - 0.6
0.7 0.7 0.8 0.3 0.8 0.4 0.6 0.7 0.6 0.6 FT
0.5 0.4 0.6 0.3 0.5 0.4 0.4 0.5 0.3 0.5
m
A-- 12.7 S D - 0.5
0.4 0.8 0.7 0.7 0.7 0.5 0.4 0.7 0.6 0.6 FT
0.5 0.6 0.6 0.7 0.5 0.6 0.3 0.5 0.6 0.4
Note: A is the mean age, SD is standard deviation.
433 We constructed auxiliary parameters that have the same sense as the corresponding Torrence characteristics. For instance, the parameter of searching activity in general we determine as 11+ 12+ al + a2
(3)
hi+ b2 We have compared parameter a with Torrence characteristic which may be called the fluency of thinking (FT). Data for ten subjects (5 boys and 5 girls) from three age intervals are represented in Table 1. The choice of subjects was arbitrary. Parameters a and FT in the Table 1 are given in relative units. Similar comparison was fulfilled for other parameters. The correlation coefficients between Torrence parameters and received by our computer testing method are 0.7 - 0.9. The probability of correlations 95 Yo. Thus our results are corresponded to those which are received by Torrence method, but our method allows to get precise information about creative thinking mechanisms. 2. C O R R E L A T I O N S BETWEEN CREATIVE T H I N K I N G AND S E L F - E S T E E M LEVELS The aim of these investigations was to make clear how creative abilities of children are connected with their self-esteem. It is believed that a self-esteem of each person influence general characteristics of a group. Therefore correlations between creative thinking and self-esteem levels of people may be used for optimum organization of different groups in educational, research and other institutions. We have chosen in this study two groups of students in special school for gifted children in Odessa (Ukraine). Each group includes twenty persons with an average age 11 and 13 years. The information about creative thinking level was obtained by help of new computer testing methods described above. Students were prepared to computer games using during studies on computer disciplines. We proposed to children our computer tests that were hidden among usual computer games. Thus children were tested without any negative emotions. We used two ways in order to obtain an information about the self-esteem of children [7]. The first way is testing questions that were proposed to children during a conversation. Ten questions were formulated. For example: Can you reach very important decision independently? Does your mood depend on the outward conditions? Do you believe in a realization of your dream? All ten adequate answers corresponded the highest level of self-esteem (1,0). In other cases the level of self-esteem was proportional to the number of adequate answers. Another way of self-esteem of students estimating was an analysis of students' characters that were prepared by teachers. Thus we obtained an additional information concerning self-esteem of children. The results are given in Table 2. Results represented in Table 2 correspond to children of 13 years average age. We have shown general results in Fig.1. The results are not simple. Two dashed curves show the layout of points corresponded to different children. We can see that there is an uniform density population in the fight of the figure between dashed curves. However in the left side we see only large or small values of self-esteem level. Here we have the effect of a polarization of self-esteems for children with a low creative abilities level.
434 Table 2 Number of a student
Psychological Parameters I
L
T
Z
S
1
0.77
0.92
0.73
0.81
0.85
2
0.33
0.64
0.62
0.53
0.84
3
0.86
0.79
0.71
0.79
0.62
4
0.61
0.59
0.62
0.61
0.32
5
0.78
0.88
0.58
0.75
0.92
6
0.86
0.90
0.81
0.86
0.4 l
7
0.68
0.90
0.72
0.77
0.52
8
0.61
0.87
0.74
0.74
0.82
9
0.39
0.66
0.62
0.56
0.32
10
0.85
0.82
0.80
0.82
0.88
Results obtained by our computer testing methods (I - parameter of intuition, L parameter of logic), and by Torrence method (T). S is a self-esteem parameter. Z is parameter of creative abilities that obtained as average value of parameters l, L and T.
1.0
/.
•
\-
0.8
/ x~.
/
•
J 0.6
0.4
/
°
,,~
/.
• x,.
0.2
0.2
0.4
0.6
0.8
1.0
Z
Figure 1. Dependence between a self-esteem level (S) and an avarage level of creative abilities (Z)
435 3. CONCLUSION The dependence of self-esteem of children on their creative abilities level was investigated and new computer testing methods were used for measuring creative abilities levels. It was established that children with a low level of creative abilities are characterized by nonuniform distribution of the self-esteem level. Effect of polarization of self-esteems takes place. REFERENCES 1. H. Eysenk The Structure of Human Personality. London: Harmondsworth Penquin Books, 1971. 2. A.E. Kiv, V.G. Orishchenko, I.A. Polozovskaya, I.G. Zakharchenko. Computer Modelling of the Learning Organization. Advances in Agile Manufacturing. P.T. Kidd and W. Karwowski (Eds). Amsterdam: lOS Press, 1994. 553-556. 3. A.E. Kiv, V.G. Orishchenko, I.A. Polozovskaya, I.G. Zakharchenko, V.V. Chislov, V.L. Maloryan. Creative Thinking Process Simulation and Computer Testing. Proceedings of the Symposium on Human Interaction with Complex Systems, Greensboro, North Carolina A&T Univ., 1994. 4. A.E. Kiv, V.A. Molyako, V.L. Malorayn, I.A. Polozovskaya, Z.I. lskanderova, The Creative Thinking Testing by Using of Testing Problems Based on Different Logical Schemes. In: Y. Anzai and K. Ogawa (Eds.) Proceedings of 6th International conference on Human-Computer Interaction (HCI International'95), Amsterdam: Elsevier Science Publishers 5. V.V. Chislov, V.L. Maloryan, I.A. Polozovskaya, G.V. Shtakser, A.I. Uyemov, I.G. Zakharchenko, M. Athoussaki The interface improvement for the creative thinking computer testing In: Y. Anzai and K. Ogawa (Eds.) Proceedings of 6th International conference on Human-Computer Interaction (HCI International'95), Amsterdam: Elsevier Science Publishers 6. E. Torrance (Research ed.). Thinking creatively in action and movement, Bensenville IL: Scholastic Testing Service, 1980. 7. U. Zabrodin (ed.), Psychological diagnosis of students, Moscow, 1990 (in Russian).
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
Computer-based testing of reflective thinking: performance in 9 to 12 year old children
437
Executive control of erroneous
Uri Shafrir Department of Applied Psychology, Ontario Institute for Studies in Education, 252 Bloor St., Toronto, Ont. M5S 1V6, Canada Attention to errors was operationalized recently by the measure of post-failure reflectivity (Shafrir & Pascual-Leone, 1990). Post-failure reflective children spend long periods of time following the production of incorrect response, compared to the time they spend following the production of correct response. Shafrir and Pascual-I.eone (1990) conceptualized post-failure reflectivity as an exploratory mental executive, a spontaneously activated "debugging" procedure that helps children reexamine and correct faulty internal plans. Researchers reported that children who show high executive control following the production of erroneous performance (post-failure reflective children) scored higher than post-failure impulsive children on an inference task, as well as on other measures of intellectual functioning and academic achievement (Shafrir & Pascual-Leone, 1990); that post-failure reflectivity generalized across tasks and across domains, and that post-failure reflective children were intentional learners (Shafrir, Ogilvie & Bryson, 1990, Experiment 2); and that those children used deep planning in constructing their action plans. In other studies, children with a reading disability were found to be significantly less post-failure reflective than normal controls (Shafrir, Ogilvie & Bryson, 1990, Experiment 1; Shafrir, Siegel & Chee, 1990). Shafrir, Ogilvie and Bryson (1990) claimed that attention to errors plays an important role in learning. This study investigated additional aspects of erroneous performance; in particular, we hypothesized that executive control of performance prior to the production of erroneous response, may also play an important role in learning. A spontaneously activated "rechecking" procedure, cued by a feeling of uncertainty about the planned, impending response, may help the child to reexamine internal plans prior to the production of an incorrect response. Attention to expected error production was operationalized by pre-failure reflectivity, defined as the ratio between mean pre-failure response latency and mean response latency; pre-failure reflective children spend long periods of time prior to the production of incorrect response, compared to the time they spend prior to the production of correct response. Finally, we operationalized overall executive control of erroneous performance as behavior that is both pre-failure and postfailure reflective. The development of executive control structures for the regulation of erroneous performance may be related to the transition from concrete to formal operations (Case, 1985; Pascual-Leone, 1987; Piaget, 1950). Paying close attention to potential as well as to actual disconfirmation of an hypothesis, i.e., pre- and post-failure reflective behavior, may be a necessary part of interpropositional, hypothetico-deductive thought, of vectorial operations, and of the manipulation of abstract sets. The development of such executive control structures may also signal the emergent increase in representational competence that enables children to expand
438 the scope of task representation to include a representation of their interaction with the task in real time. The two specific hypotheses tested in this study were: (1) twelve year olds have higher overall executive control of erroneous performance, and score higher on an inference task as well as on other intellectual and academic tasks, than younger children; (2) within each of the four age groups (9, 10, 11, and 12 years old), children who have high overall executive control of their erroneous performance, score higher than children with low overall executive control of their erroneous performance on an inference task as well as on other intellectual and academic tasks.
The PAR computer-based inference task The PAR (Pattern Recognition) task was administered individually. PAR is a computerbased induction task, with 80 stimuli of repeated designs shown through bars of different colors, heights, colors + heights, and colors + heights + sounds of varying pitch, where inter-trial intervals are subject controlled. Subjects were asked to decide whether the stimulus was a repeating design; if the subject's answer was "no", he/she was asked to point to the location of the "mistake" in the design with a blinking light on the computer screen; response was immediately followed by a yes/no feedback; the subject had to strike a key in order to see the next stimulus. The unobstrusive nature of the computer-based PAR task made it possible to record both response- and post-response latencies. The response latency was decomposed into pre-success and pre-failure; similarly, the post response latency was decomposed into post-success and postfailure. Pre-failure and post-failure reflectivities were calculated from the formulae: Mean pre-failure latency Pre-failure reflectivity
(1) Mean response latency Mean post-failure latency
(2)
Post-failure reflectivity Mean post-response latency
Subjects and procedures Subjects were students in grades four through seven in five public schools in Arad, a town in the south of Israel. We tested an unselected sample of 377 subjects, aged 9 (.q = 109), 10 (.n. = 114), 11 n(.n_= 85), and 12 (.n = 69). Children in each age group were divided by a double median split on pre-failure reflectivity and on post-failure reflectivity, into 4 quadrants: children who were both pre- and post-failure reflective, were defined as having high overall executive control of erroneous performance; children who were both pre- and post-failure impulsive, were defined as having low overall executive control of erroneous performance; finally, there were two groups of mixed conditions. Scores for the Israeli version of an IQ test (M~ = 106.1, SD = 11.5), and for Raven's SPM ~ = 32.5, SD = 8.6), were obtained when children entered 3rd grade. Scores for computer-based drill and practice in arithmetic and for teachers' evaluations of the student's level of intellectual functioning (not level of academic achievement) in percentiles were available at the time of this study. The Figural Intersection Test (FIT) for attentional capacity (Pascual-Leone & Ijaz, 1989), was group-administered.
439 Results Descriptive statistics are shown in Tables 1 and 2. Results of 2-way ANOVAs, age (4 levels: 9, 10, 11, and 12 years old) by overall executive control of erroneous performance (4 levels: high, low, and two mixed conditions) are shown in Table 3.
Table 1 Mean (SD) score on PAR, pre-failure reflectivity and post-failure reflectivity by age group Age group
Measure
9 years (n = 109)
10 years (n = 114)
11 years (n = 85)
12 years (n = 69)
PAR
.59 (.18)
.60 (.20)
.63 (.20)
.76 (.12)
Pre-failure reflectivity
1.16 (.22)
1.21 (.92)
1.15 (.21)
1.34 (.42)
Post-failure reflectivity
1.60 (.55)
1.73 (.80)
1.67 (.61)
2.14 (.92)
Note. PAR = proportion of correct response Twelve year olds scored significantly higher on the FIT task for attentional capacity, showed higher overall control of erroneous performance (were more pre-failure and more postfailure reflective) than children in the 9 to 11 years age range; the 12 year olds also scored higher on PAR, and on a variety of intellectual and achievement measures. Within each age group, children with high overall executive control of erroneous performance on the PAR task, scored significantly higher on a variety of tasks of intellectual functioning, on teacher's evaluations of intellectual functioning, and on arithmetic drill and practice, than children with low overall executive control of erroneous performance; the two groups of mixed conditions scored in the intermediate range. The younger children in the 9 and 10 years old age groups with high overall executive control of erroneous performance, scored as high as the 12 year olds on a measure attentional capacity (a score of 5 on the FIT task); these younger children scored significantly higher than the children in the 12 years old age group who had low overall executive control of erroneous performance, on the various tasks. Table 4 shows a commonality analysis of the variance of the score on PAR as the dependent variable, and age, FIT, IQ, SPM, Math, and pre- and post-failure reflectivities as the independent variables. Each R 2, the squared zero-order correlations between the dependent variable and each independent variable, was partiaUed into two components: unique variance of each independent variable, and common variance shared between the particular independent variable and one or more of the other independent variables (Kerlinger & Pedhazur, 1973). The unique contribution of the overall executive control of erroneous performance (pre- and post-failure reflectivities) to the variance of the score on the PAR inference task is higher (about
440 15 %), than the unique contributions of each of the other independent variables, age (2.3 %), Mcapacity (0.8 %), IQ (2.7 %), SPM (0.6 %), and arithmetic drill and practice (0.0 %). Table 2 Mean (SD) gores on tasks of intellectual functioning and academic achievement, for the low and high groups of overall executive control of erroneous performance by age
Overall executive control of erroneous performance Low
High
Age group
9
10
11
12
9
10
11
12
Number of subjects
30
39
28
20
31
39
29
21
PAR
.41 (.11)
.42 .46 (.15).21)
.63 (.09)
.72 (.13)
.73 (.12)
73 (.14)
.83 (.08)
Pre-failure reflectivity
1.0 (.07)
1.1 (.09)
1.0 (.10)
1.0 (.11)
1.3 (.19)
1.4 (.37)
1.3 (.17)
1.7 (.48)
Post-failure reflectivity
1.2 (.12)
1.2 (.14)
1.3 (.17)
1.6 (.25)
2.1 (.70)
2.2 (.96)
2.0 (.68)
2.7 (1.10)
IQ
100.4 98.6 98.0 102.6 (10.4) (10.3) (14.6) (8.1)
110.7 108.3 104.3 109.3 (11.9) (11.4) (10.6) (11.6)
SPM
26.4 (8.3)
26.5 (7.8)
28.8 (7.2)
28.4 (5.1)
35.2 (7.4)
34.5 (6.7)
31.3 (8.3)
35.8 (6.4)
FIT
3.8 (1.3)
4.1 (1.3)
4.2 (1.5)
5.1 (1.3)
4.7 (1.3)
5.1 (1.3)
5.1 (1.5)
5.2 (1.6)
Math
-7.3 (7.1)
-10.5 -12.0 -7.4 (7.6) (14.7) (13.6)
-0.9
-1.6
-8.5
-0.2
Teacher
41.6 37.1 32.2 na (25.5) (30.8) (32.0) na
(10.9) (12.4) (11.3) (14.2) 66.8 70.2 40.8 na (23.9) (21.1) (24.4) na
Note. Low = below median for age group on both pre- and post-failure reflectivity; High = above median for age group on both pre- and post-failure reflectivity; PAR = proportion of correct response; IQ = full scale score; SPM = Raven's Standard Progressive Matrices; FIT = Figural Intersection Task; Math = arithmetic drill and practice (months ahead or behind expected grade level); Teacher teacher's evaluation of intellectual functioning (percentile).
441 Table 3 F-ratios for two-way (age X overall executive control of erroneous performance) ANOVAs for the score on PAR and other test measures, for the whole population
Measure
PAR
IQ
SPM
FIT
Math
Teacher
Age
25.7***
ns
ns
4.2**
2.8*
4.9**
Exec
107.4"**
12.6"**
18.5"**
7.8***
7.6***
16.2"**
Age X Exec
ns
ns
ns
ns
ns
ns
MS~
0.018
121.8
51.5
1.9
127.5
684.9
Note. Age = age group; Exec = overall executive control of erroneous performance; Table 4 Commonality analysis of the variance of the score on PAR as a dependent measure Independent measures Age
FIT
Unique variance
.023***
Common variance
.054
R2
.077
.008*
.107
.115
IQ
SPM
Math Prefailure
Postfailure
.027***
.006* .000
.013"*
.139"**
.155
.150
.094
.108
.219
.182
.156
.094
.121
.358
Note. ***p < .0001; *'12 < .001; "12 < .05. Conclusions These results may be interpreted in terms of the neoPiagetian constructs of the growth in attentional capacity (Pascual-I.eone, 1987), the maturation of executive control structures (Case, 1985), and optimal levels of development and skill acquisition. The results lend support to the two hypotheses. The significant increase in executive control of erroneous performance at age 12 corresponds to an increase of attentional capacity from 4 to 5 units as predicted by Pascual-Leone (1987) and to the onset of the stage of formal operations (Case, 1985; Piaget, 1950). The newly acquired ability of 12 year old children to operate on operations and not only on concrete entities facilitates the development of internal procedures that "debug" and improve
442 currently operating action plans. The longer periods of time that 12 year olds spent both prior to, as well as following the production of incorrect response, compared to the time they spent
prior to as well as following the production of correct response, appear to signal the emergence of a new type of executive control structure, aimed at optimizing performance. However, we note that the 12 year olds with low overall executive control also scored 5 on M-capacity. This unexpected result may mean that an increase in M-capacity to 5 units at age 12 is only a necessary but not a sufficient condition for the development of effective executive control. This as well as other results of this study can not easily be interpreted in terms of current neoPiagefian theory. About 1/3 of the younger children in the 9 and 10 years old age groups showed a high level of executive control of erroneous performance, and scored 5 on the FIT measure of attentional capacity, a value predicted by neoPiagetian theory for 12 years old children (Case, 1985; Pascual-Leone, 1987). These younger children also scored significantly higher on various intellectual and achievement tasks than the children in the 12 years old age group who showed low overall executive control of erroneous performance. Similar results were reported in a study of precocious cognitive development at the level of formal operations; 'psychometrically bright' 5th graders were "at a more advanced cognitive developmental level" than the 'psychometrically average' 7th graders (Keating, 1975, p. 279). Our results, showing large individual differences within age groups, suggest that a significant number of children within each age group incorporated executive control of erroneous performance in their mental representation of the task. These results suggest that this emergent representational competence is an important measure of intellectual development. However, in its present form this measure lacks the necessary metric for testing the consistency of these findings with the current framework of neoPiagetian theory. References
Case, R. (1985). Intellectual Development: Birth to Adulthood. New York: Academic Press Keating, D. P. (1975). Precocious cognitive development at the level of formal operations. Child Development, 46, 276-280. Kerlinger, F. N., & Pedhazur, E. J. (1973). Multiple regression in behavioral research. New York: Holt, Rinehart and Winston. Pascual-Lexme, J. (1987). Organismic processes for neoPiagetian theories: A dialectical, causal account of cognitive development. In A. Demetriou (Ed.), The neoPiagetian theories of cognitive development: Toward an integration (pp. 25-64). Amsterdam: North-Holland. Pascual-I_eone, J., & Ijaz, H. (1989). Mental capacity testing as a form of intellectual developmental assessment. In R. Samuda, S. Kong, J. Cummings, J. Pascual-Leone & J. Lewis (F~s.), Assessment and placement of minority students: A review for educators, (pp. 141-171). Toronto: C. J. Hogrefe. Piaget, J. (1950). The Psychology of Intelligence. London: Roufledge and Kegan. Shafrir, U., Ogilvie, M., & Bryson, M. (1990). Attention to errors and learning: Across-task and across-domain analysis of the post-failure reflectivity measure. Cognitive Development, 5, 405-425. Shaffir, U. and Pascual-Leone, J. (1990). Postfailure reflectivity/impulsivity and spontaneous attention to errors. Journal of Educational Psychology, 82, 2, 378-387. Shafrir, U., Siegel, L. S., & Chee, M. (1990). Learning disability, inferential skills and postfailure reflectivity. Journal of ~ i n g Disabilities, 23, 506-517.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
The creative thinking testing on different logical schemes
by using
443
of t e s t i n g
problems based
A.E. Kiva, V.A. Molyako b, V.L. Maloryan a, I.A. Polozovskayaa and Zelina I. Iskanderova c aSouth-Ukrainian Pedagogical University, 26 Staroportofrankovskaya, 270020 Odessa, Ukraine blnstitute of Psychology, 2 Pankovskaya, 252033 Kiev, Ukraine CUniversity of Toronto, 4925 Dufferin Street, Downsview, Ontario, M3H 5T6 Canada Abstract New methods of testing of creative thinking are considered. They based on a mathematical model of thinking processes described in [1, 2]. This study consists more general consideration of computer testing programs structure. We have shown that there is a possibility of measuring of creative thinking parameters by using of computer testing problems based on different logical schemes. 1. CONCEPTION OF CREATIVE T H I N K I N G TESTING It is very difficult and may be impossible to determine the general peculiarities of creative thinking of a person. Complex creative problems are decided thanks to using enormous possibilities of a person brain. We model creative thinking processes taking into account that the significant part of the necessary information is absent. Our base hypothesis consists the following items: 1. The creative thinking processes proceed according to certain logical schemes. These schemes correspond to different variants of the modern logic and far and by may not correspond to any logics that are known today. The intuition component is not less essential and in many cases may be responsible for the success of a solution of creative problems. 2. The creative thinking quality depends on a set of logical schemes that a person uses in order to solve the creative problem. It is significant the degree of complexity of logical schemes using by a person. We assume that the intuition component is important in solution of creative problems any way. So we need a criterion for the evaluation of intuition component. The general procedure of development of testing programs is following: After choosing the testing problems corresponding to certain level of difficulty (certain logical scheme) the set of adequate differential equations are constructed. The equations are investigated on the basis of the quality theory of differential equations.
444 The software must be developed which can obtain the temporal function of the number of thinking elements as corresponding steps that a person fulfills during his moving to the solution. Testing problems have a computer realization and look like computer games. The software on the next stage must calculate the coefficients of differential equations that correspond to the best approximation of testing experimental curves by theoretical functions (solutions of differential equations). These coefficients has a sense of corresponding psychological parameters. In such a manner we obtain computer tests that sound different types of creative thinking. The final conclusion about the creative abilities of a person may be drawn by using of a set of testing problems in each case. Thus we suggest that the model of creative thinking processes may be given by the following scheme. Scheme 1.
Volume of memory I
Logic 1
I
Il
,
Logical operations Logic 2
Logic 3
Intuitive operations I
II !11
Logic N
According to this scheme the general level of creative thinking (Z) may be expressed as Z = X k Lk + 1 + V,
(1)
where Lk is the degree of complexity and k is the degree of mastering by a person of a given logic, I and V are parameters of intuition and memory. 2. COMPUTER SIMULATION OF CREATIVE T H I N K I N G PROCESSES 2.1. Description of the model The model of creative thinking processes which was proposed [2 - 4] is not intended for the general conformities to natural laws of a creative thinking description. This model corresponds to concrete processes in the person's mind connected with his aspiration to solve given logical problem. The modelling of creative thinking has many aspects. However, it has been generally assumed that solving complex intellectual problems entails the initiation of certain model structures in the human's mind. The examination of different authors' approaches to the concept of creative thinking modelling suggests that the usages of this notion are numerous and varied, and the relationship between them is not obvious [5 - 7]. Advantages of our model are in its practical direction. It can be applied directly to development and exploration of computer programs for creative thinking measurements.
445 The efficiency of developed testing methods in such a manner may testify that the model is correct. In order to explain the mathematical structure of equations let us briefly consider the main peculiarities of the model [2]. A thinking space existence is postulated that contains discrete elements corresponding to certain steps of a person during his moving to the problem solution. The thinking steps (or thinking elements) arise in local regions of thinking space and they are divided in three groups: effective steps, wrong steps and intermediate steps. Effective steps form the trajectory of a person moving straight towards the solution of the problem. If a person digresses from the direct route his trajectory includes wrong steps. The intermediate steps produce effective steps if they form complexes. As soon as the certain critical number of effective steps is accumulated the problem is solved. All steps may arise in regions II and III of Scheme 1. Region I provides other regions by resources (necessary information). It is in fact a realization of logic with resources [8]. 2.2. Construction of equations Our creative thinking model [ 1 - 4] is based on the assumption of equivalent contribution of irrational-intuitive and formal-logical steps in the process of creative problem solving. So according to the model the example of differential equations for the creative thinking process my be written in the following form: dNl dt - I1 + aiN1 + blN~- ClN2 dN2 dt = 12+ a2N2- b2Na- c2N1
(2)
dNa dt = I3+ aaNa+ baN1-c3N2 where N 1, N2 and N3 are numbers of thinking elements of three types: effective steps, wrong steps and intermediate steps; I1,12 and 13 are coefficients of the intuitive component, al, a2 and a3 are coefficients of mastering of accumulated information, bl, b2 and b3 and also c l, c2 and c3 are coefficients of the mutual influence of different thinking elements, k is the order of "psychological reaction". In every case the structure of these equations corresponds to the particular type of test problems which have computer realization. The coefficients in the above mentioned equations characterize the creative thinking process. Here it is notable that we introduce nontraditional characteristics for creative abilities of a person, such as coefficients of intuition, formal logic, the volume of thinking space etc. The quantitative determination of these coefficients by new computer testing methods allow to get a precise information for the creative thinking structure of a person. The traditional tests give only a general description of a person abilities without detailed information regarding the creative thinking structure.
446 3. G E N E R A L I Z A T I O N OF E Q U A T I O N S FOR D I F F E R E N T L O G I C A L SCHEMES In [9] the peculiarities of child's creative thinking were considered. Each person has concrete abilities to operate with some logical schemes. Our aim is to construct equations that describe thinking processes according to different logics. Equations (2) reflect principles of classical logic. The main principle is twice-valued character of any statement. Each step in thinking processes is true or false. The first equation described processes of accumulation of correct steps, the second equation is for wrong steps. In fact the third equation is equivalent to the first because correct steps are formed by intermediate steps. Another set of equations must be written for thinking processes based on nonclassical logics. Here we discuss only one example. Let us consider a set of equations that correspond to many - valued logic with resources. The typical equation is following: dNi dt = I (R) + a Ni + a ' Nkm -/ff Njp N[
(3)
Here Ni is a number of steps that are characterized by a certain value of probability to reach the solution. Functions Nm, Nn, Nj, Np have the same sense and are characterized by their probabilities, parameter of intuition I (R) depends on resource R that may change. Functions Nm and Nn describe intermediate steps that form Ni steps in processes of complexes formation. Steps Nj a n d Np form complexes that prevent to Ni steps accumulation. Other parameters and coefficients are the same as in equations (2). Thus the main differences of equations (3) are following: 1. There are no wrong steps. 2. During the accumulation of thinking steps and corresponding moving of a person to a solution the resources are changing. 3. The number of equations depends in each case on creative thinking processes of a person. It is very important to choice testing problems that have logical structure in accordance with equations structure. 4. C O N C L U S I O N It is shown that there are new ways of testing of creative thinking by mathematical modelling of different logical schemes that are realized in person's mind. Differential equations must describe this logical schemes. The next task is to choice testing problems that consist these logical schemes and to develop corresponding computer testing programs for different levels of creative thinking. We developed such programs for children of 7 14 years [1, 9, 10]. REFERENCES 1. A.E. Kiv, V.G. Orishchenko, I.A. Polozovskaya, I.G. Zakharchenko. Computer Modelling of the Learning Organization. Advances in Agile Manufacturing. P.T. Kidd and W. Karwowski (Eds). Amsterdam: IOS Press, 1994. 553-556. 2. A.E. Kiv, V.G. Orishchenko, I.A. Polozovskaya, I.G. Zakharchenko, V.V. Chislov, V.L. Maloryan. Creative Thinking Process Simulation and Computer Testing.
447 Proceedings of the Symposium on Human Interaction with Complex Systems, Greensboro, North Carolina A & T Univ., 1994. 3. A.E. Kiv, V.A. Molyako, V.G. Orishchenko, I.G. Zakharchenko, I.A. Polozovskaya, A.M. Solodovnikov. The Computer Testing System of a Group and Psychological Correction of Teaching Methods. Proceedings of 5th International Conference on Human-Computer Interaction, HCI'93, Orlando, Florida, USA, August 8-13, 1993. 153. 4. A.E. Kiv, V.G. Orishchenko, I.A. Polozovskaya, I.G. Zakharchenko. Criterions of Intellectual Level. Proceedings of International Conference "Gifted Children: Family, School, State", Kiev, 1994. 5. Mind Design. J. Haugeland (Ed.). Cambridge, Mass.: MIT Press, Bradford Book, 1981. 6. R.N. Shepard, L.A. Cooper. Mental Images and Their Transformations. Cambridge, Mass.: MIT Press, Bradford Book, 1982. 7. J.A. Fodor. Modularity of Mind. Cambridge, Mass.: MIT Press, Bradford Book, 1983. 8. N.J. Nilsson. Probabilistic logic, Ibid. 28 (1986) 71 - 87 9. V.V. Chislov, V.L. Maloryan, I.A. Polozovskaya, G.V. Shtakser, A.I. Uyemov, I.G. Zakharchenko, M. Athoussaki The interface improvement for the creative thinking computer testing In: Y. Anzai and K. Ogawa (Eds.) Proceedings of 6th International conference on Human-Computer Interaction (HCI International'95), Amsterdam: Elsevier Science Publishers 10.A.E. Kiv, V.A. Molyako, Stephen T. McHale, V.G. Orishchenko, I.A. Polozovskaya Computer analysis of characteristics of creative thinking and self-esteem level In: Y. Anzai and K. Ogawa (Eds.) Proceedings of 6th International conference on Human-Computer Interaction (HCI International'95), Amsterdam: Elsevier Science Publishers
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
449
F r o m N o v i c e to Expert D e c i s i o n Behaviour: a Q u a l i t a t i v e M o d e l l i n g A p p r o a c h w i t h Petri Nets. Matthias Rauterberg Work and Organizational Psychology Unit, Swiss Federal Institute of Technology (ETH) Nelkenstrasse 11, CH-8092 Zurich, Switzerland
Abstract To support the human factors engineer in designing a good interactive system a method has been developed to analyze the empirical data of the interactive decision behaviour described in a finite discrete state space. The sequences of decisions and actions produced by users contain much information about the mental model of this user, the individual problem solution strategies for a given task and the underlying decision structure. We distinguish between (1) the logical structure, (2) the sequential goal structure, and (3) the temporal structure. The analysing tool AMME can handle the recorded decision and action sequences and come up automatically with an extracted net description of the task dependent decision model (the logical structure). This basis model was filled up with additional elements to reconstruct one empirical action sequence of an expert user. Four different models are presented and their predictive power discussed. 1.
INTRODUCTION
Learning is a permanent process that changes our long-term knowledge base in an irreversible way. The structure of our long-term memory changes to more complexity and higher abstraction. Learning increases constantly the complexity of the mental model. What mental models are and how they work, is quite unclear. Carroll and Reitman-Olson [2] summarise their research recommendations as follows: "(1.) Detail what a mental model would consist of and how a person would use it to predict a system's behaviour . . . . (2.) Investigate whether people have and use mental models of various kinds . . . . (3.) Determine the behaviours that would demonstrate the model's form and the operations used on it . . . . (4.) Explore alternative views of Sequence/Method representations and the behaviour predicted from them . . . . (5.) Explore the types of mental representations that may exist that are not mechanistic . . . . (6.) Determine how people intermix different representations in producing behaviour . . . . (7.) Explore how knowledge about systems is acquired . . . . (8.) Determine how individual differences have an impact on learning of and performance on systems . . . . (9.) Explore the design of training sequences for systems . . . . (10.) Provide system designers with tools to help them develop interfaces that invoke good representations in users . . . . (11.) Expand the task domain to more complex software" ([2] pp. 59-61). In this paper we present a modelling approach that contributes to points (1), (3), (4), (5), (7), (10) and (11). We are primarily interested in a bottom-up, behaviour driven and not in a top-down, theory driven approach. One of the most elaborated modelling approach is SOAR [5]. Newell [7] describes SOAR as follows: "Soar is ... a symbolic computational system .... Soar is organised around problem spaces, that is, tasks are formulated as search in a space of states by means of operators that produce new states, where operators may be applied repeatedly, to find a desired state that signifies the accomplishment of the task .... Soar is organised entirely as a production system, that is, its long-term memory for both program and data consists of parallel-acting condition-action rules . . . . Soar incorporates a goal hierarchy .... Soar learns continuously from its experience by chunking, which constructs new productions (chunks) to capture the new knowledge that
450 Soar developed (in working memory) to resolve its difficulties" ([7] pp. 30-32). Soar is based on impasse-driven learning. "While Soar is performing a task by using the behaviour model in working memory, it is also learning. It is building chunks every time it impasses from one problem space to another . . . . These chunks constitute the acquisition of knowledge for doing the task" ([7] pp. 62-62). The knowledge generated by chunking and stored in the long-term memory represents only successful trials. Knowledge of unsuccessful attempts is not in memory. Learning in Soar means that long-term memory contains evidence only of the sequence of effective actions. But, what could it mean if the majority of our long-term memory consists only of unsuccessful trials ? Soar seems to be a typical representative of a top-down, theory driven approach for error-free skilled behaviour. Why do we believe that a bottom-up is better than a top-down approach? The answer refers to the following assumption. Most of the known modelling approaches is based on the assumption that the "mental model maps completely to the relevant part of the conceptual model, e.g. the user virtual machine. Unexpected effects and errors point to inconsistency between the mental model and the conceptual model" ([ 12] p. 258). This one-to-one mapping between the mental model and the conceptual model of the interactive system implies a positive correlation between the complexity of the observable behaviour and the complexity of the assumed mental model. But this assumption seems to be wrong. Based on the empirical result in [ 11 ], that the complexity of the observable behaviour of novices is larger than the complexity of experts, we must conclude that the behavioural complexity is negatively correlated with the complexity of the mental model. If the cognitive structure is too simple, then the concrete task solving process must be filled up with a lot of heuristics or trial and error behaviour. Learning how to solve a specific task with a given system means that the behavioural complexity decreases and the cognitive complexity increases. Now, one of the central question is: What kind of knowledge is stored in the cognitive structure? Before we are able to give a preliminary answer to this question, we have to introduce our complexity measure.
2.
THE MEASUREMENT OF COMPLEXITY
The symbolic representation of the machine system consists of the following elements: 1. objects (things to operate on), 2. operations (symbols and their syntax), and 3. states (the 'system states'). The mental model of the user can be structured in representing: objects, operations, states, system structure, decision and task structure. A net can be described as a mathematical structure consisting of two non-empty disjoint sets of nodes (S-elements and T-elements), and a binary flow relation (F). The flow relation links only different node types and leaves no node isolated [8]. Petri nets can be interpreted in our context by using a suitable pair of concepts for the sets S (signified by a circle '()') and T (signified by a square '[ ]') and a suitable interpretation for the flow relation F (signified by an arrow '->'). Bauman and Turano [ 1] showed, that Petri nets are equivalent to formalism based on production rules (like CCT of Kieras and Poison [4]). In this sense, our approach can be subsumed under 'logic modelling', too. The main operations (relations) between two Petri nets are abstraction, embedding and folding [3]. The folding operation in the Petri-net theory is the basic idea of the approach presented in this paper. Folding a process means to map S-elements onto S-elements and T-elements onto Telements while keeping the F-structure. The result is the structure of the performance net. Each state corresponds to a system context, and each transition corresponds to a system operation. This sequence is called a 'process' (see Figure 1). An elementary process is the shortest meaningful part of a sequence: (s') -> [t'] -> (s"). If the observable behaviour can be recorded in a complete ...-> (state) -> [transition] -> (state) ->... process description (see Figure 1), then the analysis and construction of the net structure of this process are simple: you have only to count the number of all different states and transitions used, or to mark on a list the frequencies of each state and transition used in the process. But, if the observable behaviour can only be recorded in an incomplete (e.g ..... ->
451 (state) -> [transition] -> [transition] ->... or ...-> (state) -> (state) -> [transition] ->...) process description, then the analysis and construction of the net structure of this process are difficulty. You have to find out the correct state (transitions, respectively) between both transitions (states, respectively). Unfortunately, this is the most frequent case in practice. For these cases we need automatic tool support. In the last years we developed a tool, that gives us the possibility to analyze any processes with an incomplete process description, that are generated by finite state transition nets (cf. [ 11 ]). The aim of the 'folding' operation is to reduce the elements of an observed empirical decision process to the minimum number of states and transitions, with the reduced number of elements being the 'logical decision structure'. Folding a decision process extracts the embedded net structure and neglects the information of the amount of repetitions, the sequential order, and the temporal structure. A simple pattern matching algorithm looks for all 'elementary processes' in the sequence. A composition algorithm (the folding operation) is now able to build up the Petri net combining all elementary processes. The result of a folding operation of our example sequence (Figure 1) is the Petri net given in Figure 2. Measurable features of the behavioural process are: number of states and transitions totally used, number of different states and different transitions used, dwell time per state and transition, etc. These measurements can be easily done based on a protocol of the user's behaviour automatically recorded by an interactive software program (the dialog system) in a 'log file'. To measure complexity we use the Ccycle metrics of McCabe [6]. With Ccycle we have a useful quantitative metric to measure behavioural complexity. We are discussing the advantages and disadvantages of four different quantitative metrics in the context of an empirical investigation elsewhere (see [ 10]). The complexity measured with Ccycle is defined by the difference of the total number of connections (F: arrows) and total number of net elements (T-transitions plus S-states). The parameter P is a constant to correct the result of Formula 1 in the case of a sequence (F - (T + S) = - 1); the value of P in our context is 1. Ccycle
-
F
-
(T
+
S)
+
P
(I)
The measure Ccycle of the model- 1 in Figure 2 is [ 18 - 13 + 1 = 6]; the complexity of the net shown in Figure 2 is six. But, what could this number mean? McCabe [6] interprets Ccycle as the number of linear independent paths through the net. Other interpretations of Ccycle are number of holes in a net or number of alternative decisions carried out by the users. Observing the behaviour of people solving a specific problem or task, is our basis for estimating 'model complexity (MC)'. The cognitive structures of users are not directly observable, so we need a method and a theory to use the observable behaviour to estimate MC. We call the complexity of the observable behaviour the 'behavioural complexity'. This behavioural complexity can be estimated by analysing the recorded concrete task solving process. The necessary task solving knowledge for a given task is constant. This knowledge embedded in the cognitive structure of the mental model can be reconstructed. 3.
R E C O N S T R U C T I O N OF T H E M E N T A L M O D E L
We carried out an empirical investigation to compare different types of interfaces (see [9]). For the reconstruction we chose one part of a log file of an expert user (see Figure 1). The whole process of the shown example is based on 12 transitions and 12+1=13 dialog states. The expert user started from the main menu and made with the ASCII-key 'd' the module 'data' active and with the ASCII-key 'a' the routine 'browse'. Pressing the function-key 'F3' he tried to reach a dialog state where he could change the actual data set, but this operation was only possible in the main menu. The system changed to the dialog state 's3' (wrong input state) and responded with the output message "Press space to continue." This message was incorrect implemented, so that the user tried unsuccessfully to leave the 'wrong input state'. Only when he 'found' the function-key 'F9', he could escape from 's3'. The wrong output message was the reason to press three times the space key '_'.
452
8.I s: main menu ascii key "d"
8.2 s: modul "data" ascii key "a"
8.3 s: routine "browse" function keg "3"
1.5 s: "wrong input" state ascii key "SPRCE" 8.4 s: "wrong input" state ascii keg "SPRCE" 3.3 s: "wrong input" state tabulator key
Figure 2. Model-l: the pure 'logical structure' of our example sequence in Figure 1.
3.8 s: "wrong input" state function key "2" 3,4 s: "wrong input" state gz
j¢
-
,%
g3
tabulator key 8.7 s: "wrong input" state carriage return 3.6 s: "wrong input" state ascii key "SPRCE" 4.8 s: "wrong input" state function key "9"
8.4 s: modul "data" ascii key "h" 8.1 s: main menu
Figure I. The original behavioural sequence of an expert with a relational database system (cf. [9]).
Figure 3. Model-2: the model-1 was supplemented with S-elements as goals (gl) ..... (g3), as memory places (ml), and as extinction places (e 1), (e2)..... (e4).
Mental models consist of three different types of knowledge: (1) the 'pure' logical structure of the task, (2.) the sequential structure of all goals, and (3.) the temporal structure of all operations. The pure logical structure is automatically extracted with our tool AMME (cf. [ 11 ]). This net is called model-1. Model-1 does not contain any knowledge about goals and time. For a first attempt to simulate the sequence in Figure 1 we build model-2 with goals and memory elements. In model-2 (see Figure 3) the S-element (ml) is free for 'recall' of the solution. The three S-elements (gl) ..... (g3) are included in model-2 to simulate the sequential goal structure of the valid transitions outside state 's3'. The four marked S-elements (e 1) ..... (e4) are included in the model to simulate the 'extinction'-rate as one aspect of the memory. The number of marks is positively correlated with the extinction-rate of the corresponding transition. If the transition is so often made active as marks are on the S-element then this transition can not be fired anymore. This consequence is a model of learning caused by unsuccessful trials. Model-3 has a complete goal structure for all transitions (see Figure 4). We combined the
453 goal-element of transition'_' with the S-element for the extinction-rate. The activation of transition 'F3' fills this S-element with an additional mark, so that we have then the same extinction-rate as in model-2. Model-4 has exactly the same structure as model-3 (cf. Figure 5). The only difference between model-3 and model-4 is the time delayed transition 'F2'. This delay increases the probability that the transitions '_' and 'TAB' exceeds completely their extinctionrate before transition 'F2' is fired.
Figure 4. Model-3" this Petri-net is equivalent to model-2 with four additional goals to simulate the sequential structure.
4.
Figure 5. Model-4: this Petri-net is equivalent to model-3 with an additional time delay of transition 'F2'.
VALIDATION OF THE FOUR MENTAL MODELS
To validate the four different mental models, a simulation study was carried out. With a Petri-net simulator each of our four models was implemented, marked and executed. We generated different task solving sequences with each model. To estimate the difference between the original sequence (cf. Figure 1) and each simulated sequence, we used the following procedure: 1. We numbered consecutively all operations ('transitions', respectively) in Figure 1 ['d'= '1', 'a' = '2', 'F3' = '3', .... 'h' = '12']. The number R is the rank-position of each transition t in the original sequence. 2. We attached these numbers to all generated transitions (t) of each simulated sequence. For example, the shortest sequence we found, was generated with model-1: ['d', 'h']. The rankpositions R of these both transitions are: ['1', '12'] (compared with the original sequence). 3. We calculated a 'similarity ratio' (SR) as follows:
[{Nt~II 'N°rg I I}/N'i:~100%
SR = 1 -
Rorg,t- Rsim,t + Z max Rorg = Nam+l SR is a sufficient measure of the difference between the simulated sequence and the original sequence. N is the number of all transitions in a sequence. The maximum of Rorg is equal to Norg (Norg in Figure 1 is 12). SR is only valid for simulated sequences that fulfil the following condition: Nsim < Norg. For example, SR of the shortest sequence ['1', '12'] is 10%. 4. We averaged the similarity ratios of all simulated sequences per model (see Table 1). The results in Table 1 show that with increasing complexity of the mental model (MC) the similarity ratio (SR) tends to 100%. Interesting is to note that the structure of model-3 does not distinguish from the structure of model-4. Only the delayed transition 'F2' increases SR by 9%. Additionally to the positive correlation between MC and SR we can see that the variance (measured by the standard deviation) decreases continually. This result indicates that the predictive power increases from model-1 to model-4. rg
"
454 Table 1. The model complexity (MC) and similarity ratios (SR) of model-1, -2,-3, and -4. MC: absolute value SR: mean SR: standard deviation SR: minimum.., maximum number of simulated sequences
5.
Model- 1 6 43 % + 33 % 10% ... 83% 11
Model-2 8 57 % + 23 % 32% ... 93% 12
Model-3 13 86 % + 12 % 68% ... 94% 8
Model-4 13 95 % + 1% 94% ... 96% 8
D I S C U S S I O N AND C O N C L U S I O N
Three different results are important: (1.) Our assumption that 'learning how to solve a specific task with a given system means that behavioural complexity decreases and cognitive complexity increases' seems to be correct. (2.) We can conclude form the validation results that we must discriminate between the logical decision structure of a task, the sequential goal structure, and the temporal structure. The logical structure of a task can be extracted automatically with our analysing tool AMME, and the complexity of this logical structure can be measured with the McCabe-measure. The temporal structure can be measured with the planning time per operation and transition, respectively. Learning the temporal structure means to accelerate the task solving process. (3.) The results of the goal structures of model-3 and -4 show us that we must take into consideration the knowledge of unsuccessful attempts. Our hypothesis is that the majority of our long-term knowledge consist of inhibitions. To model this aspect means to change the type of the arc between (e 1) - > [_] from an activator to an inhibitor in model-4. In our modelling approach we can not neglect knowledge of unsuccessful trials. One psychological dimension of the goal and time structure seems to be self-confident in doing the fight things at the right time avoiding all unsuccessful ways tried before. REFERENCES: [1] R. Bauman and T.A. Turano, Production based language simulation of Petri nets. Simulation 47 (1986) 191-198. [2] J. Carroll and J. Reitman-Olson, Mental models in human-computer interaction. In: M. Helander, Ed., Handbook of Human-Computer Interaction (North-Holland, 1991, pp. 45-65). [3] H.J. Genrich, K. Lautenbach and P. S. Thiagarajan, Elements of general net theory. In: W. Bauer, Ed., Lecture Notes in Computer Science 84 'Net Theory and Applications' (Springer, 1980, pp. 21-163). [4] D.E. Kieras and P.G. Poison, An approach to the formal analysis of user complexity, International Journal of ManMachine Studies 22 (1985) 365-394. [5] J. Laird, A. Newell and P. Rosenbloom, SOAR: An architecture for general intelligence. Artificial Intelligence 33 (1987) 1-64. [6] T. McCabe, A complexity measure, IEEE Transactions on Software Engineering, SE-2 (1976) 308-320. [7] A. Newell, Unified theories of cognition and the role of SOAR. In: J. Michon and A. AkyiJrek, Eds., SOAR: A Cognitive Architecture in Perspective (Kluwer, 1992, pp. 25-79). [8] C.A. Petri, Introduction to general net theory, pp. 1-19. In: W. Bauer, Ed., Lecture Notes in Computer Science 'Net Theory and Applications' (Springer, 1980). [9] M. Rauterberg, An empirical comparison of menu-selection (CUI) and desktop (GUI) computer programs carried out by beginners and experts. Behaviour and Information Technology 11 (1992) 227-236. [10] M. Rauterberg, A method of a quantitative measurement of cognitive complexity. In: G.C. van der Veer, M.J. Tauber, S. Bagnara and A. Antalovits, Eds., Human-Computer Interaction: Tasks and Organisation (CUD, Roma 1992, pp. 295-307). [11] M. Rauterberg, AMME: an automatic mental model evaluation to analyze user behaviour traced in a finite, discrete state space. Ergonomics36 (1993) 1369-1380. [12] G. Van der Veer, S. Guest, P. Haselager, P. Innocent, E. McDaid, L. Oesterreicher, M. Tauber, U. Vos and Y. Waern, Desiging for the mental model: an interdisziplinary approach to the definition of a user interface for electronic mail systems. In: D. Ackermann and M. Tauber, Eds., Mental Models and Human-Computer Interaction 1 (North-Holland, 1990, pp. 253-288).
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
455
M o d e l i n g and Simulation o f H u m a n Operator in Mental T a s k H a n d l i n g Qualities Celestine A. Ntuen Department of Industrial Engineering, North Carolina A&T State University, Greensboro, NC 27411
Effective human control of a complex system depends in part on the design of decision aids that have (high-level) models of human mental processes. Since the human operator utilizes the computer as a medium of interaction with the task environment, it is important that tightly coupled and interoperable human mental models be incorporated into HCI. This paper reports on a pilot study of simulating mental task performance in a desk top HCI platform.
1. INTRODUCTION Researchers who advocate the use of mental models in HCI design are often trapped in two epistemological myths. First, the notion of representing tasks as concepts defies the physical laws with which systems operate. Second, the use of mental models and mental tasks interchangeably reflects a lack of understanding between derived representation of knowledge about how humans perceive the objective world and how actions are used to control the tasks in the real world. Unequivocally, Gentner and Stevens (1983) have noted that "mental models research is fundamentally human knowledge about the world". Mental tasks on the hand are more akin to memory events such as perception and cognition. Humans utilize memory events when executing mental tasks. The memory structure have been shown by psychologists and cognitive scientists (Anderson, 1974, 1982) to contain information symbologies used to activate various forms of stimuli generated by the sensory o r g a n s - auditory, tactile, visual, etc. This clearly demands that HCI should be designed to complement such dependency.
2. MENTAL TASK HANDLING QUALITITES (MTHQ) A mental task handling quality is the study of perception and control of memorybased tasks. These tasks may include event prediction, pattem analysis (recognition, association, discrimination) and information processing (data fusion, data estimation, etc.) Mental tasks of the human operator can be conceptualized at four levels of memory events. These are:
456 1. Visual memory. This consists of chunks of tasks for pattern recognition, pattern matching, pattern discrimination and pattern association. 2. Spatial memory.. This consists of spatio-temporal tasks such as understanding geometric position and orientation and object dimensionability (size, shape, etc.). 3. Abstract memory. This consist of abstract tasks which hold facets of information in various forms such as symbologies, graphics and texts. 4. Decision memory.. This consists of tasks for decision, judgment, and inference w
3. MODELING AND SIMULATION OF MTHQ Mental handling quality models can be coupled with the human operator model through optimal control model as shown in Figure 1. Ntuen and Fang (1994) have shown that Fitt's law of stimulus-response compatibility can be used as quasi-linear models of the human operator (Sutton, 1990). Some of the human describing functions are available. . . . . . . . . .
~.~.~-~.o~r..o;~
. . . . ~,~oT,o~.s: . . . . . . . . . . .
{COGNITION}
/~
ACTION}
Figure 1" A representative systemfor application of optimal control model for mental task process simulation. In the HCI, we need to conduct experiments to derive tasks describing functions. Thus, we can consider the computer as the plant to be controlled.
4. EXPERIMENT WITH MTHQ USING HCI In our pilot experiment, we study MTHQ through perceptual/cognitive control of actions. The perceptual tasks consist of the use of graphical symbologies and a TIM (Textin-menu) box; control tasks require the use of a Microsoft mouse system and keyboards (functional keys); the actions to be performed were cognate tasks. Seven graduate students (considered "experts") and five undergraduate freshmen ("novices") with age ranges between 17.5 years to 26 years took part in the study; participation was voluntary. In order to induce mental tasks, visual stimuli were associated with the actions as follows" {Blue} = Save file & continue {Blue, Yellow} = {Save file, exit to DOS} {Yellow, Red} = {Cut and paste document} {Red} = {Cut document} {Green} = {Print current document} {Yellow, Green} = {Print review document} Document typing tasks were assigned to the students on Word Perfect (WP6.0) Windows Version. Each subiect took part in a two-hour review of the visual stimuli. Subjects
457 performed the desired tasks upon the stimuli emittance. Each stimulus was randomly generated from exponential distribution function with mean emittance rate (MER) of 40 seconds. The subjects reaction to light stimulus closely matches the Fitt's law of S-R compatibility and allows for optimal control simulation of the human operator in MTHQ.
5. SAMPLE EXPERIMENTAL RESULTS AND CONCLUSION The preliminary experiment was evaluated in terms of human errors (Booth, The aggregate results are shown in Table-1.
1990).
Table 1" Human Errors in Perceptual/Cognitive Control of Actions in HCI Mental Handling GM Selected Error Models Automacity Familiar ShortCut Signal Sustitution Task-Similarity Signal Interference
TIMS - M
GK
TIMS - K
]
Expert
Novice
Expert
Novice
Expert
Novice
Expert
Novice
0.045 0.003
0.090 0.106
0.008 0.006
0.114 0.121
0.08 0.100
0.098 0.137
0.052 0.0910.
0.115 0.106
0.028
0.088
0.I01
0.096
0.052
0.116
0.058
0.008
0.092 0.03
0.115 0.0431
0.0429
0.078 0.112
0.008 0.041
0.082 0.111
0.07 0.09
0.045 0.011
0.011
GM = Graphic - Mouse Control. GK = Graphics-- Keyboard Control, TIMS - M = Text-in-menu and Mouse Control, TIMS - K = Text-in-menu and keyboard control. The above results show human errors associated with mental tasks. A separate paper that reports on the response times and the use of optimal control models to predict human performance contains all the details of the experiment. The preliminary results show that as advanced HCIs are being developed, the human mental tasks should be studied and validated before developing related mental models.
REFERENCES 1. 2. 3. 4.
5.
Gentner, D. & Stevens, A. L. (1983). Mental Models, Hillsdale, N. J.: Lawrence Erlbaum. Anderson, J. R. (1974), Retrieval of prepositional information from long-term memory, Cognitive Psychology, 6, 471-474. Anderson, J. R. (1982). Acquisition of cognitive skills, Psychological Review, 89, 369-496. Ntuen, C. A. & Fang, J. (1994). A simulation model of the adaptive human operator, First Industry/Academy Symposium on Research for Future Supersonic & Hypersonic Vehicles, Vol.l., Homaifar, A & Kelly, J. C., Eds), Albuquerque, New Mexico: TSI Press, 466-471. Sutton, R. (1990). Modeling Human Operators in Control System Design, New York: John Wiley.
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
The interface testing
improvement
for the creative thinking
459
computer
Chislov a, V.L. Maloryan a, I.A. Pc~ozovskayaa, G.V. Shtakser a, A.I. Uyemova, ~• . .Zakharchenko a and M. Athoussaki aSouth-Ukrainian Pedagogical University, 26 Staroportofrankovskaya, 270020 Odessa, Ukraine bSIEM Ltd., Markou Mousouri 25 - Mets, Athens, 11636, Greece Abstract The interface design is one of significant components which the successful creation of computer testing programs depends from. This is especially important for children's creative thinking testing. A new approach to this problem that takes into account modern tendencies in interfaces design and the Language of Ternary Description [ 1] is supposed. 1. COMPUTER T E S T I N G OF CREATIVE T H I N K I N G Modem tendencies of human-computer interfaces design are connected with a maximal simplification of problems for human-operator in HCI process. These tendencies are realized by a gradual transition from a conversational style of interaction to immediate operator activity in a virtual world and by a wide using of the principle of direct manipulation by concrete objects. It is especially important for the creative thinking computer testing and particularly for children testing. A researcher must stimulate a strong interest to testing problems or puzzles among children. Another word he must create a good motivation for child's thinking. It goes easier if a test puzzle is formulated in the game form and if a child can manipulate objects of his/her interest directly. This situation will be realized if children will perceive testing programs as usual computer games. It is necessary also in order to have a pure psychological experiment. 1.1. Some features of child's logic The problem of a child's logic is connected with a paradox. On the one hand, the formal-logical thinking of a child is formed after 11-12 years. But on the other hand, a maximum of a human creative abilities achieve just before 11 years. From the our point of view a relevant solution of this paradox is based on various logical systems construction. Ideally it is necessary to learn children to create a free-reconstruct logic frame which can be transformed into the formal systems with various logic schemes in correspondence with own children's logic. We suppose that a child thinks logically. But his/her used logic relations are weak. Therefore usual psychological tests do not reveal real creative abilities of a child if only logic components of child's thinking is considered.
460 Some psychologists, for example J. Piaget [2], believed that children till 11 years can not use and can not understand, for instance, the set conception. But Piaget did not know a relevant logic which is connected with child's thinking and appeared later. It is a logic theory which is called the Language of the Ternary Description (LTD) [ 1,3 ]. The traditional logic of predicates is based of a mathematical deductions. But children think in the frame of more weak connections between things. It is more close to the natural language grammar rules. It's known that a child not only easily learns different languages but also create own language. K. Chukovskij collected many examples of children's creative activities in the age from 2 to 5 in [4]. One of the most interesting results is following: many children think up a "new" words that were used in Russia in ancient times or correspond to some dialects today. So children reveal theirs abilities by using logic which is close to natural language rules. 1.2. The Language of Ternary Description (LTD) We shall show that it can be more relevant for our aim the nontraditional logic system as LTD [ 1, 3]. Structures of this logic is approached to structures of the natural language. This logic can describe the children's thinking. The basis of LTD consists of two group categories (at three in each): "things, properties, relations" and "definite, indefinite, arbitrary". With the help of this concepts LTD's structures allow to underline the most essential objects or its elements and visa versa - hide inessential or dangerous ones. Moreover objects can be connected in different hierarchical chains by means of properties and relations. LTD is an alternative to logic of predicates, oriented at mathematical relations, because permits to describe relations, which can be often found in usual situations. Nevertheless, this logic system has own axioms, rules and theorems. With the help of categories "things, properties, relations" we construct well-formed formulae (WFF): 1) (X) Y means that a thing X has a property Y; 2) Y (X) means that a thing X has a relation Y; 3) [ (X)Y] means a thing X with a property Y ( contrary WFF (1) which describes propositions (relations) describes things); 4) [Y (X) ] means a thing X with a relation Y. The opposite direction of the predication relation is expressed by the next four WFF: 5) (X*) Y means that a property Y is attributed to a thing X; 6) Y (*X) means that a relation Y belong to a thing X; 7) [ (X*) Y] means a property Y attributing to a thing X; 8) [Y (*X) ] means a relation Y belonging to a thing X. Last WFF is a free list: 9) X,Y. The second group of categories is used in order to introduce elementary objects which substitute into a WFF instead of X and Y. Objects are denoted: "A" is arbitrary object, "a" is indefinite object, "t" is definite object. These elementary objects can be combined at a various manner. As a result we obtain different formal objects: identity, implications etc. Combined properties of these objects permit children to create theirs own new objects as they do it in the frame of natural language with "definite", "indefinite" and "arbitrary" concepts.
461 Besides mentioned properties LTD permit to decide problems in HCI's frame. Modern tests control only how a child master adult logic. 2. NEW T E S T I N G M E T H O D S 2.1. General description From the point of the LTD view we can suppose a new version of the creative abilities testing system. First, it's necessary to suggest a set of testing games that may be represented by its metaphoric pictograms for a free child choice. The menu must give to children a wide possibilities of the computer games choice. Moreover the ideal variant is to design such virtual worlds which can contain the orientation items for children. This position will give an additional information about the child abilities and the interests of the child. Second, we would like to open for children the possibilities of the independent investigation of the game rules and its peculiarities. A part of the possible operations and/or the game rules can be unknown for the child and another part can be arbitrary with accordance of the child wishes. Third, the test game can have a solution based upon the LTD logic or another logic that similar to the "child's" logic. Undoubtedly the new system saves all main properties of the previous system [5] and allows to demonstrate the creative abilities of the tested child more completely. 2.2. An example of creative thinking test For example, consider one of the creative thinking tests from [5]. This is one of River Crossing tests. The game rules are very simple. Three knights with servants have to cross a river in a boat that can hold not more than two people. The boat can't float itself. The game aim is to transport all personages on the other bank. The main rule says: "Any servant has no fight to stay in the presence of a strange knight without his master". This puzzle has natural computer realization in accordance with the above mentioned principles of an interface design. The graphical imagines' motion is controlled by the mouse. Researcher introduce the child with the all game rules before the test starts. Then the child plays, his/her decisions store on the disk and thinking parameters are calculated by the special program. But the thinking model [5] uses a traditional, "adult", logic. 2.3. More general examples A human operator influences on a computer processes but also he/she feels a strong influence from a computer on himself. A logic component of the operator thinking feels especially strong stress. The thinking style of human operator accepts new features. Therefore, designing HCI, we need a logic of the human operating in visual and natural situations. The traditional formal logic systems also are unsuitable to this aim. The nontraditional logic system as LTD, can be more relevant to it.
462 For more general example, we consider a typical file system as MS DOS. At the first approximation one can pick out the main categories in the such manner: "things" are files, "properties" are the file descriptors, "relations" are defined by directories. From this point of view, the typical file system has the next drawbacks: - it is impossible include a file into two different independent relations without copying; the categories "indefinite" and "arbitrary" are unexpressable; - properties from the fixed set and one unfixed are effectively expressible only. Our analysis exposes the directions of the file system improvement. Particularly we would like to express any properties and relations between files. Also if we introduce concepts "indefinite file" and "arbitrary file" then a number of operator decisions during a session are decreases. Now we suppose an idea of the file system without above mentioned defects. Any file must to have two lists for its properties and relations. We allow cross references in these lists. Therefore an operator can express properties of relations, relations between properties, any properties and any relations of files and so on. Moreover we allow "indefinite" objects and "arbitrary" objects. So LTD approach can give new ideas in the file system design area too. -
3. C O N C L U S I O N It was demonstrated new ideas in the interface design for creative thinking computer tests development. Similar approach may be applied to other problems of Human-Computer Interaction. It is possible that Language of Ternary Description and corresponding Logic can be useful for many areas of HC1. REFERENCES 1. A.I. Uyemov Fundamental features of the language of the Ternary Description as a logical formalism in the systems analysis / / 8 Congress LMPS, vol.5, part 1, sect. 1-6.Moscow, 1987.- p.p. 337-340. 2. J. Piaget The psychology of intelligence.- London, Routledge and Kegan Paul, 1950. 3. A.I. Uyemov A Formalism for Parametric General Systems T h e o r y / / S y s t e m research. Methodological problems. Yearbook 1984.- Moscow, Nauka, 1984.- p.p. 152-180. 4. K.I. Chukovskij From two to five.- Moscow, Det. Lit., 1986. 5. A.E. Kiv, V.G. Orishchenko, I.A. Polozovskaya, I.G.Zakharchenko Computer Modelling of the Learning Organization / / Abridged Proceedings of 4-th International Conference Human Aspects of Advanced Manufacturing and Hybrid Automation, Manchester, UK, 1994
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
463
Evaluating Human Operator Models in Tool-based User Interface Design Maria Athousaki SIEM Ltd, M. Mousourou 25, Metz, 116 36 Athens, Greece
This paper describes the context of use of a software tool aiming to provide assistance for the ergonomic design of man-machine interfaces as well as to facilitate usability engineering. The tool is currently being developed by SIEM Ltd and constitutes one of the company' s lines of activities towards innovative solutions in the area of user interface design and evaluation. The novelty of the tool being described is that it relies upon encoded knowledge of the human operator (user profile, mental model) and the tasks to be performed, to construct prototypical implementations of alternative potential designs. These, subsequently become subject to evaluation and usability testing.
1. INTRODUCTION The role of ergonomic design for man-machine interaction is widely acknowledged as critical for the overall performance of interactive systems. There is, consequently, an ever increasing need for promoting and supporting ergonomic design to arrive at usable systems that take into account the human operator characteristics, requirements and preferences. In recent years, there has been an increasing interest in the development of methodologies and frameworks for evaluating the usability of new products. The primary objective of these efforts has been to develop methods and tools for the specification and measurement of usability [1 ]. As a result, several methodologies, guidelines and checklists have been proposed by industries, research practitioners and international standards committees. Some of the studies which are frequently cited in the relevant literature include the work by Smith et al [2], which has been used as the basis to arrive at a more comprehensive set of guidelines published by ISO as international recommendations. Relevant checklists include EVADIS [3], [4] as well as the work of Ravden et al [5]. More recently, the MUSIC project [1], in the context of the ESPRIT programme of the European Commission has worked in conjunction with industry to produce a collection of methods and tools for the specification and measurement of usability. In this context, measuring usability during design aims to: (i) ensure that the final product reaches at least a minimum required level of usability; (ii) provide feedback during design on the extent to which the objectives are being met; and (iii) identify potential usability shortcomings of the product. Evaluation of the usability of interactive systems can be used to assist the design of ergonomic products, taking into consideration the human operator and his mental model (user-centered approach). Even today, design decisions largely depend on the designer's interpretations, beliefs and assumptions regarding the human operator of a system. According
464 to Christie et al [8], the development of appropriate human operator models regarding manmachine interaction can provide an appropriate basis for evaluating and implementing interactive systems. Such models should be sufficiently "rich" and contain information at different levels (lexical, syntactic, semantic and conceptual), considering the human operator as a psycho-physiological and cognitive system. 2. SUPPORTING E R G O N O M I C DESIGN T H R O U G H USABILITY ENGINEERING SERVICES
Usability evaluation at SIEM Ltd, is currently at the core of the company's activities. In the past, there have been a number of efforts towards the experimental evaluation of user performance in various tasks, such as CAD tasks [6]. As the role of man-machine interaction becomes increasingly more critical for the overall performance of an interactive system, there is a growing need for the development of methods and tools to support the measurement of usability of such systems. Towards this direction, SIEM Ltd seeks to develop innovative solutions to address the numerous problems associated with determining various aspects of usability (e.g. measurement, analysis of usability results), by building upon recent experience and the state of the art. One of the current activities of the company, is targeted towards the development of software tools for the design, rapid prototyping and evaluation of man-machine interfaces. Our objective is to incrementally integrate in one software architecture, a number of critical elements as they may be related to the human operator (e.g. user profile, mental model) and the tasks under consideration, in an attempt to determine and assess the usability of potential designs. At SIEM Ltd, our current objective towards measuring the usability of man-machine interfaces of interactive systems is facilitated through the process depicted in the diagram of Figure 1. More specifically, we are developing a software tool, called EDS (Ergonomic Design Support), which can quickly produce a mock up of different potential designs, given knowledge about the human operator (e.g. characteristics, preferences, knowledge of the domain of discourse) and the task(s) that this operator is to accomplish with the user interface. These mock-ups are subsequently subject to laboratory evaluation by establishing appropriate measurements as they are needed by the task(s) being considered. The usability results are interpreted in cooperation with the customer and are used in two different ways. First, they are translated into a number of recommendations for the customer, depicting specific guidelines for optimal, ergonomic design of the user interface. Secondly, they are fed into the system as past usability data, to extend EDS's functionality and decision making process. In the following, the diagram of Figure 1 is elaborated, exemplifying the way in which usability measurement during early design is serviced. 3. OVERVIEW OF EDS EDS aims to support the ergonomic design, simulation and prototypical implementation of man-machine interfaces to interactive systems, facilitating usability evaluation at design time. The tool which is currently under development, provides support for encoding operator
465 models, alternative dialogue syntax suitable for different operators, as well as usabilityspecific data; all these constitute the aggregation policies towards prototypical designs.
~
~ Task structur'e'sI< I
translates
equirements
A & B->C B & Z->G
~i
J Usability Testing I
Figure 1" Supporting ergonomic design: Usability engineering services using EDS The novelty of the tool which makes it attractive in the context of usability measurement is that it is geared towards experimentation with altemative operator models, task structures and measurement criteria. In other words, the usability engineer is allowed to experiment with the encoding of altemative models of human operators and usability measures to determine the value of different designs in various contexts and scenarios of use. A prototypical design is produced by fitting the task to the user. This is achieved by tailoring the dialogue structure and lexical interface design to the operator's characteristics and knowledge of the application domain, by applying rules and usability-related aggregation policies. Aggregation policies are specified by the usability engineer, during design, as metarules and draw upon past experience and cases that may influence or determine choices (of dialogue structure, interaction style, choice of interaction object) based on usability criteria (e.g. time taken to complete a task). For example, let us assume that the objective is to measure the usability of a potential design with respect to the time taken by the operator to complete the task. Furthermore, we assume the hypothetical operator model which depicts the information shown in Figure 2 (in a declarative notation).
466
o p e r a tor( n u m e ric_i nput, [ knows ( value_range, [1,25] ) , knows ( c o n c e p t , [ potentiometer,gauge] familiar ( device , [keyboard , mouse
), ])])
Figure 2" Instance of the operator's model The operator model depicted in Figure 2 declares that for a given task (e.g. entry of a numeric value represented by the constant n u m e r i c _ i n p u t ) the following hold: (i) the current operator knows the value range from which input is to be accomplished; (ii) the current operator knows the concepts of a potentiometer and gauge; (iii) the current operator is familiar with the keyboard and mouse input devices. In order for EDS to propose a design suitable for a numeric input task and the current human operator, it initiates a search to retrieve past usability data (if any) in which an interface design has been developed for a numeric input task. Usability data is encoded into the system as declarative statements depicting average test results such as those indicated in the example of Figure 3. case ( [criterion ( t i m e _ c o m p l e t i o n ) ] , numeric_entry, [known ( 2 5 s e c , [ p o t e n t i o m e t e r , keyboard ], user ( [ knows ( v a l u e _ r a n g e , _ ) , knows ( c o n c e p t , p o t e n t i o m e t e r ) , familiar( d e v i c e , [keyboard , mouse]) type (manager) . . . . ] ]) . case ( [criterion (time_completion) ] , n u m e r i c _ e n t r y , [known ( 4 2 s e c , [ g a u g e , m o u s e ] , user( [knows ( v a l u e _ r a n g e ,_) , knows ( c o n c e p t , p o t e n t i o m e t e r ) , familiar( d e v i c e , [keyboard , mouse]) type (manager) . . . . ] ] )
,
,
Figure 3" Relevant cases retrieved from the usability knowledge base For the purposes of our example, the information depicted in Figure 3 identifies two past cases in which a task classified as n u m e r i c i n p u t was evaluated with the 'timecompletion' usability criterion (represented by the term cri teri on (t i m e _ c o m p I e t i on) of the listconstituting the firstargument of the predicate case). In the first case, the average time taken to complete the task was 25 sec, while this was achieved with the use of a potentiometer and a keyboard device (all these are explicitly declared by the predicate k n o w n ) . In addition, the operators who took part in the experiment have exhibited a minimum set of characteristics, declared as members of the list appearing as the argument of the predicate user (i.e. they all knew the value range from which input was to be accomplished, etc). In the second case, the time taken to complete a numeric input task was substantially larger, but this was achieved through the use of a gauge and the mouse, while the operators who participated in the experiment exhibited another set of characteristics, which is also declared in the list.
467 Past cases are retrieved by matching the current task under consideration (i.e. numeric input) and the characteristics of the current operator, against task and operator characteristics addressed in the past cases. Consequently, all cases thus retrieved are relevant and can be used to reason towards a conclusion. The reasoning process, selects the result which indicates the shortest time for task completion. In the case of our example, EDS would have selected the first case and it would configure the interface accordingly, since there is evidence to suggest that this configuration is likely to be more usable than any other. In the case that such evidence does not exist, then EDS will simply inform the usability engineer. From the example, it can be seen that as new usability data are interpreted and encoded as new knowledge in EDS, the decisions that the tool makes, take full account of this incrementality in the semantics depicted in the knowledge base. Consequently, the tool exhibits a certain degree of "dynamic behaviour" which is required when conducting usability measurements. In order for EDS to be able to retrieve cases which are relevant to the current usability test, it uses additional knowledge related to the task being addressed. More specifically, tasks are characterised in terms of requirements. A number of criteria have been devised which serve this objective. Indicative examples of task characterisation criteria include: whether or not the task is safety critical, frequency of the task, accuracy, type (e.g. input or output), feedback, etc. Each one of these characterisations are associated with rules in the rule base of EDS and may potentially trigger different user interface versions for the same task. For instance, if the task is safety critical a general flag is set which activates particular types of (initiation, interim and completion) feedback on different interaction objects.
3.1. Interface design using operator and task knowledge The tool which interprets human operator models and task descriptions is equipped with a number of rules which operate on previous knowledge, so as to determine some of the design aspects of the user interface. Typical rules encode information such as the following: If the task that the operator is to perform with the man-machine interface is safety critical and the operator has the ability of visual comprehension, then the feedback to be offered by an interaction object may be of a particular type (e.g. blinking), as opposed to the feedback required in case that the operator's ability of visual comprehension is limited or reduced. Rules such as the above, relate the context of interaction (e.g. what the user in doing, or what the user interface is trying to convey, at a given time), to the operator's abilities. Another set of rules is used to relate tasks to the operator' s cognitive load and mental model. For instance, if from the human operator model, it can be inferred that the operator is not familiar with the use of a bar chart for numeric entry then the selection of a bar chart for numeric input, should be avoided.
3.2. Usability analysis There are a number of general criteria which are frequently used to evaluate userinterfaces. Some of the most common ones which are usually employed when conducting user trials include [7]: time, speed of response, activity rate, etc; accuracy and errors; convenience and ease of use; comfort; satisfaction; physical health and safety; amount achieved or accomplished; job satisfaction; etc. Usability testing of a user interface usually involves user trials in which several of the above criteria may be used to evaluate and compare alternative
468 designs. The results are incorporated into the usability results knowledge base to facilitate ergonomic design of new interfaces, as described in Section 3. 6. DISCUSSION AND CONCLUSIONS
The preceding discussion identified the main building blocks of a tool aiming to provide assistance for the ergonomic design of man-machine interfaces. The rationale for building such a tool has been to support usability engineering as well as the incremental embodiment of the state of the art in HCI research and development (R&D), past experiences and practices, so as to enable SIEM Ltd to successfully address industrial needs. The commercial targets that SIEM Ltd aims to address include domains in which the ergonomic design of the manmachine interface is crucial to the overall system performance or to the wide diffusion and user acceptance of the system. Such domains include interfaces to public or private terminals and equipment (e.g. ATMs, photocopiers), but also market niches, such as the assistive technology industry, where terminal/application/service access problems confronting the socio-economic integration of people with disabilities, are currently predominant. The added value of the usability cycle is that the recommendations produced at the end of this cycle, can be incorporated and subsequently used to extend the capabilities of the EDS tool. In addition, a conscious effort is expended on collecting past data on usability problems and corresponding solutions, in an attempt to establish a pool of knowledge revealing past data and experience. SIEM Ltd, seeks to exploit the above usability cycle, in a number of domains and application areas. One such application domain is user interfaces in smart home environments, accessible to disabled and elderly users. Currently, the company participates in European collaborative R&D activities related to identifying user requirements in smart home environments and conducting usability tests (SIEM Ltd is a partner in the CASA TP 1068 project of the TIDE Programme of the European Commission). REFERENCES
1. N. Bevan and M. Macleod, Usability measurement in context, Behaviour & Information Technology, 3(1&2), 132-145, 1994. 2. L. S. Smith and J. N. Mosier, Guidelines for Designing User Interface Software, MITRE Corporation, Bedford, MA, USA, 1986. 3. R. Oppermann, B. Murchner, M. Paetau, M. Pieper, H. Simm, I. Stellmacher, Evaluation of Dialogue Systems, GMD, St Augustin, Germany, 1989. 4. H. Reiterer, EVADIS II: A new method to evaluate user interfaces, in D. Diaper (ed.), People and Computers, Vol. VII, Cambridge University Press, 1992. 5. S. Ravden, G. Johnson, Evaluating the Usability of Human-Computer Interfaces, Chichester: Ellis Horwood, 1989. 6. L. Laios and M. Athousaki, An Experimental Evaluation of User Performance in CAD tasks, Fourth International Conference on Hunan aspects of advanced manufacturing & hybrid automation, July 6-8, 1994, Manchester, England. 7. I. McClelland, Product assessmentand user trials, in J. R. Wilson and E. N. Corlett (Eds), Evaluation of Human Work, Taylor and Francis, 1990, pp.231-232. 8. B. Christie (ed), Human factors of the user-system interface, North-Holland, 1985.
III. 16 Modeling 1
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
471
Associative User Modeling: A Neural Network Approach Qiyang Chen a and A. F. Norcio b aDepartment of Computer and Information Sciences Towson State University, Towson, MD 21204, U. S. A bDepartment of Information Systems University of Maryland, Baltimore County, MD 21228, U. S. A. This paper presents and discusses an approach of user modeling. A set of neural networks is utilized to store, maintain and infer users' task-related characteristics. Such networks function as associative memories that can capture the causal relationships among users' characteristics for the system adaptation. It is suggested that this approach can be expected to overcome some inherent problems of the conventional stereotyping approaches in terms of pattern recognition and classification. It can also avoid the complexity of truth maintenance in default reasoning that is required in previously known stereotyping approaches. 1. INTRODUCTION User models have become important components of adaptive human-computer interfaces. It is recognized that a system will be able to exhibit flexible user-oriented behavior only if it has access to a model of user which consists assumptions about users' characteristics regarding certain task being performed. These characteristics may vary depending on the types of tasks. Usually, they are related to a user's plans, goal, domain knowledge as well as cognitive preferences [ 1]. There are different taxonomies about these characteristics depending on the time period while they hold, the way by which their are elicited and represented as well as the degree by which they are specified [2]. It has been a common practice to use a set of predefined assumptions to initialize the system's beliefs about its users and the tasks they are performing. It is usually referred as stereotype approach. The pre-defined stereotypical knowledge is organized into a generalization hierarchy in which the stereotypes inherit knowledge from their ancestors. The modeling process proceeds with stereotype assignment in terms of default reasoning that allows the model to retain the stereotypical knowledge about a user in the absence of evidence to the contrary. Although the stereotype approach provides a simple way to initialize the modeling process and was successful in some applications, we belief that this approach limits the representation power of a user model in following aspects. Since the reasoning is conducted with extensive default assumptions that may conflict with the new evidence obtained as the interaction progresses, the revision of stereotypical knowledge is necessary to handle the inconsistencies. A common suggestion is to use dependency-directed backtracking process to accomplish the truth maintenance which examines one piece of evidence at a time in a nonmonotonic way [3]. This approach is often inefficient
472 and lacks the ability to detect noisy or inconsistent information that should be ignored [4]. Therefore, it is very possible that current effort of maintaining consistency may bring further conflicts in the subsequent interaction. Thus, model construction may fall into a dilemma where a non-monotonic process of reconciling conflicts is frequently involved and eventually no decision can be made after a period of interaction [5]. In addition, the pre-defined hierarchy confines the system beliefs within each stereotype and can be only inherited by the descendant stereotypes. Therefore, it is hard to effectively update those system beliefs that are no longer significant in the context of task performance. Also, since a user may fail to fit any set of stereotypes, so that the modeling process fails to associate any system decision to that user. In such situation, however, some of the assumptions distributed among the stereotypes might be still useful for characterizing that user. In this sense, the hierarchical structure of stereotyping approaches is limited in the degree of individualizing a user. We suggest that the users' task-related information should be examined in terms of pattern recognition and classification so that the interface system can establish a complete and consistent profiles about users in order to exhibit the cooperative behavior. 2. ASSOCIATIVE USER MODEL
2.1. Basic concepts We propose that associative networks can be used as an efficient mechanism for user modeling. In associative network based user modeling process, the stereotypical knowledge is organized as a set of patterns. Since the system's beliefs about a user should be determined through the context of task performance, fragmented pieces of observation may not bring any meaningful implication. In addition, the observed information about user's characteristics may be mixed with noise or inconsistencies. Therefore, all the aspects of the user's performance patterns have to be examined before any system decision can be made. In other words, user modeling process should be conducted in terms of pattern recognition, which requires that the modeling system has the capability of fault tolerance, graceful degradation, and signal enhancement. Therefore, the neural network techniques become natural tools for the implementation. Whereas, as suggested in above section, the conventional stereotyping approaches, in which the inference proceeds a step at a time through sequential logic, may become seriously inadequate for processing pattern-formatted knowledge especially when there are incomplete, noisy or inconsistent observations involved [6]. There are several paradigms of associative memories that can capture the associations between input pattern and output patterns despite incomplete or inconsistent inputs. In our approach, an associative memory is implemented by a single set of interconnected units. Each unit represents an assumption or an attribute as the system's stereotypical knowledge about users. All assumptions consist of a universal stereotype. The modeling process extracts some of assumptions to dynamically form a unique stereotype that fits a particular user. Unlike the hierarchical stereotyping approaches that only associate the user with a single or a set of stereotypes, an associative user model includes various assumptions associated to a user. In other words, associative user modeling proceeds at the level of the assumption rather than the level of the stereotype. In this approach, all assumptions of the universal stereotype are considered to be
473 relevant to each other in a spectrum which is valued from negative to positive (i. e., from contrary, via irrelevant, to consistent). Thus, it overcomes the limitation of the hierarchical stereotypes which is unable to extract assumptions from different stereotype structures to form a new profile about the current user and task. Therefore, this approach has a better ability to personalize a user. To test the proposed approach, several network models are used for modeling users' genetic domain knowledge in the field of database programming. 2.2. P a t t e r n a s s o c i a t o r s
A bidirection linear associator (BLA)is used as a knowledge processing paradigm to capture the causal relationships between an arbitrary number of assumptions. The associations among the assumptions are weighted under certain conditions. Figure 1 shows a structure of such paradigm. Once a user's input from the dialogue channel is observed, it forms an input to BLA. The modeling process is conducted by propagating the activation level throughout the network, to associate this input with an output pattern. This output pattern is considered as the current system beliefs about that user. This process simulates the behavior of default reasoning.
II o Assumptions
wij: weightfromnode i to nodej Figure 1 The structure of BLA The kernel part of the BLA is the weight matrix. The data for constructing this matrix is collected by the card sorting method [7]. Twenty database programming concepts (refer to Table 1) are used in the test. 49 undergraduate students who are majoring either information systems or computer sciences participated in the data collection procedure. Each subject is asked to create a weight matrix expressing the relationships among the relevant concepts. Given an assumption that a user knows (or not know) a concept, subjects are asked to choose other possible concepts the user might also know (or not know) and assign the belief values to the corresponding cells. For example, if it is believed that a user who knows concept x may also know concept y, then fill "1" into the corresponding cell (x,y). Subjects may use any number between -1 and 1 to characterize such beliefs. A simple average function is used to integrate the matrices from the subjects. No. 1 2 3 4 5
concept integer real loop array subroutine
No. concept No. concept 6 local variable 11 recursive 7 tree 12 record 8 stack 13 index 9 inheritance 14 weakentity 10 interrupt 15 dataintegrity Table 1. Concept index
16 17 18 19 20
No. concept FD 3NF concurrence locking NP-complete
474 A linear propagation activation algorithm is used: 1. Initialize weight matrix M M = ( wij ) n ' n , n is the total number of nodes. 2. Apply input vector V ( v 1, v 2 ..... v n ) on M to produce a stimuli vector V': n
v'= i=l
where fi (w × w u) = v,' =
f -1 1
if v i ' < - l / 2 if v~' > 1 / 2
0 others 3. a) If the network converges ( i.e., V '= V, ), stop. b) If the V' is previously created, set V' = V' u V (logic union) stop c) otherwise set V'= V, go to 2. 110 different input patterns are tested, including the input patterns that include inconsistent concepts (e.g. for same concept, both known and unknown nodes are fired). 100% of the output patterns satisfied the conditions: (a) the advanced concepts in the input yield less advanced concepts, and (b) the inconsistent input does not produce inconsistent output. 2.3. Feedforward Model A feedforward network trained by back-propagation method is also used in the study. This network (BP) is used to generalize the associative knowledge stored in BLA. The training data comes from BLA's testing results. A three-layer network is implemented shown in Figure 2. Table 2 shows the training and testing results. concepts:
int ......
NP
"1 i n t ,..,
. ~
--r N P
• • •0
output
~
input
!
. ~.. concepts:
mt
,..,
NP
"I i n t
,..,~ NP
Figure 2. Feedforward network Learning rate Momentum Output threshold Number of training cycles Number of training data testing/recall data recall accuracy
0.50 0.50 0.85 25,000 110 70 100%
Table 2. Training and testing results
475 For the unseen data, 100% of the testing results satisfy conditions (a) and (b) mentioned in (2.2). This network has been also tested for generalization: Introducing a new concept node into the network (i.e. this implies a new system beliefs coming into the modeling process) causes the structural change to the network. In order to retain the effects of previous training, a partial training is conduced as follows: • Fix all weights except those that are on the newly added connections. (i.e., the fixed connections are not participating the training process). • During the training period, present the new training samples on both input layer and output layer. The input vector contains the stimulus on the new concept, the output vector contains all concepts implied by the new concept. The representation pattern for a new concept (i.e., the training data) reflects the closeness between the new concept and the existing concepts. The test result shows that the functional correlated concepts yield similar concepts in output. In other words, a new concept makes the network to turn in the nodes that the correlated concepts might turn on. For example, once a concept "queue" is added, it yields the similar concepts in the output as the concept "stack" yields. This result implies that the network can generalize its reasoning ability to adapt new system assumptions without being totally retrained. This feature is particular important for the dynamic modeling process in which it is often necessary to update the structure of system belief space. 2.4. Regularity detector
In a user modeling system it is also often necessary to classify users' characteristics for system adaptation. An ART model is therefore used to further classify the outputs from above two networks. The classification is based on the closeness of the input patterns. Sixteen of the twenty concepts are represented by input nodes. Four output nodes are used to indicate user categories as expert, expert-intermediate, intermediate-novice, and novice respectively. The unsupervised training process stores the typical patterns for each category. The test result shows that the network successfully associates the test patterns to the closest stored-patterns and activates the corresponding categorical node in output layer. Figure 3 shows the example of such network behavior. 2.5. Simulating a blackboard framework
Above network models can be integrated as a blackboard system where each model can function either independently or cooperatively (refer to Figure 4). This system framework has been simulated by passing the input or output from one network to another. For example, the input and output from BLA model can be used as input (or training data) to BP model; the output from BP or BLA can be represented to ART model for classification. The output from any network is viewed as the current user profile in the dialogue context. Thus, this framework provides an effective way for dynamic user modeling. 3. CONCLUSION This study tested and integrated several neural networks as associative memories in user modeling process. It has shown several advantages such as fast default reasoning and generalization, insensitivity to inconsistent input, personalization, and learning ability. In
476 addition, comparing with the rule based systems used in conventional user modeling systems, it is easier to implement and maintain the proposed system. Also, the knowledge elicitation process is simpler than the rule-base construction, because only the causal relationships are considered to initiate the modeling process. The further research is aimed at incorporating the task information into user models so that the system can have a more comprehensive picture about not only who the user is but also what he is going to do.
2GATEGORIES 3
!
I;I;I;I;I
l*lel*l*l le!e!e!ol
4
lll:,O,O 11119 I • o!o!o
IoOl-lOlI O l O I O I I ; I ; " I"OIl O
I:1:1o'1o'1 I:1:1:1:1 o
ol • ]
• I:I..
ol ol ol o
o l _o l_ o l o I
IO°l ' :I'O°l O"! ' "• • • i:1-1:1: • l l oo~.o !aolo
OlOlOlO 0101010
~I~IoI o i'""l
o I o l ol ol o I
lele/ole I lelololo o i I o l ol ol
I O I OI OI 0
•.. ................................
•
,,.,.,.,. I"""" I I:l:I:: !i-,-,-,ololol •
diatog ~ . . ~ , d i x i........................... ,..:.
I I'1"1"!
~ BLA
lele • • I elololol I lelolo o Io!~ olo
uonolO •
"" • i:1:1 o
llol
•
l ""'" I l o l o"Oo1 '! ol o i:l:lOn:l,o,olo,o,,o,ol i,ojo,o,o ~,.,.,o,o ,,.,o,~I ° I o l o l o l o !lololole
i
o o I I oo 1~~ o ol o l l o ol oo l
0
t
L
II
' ............................ 1
ii
'~" ;"
ART
i
BP
x, t i! ~"v
"
•
no!ouono n,o,~ o, o , o , ~
.~ ..................... " ..................i;.~.:
::
1
..........................
x: stimulus training data , v: output '
o, o lloUololo I
iono!olo. II o! ~_ ol o II o! ~_ ol o
OlO101o
Figure 3. An example of the behavior of the ART for user classification
Figure 4. A blackboard framework for user modeling
REFERENCES 1. Norcio, A. F. and Stanley, J., "Adaptive Human-computer Interfaces, A literature Survey and Perspective," IEEE Trans. on System, Man and Cybernetics, Vol. 19, No. 2, pp. 399408 (1989). 2. Rich, E., "User Modeling via Stereotypes," Cognitive Sci. Vol 3, pp 329-354, (1979). 3. de Kleer, J., "An Assumption-Based TMS," Artificial Intelligence, Vol. 28 No. 2, pp. 127162 (1986). 4. Chen, Q. and Norcio, A.F. " An Associative Approach in Dynamic User Modeling ," Proceedings of 5th Intentional Conference of Human Computer Interaction, pp 909-914 (1993) 5. Huang, X., Mccalla, G. I., Greer, J. E. and Neufeld, E., "Revising Deductive Knowledge and Stereotype Knowledge in a Student Model," User Modeling and User-Adapted Interaction, Vol. 1, No. 1, (1991) 6. Pao, Y., Adaptive Pattern Recognition and Neural Networks, Addision-Wesley Publishing Co., (1989). 7. Wilson, M., "Task Models for Knowledge Elicitation," Knowledge Elicitation: Principles, Techniques and Applications, Dan Diaper (ed.), Ellis Horwood Pub., (1989).
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
477
Personality engineering: Applying human personality theory to the design of artificial personalities Linda S. Endres Johnson Controls, Inc., 507 East Michigan Street, Milwaukee, Wisconsin 53201 [email protected] We should take care not to make the intellect our god; it has, of course, powerful muscles, but no personality. - Albert Einstein (1950, p. 260)
Abstract
To understand and explain our world, we often resort to "anthropomorphism," attributing human characteristics to things that are not human. For example, intelligence and friendliness are often perceived--in varying degrees and by various definitions--in computers and other machines. This perception of human traits in nonhumans can be interpreted as "artificial personality." Because anthropomorphic systems have proven controversial, more research is required to give us the insights needed to develop more compatible and effective artificial personalities. The purpose of this paper is to introduce the concept of "personality engineering" to enable us to better understand artificial personalities and ultimately to improve the interaction between human and artificial personalities. To accomplish this, personality psychology is recommended as a framework within which to design, develop, and evaluate any system or product. A brief example is given to illustrate how one human personality theory could be adapted for the study of artificial personalities. 1. INTRODUCTION
Attributing human characteristics to things that are not human is natural in that we tend to "see the world not as it is but as we are" (Frude, 1983, p. 103). Particularly with the development of human-inspired software--agents, assistants, experts, and guides, as well as the eventual evolution of sophisticated virtual beings--it becomes easy to perceive characteristic human-like behavior, or "artificial personality." The tendency to attribute human characteristics to nonhumans is known as "anthropomorphism." Whether computers and other nonhumans actually have personalities is left for debate elsewhere. This paper begins with the observation that people do perceive personality characteristics in nonhumans and it is this perception that is referred to here as "artificial personality." As background, human personality will briefly be defined and examples will be given of personality characteristics perceived in nonhumans. Because anthropomorphic systems have proven controversial, several concerns will be noted.
478
The concept of "personality engineering" will then be introduced to enable us to better understand artificial personalities and ultimately to improve the interaction between human and artificial personalities. To accomplish this, personality psychology is recommended as a framework within which to design, develop, and evaluate any system or product. An example illustrates how we might adapt one personality trait theory to study artificial personalities. 2. EXPLORING HUMAN AND ARTIFICIAL PERSONALITIES
Personality has several definitions, including: "the complex of characteristics that distinguishes an individual," "the totality of an individual's behavioral and emotional tendencies," and "the organization of the individual's distinguishing character traits, attitudes, or habits" (Webster's, 1979, p. 848). According to the trait theory developed by Cattell (Cattell, Eber, & Tatsuoka, 1970), varying degrees of each of 16 core factors make up the normal human personality. Of these factors, warmth and intelligence are considered the most important because they account for more variation in personality than the other 14 factors (Karson & O'Dell, 1976). Not surprisingly, these two human traits are already represented--albeit in varying degrees and using various definitions-by the machine traits of user friendliness and artificial intelligence. Countless products have been advertised as "smart" or "intelligent," including batteries, buildings, highways, sensors, mops, tires, and toilets. Equally impressive is the number of products that have been advertised as "user friendly." Compared to the richness and complexity of the human personality, today's artificial personalities are typically shallow. Our quest for user friendliness and artificial intelligence has focused attention on only two dimensions of personality. We may wonder whether other personality traits can similarly be perceived in machines. A study by Asch (1946) found that when humans were described in terms of personality traits, those not described could still be inferred. Therefore, it is hypothesized here that traits that are not purposely designed in machines can still be inferred. Artificial personalities can take many forms. For example, Laurel noted that "even the most technologically savvy user will feel quite comfortable comparing the Macintosh and the IBM PC in terms of their 'personalities'" (1990, p. 356). Regardless of whether a system is overtly anthropomorphic (for example, one that uses a human image or otherwise presents itself as if it were human), personality traits can be perceived in the system. Personality may be inferred from even the choice of words used in a system. The degree of fidelity of an anthropomorphic system in simulating a given human behavior may vary. The goal of a product or system will dictate what level of fidelity in personality simulation is appropriate. For example, in the role of agents, Laurel (1990) advocates the use of stereotyped personalities--as opposed to full personality simulations. Anthropomorphic systems may also vary in the degree to which they attempt to model a complete person. The artificial nose (Coghlan, 1994), for example, is anthropomorphic in name and function, but it models only a narrow range of human capability.
479
3. ANTHROPOMORPHISM Anthropomorphism is both natural and useful and may be the most common metaphor used in formal computer discourse (Johnson, 1994). By making use of our past experience, metaphors help us understand our world. They enable us to think differently, in terms in which we might not otherwise have thought. Despite the naturalness of anthropomorphism, anthropomorphic systems are controversial and have a formidable opponent in Shneiderman (1993), a pioneer in human-computer interaction, who challenges us to rethink our anthropomorphic systems. Where children are involved, Shneiderman raised three concerns over an anthropomorphic view of machines in that it may: (1) give an erroneous impression of the workings and capabilities of computers, (2) confuse a child's developing sense of humanity, and (3) produce anxiety or confusion. To address these concerns, Shneiderman advocates a "nonanthropomorphic style" of language that attributes control to the user and avoids anthropomorphic references. Shneiderman's concerns are valid, but they are not limited to anthropomorphic machines or to children. When the concerns are recast to apply to both nonanthropomorphic systems and adults, other alternatives that address these concerns then become apparent. For example, we can acknowledge in the first concern that both anthropomorphic and nonanthropomorphic machines can give an erroneous impression to both adults and children as to the workings and capabilities of computers. However, if this information is needed, users can be trained. The second concern, that children may be confused over their own sense of humanity, can likewise be extended to include adults. Instead of attributing this confusion to anthropomorphic systems, perhaps we should work to strengthen our sense of humanity through some form of education. The third concern, recast to include nonanthropomorphic machines in producing anxiety or confusion in both adults and children, may also be addressed by prescribing training for the user, the system's designer, or both. In his nonanthropomorphic style guide, Shneiderman (1993) addressed the user's need for a sense of control by, for example, crediting a child with printing a list of animals instead of crediting the computer with printing a list. However, in crediting only the child, we lose a prime opportunity to acknowledge and appreciate teamwork between humans and their automated assistants. The human-computer relationship is an honorable one and it need not be perceived as threatening to our sense of control. The issue of control is not absolute. While there are times when a user wants complete control, there may be other times when such control would defeat the purpose of a given system. For example, consider future scenarios where an intern practices his or her bedside manner on a simulated patient or a person practices assertiveness techniques while role-playing with a virtual being. In these scenarios, users may desire strict control over such program considerations as on/off or in the setup of the role-play situation. Once these programming controls are defined (perhaps in part with sliders; see Ahlberg, Williamson, & Shneiderman, 1993), a user may then benefit from roleplaying with an anthropomorphic interface with predetermined or user-selectable personalities. In this case, both anthropomorphic and nonanthropomorphic interfaces are used as appropriate to the goal at hand. The needs of the user and the purpose of the system help determine whether it is best to employ an anthropomorphic interface, a nonanthropomorphic interface, or both in the same system, as either alternate or
480
complementary modes of interaction. In considering the needs of users, designers should be aware, for example, that an overtly anthropomorphic system might offend certain cultures (Marcus, 1993) or people who want only to view their machines as tools. A final criticism is that some anthropomorphic systems at first seem cute, but are later viewed as silly or annoying (Shneiderman, 1995). However, that such systems have been created does not mean that all artificial personalities are annoying; it only means that we have succeeded in creating personalities that can be annoying. Addressing the question, "Who wants obnoxious machines?" Kurzweil (1990) predicted that "shaping the personality of intelligent machines will be as important as shaping their intelligence" (p. 413). In the next section, a framework is introduced that will enable us to create more compatible and effective artificial personalities. 4. PERSONALITY ENGINEERING
The term "personality engineering" is introduced here to refer to the use of human personality psychology as a framework within which to design, develop, and evaluate any system or product. In effect, it proposes that we adapt traditional methods of understanding human personality for the study of artificial personalities. By drawing on psychology, personality engineering addresses two important needs: for a more scientific evaluation of interfaces (Shneiderman, 1995) and for multi-disciplinary collaboration (Laurel, 1990). It provides an excellent framework for researching products, users, and user-product interactions. Diversity and user-centered design can be taken to new levels, as designers are given one more tool to meet the needs and expectations of diverse users. By adapting the theories and tools of personality psychology, designers can anticipate how users might perceive systems and products. Of note, personality engineering is anticipated to be useful whether or not a system is overtly anthropomorphic. Furthermore, it is proposed as a supplement, not as a replacement, for traditional user interface techniques. Applications for personality engineering are wide-ranging, from planning the personality traits that users will perceive in simple text or voice output to designing fullscale artificial personalities for sophisticated virtual beings. At a basic level, traits are impossible to control if they cannot be measured. By considering personality theories, designers are at least made aware of potential interpretations that they may then decide to measure. Depending on the application and the intended users, certain personality traits may be more crucial to engineer than others. At a more sophisticated level, designers can match the personalities of users and machines to facilitate interaction and enhance productivity. Users could be allowed to implement their preferences in machine personality. Personalities of sophisticated virtual beings could be created from a palette of traits, perhaps implemented as a control panel. Various personalities could be codified in expert systems that provide the advice of multiple experts. Different artificial personalities could be supplied with computer-based training packages and students could select a teacher personality compatible with their learning style. One way to engineer the artificial personality is via the systematic application of personality trait theory to a product or system being designed or evaluated. Personality engineers might select, for example, the 16 personality factor theory of Cattell (see Table 1). By adapting Cattell's trait theory, they ensure that all essential personality
481
dimensions have been considered and thereby increase the probability that artificial personalities will meet the needs of the human personalities that will interact with them. To adapt the theory, each trait must first be defined for the application and a determination made as to how that trait might be manifested in behavior. Certain traits, such as suspiciousness and imagination, for example, have obvious counterparts in computer security and creativity. Potential users can be surveyed to determine whether they have preferences for certain traits, an inability to perceive certain traits, or an unanticipated ability to perceive or impute others. Users can also be surveyed to determine how they might perceive these traits being manifested in the behavior and/or physical characteristics of the particular product or system. The artificial personality can then be revised based on conclusions drawn from the survey. The adaptation of theories of both normal and abnormal personality will likely give us insights into both human and artificial personalities. Table 1 The 16 personality factors of Cattell's personality trait theory Description of Factor Name of Factor Cool vs. Warm Warmth Concrete Thinkincj vs. Abstract Thinkin 9 Intellicjence Affected by Feelings vs. Emotionally Stable Emotional Stability Submissive vs. Dominant Dominance Sober vs. Enthusiastic Impulsivity Expedient vs. Conscientious Conformity Shy vs. Bold Boldness Tou~lh-minded vs. Tender-minded Sensitivity Trusting vs. Suspicious Suspiciousness Practical vs. Imacjinative Imagination Forthright vs. Shrewd Shrewdness Self-assured vs. Apprehensive Insecurity Conservative vs. Experimentincj Radicalism Group-oriented vs. Self-sufficient Self-sufficiency Undisciplined Self Conflict vs. Followin 9 Self-ima~e Self-discipline Relaxed vs. Tense Tension Copyright © 1986, IPAT, Inc. Adapted from the 16PF Profile with permission. 5. CONCLUSIONS Regardless of whether a system is overtly anthropomorphic, personality traits, such as friendliness and intelligence, can be perceived in the system. This perception of personality traits in nonhumans can be interpreted as "artificial personality." Although anthropomorphism is natural, anthropomorphic systems have proven controversial. Research into the artificial personality should allay some of our concerns and enable us to create more compatible and effective artificial personalities.
482
The concept of "personality engineering" was introduced here as a way to research the artificial personality. Personality engineering recommends the use of personality psychology as a framework within which to design, develop, and evaluate any system or product. Given our ability to perceive personality traits in nonhumans, it is likely that the theories and tools of personality psychology can be adapted to enable us to better understand the artificial personality and to improve the interaction between human and artificial personalities. Cattell's 16 Personality Factor theory, for example, seems a viable way to explore the personality of a system or product. This and other personality theories can be adapted to give us insights into both human and artificial personalities. REFERENCES
Ahlberg, C., Williamson, C., & Shneiderman, B. (1993). Dynamic queries for information exploration: An implementation and evaluation. In B. Shneiderman (Ed.), Sparks of innovation in human-computer interaction (pp. 281-294). Norwood, NJ: Ablex Publishing Company. Asch, S. E. (1946). Forming impressions of personality. Joumal of Abnormal and Social Psychology, 41, pp. 258-290. Cattell, R. B., Eber, H. W. & Tatsuoka, M. M. (1970). Handbook for the Sixteen Personality Factor Questionnaire. Champaign, IL: IPAT, Inc.. Coghlan, A. (1994). Electronic nose detects the nasty niffs. New Scientist, 141(1911), p. 20. Einstein, A. (1950). Out of my later years. NY: Philosophical Library, Inc. Frude, N. (1983). The intimate machine: Close encounters with computers and robots. New York, NY: New American Library. IPAT, Inc. (Institute for Personality and Ability Testing, Inc.), P.O. Box 188, Champaign, IL 61820. Johnson, G. J. (1994). Of metaphor and the difficulty of computer discourse. Communications of the ACM, 37(12), pp. 97-102. Karson, S. & O'Dell, J. W. (1976). A guide to the clinical use of the 16PF. Champaign, IL: IPAT, Inc. Kurzweil, R. (1990). The age of intelligent machines. Cambridge, MA: MIT Press. Laurel, B. (1990). Interface agents: Metaphors with character. In B. Laurel (Ed.), The art of human-computer interface design (pp. 355-365). Reading, MA: AddisonWesley Publishing Co., Inc. Marcus, A. (1993). Metaphor design and cultural diversity in advanced user interfaces. In M. J. Smith & G. Salvendy (Eds.), Human-computer interaction: Applications and case studies, Proceedings of the fifth international conference on human-computer interaction, (HCI International 1993), Orlando, FL, August 1993, Vol. 1, pp. 469-474. Shneiderman, B. (1993). A nonanthropomorphic style guide: Overcoming the Humpty Dumpty syndrome. In B. Shneiderman (Ed.), Sparks of innovation in humancomputer interaction (pp. 331-335). Norwood, NJ: Ablex Publishing Company. Shneiderman, B. (1995). Perspectives: Looking for the bright side of user interface agents. Interactions, 2(1 ), pp. 13-15. Webster's new collegiate dictionary. (1979). Springfield, MA: G. & C. Merriam Co.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
483
Using the template model to analyse interface specifications C. R. Roast and J. I. SiddiqP ~Computing Research Centre, School of Computing and Management Science, Sheffield Hallam University, Sheffield, Sll 8HD, UK. This paper describes the application of a formal modelling technique within human computer interface design. The model described is termed the template model which is a system based model which explicitly identifies system abstractions (known as template abstractions) that have a defined relation to intended task and users' capabilities [4]. Using template abstractions is it possible to express and analyse general interface usability properties within a formal framework. We describe the application of the template model in the analysis of a wysiwyg-style word processor. In this way we are able to demonstrate the potential role of the model and the impact of formal analysis with a familiar yet non-trivial application. In the case of the word processor analysed, we are able to characterise system constraints that determine what user tasks are supported. In general, we argue that employing the template model helps explicate assumptions about interface requirements. 1. I N T R O D U C T I O N Within HCI research, models of interaction are based on two distinct perspectives of
either the users or the system which offer quite different benefits ([1]). User models conventionally are used to assess the ease of use of a system, that is the demand it makes upon a users' capabilities. System models, by contrast, focus upon the expression and comprehension of system behaviour and the role of these in the design process. One consequence of this is that usability requirements are often expressed solely in terms of users, and are not explicitly associated with system based requirements. This complicates the design process and over burdens the role of iteration in interface design. In this paper we introduce and employ a novel model of interaction termed the template model which introduces user modelling concepts into system modelling thereby enabling usability requirements to be expressed as system properties [4]. The model is applied to analyse a wysiwyg-style word processor this demonstrates its potential role in explicating assumptions about interface requirements. The analysis shows that the consequences of a formal analysis can help determine system constraints and the user tasks supported. 2. T H E T E M P L A T E
MODEL
The template model is an extension of a basic state based model of interaction proposed by Harrison and Dix [2] in that it provides a deterministic and synchronous account of
484 interaction that is adequate for modelling stand alone interactive systems. The basic state based model is defined by the tuple (S, so, D, v, K), in which: S is a set of system states; So E S identifies an initial state; D is a set of system outputs; v • S ~ D determines the output for any state; and, K is an input alphabet. Each input of the system k E K is assumed to identify a function over states, thus we have: K C_ (S --. S). The template model explicitly accommodates the notion that certain system abstractions are germane to successful system u s e - these abstractions are termed templates. The use of template abstractions focuses attention upon users intended tasks and perceptual capabilities, and the extent to which systems can support them. Being based upon system abstractions, the template model has the potential to be employed constructively within system development and avoids the complexity of having to speculate about users' psychological and cognitive activities. Two types of template abstraction are used, termed results and display templates. Results are system abstractions required by users in order to determine the successful completion, or otherwise, of their tasks. For example 'the document' within a word processor is the focus of many user tasks, and thus may be identified as a result. Since users perform a variety of tasks and sub-tasks, the system will be required to provide different and appropriate abstractions of the system state each of which may serve as a result. By contrast display templates are abstractions of the system output which denote perceived display features such as menus, icons, cursors, etc. independently of details not affecting their perceived role ([3]). We model these components as a set of results (R) each of which is a function from states to a range of values particular to that element. Thus, for a result r E R we shall denote its range by Ur" A r e s u l t r E R is a function upon states which extracts information necessary for users to determine if a task (or sub-task) is complete r" S ---, Ur. To model users' perceptions as system abstractions we shall identify a set of display templates Dt. As with results we shall treat each display template dt as a function with a range of particular type Udt. One distinct feature of the formal characterisation of display templates is that the are take to be partial functions. A d i s p l a y t e m p l a t e dt E Dt is a partial function upon displays which extracts details that can be perceived by users as potential sources of information dt • D -++ Udt. Modelling display templates as partial functions accommodates the possible absence of whatever feature a display template extracts. For instance, if a display template extract a certain form of icon, we would interpret the icon not being displayed while the display template undefined. The template model can be used to express general interactive properties that support ease of use. Roast and Harrison [5] give the derivation of two properties namely output correctness and structural consistency. In this paper we focus upon system behaviour that enables the user to determine information held within the system state - - we state, illustrate and apply the property of output correctness. Output correctness requires that each value a result may take is associated with a unique display template value, such that whenever the display template has that value the result has the corresponding value. This
485 characterises many informal notions familiar within human computer interaction, such as
wysiwyg. However, in order to accommodate situations when the display templates may be undefined we make use of the notions of weak and strong equality to arrive at the definition below. Strong equality (written =~) represents equality only between defined display template values, whereas weak equality (written =~) holds when display template values are the same or either values are undefined. For a template model, the display template dt and the result r are related in an o u t p u t c o r r e c t manner, written OC(dt, r), iff:
VSl, S2 e S :
r(Sl)-- r(s2)~ dt(v(sl)) =~ dt(v(s2)) A dt(v(sl)) =, dt(v(s2))=~ r ( S l ) = r(s2)
(1)
The conditions for output correctness requires that: if the result value is the same in any two states, then the display template in those two states has the same value (or either is undefined), and; if the display templates are defined and equal in any two displays, then the result values in the underlying states are the same. For example, with a conventional word processor a variety of tasks are dependent upon the number of pages a document has. Thus, the number of pages represented within the system state may be treated as a result which we shall term pageCount. In addition, a visual indicator of the number of pages in a document can be modelled as a display template, termed pageDisplay. For instance, a particular instance of pageDisplay may be considered to 'extract' the numerals framed and labeled in a particular manner, say, IPages" 14 i. To ensure that the display template serves as a reliable indicator of the result we to simply assert OC(pageDisplay, pageCount), this would ensure that whenever the frame and label were displayed the numerals shown would be in one-to-one correspondence with the actual page count (pageCount). In this way users engaged in tasks dependent upon a document's number of pages would be supported by the appropriate template abstractions and system behaviour. 3. A P P L Y I N G T H E M O D E L To illustrate the impact of template properties such as output correctness, we have developed an abstract specification of a wysiwyg-style word processor, this is expressed in the Z specification notation [7]. Working from the abstraction specification the impact of usability requirements expressed as instances of output correctness can be assessed. Our abstract model of a word processor introduces a framework, where a document is viewed as including 'printable' and 'control' characters. The image of a document is defined in terms of positioning the bitmaps of printable characters on an output medium, using styles and fonts defined by 'control' characters within the document. The imaging of a document is defined for two resolutions corresponding to the VDU output and the printed output. Given this framework we are able to consider two informal notions of wysiwyg that represent alternative instances of output correctness, each of which supports particular tasks. First, print correctness ensures that there is a perceivable correspondence
486 between the VDU output of the word processor and what appears on the printed page. This correspondence supports a variety of tasks, in particular those where the physical layout of the printed document is important. Second, document correctness concerns the correspondence between the VDU output and the underlying document content. In general, editing tasks demand that this form of correspondence is supported. Figure 1 illustrates the alternative views of the document suggested by print correctness and document correctness.
document j
cor r e ~
Display: [ - ~
Document" " ... ,n,t,boldOn,b,r,... ,,
correct
Page:
Figure 1. Alternative document views" the document itself, the displayed document and the printed document.
In order to express print correctness and document correctness formally, and analyse their impact upon the word processor, it is necessary to make certain assumptions about the nature of user perception. In particular, we model two forms of user perception with two relations • and gts. Briefly: we write b • f to represent that the bitmap b is seen by users as including the visual feature f, and b ~ s s represents that the bitmap b is seen as displaying the character sequence s. From this minimal view of perceptual properties we are able to express the notions of print correctness and document correctness formally and analyse the system's adherence to them. P r i n t c o r r e c t n e s s requires that for any visual feature f, VDU output bx and any printer output b2, we have:
OC(bl qt f, b2 ~1 f)
(2)
(Where, bl and b2 are bitmaps.) D o c u m e n t c o r r e c t n e s s requires that for any string of character s, VDU output b and document doc:
oc(b •
i. aoc)
(a)
(Where, b is a bitmap, and s in doc represents that s is a subsequence within doc.)
487 Since the word processor is modelled in generic terms, the requirements of print and document correctness can be interpreted as constraints upon the word processor model• The consequences of these constraints represent conditions which can focus further analysis and development. A full account of the specification and its analysis can be found in [6]. 4. O U T C O M E S The outcome of the analysis identifies general system constraints that satisfy the usability requirements expressed as instances of output correctness. The potential for a system to adhere to these constraints focuses attention upon the exact nature of the requirements expressed. Difficulties with satisfying such constraints can be interpreted as identifying the limitations of any eventual system's successful, or focusing the re-examination of the original assumptions. In the case of the word processor the constraints found have familiar counterexamples within commercial word processing packages we propose that these counterexamples illustrate potential usability problems. The analysis of print correctness results in two constraints upon the word processor model developed: (i) that the perceived features of any character bitmap at display resolution also should be present for the same character at print resolution, (ii) the relative angle and distance between any two characters in an image at display resolution should be preserved at print resolution. This constraints do not prevent two characters (in a certain font and style) from appearing identical, they require that whatever perceived features characters have they are preserved at alternative resolutions. Counterexamples of print correctnessrequirement can often be found in commercial word processing packages. For example: it common for the numeral ' l ' and the letter '1' are rendered identically on the display to appear distinct on the printed page; and the spacing of fonts is often not preserved at alternative resolutions (see figure 2). The significance of potential contradictions of this sort will depend upon the exact model of perception represented by @, and expected nature of users tasks.
h
.
•
.
.
•
.
h
.
•
.
.
.
.
.
.
.
.
B
B
Figure 2. The same document at different resolutions may not preserve relative angles between characters.
For document correctness the analysis yields the following constraints: (i) if a printable bitmap is perceived as a character, then that bitmap should be generated by the same character, (ii) any displayed bitmap which is perceived as two adjacent characters should be generated by two adjacent printable characters in the document. These conditions prevent any character bitmap from being perceived as another character or sequence of characters. Again contradictions are common: the character double quote '"' is often rendered on the display as though two single quote characters " ' '. A more dramatic
488 contradiction arises if we consider control character sequences that are ineffectual. For instance, a document containing "...a,boldOn, boldOff,b ..." may appear the same as though "...a,b ...". For each of the counterexamples of output correctness discussed above we can ask a number of questions that would motivate a further analysis of the requirements for the interactive system being developed. First, we can question whether or not the contradictory situation is likely to arise. For example, is it likely that the control characters boldOn and boldOff would be juxtaposed in a document? Second, we can examine the significance of the contradictory situation upon successful use. For example, we could analyse whether expected users were aware of the possible confusion of '1' and '1'. 5. C O N C L U S I O N S Our use of the template model and its associated properties illustrates that usability requirements can be formally analysed in a system based framework. Given the familiarity of the domain analysed one might argue that notions of "correctness" are well understood, but our analysis has revealed that such notions make complex demands upon successful interaction. In particular, usability requirements may be refined and explicated early in development through the use of the template model and properties such as output correctness, in terms of system properties, thereby enabling usability requirements to be an integral part of the system and its subsequent development. REFERENCES
1. P.J. Barnard and M. D. Harrison. Towards a framework for modelling human computer interactions. In J. Gornostaev, editor, Proceedings International Conference on HCI, EWHCI'92, pages 189-196. Moscow:ICSTI, 1992. 2. M.D. Harrison and A. J. Dix. A state model of direct manipulation. In M. D. Harrison and H. W. Thimbleby, editors, Formal Methods in Human Computer Interaction, pages 129-151. Cambridge University Press, 1990. 3. M. D. Harrison, C. R. Roast, and P. C. Wright. Complementary methods for the iterative design of interactive systems. In G. Salvendy and M.J. Smith, editors, Designing and Using Human-Computer Interfaces and Knowledge Based Systems, pages 651-658. Elsevier Scientific, 1989. 4. C.R. Roast. Executing Models in Human Computer Interaction. PhD thesis, Department of Computer Science, University of York, 1993. 5. C. R. Roast and M. D. Harrison. User centred system design using the the template model. In F. Paternb, editor, Proceedings, EUROGRAPHICS Workshop on
the Design, Specification, Verification of Interactive Systems, Bocca di Magra, Italy, Eurographics Seminar Series, pages 381-392. Springer-Verlag, 1995. 6. C.R. Roast and J. I. Siddiqi. A formal analysis of an interface specification using the template model. Technical Report in preparation, 1995. 7. J. M. Spivey. The Z Notation: A Reference Manual. Prentice Hall International, 1988.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
489
T a s k m o d e l - s y s t e m model: t o w a r d s an u n i f y i n g f o r m a l i s m Ph. A. Palanque a'b, R. Bastide a'b, V. Senges a 1 aL.I.S., Universit6 Toulouse I, Place Anatole France, 31042 Toulouse Cedex, France {palanque, bastide, senges }@cict.fr. Tel. +33 61 63 35 88 - Fax. +33 61 63 37 98 bCENA, Centre d'l~tudes de la Navigation Arrienne, 7 avenue Edouard Belin, 31055 Toulouse Cedex, France.
1. I N T R O D U C T I O N The ultimate goal of many current approaches in HCI design is to enhance the quality of interactive systems, meaning to produce systems that are both usable and reliable. To reach that double ended goal, design practices must include in their scope the modeling of the two actors of HCI : the user and the system. The goal of usability is tackled by the integration of the user' s point of view at several stages of the design. Several notations have been devised to describe user tasks, such as UAN [ 1], GOMS [2], MAD [3], TAG [4]. Those formalisms are generally easy to use and to understand, in the hope that they can be directly validated or modified by the users themselves. The reverse side of that ease of use is that those formalisms often lack a solid formal foundation, and thus may present incompleteness or ambiguities that hinder their seamless integration in the design of the interactive system itself. In the field of softwareengineering, a promising approach to reach the goal of reliability is the use of formal methods. The use of a formal notation which is both mathematically analyzable and directly executable, is of great help in the design of an interactive system, allowing for an " a priori "validation of correctness. This is especially true since modem interactive systems are in general very difficult to test, due to their graphical and event-driven nature, and that their reliability cannot generally be asserted by an " a posteriori" testing. The authors of this paper have previously devised a formal approach for the design of interactive systems [5]. This approach, called ICO (for Interactive Cooperative Objects) uses high-level Petri nets as a formal model for the dynamics of the system, and the structuring constructs of the object-oriented approach. The goal of this paper is to provide a gateway between task models defined with the UAN formalism and system models defined by ICO. This is accomplished by giving formal Petri net equivalencies for every UAN constructs. We begin by briefly summarizing the main features of the two formalisms (UAN and Petri nets). Then we describe how each of the nine constructs of UAN can be described in terms of Petri nets. Finally, we show on a simple example how those transformations are performed. In the conclusion we present what are the benefits of making a bridge between user's task models and a model of the system. 1 The authors would like to thanks the HCI group of the Department of Computer Science of the University of York (U.K.) where Ph. Palanque was a visiting researcherduring the developmentof this paper.
490
2. PRINCIPLES OF THE TWO FORMALISMS
2.1. UAN Principles The UAN formalism is a task oriented and a user centered notation [6]. The tasks are described at two levels: Action level: which describes the primary user actions and their corresponding interface feedback. Those actions correspond to the physical actions performed by the user when he/she interacts with the devices (e.g., Depress mouse button). This level is related to implementation and allows designers to have a complete view of all the possible actions and their corresponding impact on the user interface. Its representation in a tabular way improves readability and gives a very clear distinction amongst inputs and outputs of the interactive system. The information described at this level is of few interest regards to the high-level structuring of the tasks, which is the aim of this paper. Actually to take into account this kind of information, a more powerful formalism than basic Petri nets is needed. We have already proposed such a formalism [5] and are currently working on its merging with the action level, but this description is beyond the scope of this paper. Task level: manipulates tasks names and temporal relations between them. It corresponds to a high-level description of the tasks. It usually takes into account the descriptions of the Action level, and their inter-relations are described in [6]. There are nine constructs for the description of the temporal relations between tasks: sequence, iteration, choice, repeating choice, order independence, interruptibility, interleavability, concurrency, intervals-waiting. These constructs are described in the s~tion 3. 2.2. Petri nets: principles Petri nets [7] are a mathematically founded formalism designed for the modeling of concurrent systems. When modeling with Petri nets, a system is described in terms of state variables (called places, depicted as ellipses) and by state-changing operators (called transitions, depicted as rectangles), connected by arcs. The state of the system is given by the marking of the net, which is a distribution of tokens in the net's places. State changes result from the firing of transitions, yielding a new distribution of tokens. Transition firing involves two steps: (1) tokens are removed from every input places, and (2) new tokens are deposited in every output places. A transition is enabled to fire when all of its input places contain tokens. Figure 1 depicts a small Petri net which illustrates the simple concept of sequences of actions. When a variable n labels an arc between a place and a transition, it means that the place has to hold at least n tokens in order for the transition to be fireable. When it fires n tokens are removed from the place (cf. Figure 3). A variable n may label an arc from a transition to z place. In that case, when the transition is fired n tokens are deposited in the place. Using Petri nets it is possible to include temporal aspects in models. Sifakis [8] proposed for each place to have a duration, meaning that a token must remain in a place a certain amount of time before it can be used by the occurrence of a transition. This amount of time is described between brackets next to the corresponding place (cf. Figure 9). Petri net theory offers a set of structuring mechanism for describing complex systems [9]. Among them, hierarchical refinement is made possible using substitution nodes which are places or transitions related to a sub-Petri net. For instance, in order to model complex behaviors a transition can describe high-level actions, and a sub-Petri net can be associated to
491 this transition in order to describe precisely its behavior without enlarging the Petri net modeling the system at a high-level. However, in order for the Petri net to be rebuildable (i.e. to be able to build a single model from the main Petri net and its sub-Petri nets) and analyzable (i.e. to apply analysis techniques provided by the Petri net theory on the model of the system), it is requested for the sub-Petri nets to have at least one transition with no input place (called Begin) and one transition with no output place (called End). Of course this refinement can be used recursively in order to describe hierarchically complex behaviors.
3. F R O M A UAN MODEL TO A PETRI NET In this section we show how every UAN construct of the task level is represented by a subPetri net. The global task is thus represented by a Petri net featuring substitution nodes (precisely transitions called macro-transitions).
3.1. Description of the constructs O S e q u e n c e : There is a sequence relation between tasks when a task has to be completely performed immediately after another task (see Figure 1). @ Choice: There is a choice relation between tasks when the user has to choose (equally) and perform one of these tasks (see Figure 2).
UAN description A g
description UAN description A IB
Petri net
B,
Figure 1. Sequence
Petri net
Figure 2. Choice
UAN description :)etri net description UAN description (A) + Begin (A )n .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Petri net
description
.
¢
1,, A
,
description
Begin
Begin
I
Enl ~nn 'l UAN description
Petri net description .
.
.
.
.
.
.
.
(A)*
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Begin ' .
- _ -.
.
.
.
_.
.
.
.
.
End ¥ .
.
.
.
.
.
.
.
.
.
Figure 3. Iteration
.
.
.
.
.
492 • Iteration: There is an iteration relation if the task can be performed zero or several times (*) or has to be performed exactly n times (n) or one or more times(+) (see Figure 3). @ Repeating Choice: It is a combination of the two previous temporal relations (iteration and choice) (see Figure 4). The left part of the picture describes the repetition of exactly n choice, while the fight one describes zero or several choices. Due to space reasons the sub-Petri net corresponding to one or more choices is not presented here. UAN description Petri net description ( A Begin "n IB) n . .
. .
. .
. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. .
.
. .
. .
. .
. .
. .
.
.
UAN description
.
.
.
.
.
.
(A
Petri net description Begin .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
t
IB)
I
A
~
B
I
I
Figure 4. Repeating choice • Order independence: There is an order independence relation between tasks if the user performs all the tasks without any constraint of ordering (see Figure 5). OAN description Petri net description A
......~ - - - - :
.............
& B
I
I
Figure 5. Order independence • lnterruptibility: There is an interruptibility between tasks if one task (A) can be interrupted by another task (B) and this last task (B) have to be entirely performed before to return to the interrupted task (A) for completion (see Figure 6). UAN description
Petri net description - -
B
-->
.
.
.
.
.
.
.
.
.
.
.
A
.
.
.
.
.
.
-
-
-
-
-
-
-
-
-
-
Petri net description Begin ._~
UAN description -
-
r~..,,i,,
-
-
-
=
.
A
With: Task: A A1 A2
¢=~
.
.
.
- - _ - _ - _ - - - - - _ - - - - _ - _ -
B
.
.
.
.
.
.
.
.
.
.
.
.
.
-
.
.
.
.
.
.
- - _ - _ - _
With: Task: A Task: B A1 BI A2 B2 c/~E~ tran,, .
.
.
.
.
.
.
transition .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Figure 6. Interruptibility
.
.
.
.
.
.
.
.
.
.
.
.
.
q -" Macro• transition B
~ .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
F i g u r e 7. Interleavability
@ Interleavability" There is an interleavability relation between tasks if each task can interrupt each other (see Figure 7).
493 • Concurrency: There is a concurrency relation between tasks if some (two or more) of these tasks can be performed at the same time (see Figure 8). UAN description All
Petrinet description
B
UAN description Petri net description A (t > n seconds) B Begin
I
I
[0, n] P. Temporisation
I End
I +
Figure 9. Intervals - Waiting
Figure 8. Concurrency
• I n t e r v a l s - Waiting" There is a waiting (or interval) relation between tasks if a task have to be performed before another task and before the waiting of a period of time (Figure 9).
3.2. Combination of constructs As stated in section 2.2, our translation scheme is of hierarchical nature, allowing to combine several macro-transitions to form a more complex net. Figure 10 for example presents the Order independence combination of two sequences of two subtasks. The merging of a subPetri nets with a transition T in the upper one is done by: - connecting the arcs from the input places of the transition T of the Petri net to the transitions without incoming arcs in the sub-Petri net (Begin transitions), - connecting the arcs from the output places of the transition T of the Petri net to the transitions without outgoing arcs in the sub-Petri net (Endtransitions). UAN formalism (A1 A2) &(B1 B2)
Petri net formalism ..... ~t-.... ; ..... ~ , - - -
W
I A2
II
I B2
I
r
Macro-transition A ~ ~__~ ~
~ "Macro-transition B
Figure 10. Combination of the constructs
494 4. C O N C L U S I O N In this paper we have proposed a way for translating from a UAN description to a Petri net one. For every UAN construct, we have proposed a Petri nets which interpretation is the same as the one of the UAN description. Lastly, we have shown how those sub-Petri nets can be merged in order to model complex task models. This approach provides significant advantages : • the ambiguities of the task models are solved during the translation process. This process, while automated to a certain extend, is interactive as it asks the designers of the task models to make more precise their specification if needed. Afterwards, the task models can be mathematically validated by analysis the Petri net model; • as we use Petri nets for modeling the interactive application, cooperation between the model of the tasks and the model of the system can be mathematically checked, in order to ensure before implementation that the system will be able to perform all the user's requests included in the task models; • as the model of the system is embedded at run time, it can be used to provide contextual help about the behavior of the system, as described in [10]. When the task model is also embedded within the same formalism, the help may be given not only with respect to the system behavior but also with respect to the user's task and goal.
REFERENCES
1.
D. Hix, H.R. Hartson, Developing user interfaces Ensuring, Usability Through Product & Process, John Wiley & Sons, inc., 1933. 2. S. Card, T.P. Moran, A. Newel, The psychology of Human-Computer Interaction, Hillsdale, NJ: Erlbaum, 1983. 3. Scapin D. L., Pierret-Golbreich C., Towards a method for task description: MAD, Work With display Units'89, Amsterdam, Elseiver, 1989. 4. S.J. Payne, T.R.G. Green, Task Action Grammar: A Model of the Mental Pepresentation of Task Languages, Human-Computer Interaction, 2, pp 93-133, 1986. 5. P.A. Palanque, R. Bastide, Petri net based Design of User-driven Interfaces Using Interactive Cooperative Object Formalism, In proceedings of 1st Eurographics Workshop on Design, Specification and Verification of Interactive Systems, Paterno editor, Carrara, Italy, 8-10 June 1994. 6. Hartson H.R., P.D. Gray, Temporal Aspects of Task in the User Action Notation, Human Computer Interaction, Vol. 7, pp 1-45 - 1992. 7. Peterson J.L., Petri net theory and modelling of systems, Prentice Hall 1981. 8. Sifakis J., Use of Petri nets for performance evaluation, In Measuring, Modelling and Evaluating Computer Systems. H. Beilner and E. Gelenbe (Eds.), North Holland 1977. 9. Huber P., Jensen K, Shapiro R., Hierarchies in coloured Petri nets, 10th international conference on application and theory of Petri nets, Bonn 1989. 10. P. Palanque, R. Bastide, L. Dourte, Contextual Help for Free with Formal Dialogue Design. "HCI International 93", 5th International Conference on Human-Computer Interaction, North Holland.Orlando, Floride (USA), 8-15 August 1993.
III. 17 Modeling 2
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
497
Scenario based specification of interaction metaphors C. Stephanidis, C. Karagiannidis and A. Koumpis Institute of Computer Science, Foundation for Research and Technology - Hellas, Science and Technology Park of Crete, P.O. Box 1385, GR-71110 Heraklion, Crete, Greece
A scenario-based formal specification of interaction metaphors is proposed, which assists designers to develop interaction dialogues at a high-level of abstraction. A formalism has been developed which provides means for describing both application and metaphor semantics, as well as a descriptive model for the specification of interaction scenarios. The proposed approach has been applied for the construction of application scenarios in the domain of special education (i.e. supporting students with learning difficulties), so that different user interaction requirements and cognitive abilities are addressed.* 1. INTRODUCTION Human-Computer Interaction research has paid considerable attention to interaction metaphors, recognising their impact on achieving various usability goals for interactive computer-based applications [1], [2]. In this paper, a method is described which enables the 'high-level', scenario-based specification of interaction metaphors, independently from the specifics of the target development environment. Instantiations of the model can be elaborated as prototypes through the use of storyboard, video, and rapid prototyping tools [3]. The work reported is based on a proposed formalism that captures application and metaphor semantics. This formalism serves as the basis for the development of a descriptive model for the specification and implementation of interaction metaphors, and complies with the scenario-based design approach, taking into account typical user activities in the development process [3]. 2. F O R M A L I S M The formalism captures application and metaphor semantics. In the context of this formalism, implemented interactive applications are abstracted to the 'combination' of behavioural aspects of the functional core of an application, with the presentational characteristics attributed to a specific metaphor. Applications are described by means of entities and actions. Each entity has a set of possible states, and a set of possible transitions between its states, which determine its behaviour. Each action, on the other hand, is characterised by the entity on which it is performed, and the state transition it causes. Part of this workhas been carriedoutin the contextof the HORIZONProjectof the Region of Crete, Greece,which was partiallyfundedby the Commissionof the E u r ~ Union (DG V).
498 Following this abstraction, we define relationships between applications (e.g. equivalence, sub- and super-application), as well as operations on applications (e.g. composition). These definitions are based on the relationships between application-specific entities and actions sets. Metaphors are also described by means of entities and actions. Entities are divided into objects on which actions are performed, and actors that perform the actions. Each entity has a set of possible appearances, and a set of possible transitions between its appearances, which determine its presentation. Each action, on the other hand, is characterised by the actor that performs it and the object on which it is being performed, as well as the appearance transition that is caused by its activation. In this context, each metaphor is defined by a triplet [ A, O, P ], containing its actors, objects and presentation, respectively. For example, the well-known desktop metaphor may be defined by the sets: • Actors, relating to the user (or users in the case of a CSCW application); for a single user, there might be no presentation, while for multiple users, there might be appearance states such as visible, non-visible, etc, and transitions between their appearances such as useri.visible --->useri.non-visible, etc; • Objects, relating to icons, windows, window managers, trashbins, etc; the window object has possible appearance states defined by its size, position, etc, and possible appearance transitions such as window.size1 --->window.size2; • Presentations, relating to the transitions between the appearance states of the objects and actors, caused by the actions; for instance, such a transition could be window.position1 ---) window.position2. Similarly with the case of applications, we define relationships and operations for metaphors. In particular, two metaphors M~ and M2, defined by [ A~, Ol, P~ ] and [ A2, 02, P2 ] respectively, can be related in one of the following ways: MI -- M2 (Ml equivalent to M 2 ) ¢ ~ :! fA : A~ --->A2, fo : O~ --->02, where fA, fO are bijective functions, such as V a.pi, a.pj e Al, if ( a.pi ---> a.pj ) ~ Pl then ( fA ( a.pi ) --->f^ ( a.pj ) ) e P2, and Vo.pi, o.pj~ O1, if(o.pk ---> o . p l ) ~ P l then ( f o ( O . p k ) - - - > f o ( O . p l ) ) ~ P2 MI :9 M2 ( MI super-metaphor of M2 ) ¢:* 3 f^ : AI --->A2, fo : O1 --->02, where fA, fo are surjective functions, such as V a.pi, a.pj e Al, if ( a.pi ---> a.pj ) e Pl then ( fA ( a.pi ) ~ fA ( a.pj ) ) ~ P2, and V o.pi, o.pj E O1, if ( o.pk -"> o.pl ) e PI then ( fo ( O.pk )--->fo ( o.pl ) ) e P2 M1 c M2 ( M~ sub-metaphor of M2 ) ¢=> M2:9 M~ ( M~ super-metaphor of M2 ) where a.p and o.p describe possible presentations of actor a and object o respectively. The equivalence relation segments the Metaphor space into classes of equivalent metaphors. Also, operations on metaphors are defined (where M3 is defined by [ A3, 03, P3 ]): M3 = MI u M2 ( composition ) M3 = MI c3 M2 ( common part )
¢:, A 3 = A I u A 2 , O 3 = O i u O 2 , ¢=> A3 = AI c3 A2, O3 = O1 {"~ O2,
P3 = Pl u P3 P3 = P I n P3
which have all the properties that hold in the respective operations on sets (e.g. commutative, transitive). Furthermore, we define relationships between applications and metaphors. A metaphor, in this context, can serve an application, if there is a surjective function which maps its entities and actions sets, onto the entities and actions sets of the application; in any other case, the
499 metaphor serves the application only partially. This surjective function maps the application semantics (behaviour) to the appearance attributes of a metaphor (presentation). In addition, as the equivalence relationships segment the Application and Metaphor Spaces into subspaces of equivalent applications and metaphors, we can conclude that equivalent metaphors can serve (or serve only partially) equivalent applications. The formalism follows a structural approach [ 1] in describing metaphors, in the sense that metaphors are defined/described in terms of primitives and relations between primitives, in both the source (i.e. application) and target (i.e. metaphor) domains, and mapping functions are defined between the two domains, relating portions of the descriptions. Apart for enabling the pure theoretical construction of composite artificial structures, the formalism also serves as the basis for the development of a descriptive model for the specification of metaphors. 3. MODEL Based on the above formalism, a descriptive model has been developed consisting of two submodels, namely scenario specification and scenario implementation submodels (Figure 1). The proposed model aims to be utilised mainly for the provision of alternative metaphors of an interactive application, such as an information system application that aims to be used by a variety of users with different requirements, needs and abilities.
/
With l~ospect_To ~
~
...... IAaivalion
•..
To_Rntity
I ~''~0"
_Sehaviour iitte;
'.....
Actor_~ Interact Eas_Subt,
:
/
\
InleraclionTechnique
I°b~; IlU~e, ...... ", I I Sy=emO~ver'I Dr~ve. Has ~un_~'m~_
: :
Bntity
:- ".
.,
multivalued attribute link
i s~-
Enli~
'/ llnterad0n
UsG Int
=tque i
lkploys_ r~ogtcal Devlc,
kploys_
l
............. isA Relation (Specialisation)
~ e r a c t ion_Technique_ ~a!Behavlour
~T~o_
I
""...,,..°. I
I~~,
/ t /
I'- i Figure 1. The scenario specification and scenario implementation submodels and their relations
I
I
500
3.1. Scenario specification submodel The scenario specification submodel enables the specification of metaphors and can be treated independendy of a computer-based environment. It provides a basis for the construction of metaphoric 'worlds' [4]. More specifically, theatrical plays, movies or even usual stories could be, in principle, described by means of the scenario specification submodel, without necessarily aiming towards a computer-based implementation. The metaphoric worlds to be created by employing the scenario specifcadon submodel follow a linguistic approach [5], since the model informally adopts a scheme whereby the primitive entity within a metaphor is the analogy of a sentence in a language, containing a subject, a verb and (optionally) an object. Of course, there might be more than one subjects, verbs or objects in a sentence. A series of sentences make up a paragraph and a series of paragraphs make up a story [6]. A typical sentence in the context of a specific metaphor includes: • an entity as the equivalent of a subject in the linguistic paradigm, • an activity as the equivalent of a verb in the linguistic paradigm, • an entity as the equivalent of an object in the linguistic paradigm. Logical Condition classes are introduced in order to assist the process of specifying dialogue control and sequencing issues in the metaphor, and are specialised in four sub-classes, namely: • precondition, which is a set of predicates that should be true, or a set of conditions/constraints that should be satisfied, so that the activity is enabled; • activation condition, which is a set of conditions that trigger an activity; • termination condition, which is a set of conditions that stop the activity; • postcondition, which is a set of predicates or conditions/constraints that should be satisfied after the termination of the activity. The metaphor designer is able to proceed to a customisation of the scenario specification submodel by means of a specialisation for a specific domain. More specifically, the metaphor designer is encouraged to create subclasses of the existing classes in the model, with characteristics considered important. For instance, when applied in the domain of computerbased education, the actor class in the scenario specification submodel might be further specialised in two new subclasses, e.g. tutor_actor and student_actor, that apart from the 'standard' (i.e. inherited) attributes of their class, they might be annotated with new attributes, which are due to the real world properties of the domain-specific classes. This possibility of custornisation (i.e. modularly modifying, adding or deleting classes of the model and their attributes) contributes towards the development of more complex or composite metaphors capable of conveying all those semantics that are considered by the metaphor designer to be important for the design procedure. 3.2. Scenario implementation submodel Any already specified metaphor by means of the scenario specification submodel might be described in terms of the scenario implementation model. Though the scenario specification submodel could be characterised as generic, the scenario implementation submodel is strictly limited to computer-based implementations of metaphors. The transition from the specification of a scenario to its implementation could potentially take place by means of an algorithm which implements the mappings between real-world entities and their attributes, to corresponding interface entities and attributes; this mapping
501 could be accomplished according to an automated procedure, where: specific entities of the specification submodel are instantiated in the implementation submodel as specific runtime entities, specific activities are realised by means of specific interaction techniques, etc. For the last step of the scenario implementation, namely the selection and realisation of specific interface entities, the metaphor designer could be assisted by a user interface developer or an application engineer, so that they could, in cooperation, decide on the implementational details of the metaphor. The correspondence of classes between the two submodels is shown in Table 1. It should be noted that the Scenario Implementation Classes are independent from the specificities of the various technology platforms. For instance, the Interaction Object class depicts abstract behaviours that can be assigned to interaction objects. In the present context, it might be thought of as referring to Interactors [7] or Abstract Interaction Objects [8]. Scenario Specification
Scenario Implementation
Entity Run time Entity Object Interaction Object Actor User or System Agent User driven Actor User System driven Actor System Agent System driven Actor or Object System Entity Activity Interaction Technique Table 1: The correspondence between the two submodels 4. APPLICATION OF THE MODEL The proposed approach has been employed in the construction of applications in the domain of special education. More specifically, a multi-disciplinary group, consisting of educators, psychologists, computer scientists, and rehabilitation specialists, have utilised the model as a 'formal reference description', in order to develop metaphoric worlds to assist students with learning difficulties acquire elementary mathematical concepts. More specifically, the following process has been adopted: • Initially, the multi-disciplinary group posed the educational objectives and goals to be satisfied by the educational application. • By means of a stepwise refinement process, the overall objectives have converged into a high level interaction scenario. • The high level scenario was iteratively reformulated into a specific instantiation of the scenario specification submodeL • The design group cooperatively arrived at the transition from the scenario specification to the scenario implementation. • The resulting instatiation of the scenario implementation submodel has been implemented in two application development environments for evaluation purposes.
502 5. CONCLUSIONS AND FUTURE WORK The utilisation of the proposed model in the above application has resulted in a significant reduction of the time spent on the specification and implementation of scenarios, by the multidisciplinary group. The main benefits of our approach, can be summarised as follows: • The proposed formalism facilitates the development of metaphors as a multidisciplinary task to be accomplished through the synergy and collaboration of various experts from different fields, such as human factors experts, psychologists, interface designers, artists, who need not be expert programmers. • The transition from the specification to the implementation of a specific scenario is not unique; multiple implementations of the same specification may exist, i.e. the same scenario can be implemented using different presentation and interaction means (interaction techniques, media/modalities, devices), as well as different technology platforms, depending on the specific abilities, needs and preferences of the target user group under consideration. • The proposed model facilitates the development of 'multi-user' metaphors, through the creation of multiple user-driven actors, hence it may be particularly suitable for ComputerSupported-Cooperative-Work (CSCW) applications. • The proposed model may be easily expanded and specialised through the introduction of new subclasses and attributes, in order to address the requirements of specific application domains and/or user groups. Specific aspects of the model are currently under implementation, concerning the automated transition from the specification to the implementation of interaction metaphors. REFERENCES
1. J.M. Carrol, R.L. Mack and W.A. Kellog, Interface Metaphors and User Interface Design, in M. Helander (ed.), Handbook of Human-Computer Interaction, North-Holland, 67-85, 1990. 2. Proceedings of the 1994 FRIEND 21 International Symposium on Next Generation Human Interface, Meguro Gajoen, Tokyo, February 2-4, 1994. 3. J.M. Carroll, Making Use A Design Representation, Communications of the ACM, 37(12), 2935, 1994. 4. B. Laurel, Computers as Theatre, Addison Wesley, 1991. 5. J.R. Hobbs, Metaphor and Abduction, in A. Ortony, J. Slack and O. Stock (eds.), Communication from an Artificial Intelligence Perspective, NATO ASI Series, Springer Verlag, 1992. 6. R.L. Campbell, Will the Real Scenario Please stand up?, SIGCHI Bulletin 24(2), 1992. 7. B. Myers, Encapsulating Interactive Behaviours, CHI '89 Conference Proceedings on Human Factors in Computing Systems, 319-324, 1989. 8. J.M. Vanderdonckt and F. Bodart, Encapsulating Knowledge for Intelligent Automatic Interaction Objects Selection, INTERCHI '93 Conference Proceedings on Human Factors in Computing Systems, 424-429, 1993. 9. M.D. Harrison and A.J. Dix, Modelling the Relationship between State and Display in Interactive Systems, in P. Gomy and M.J. Tauber (eds.), Visualisation in Human-Computer Interaction, Springer-Verlag, 241-249, 1990.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
503
Cocktail-Party Effect with Computational Auditory Scene Analysis Preliminary Report Hiroshi G. Okuno a, Tomohiro N a k a t a n i a, and Takeshi Kawabata a aNWW Basic Research Laboratories, Nippon Telegraph and Telephone, 3-1 Morinosato-Wakamiya, Atsugi, Kanagawa 243-01 JAPAN One of important and interesting phenomena in sophisticated h u m a n communications is the cocktail party effect: t h a t even at a crowded party, one can attend one conversation and then switch to another one. To model it in a computer implementation, we need a mechanism for understanding general sounds, and Computational Auditory Scene Analysis (CASA) is a novel framework for manipulating sounds. We use it to model the cocktail party effect as follows: sound streams are first extracted from a mixture of sounds, and then some sound stream is selected by focusing attention on it. Because sound stream segregation is an essential primary processing for the cocktail party effect, in this paper, we present a multi-agent approach for sound stream segregation. The resulting system can segregate a man's voice stream, a woman's voice stream, and a noise stream from a mixture of these sounds. 1. I N T R O D U C T I O N Looking and listening are more active t h a n seeing and hearing [1]. Some of important and interesting phenomena in sophisticated h u m a n communications are (1) the cocktail-party effect m selectively attending some conversation or sound source and then changing the focus of attention to another [2], and (2) the Prince Shotoku effect ~ listening to several things at the same time [3]. The latter effect is named for Prince Shotoku (574-622) in Japan, who is said to have been able to listen to seven people petitioning him at the same time. Although research on speech understanding has a rich history, it is still difficult to understand sounds under real-world conditions. In psychoacoustics, some efforts have been made to build a general framework for understanding sounds. This research area, called auditory scene analysis [4], has recently been explored by AI and computer science researchers trying to create a general representation of sounds in order to deal with more realistic acoustic environments and to integrate computational auditory frameworks into multimodal perception systems. This emerging research area is called Computational Auditory Scene Analysis (CASA)
[5]. The main research problem in listening to multiple sound sources is sound stream segregation, which may yield clues to a modeling of the cocktail-party effect. In this paper, sound stream segregation is modeled by the multi-agent paradigm.
504
t,,.,=.,,,,.,...,,,.,.,.....,,. •
.......
,,..,,,,,..,
.....
,..=.,..,..
.....
,,,,.,.,,,.,,=.,,~
,
sound streams mixture of sounds
Sound stream segregation
~
i multi-agent i system
i !
% ....................................
~'
%,% ..~o.. "%'°%,
Focus of attention mechanism
multimodal information
Figure 1. Cocktail party effect modeled by Computational Auditory Scene Analysis.
2. COMPUTATIONAL AUDITORY SCENE ANALYSIS Auditory scene analysis aims at an understanding of acoustic events (or sources) that produce sounds. An acoustic event consists of sound streams (hereafter simply stream), each of which is a group of acoustic components that have consistent attributes. The process that segregates sound streams from a mixture of sounds is called sound stream segregation, and the cocktail party effect can be modeled by it (Figure 1). The segregation system extracts individual sound streams from a mixture of sounds, and the focus-of-attention mechanism then selects one stream. Such a selection may be affected by the sound streams themselves as well as by other kinds of information, such as visual cues. Consider the sound stream selected by the focus-of-attention mechanism and given to a spoken language processing (SLP) system. If, for example, a segregated sound stream is that of a closing door or of breaking glass, the SLP may change the thread of discourse to a sudden happening by switching attention to the stream. This simple application of sound stream segregation is expected to make SLP robust and adaptive in a noisy environment, since it extracts a voiced speech stream from the real-world nonvoice sounds. Many techniques for this have been proposed so far. Brown, for example, uses auditory maps in sound stream segregation [6,7], but it is an off-line (or batch) algorithm in the sense that any part of the input is available to the algorithm at any time and for many applications off-line algorithms are not suitable. Additionally, it is not easy to incorporate schema-based segregation and grouping of streams into such a system because it does not support a mechanism for extending capabilities. We adopted the multi-agent paradigm to model sound stream segregation, partly because the number of sound streams is neither fixed nor constant and partly because it allows us to design a more flexible and expandable system. 3. SOUND STREAM SEGREGATION BY MULTI-AGENT PARADIGM The sound stream segregation system has four functions: determining when a stream appears, tracing the stream, detecting the end of the stream, and resolving
505
Tracer generator I tR~sidu~ I
Event ~'_ detector
'l I I I I I
Tracer Extract rl information Predict I .~ next input
I I i
"-
Predictednextinput
r'I I I I I
I
L._
I
'--~ L- -I
i
-----I
I
Tracer Extract information Predict v nextinput
I~
FI
Streamfragment
'~Streamfragment ~
J
Predictednextinput
I I
I '__
.~
Noise-Tracer Extract I [~i information Predict next input
I'-~Noisestreamfragment Predictednextinput
Figure 2. Residue-driven architecture for sound stream segregation. interference between simultaneous streams. We have designed and implemented a multi-agent stream segregation system MASS [8,9]. Since a number of sound attributes (such as onsets, pitch, and common modulation) are available for use in segregation, there are m a n y possible kinds of MASS systems differing in the attributes used. The modeling of the MASS systems is unique, however, and is called the r e s i d u e - d r i v e n a r c h i t e c t u r e . Each MASS system t h a t extracts sound streams by using some sound attributes is called a n agency. Agencies interact with each other to refine segregated streams or to replace other segregated streams with their own streams [2]. The residue-driven architecture uses three kinds of agents: an event detector, a tracer generator, and tracers (Figure 2). An agency based on this architecture extracts streams as follows: 1. An event detector subtracts a set of predicted input from the input and sends the residue to the tracer generator and tracers. 2. If the residue exceeds some threshold value, the tracer generator searches for the values of focused sound attributes. If it finds appropriate values of the attributes, it generates a tracer to trace on the attributes. If it fails to find such values, it generates a noise tracer. 3. Each tracer extracts a stream fragment by tracing the attributes of the stream.
506 It also composes a predicted next input by adjusting the segregated stream fragment with the next input and sends this prediction to the event detector. In other words, tracers are dynamically generated and terminated in response to the input. Amd because segregation is performed incrementally, on sufficiently fast hardware the system will be able to operate in real-time. The first implementation of the MASS uses only the harmonic structure of sound. The reasons are twofold: (1) the general representation of a sound has not yet been proposed, and (2) we need an attribute of a sound as simple as possible so that we can use it as a building block to manipulate more sophisticated attributes. The tracer generator is designed as a set of pitch-watcher agents. It is a kind of filter bank. Each pitch watcher has its own frequency region (about 25 Hz wide) and watches a residual input to see whether a new harmonic stream with its fundamental frequency in the watcher's region appears. The harmonic intensity Et(w) of the sound wave xt(7-) at frame t is defined as
Et(w) = ~ ]1 Ht,k(w) I12, k
where
Ht,k(w) = ~ Xt(T)" exp(--jkwT), r
T is time, k is the index of harmonics, xt(T) is the residual input, and Ht,k(w) is the sound component of the kth overtone. For the sound consistency check, we use a valid overtone for a harmonic stream and dormant period for a noise stream. An overtone is defined valid if the intensity of the overtone is larger t h a n a threshold value and the local time transition of the intensity can be approximated in a linear manner. The period is defined dormant if there are only non-harmonic sounds. The average spectrum intensity of the noise is calculated during the dormant period. We also use valid harmonic intensity, E~(w), which is defined as the sum of the [I H t , k ( w ) [ I of valid overtones. A pitch watcher is activated when the following conditions are satisfied: (1) E~(w)/Et(w) > r (r - 0.1), and (2) there is a power peak near frequency v in the residual input, where w is the frequency t h a t maximizes E~(w) within the region. Since there is more t h a n one activated pitch watcher, the one t h a t gives the maximum Et(w) is selected by the tracer generator and generates a tracer. If there is no active pitch watcher during a dormant period, the noise tracer is activated. When a harmonics tracer is generated, it gets the initial fundamental frequency from a pitch watcher, and at each residual input each tracer extracts the fundamental frequency t h a t maximizes the valid harmonic intensity Et(w). It then calculates the intensity and the phase of each overtone by evaluating the absolute value and the phase of Ht,k(v). It creates a predicted next input in a waveform by adjusting the phase of its overtones to the phase of the next input frame. It also recovers the actual input by adding its predicted input to the residual input before calculating the fundamental frequency. If there are no longer valid overtones, or if the intensity of the fundamental overtone drops below a threshold value, it terminates itself. The noise tracer segregates the static noise stream by means of the average spectrum intensity [10]. It calculates the spectrum intensity time average of the residual input during the dormant period. The noise tracer creates a predicted next input in the form of the spectrum intensity and sends it to other agents. When
507
300
300
250
250
200
200
Hz 15o
Hz 150
1OO
100
50
50 ,
0 0
50
100
J
150
i
,
,
200
250
300
O 350
time in unit of 7.5 ms (a) without a noise
O
100
200
300
400
500
600
time in unit of 7.5 ms) (b) with a white noise
Figure 3. F u n d a m e n t a l frequency patterns of segregated sound streams.
a tracer receives a spectrum intensity, it estimates the intensities of its sound components at each frequency by subtracting the predicted values. The predicted next input of the noise tracer inhibits the generator from generating unnecessary tracers and makes the harmonics tracers robust against nonharmonic noise. The noise tracer calculates the average spectrum intensity for a long time range as well as for a short time range; and it terminates itself when the short-time-range average intensity falls below a threshold.
4. E x p e r i m e n t s and E v a l u a t i o n s The performance of the MASS system in segregating harmonic sounds has been evaluated by using several benchmark sounds, and here we present the results only for two benchmarks; a mixture of the voiced speech of a m a n and of a woman both saying "ah-ee-oo-eh-oh" (the vowel sounds of Japanese) and with or without a white noise. Both speeches do not have any common fundamental frequency but fundamental frequencies of the man's speech coincided with overtones of the woman's speech. Such coincidence may cause incorrect segregation. The SignalNoise ratio of the white noise to the man's speech is 1. Sounds are put into the system at each time frame (7.5-ms frame period, with a 30-ms hamming window). Figure 3 shows the fundamental frequency patterns of segregated sound streams. The upper curves are those of the man's voiced speech and the lower ones are those of the woman's voiced speech. Figure 3(a) is the result of segregation from a mixture of the two voiced speeches and shows t h a t one tracer segregates the woman's speech and another segregates the man's speech. Several harmonic tracers were generated because of an incomplete subtraction but were killed immediately. We also synthesize a segregated sound from each segregated sound stream by a simple method and had its quality evaluated by h u m a n listeners. Figure 3(b) is the result of segregation of the same two voiced speeches under conditions with a white noise. Each speech is segregated by one harmonic tracer and it is easy to synthesize a segregated sound because no regrouping is needed.
508 The total number of generated harmonic tracers is 13, and 11 false tracers were generated but killed immediately. 5. CONCLUSIONS The MASS system described here segregates sound streams from a mixture of sounds. Its design and implementation are based on the multi-agent paradigm so that it can extract a variable number of sound streams in accordance with the input. Even though the current implementation uses only the harmonic structure of sounds, it can segregate a man's voiced speech and a woman's voice speech from the mixture of these two speeches and a white noise. We think that the MASS system provides a primitive function that can be used to implement the cocktail party effect. We are now designing and implementing the localization agency by using the MASS system as a building block. This agency will determine the direction of sound sources from input binaural sounds. A future project will be to integrate these agencies with multimodal information, implementing the cocktail party effect by computer in order to widen the communication channel for humancomputer interaction. REFERENCES 1. S. Handel: Listening. MIT Press, 1989. 2. H.G. Okuno: Cognition Model with Multi-Agent System (in Japanese), in Ishida T. (ed.): Multi-Agent and Cooperative Computation H (MACC "92), Kindai-Kagaku-sha (1993) 213-225. 3. M. Cooke, G.J. Brown, M. Crawford, and P. Green: Computational Auditory Scene Analysis: listening to several things at once. Endeavour, Vol. 17, No. 4 (1993) 186-190. 4. A. Bregman: Auditory Scene Analysis, MIT Press, 1990. 5. D. Rosenthal and H.G. Okuno (eds.) Proceedings of the 1995 IJCAI Workshop on Computational Auditory Scene Analysis, AAAI Press, to appear, Aug. 1995. 6. G.J. Brown: Computational auditory scene analysis: A representational approach, PhD thesis, Dept. of Computer Science, University of Sheffield, 1992. 7. G.J. Brown and M.P. Cooke: A computational model of auditory scene analysis, In Proceedings of International Conference on Spoken Language Processing (ICSLP-92), IEEE (1992) 523-526. 8. T. Nakatani, H.G. Okuno, and T. Kawabata : Auditory Stream Segregation in Auditory Scene Analysis with a Multi-Agent System. In Proceedings of the 12th National Conference on Artificial Intelligence (AAAI-94), AAAI (1994) 100-107. 9. T. Nakatani, T. Kawabata, and H.G. Okuno: Unified Architecture for Auditory Scene Analysis and Spoken Language Processing. Proceedings of International Conference on Spoken Language Processing (ICSLP-94), IEEE (1994) 14031406. 10. S.F. Boll: A Spectral Subtraction Algorithm for Suppression of Acoustic Noise in Speech, In Proc. of International Conference on Acoustics, Speech, and Signal Processing (ICASSP-79), IEEE (1979) 200-203.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
509
The Effects of Rehearsal on Visual Memory Mamoru UMEMURA a, Hiroshi ICHIKAWA b and Kenichi TEGURI ° Science University of Tokyo 1-3 Kagurazaka Shinjuku-ku Tokyo 162 Japan.
bSanno College, Jiyugaoka 6-39-15 Todoroki Setagaya-ku Tokyo 158 Japan. ° Asahi Orikomi Co., Ltd. 3-9-14 Kudanminami Chiyoda-ku Tokyo 102
Japan.
1. INTRODUCTION Recently the development of information equipment including computers has advanced rapidly. Accompanying it, the opportunity of using the man-machine interface using CRT display has increased. It is considered that in the case of the works that are carded out by reading the information displayed on a CRT, the easiness of observing the displayed information exerts large influence to the accuracy and efficiency of the works, fatigue and others. In the works accompanied by danger, its mistaken cognition may sometimes be linked with serious accidents. Accordingly, the easiness of its cognition is an important subject in view of safety. The easiness of recognizing the information displayed on CRTs has been reported regarding the color, shape, size and so on of display [1]. Besides, investigation has been carried out on the information processing by humans after reading display, especially on the maintenance of short-term memory [2-3]. In order to take proper judgment and action after reading displayed information, it is necessary to make information easy to be maintained in short-term memory. As its means, there is rehearsal, and it has been said that it is effective for maintaining information. Also in real scene, in the case of the works accompanied by danger, the confirmation of information by voicing has been frequently carried out. Therefore, this study aims at quantitatively investigating the following items by experiment. (1) The effect that rehearsal exerts to the time interval of presentation. (2) The effect of the rehearsal carried but by voicing in the case of continuously presenting information on CRTs.
510
2. METHODS The subjects were seven males of 20-odd years old having normal eyesight. At the same positions of CRT display, seven capital letters of alphabet and numerals were displayed at random, and the subjects were asked to answer what were the displayed letters. The answer was decided to be free recall experiment, the order of which may not always be the same as the order of presentation. The size of the letters was 32 x 32 dots. As for the presentation time, seven kinds from 100 to 700 ms at 100 ms intervals were set up, and the presentation for respective times was carried out 25 times at random. Before and after presenting a series of letter stimulus, as the masking letter, the square of 32 x 32 dots was displayed for 0.5 sec at the same position where the letter stimulus is presented. The experiment was carded out in two ways, namely the case of carrying out rehearsal and the case without rehearsal.
3. RESULTS
3.1. Presentation time Figure 1 shows the relationship between the presentation time and the probability of recall. In all the subjects, the probability of recall became higher when rehearsal was carried out, and the effect of the rehearsal on the maintenance of short-term memory was recognized. The probability of recall was compared with the mean values of all the subjects, and it is shown in figure 2. From the finding show in this figure, the value in the case of carrying out rehearsal is higher about 7-15% than that in the case without rehearsal. The presentation time when this difference between both values became the maximum was the case of 700 ms with 15.4%. The presentation time when it became the minimum was the case of 100 ms, and the difference was 6.7%, in this way, as the presentation time became longer, the tendency of the difference becoming larger was observed. Moreover, as to the number of recalled letters, in the case without rehearsal, the dispersion among the subjects was large, but in the case of carrying out rehearsal, the dispersion among the subjects became small. In the presentation time of 100-300 ms, the probability of recall increased relatively largely. However beyond 300 ms, the increase became slow. This is the similar tendency in both cases of carrying out rehearsal and without rehearsal, but in the case of carrying out rehearsal, the increase of the probability of recall became larger. When the probability of recall for each subject was observed, in the presentation time of 100 ms, the subject, whose probability of recall was smaller in the case of carrying out rehearsal than in the case without rehearsal, was observed. 3.2. Serial position Figure 3 shows the typical example (Subj. A) of the relation of the order of presenting letters (serial position) with the probability of recall. In both cases of
511
With rehearsal 1O0
~
A~-~.
-J-" Subj.A
80
--ffl- Subj.B - * - Subj.C Subj.D - A - Subj.E
O
40 ..--4
~
--ZX-- Subj.F
2o
O
Subj.G 0
I
100
200
I
I
,
I
,
300
400
500
600
700
Presentation time(ms) Without rehearsal 100
Subj.A
~-
80 ~
o
60
- U - Subj.B - * - Subj.C --O-- Subj.D - A - Subj.E
O
. ,.,,q
Subj.F Subj.G 0
i 100
200
,
,
,
I
I
300
400
500
600
700
Presentation time(ms) Fig. 1 The relation between presentation time and probability of recall
80 t
Proba ilityofrecall
~" o 70
35 r
/.23
D------_{2------~----.-.13
- I I - With rehearsal
30 ~" -tzy-- Without rehearsal
~-, 60 o
25~ '-' Increase rate
;~ 50
15~
.~
20
-
10 -`=
I
d: 40
100
200
300
400
500
600
700
exposure duretion(ms)
Fig.2 Probability of recall and Increase rate carrying out rehearsal and without rehearsal, in the former part and the latter part of serial position, the probability of recall was high. Besides, in the case of carrying out rehearsal as compared with the case without rehearsal, in the former part like the serial position 1-4, the probability of recall has increased.
512 90 80 ~" 70 ~O .~ .....0 ~ .0
50 40 30 20 b
-0-- Withoutrehearsal
10 0 1
2
3
4
5
6
7
Sirial position Fig.3 Probability of recall on Sirial position (Subj.A)
[~ Subj.A N Subj'B Subj.C n Subj.D Subj.E O l=
I~ Subj.A D Subj.B Subj.C 1 Subj.D
~ 50t ~'40 i!~!i!i!ili "~ !:!:i:i:!:i:i ~0 3 0 •:~:~:~i~!i ~ 20 !i~i~i:~i!
1[-] Subj.F Subj.G =1001L , 1
2
3
4
5
6
7
-100
,
,
i in ,
,
Subj.F 01 Subj.G
Sirial position
3 4 5 6 Sirial position
Fig.4 The effect of rehearsal in Sirial position
Fig.5 The effect of rehearsal in sirial position(Presentation time:300ms)
1
2
,
7
In the latter part from the serial position 5, even if rehearsal is carded out, the increase of the probability of recall was not observed, and on the contrary, also the case of lowering the probability of recall was observed. The effect of rehearsal in serial position is shown in figure 4. This is the summation of the increase rate of recalled letters for all the subjects when rehearsal was carried out. At the serial position 2, the rate of increase was largest, and the serial positions 1, 3 and 4 followed as for its values. It is known according to this figure that the effect of rehearsal was large in the former part 1-4 of serial position. Figure 4 shows the mean value of all the presentation time. However in the short presentation time, it is considered that the time sufficient for executing rehearsal cannot be taken. Therefore, the case of the presentation time 300 ms is shown in figure 5. From this figure, it is known that at the serial positions 1, 2 and 3, the rate of increase was large, and subsequently, it was large at the serial position 4.
513 4. DISCUSSION
4.1. Optimal presentation time In the case of the presentation time longer than 300 ms, it was found that by carrying out rehearsal, the probability of recall heightened by more than 20%. By carrying out rehearsal, the attenuation of information in short-term memory can be stopped, and it can be maintained. From this fact, it was expected that by carrying out rehearsal, even if the presentation time is short, the probability of recall is high. However, the results of this time became such tendency that in both cases of carrying out rehearsal and without rehearsal, at the presentation time of 100-300 ms, the probability of recall increased rapidly, and after exceeding 300 ms, much increase did not arise. This is considered that in the rehearsal by voicing, letters are recognized and those are pronounced, consequently, in the short presentation time, the cognition of the following letters cannot catch up, and in this way, it occurs. Accordingly, in the continuous presentation of letters using CRT display, at the presentation time shorter than 200 ms, the cognition of letters cannot catch up the presentation, and the maintenance of information in short-term memory becomes difficult. Moreover, at 400 ms or longer, even if the presentation time is increased, the probability of recall does not much heighten. Accordingly, the efficient presentation time is considered to be about 300-400 ms. Generally regarding the maintenance of short-term memory, it has been known that the difference according to individuals is large. Consequently, in the visual sense monitoring works of carrying out control and monitoring by using VDT equipment, there are the workers who are apt to cause mistaken reading and those who are not so. This is an important problem in view of safety, for example, in nuclear power plants. By seeing figure 1, it is observed that the execution of rehearsal makes the difference according to individuals in the probability of recall small. From this fact, it is considered that the execution of rehearsal in visual sense monitoring works makes the difference according to individual workers small, and is effective for the prevention of the mistaken reading of information.
4.2. Serial position and effect of rehearsal The probability of recall is high in the former part and the latter part of serial position in both cases of carrying out rehearsal and without rehearsal. This is the typical serial position curves [4]. Namely, the former part is that called primary effect, and the latter part is that called recency effect. As for the recency effect, the similar probability of recall was observed in both cases of carrying out rehearsal and without rehearsal. However, the primary effect in the case without rehearsal has not arisen conspicuously. Unless rehearsal is carried out, the information is short-term memory vanishes in relatively short time [5]. Generally, it has been said that the vanishment occurs due to the following causes. (1) Proactive inhibition that is the interference from the information inputted earlier [6].
514 (2) Retroactive inhibition that is the interference from the information inputted later [7]. The increase of the probability of recall when rehearsal was carded out was large in the former part of the serial position 1-4. To the former part of serial position, the effect of proactive inhibition is small, and it is considered that by strongly undergoing the effect of retroactive inhibition, information progressively vanishes. According to these facts, it can be said that the execution of rehearsal lowered the effect of retroactive inhibition, and increased the probability of recall as a whole. In the visual sense monitoring works using the CRT display that continuously presents information, the information in the former part of serial position (the information that was displayed earlier) is hard to be maintained in short-term memory. However, it is considered that by carrying out the rehearsal by voicing, the information in the former part of serial position can be maintained, and mistaken cognition can be prevented.
REFERENCES 1. T.Hatada, Characteristics of human vision for VDT, The Japanese Journal of Ergonomics, 22, 2, 45-52 (1986). 2. H.Yokoi and H.Yuma, Optimum presentation speed of character sequence based on time continuous model of short-term memory, The transaction of mICE, J70D, 11, 2327-2337 (1987). 3. G.Sperling, A model for visual memory tasks, Human Factors, 5, 19-39 (1963). 4. B.B.Murdock Jr., The serial position effect of free recall, Journal of Experimental Psychology, 64, 5, 482-488 (1962). 5. L.R.Peterson and M.J.Peterson, Short-term retention of individual verbal items, Journal of Experimental Psychology, 58, 3, 193-198 (1959). 6. G.Keppel and B.J.Underwood, Proactive inhibition in short-term retention of single items, Journal of verbal Learning and verbal Behavior, 1, 153-161 (1962). 7. N.C.Waugh and D.A.Norman, Primary memory, Psychological Review, 72, 89-104 (1965).
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
M e c h a n i s m s o f Slips in D i s p l a y - B a s e d H u m a n - C o m p u t e r
515
Interaction:
A Model-Based Analysis Muneo Kitajima a and Peter G. Polson b aNational Institute of Bioscience and Human-Technology, 1-1 Higashi Tsukuba Ibaraki 305, JAPAN blnstitute of Cognitive Science, University of Colorado, Boulder, CO 80309-0344, U.S.A.
1.
E R R O R S O F S K I L L E D U S E R S IN H U M A N - C O M P U T E R
INTERACTION
A puzzling and frequently-ignored fact in the human-computer interaction literature is that skilled users have surprisingly high error rates (10 ~ 15%). Card, Moran and Newell (1983) studied individual skilled users performing two tasks, manuscript editing and electronic circuit design editing. The manuscript editing experiment involved a detailed evaluation of a single expert user doing 70 edits presented in marked up manuscript. Errors were made on 37% of the command sequences describing edits. Over half of the errors were detected and corrected during generation of the editing commands. 21% (15 out of 70) of the commands issued by this very skilled user generated the wrong result and required additional edits to correct these errors. In a second study of a single expert carrying out an electronic circuit design editing task, the user had an error rate of 14% on 106 edits. Hanson, Kraut, and Farber (1987) studied 16 researchers and managers who were intermediate and expert level users of UNIX performing document preparation tasks and e-mail. They logged over 10,000 commands. The overall error rate was 10% with error rates ranging from 3% to 50% on different commands. The experiments briefly reviewed here are representative of results from a wide range of studies in the human-computer interaction literature. Error rates for expert users range from 5 to 20%. In all studies of experts, users eventually produced the correct results. Approximately 50% of the errors are detected during the generation of a command and corrected. Detection and correction of errors is an integral part of expert skill. The literature on errors has concluded that there are two qualitatively different types of errors (Norman, 1981; Reason, 1990). The first is errors of commission, or mistakes. Such errors are committed by users who are carrying out a novel tasks and fail to immediately discover the correct action sequence. The other is slips, where users have the correct intention but fail to successfully execute the correct action sequence. Most part of errors described above is slips.
516 Sellen (1990) reviews classes of models that provide principled, qualitative accounts for slips. She argues that all of these models have a hierarchical representation of action sequences that include representations of top-level task goals and lower-level goals that actually control execution of elementary actions. Reason (1990) argues that control of attention is a critical determinant for generating correct performance from a hierarchical representation of action sequences that include representations of top-level task goals and lower-level goals that actually control execution of elementary actions. Failure to adequately attend to the ongoing process and coordinate the interaction between the various schema causes a wrong low-level schema to become activated, generating related but incorrect actions for the current task. In HCI tasks, the users could be focusing on the task of composing new text or drawing a figure, and so on. This would lead to insufficient attention being allocated to subtasks involved in operating the interface. Card, et al. (1983) proposed that experts accept high error rates in order to increase their productivity, because for them error recovery can be done easily and rapidly. Experts trade speed for accuracy, causing slips. In this paper, two mechanisms of slips, attention failures, and speed-accuracy tradeoffs are simulated by a comprehension-based cognitive model of display-based human-computer interaction proposed by Kitajima and Poison (1992, 1994a, to appear), showing that they could account for the rate of slips made by skilled users interacting with graphical user interfaces (Kitajima and Poison, 1994a, 1994b).
2. A MODEL OF DISPLAY-BASED HCI The model developed by us (Kitajima and Poison, 1992, 1994a, to appear) is shown in Figure 1. The model elaborates Hutchins, Holland, and Norman's (1986) action theory framework which consists of the following four basic components: (1)
goals representing what the user wants to accomplish which are a schematic outline of the action sequence that will accomplish the task,
(2)
a task environment which is the world that reacts to the user's actions and generates new responses by modifying the display,
(3)
the stage of evaluation comprised of the processes that evaluate and interpret the display, and
(4)
the stage of execution comprised of the processes that select and execute actions that affect the world.
Our model of the Hutchins, Holland, and Norman's (1986) action theory incorporates goals, two processes for the stage of evaluation and two for the stage of execution.
517
Task Goals Device Goals e of Evaluation
Stage of Execution Selecting Candidate Objects for Next Action
Elaborating the Display
Action Cycle Se'ecting' cti°n
I
I
Generat'ng ,=,eoresen,ations °'s°'a''
World Figure 1.
2.1.
The comprehension-based cognitive model of skilled use of graphical user interfaces, mapped onto Hutchins, Hollan, & Norman's (1986) action cycle.
Task Goal and Device Goal
The model assumes that skilled users have a schematic representation of the task that is in the form of a hierarchical structure involving two kinds of goals: task goals and device goals. Our goal representation is taken directly from the Yoked State Space Hypothesis proposed by Payne, Squibb, and Howes (1990). Payne, et al. assume that discovering how to carry out a task involves searching of two problem spaces. The first is a space of possible task states. The second is a space of possible device states that are required to achieve a given task state. We assume that each task goal is associated with one or more device goals. The device goals specify device states that must be achieved in order to satisfy an associated task goal. Given a task goal and its associated device goals, the model simulates a sequence of action selections as follows.
518
2.2.
Generating Display Representations
At first, the model generates a representation of the display. The display representation only includes information about identity of each object on the display and its appearance, e.g. highlighted, pointed-at, dragged, etc. No information about what actions can be taken on the object, or its meaning and relationships to other objects in the display is included in this initial display representation.
2.3.
Elaborating the Display
All such information is generated by the elaboration process which retrieves information from long-term memory by a random memory sampling process. The retrieval cues are the representations of the current display, the task goal and the device goals. The probability that each cue retrieves particular information in a single memory retrieval process is proportional to the strength of the link between them. The model carries out multiple memory retrieval in a single elaboration process. A parameter, the elaboration parameter, controls the number of times each argument in the display and goal representations is used as retrieval cues 1. The retrieved information elaborates the display representation, providing information about interrelationships between display objects, relationships between the task and display objects, and other attributes of display objects. The elaborated display representation is model's evaluation of the current display in the context defined by the task goal and the device goals.
2.4.
Selecting Candidate Objects for Next Action
In the stage of execution, the model first limits its attention to a few number of screen objects out of---100 objects displayed on the screen. These screen objects are candidates for the next action to be operated upon. The candidate object selection is performed on the basis of the evaluation, defined by the elaborated display representation. The model uses the spreading activation mechanism to select candidate objects. The process is dominated by two factors: the strengths of links from the representation of the goals, which is parametrized by a parameter, the attention parameter, and the number of propositions that are necessary to bridge the goals and the candidate objects 2.
1The model represents goals and display in propositions, like (is-on-screen OBJECTI2). In the memory sampling process, the argument, such as OBJECT12, is used to retrieve information from long-term memory that has OBJECT12 as its argument. 2The model assumes the argument overlap mechanism to link up propositions. For example, the two propositions, (is-on-screen OBJECT12) and (has OBJECT12 CalculatorMenultem), are linked by the shared argument, OBJECT12.
519
2.5.
Selecting Action
The model considers all possible actions on each candidate object. The model incorporates 18 possible actions 3, such as "moving the mouse cursor to a menu item in order to display a pull-down menu." The process is dominated by the same two factors described above. Furthermore, the action representations include conditions to be satisfied for their execution. The conditions are matched against the elaborated display representations. Some conditions are satisfied by the current screen, others by information that was retrieved from long-term memory in the elaboration process. For example, the model cannot select an action to double click a document icon for editing unless the icon is currently pointed at by the mouse cursor and the information is available that the icon can be double clicked. Observe that if information about a necessary condition is missing from an elaborated display representation, the model cannot perform that action on the
3.
incorrectly described object.
HOW THE MODEL ACCOUNTS FOR ERRORS
In a set of experiments we conducted so far, where a graph drawing task was simulated, we found that the model could cause errors due to the following three reasons. The first is that the process of selecting candidate objects for the next action fails to include the correct object on the list of candidate objects. The second possible cause of errors is that the correct action fails to become the highest activated action among executable actions. In the model's terms, these kinds of errors are ascribed to both or either of small values of the attention parameter (A), and/or missing bridging knowledge that had to be retrieved from longterm memory (B). The third is that the elaboration process fails to incorporate all of the conditions for the correct action in the elaborated display representation. Low values of the elaboration parameter cause this error (C). Parameter values in the range of 12 to 20 caused the model to simulate error rates in the range of 10% to 20% (Kitajima and Poison, 1994, to appear). We argue that the elaboration parameter describes a speed-accuracy tradeoff process where low values of the parameter reduce the amount of time taken by the elaboration process.
4. C O M P A R I S O N W I T H O T H E R M O D E L S The strength of the model is that the model generates correct actions as well as occasional errors without assuming a special set of mechanisms to produce erroneous actions. In this
3Representations of actions define different functions of single physical actions in many different contexts. For simulating a graph drawing task, the model defines eighteen cognitive actions on six physical actions; MoveMouse-Cursor, Single-Click, Double-Click, Hold-Mouse-Button-Down,Release-Mouse-Button,and Type.
520 respect, the model is strikingly different from typical models of expert performance and error (Anderson, 1993; Reason, 1990; Card, et al., 1983). Typical models assume that skilled performance is mediated by detailed, large grain size action plans stored in long-term memory. Card, et al. (1983) refers to them as methods; Reason (1990) assumes that skilled performance is mediated by action schemata (Norman, 1981). Thus they have to be equipped with erroneous plans to generate errors. The grain size of action is much smaller in our model, at the level of individual pointing action. When the model makes an error, it has attempted to select a correct action based on incomplete knowledge, and/or insufficient attention. The incorrect action will be highly constrained by the user's current goals, the current state of the display, and the partial knowledge that was successfully retrieved from long-term memory. The candidate objects and the next action selected by a simulation are the model's best selections given the context represented by the elaborated display representation.
REFERENCES
Anderson, J. R. (1993). Rules of the mind. Hillsdale, New Jersey: Lawrence Erlbaum Associates. Card, S. K., Moran, T. P., & Newell, A. (1983). The Psychology of Human-Computer Interaction. NJ: Lawrence Erlbaum Associates. Hanson, S. J., Kraut, R. E., & Farber, J. M. (1984). Interface design and multivariate analysis of UNIX command use. ACM Transactions on Office Information Systems, 2, 42-57. Hutchins, E.L., Hollan, J.D., & Norman, D.A. (1986). Direct manipulation interfaces. In Norman, D.A. & Draper, S.W., Eds. User Centered System Design. Hillsdale, NJ: Lawrence Erlbaum Associates. Kitajima, M. and Poison, P.G. (1992). A computational model of skilled use of a graphical user interface. Proceedings of CHI'92 Conference on Human Factors in Computing Systems. NY: ACM, pp. 241-249. Kitajima, M. & Poison, P. G. (1994a). A comprehension-based model of correct performance and errors in skilled, display-based human-computer interaction. ICS Technical Report. 94-02. Boulder, CO: Institute of Cognitive Science, University of Colorado. Kitajima, M. and Poison, P.G. (1994b). A model-based analysis of errors in display-based HCI. Conference Companion CHI'94. pp. 301-302. Kitajima, M. & Poison, P. G. (to appear). A comprehension-based model of correct performance and errors in skilled, display-based human-computer interaction. International Journal of Human-Computer Studies. Norman, D. A. (1981). Categorization of action slips. Psychological Review, 88, 1-15. Payne, S. J., Squibb, H. R., & Howes, A. (1990). The nature of device models: The yoked state hypothesis and some experiments with text editors. Human-Computer Interaction, 5, 4, 415-444. Reason, J. (1990). Human Error. New York, NY: Cambridge University Press. Sellen, A. Mechanism of human error and human error detection. Unpublished doctoral dissertation (1990).
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
521
Computation Model for Human Communication Masahiro HIJ1.1, Hiroshi NUNOKAWA .2 and Masatoshi MIYAZAKI *l *l Graduate School of Information Sciences, Tohoku University, kawauchi Aoba, Sendai 980-77, Japan .2 Institute for Science Education, Miyagi University of Education, Aoba aza Aramaki Aoba, Sendai 980, Japan Abstract We have propose a computation model for human communication which has both dynamism and variety. And we propose a possibility to represent human communications as a figure of communication. This makes the computation model enable to model a dynamism in human communication smoothly. We design a programming language based on the computation model.
1. Introduction
Due to the increase in size, popularity and individual use of the computer network nowadays, their use as a means for inter-personal communications becomes increasing. Furthermore, as a communication system, they can also be common platform for cooperative activities which occurs while exchanging informations with other people via communications and coordinating activities and informations with each other. This is because, in the cooperative activities, communication is one of indispensable primary factors and even the cooperative activity itself can be considered as a particular activity caused by communications. Consequently, the communication system on computer network, as a recent information technology, is widely applicable. The development of such system becomes more and more important. For constructing system for computer-mediated communications on computer networks, this system must be able to correspond to varieties of human communications ,- e.g. a real-time person-to-person conversation, sending letters to specific person, creating a group concerned with a particular topic along with conversations in the group. In addition to this, it must be able to correspond to a dynamism of communication while switching among these various communications occur. Researches on a human communication so far emphasize analyzing of the influence of communications on human-beings and have been carried out in a field of sociology [1][2]. Therefore it is impossible to directly make use of the results of these researches as a basis to create a system that supports computer-mediated communications above. Based upon these background , we emphasize our research on the way to model a human communication, its implementation of and the development of the computer-mediated communication system on computer networks. Our aim is to provide a model for analyzing a human communication and also a basic theory of constructing a computer-mediated communication system. In this papers we propose a method of modeling a human communication from a view point of communication figure and its implementation.
522
Interaction~ ~ ~ ) - "
c,. . . .
~,,,,; . . . .
Communlcauons
.[ mteracuon ':. Computer-Mediated % Communication System
| ~::~
Interaction
"- ( " ~ ~ Q ~"~1~'~
•=======================================.
Interaction 4"~t.~~! .... :ii~ ..........~......:.:.:.:.:.:.:.:~:.:.:.:.:.:.:.:.::::::::::.:~:.::::::~:~:~::"
I~mmm
l1,,, u,,,
Figure 1. Computer-Mediated Communication System
2. Analysis of the Human Communications 2.1. Human Communications as the Target of Analysis In general, human communications can be analyzed from several view of points. In this paper, our meaning of human communications is as follows. In order to clearly define the computation model for human communications which will be the basis for the development of computer-mediated communication system. Generally speaking, human communication can be considered as choosing a selection of communicative partners for exchanging informations, and a process of exchanging informations among these selected partners. First, we think that human communications consist of some primary objects which exchange informations and the figures of these information exchanges [3]. In other words, it is a process of information exchanges consisting of these two elements. We call the primary object which exchanges informations and interpret these informations as communicator. There are two kinds of communicators. One of them is the sender which gives informations, the other is the receiver which takes informations from the sender in an exchange of informations. Communicator can be either sender or receiver. It means that a relation between the sender and the receiver is not fixed, communicator can be either the sender or the receiver in the process of information exchanges. Communicators includes not only individual person but also a group of people. When a group of people is the object for communication, it is considered as one independent object which communicates and is represented as one communicator. We call these figures of information exchange among communicators as figure of communication. It is a mode of an exchanging informations defined as a triple of-- the number of communicators, the direction of information flow and the timing of information exchange. The number of communicators means the combination of numbers of senders and receivers. It is represented in the form of 'number of senders TO number of receivers', and is any one of these four types of '1 TO 1', '1 TO n' , ' n TO 1' and 'n TO n', where n means any number more than 1. The direction of information flow is the direction which information flows among communicators. This is either' one-sided direction' or' mutual direction'.' one-sided direction' is the case when sender gives informations to receiver in one way.' mutual direction ' is the case when sender and receiver exchange informations mutually. The timing of information exchange means the time necessary to exchange informations between sender and receiver. This is either' realtime ' or 'non-realtime '. 'realtime 'is the case when the receiver reads out informations as soon as receiv-
523 ing it.' non-realtime' is the case when some time is necessary for the receiver until it reads out the information being sent. This is because either the means of communications itself need some time in transmission (non-realtime means ) , or the receiver needs some time in reading out (non-realtime from the activity of receiver). For realtime information exchanges, means of communication must be realtime and the receiver communicators must read out the received informations at once. From the viewpoint of information exchanges in the process of human communications, human communications can be modeled as a process in which the sender communicator selects the receiver communicators and exchanges information based on the figure of communication which is decided by number of communicators, the ability of the receiver communicator in doing a real time information exchanges as well as the direction of information flow. A sequence of communication proceeding with the time is modeled by combining the figures of communication with the flow of time.
2.2. Dynamism of the Human Communications Human communications are generally dynamic. It means that the number and the state of communicators as well as the figure of communications vary according to the process of human communication. Also the relation among communicators corresponding to the communication in similar way vary by changing an figure of communication. This is called as the dynamism of communication. So, when computer-mediated communication system are constructed, it must be able to correspond to the dynamical side of the human communication. To cope with such a dynamism of the human communication, we modeled the human communication based on the cooperative computation model [4] with the concept of autonomous decentralization. This dynamism of communication has not been considered yet in the communication model so far.
3. Computation Model for Human Communication 3.1. Modeling of Human Communication We propose a computation model for modeling a human communication based on the analysis of human communications in chapter 2. Our computation model uses communicators and an exchange of informations among them. In this model, a kind of information exchanged in communications and behavior of communicator varied with communications are not taken into consideration. It means our computation model considers human communication as a selection of communicators as communication partners and an exchange of information among them. This computation model gives basis for expressing human communication as described in section 2.1 In this computation model, an individual communicator is represented by an autonomous object, a group communicator is represented by afield. An autonomous object is similar to an object in Object-Oriented computation model. The autonomous object is same as the object described in Object-Oriented computation model, except that, it has an interpreter in itself to change its function by the execution of receiving scripts from other objects. The interpreter in autonomous object has high-order function which can interpret script as data. We represent communicators as autonomous objects or fields, communicators can be either a human or a software. Software communicators are different from human communicators in the sense that their ability to interpret the exchanged informations in communication are limited and these informations must be structured in some forms. Field is a boundary which distinguishes a group of autonomous objects from other autono-
524 mous objects. A Field as communicator sends information received from other communicators to all autonomous objects or certain autonomous objects belonging to it. Also information from an autonomous object belonging to a field is sent to other receiver communicators through the field. These fields do not directly exchange information by their own, but autonomous objects belonging to the field indirectly exchange information through the field. The computation model can represent any communication based on the figure of communication described in section 2.1 as follows. The number of communicators is decided upon whether communicator is described as autonomous object or field. The direction of information flow is represented as two kinds of correspondence form named as the Send and ContractSend. Send is that sender only send informations to receiver in one way. ContractSend is that communicators contract and exchange informations among them. The timing of information exchange is represented as two kinds of correspondence form named as lnformSend and CooperativeSend. CooperativeSend is the CooperativeScope which recognize all communicators able to do a real-time communication. A real-time communication is then done among communicators with the same CooperativeScope. CooperativeSend is a figure which assure that information will not only be sent to the receiver in real time but also be read out by the receiver immediately, lnformSend differs from CooperativeSend in the sense that it does not assure neither a real-time sending nor an immediate reading out of informations. It leaves the reading out as the responsibility of the receiver. The figure of communication are represented as combinations of individual correspondences form above these are InformSend, lnformContractSend,
CooperativeSend, CooperativeContractSend and MessengerSend. A MessengerSend specifies the conditions as the destination as well as communicator list (called known-information ) and then sends a message [5]. The message being sent will search for communicators which satisfies the specified conditions according to known-information. The message gets these known-information which from other communicators while moving to those communicators. In other words, MessengerSend is a communicative figure searching for communicators which satisfy the specified conditions while getting known-information from other communicators. That is to say, the figure of communication in section 2.1 is represented by the above five communicative figures which specified the receiver communicator to each destinations in these figures. 3.2. Describe Human Communication based on our model The programming language which we designed based on our computation model in section 3.1, it is possible to describe communicators and figure of communication using each function in Figure 2 and Figure 3. Communicator which is a partner in communication is designated as parameter < toField>
( CreateAutonomousObject ) ( CreateField <ScopeName> ) ( DeleteField ) ( InField ) ( OutField ) ( OpenScope <ScopeName> ) ( CloseScope <ScopeName> ) Figure 2. Functions for describing communicators
525 ( InformSend <Script> ) ( Contract ) ( InformContractSend <Script> ) ( ContractReply ) ( ContractEnd ) ( WholnScope <Scope> ) ( CooperativeSend <Script> ) ( CooperativeContractSend <Script> ) ( CreateMessenger <MessengerName> ) ( MessengerSend <MessengerName> ) ( MessengerReceive ) Figure 3. Functions for describing figure of communication
and in each communicative functions. An exchanged information through communication is described as a script <Script> in these communicative functions .A script as described in <Script> in these functions is exchanged among communicators as data by using the interpreter which the autonomous object has. A receiver communicator interprets and executes the received data as script. In our model, the interaction method between communicator along with informations are included inside the exchanged data. A process of a sequence of communication is described as combination of these communicative functions. Moreover, dynamism in human communications is interpreted as follows. A dynamism of number of communicators is described as creation and elimination of an autonomous object and field, participation and going out of autonomous objects using these functions in Figure 2. A dynamism of figure of communication is described as selection of the communicative function corresponding to the figure of communication as well as information exchanges by using it. An autonomous object can change its internal status as well as its script and can vary the figure of communication by interpreting the script which is exchanged as data among these objects. As a result, our computation model can be used to interpret human communication as described in chapter 2.
3.3. Example of modeling human communication Figure 4 shows an example to describe a communication when a person A sends a question to a group G. Then other two persons B and C belonging to group G make a discuss about the question in real time. Here, each person (A,B,C) is described as an autonomous object and group G is described as a field. Autonomous objects B , C belong to Field G by executing InField function. Autonomous objects B, C open CooperativeScopewhich shows a possibility of realtime exchanging informations by executing OpenScope function. When autonomous object A asks a questions to Field G, it sends the question by using lnformSend function. The question sent from A is transferred to each autonomous object B, C belonging to Field G . An autonomous object ( for example B ) receiving a question knows about a partner ( partner C with the same CooperativeScope ) to whom he can exchange the informations in real time by using WholnScopefunction. Autonomous object B exchanges the informations in real time with autonomous object C by using CooperativeContractSendfunction. If a new person D wants to participate in this discussion, autonomous object D must open the same CooperativeScope as the one of autonomous objects B, C. At this point, informations
526
Group G
person • B r,
"~.... . __......-.../~utonomous~ person : C ~Fleld : ~~'l~k,~ Object : .C-~..~ '~ ............. /~utonomous~ Cooperative ~ f ' " N J ~ t t- Obiect • B -1 ContractSend F ..... ..
answer }n:A
_
pamclpate I n f o r m S e ~ r m S e n ~'~ /rAutonomous~ ~person : D k, Object : A J
"-K..O.__bJ_e__c.t___;_P.,,r~.
~ . ,
d ............................ InField fAutonomous~ k, Object : D J
Figure 4. A sample of modeling human communications
exchanged by using sent to D also.
CooperativeContractSendfunction among autonomous objects B,C will be
4. Conclusion We have proposed a computation model for human communication which has both dynamism and variety. We try to make the range of human communication aimed used in modeling as wide as possible and we also model a dynamism of human communication. Therefore this kind of various human communications can be represented as process in which some communicators select other communicators and exchange informations according to one of the communication figures. We also developed a programming language based on our computation model. We have implemented the programming language based on the computation model and a system for supporting communication in education and production planning field. This programming language is defined as seven functions for describing communicators and eleven functions for describing communication form .This language is implemented on computer network consisting of UNIX Workstation. Human interaction is the essential elements in the information exchanges on computer-mediated communication .In a computer-mediated communication so far, the information being sent is just kept until being read by the receiver, sender can not specify the receiver to read it. In the new computer-mediated communication based on our computation model, sender is able to specify receiver to read an exchanged information or interact to sender in communication.
REFERENCES [1]Barnlund D.C. : "A Transactional Model of Communication" in J.Akin, A.Goldberg, G.Myers, and J.Stewart (eds),Langiage Behavior : A Book of Readings in Communication , Mouton & Co.N.V.Publishers, The Hague ,1970 [2]Giffin K. and Patton B.R. : "Fundamentals of Interpersonal Communication", 2nd ed. Harper & Row, Publishers, NewYork ,1976 [3]Hiji,M.,Nunokawa,H.,and Shiratori,N. :A Design Language Describing a Cooperative Work Model, IPSJ SIG Notes,93-GW-3,pp.33-40,1993 ( in Japanese ) [4]Takemiya,H.,Yano,H.,Nunokawa,H. and Noguchi,S. :Coordinated Computation Model with Coordination Scope, IPSJ SIG Notes,91-PRG-3,1991 ( in Japanese ) [5]Igarashi,T.,Nunokawa,H. and Noguchi,S. :Description of the communication tool using computation model based on an autonomous decentralized cooperative concept, IPSJ SIG Notes,92-DPS-58,pp.165-172,1992 ( in Japanese )
III. 18 Voices and Faces
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
529
Delivering the promise of speech interfaces Charanjit K Sidhu and Gerry Coyle BT Laboratories, Martlesham Heath, Ipswich, IP5 7RE, UK Key words: Speech interfaces, product life cycle, usability engineering.
Abstract We compiled a style guide that includes checklists and methods based on our experience of speech interface design. These have been applied in user trials and field evaluations throughout the development of Call Minder, a network-based telephone call answering service. Numerous improvements were made to the usability of the service as a result.
1 INTRODUCTION Speech technology offers intuitive information services accessible from any telephone worldwide. However, the design of a service has a major impact on ease of use and so we involved customers to steer the design process towards optimal usability [1]. We used this approach throughout the design of Call Minder--a network-based telephone call answering service. We believe that multi-disciplinary teams of software, systems, marketing and Human Factors people are needed to design successful interactive services [2]. The Human Factors contribution includes requirements capture, design, prototyping and evaluation supported by research into user psychology. Checklists, guidelines and methods have been compiled to support the design of usable services. This provides designers and engineers with guidance in the following areas: •
Requirements capture checklists to help specify user requirements;
•
Dialogue Structure and message composition design guidelines for the format and content of speech messages;
•
Keypad guidelines for mapping functions to the telephone keypad;
•
Evaluation guidelines for evaluating services at all stages of development.
Figure 1 demonstrates how information from different sources is utilised in the design of speech services.
530
Tools and Techniques RequirementsCapture,ExpertWalkthroughs,User Trials, Questionnaires,FocusGroups,FieldTrials
Ia
Case Studies ~' Evaluationof SpeechBasedServices eg. Call Minder,NetworkServices, dioPaging,Chargecard,Voicemessaging
¢ Speech Research Speechtechnology,evaluationmethods,user psychology
Figure 1: Optimising 2
Internal Standards SpeechStyleGuide UsabilityDesignGuide
External Standards ISO, ETSI,etc.
usability in developing speech based services
DESIGNING USABILITY INTO CALL M I N D E R
The Call Minder answering service generates voice prompts and interprets spoken or keyed responses. There are two main dialogues: Caller dialogue: When someone calls and no-one answers, or the phone is engaged, Call Minder answers the call and prompts the caller to leave a message. Figure 2 shows a typical interaction with Call Minder. Good afternoon. There is no reply from that number at present, but I can take a message for you. Can I have your name please ? "Charanjit Sidhu" Who are you calling ? "Gerry Coyle" Please leave your message and telephone number after the tone. "Hello Gerry, just demonstrating the system". I recorded your message as: 'Hello Gerry, just demonstrating the system'. You now have a choice. If you are happy with your message, please hang up now. I f you would like to replace it with a new message, please hold on. Your new message will overwrite the old one. Please leave your new message after the tone. "Hi Gerry, I'm now demonstratin~ the overwrite facility,". Figure 2: Call Minder- Caller dialogue
531
Customer Dialogue: Call Minder customers can listen to their messages from their own phone or, after keying a Personal Identity Number (PIN), from any other telephone. The service informs customers how many messages have been recorded and allows them to be retrieved. Customers can also change various options including the greeting message played to callers, the number of rings before answering and the customer's PIN. Call Minder's success depends on dialogues that customers can use effortlessly. Human Factors involvement in the development of the service can be seen in Figure 3. Service Co nc ept/design
Service development
Defivery o f completed Service Future modifications: User trials
Implementation of prototypes
quiremntsa ,tur , impl m ntation°f m ,l m ntation
functional specification o: dmlogue"
Concept testing (Focus groups) Paper walkthroughs
prototype ~Iterative "
Prototype walkthroughs User trials Paper walkthroughs
the final system FIELD ~,FIELD TRIAL 1.,
Minder ~LAUNCH
Field Trial 1: One-to one interviews~questionnaires Field Trial 2: Focus groups~questionnaires
Figure 3: Stages in the usability engineering of Call Minder As soon as Market Research confirmed the basic service concept, preliminary dialogue designs were constructed following principles such as those below: •
Provision of adequate feedback so users feel in control of the interaction, know where they are, what they can do next, and how to correct errors.
•
Complexity should not strain the users' memory. They should not have to remember information from one part of the interaction to the next.
•
Speak the users' language and provide easy to understand prompts.
We then improved the specification of the service by conducting a series of paper walk-throughs. A dialogue prototyping tool had been developed to build rapid prototypes and simulate the service. This allowed us to gather objective and subjective usability data in laboratory trials.
532 The results from these trials allowed many improvements to be made. Early trials showed that voice prompts did not always elicit the expected responses. For example the prompt: 'Would you like to leave a message ?'
elicited responses such as, 'Um. Yes I would' or 'Thank you, yes' or 'Hello Gerry ...... '
rather than with the expected 'yes' or 'no'. As a result the Caller dialogue was radically changed by eliminating the "YES/NO" questions and introducing open ended responses. Another benefit from carrying out these trials was that timing data we collected allowed us to specify appropriate time-out durations. Early prototypes had time-outs that were so short that hesitations triggered the next stage in the dialogue. Not surprisingly, users found this very frustrating! When all the improvements identified in the laboratory trials had been incorporated, we undertook an extensive field evaluation to test Call Minder in a real life environment. The aim of the field evaluation was to refine and establish: •
the robustness and effectiveness of the technology
•
the processes needed to deliver the service to market
•
data on usage, customer attitude and usability
The field evaluation involved over 200 customers for one year. Usability was investigated by means of questionnaires and interviews, which indicated that customers were satisfied with the service. However, there was still scope for improvement. For example, the high level of background noise in some environments reduced the recognition performance leading to timeconsuming error correction dialogues. As a result, some found the system too slow. Fast-track dialogues and keying options were thus introduced. 3
O B J E C T I V E AND SUBJECTIVE BENCHMARKING
Both objective and subjective measures were taken throughout the course of the trials and the field study which enabled us to benchmark different evaluation techniques. Objective measures are taken either from the service platform or by observing users. These provide data on performance and usage, such as the number of times and at what point in the dialogue users hang up, recognition performance and responses to voice prompts. They allow us to identify problem areas within the dialoguemfor example, we can pinpoint confusing voice prompts which cause users to hang up. Subjective assessment through interviews and questionnaires allow customer satisfaction monitoring. Questionnaires measure key dimensions relevant to speech services including speed of use, level of concentration needed, reaction to the voice and clarity of messages. The results from our questionnaire are represented on attitude profiles (see Figure 4), which allow easy comparison between different versions of a service. The questionnaire was validated and refined during the development of Call Minder and is now part of the Human Factor tool-kit for designing future speech services.
533
Clarity of system messages
t ~ , , /
m
Level of concentration required Perceived Recognition
m_.
Speed of operation
Im
.~\m
Voice used for the service Overall clarity of service
¢#
Woulduse serviceagain
,,m
Level of improvement *O m
Option sequence used Reliability of operation Overall ease of use -2 -1.5 Very Negative
-1 -0.5
0
0.5
1
1.5
2
Very Positive
Response score (Mean) KEY
--
-m - -
Service A ----------!:3-.-------Service B
Figure 4: Attitude profile comparing two versions of Call Minder Responses to the questionnaire statements and the objective data are considered together to identify potential improvements. An overall assessment of the service is based on all the available data. 4
CONCLUSIONS
The Call Minder project demonstrates the importance of designing usability into speech services. We involved users throughout development and identified numerous improvements to the service. Implementing such improvements in future speech services will yield enhanced customer satisfaction, increased return on investment and competitive advantage. REFERENCES 1 M. B. Cooper, Human Factor Aspects of Voice Input/Output, Speech Technology March/April 1987, 82-86. 2 M. Atyeo, R. Green, User friendly weapons for the competitive fight, British Telecommunications Engineering Journal Vol. 13.3, 1994, 201-205.
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
535
VOICEDIC: A Practical Application of Speech Recognition Technology Kenji Kita, Kazuhiko Ashibe, Yoneo Yano and Hiroaki Ogata a aFaculty of Engineering, Tokushima University Minami-josanjima, Tokushima 770, JAPAN
This paper presents a practical application of speech recognition technology, a voiceaccessible or voice-activated dictionary, which is undertaken by the VOICEDIC project at our laboratory. The paper will show an outline of VOICEDIC as well as its advantages. We will also describe implementational issues required to attain high speech recognition performance.
1. I N T R O D U C T I O N Over many years, researchers in the speech field believe that speech provides the ultimate medium for human-computer interaction. Some of the reasons include the following
[1,2]. • Speech is the most natural and easiest communication medium for humans. • Speech is more efficient than other modes of interaction. • Speech input can be used in hands/eyes busy situations. • Speech is very helpful to handicapped persons. Speech researchers have tackled many difficult problems towards this ultimate goal. As a result, several remarkable speech recognizers have emerged in recent years [3,4]. Now, at the research level, we are capable of attaining a word accuracy of over 90% in connected word recognition of several thousand words. But speech recognition technology still remains little used in real applications. Many possible reasons come into our mind. For example, (1) an accuracy of 90% is poor for a real use, (2) we do not have an effective error correction strategies at hand, and many other reasons. But the greatest reason is that we could not find an appropriate application. The importance is to find a particular application for which speech is appropriate. In this paper, we present a practical application of speech recognition technology, a voice-accessible or voice-activated dictionary, which is undertaken by the VOIOEDIC project at our laboratory. The paper will show an outline of VOlCEI)IC as well as its advantages. We will also describe implementational issues required to attain high speech recognition performance.
536 2. V O I C E D I C : O V E R V I E W A N D ITS A D V A N T A G E S V O I C E D I C allows us to consult a dictionary through human voice. Consulting a dictionary is rather a bothersome task. It takes a few seconds at the lowest estimate, even worse sometimes takes ten seconds or more. Of course, on-line electric dictionaries are recently available and they provide faster access. But, even in that case, keyboard typing is much slower than speech input. In addition, VOICEDIC is effective particularly in conjunction with tasks where hands are already occupied, such as while keyboard and/or mouse operations. Figure 1 shows a block diagram of the V O I C E D I C system. First, the system transforms an utterance, which is an analog form, into a digital form, and then extracts speech features that facilitate a subsequent recognition task. The speech recognition module produces a symbolic representation by comparing the feature pattern of the input to members of a set of ready-made reference patterns. On completion of the recognition, the result is converted into a network of alternative matches, called lattice. The next step is to obtain multiple word candidates from the lattice using a dictionary entries (lexicon). From the output candidates, a right result is selected by a user, and finally the chosen word is retrieved from the dictionary. VOICEDIC has the following advantages:
requires only isolated-word recognition technology. From current speech recognition standards, a high recognition accuracy is easy to achieve for isolatedword recognition.
• VOICEDIC
•
VOICEDIC displays several candidate words as recognition results. If the candidate set does not include the correct result, we can simply speak the word again. Thus, misrecognitions do not result in critical situations.
• We do not have to utter a complete word in order to retrieve words from a dictionary. Uttering the first few syllables makes it possible to list candidate words that begin with the uttered syllables. Then, we can select the desired word from the candidate set. •
VOICEDIC has an option of restricting the search within an active window (screen). With this option, words to be recognized are resrticted to those displayed in the current editing window. Therefore, very fast and accurate recognition is possible.
We are now considering two forms of realization according to the dictionary component" (A) Using a text-based electric dictionary. (B) Using a speech dictionary. In case (A) retrieved results (the meaning of words) are displayed on the screen, while in case (B) the results are given by sound. Speech dictionary holds real speech uttered by humans, and thus it gives more natural sound than a synthetic one. Also, using real speech data would be of great help for blind or visually impaired users.
537
Microphone
input speech AD Conversion ~ digitizedspeech Feature Extraction acoustic parameters I SpeechRecognitionModule ] -~
Syllable HMMs
syllable lattice I N-bestCandidatesGenerator ] -"
Trie Representation of DictionaryEntry Words
~ N-bestcandidates Candidate Selectionby User ~word to be retrieved l DictionaryInterfaceModule
l~
~ retrievedresult
Figure 1. Block diagram of the VOICEDICsystem
Electric Dictionary
538 3. I M P L E M E N T A T I O N A L ISSUES This section describes some parts of the system, which is highly important for the implementation. 3.1. Acoustic Models As acoustic models, we adopt hidden Markov models (HMMs for short) [5], which have been successfully used in recent state-of-the-art speech recognition systems. HMMs are stochastic models suitable for handling the uncertainty that arises in speech, such as contextual effects and speaker variabilities. In VOICEDIC,Japanese syllables are used as the basic HMM unit because the whole word-based approch is difficult to meet the real-time requirements in case of the large vocabulary size. There are about 100 phonetically different spoken syllables in all. Each syllable is represented by a continuous mixture HMM, in which an output probability density function is characterized by a 39-component diagonal covariance Gaussian mixture. See Table 1 for speech analysis conditions. 3.2. Speech R e c o g n i t i o n M o d u l e As stated above, VOICEDIC uses hidden Markov models of syllables as the basis for speech modeling. Word models are built by concatenating syllable models. The speech recognition module performs a time-synchronous Viterbi beam search, matching syllable models against the speech input. That is, it maintains a beam of the best scoring candidates and extends these one frame at a time. Recognition candidates with a low likelihood score are pruned. All candidates cover the utterance from the beginning to the most recently processed frame. As a recognition path reaches the end of a syllable model, the search transits to the beginning of syllable models that can follow the current syllable which ends the path. Currently, the speech recognizer uses no restrictions concerning syllable connections (i.e. any syllable can follow any other syllables), which sometimes overgenerate the lexical vocabulary. To avoid this problem, the recognizer gives a lattice of alternative syllable matches in order to get a high inclusion rate of the correct result. Figure 2 shows an
Table 1 Speech analysis conditions Sampling frequency and precision Pre-emphasis Hamming window Frame period Acoustic parameters
16 kHz, 16 bit 1 - 0.97z -1 25 ms 10 ms 12 MFCC (mel-frequency cepstral coefficients) + 12 A MFCC + 12 A A MFCC + power + A power + A A power (39 dimensions in all)
539 example of a lattice, where symbols A, B and C indicates distinct syllables. As is clear from the figure, a lattice is characterized as a set of hypothesized syllables with different starting and ending positions in the input speech.
A
C
B Time
tl
t2
t3
t4
t5
Figure 2. An example of a lattice.
3.3. N - b e s t C a n d i d a t e s G e n e r a t o r a n d V o c a b u l a r y R e p r e s e n t a t i o n The job of the N-best Candidates Generator is to generate N most likely word candidates from the syllable lattice. The entire vocabulary (a set of dictionary entry words) is represented as a trie structure, from which this module selects word candidates that matche well with paths in the lattice. The size of total vocabulary amounts to more than tens of thousands to a hundred thousand, and thus it is necessary to adopt both a compact and efficient representation of vocabulary. For these purposes, VOICEDIC uses a double-array-based trie structure [6], which combines the fast access of a matrix form with the compactness of a list form. In short, the double-array-based trie structure uses two one-dimensional arrays, called B A S E and C H E C K respectively. Suppose 6(n, a) = m is a notation indicating that there is an arc labeled a from node n to node m, then it is defined using B A S E and C H E C K such that the following relation holds.
6(n,a) = m
~
BASE[n] + 'a' = m & CHECK[m] = n
(1)
In this way, each arc of the trie can be retrieved from the double-array in 0(1) time, providing very fast access to the dictionary entry words.
540 4. C O N C L U S I O N S This paper presented the VOICEDIC project undertaken at our laboratory, which aims at a practical application of speech recognition technology. The main characteristics are: • Because of requiring only isolated-word recognition technology, it is expected to attain a high recognition performance from current technological standards. • It offers simple error handling for misrecognitions. That is, the system produces N most likely recognition results, from which a user selects a correct one. We are currently developing each module of VOICEDIC separately on workstations. These modules will be integrated into one system before long. Finally, let us turn our eyes to the future. Progress in device technology will surely make it possible to increase gate numbers in chips and clock frequencies more than 10..~100 times. Also, the development of large-scale application-specific ICs is going to enable us to realize a one-chip speech recognition device. In the future, it would not be a mere dream to make a portable pocket VOICEDIC so that anyone carries it to anywhere anytime. REFERENCES
1. C. Baber and J. M. Noyes (eds.), Interactive Speech Technology: Humans factors issues in the application of speech input/output to computers, Taylor & Francis Publishers
(1993). 2. D.B. Roe and J. G. Wilpon (eds), Voice Communication between Humans and Machines, National Academy Press (1994). 3. K . F . Lee, Automatic Speech Recognition: The Development of the SPHINX System, Kluwer Academic Publishers (1989). 4. K. Kita, Y. Yano and T. Morimoto, Continuous Speech Recognition Based on LRParser-Driven One-Pass Search Algorithm and Hidden Markov Modeling of Speech: Towards Real-time Intelligent Human-Computer Interaction by Voice, Proc. of Second Singapore International Conference on Intelligent Systems, pp. B347-B352 (1994). 5. X.D. Huang, Y. Ariki and M. A. Jack, Hidden Markov Models for Speech Recognition, Edinburgh University Press (1990). 6. J. Aoe, K. Morimoto and T. Sato, An Efficient Implementation of Trie Structures, Soj2ware-Practice and Experience, Vol. 22(9), pp. 695-721 (1992).
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
541
An Operation Analysis of An Address Input System with Speech Recognition Kazuhiro A R A I , Osamu Y O S H I O K A , Shigeki S A G A Y A M A and Noboru S U G A M U R A NTT Human Interface Laboratories 1-2356 Take, Y o k o s u k a - s h i , K a n a g a w a 238-03, J A P A N Abstract Address input is one of the most important processes in business. This process is needed for managing customer information and the delivery of commodities. In order to reduce the time necessary for address input, we have developed an address input system that employs speech recognition. Users of this system can input addresses by using not only a keyboard and a mouse but also speech. The input time was measured for 25 adult novice users in order to test whether or not the speech recognition function is useful. The results revealed that speech recognition reduces the time required for address input. This paper describes the system outline and the experiment measuring input time.
1
INTRODUCTION
Recently, the performance of speech recognition has reached new highs. Due to this remarkable progress, many new avenues have opened up for application systems employing speech recognition [1]. Architecture and user interface of speech recognition are the important factors for bringing the speech aspect of human-computer interface to a more readily accessible state. We have developed speech recognition architecture according to the following concepts. The core of speech recognition must be developed independently of any application system. From this viewpoint, we introduce a server-client architecture for the development of a speech recognition process. The server processes speech by accepting as input words that are to be recognized and speech data from the client. The results of speech recognition are sent back to the client. This server is developed using only software and does not need special hardware such as a digital signal processor. For making a useful user interface, speech recognition should be introduced which is complementary to conventional input devices such as a keyboard and a mouse [2]. By using speech recognition, the time required for data input may be reduced in comparison to that of the conventional devices [3]. However, sometimes the results of speech recognition may be incorrect. Therefore, we developed a support environment that corrects the results of speech recognition using conventional devices [4]. We have developed an address input system employing speech recognition using the architecture described above. Address input is one of the most important processes in business. The process is needed for managing customer information and the delivery of commodities. The user of this system can input an address using not only a keyboard
542
[ 03.3509_XXXX I AreaandI°calc ° d e ~ ITokyo-to lTokyo-to Tokyo-to Tokyo-to ITokyo-to Tokyo-to Tokyo-to Tokyo-to
Chiyoda-ku Uchisaiwai-cho"1~,,,,AddressRetrieval Chiyoda-ku Kasumigaseki Chiyoda-ku Nagata-cho ~ Dividing according to the classes Chiyoda-ku Hibiyakouen Tokyo-to Chiyoda-ku f Uchisaiwai-cho Chiyoda-ku Yuuraku-cho Kasumigaseki Minato-ku Shinbashi Nagata-cho Minato-ku Toranomon Hibiyakouen Yuuraku-cho Minato-ku Nishishinbashi Minato-ku | Shinbashi r Toranomon t__Nishishinbashi
'
1 t
Storing words into each class
1
Figure 1" The flow of address retrieval and a mouse, but also speech. Area codes and local exchanges of telephone numbers are generally assigned depending on the locality. The system has a database that describes relationship between addresses and telephone numbers. The user first inputs the telephone number. By retrieving addresses using the area code and exchange, the system creates address candidates.
An Address Input System 2.1 System Outline
Speech recognition accuracy will deteriorate as the system is forced to deal with more and more words. In order to reduce the number of words that must be handled, this system uses the relationship between the telephone number and addresses described below. The area code and the exchange of telephone numbers generally depend on the locality. The system has a database that describes the relationship between the area code and exchange of the telephone numbers and the regions. Figure 1 shows the flow of the address retrieval using the area code and exchange. The user first inputs a telephone number using the keyboard. Using the area and code the exchange, the system retrieves several addresses corresponding to the telephone number input. The addresses are divided into words according to the address classes such as prefecture, city, ward and town. Then the relationship between the words are assigned and are then stored into each class. Those address words form the word speech recognition input. A sample window for this system is shown in figure 2. The system's window consists of one entry field for telephone numbers, five fields for address such as prefecture, city and town, two fields for block numbers and building names, and three buttons. The fields and buttons are developed as parts of a graphical user interface employing speech recognition that works on the X window environment. By arranging these parts in a window, a developer of a system with speech recognition can easily create a specific window. The
543
PhoneNumber I 03-3509-0000I
AddressI Tokyo-to II Chiyoda-ku
I
I
••°•"'•a'•a'•"•••liiiiiiliiIi1•iiiiIi1•i••iiI!•i•I!ii•i1ijiii1•!••••iiiijIiIii•iIijijiiiiliIiiiijiilii•iIiil
lCearI lSavel
nOutl
Figure 2: A sample of the system's window user interface and recognition procedure are implemented as functions of the fields. In the address input system, the fields for address accept speech as input. After retrieving addresses using the area code and the exchange, the address words are assigned to each field according to their classes. If no address word is assigned to a field, the field turns dark to indicate that there is no entry. The user can input the address word using a mouse operation on a pulldown menu which shows the words stored in each field. The user can also input an address word using speech recognition. The procedure for speech recognition is described briefly as follows. When a field for address input is focused by the user, the client sends the words stored in the field to the recognition server to set a vocabulary. While a user is pushing a key for speech input, the server accepts the speech data from the client and processes it. After the server has finished the process, the server returns the words to the appropriate fields with plausibility scores. The user interface for speech recognition is described in detail in section 2.2. 2.2 User Interface The following user interface supports speech recognition.
Indicator for Speech Input Extracting the speech section correctly is an important issue for speech recognition. The speech recognition client should accept speech partitioning but not noise as input to get highly accurate speech recognition. In our server-client architecture, the client sends speech data to the server, while the user is pushing a key for speech input. Some users, however, start speaking before they push the key or release the key before completing the speech input. In order to avoid such error, the client darkens the field to notify the user t h a t speech data are being sent. The user can start speaking after the field has returned to its original state.
The Following Candidate When speech recognition is finished, the candidates are arranged into the field's menu list according to the plausibility scores. The most plausible candidate is shown in the field. If the word shown in the field is incorrect, the user can look for the correct word by depressing the space key and selecting the correct word using the return key. By operating the mouse, the user is also able to look for the correct word in the menu list which shows ten candidates at a time. It is easy for users to
544
Tokyo-to
Chiyoda-ku "-7--- Uchisaiwai-cho F Kasumigasoki '--- Nagata-cho F Hibiyakouen ' - - - Yuuraku-cho Minato-ku | Shinbashi F Toranomon L--- Nishishinbashi
In case that "Uchisaiwai-cho"is selected.
Tokyo-to II Ch,,o,,-ku Iluchis=w=-choI
In case that "Minato-ku"is selected.
Tokyo M,no-ku,
Shinbashi
Toranomon Nishishinbashi
Figure 3: A sample of fixing and reducing of words understand the key and the mouse operation required because similar actions are used by many other systems for transaction entry.
2.3
Use of Address Class
Using the user's selection of an address word in a class, the system fixes word in the larger classifications and decides if each word in the smaller classification is related to the selected word. Figure 3 shows an example for fixing and determining the word. In this figure, when the user selects the address word "UchisaiwaJ-cho," the words in the larger classifications such as "Chiyoda-ku" and "Tokyo-to" are automatically fixed using the relationship between the words. On the other hand, if the user selects the word "Minato-ku," the word "Tokyo-to" is fixed and the words in the smaller classification are reduced to "Shinbashi," "Toranomon," and "Nishishinbashi." By reducing the number of selectable words, it becomes easier for the user to decide the next classifying words and for the system to recognize the words spoken by the user.
3
Experiment
An experiment was carried out to test if speech recognition is useful for address input. The experiment focused on the input time needed for address input because the goal of this system is to shorten this time required. The conditions and results axe described below in detail.
3.1
Condition
25 female adult subjects were used as operators for the experiment. None of the operators had previous experience using speech input. Telephone numbers were arranged so that 10, 20, 30, or 40 addresses each would be retrieved from the database. 24 sets of telephone numbers and addresses were given to each operator. Half of them were used for speech input, and the others were for conventional input. Use of speech recognition for address input was left to the operator's discretion. Therefore, some users could opt not to use speech recognition even though it is available. The database contained 7,925 combinations of area codes and exchanges and 89,59? addresses in the Tokyo area.
545
"~' "6
10
z
I,
II
10
I
Input Time [sec]
Figure 4: A sample of histogram of the operation time The experiment consisted of first, an explanation of the system and its operation; second, practice operations; and finally, 24 operations for time measurement. When all words had been retrieved from the database and had been stored into fields, the system started the time measurement. The system ended the measurement when the operator had fixed all the fields. In the experiment, the input times of the system were compared to those of another system that used the same user interface excluding speech recognition. All key strokes, mouse operations, and speech data were recorded into files together with the time at which each action occurred.
3.2
Results
All records of the operation are categorized based on how many addresses the operation dealt with and whether or not the operation used speech recognition. Namely, all records were categorized into one of eight cases. For each case, histograms of the operation time were made within 2 sec. of the class internal. Figure 4 shows a sample of the histogram made from the operations that dealt with 20 addresses and did not use speech recognition. In the figure, the largest number of trials, that is, the mode of this histogram, is 18 trials at 12 sec. In this experiment, the operation time in each case was compared in the mode of their histogram. Figure 5 shows the modes of operation time in cases. Where the user used the system without speech recognition, the tendency shows that as the number of addresses increased, the amount of time required to input the addresses increased. In this system, accordingly as the number of addresses increased, the time may proportionally increase, because it takes more time and operation for the operator to find out the word to be input. However, for the system with speech recognition, the increase in the time was comparatively less than that in the conventional system. Therefore, these experimental results support the fact that the time required for address input by speech recognition did not significantly increase as the number of addresses increased.
546
30
I
25 _ "G' 20
I
I
I
With Speech Recognition. -.a-Witho
m
¢D ,____, (D
-o 0
15
m
10
0
0
I
I
I
I
10
20
30
40
Number Of Addresses
Figure 5: Comparison of the time for address input
4
Conclusion
This paper has described an address input system with speech recognition. This system was implemented based on a server-client architecture for speech recognition, the graphical user interface developed using parts for speech recognition, and address input using address classes. The experimental results revealed that when the input volume is large, our speech recognition system significantly decreases the time required for data input in comparison to conventional means. Future works needed to further improve speech recognition applications include improvement of the user interface and consideration of a support environment for user instruction.
Acknowledgment The authors would like to thank Dr.Nobuhiko Kitawaki for his administrative supports in this work. We would also like to acknowledge Tomokazu Yamada, Takashi Imoto and Yoshiaki Noda for their helpful technical supports.
References [1] Kai-Fu Lee, "The Conversational Computer: An APPLE Perspective," Proceedings of The Third European Conference on Speech Communication and Technology; EuroSpeech'93, Vol.2, pp.1377-1384, (1993). [2] Alexander I. Rudnicky, "Factors Affecting Choice of Speech Over Keyboard and Mouse in A Simple Data-RetrievM Task," Proceedings of The Third European Conference on Speech Communication and Technology; EuroSpeech'93, Vol.3, pp.2161-2164, (1993). [3] Lewis R. Karl, Michael Pettey and Ben Shneiderman, "Speech versus mouse commands for word processing: an empirical evaluation," International Journal of Man-Machine Studies, 39, pp.667-687, (1993). [4] Osamu Yoshioka, Yasuhiro Minami and Kiyohiro Shikano, "A Multi-Modal Dialogue System for Telephone Directory Assistance," Proceedings of 1994 International Conference on Spoken Language Processing; ICSLP'9.~, Vol.2, pp.887-890, (1994).
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
547
A Menu-Guided Spoken Dialog System and Its Evaluation Mikio Yamamoto, Takeshi Koike and Seiichi Nakagawa Department of Information and Computer Sciences, Toyohashi University of Technology, 1-1, Tempaku, Toyohashi, 441, Japan
Abstract This paper describes a man-machine spoken dialog system that integrates speech recognition and menu-based natural language understanding technique. One of the difficulties in speech recognition is that the recognition rate is inversely proportional to the size of the set of acceptable sentences in the recognition system. That is, there is a serious tradeoff between the recognition precision and coverage. To resolve this problem, we employ a menu-based natural language understanding technique as the guide for spoken input. We developed the preliminary system and evaluated it. The experiment shows that the menu-based spoken dialog system is suitable for novice users. 1. I N T R O D U C T I O N A spoken natural language dialog system is one of the best man-machine interfaces for novice users, if the system can understand and answer almost all the user's queries. However, current technology in speech recognition and natural language understanding cannot perfectly recognize and answer all the user's queries. The recognition rate depends on the perplexity of the grammar for the recognition system. Perplexity means the size of search space in speech recognition. If we decrease the perplexity of the grammar in the system, the recognition rate may improve. However, decreasing perplexity means decreasing the number of recognizable input sentences. Although the recognition rate for acceptable sentences is high in limited conditions, it will be difficult for the user to utter an acceptable sentence. The grammar-driven speech recognition system never recognizes unacceptable sentences and it is difficult for the user to know system's capabilities. This tradeoff between precision and coverage is a serious problem of speech recognizer. We think the speech input system has to show explicitly its capabilities to the novice user by some guidance facility. To show them, we employ the menu-based natural language understanding technique. Sentences inputted to an original menu-base natural language understanding system[l] are made by selecting words or phrases from a set of dynamically changing menus on screen. When a user selects a word or phrase from the menu, the system predicts the next words or phrases that can grammatically follow after the sequence of words selected previously and displays the predicted words or phrases as menus. This technique can show explicitly system's capabilities using menu. However, Thompson reported limitations of the original menu-based natural language understanding technique[2]. He pointed out the "big menu" problem in his paper.
548 This is a problem in that too many items are active for the selection in relatively bigger tasks. It is difficult for the user to select a word or phrase from such a big menu. Reducing the number of units for the selection solves this problem. For example, a phrase consisting of several words can be subdivided into words. The word menu may be smaller than phrase menu. However, two new problems arise by reducing the number of selections. One is that there is low input efficiency and selection time increases. Another problem is that it is unnatural to speak a small item in a menu. It is natural for user to be able to select an item that has a lump of meaning. The Japanese language has phrases called 'bunsetsu' that is a sub-sentential unit. A "ounsetsu' is a phrase that has a content word followed by a sequence of functional words. Most of 'bunsetsu' take one or more functional words. Since a 'bunsetsu' is a lump of meaning and is familiar for Japanese people, it is easy for Japanese to speak a sentence with pauses between 'bunsetsu' phrases. In addition, functional words are short and the sequences of them may be long, so the user goes to the trouble of selecting each functional word one by one. However, if we choose a ~ounsetsu' as the selection unit, the menu becomes big because 'bunsetsu' is the combination of content words and functional words. Also in Japanese, we must display several words for one root verb since the inflection of verbs changes the mood of sentence. In the inflection of Japanese verb, only the end of word changes, so dividing a verb into an unchangeable part and a changeable part decreases the size of the menu. But in this case the unit of selection is too small and meaningless. Thus the decision of units for the selection includes a tradeoff. That is, although we want a big selection unit as possible, it leads to a big menu. On the other hand, although a small menu is preferable, it leads to an unnaanal speaking manner. In the Section 2, we propose a new technology that solves both tradeoffs of speech recognition and the menu-based technique by integration of the both advantages of them. In the Section 3, we compare our method with other methods and evaluate it. 2. M E N U - G U I D E D S P O K E N D I A L O G S Y S T E M To solve the problems discussed above, we propose a new interface, which uses spoken language input in combination with the menu-based technique. The speech recognition system has enough speech recognition rate when the input sentences are in the range of a limited grammar for speech recognition. However, it means that the limitations reduce the set of acceptable sentences, so user's utterances are often not acceptable to the system. We need a guiding facility to help for novice users to speak acceptable sentences. We think that the menu-based technique can be used as a guiding facility for user input. If we choose a 'bunsetsu' as a unit for the selection and the system displays all 'bunsetsu' as a menu, the menu may be too big. To avoid this problem, we employ the mechanism that displays only content words in a menu and an user utters a 'bunsetsu' that is made from the selected content word in the menu or the inflection form, and following any sequence of functional words. This is a new method integrating the advantages of the menu-based technique --- which explicitly shows the capability of the system to the user and guides to make an acceptable sentence --- and the speech recognition --- by which the user can input a phrase that includes functional words out of the menu.
549 To evaluate our basic idea, we developed a preliminary spoken dialog system that can accept Japanese language for the task of "sight-seeing guidance for Mt. Fuji." Figure 1 shows the system configuration. The next word predictor predicts content words and 'bunsetsu' categories that can legally follow the partial sentence inputted previously. Since this part uses the prediction mechanism for the continuous speech recognition system controlled by a contextfree grammar(CFG), the grammar of the predictor is written by CFG. The basic algorithm for the prediction of the content words and "ounsetsu' categories uses the Earley's top-down parser[4]. The predicted content words are sent to the menu manager and are displayed. The predicted 'bunsetsu' categories are sent to the part of the 'bunsetsu' recognition, and a 'bunsetsu' spoken by the user is recognized using the grammar and dictionary related to the predicted 'bunsetsu' categories. The system's recognition rate and speed are very good, because the candidates not related to the predicted 'bunsetsu' categories are not considered. Since the system displays the recognized 'bunsetsu' phrase, the user can know whether recognition result is correct or not just after the end of uttering. If the recognition result is not correct, the user can cancel the misrecognized 'bunsetsu' phrase by uttering 'cut.' When the user completes a sentence, he speaks the word that means the end. Then the system sends the sentence to the dialog system that analyzes its syntax and meaning and generates a response[5]. The dialog system has a database for answering queries about sightseeing in Mt.Fuji that has about 500 entries. Figure 2 shows a simplification of the system's display. The above list written by Kanji characters is a menu. The kanji string '~~,7~~ ~ ¢9' on the bottom is the sequence of words selected previously. ' _ ~ 7 5 ~ G' means 'from Toyohashi city' in English. If the user want to input the following sentence
Toyohashi kara (from Toyohashi)
Fujisan made (to Mt. Fuji)
douyatte (How)
ikemasuka? (can I go)
that means 'How can I go to Mt. Fuji from Toyohashi city?', he may input '~:]zLI_I ~ "~' (to Mt. Fuji) as the next 'bunsetsu'. Bunsetsu ' ~ ' ~ ILl ~ ~ ' consists of the content word " ~ ' ~ ~ ' Menu-GuidedSentenceInputS~,stem
I Menudisp,ayI/--
Utterance
~.
nce
. ts,ste j ' o o
•
| |
Dialog /
Figure 1 System configuration
ply
MENU ~ E I ~ (Lake Kawaguchi) I11~'~ (Lake Yamanaka) ~ 1 1 1 (Mt. Fuji) I~ ~O')-~ (White String Fall) ~±-1" > ~ --(Fuji Interchange) ~~ -1" > "Y -- (Gotenba Interchange)
WO,DS OUENCE, CTED,,romTo,oha.h . P.,V,OU,., ...c. °
Figure 2 Display of the menu guided spoken language input system
550 (Mt. Fuji) in the menu and the Japanese post-position ' ~ "~'(to) that is not displayed as an item in the menu. Since there are only words that mean destinations in the menu, the user can easily know that he must specify a destination and decide the next utterance. The system can easily recognize user's input 'bunsetsu', because the allowable 'bunsetsu's are limited by the previous speech and the menu. While the system does restrict the user utterance for the speech recognition, it guides the user to decide easily on the next utterance by displaying the list of content words that are allowed by it. 3. EVALUATION
3.1 Compared system For the evaluation of our technique, we compare four systems which are: (i)the menu-based spoken word input system('word menu)[3] that accepts only utterances of the words in the menu, (ii)the full spoken sentence recognition system('sentence recognitionS[4] that accepts a sentence and displays no menu, (iii)the menu-based mouse input system('mouse menu3 that the same systern as the word menu system except the selection is done by pointing device instead of speech recognition and (iv)our menu-based spoken 'bunsetsu' input system('bunsetsu menu ~. The speech recognition program for each system is the same. The recognition program recognizes sentences, 'bunsetsu's or words according to the grammar of each system. The program uses the Hidden Markov Models(HMM) of syllables and connects them dynamically in accordance with the word dictionary and grammar of the system. Although each system covers the same task domain, they do not cover the same sentence set for user inputs. Because the user of menu-based systems has different information about input guidance with the user of the sentence recognition system without the guidance, the user of the menu-based spoken word input system has a complete guide information and the user of the menu-based spoken 'bunsetsu' input system has the intermediate information of other two systems. The less information of guidance the user has, the more degree of freedom for input he has. Thus the sentence recognition system has to have larger coverage than other systems. We have tuned each system for the possible inputs. As results, the sentence recognition system has the largest set of acceptable sentences and the menu-based spoken word input system has the smallest set of acceptable sentences. The number of words in the dictionary of the sentence recognition, the bunsetsu menu and the word or mouse menu, are about 180, 120 and 100, respectively. In the menu-based spoken 'bunsetsu' input system, about eighty words are used for displaying the menu. The word and 'bunsetsu' recognition rates of the menu-based systems are about 90 percent. The sentence recognition rate of the sentence recognition system is about 60 percent, but the sentence understanding rate --- that is the rate when the misrecognized sentence that has the same meaning as input sentence is also correct --- is about 80 percent, where each recognition rate is the case when the user speaks the acceptable word, 'bunsetsu' or sentence for each system. Of course, each system cannot recognize the word, 'bunsetsu' or sentence outside its grammar and dictionary. In the next section, we describe the experiment in that we give the subjects a scenario of a
551
Table 1 Results of evaluation Average Task Recog. time per complerate ition rate task
---..... .~
Word Menu Bunsetsu Menu
25 min.
21 min. 7° Sentence Recog. 31 min. 24 min. Mouse Menu d. Sentence Recog 12 min. 7 min. Mouse Menu
88% 98% 85% 100%
91.5% 87.1% 73.5%
100%
93.3%
100%
Novices [i~i~ '................................................................................................................. il:Acceptable(38%)iiiiiii~Unacceptable(52%)'~':':':~ ':' ........................
Average time per sentence 120(58) sec. 99 (54)sec. 133 (56) sec. 62 sec. 69 (29)sec. 40 sec. :'~;~
l.:i:i:~:!:~:!:~!............. : ~~.~:~.~:~:~.:~:~:~:.~:.~:.~:..................................................~:..:.::.~......~...................fff~f-1 ... [.'i~i~i!iiiiiiiiiii~ii::~i!!!iiiii
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::iii~6..'./-...'.`6..~`1~
Figure 3 Acceptable and unacceptable utterances task that they have to perform.
3.2 Evaluation by task performance Using the systems in the section 4.1, we make an experiment to evaluate each system's performance in the task execution. The scenario of the task is that subjects play a manager of an overnight trip to Mt.Fuji by car and make a plan for the trip. The subjects have to decide where to go, what to do and which hotel to stay at. Also the subjects must retrieve some information about routes and time for driving. The subjects were volunteer students, twenty-two novices and six experts of the systems. Each system based on speech recognition is used by five novice users respectively. Mousemenu system is used by seven novice users. Three experts used the sentence recognition o system and other three experts used the mouse-menu system. The results of this experiment are given in Table 1. "Novice" in the table shows the results by the novice users. "Exp." shows the experimental results by the expert users. The time in "Average time per task" means the average time the subjects completed a task. "Task completion rate" means the rate of completion of scenario we gave to the subjects. "Recog. rate" is the recognition rate of utterance units of each system that are words, 'bunsetsu' phrases or sentences. The time in "Average time per sentence" is the average time of inputting a sentence by each system. The numbers in parentheses are the time assuming the real time speech recognition that means finishing recognition at the same time of end of speech. Note that although the average time per sentence of the mouse-menu system is shorter than the bunsetsu-menu system, the average time per task of the mouse-menu system is longer. This reason is that subjects of mouse-menu system input more questions than speech-based system. The mouse-menu system has the best performance. But if we assume real time speech
552 recognition, the average time per sentence of all systems based on speech recognition is shorter than the mouse-menu system. In the systems based on speech recognition, the experts using the sentence recognition system has the best result. However, the menu-based spoken 'bunsetsu' input system has the best results for novice user except the recognition rate. In the menu-based spoken word input system and the spoken sentence recognition systems, some subjects gave up inputting some queries, since the system could not understand user's input utterances although the user repeated speaking the same content utterances many times. In contrast, the misrecognition of the menu-based spoken 'bunsetsu' input system isn't too consecutive for the user to give up inputting queries. In the menu-based spoken word input system, this trouble is caused by the fact that the length of Japanese postposition is too short for the speech recognition system. In the spoken sentence recognition system, the misrecognition is caused by the fact that the user's utterances are unacceptable for speech recognition system. Figure 3 gives the proportion of acceptable and unacceptable input sentences to the spoken sentence recognition system. Figure 3 shows that the novice users speak acceptable sentences at a rate of only 38 percent, but the expert users at 88 percent. This is the reason of different input efficiency of the novice and expert. This result shows the advantage of the menu-based system for the novice user, because it is difficult for the novice user to utter acceptable sentences for the spoken sentence recognition system. Thus we can say that the menu-based spoken 'bunsetsu' input system is the best for the novice users who are not familiar with the system. 4. CONCLUSION We proposed and evaluated a new spoken natural language dialog system that integrates the advantages of the menu-based natural language understanding and speech recognition techniques. By evaluation with other systems, we showed that our method was suitable for novice users and revealed the reason that the sentence recognition system isn't suitable for novice users. More than half of utterances of novice users were not acceptable sentence to the system. In the other hand, about 90 percent of expert's utterances were acceptable. REFERENCES [1] H.R.Tennant et al: "Menu-based natural language understanding", Proceedings of the Conference of the ACL, pp. 15 l- 158, 1983. [2] C.W,Thompson: "Constraints on the design of menu-based natural language interface", CSL Technical Note #84-03, Texas Instruments Inc., March, 1984. [3] M.Yamamoto and S. Nakagawa: "Menu-based dialog system on Japanese", Proceedings of the 7th Annual Conference of JSAI, 19-7, pp.537-540, June, 1993(in Japanese). [4] A.Kai and S. Nakagawa: "A frame-synchronous continuous speech recognition algorithm using a top-down parsing of context-free grammar", Proceedings of ICSLP92, pp.257260, 1992. [5]M.Yamamoto, S.Kobayashi, Y.Moriya and S.Nakagawa: " A spoken dialog system with verification and clarification queries", IEICE Trans., VoI.E76-D, No.l, pp.84-94, 1993.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors)
553
© 1995 Elsevier Science B.V. All rights reserved.
Face Observation
Using an Active Camera
Qian CHEN, Takeshi FUKUMOTO, Haiyuan WU and Masahiko YACHIDA Department of Systems Engineering, Osaka University, Machikaneyama-cho 1-3, Toyonaka, 560, Japan. {chen, fu ku m oto ,wu hy, yachid a} @yachi-lab, sy s. es. osakau, ac.j p
Abstract This paper describes a new approach of taking good images for the face recognition such as the extraction of facial expression. The face like parts in an input image are first extracted by comparing the skin color regions and the hair regions detected from the image with several pre-defined 2-dimensional face pattern models using the fuzzy pattern matching method. The 3 dimensional pose of the extracted face relative to the camera is estimated using the area and the center of the gravity of the skin color region and the ones of the hair region on the face, which is then used to guide an active camera changing its view point in order to taking the image where the face will appear in the desired pose. face recognition, active vision, face detection, pattern matching, pose estimation, skin color
Keywords:
1. I n t r o d u c t i o n Automatic face detection and obtaining the image containing the face of interest in the desired pose is a key problem in the face information processing research such as face recognition and expression estimation. Most of the studies about facial expression extraction, face recognition and facial feature extraction assume that the input is an image where the face in it is a frontal one and its size and position is known. However, such images are not available in general using a single fixed camera because the face obtained from a camera may not in the desired pose, and with a fixed camera, one has no chance to change the view point to get the desired image. Obtaining the image containing the face of interest in the desired pose is a very important subject in man-machine communication and human interface although very little attention has been paid to that subject. There are two reasons that we want the image containing the face of interest in some desired pose. First, the face in some desired pose (such as in frontal pose) is required by many face recognition and expression estimation approaches. The second one is that the face in some frontal pose can be verified easily. The active camera has been introduced in computer vision research to solve many problems such as the structure from motion problem and eye-hand cooperation problem and so on. It is proved that using an active camera in vision system is an efficient approach to make many difficult vision problems becoming easily to solve. We introduce the active camera into our research in order to obtain the image containing the face of interest in some desired pose. When a face like part is extracted from an input image, the pose of the "face" relative to the camera is estimated, which is then used to guide the camera to get an image containing the "face" in frontal pose.
2. D e t e c t i n g Face C a n d i d a t e s U s i n g Color I n f o r m a t i o n The first job of obtaining a frontal face image is the face detection. Since the subject about automatic face detection has been studied by many researchers, and it is not the main subject of our research, we will not discuss it in detail. The face like parts are detectedusing the information of the skin color of faces in the input image: 1 Extracting the skin color regions and hair like regions in the image. 2 Detecting the face like region by comparing the face models with the extracted skin color regions and hair like regions using fuzzy pattern matching approach. In our research, the perceptually uniform color space described in [2] is used to represent color information in order to obtain reliable results. A model called skin color distribution function (or SCDF) is developed for representing the concept of skin color. The SCDF represents the frequency that each color appears in the human skin regions in images, thus can be used to estimate the degree of how well a color looks like skin color.
554 The skin color regions are extracted by estimating a continuo measure for each pixel in the image indicating how well it looks like skin color using S C D F , and we call it as Skin Color S i m i l a r i t y (or S C S ) . The hair like regions are extracted by finding out the pixels with relatively low luminance and low saturation. We modeled the shapes of the faces appearing on the image as three 2-dimensional patterns, one for frontal face and two for side view faces. The cell in a model indicates how the corresponding part of a face should look like. The model can be changed to any size by enlarging or reducing the size of the cells in it. Therefore we can use the models to detect faces of any arbitrary size. The face candidates in an image are found out by comparing the various rectangular regions of the image with the face models to see if it looks like a human face. To detect faces of any possible size in an image, we prepare a series of face models of various sizes. To detect faces of a particular size, we set the size of the face model to the specified size by changing the size of each cell in it. We introduce the fuzzy theory to perform the classification. We define five fuzzy sets, each corresponding to a particular kind of cell in the face models as follows: #y : F ~ Face cell #h : H ~-* Hair cell I~b : B ~ B a c k g r o u n d
(1)
cell
lZh/f : H / F
~ H a i r or f a c e cell
#h/b : H / B
~-~ H a i r or b a c k g r o u n d cell
To perform the classification, we first compute 3 kinds of measures for each square area in the selected image region. These measures are hair measure ( H ) , face measure (F), and background measure (B). Then the matching degree of each cell in a face model with the corresponding square areas in the image is estimated using the fuzzy membership functions defined above as follows: / us(F), M(cell) =
lzh(H)'
gh/!(H,F), #h/b(H,B),
if if if if
the the the the
cell celt celt celt
in in in in
the the the the
model model model model
is is is is
a a a a
F cell; H cell; H / F cell; H / B cell.
(2)
To detect regions containing faces, we use the face models to scan the entire image in order to estimate the matching degree between the face model and each rectangle region in the image. 3 Obtaining Good Face Images Most existing methods of the extraction of facial expression and of the face recognition require that the face in the input image must be in some special pose, such as frontal and its size and position must be known (in most case, the face must in the center of the image plane and the size of it must be big enough). The images of this kind can not be always obtained using a fixed camera. This problem causes most of the face recognition methods to be difficult to applied to a practical application. To solve this problem, we use an active camera that can change its view point freely, such as a camera mounted on a robot manipulator, to take good input images for face recognition. The good image for face recognition is defined as the following: 1. The face in the image is in the desired pose. 2. The face is at the image center. 3. The face in the image is big enough. We use an active camera to obtain the images containing the face of interest in the desired pose. This operation is carried out by the following steps: 1. Find out the face in the input image. 2. Estimate the pose of the face. 3. Check if the face is a frontal one. If it is, then the process terminates.
555
4. Make a plan of the for the active camera to take the picture of the frontal face. 5. Guide the camera. 6. go to step 1. 3.1 E s t i m a t i n g t h e p o s e a n d t h e p o s i t i o n o f a face Much information can be used to estimate the pose of a face in the image, such as facial features, etc. However, the approach based on facial features often fails due to the low reliability of the extracted feature. In our research, the pose and the position of the face is estimated by making use of the active cam,era and the image processing techniques. The manipulator is calibration is done before hand thus the relation between the robot coordinates system and the camera coordinates system is known. At first, the face is detected from the input image and its position, sizel and pose is estimated. This is done by extracting the hair part and skin color part in the image, then find out the combination of them to compose a head like region. The position of the face in the image is computed the center of gravity of the hair region and skin color region. To estimate the 3D position of the face relative to the camera, we first bring the face in the image to the center of the image plane. This is done by rotating the camera around the x-axis and y-axis of the camera coordinates system. Since the camera is calibrated beforehand, the required rotation angle to bring the face to the image center can be computed from the displacement of the position of the face in the image from the center of the image plane. Then we know that the position of the face in the 3D environment will lie on the optical axis of the camera. Since the head of people does not change very much from person to person, we can use this information to estimate the approximate distance between of the camera and the face:
d= f wf
(3)
wi
where d is the distance, the W: is the average width of human head, and the w~ is the width of the face appeared in the image. The pose of the face is determined from the center of gravity of the skin color region and of the hair region. Since the face is approximately symmetric, when viewing a face in frontal pose, the face part will be at the center of the head and the hair part will be around the face. When viewing a head from the left side, the face part will appear on the right and the hair part on the left. Thus, the pose of the face can be estimated by considering the center of gravity of the hair part and the one of the skin color part (face). When the face is in frontal pose, the horizontal position of the center of gravity of the skin color region and the center of gravity of the hair region will be same. Thus, we can find out whether a face is in frontal pose by check the z-coordinate of the center of gravity of the hair region and the one of the skin color region. If the two are not same, we can estimate the approximated direction of the face relative to the camera from those data. Although the shape of face and the hair style changes for different person, the pose of the face estimated using that information can tell whether the face is in frontal pose successfully, and it can give the right direction that the active camera should move along to get the frontal pose image if the face is not in frontal pose. 3.2 A c t i v e C a m e r a C o n t r o l The detail discussion of the strategy for controlling the active camera can be found in [8], [9] listed in the references. When a face candidate is detected from an input image, the camera changes its direction to bring the face in the image to the image center. Then the pose of the face is estimated using the method described above. If the face is not in the desired pose (frontal), the camera can rotate along a circular path of which the radius is the distance between the face and the camera, the center is the 3D position of the face, and the optical axis of the camera keeps directing to the center of the face. (See figure 1). The rotation angle to do the rotation can be computed from the face direction estimated. Although the estimated direction of the face is not accurate, since we just use it as the error measurement while perform the visual feedback control, at the final state, the face
556 in the image is very near a frontal one. To take the frontal pose image of the face, the camera needs to rotate along a circular track of that the center is on the face in 3D space, the radius is the distance between the camera and the face. Head
r track Camera R: distance between the camera and the head O: estimated pose of the head relative to the camera Figure 1 Using an active camera to take the frontal pose image of a face.
4.
Experimental Results We have made some experiments to examine if it is possible to obtain the pose of the face relative to the camera using the face region and the hair region on it, and what kinds of information are useful for pose estimation. The experiment environment is shown in figure 1. The testee sits on a chair in front of the camera. For each testee, we take an image of frontal pose face, then the testee turns on left for 15 degrees, and we take the next and so on. For each image, we extract the skin color region and the hair region, then compute the area, center of gravity of the two regions. Figure 2 shows some results of the extracted skin color region and the hair region of a face. The results are shown in figure 3. We find out that the difference between the center of gravity of the skin color region and the one of the hair region is most useful for the pose estimation of faces. In all case, the distance become zero when the face is a frontal one. When the face rotates to left (relative to the testee), the distance increases monotonously, when it rotates to right, the distance decreases and becomes minus.
left 90
left 60
left 30
0
right 30
right 60
right 90
Figure 2 The extracted skin color regions and hair regions from a test image sequence. The number under each face indicates the rotation angle relative to the frontal face.
We also made some experiments using an active camera to obtain the face images in the frontal pose. The experiment environment is same as showed in figure 1. At first, we take an image and estimate the position and the pose of the face relative to the camera, the we use the method described in the above section to build a motion plane for the camera so that after the camera change its view point the image taken from the camera will contain the face in frontal pose. The image sequence obtained while guiding the
557
active camera towards the frontal face is shown in figure 4. The experiment results are shown in figure 5. . Fx (pixel) '"".
~ Hx (pixel)
.~Dx (pixel)
100
,-''~,,
8
t St
6 4
~.~ '"-.
-40
,' , , -==".~
4 .Rotation(deg..ree)
20"~K)'-6~) " - ~ ~
Fx: The x-coordinate of the center of gravity of the face part relative to the one of the whole head.
40
~ 0 " ~0 : j l e e )
2
:
' ' "x
6
"
f -';C'~"
-40
"~',,
8
",
. ~, "-.
',~ Rotation!de?re.ee)
-40
Hx: The x-coordinate of the center of gravity of the hair part relative to the one of the whole head.
~.,,
" ......
Dx: The difference between the x-coordinate of the center of gravity of the face region and the of of the hair region.
Hair Area (%)
Face Area(%)
Man1 lO£ 80 ...... .
100 80 6O 40 20
......... •
40
Rotation (degree)
~o;-~ '-38 ' 6 ' ~o' do: ~o~
2O
Man3 Man4
Rotation (degree)
Man5
-~o:-~ ~-ab ' b ':;o:do:~" ................
Face Area: Area of the face part measured asthe percentage of the frontal head.
Man2
Woman1
Hair Area: Area of the hair part measured as the percentage of the frontal head.
Rotation: rotation angle of the face relative to the frontal pose.
Figure 3 The results of the pose estimation of f a c e s . iT
ii!!!:i "~:i!~ii!::ili:ili!ili :~:...... ii!~iiiiiii:i::~:::~ ~:~ • - 30 degree
-5 degree
-4 degree
-3 degree
-6 degree
Figure 4 An image sequence obtained while guiding the active camera. T h e angles are real the direction of the face relative to the c a m e r a .
6. D i s c u s s i o n s and Conclusions This paper described a new approach for to obtain the frontal pose image of a face using an active camera. By using an active camera and visual feedback approach to get desired views of face that are required by many face recognition methods. This approach does not require the pre-built 3D model of a human head, and does not require 3D shape reconstruction during the operation, thus is very efficient and flexible. The experiment of obtaining frontal face shows that the desired views of faces can be get successfully using an active camera. Merging this method with expression recognition system or person
558 identify system, the facial expression system can achieve a big progress toward the real application. The experiments of the pose estimation of faces showed that the method described in this paper is very robust and the result of it is easy to use to control the active camera. Face direction
-30
-15
-9
-4
-2
Face direction
60
28
16
9
6
Estimated directior
-30
-12
-9
-4
-4
Estimated directior
64
23
13
5
8
- 15
-6
-5
-2
-2
Camerarotation
32
12
7
3
4
camera
rotation
Experiment 2
Experiment 1
Face direction
-30
-5
-4
-3
-6
Face direction
60
12
0
-2
-1
Estimated directior
-50
-2
-2
5
-4
Estimated directio~
95
23
3
-2
-2
Camera rotation
-25
-1
-1
3
-2
12
2
-1
-1
Experiment 3
Camera rotation
48
Experiment 4
Figure 5 The experimental results of obtaining frontal face using an active camera. The "Face direction" are real direction of the face relative to the camera. The "Estimated direction" is the direction of the face relative to the camera estimated using the method described in section 3. The "Camera motion" is the rotation angle of the camera along the circular path, we assign it as 1/2 of the estimated face direction since the face direction often over estimated.
References [1] Qian Chen and Saburo Tsuji, "Real Time Motion Estimation of Fingers in Color Image", Proc. 24th Conference on Image Engineering, pp.103-pp.106, 1993, Tokyo, Japan. [~t Giinter Wyszecki and W. S. Stiles, COLOR SCIENCE, John Wiley K: Sons, Inc. Guangzheng Yang and Tomas S. Huang, "Human Face Detection in a Scene", Proc. CVPR'93, pp. 453-458. [4] Xinguang Song, Chil-Woo Lee, Gang Xu, and Saburo Tsuji, "Extracting Facial Features with Partial Feature Template", Proc. Asian Conference on Computer Vision,pp. 751-754, 1994. [5] Akitoshi Tsukamoto, Chil-Woo Lee, Saburo Tsuji, "Detection and Tracking of Human Face with Synthesized Templates",Proc. Asian Conference on Computer Vision,pp. 183-186, 1994. [6] H. Zabrodsky, S. Peleg and D. Avnir, "Hierachical Symmetry", Proc. CVPR'92, pp. 9-12, 1992. [7] Roberto Brunelli and Tomaso Poggio, "Face Recognition: Features versus Templates ", IEEE Trans. Patern Analysis and Machine Intelligence, Vol. 15, No. 10, pp. 1042-1052, Oct. 1989. [8] Jiang Yu ZHENG, Qian CHEN, Fumio KISHINO and Saburo TSUJI, "Active camera guided manipulation", IEEE Int. Conf. Robotics and Automation, pp. 632-638, 1991. [9] Jiang Yu ZHENG, Qian CHEN, Fumio KISHINO and Saburo TSUJI, "Active camera controlling for manipulation", CVPR'91, pp. 413-418, 1991.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
559
Facial Features and Configurations Affecting Impressions of Faces Takashi Kato*, Masaomi Odd, Masami K. Yamaguchi, and Shigeru Akamatsu ~t ~ATR Human Information Processing Research Laboratories 2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-02 Japan {tkato, odd, kanazawa, akamatsu}~hip.atr.co.jp The present study investigated the relationships between physical characteristics and perceived impressions of human faces, namely fierceness and gentleness of the face. The study demonstrated that the spatial properties and configurations of facial parts that distinguish between fierce and gentle faces can be identified. Such information on physical characteristics of the face may be useful to interface designers who wish to incorporate faces into human interface in order to improve the intelligibility and/or perceived friendliness of human-computer communication. 1. I N T R O D U C T I O N Human-computer interaction (HCI) to date has been mostly text- or iconic-based. Previous studies show that a variety of factors affect the intelligibility and/or friendliness of such interaction. For example, the wording of system messages not only can confuse the user (thus resulting in unnecessary errors) but can make the system appear to be unfriendly or even hostile to the user [1]. It is not surprising that one would hope to utilize a (human) face as a friendly element of human interface. The intent is to bring some flavor of face-to-face communication into HCI, much in line with a general argument for anthropomorphizing interface agents [2]. If, however, a human face (whether a real or synthetic one) is to be incorporated into future human interface in order to improve its friendliness as well as intelligibility to the user, interface designers must first understand how the user's perception of the face might be affected by the visual cues of the face which they would intentionally or unintentionally make available to the user. This design issue on computer output should be distinguished from that on computer input, such as automatic understanding of the user's facial expressions or gestures [3]. In this paper we will report an experimental study which investigated the relationships between the spatial properties and configurations of facial parts and subjective impressions of the face. The main objective of this study was to clarify whether a particular facial feature (e.g., eyebrow tilt) and/or configuration (e.g., distance between eyebrow and eye) might determine whether a face of neutral expression is perceived to be gentle or fierce. *This research was conducted while the first author was a visiting researcher from Kansai University. tWe thank Ian Craw for producing the average faces presented in the Discussion, and Hideo Fukamachi and Shigeru Mukaida for technical assistance.
560
2. M E T H O D 2.1. Subjects A total of 40 undergraduate students from Doshisha University were paid to participate in the experiment. They were randomly divided into two groups of 20 subjects each. 2.2. Stimuli The facial stimuli used were black and white photographs of 101 males and 102 females who were in their 20's and 30's. The original pictures of 512x512 pixels were cut into 236x236 pixels in order to reduce the visible portion of the external features of the face such as hair, neck, and clothes. The visible portion of the face was normalized across the faces by keeping constant the proportion of the retained area to the face's baseline unit, which was defined to be the length between the mouth center and the line connecting the centers of the eyes. 2.3. Facial features and configurations For all the original faces mentioned above, the coordinates of a pre-defined set of 26 facial points were manually measured with the aid of an in-house tool developed on a Silicon Graphics system. We then defined three types of facial characteristics using another in-house tool which allows us to define any feature or configuration as long as it can be specified by the set of predefined points of the face. The first type of facial characteristics, which we termed individual features, is related to the properties of individual facial parts such as the eyebrow, eye, nose, mouth, chin, cheek, and face outline. There were 23 individual features which characterized the width, height/thickness, area, tilt, and/or (curving) shape of these facial parts. The second type, termed here as positional configurations, is concerned with the positional relationships between two points of different facial parts. There were 16 positional configurations which characterized the straight-line, vertical or horizontal distance between, or the tilt of the line connecting, two points of different facial parts. The third type, called area configurations, has to do with the areas defined by three or more points of at least two different facial parts. There were 8 such area configurations. It should be noted that such an area defined by the points of a single facial part was classified as an individual feature. Finally, we calculated the parameter values of all the defined features and configurations for each of the original faces. Since individual faces may have been photographed from a (slightly) different distance, the measurements of the defined features and configurations were normalized across the faces such that each measurement of a given face was taken as a relative value to its own baseline, which was defined to be the value of the vertical distance from the mouth center to the straight line connecting the centers of the eyes. All the measurement values in pixels were then converted to standard scores with # = 50 and cr = 14 within the set of male or female faces so that the variability of user-selected faces can be compared across different features and configurations. 2.4. Procedure The experiment was controlled by a Silicon Graphics Indy system which randomly drew from a database and presented 10 faces at a time on a selection screen and kept user-
561 selected faces in a background buffer screen. Subjects were free to select any number of faces from the currently-presented 10 faces and to discard any of the previously-selected faces from the background screen. One group of 20 subjects (11 females and 9 males) was asked to select the 8 most gentle/fierce faces from a database of 102 female faces and another group of 20 subjects (10 females and 10 males) from that of 101 male faces. Subjects were instructed to proceed at their own pace using their own criteria for gentleness or fierceness of the face, and to terminate the retrieval task when they were satisfied with the 8 faces collected in the background screen. 3. R E S U L T S
For each set of 8 faces collected by a given subject as being the most fierce or gentle faces, the mean and the standard deviation (SD) of each defined feature and configuration were obtained. Those means and SDs were subjected to a two-way (fierce vs. gentle x features or configurations) analysis of variance (ANOVA). For both female and male faces, ANOVAs of means and SDs both showed significant main effects for fierce vs. gentle faces and for different features and configurations, and significant interaction between the two factors. 3.1. F e m a l e faces Table 1 shows partial results of the analysis of simple main effects between fierce and gentle faces conducted on the mean data for female faces. The first column lists the significant (at the .01 level) features and configurations that distinguish between fierce and gentle faces. The second and third columns indicate whether fierce and gentle faces tend to have a smaller or larger value than the population average for that feature or configuration. The simple main effect analysis of the mean data also showed that the line connecting the left ends of the nose and mouth is tilted toward the right more for gentle than for fierce faces. The areas surrounded by different facial parts are smaller for fierce faces and larger for gentle faces than the population average, except that the area between the eyebrow and the eye for fierce faces and that between the nose and the mouth for both fierce and gentle faces are not significantly different from their population average. The analysis of simple main effects for features and configurations conducted on the SD data for female faces indicated that fierce faces tend to have smaller variance for the shape and size of the chin region and the distances between the eyes and between the eyebrows, whereas gentle faces have smaller variance for the face size and the positional relations between the horizontal end of the nose and the eye, eyebrow or mouth end. 3.2. M a l e faces Partial results of the analysis of simple main effects between fierce and gentle faces conducted on the mean data for male faces are shown in the bottom part of Table 1. The analysis also showed that the line connecting the left end of the nose and right end of the left eyebrow is tilted toward the right more for fierce than for gentle faces, which have a mean value roughly identical to the population average. Fierce and gentle faces are not significantly different in the size of the areas surrounded by the eyes and the mouth, and
562 Table 1 Distinguishing Characteristics of Fierce and Gentle Features/Configurations Face size Face width (along the mouth) Eyebrow tilt Eye height Nose length Upper-lip shape Mouth width Female Faces Chin size Eyebrow-eye vertical distance Eyebrow-nose vertical distance Eyebrow-mouth vertical distance Eyebrow-chin vertical distance Eye-nose vertical distance Eye-chin vertical distance Mouth-chin vertical distance Eyebrow shape Eyebrow thickness Eyebrow tilt Eye shape Eye height Eye size Male Faces Upper-lip shape Mouth width Distance between eyebrows Eyebrow-mouth vertical distance Eyebrow-nose vertical distance
Faces (Partial List) Fierce Gentle smaller larger shorter longer upward downward shorter average shorter longer average flatter shorter longer smaller larger longer average shorter longer shorter longer shorter longer shorter longer shorter longer shorter longer curved flatter thinner average-thicker upward downward flatter rounder shorter average smaller average-larger average curved average longer shorter average shorter longer shorter longer
by the nose and the mouth. The analysis of simple main effects for features and configurations conducted on the SD data for male faces indicated that fierce faces tend to have smaller variance for the shape of the nose, the horizontal width of the eye, the distance between the eyes, and the area defined by the horizontal ends of the nose and mouth, whereas gentle faces have smaller variance for the curving shape of the eye, the tilt of the line connecting the left end of the nose and the chin, the distance between the nose and the mouth, and the area defined by the horizontal ends of the nose and mouth. 4. D I S C U S S I O N
One important characteristic of the present method is that subjects are asked to collect a set of faces which all meet a particular retrieval requirement. It can be expected, therefore, that subjects would collect a set of faces with consistent values for important features
563 and/or configurations while making a necessary compromise on facial characteristics which are less important to them. It follows then that the relative size of mean SDs can be taken as a viable index of the relative importance of features and configurations [4]. In order to characterize fierce and gentle faces, therefore, we took into account the relative size of mean SDs in identifying those features and configurations which seem to distinguish between fierce and gentle faces. Female fierce faces appear to have tilted eyebrows, thin and tilted eyes, and a small mouth. The face outline tends to be thin, and the internal features compacted in the vertical direction of the face. Although the spatial separation between the eyebrow and the eye tends to be larger, the area surrounded by the inner corners of the eyebrows and eyes is relatively smaller than average. In contrast, female gentle faces appear to be characterized by horizontally longer eyebrows, bigger eyes, a larger mouth with a thinner upper lip, and a larger and rounder face. The internal features tend to be spread out along the vertical direction. Male fierce faces seem to have curved eyebrows, thinner eyes, and a thicker lower lip. The distance between the eyebrows tends to be shorter, and the area surrounded by the inner corners of the eyebrows and eyes smaller than average. Also, the eyebrows, nose and mouth are relatively compacted in the vertical direction. Male gentle faces, on the other hand, appear to have rounder eyes, a thinner upper lip, and a thicker lower lip. While the distance between the eyes tends to be shorter than average, the eyebrows, nose and mouth tend to be spread out along the vertical direction. The average fierce and gentle faces for females and males shown in Figures 1-4 clearly illustrate the general patterns of the features and configurations discussed above for fierceness and gentleness of the face. These faces were produced by averaging 5 most frequently selected faces in each category. The present study demonstrates that the spatial properties and configuration of facial parts that distinguish between different impressions of faces can be identified. Such information on physical characteristics might be useful to an interface designer who would want to select or design an appropriate face for a particular type of interaction between a system and the user. For example, the designer might want to adopt a (genuinely) gentle face for normal communication with the user but decide to introduce a fierce face where a serious warning is to be called for. Another example might be that in designing a synthetic face an interface designer might wish to manipulate the distinguishing characteristics of the face so that a standard face might appear gentler or fiercer depending on the intent to be communicated to the user. It should be pointed out that the experimental method used in the present research can easily be applied to the investigation of other types of impressions. In fact we have successfully used the method to investigate the physical characteristics of cheerful and gloomy faces [5]. A cautionary note, however, is warranted here. The present study has measured only the spatial properties of facial parts and their configurations. We have no intention of claiming that such spatial information is the most effective or sufficient means of distinguishing between different impressions of the face. The perceived impression of a neutral face is likely to be affected by other visual cues, such as the fairness and fineness of the skin and the three-dimensional shape information indicated by shading, both of which are
564 noticeable even in black and white photographs. We are currently pursuing other ways of representing facial images so that a more effective set of distinguishing characteristics, spatial or otherwise, of different types of faces can be identified.
:: : ::::::::::::::::::::::
Figure 1. Average Fierce Female
Figure 2. Average Gentle Female
~::! iiii!i!i!~!iiiii i~'~ .... ....
~:ili:~::::~
Figure 3. Average Fierce Male
ill ~
Figure 4. Average Gentle Male
REFERENCES
1. B. Shneiderman, Designing the User Interface, Addison-Wesley, Massachusetts, 1992. 2. B. Laurel, Interface Agents: Metaphors with Character. In B. Laurel (Ed.), The Art of Human-Computer Interface Design, Addison-Wesley, Massachusetts, 1990. 3. R.E. Eberts, User Interface Design, Prentice-Hall, New Jersey, 1994. 4. T. Kato and M. Oda, Indirect Measurement of Feature Saliency in Face Processing. ATR Technical Report TR-H-022, 1993. 5. T. Kato, M. K. Yamaguchi and R. Takahashi, Categorizing Faces Based on Subjective Impressions, 1995 (submitted for publication).
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
565
Anthropomorphic Media Approach to Human-Computer Interactive CommunicationUsing Face Robot H. Kobayashi* and F. Hara *
ABSTRACT
This lXttmr deals with the realtime response face robot to demonstrate the integration of three fimctions of active human interfaee(AHI) that realizes interactive human-computer communication through an anthropomorphic modality of face robot. As the 1st fimction of AHI, the face robot recognizes the existence of a human being in her view by using hi/her face image data and measures the distance between human being and herself, as the 2nd function of AHI, according to the distance obtained above, the face robot determines the facml expression to be displayed, and as the 3rd function of AHI, the face robot realizes the facial expression on her face. The experiment accomplishes the realtime response of the face robot in terms of her facial expression to the distance recognized in 80ms per one recognition. 1. INTRODUCTION
Since our dailyhmmn-to-human connnunicatim is fullybased on face-to-faceconnnunicativeintcmaion, the idealcommunication bctweon hunmn and computer iscamsideredto be establishedin the form of ~ f a c e interactiveinformationcxdmngc and thiskind of ideal communicationseems to be our goal in designingthe interfacefor hmrm.computer communication. The most critical feature of faceto-face communications ~ humam is the multiplicity of communication channels, such as facial expression, voice, gesture and so forth. The channel is, of course, a communication nxxtality associated with a lmrticular way of encoding infmmtion, for instance, an auditory chatmel of can34ng voice as ~ uttfxance, i n , ration and so on, and the visual chamiel for face actions including nodding motion ofa MM, facial expression and so fa~h. Thus the face-to-facecommunicationbetween htmam~ is a multi-modal communicative interaction and has been a long-time,hard-worksubject in socio-psychologyas well as in cognitivepsychology[1]. the face,to-face communication,Mehrabian[2] indicatedthat only7°,4ofmessage is carriedby linguistic ~ , 38°4 is dueto ~ e and 55% ofit is tmnffetred by facial exprmsims. Althoughthis kind ofstatislics seems to be mmgly ~ ~ t on ~ situation involved, it implies that the facial expressionis sndy a major modalityin the face-to-fac~communicationbetweon hunmns. We thus inmgim that, wlma designing an intezface between human and comtmtcr, the facial expression seems to be a good modality for hnpmving the communicabilityof message even in human and conqmtercommunication. The present paper assumes that the facialexpression is a communicativesignal to transfermostlypsychologicalmessages in human4ohmnmi commtmication. In this view of facial expression, there have bern quite a few wvdcs[3-6] on use of facial cxtxession for communication modalityin hunmn-comtmterint~mtion, which have been completelybased on comtmtergta#aica(CG) aplxoaclt We think that the CG ~ m c h can surely offersome advantagesin the human inlefface~ computerand human, but lime is no necessityto confinethe facial expressionwithin the 2Mimensional CRT displaydevices,wlxmwe want to expand the use of computer to artisticmedia, entmainment, and other new field~ We thoughtthat a 3Mitmnsioml, htmmn-likeface mightbe more realistic in hmmlr-col~uter communication iftmliz~. Then we developed a 3.dimmsioml, realistic fire robotas a conanunication modalityin h ~ communication. This in the firstplace will p m l x ~ a new conce~ of "ActiveHtma~ Interface:AHP' to realizethe intcracfvecommunication between h m m n and ccmlxaer.Thcxcafler,inorderto d c m ~ lhe integrationofAHI, wc willixcsmt the rcaltime~ ofa psycl~logieal distance of the face robot from its hurrah tmrmer, and also the realtime disphy of the tabors facial ~ o n in ~ to its "psF~logical distance" of the face robot measmxt Finallythe works to be done in future will be briefly disoamcd. 2. ACTIVE HUMAN I N T E R F A C E W h ~ we thinkabout h u r m n ~ communication as an advam~ form ofhlamn-eomlm~ intcamaion,cctnputerneeds to posses a specialintcxfaceto commtmicatc both ~ and psyr.hologicalinformationintcra~vcly with hunmn user as in hunmn-to-hmmn conanunication. The computer intca4_ace,then, must equip the conmaunicationfimction similar to that evolved in htmmn beings The function is, at least, thought to be ~ by the followingthree as pointed out by Arbib[7]:
* Departmentof Mechanical EngineeringScietr.,eUniversityof Tokyo 1-3 Kagurazaka,Shinjuku-ku,Tokyo 162 Japan E-mail: ~0103.me.kagu.mt.ac.jp
566
(1) Recognition of human user's intention, feeling or state of mimi, (2) Decifion of ~ action, and (3) Display ofitin an a l ~ form. As, on human useds side, he/she is alrmdy equipped with these three fimctions, the human user can tmdettake mutually inter communicatim with a ~ . This intamfive ~nmaunicalim mayresult in tmking it posst~olefor user's intdlismce to be activatm effectively. The firstfimctionoftlz~ threeisi n ~ as a "sensor"agentto detectthe message offeeling,ear~tionor the state ofhunmn 2nd fiatctimmay be a "c~nlzollc~agentto dctcm~e what kind ofactionshouldbe taken tohuman user,and 3rd one an "actuator" to outImtthe proper actim to ~msfcr the computers message. The hunmn usermay then bc i n ~ as a "controlobject"inthe v~ oon~olwhen we consider the interactive¢mlmunicationbetweenhunm and computer.From the discussion statedabove, we have re the cotw~t of"Active Human Intcfface(AHO"as a mw paradigm for developingthe technologyto realize a "face-to-face"communi, hunmn user and computer.The AHI is,of course,multi-modalconnuunicationmcdi~ we think about the &amns~tim of the AHI integrationby using visual infonmtion as communicatim modality,the folh information tm2cessing is neo4__~for AHI: 1. facerobot recognizes the appemmlce ofahuman being and ~ the psychologicaldistmc~ between the face robot and htmm 2. when the hmmm is getting close to the robot, the face robot starts interactiveresponse and than recognizes the facial extxessim hunm and resixmds to it. From the view point of'truman space"ineamnunicatim psyd~logy, the dimmcefnxn otherparty is very influential and which leads us to think that the measmml~t of the dimmce between the fw,e robot and its human pmlne, is also important fact ~'veloping AHItechnology.To dmmmate the integrationofAHI three fimOionsforthe situationstated above 1. and2., we have devc a realtime ~ ~ ~ tl~ facerobotand its hunmnimrtnerand also tmltirm displayo f ~ extxemkmof the face in reslmme to the psychologicaldimmce. In other mints, as the 1st function ofAHI, the face robot ~ the aplxamme of a h being inthe robotsview by usingthehunmn image dataand n~usurcs the distancebetween thehuman being and the robotsclf,as th function of AHI, according to the distance obtained above, the face robot dOmnines the facial expression to be displayed, and as tt function of AHI, the face robot getmates the facial expression on her face. 3. REALTIME MEASUREMENT OF ROBOT-HUMAN DISTANCE
In orderto r e c o ~ a human kmge by computer,parallelprocessingwas thoughtto bc also~ t in our researchwork sinc pothole to calculate all ~ data fast by dividing the tasks into several CPUs. We uscda ~ t c t ( T S 0 5 : 1 5 M ps,2.3 Mflops) ~. that performedImrallel~ for the realtitm distance~ t The tnmstmtersystemis camct with 4 other ~ by 4 serial links and performs plural tasks in one ~ by using the thread f i n ~ forpmallcl~ . In sideoftheeyeballsofthe facerobot,we installedtwo smallC C D cmmax~ 12*40nun, F=I 8mm) forvisualdataacquisition,we the Innmn image obtained from them for our realtimedistame mea.mrmlenL The clunaO.ai~'csof the pamlld ~ sy lmffonnmce constraintsthe system and the distame measuremmt procedme are briefly shown in the following sections: 3.1 Characteristics of Parallel Processing System The distame between the face robot aml its human lmrtner is measured in the following c~nditions: (a) We calculate the distancebyufingdiffaetreofthe gravitycenter betmma the two ~ ofo~~m~ installed in the right and left eyes of the face robot. (b) One color CCD camera is used to detect the hair and skin color( The other is black & white ). (c) Not necessaryto take a specialattentionto the ~ scene and lighting.
twoCCD carm
Since the memory catmcityfor color image data needs 3 times more than that for black and white one, it requires longer time for inmge lXOCessi~and thus we maployedblack and white ~ data for obtainingimage diffemitial. For acquiringthe data of hair anc color tmntiomd (b), the color image was used, but for others we utilized black and white monochrome image data by tmmfor colot(RGB) data to black and white one. The tnmsfommtionmethod from RGB colordata to mmochrome data (Y) is as follows by 1 YIQ color system: Y = 0299R + 0.587(3+ 0.114B It is noted here that the dimme m
~
(1)
has the followingconditions:
(1) Thisvis~ d i s t a m e ~ ~ e n t system wasusedimidea romL (2) The target person for visual dimmce tratstmanent was asstm~ not stayingand not to identifythe human itself ( an anthropano~r,,_ face robot was assumed to reqxmd to moving Ix~mn activdy). (3) The mngeoftl~ dimmce mmstmmm~was from 1.5mto 5.0m(the range depends ofcourse on the focus distameofthe CCD camera used). 3.2 Distance Measurement Procedure Fig. 1 shows the conmuOion of the mmsputer~ and data flowfor the realtime distance tmmsmxanent In this figure,each square enclosed bold line indicates the ttmlsputet used and each dam processed is tnmsferred along the allows by using data channel conummica-
567
\ CCD ~ 12nm
\
/ Black and White
TRIP with DSP
,~____'I~P with DSP
1/3 ®-I
' Rlghfl, L ~ l
,
CCD 12B
Color (RGB) (256*240pixel)
(256*240pixel)
t,U-~ _ Headpartl_ .I
/
I I'll tBlack&~hite~
',1 I I l i f t
,
- .....
.
~i~) (" 86*61pixel 1 I Trans. RGB into ~iK~!iiiii!J ~ Black & White
(~-2
(9-2
I--l[~::i::ili::iii::(::~$ii::iii::i::i::iii::!::i::iil
_
®-2
/ - Headpatt2 (.7)-2 -ti ," T ep, llettem,-'~I I\ Right, _ t ~ _ , \ ================================
~. . . . .
~: : ~ - ~ ' ~ . ~ J (~) (::i:::................. :::::~
®-x
~
CaLeflhec.aterefgravily o q t h e H e a d Partl with i860
III1®-1 v [_J~/,
lit
®_2
-Headpatti - ,~ /, Headpart2 ,~ Rlghtl&IMtl I iRightl&IMl2 t-
CaL of the crater of gravity af~H~d
"l]-"
]
Parl2
TRP with i860
TRP with i860 (~)d CaL ef the aimace bawem " l humm u d the face robot ~
F
RS232C
C o n t r o l l e r of Face R o b o t
Fig.1 The conamcfion of the tmmlmter system
~X
tion, which implies that one processing does not start the task until the information comes from the other processor according to the allow indicated in Fig,1. The timing of obtaining in~es from the right and left CCD cameras was conlrolled to be synchronous by the channel eommunicatio~ The spatial resolution of the one frame image is 256*240 pixels and 8 bit depth in color speclmn. The image filteringwas taken at the rate ofl/30s. In Fig.l, the shadowed portions indicate the processv ing for the colorimage data to acquire the head and Color Black & White skin colormentioned before. The details in each ~ are described a Fig.2 The conceptual procedure for vocessing @ to ® bellow. It is noted here that the spatial resolution of one image data was changed ifneeded a high speed calculation. The i.d. numberslocatedat the head of each explanation corresponds to that
- 1 To reduce the obtained imagedata to 52"48 pixel image data( one for 5 pixel data in horizontal and vertical directions)and forward
thena to ® -1. The same as 0 ) -1 for RGB colorimage data.
O) -2
In order to establish a fast calculation of image data, we divide the image data into 3 portions from the top to t_hebottom of the image
and eachofthoseistmnsferredtoe~ahof3 l m m p u t e r s . F o r ~ i n g f r o m @ to® mentiomdbelow are performedin each lrans[x~
~dently. -1 To reduce the obtained image data to 86 *61 pixel imagedata( one for4 pixel data for verticaldirectionand one for 3 pixel data for hmmntal direction)and is f ~ to ® -1. (~) -2 Byusing eq.(1), to transform RGB col~ image data to momchronaeinmge data and per fcrm the same as (~) -1. l n ~ ® , gmvitycenter is calculatedby ® usingin~edatanmationed~ -land(~ -2.Althot~toobtainarn~ewxanate gravityconter,it is better to use 256*240 pixd origiml inmgedata, t h e ~ ® needsrncretimettmnothertx'ocessing.Thenweused the transfnred image data. ® -1 To receive the f ~ i e d data from @ -1. @ -2 To receive the forwarded data fixxn0) -2. Forvocessing@ t o ® , Fig,2 shows the conceptual procedure oftheir data ~
.
568
-1 To calculate the hnage diff~mtial, i . e . , i f t h c d i f f e r ~ b a w e e n t h c p r e s e n t i m a g e d a t a ~ ~ ~ ~ 1 stcp before is larger than 21, which is empirically ~ for each pixel, we set the pixel value to 1 and others are set to 0. (g) -2 1)ByusingRGBinmgcdata, thc l ~ and skin color area arc damnined and sct the pixd ~ to 1 for their area and 0 for other are~ It is noted here that, if there is no hair and skin col~ area, the tnmsputer system judge that human being does not appear. 2) By using eq.(1), to transfer RGB image data to momchmme data and perform the same as (g) -1. To detmnixe the skin color, we employ eq.(2)[10]: if
l10
4 0 < R < 50,40
(3)
To obtain the top and the b o ~ portions of the head ( hair and face) shown in Fig,2, the following two are pcrfomled: For thetop portion, by using the image ditferential mentimed (~ -2 2) where the pixel value is set to 1, we seek the smallest ycoordinate value within the area and assume the point as "top". For the bottom portion, byusingtl~rcsultof(~ -2 1 ) a n d ~ -2 2), we seek the largest y ~ value as for the bottom, within the area ofpixel value ~ to I. We assume that, since the right and left CCD cameras are installed horizontallyat the same level, the top and the bottom men&reed above are equal to each other from both CCDs. -1 By using the image diffaenfial result(~) -1), we acquire the x-coordinates ofthe right and lelt ends within the area of the pixel value assigned to 1 and between the top and the bottom already obtained, and denote each of their values as "fight1" and "left1". @ -2 Thesameas@ -l is pa'formed by using the result of @ -22). -1 Usingthetlneecombinatiousofthewduefor thetop, bottom, rightl andlefll obtain~from3 lransputers, we calculate the top, bottom, fight and left end values for the whole image data. (~ -2 T h e m n ~ a s ~ -lispefformedbyusingthcmmltof® -2.. ® - 1 1) To obtain the black area ofhair for the area within the top, bottom, right and left value obtained ~ - 1, we assign 1 value to the pixels if the value of the pixd is under 60, and 0 value to others. 2) To calculate the gravity center of the black are~ The calcuhtim method is shown below:. (xi, yi) shows the x-y coordinates of image data and the pixcl value expresses Pij for (xi, yi). Namely, (xi, yi) = Pij =( Oor 1 )
-
(4)
Since we asam~ that the fight and left CCD cameras are installed horiamtally at the same level mmtiomd before, there exists the diffaet~ in thehorizontalcoordinatesofthc gravitycam~ bctwe~ rightand Icflimage. Then the gravitycenterforIeR image(gOis askedby cq.(5).
Namely,
eettem l),i~ I
2(xi × Pij) g~
=
J-~"-~" Bettem R i ~
2 j-'r~
l
(5)
2 (Pij) i -td
1
® -2 Thssamcas® -l is paformed to obtain thc gravity center for thc right imagc (gr).
® By using the method for stereo image mcasm~mt shown in eq. (6), we obtain the diamcc (z) ~ CCD camera ) z=B*F/d Here,
the human and the face robot(
(6)
B : the length between right and left CCD camera F : focus length ofCCD camcr~ = 18ram d:=gr-gl
3.3 Experimental Results Let us explain what kind of ~ e n t was ~ with ~ to the realtime distance meastmmlent by using th~ system We asked a subject to walk at a normal speed from the position of 5.0m to that of 1.5m from the face robot m ~ ~c ~ ~ of which two C C D ~ were installe~ In ¢~lcr to cah~te tl~ distanc~ of the subject from li~ CCD cameras, the subject was asked to put on the one end of a tape meastae on his/her waist and to walk with it. We ~ the scale indication at ofthe other end ofthe ~ cn the video tape by ush~ other CCD canm-'a_R is no~dl~c ~ t h c ~ c r s ~ l ofthe CCD ~ was set 1/2000s. Thou we could measure
569
6
position of human --
5
--. ~ ~ t the ~
result by
Table 1 Error ratio ofmeastmanaat result unit: [%] ,,
Subject Experimentno.
2 1
0
J
0
1
A
i
i
2
3 4 5 T i m e [s] Fig.3 An example ofmmmnement result
A
B
C
1st 2rid 3rd
1st 2nd 3rd
1st 2rid 3rd
Max. abs. errorratio
4.93 7.00 6.70 10.2 9.49 7.12 8.73 7.58 6.62
Ave. of error ratio
1.89 1.78 2.86 2.80 3.89 3.74 3.26 2.99 2.62
J
6
the position of the subject in amm order. We employed three subjects (A, B and C ) and the exlxtinmit was perfom~ three times far each subject Pig,3 shows an example ofthe ~ result, where hhe solid line ~ the Ix~'on ofsubject fiam the face robet measta~J by the tape rmamre (line position) and the dotted line means the measurement result obtained from the ~ system (n-easmed position). Table 1 shows the maximum absolute error ra~o ~ as the ~ value obtained by dividin8 etmt((n-amaxxlposition)- (lmeposifion)) by ~ueposifion andthe avaage afar ratio for each oqxaJmmt. Although the largest value among the nmximum absolute error ratio is about 10%, we find that the avaragem~rrati~istmder4%.Itwasf~undc~earthatthemmsputerneet~thetimefr~mV~x~ssing~ t o ® needs about S0ms. This result indicates that, by using mmslmter system, the distmce ~ m t is perfumed accurately at a high spe~ of about gores per one mmsmmma. 4. REALTIME RESPONSE OF THE FACE ROBOT
As pointed out "Hmmn Space" in communication psychology[8,9], the distance between the face robot and its human pmlx~ is considered influential and important factor in robot-hmmn communicatiotr i.e., we think that the realtJme reslXxlse of a face robot to file '~ychological distance" is ~ t o r for robot-hmmn ~lmunicafic~ Thus we ~ the ~ e n t far the integration of AHI by and changing f~Jal expression of the face robot to human pruner activelyin response to the distance between the face robot and its human partner. Far the 1st fm~on of AHI, we developed ~ and realtime distance measmement by using the t m ~ u t e r system mentioned above. Far the 2nd and 3rd fu~tions of AHl, we employ a v a t prinfitive c~lzoller far tabors simple mind flint, when the distance is 5m, the face robot shows nonml face( Fi~4 (a)) and, when the distance bec(xnes 1.Sin, the face robot shows "surprise"(]0~4 (b)). We connected the tramputer system and the PC conlroller of the face robot with RS-232C and then the mmstmmm~ result of the distance was Wansferred to the PC from the lnms~ter system In the PC, accordingto the distance m ~ result and the 2nd finx~on of AHI, the facial expression to be expressed on the face robot was detmnimd as in mentioned just before and, as the 3rd function of AHI, its facial expression was displayed on the flee robot sequentially in 30ms. Since the distance ngastaemeot bythe ~ system(80ms) and the display of facial expression on the face robot(30ms) were perfmI~ in parallel, the speed far ~ facial expression of the face robot seems to need 80ms per om facial appear~ce change. An example ofthe xeallime response ofthe face robot as an AHI is shown in Fig,5. In this figtae, the number axreslxlxts to tliat in 1~g.6 which indicates the attention points on the face robot for displaying the facial expression. The dotted line shows the target displacement of the attention point determined by 2nd function o f A ~ meafioaed just befare and solid line shows the actmd d i s p ~ of the attention poinlx It is noted hoe that the attention point indicates the point to be controlled for displaying facial extxession. We have found flora Fig.5 that the display ofthe facial expression to be expressed is realized exactly. This realtime response of the face robot is simple, but the realtime integI~on of those AHI three fm~ons should be ~ for develqxnent to generate mine sophisticatedrespons~ namely,the face robot meastaes file distance between the human pelmr and itsel~ and ~ to the distance, tie face robot "mind" decides facial eXplession and changes its ficial expression at the speed of 80ms per cycle. ~ , of course, how to change the facial extxession is strongly~ on the situation and context, this paper shov~xl the possibility ofrealtime faco-to-facecommunication b a w e ~ human and computer through an anfluopc~rphic, 3 dimensional modality of face robot. 5. DISCUSSION AND FUTURE WORKS
The 2nd fimaion of AHI is an essential part of"artificial mind" which inchrks variouskinds ofinfartmtion Vocessing paradigms such as artificial intelligence, artificial emotion, computatioml psychologyand so m. Although there are quite a few research works on artificial intelligare, we are afraid that they have not yet developed any artificial intelligence to be used in human
570
iiiii L (a)nmnal face
2-Y
i~'~~~ ~
2-X
Co)strprise
FigA Facial expressious disphyed by the face robot ~,~. 10
(._o/
. . . . . . . .
8
6 4
•
7-X7 . y ~ 6
actual d i s p ~ e n t
2
..-- .---
>X
target d i s p ~ c n t
4
0
2
Time[s]
4
6
Fig.6 Attention points for expressing facial expression
Fig.5 Result of realtime response ofthe facembot As pointed out beftxe it is sta'dy better to additionally ~nploy other kinds ofcommuaicatiou modalities such as facial exptessiom and voices, and we are specially interested in Vostxiy infonnafion processing in ~ recognition and synthesis. As a new directiou of our research, lhe integtafiou ofrealtime facial expression ~ t i o n with prosodicspeechrecognition is the most urgent and exciting one to improve file ~ u n i c a b i l J t y . Finally it should be pointed out the followings: Multiple channels in h ~ u t e r communication can surely redtre the current necessity ofcormnunicatiou regulafiou ~ upon hmmn users and can invoke more ~ u s , human-like communicatiol~ This will flourish when we would devdop li~eessential technology or "artifioal mind" for dynamically coordinating the messages transferred through various om~nunication modalities. Seomdly the anlhropotm~hism in h u t m n ~ p u t e r interface will be surely realized by using a 3dimcmsioual, hmnan-like face robot. The face robot can recogfize human user's fieial expressions as well as prosodic speech and then can display the messages through the robofs ~ expression and prosodic speedt This might imply a hunmnoid computer with'which we would omrantmicate as in human-to-human c~mmunicati~
ACKNO~MENTS This researchwork was pmliallys t ~ ~on, through 1992-1994.
by the ScientificC_aants04236107 and 05452172, Japanese~
of Cultme and
REFERENCES (1)Brace, V., ' ~ R ~ g Faces(Iranslatedinto J a l ~ n e s e ) " , SciencePub. Co. pp.l-205 (1990) (2)M~ A., '~_,ommunicalionwithoutWords" PsychologyToday,Vol2, No.4 (1968) (3) Takeuchi, A. andNagao, K., "CommuaicativeFadalDisplays as a New Convotsafional Modality", ACM/IFIP INTERCHI~3, pp.187-193 (1993.4) (4) Seto, S. ot al, ' ~ / l ~ o d e l Response ofa R~fl-TtmeSpeech DialogueSystem(inJapanese)", 8th Syrup.Human Intorfa~, SICE, pp. 693-698 0992) 5) Hara, F., "ANow Panutigm far Robot and Human Communicalion(inJatxmoso)",JSME Proc. Robolicsand Mocha~nics, No.940-21, pp.l-9 (1994) (6)Hasogawa, O., Yokosawa, K. and Ishizulag M.:'~RoallimeParalleland ~ v e Roc~gnilionof Human Face for a Naturalistic Visual Human Intoffs~in Japancso)", The Trans. ofIEICE D-II, VoI.J-77-D-H,no.l, pp.108-118 (1994) (7)Arbib, M. A., '~eural Netv~rks and Brain (translated into Japanese)", Scien~ Pub. Co., pp.l-507 (1992) ($)N. L. Ashton, M. K Shaw and A. P. Worshanr "Affo~ive Reaction to Intorporsonal Dislanc~s by Friends and Slrangors", Bulletin of file Psychonomic SociotyVoL15(5), pp306-308 (1980) (9)Fulmda, F., Shigenofi, I. and Anti, F.:"Retx~fifionof Human FaceUsing Fuzzy Inference and NeuralNetwork (in Japanese)", JSME Tmas.C, Vo1.59,No.558, pp200-206 (1993)
IV. Ergonomics and Health Aspects of Work with Computers
This Page Intentionally Left Blank
IV.1 Health Aspects
This Page Intentionally Left Blank
Symbiosisof Human and Artifact Y. Anzai, K. Ogawaand H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
575
Symptom clusters among VDU- workers. Part of the Norwegian field study: The VDU workplace - A new ergonomic concept*. K. I. Fostervold a, I. Lie b, S. Larsen c, G. Horgen d, A. Aarfis e & A. V~gland f a,b Vision Laboratory, Institute of Psychology, University of Oslo P.O. Box 1094, Blindern, N - 0317 Oslo, Norway E-mail: K.I.FostervoldC~sykologi.uio.no CMedstat Research AS. P.O. Box 210 N-2001 LillestrOm dCollege of Engineering, Dep. of optometry. P.O. Box 235 N-3601 Kongsberg eAlcatel STK AS. P.O. Box 60 Okern N-0508 Oslo fErgonomidesign AS. Huitfeldsgt. 25 N-0253 Oslo
1. I n t r o d u c t i o n Over the last decade implementation of advanced information technology has revolutionised ordinary office work, and video display units (VDU) are today a dominating device in most western workplaces. Due to this rapid implementation very little was known about possible health risks that could originate from prolonged interactive VDU work. When reports about adverse health reactions among VDU-workers started to appear, this was taken seriously by many scientists and health professionals. The initial concerns were mainly directed toward eye problems. Visual and oculomotor consequences of sustained excessive near-work have been research topics in visual perception and optometry for almost 50 years (1,2,3,4,5). Since typical VDU-work tasks are dominated by intensive and sustained visual work at short distances, a renewed interest in these issues was therefore a logical consequence. Subsequently, also other health concerns appeared, and typical problems discussed in the literature are: musculoskeletal problems, psychosocial factors, concentration problems, skin symptoms, headache, dizziness and general tiredness (4,6,7,8,9,10). Among these, musculoskeletal problems and psychosocial factors have gained particular attention. Musculoskeletal problems are reported to be very common in the industrialised world (11,12). Repetitive movements and long lasting static muscle load are assumed important factors in the aetiology of
* "The VDU workplace- A new ergonomic concept", are financially supported by: The Norwegian Research Council and The National Insurance Administration.
576 these conditions (13). A causal relationship between musculoskeletal problems and intensive VDU-work are therefore expected since most VDU operations are sedentary and necessitate repetitive and restricted body movements. Recent research reports have shown an increase in musculoskeletal problems among VDU-workers even though they are not always consistent about whether the increase is higher among VDU-workers than other employees (14,15,16). Other studies emphasise psychosocial factors as an explanation. In this perspective, focus is directed toward environmental factors such as job design, time pressure and obsolescence or more person centred concepts like mental workload, coping style and cognitive failure (9,10,17,18). In the public opinion new technologies always represent a source of potential hazards. In such situations people tend to blame existing symptoms already associated with certain causes, on new causes. The symptoms associated with VDU work could therefore be interpreted as nothing else than the normally occurring levels of non-specific symptoms that are present in any population. Research projects on VDU-related complaints use questionnaires providing frequency distributions of visual, musculoskeletal and vegetative symptoms. Little is known, however, as to whether or not these exist as symptom clusters among VDU-workers. If such clusters exist, advocates of the psychosocial explanation must explain why people develop different symptoms. If clusters do not exist, advocates of a more direct causal link between VDU-usage and symptoms must explain why this specific influence gives rise to different symptoms. The aim of the present study is to search for symptom clusters among VDU-workers participating in an on-going field study in Norway. 2. M e t h o d
The on-going field study "The V D U workplace - A new ergonomic concept", emanate from laboratory research which has shown significant health effects of gaze angle and optical corrections during VDU-work (19,20,21). The objective is to implement these variables in an ordinary office environment and to investigate whether they also reduce complaints in natural settings. The field study design includes the following health measures: a subjective symptom questionnaire, EMG-measurements, a visual examination and a health examination. The present paper is based on the symptom questionnaire administered as a part of the baseline measurements in this field study. Subjects were recruited among employees in The National Insurance Administration, The National Ensurance Office for Social Ensurance Abroad and Local Social Ensurance Offices in Oslo. A sample of 150 subjects out of a total pool of 500 employees was selected according to a given set of criteria. Subjective symptom reports are often subject to criticism. Scientists claim that they may produce biased results as a consequence of self-selection. This represents a danger of an over-reporting of symptoms, since respondents who complete questionnaires, are in most cases highly motivated. To prevent this, the symptom
577 questionnaire was distributed personally to each subject, together with short oral instructions. Each subject was told to take the questionnaire home, to read the instructions carefully and to complete it under quiet conditions. The questionnaire h a n d in procedure included a verification system that made it possible to distribute personal reminders. Very few reminders were necessary and all subjects, except one, (99.33%) returned completed questionnaires. To enhance the validity further, a comparison of the symptom frequencies found in this study was performed with symptom frequencies reported in other research papers (4,8,10,14). This comparison did not indicate that our results were biased toward an over-reporting of subjective symptoms. The sample consisted of 111 women and 38 men. The mean age was 40,9 years, with a standard deviation of 9,7 years. All subjects were experienced VDUusers and had VDU-~ork, mostly routine and non-routine data dialogue, as a major part of their Working day. No particular change regarding the type of VDU-equipment was done as a part of the study. The equipment most subject used on a daily basis was a 14-inch low radiation colour monitor, 70,5% used a TDV 5330 (Tandberg Data A/S, Oslo, Norway), a monitor with a refresh rate of 72 Hz, non-interlaced. The symptom questionnaire is developed by the authors and precursors of this questionnaire have been used in other studies (20,21,22). Each symptom category in the questionnaire was recorded through several items, in order to endeavour different facets related to the complaint. This diversity in items made it necessary to reduce and standardize items applied in the present analysis. 17 key items, one from each symptom category, were chosen a priori and factor analysed using principal components analysis with Varimax rotation. A list of the symptom categories included in the questionnaire is given in table 1. A 4 factor solution explaining 52,6% of the variance was selected on the basis of its ease of interpretation and relatively high communalities. The 4 factor solution was also in accordance with factor solutions found in other studies (23,24). Items were summed to form the following 4 symptom variables: 1. Visual symptoms 2. Musuloskeletal symptoms, 3. Vegetative symptoms and 4. Specific symptoms (Skin and forearm]hand symptoms). A hierarchical cluster analysis was used to produce empirical clusters of subjects classified on the basis of their scores on the four symptom variables. Different cluster solutions were then compared and validated according to the criteria of replication and interpretation. Replication is in particular regarded as one of the better ways to validate cluster solutions (25). 3. R e s u l t s .
The analysis was performed by use of the hierarchical agglomerative cluster procedure in SPSS, with Within-groups average linkage as method and Pearson correlation as similarity measure. The dendrogram was inspected and solutions of 2,3,4,5,6,7 and 8 clusters selected to undergo further examination.
578 A split half procedure was used to examine replicability of different cluster solutions. The sample was randomly split into two subsamples and cluster solutions from the subsamples compared. The analysis show t h a t the 4 and 5, 6 and 7 cluster solutions were quite similar in the two subsamples. The criteria of interpretation show t h a t the 4, 5 and 6 cluster solutions were easily interpreted. The 7 and 8 cluster solutions gave some clusters with very few subjects. Based on the information about the validity criteria, the 5-cluster solution was chosen. It was stable, easily interpreted, parsimonious and represented the clustering p a t t e r n of the sample in the most appropriate manner. The five clusters may be described as follows. Cluster 1 is characterised by a relatively high frequency of musculoskeletal symptoms and very few other symptoms. Cluster 2 is characterised by a relatively high frequency of visual symptoms and very few other symptoms. Cluster 3 is characterised by a high frequency of specific symptoms and some musculoskeletal symptoms. Cluster 4 is characterised by a high frequency of vegetative symptoms and very few other symptoms. Cluster 5 is characterised by relatively high frequencies of visual and musculoskeletal symptoms, some vegetative symptoms and very few specific symptoms. To enhance the understanding of the 5 cluster solution, each symptom category was recoded into a binomial variable. The percentage of subjects, in each cluster group, experiencing a symptom was then calculated. The results are shown in table 1. Table 1. Symptoms in the questionnaire and yes answer distribution in the clusters.
Symptoms in the questionnaire 1. Focusing problems 2. Headache 3. Pain/tension- neck/shoulder 4. Pain/tension in the back 5. Pain/tension- forearm/hand 6. Pain/tension- leg/foot 7. Skin problems 8. Dizziness 9. Nausea 10.Concentration problems 11. A general feeling of tiredness 12. Pain in the eyes 13. Tired eyes 14. Problems with line tracking 15. "Foggy" letters or words. 16. "Doubling" of letters or words 17. "Shivering" text
Clust. 1 Clust. 2
Clust. 3
Clust. 4
Clust. 5
% Yes 16 89 95 84 11 05 16 16 05 37 100 37 53 00 21 00 00
% Yes 19 52 93 62 55 38 33 07 00 26 71 45 76 14 33 17 17
% Yes 47 73 80 53 20 13 40 33 60 60 100 60 47 13 27 13 07
% Yes 48 80 80 40 16 20 00 68 08 64 00 72 88 44 44 28 28
% Yes 30 13 33 13 13 10 33 07 03 30 57 63 63 40 50 40 30
579 Except for the main tendency found in the cluster analysis, the results revealed some interesting symptom distributions among the groups. It is particularly noteworthy that some symptom categories (Nos. 2, 3, 11, 12 and 13) seem to be quite common in most groups. 4. D i s c u s s i o n
The results from the cluster analysis imply the existence of differences in symptomatology between subgroups of VDU-workers. The existence of such subgroups agrees with the notion of different risk factors associated with VDUwork. Subgroups experiencing mostly visual symptoms, musculoskeletal symptoms and their combination are, in this perspective, accounted for; visual symptoms because of sustained near work and probably minor oculomotor and refractive errors; musculoskeletal symptoms because of the increased in sedentary work, and the combination of symptoms because of both risk factors or, as recent research proposes, through the influence of oculomotor factors in the aetiology of occupational cervicobrachial diseases (12,26). The subgroup experiencing mainly vegetative symptoms is also accounted for, since several studies have demonstrated a strong relationship between vegetative symptoms and psychosocial factors (12,24,27). The aetiology involved in the subgroup associated with skin and forearm/hand symptoms are less obvious. The forearm/hand symptoms could be associated with the so called "mouse syndrome", while a possible causal link between skin symptoms and VDU-work are still under discussion (8,12,14). Perhaps these symptoms are not functionally inter related at all. Alternatively, the existence of symptom clusters may be explained on the basis of psychosomatic theory. Already in 1949 Malmo and Shagass described the principle of "symptom specificity" (28). This principle is derived from evidence showing a specific association between symptoms and physiological mechanisms being susceptible to activation by stressful experiences. The observed clusters could, therefore, be explained solely by symptom specific reactions to workplace stress. However, the symptom clusters are not that clear-cut. A more subtle picture emerges if one considers the frequency distribution in table 1. This table shows that most symptoms are represented among most VDU-users, even though there is an overrepresentation of certain symptom categories in the different clusters. This fact presupposes that more general factors have to be taken into account when discussing the causal relationships of symptom clusters. The baseline health measurements of the present study are to be replicated during the autumn of 1995. The repeated measurements design, which makes it possible to investigate changes in magnitude of symptoms, also opens the possibility to investigate the stability of those clusters reported in this paper.
580 References
1. C. Behrens, and S.B. Sells, Arch. Ophth., 31, (1944) 148. 2. K. Brozek, E. Simons and A. Keys, Am. J. Psych., 63, (1950) 51. 3. W.J. Smith, In, Proceedings of the Human Factors Society, 23rd Annual Meeting. (1979) 362. 4. M.J. Dainoff, and A. Happ, Hum. Fact., 23(4), (1981) 421. 5. R.A. Tyrrell and H.W. Leibowitz, Hum. Fact., 32(3), (1991) 341. 6. E. Grandjean, Ergonomics and Health in modern Offices, Taylor & Francis: London, UK, 1984. 7. M. Hagberg, and G. Sundelin, Ergonomics, 29, (1986) 1637. 8. J. Evans, Work and Stress, 1(3), (1987) 271. 9. A. Smith, D. Peck, and T. Clatworthy, Work and Stress, 3(2), (1989) 195. 10. P.T. Yeow, and S.P. Taylor, Appl. Ergonomics., 21(4), (1990) 285. 11. A.C. Mandal, In, D.J. Oborne (eds), Contemporary Ergonomics 1985 Taylor & Francis: London, UK, 1985. 12. U. Bergquist, Scand. J. Work Environ. Health, 10, suppl. 2, (1984). 13. A. Aar~s, Scand. J. Rehab.-Med., Supplement No 18. (1987) 14. U. Bergqvist, B. Knave, M. Voss, and R. Wibom, Int. J. Hum. Com. Int., 4(2), (1992) 197. 15. M.J. Smith, P. Carayon, K.J. Sanders, S.Y. Lim, and D. Le Grande, Appl. Ergonomics., 23(1), (1992) 17. 16. K. M. Zyla, Appl. Ergonomics., 24(6), (1993) 432. 17. H. Kahn, and C.L. Cooper, Curr. Psych. Res. & Rev., Summer, (1986) 148. 18. S.Y. Lim, and P. Carayon, In, A. Grieco, G. Molteni, E. Occhipinti and B. Piccoli, (eds.), Book of Short Papers, Work With Display Units '94, Vol 1, C9 University of Milan: Milan, Italy, 1994. 19. T. Paulsen, Proof-Reading from VDU: The Effect of Vertical Gaze Direction on Speed, Acuity, Subjective Discomfort and Preference. Institute of Psychology. University of Oslo: OSLO, Norway, 1990. (In Nor.) 20. I. Lie and R. Watten, Ergonomics, 37(8), (1994) 1419. 21. I. Lie and K.I. Fostervold, In, A. Grieco, G. Molteni, E. Occhipinti and B. Piccoli, (eds.), Work With Display Units '94 Elsevier Science: Amsterdam. 1995. (In press) 22. R. Watten, I. Lie and S. Magnussen, Behav. & Inf. Techn., 11(5) (1992) 262. 23. R. Watten and I. Lie, Nord. Ergonomi, 5 (1987) 8. (In Nor. Eng. summary) 24. H. Ursin, I.M. Endresen and G. Ursin, Eur. J. App. Physol., 57, (1988) 282. 25. P. Carayon, Ergonomics, 37(2) (1994) 311 26. I. Lie and R. Watten, Eur. J. App. Physol., 56, (1987) 151. 27. O. Vassend, R. Watten, T. Myhrer and J.L. Syvertsen, Soc. Sci. Med., 39(4) ( 1994 ) 583. 28. R.B. Malmo, C. Shagass and F.H. Davis, Psychosom. Med., 12, (1950) 362.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
581
C o n s t r u c t validity of c o m p u t e r anxiety as m e a s u r e d by the C o m p u t e r Attitudes Scale* Deane, F. p.1, Henderson, R. D. 2'3, Barrelle, K. 2, Saliba, m. 2'4, & Mahar, D. 2 Department of Psychology, Massey University, New Zealand. Trust Project, Computer Sciences Lab., RSISE, ANU, ACT 0200, Australia. Now at Department of Applied Psychology, UC, P.O.Box 1 Belconnen, ACT 2616, Australia. Now at DSTO Aeronautical and Maritime Research Laboratory, Air Operations Division, Human Factors, PO Box 4331, Melbourne, Victoria 3001, Australia. 1.
Introduction It is becoming increasingly difficult for individuals to avoid contact with computer technology. Along with these technological advances come a number of associated stressors. There may also be significant job task changes as a function of computerised technology and the need to master a variety of computer software packages. Individuals vary considerably in their response to these challenges. A significant proportion will experience negative reactions. One reaction which has generated much interest and research is computer anxiety. This is attested to by the fact that at least 12 questionnaires have been developed to measure computer anxiety [ 1]. Most people initially experience hesitation when confronted with new computer technology, or have heard anecdotal reports of such experiences [2]. Anecdotal reports are typically restricted to examples of people in the work environment avoiding wordprocessers and handwriting reports [3]. Estimates of the prevalence of computer anxiety range from 5% [4] to 30% [5] with specific sub-groups having higher prevalence rates [6]. Despite the high prevalence rates and apparent face validity, the construct validity of computer anxiety requires considerable clarification. Inconsistent use of terminology has resulted in some conceptual uncertainty regarding the construct of computer anxiety. There have been several terms which all relate to the construct of computer anxiety :- computerphobia, computer fear and computer anxiety [7,6,5]. Different criteria for each definition has substantial implications for both estimates of prevalence and subsequent corrective interventions [5]. The construct of anxiety, per se, has historically been steeped in definitional ambiguities. Resolution of the definitional and measurement problems associated with anxiety has been facilitated by the state-trait anxiety distinction. State anxiety is considered a relatively unstable, transitory, situational anxiety response [8,9]. A person experiencing state anxiety is typically experiencing an unpleasant emotional state characterised by apprehension and worry. Trait anxiety, on the other hand, is a more stable and enduring characteristic, more consistent with a personality trait, but does not imply permanent anxiety. Trait anxiety refers to "relatively stable individual differences in anxiety-proneness." [9, p. 1]. Theoretically, people with high levels of trait anxiety are considered more likely to respond to stress with elevations in state anxiety. Whether people high in trait anxiety react to stressors with high levels of state anxiety depends on the extent to which they perceive the specific situation as threatening, and this is influenced by each individual's past experience [9]. The state-trait distinction has rarely been made with regard to the construct of computer anxiety. In their efforts to clarify the definition of computer anxiety Cambre and Cook [ 10] cited two unpublished studies which explored the relationship between computer anxiety and state-trait anxiety. Both studies concluded that computer anxiety comprised state anxiety resulting form exposure to computer use. Cambre and Cook [ 10] concluded that "the relationships between computer anxiety and
* This reseach has been carried out on behalf of the Harry Triguboff AM Research Syndicate.
582 state and trait anxiety need to be explored further and classified if computer anxiety is to be defined accurately", (p.53). Several studies have done this, but the results have offered little theoretical clarification of the relationships and produced contradictory results [ 11,12,7]. In assessing cognitive factors in computer anxiety Glass and Knight [ 13] found that high computer-anxious subjects scored significantly higher on the STAI trait scale than low computeranxious subjects. They found highly computer-anxious subjects experienced significantly higher levels of anxiety than low anxious individuals immediately after beginning the task and after being informed of an error in their performance. This implies that computer anxiety is related to both trait and state anxiety. The authors conceptualised computer anxiety as a "situation-specific anxiety", but stated that unlike other situation-specific anxieties, computer anxiety research has not been based on clear theories or models. This criticism was also indirectly highlighted by Harrington et al. [5] who found that computer anxiety has less effect under certain conditions, "suggesting that computer anxiety may be a temporary, normal form of state anxiety", (p.354). This conclusion was viewed as highly speculative because state anxiety had not been tested or controlled for. The present study focuses on the use of the Computer Attitudes Scale (CAS) [ 14], in particular the Computer Anxiety subscale (CAS-Anxiety) as the measure of computer anxiety. The CAS has been used extensively in studies of computer anxiety and is fairly typical of the range of anxiety scales [ 1]. The CAS-Anxiety appears to be consistent with other forms of situational anxiety where anxiety occurs in a specific class of situations [ 15]. Respondents are asked to respond to 10 items describing reactions when working with computers. While the CAS-Anxiety appears to be an example of "trait" situational anxiety, there is some ambiguity in that respondents are not asked to respond as to how they "generally" feel. The present study aimed to clarify these constructs by looking at the theoretical relationships between state anxiety, state anxiety when imagining recent computer interactions, trait anxiety and computer anxiety as measured by the CAS-Anxiety. Theory and supporting empirical evidence [ 12] suggest that those individuals with high scores in trait anxiety should react to relevant stressors with higher scores in state anxiety than those low in trait anxiety. Given, that the stressor must be perceived as threatening, we predicted that those high on a trait oriented measure of computer anxiety (CAS-anxiety) would respond with higher levels of state anxiety in situations involving computer use, than those low in trait oriented computer anxiety. Construct clarification is also attempted by examining the theoretical relationship between computer anxiety and avoidance. An anxiety-response pattern involves "an array of avoidant-defensive behaviours", [ 16](p.183). Despite a number of theories predicting avoidance of anxiety arousing situations, empirical findings related to computer anxiety have produced theoretically incongruent results. There have been few studies in which a relationship between computer anxiety and avoidance has been established [17,7,11]. The implications of the relationship between computer anxiety and computer avoidance are far reaching. There are serious implications for productivity if up to 30% of the workforce are computer anxious and avoiding computer use. Preliminary studies suggest that information system failure (especially non-use) may be associated with a number of psychological variables [ 18, 19, 20]. One such variable may be computer related anxiety and associated avoidance. Given the theoretical causal chain leading to avoidance involving computer anxiety, we also aimed to clarify those predictors of computer anxiety. A partial replication of a prior study [21] looking at key psychological predictors of computer anxiety in health care workers was attempted. In that study it was found that self-efficacy expectations was the best single predictor, accounting for 72% of the variance [21 ]. 2.
Method Subjects and Procedure. 197 Undergraduate university psychology students agreed to complete the questionnaire. 58% were in their first year of university study. Age ranged from 17 to 50 years, (M=21.0, SD=5.2), with a mode of 18 years. Sixty percent were female. Measures. The Computer Attitudes Scale (CAS) [ 14] is made up of three subscales which measure computer related anxiety (CAS-Anxiety), computer attitudes (CAS-Liking) and computer
583 self-efficacy (CAS-Confidence). Reliability coefficients and factor-analytic studies suggest each of the three subscales are sufficiently discrete to be used separately [22, 14]. All subscales have high internal reliability with coefficient alphas ranging from .86 to .91 [ 14]. In the present study similar levels of internal reliability were found with Cronbach alphas of .89 for CAS-Anxiety and .88 for both CAS-Liking and Confidence subscales. The CAS-Liking was used to assess computer attitudes and has been found to correlate with a variety of other computer attitude measures in the .80 to .89 range [23]. The CAS- Confidence subscale was used to assess self-efficacy. This was considered appropriate given the match between item content and the definition of the self-efficacy coostruct by Bandura [24]. The State-Trait Anxiety Inventory (STAI), [9] is one of the most extensively used and researched self-report measures of anxiety. The STAI-Trait has high internal reliability. A coefficient alpha of .91 in the present study is commensurate with Spielberger's [9] normative sample. In the present study a shortened 6-item version of the state anxiety scale [25] was used given the need to collect repeated measures of state anxiety. Spielberger [9] indicated that the state anxiety scale of the STAI may "be used to evaluate how they felt at a particular time in the recent past...in a specific situation...or in a variety of hypothetical situations" (p.2). Consequently, this measure was completed twice. The first version with the standard instructions to respond to the items as they feel "RIGHT NOW, that is AT THIS MOMENT", was considered the normal level of state anxiety and will be referred to as "STAI-norm". The second version instructed respondents to answer as they felt the "LAST TIME YOU USED A COMPUTER", and will be referred to as "STAI-compute". Since measures of computer avoidance have been limited [26] this construct was measured using a custom developed scale. This scale comprised seven items rated on a 4-point Likert-type scale. Subjects were instructed to rate the extent to which each item described their" behaviour during the present academic year". No prior psychometric evaluation on the scale had been conducted, but the Cronbach alpha coefficient of .85 in the present study suggested good internal reliability. Similarly, theoretically consistent correlations with other measures suggested some construct validity (see Table 1). The items were as follows: 1. I avoided taking courses that required using computers; 2. I avoided using computers in my daily activities; 3. I avoided learning about computers; 4. I put off working with computers; 5. I completed some things by hand rather than use a computer; 6. I avoided talking about computers and; 7. I avoided using computers to help with my course work. Subjects were asked two questions to determine computer experience. The first involved an estimation of the length of time they had been using computers as part of their daily activities, and the second an estimation of the number of hours of computer use in the previous. An index of computer experience comprised the product of the total number of months of computer use and the average number of hours of computer use. This represented the total average hours per month for each subject. 3.
Results
In order to determine the differential changes in state anxiety from the "normal" state to the computer situation, the difference between STAI-compute and STAI-norm was calculated. This variable is referred to as STAI-change and reflects changes in state anxiety from a "normal" resting state to the state when last using a computer. Subjects were divided into high and low trait anxious groups using a median split on the STAI-trait. Given that theory suggests that people who differ in trait anxiety only show corresponding differences in state anxiety in situations they perceive as threatening, high and low trait anxiety groups were also determined using a median split on CASanxiety. A between groups t-test was calculated between mean STAI-change scores for high and low trait anxious groups. No significant difference in STAI-change was found for high (M=-.15, SD=4.82) and low (M=.98, SD=3.98) trait anxious groups using the STAI-trait. However, when high and low situation specific trait anxiety (CAS-anxiety) was used, those high on the CAS-anxiety (M=-.85, SD=3.95) experienced significantly higher (t(196)=-4.2, I2<.001) changes in state anxiety than those in the low CAS-anxiety group (M= 1.70, SD=4.95). In order to clarify the theoretical relationships between computer anxiety and other relevant constructs a correlation matrix of the variables included in the study was constructed (see Table 1).
584 Table 1. Intercorrelation matrix of the variables examined in the present study. The diagonal represents the coefficient alpha, off dia~onal indicates the correlation, and probability (2 tail) of the correlation (N = 197). STAISTAICASCASSTAICASComputer Normal Trait Anxiety Confidence Compute Liking Avoidance STAI-Normal (.76) STAI-Trait .4428 (.91) P= .000 CAS-Anxiety -.1191 -.2601 (.89) P= .096 P= .000 CAS-Confidence -.0760 -.2550 .7817 (.88) P= .288 P= .000 P= .000 STAI-Compute .2023 .2810 -.4937 -.3967 (.84) P= .004 P= .000 P= .000 P= .000 CAS-Liking .0405 -.1205 .5787 .6594 -.2931 (.88) P= .572 P= .092 P= .000 P= .0o0 P= .000 Computer .0332 .1431 -.5397 -.4611 .3260 -.4986 (.81) avoidance P= .643 P= .045 P= .000 P= .000 P= .000 P= .000 Computer .0285 .0363 .2246 .3662 -.1471 .3210 -.1772 experiencea P= .697 P= .619 P= .002 P=. 000 P= .043 P- .00o P=.01 Relationships between the various anxiety measures and Computer Avoidance were examined, and all correlations were in the expected direction, with CAS-anxiety having the highest correlation with Computer Avoidance (r=-.54), suggesting that the higher a subject's computer anxiety the more likely they were to have avoided computer related activities over the previous year. This was followed by moderate correlations between CAS-Liking, CAS-Confidence, STAI-compute. While the correlations of avoidance with computer experience and trait anxiety reached significance they appeared relatively small in magnitude. There was no significant correlation between STAI-normal and avoidance. In order to clarify the relative importance of the correlates in predicting computer avoidance, a regression analysis was conducted. As there was no a priori ordering for the independent variables, a stepwise regression procedure was employed with Computer Avoidance as the dependant variable and STAI-Trait, STAI-Normal, STAI-Computer, CAS-Anxiety, CAS-Liking, CAS-Confidence and Computer Experience as the independent variables. Computer anxiety (CAS-Anxiety) was entered into the equation at step one and accounted for 29% of the variance in Computer Avoidance (R2 =.29, F(1,188)=77.13,12<.0001). Computer attitudes (CAS-Liking) was entered at step two and accounted for an additional 5% of the variance in Computer Avoidance (R2=.34, F(1,187) =48.81, I2<.0001). No other variables were accepted into the stepwise regression equation. Given the theoretical and practical importance of computer anxiety predicting avoidance, it was decided to test those factors which best predicted computer anxiety. While prior research suggested that self-efficacy would be the best predictor, a stepwise regression was again used since this was the analysis used by Henderson et al., [28]. Computer anxiety (CAS-anxiety) was the DV with CASConfidence, CAS-Liking, STAI-Normal, STAI-Computer, STAI-Trait, and Computer Experience as independent variables. Computer self-efficacy (CAS-Confidence) was entered into the equation at the first step and accounted for 62% of the variance in computer anxiety (R2=.62, F(1,188)=300.31, 12<.0001). Computer state anxiety (STAI-Computer) was entered second accounting for an additional 3 % of the variance in computer anxiety (R2=.65, F(1,187)=174.53, 12<.001). No other variables were entered into the equation.
4.
Discussion The results clarify and confirm the construct validity of computer anxiety. Individuals high in computer anxiety, as measured by the CAS-anxiety, reported significantly higher levels of state anxiety change when considering the last time they used a computer compared to those low in a Computer experience reliability not computed due to the nature of the scale.
585 computer anxiety. State-trait theory suggests that people high in trait anxiety react to stressors with higher levels of state anxiety than those low in trait anxiety. While this was not found for general trait anxiety (STAI-trait), it was found for computer related anxiety (CAS-anxiety). This too is theoretically consistent in that, whether or not individuals differ in trait anxiety depends on whether they perceive specific classes of situations as threatening. It appears that computer anxiety as measured by the CAS, constitutes a situationally specific (computer) measure of anxiety more similar to trait anxiety than state anxiety. This may explain inconsistencies in prior research which has failed to take into account the state-trait distinction, and the situational specificity of anxiety. In certain contexts it may even be fruitful to increase the situational specificity of the general computer anxiety measure. This is analogous to the development of more situationally specific test anxiety measures which assess specific test situations such as multiple choice, time limited, or essay tests [27, p .558]. The present finding that the avoidance measure was most highly correlated with computer anxiety (CAS-anxiety) has both theoretical and practical implications. Theoretically it provides preliminary evidence that computer anxiety may lead to computer avoidance. Practically, finding that the CAS-anxiety was the best predictor of avoidance suggests it may have utility as a screening measure. In large organisations where new information systems are being introduced, there is usually high demand for training. By identifying those most at risk of avoiding system use due to computer anxiety, specific training programmes can be developed to reduce these barriers [21 ]. It would be more efficient to provide appropriate interventions to those identified as at risk of experiencing computer anxiety than to provide a genetic training programme to all staff. The present study confirmed that self-efficacy expectations were the best predictor of computer anxiety. This is consistent with the findings of Henderson et al. [21 ] who found self-efficacy predicted computer anxiety in a sample of health care workers. While a number of interventions aimed at reducing computer anxiety have been developed [28, 29], these programmes may be improved by targeting changes in self-efficacy more specifically. Despite these theoretically consistent findings, a number of methodological limitations suggest the need for further research particularly in other samples. The measure of computer state anxiety was retrospective in nature, and future work would benefit from ratings during actual computer interactions. Similarly, additional measures of computer avoidance are needed. Indirect measures such as computer usage may provide additional information. This would ideally be more direct than traditional measures of computer experience which typically require respondents to estimate the time spent on computers. It is suggested that computer system implementations provide a naturalistic opportunity to conduct such research [21 ]. Prior to the implementation of a new system measures of computer anxiety (CAS) could help identify those who are most at risk of avoiding use of the system. In summary, the present study suggested the anxiety subscale of the CAS is consistent with a situation-specific measure of trait anxiety, and was related to self-reported computer avoidance. Computer anxiety as measured by the CAS was best predicted by self-efficacy and computer related state anxiety. References
1. LaLomia, M. J., & Sidowski, J. B. (1993). Measurements of computer anxiety: A review. International Journal of Human-Computer Interaction, 5, 239-266. 2. Dickson, G., Simmons, J., & Anderson, J. (1974). Behavioural reactions to the introduction of a management information system at the US Post Office: Some empirical observations. In D. Sanders (Ed.), Computers and Management (2nd Ed), NY: McGraw-Hill. 3. Custer, G. (1994, August). Trepidation slows adults making move to high-tech. APA Monitor, 25(8), 34. Weinberg, S. B., & Fuerst, M. L. (1984). Computerphobia: How to slay the dragon of computer fear. Wayne, PA: Banbury Books. 5. Harrington, K. V., McElroy, J. C., & Morrow, P. C. (1990). Computer anxiety and computer-based training: A laboratory experiment. Journal of Educational Computing Research, 6, 343358. .
586 6. Rosen, L. D., & Maguire, P. (1990). Myths and realities of computerphobia: A meta-analysis. Anxiety Research, 3, 175-191. 7. Rosen, L., Sears, D. C., & Weil, M. M. (1987). Computerphobia. Behaviour Research Methods, Instruments, & Computers, 19, 167-179. 8. Cattell, R. B., & Scheier, I. (1958). The nature of anxiety: A review of 13 multivariate analyses comprising 814 variables. Psychological Reports, 4, 351-388. 9. Spielberger, C. D. (1983). Manual for the State-Trait Anxiety Inventory, STAI (Form Y). Palo Alto, CA: Consulting Psychologists Press. 10.Cambre, M. A. & Cook, D. L. (1985). Computer anxiety: Definition, measurement, and correlates. Journal of Educational Computing Research, 1, 37-54. 11.Keman, M. C. & Howard, G. S. (1990). Computer anxiety and computer attitudes: An investigation of construct and predictive validity issues. Educational and Psychological Measurement, 50, 681-690. 12.Spielberger, C. D., Gorsuch, R. L., & Lushene, R. E. (1970). Manual for the State-Trait Anxiety Inventory (Self-Evaluation Questionnaire). Palo Alto, CA: Consulting Psychologists Press. 13.Glass, C. R. & Knight, L. A. (1988). Cognitive factors in computer anxiety. Cognitive Therapy and Research, 12, 351-366. 14.Loyd, B. H., & Gressard, C. P. (1984). Reliability and factorial validity of computer attitude scales. Educational and Psychological measurement, 44, 501-505. 15.Fox, E. (1990). The nature and treatment of situational anxiety: Dental surgery as a model. In N. McNaughton & G. Andrews (eds), Anxiety, pp. 136-148. Dunedin, NZ: University of Otago Press. 16.Carson, R.C., & Butcher, J.C. (1992). Abnormal Psychology and Modem Life. NY: Harper Collins Publishing. 17.Campbell, N. J. (1989). Computer anxiety of rural middle and secondary students. Journal of Educational Computing Research, 5, 213-220. 18.Ginzberg, M. J. (1981). Early diagnosis of MIS implementation failure: Promising results and unanswered questions. Management Science, 27, 459-478. 19.Jagodzinski, A.P., & Clarke, D.D. (1988). A multidimensional approach to the measurement of human computer performance. The Computer Journal, 31,409-419. 20.Zumd, R.W. (1979). Individual differences and MIS success: A review of the empirical literature. Management Science, 25, 966-979. 21.Henderson, R. D., Deane, F. P., & Ward, M. J. (in press). Occupational differences in computerrelated anxiety: Implications for the implementation of a computerised patient management information system. Behaviour and Information Technology. 22.Bandalos, D., & Benson, J. (1990). Testing the factor structure invariance of a computer attitude scale over two grouping conditions. Educational and Psychological Measurement, 50, 49-60. 23.Dukes, R. L., Discenza, R., & Couger, J. D. (1989). Convergent validity of four computer anxiety scales. Educational and Psychological Measurement, 49, 195-203. 24.Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioural change. Psychological Review, 84, 191-215. 25.Marteau, T. M., & Bekker, H. (1992). The development of the six-item short-form of the state scale of the Spielberger State-Trait Anxiety Inventory (STAI). British Journal of Clinical Psychology, 31, 301-306. 26.Henderson, R.D., Deane, F.P., Barrelle, K. and Mahar, D. (in press). Computer anxiety: Correlates, norms, problem definition in health care and banking employees using the Computer Attitudes Scale. Interacting with Computers. 27.Anastasi, A. (1988). Psychological Testing (6th Ed). N.Y.: MacMillan. 28.Bloom, A. J., & Hautaluoma, J. E. (1990). Anxiety Management Training as a strategy for enhancing computer user performance. Computers in Human Behaviour, 6, 337-349. 29.Weil, M. W., Rosen, L. D., & Sears, D. C. (1987). The Computerphobia Reduction Program: Year 1. Program development and preliminary results. Behaviour Research Methods, Instruments, & Computers, 19, 180-184.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
587
Sick b u i l d i n g syndrome: are UK libraries affected? A. Morris and P. Dennison Department of Information and Library Studies, Loughborough University of Technology Loughborough, Leicestershire L E l l 3TU, United Kingdom
Abstract Sick building syndrome is characterised by occupants of a particular building suffering from a range of symptoms which may include eye, nose and throat irritations, dry skin, headaches, coughs, wheezing, nausea, dizziness, lethargy, hypersensitivity and erythema. This paper discusses the results of a survey which examined the evidence and possible causes of sick building syndrome in libraries. The findings suggests that sick building syndrome exists in libraries and that airconditioned libraries are more likely to be affected than those that are naturally ventilated.
1. I N T R O D U C T I O N Building sickness is "recognised by the World Health Organisation as a syndrome of complaints covering non-specific feelings of malaise, the onset of which is associated with occupancy of certain modern buildings". (1) In 1983 the World Health Organisation listed the symptoms characterising sick building syndrome. They include eye, nose and throat irritations, dry skin, headaches, coughs, wheezing, nausea, dizziness, lethargy, hypersensitivity and erythema.(2) All of these symptoms are commonplace in the general population but it is the pattern of their expression that points to the diagnosis of sick building syndrome. In sick building syndrome symptoms are associated with being in a particular building and are relieved by leaving or staying away from that building. Research investigating sick building syndrome in offices indicates that: air-conditioned buildings are likely to be more affected than naturally ventilated buildings (3) •
women are more prone to the syndrome than men (4) clerical workers are more prone to the syndrome than professional or managerial staff (4) workers who perceive their workplace to be hot and stuffy and the air to be humid and stale report more health problems (5).
588 Pinpointing specific causes of building sickness is difficult because the symptoms reported are often non-specific and diverse, not all employees in a 'sick' building are affected, and causes appear to vary from building to building. Nevertheless, it is generally thought that the main cause of sick building syndrome is physical, and it is probably caused by the presence of air-borne volatile organic compounds plus interactions with other physical factors such as ventilation, humidity and temperature (6). Psychological factors such as stress may also contribute to the perception of a building being 'sick'. Most of the research to-date has been carried out on office buildings and office workers. No research known to the authors has been undertaken to establish whether sick building syndrome also affects libraries. The research summarised here aims to address this issue.
2. M E T H O D O L O G Y A questionnaire was distributed to the managers of all 147 academic libraries in the United Kingdom. In addition, the questionnaire was sent to the manager in a sample of 79 public libraries. The questionnaire was divided into four main sections: general information; library building; library environment, and staff health. Space for other comments was also available. The general information section asked basic questions about the type of library and the number of full-time and part-time female and male staff. The library building section contained questions about the age of the building, whether it was air-conditioned and, if so, if the system was maintained, whether windows could be opened, the type of lighting and if staff had control of the lighting levels. Questions in the library environment section asked the librarians to rate air temperature, freshness and humidity and also environmental comfort. In addition they were asked if their staff experienced draughts and if they had control over the air temperature and air-conditioning levels in the library. The remaining questions, in the section on staff health, asked for information about absenteeism rates, if staff lei~ work early because of illness and if staff complained about a number of symptoms including eye, nose, skin and throat problems, lethargy, headaches, flu-like symptoms, breathing problems, nausea or reduced memory. The general area being examined was referred to both on the questionnaire and in the accompanying covering letter as 'environmental conditions in libraries' rather than 'sick building syndrome'. It was felt that the latter term might possibly alarm library managers and compromise the number of respondents due to the fear of their library building being labelled as 'sick'.
589
3. R E S U L T S The most important results are briefly summarised below:
3.1 General information the overall response rate was 69%; 114 from academic libraries and 43 from public libraries no significant differences were found between the responses of managers of public and academic libraries 3.2 T y p e of l i b r a r y building •
24% of all the libraries were air-conditioned
•
68% were built between the late '60s and early '80s.
3.3 Library environment there was no difference in the perception of air temperature and air humidity between managers in air-conditioned and naturally ventilated libraries managers in naturally ventilated libraries rated their environment higher (more comfortable) and their air to be fresher than those working in airconditioned libraries 58% of managers thought their library was draughty. Surprisingly, managers of nine libraries reported that staff complained of draughts from air vents in the air-conditioning system. •
all but two of the libraries had fluorescent lighting
•
82% of libraries had windows which could be opened managers at those libraries where windows could not be opened reported that staffhad significantly more throat problems and general lethargy (sig. at p=0.05) staff could control the lighting in most libraries, however, lethargy and headaches were more frequently experienced by staff working in libraries with fixed lighting levels (sig. at p=0.01) and p=0.05 respectively) 71% of managers reported that their staff did not have any control of the level of air temperature in their libraries. The staff working in these libraries also experienced significantly more headaches, throat problems,
590 lethargy and eye problems (all at p=0.05 or above) than those working in libraries where staff could control the air temperature. 3.4 S t a f f h e a l t h
managers reported a problem with staff experiencing the following: headaches -51%, lethargy- 45%, eye problems - 34%, throat problems 32%, and nose problems - 22% headaches, nose problems, throat problems, eye problems and lethargy were significantly more prevalent in air-conditioned libraries (at p=0.05 or above) 40% of staff in air-conditioned libraries had more than six days absence per year compared with 22% in naturally ventilated libraries
4. C O N C L U S I O N S The results clearly supports previous research that suggests air-conditioned buildings are more likely to suffer from sick building syndrome than those that are naturally ventilated. Staff in air-conditioned libraries experienced more health problems, had more days off sick and rated their environment more stale than staff working in naturally ventilated libraries. The research has also confirmed that psychological factors, such as allowing staff control over environmental levels, may also contribute to the perception of a building being 'sick'. Significantly more h~ealth problems were experienced by staff working in libraries where they did not have control over the openir~g of windows and level of lighting and air temperatu~re. No conclusions can be drawn from this study about the possible effects fluorescent lighting may have on staff because nearly all libraries had this type of lighting. It is clear from the results and comments made on the questionnaire that, like offices, libraries are affected by sick building syndrome. The scale of the problem is hard to estimate, but it would appear that many people are experiencing health problems and working in unsatisfactory environmental conditions in libraries. The authors therefore agree with Finnegan et al who state: ...although the symptoms of the sick building syndrome do not represent a disease but rather a reaction to the working environment, the scale of the problem is probably considerable, and the high degree of dissatisfaction seen...demands attention from architects, engineers, and the medical profession. In particular, more research is needed, preferably of a longitudinal nature, into both air-conditioned and naturally ventilated buildings (3).
591
REFERENCES 1. Wilson, S. and A. Hedge. The office environment survey: a study of building sickness. London: Building Use Studies Ltd, 1987. 2. World Health Organisation Indoor air pollutants: exposure and health effects. World Health Organisation EURO Reports and Studies 78. Copenhagen: World Health Organisation, 1983. 3. Finnegan, M.J. et al. The sick building syndrome: prevalence studies. British Medical Journal, 8 December 1984, 1573-1575. 4. Vince, I. Sick building syndrome. London: IBC Technical Services Ltd, 1987. 5. Hedge, A. et al. Work-related illness in offices: a proposed model of the "sick building syndrome". Environment International, 1989, 15, 143-158. 6. Lundin, L. On building-related causes of the sick building syndrome. Stockholm: Almqvist and Wiksell, 1991.
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Moil (Editors) © 1995 Elsevier Science B.V. All rights reserved.
593
H e a d - c o u p l e d d i s p l a y s y s t e m s - R e s e a r c h i s s u e s on h e a l t h a s p e c t s Wolfgang Felger Fraunhofer-Institut fiir Graphische Datenverarbeitung (IGD) WilhelminenstraBe 7, D-64283 Darmstadt, Germany Phone: +49-6151-155-122, Fax: +49-6151-155-399, E-mail: [email protected] ABSTRACT Head-coupled display systems are in widespread use for virtual reality applications. This contribution introduces some technical aspects of head-coupled display systems, as well as discussing pertinent health concerns. The latter are discussed in an effort to motivate researchers conducting corresponding health studies. Finally, ten research issues on health aspects are identified. A comprehensive reference list is provided.
Keywords: 1.
head-coupled system, health aspects, HMD, 3D presentation device, virtual reality
INTRODUCTION
In concert with the great attention afforded to the field of virtual reality (VR), head-coupled display systems are becoming increasingly popular. This popularity has lead to a great variety of different head-coupled display systems. Only high system prices have restricted their wider distribution, beyond the academic and industrial research labs. Within the last few months, however, systems costing less than US$ 600 have become available. The affordability of such systems, to the mass market, may lead to a situation where they are found in everybody's work environment or home. In 1994, the Fraunhofer-IGD carried out a comprehensive market survey of head-coupled display systems [Felg-94]: commercial and research systems, as well as system components, were identified. Further to this survey, an extensive literature investigation has been performed. Surprisingly, almost no work has been found which relates health aspects to system usage. The goal of this paper is to motivate research into the possible health impact that head-coupled display systems may have on their users. This will be very important if 'careless system usage' is to be quantified and for a system's acceptance to a real VR application. Corresponding research issues are identified. The paper is structured as follows: Section 2 briefly introduces the idea of a head-coupled system, and section 3 explains main system components. Related human factors are summarized in section 4, whilst section 5 sets these in relationship to the current technology and health concerns of users. The paper concludes with an outline of open research issues relating to the health aspects of head-coupled systems usage. 2.
WHAT IS A HEAD-COUPLED DISPLAY SYSTEM ?
The idea for a head-coupled system is 30 (!) years old, having originally been proposed by Ivan Sutherland in 1965 [Suth-65]. Important developments have
594 been carried out by the military, in the 1970's [Furn-91], and by NASA Ames in the 1980's [FMHR-86]. The first commercialsystem (VPL EyePhone [Teit-90]) became available in 1989. A head-coupled display system is the classical 3D presentation device for virtual reality. It enables a user to feel immersed in a computer-generated environment. Usually, a head-coupled system presents a binocular image (a stereoscopic image with different images for the left and right eyes) to the observer: two monitors display the image pair to the eyes. Some monoscopic display systems also exist. In order to provide a wide field of view, a special optical system is applied. Most systems are directly mounted on the head (headmounted display, HMD) in the form of a helmet or goggles [CHBF-89]; others need a special mounting (e.g., BOOM [MBPF-90]). There are two main categories of head-coupled display systems: opaque systems, and see-through systems: Opaque systems block the user's view of the outside world, allowing them to concentrate on the virtual world. See-through systems superimpose computer-generated environments onto the user's real surroundings. Figure 1 shows one opaque system (left side: "Datavisor' from n-Vision) and one see-through system (right side: "i-glasses!' from Virtual I/O). Surveys of currently available systems can be found from time to time in various VR newsletters (e.g., [RTG-94]).
....i
Fig. 1: Head-coupled systems
3.
SYSTEM COMPONENTS
A head-coupled display system requires the integration of several major components [KoTa-94]" typical systems will integrate graphics displays and optical systems with devices to track head movements.
3.1.
Tracking systems Tracking systems perform head tracking (they measure the position and orientation of the head). Electromagnetic, acoustic, mechanical, optical, and inertial tracking techniques are all in general use [MeAB-92]. General purpose applications typically find the accuracy delivered by electromagnetic techniques sufficient, whilst special purpose set-ups may opt for mechanical or optical techniques. Eye tracking is a technique to provide information about the user's gaze direction, in order to detect the region of the view they find of particular interest. Accurate eye tracking is achieved using special cameras, which measure infrared light reflected off the retina.
595
3.2.
Display systems
3.3.
Optical systems
The display systems present the visual information to the user: a colour display is preferred. Display technologies include liquid crystal displays (LCD), cathode ray tubes (CRT), and fiber optics [Bola-94]. A first prototype exists, using laser technology, to scan an image directly onto the human retina [TJMF-95]. The various technologies have differing technical specifications and costs. LCDs are very economic and light-weight, but suffer from poor resolution. Typically, 150,000 primary pixels represent approximately a display resolution of 258 x 195 (horizontal x vertical). Further drawbacks are slow response times and poor contrast ratios. Best resolution (e.g., 1,280 x 960) and display characteristics can currently be achieved with CRTs, but these have the drawbacks of high price, a certain weight, and strong, high-frequency, electromagnetic emissions. Faced with these disadvantages, a few head-coupled systems physically decouple the image creation and image presentation units, using fiber optics to transport the image between both components. Optical systems typically integrate a number of lens elements. Using mirrors and/or optical lenses, the image can be magnified and a large field of view is provided [MaHH-93]. Lenses in use include: relay lenses (which relay an image to another location to increase the viewing distance), eye-piece lenses (located close to the eye and serving as simple magnifiers), and field lenses (located near the image plane and increasing the field of view) [KoTa-94]. Other elements, such as combiners or beamsplitters, have been applied to enable see-through capability. The applied optical system has an impact on image quality, focal distance, field of view, exit pupil (formed when using relay optics to produce an intermediate image plane), eye relief (distance between the last optical element and the eye entrance pupil), and also on other parameters. 4.
HUMAN FACTORS
A head-coupled system should imitate the vision characteristics of real life. In any case, their use should not impair the user's senses or body. The following main aspects have to be taken into account in the design phase [McZe-92, VeilS94]: a) Binocular vision: Depth perception enabled by binocular/stereoscopic vision should be provided. Human depth resolution is 0.47 mm. b) Field of view: The field of view of the human vision system is 180 degrees horizontal and 120 degrees vertical. The natural binocular overlap is 120 degrees. c) Resolution: The human eye is capable of discerning an element size of about 0.5 minutes of arc (this corresponds to a spatial resolution of 4,800 x 3,800). d) Focus: When viewing an object in nature, the human eyes accomodate (focus), and converge to the same point in space. With this feature, objects in a distance range from infinity to near, can all be viewed comfortably. Further characteristics to consider are: colour, brightness, contrast, and freedom from distortion and aberrations. 5.
T E C H N O L O G Y CONSTRAINTS AND HEALTH IMPACT
Besides the ergonomic aspects of head-coupled systems (such as weight, easeof-use, etc.) the health aspects of system usage are also of great importance. These aspects strongly correlate to the technology in use. By their very nature,
596 the components which make up a head-coupled system are all located very close to the human head and brain. Moreover, hygiene issues (such as transferring skin or hair infections) should not be forgetten, especially if a head-coupled system is in public use. 5.1.
Previous work Unfortunately, very little work in this area has been made openly available. The author assumes that some non-disclosed research findings (relating to military applications) must exist, because the Apache helicopter has the only operational HMD (Integrated Helmet and Display Sighting System) in service today [NeHa-94]. Adjusting a head-coupled system to individual settings can be a non-trivial task. [DeEA-94] reported that USAF aircrew members cannot obtain optimal performance from night vision goggles (NVGs), without resorting to a standard procedure (NVGs are not normally considered to be HMDs but they share many of the issues and problems). [KVBM-94] conducted a study on luning when using an HMD with partial binocular overlap displays. Luning refers to the subjective darkening that can occur in the monocular regions near the binocular overlap borders. Luning can result in fragmentation of the field of view into three phenomenally distinct regions. [KoMM-94] investigated how accomodation can be adversely affected by the use of narrowband phosphors in HMDs under dynamic conditions; i.e., the observer might accomodate inaccurately to the display if frequent changes in focus to and from the display are required, as may well be the case with see-through systems. [Kell-94] investigated depth perception with HMDs, and reported that a few subjects suffered from headaches and dizziness after the experiment. Other subjects reported perceiving double images, from time to time, during the experiment. [MoWR-93] reported on the short-term effects, of wearing a conventional HMD, on binocular stability. Subjects were examined before and after exposure to the HMD and there were clear signs of induced binocular stress for a number of subjects. [Pian-93] points out that the distance between the viewer's eyes, relative to the distance between the lenses in the HMD, can affect the visual function of most viewers. If they are improperly adjusted, optical distortion may be introduced, and subjects could become slightly esophoric, or cross-eyed, when wearing the HMD. 5.2.
O b s e r v a t i o n s at IGD Working in the field of VR since the early 1990s at IGD, in Darmstadt, we have collected a variety of observations and concerns voiced by visitors, our students, and colleagues. The most popular tracking technology uses electromagnetic fields. These are low energy fields, but the same concerns which relate to cellular phones, or high voltage cables, apply here. Furthermore, system latency and accuracy can lead to unnatural user behaviour (e.g., the user will not move their head quickly). LCD technology still has a poor resolution (e.g., 50,000 color pixels). Refresh rates and contrast need to be improved. CRTs are doing better here, but need a very high voltage to drive the cathode ray. The optical system influence the observation process. Some systems force the user to focus on a close distance, others at infinity. One head-coupled system achieves see-through capability by using one display monitor for the dominant eye, whilst allowing the other eye a clear view of the surroundings. The technology constraints just described may have an impact on human health, when users are exposed to such systems for a longer period (say, a few hours). Today, the use of such systems in research labs is mostly limited to a few
597 minutes. However, with the appearance of industrial virtual reality applications and head-coupled display systems on the consumer market, an extented usage is obvious. Large e n t e r t a i n m e n t and cable television companies have already announced their intent to support such systems. This demonstrates clearly the need for investigations on the related health aspects. 6.
RESEARCH ISSUES
The goal of this paper was to report on head-coupled systems from an engineers point of view, in order to motivate perception scientists, psychologists, and others, to perform studies on the health aspects of HMD usage~ The author is confident t h a t within the VR community, such information is eagerly sought. To conclude, the author has identified ten research issues which relate to the health aspects of HMD usage. These issues are outlined below, and can be categorised as relating to the display component (issues 1-3), the optical system (issues 4-7), and more general aspects (issues 8-10). Aspects about simulator or motion sickness are omitted, because such issues are already been well-addressed elsewhere (e.g., [Pres-92]). 1) What effects can be caused by the electromagnetic radiation of CRTs and tracking systems, used with HMDs? 2) Does a poor display resolution affect the h u m a n vision system? 3) What problems arise when the contrast and brightness is not high enough? 4) What effects can be caused by a not well-engineered optical system? How can such a bad system be identified by non-experts? 5) Can problems arise when the field of view is too narrow? Is there a lower limit? 6) Does the conflict with binocular systems -- the u n n a t u r a l behavior t h a t a viewer's eyes focus/accomodate on the screen but converge according the stereo cue -- affect the h u m a n vision system? 7) Do different effects exist for see-through systems and opaque systems, or are the problems similar? 8) What are the impacts of mid-term and long-term HMD usage (e.g., between 30 minutes and 3 hours)? 9) What individual adjustment capabilities should an HMD provide? How can a system be perfectly calibrated to the user? 10) Can the use of an opaque HMD for a certain time lead to spatial disorientation after use? Learning more about all of these issues, would enable an informed discussion on the drawbacks, and benefits, of VR applications, as well as allowing one to quantify any potential for personal risk. ACKNOWLEDGEMENTS I would like to t h a n k the Fraunhofer Society for the support in establishing the VR Demonstration Center. Special thanks to Christine Giger and Neil Gatenby for proof-reading. They are responsible for much of the readability and none of the faults. REFERENCES [Bola-94]
Bolas, M.T.: "Human Factors in the Design of an Immersive Display", in: IEEE Computer Graphics & Applications, January 1994, pp. 55-59 [CHBF-89] Chung, J.C.; Harris, M.R.; Brooks, F.P.; Fuchs, H.; Kelley, M.T.; Hughes, J.; Ouhyoung, M.; Cheung, C.; Holloway, R.L.; Pique, M.: "Exploring virtual worlds with
598 head-mounted displays", in: Proc. SPIE Vol. 1083, Three-Dimensional Visualization and Display Technology, 1989, pp. 42-52 [DeEA-94] ....DeVilbiss, C.A.; Ercoline, W.R.; Antonio, J.C.: "Visual performance with night vision goggles (NVGs) measured in USAF aircrew members", in: Proc. Helmet-and HeadMounted Displays and Symbology Design Requirements, April 5-7, 1994, Orlando (FL, USA), SPIE Vol. 2218, pp. 64-70 [Felg-94] Felger, W.: "Marktsichtung fiir kopfgebundene Darstellungssysteme", FraunhoferIGD, confidential report (in German), March 1994 J [FMHR-86] Fisher, S.S.; McGreevy, M.; Humphries, J.; Robinett, W.: "Virtual Environment Display System", in: ACM Proc. 1986 Workshop on Interactive 3D Graphics, Oct. 1986, pp. 77-87 [Furn-91] Furness III, T.A.: "Virtual Interface Technology (Virtual Reality)", Siggraph course notes #C3, 1991 [Kell-94] Kelle, O.: "Untersuchung zur Importanz dynamischer Tiefenwahrnehmungskriterien in computergenerierten interaktiven Szenen und virtuellen Environments", Dissertation, FB Sicherheitstechnik, Bergische Universit/~t-GesamthochschuleWuppertal, 1994, (in German) [KoMM-94] Kotulak, J.C.; Morse, S.E.; McLean, W.E.: "Does display phosphor bandwith affect the ability of the eye to focus?", in: Proc. Helmet- and Head-Mounted Displays and Symbology Design Requirements, April 5-7, 1994, Orlando (FL, USA), SPIE Vol. 2218, pp. 97-104 [KoTa-94] Kocian, D.; Task, H.L.: "Design and Integration Issues of Helmet-Mounted Displays", Short course notes SC05, SPIE International Symposium on Optical Engineering in Aerospace Sensing, Orlando (FL, USA), April 1994 [KVBM-94] Klymenko, V.; Verona, R.W.; Beasley, H.H.; Martin, J.S.: "Convergent and Divergent Viewing affect Luning, Visual Thresholds and Field-of-View Fragmentation in Partial Binocular Overlap Helmet Mounted Displays", in: Proc. Helmet- and Head-Mounted Displays and Symbology Design Requirements, April 5-7, 1994, Orlando (FL, USA), SPIE Vol. 2218, pp. 82-96 [MaHH-93] Ma, J.; Hollerbach, J.M.; Hunter, I.W.: "Optical Design for a Head-Mounted Display", in: Presence, Vol. 2, No. 3, Summer 1993, pp. 185-202 [MBPF-90] McDowall, I.E.; Bolas, M.; Pieper, S.; Fisher, S.S.; Humphries, J.: "Implementation and integration of a counterbalanced CRT-based stereoscopic display for interactive viewpoint control in virtual environment applications", in: Proc. SPIE Vol. 1256, 1990, pp. 136-146 [McZe-92] McKenna, M.; Zeltzer, D.: "Three Dimensional Visual Display Systems for Virtual Environments", in: Presence, Vol. 1, No. 4, Fall 1992, pp. 421-458 [MeAB-92] Meyer, K.; Applewhite, H.L.; Biocca, F.A.: "A Survey of Position Trackers", in: Presence, Vol. 1, No. 2, Spring 1992, pp. 173-200 [MoWR-93] Mon-Williams, M.; Wann, J.P.; Rushton, S.: "Binocular vision in a virtual world: visual deficits following the wearing of a head-mounted display", in: Ophthalmic & Physiological Optics, Vol. 13, No. 4, Oct. 1993, pp. 387-391 [NeHa-94] Newman, R.L.; Haworth, L.A.: "Helmet-Mounted Display Requirements: Just Another HUD or a Different Animal Altogether?", in: Proc. Helmet- and HeadMounted Displays and Symbology Design Requirements, April 5-7, 1994, Orlando (FL, USA), SPIE Vol. 2218, pp. 226-237 [Pian-93] Piantanida, T.: "Another Look at HMD Safety", in: CyberEdge Journal, Vol. 3, No. 6, Nov./Dec. 1993, pp. 9-12 [Pres-92] PRESENCE: Teleoperators and Virtual Environments, MIT Press, Vol. 1, No. 3, Summer 1992 [RTG-94] Real Time Graphics: "Head Mounted Display Survey- A comprehensive Round-Up of Products", RTG, Vol. 3, No. 2, Aug. 1994, CGSD Corp., Mountain View (CA, USA), pp. 8-12 [Suth-65] Sutherland, I.E.: "The ultimate display", in: Proc. of the IFIP Congress 2, 1965, pp. 506-508 [Teit-90] Teitel, M.A.: "The EyePhone, a head-mounted stereo display", in: in: Merritt, J.O.; Fisher, S.S. (Eds.): Stereoscopic Displays and Applications, Proc. SPIE 1256, 1990, pp. 168-171 [TJMF-95] Tidwell, M.; Johnston, R.S.; Melville, D.; Furness III, T.A.: "The Virtual Retinal Display- A Retinal Scanning Imaging System", in: Proc. of Virtual Reality World '95, Stuttgart, Feb. 21-23, 1995, Computerwoche Verlag, Munich, 1995, pp. 325-333 [VeilS-94] Veron, H.; Hezel, P.J.; Southard, D.A.: "Head Mounted Displays for Virtual Reality", in: Proc. Helmet- and Head-Mounted Displays and Symbology Design Requirements, April 5-7, 1994, Orlando (FL, USA), SPIE Vol. 2218, pp. 41-50
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
599
E s t a b l i s h m e n t of a n E x p e r t S y s t e m for V i s u a l D i s p l a y T e r m i n a l s (VDT) W o r k e r s ' Periodic E y e C h e c k u p s Hitoshi Nakaishi a and Masaru Miyao b
a Dept. of Hygiene & Preventive Medicine, Faculty of Medicine, Kanazawa Medical University, 1-1 Daigaku, Uchinada, Ishikawa 920-02, J a p a n b Dept. of Public Health, Faculty of Medicine, Nagoya University, Tsurumai 65, Showa-ku, Nagoya 466, J a p a n
With the increased use of visual display terminals (VDT) in almost every type of workplace in Japan, there have been increased instances of workers' eyestrain symptoms through viewing objects at relatively near distances. To tackle this problem, we established an "expert system" for analyzing the optimal visual conditions for VDT workers. Utilizing our system, eyestrain symptoms in workers during VDT operation can be greatly reduced, allowing for an improvement in the quality of working life.
1. I N T R O D U C T I O N
Since the technology of VDT or visual display terminals has been proliferating very rapidly, an ordinance regulating potential health risks using VDT has been enforced by the Ministry of Labor in J a p a n since 1985 [1]. In the ordinance, periodic physical checkups or medical examinations were also prescribed to be carried out at least once per year. The validity and reliability of these examinations, especially those testing eyestrain symptoms, were investigated by the J a p a n Association of Industrial Health, and a revised combination of visual function tests for VDT workers was reported in 1990 (Table 1)[2]. The latter emphasizes the importance of refraction test, since in performing VDT work, there exist several differences from conventional desk work in addition to viewing objects from a relatively close distance, for instance (1) intensive viewing of a light source itself, unlike the light reflected from a printed page; (2) changes in posture required during work, from a bent posture to a nearly vertical posture of the upper body, and the corresponding large change in the line of sight from a downward to a nearly
600 Table i Visual Function Tests Recommended by the Japan Association of Industrial Health (1990; issued in 1992) 1. Visual Acuity Test at 5 nd 50 cm/33 cm; right/left/binocular with/without correction with hypermetropia screening (Donders method) 2. Refractometry with Lensmetry for those wearing glasses 3. Astigmatism (Subjective) screening 4. Accommodative Near Point Assessment 5. Phoria Test 6. Stereoscopic Vision Test 7. Other Tests when regarded necessary by an industrial doctor horizontal view; and (3) an extension of the distance to a viewed object from an average 30 cm distance for a page, to 40-70 cm for a CRT (Cathode Ray Tube). We as members of the VDT Work Research Group in the J a p a n Association of Industrial Health have reported that VDT workers' visual correction (glasses and contact lenses) is often inappropriate for their working distance [3],[4]. Such problems in visual correction among VDT workers may account for the asthenopic symptoms seen in them. Consequently, the determination of visual correction for VDT workers should be based on an actual measurement of a person's near visual function. Improving near visual Correction would not only alleviate their workrelated needs, but would also improve the quality of their working life. Up to now, however, health examinations have rarely been performed in practice due to the inadequate ophthalmologic knowledge of many industrial health practitioners. An accurate and simple "expert system" able to perform the near visual function screening requested for VDT workers could alter this situation, greatly advancing the health management ofVDT workers.
2. M E T H O D S
The expert system we developed is designed to analyze optimal near visual function for VDT workers on an individual basis. The system consists of the following three screening instruments and a computer to analyze the data collected. (1) A visual acuity analyzer able to assess phoria status, stereo vision, and visual acuity at 3 distances (33 cm, the optimal distance for conventional desk work; 50 cm, the optimal distance for VDT work; and 5 meters, the distance evaluated in a regular health examination) (Figure 1). (2) An autorefractometer, assessing refractive status through detection of myopia, hypermetropia, astigmatism, or anisometropia, and their severity (Figure 2). A lensmeter is also included for wearers of corrective glasses. (3) An autoaccommodometer, measuring accommodative power and the
601 accommodative near point (Figure 3). Each of these instruments has been designed to allow even a novice operator to easily obtain data of high reproducibility. A detailed discussion of technical characteristics m a y be found in the literature [5],[6]. The data obtained from each instrument are automatically recorded on an IC card and the complete data for an individual is processed on a personal computer by a prediagnostic software system which we have also developed. The overall s t a t u s of visual function for a subject during VDT work is then shown via a simple diagram and printed out (Figure 4). The difficulties for physicians making a final determination or diagnosis during a physical examination (particularly nonophthalmologists) can be greatly alleviated by reference to these printed data.
3. R E S U L T S
Considerable innovations were applied to the development of the screening instruments to ensure technical ease for new users as well as test results of high reproducibility. One of these is the movement of optotypes within the autoaccommodometer at a constant diopter rather than a constant speed, making subject tracking easier and greatly improving data reproducibility ("diopter [D]" is an optical concept indicating the strength of a lens as the reciprocal of its focal length in meters). Another is the internal illumination of the visual acuity analyzer and the autoaccommodometer. These allow measurement and screening in an open environment more closely approaching natural visual conditions than the atypical environment of conventional screening devices, where subjects peer at internal optotypes. A considerable reduction in the biases accompanying subject screening can be expected from these innovations [7]. In addition, the screening results obtained from the instruments in our expert s y s t e m incorporate all the items set forth in the 1985 Ministry of Labor bulletin [1]and revisions thereto recommended by the J a p a n Association of Industrial Health in 1990 (issued in 1992 [2]). These criteria for visual function screening, in particularly the latter criteria of the Association, are scientifically seen as highly desirable for VDT workers [3,4]. However, they have not enjoyed widespread use due to the difficulties for the average industrial health practitioner in testing and assessing refraction and accommodation, the essence of the criteria. This is also the main reason why a certain industrial health organization does not conduct medical examinations for VDT operators. However, use of the expert s y s t e m (including the prediagnostic software system) we have developed allows industrial health practitioners to obtain valid results in each test without special technical training, and likewise removes major obstacles to the a s s e s s m e n t of the data and diagnosis. Since near visual function of VDT workers can be now evaluated with great ease, the government and the scientific community's demand for better VDT worker health m a n a g e m e n t m a y finally be implemented by industrial health care practitioners throughout Japan.
602 Figure 1. A Visual Acuity Analyzer
i
• i!i¸ii!ii!iii!i!ii¸i!l!!!i~I!iliili!l!ili!ili iii~ii!i!iill!i| ii~ii!l
i!ii
iiiiiiiiiiiii i~Ii:
i
............
.....
i
i!!
Figure 2. An Autorefractometer
i¸¸:¸i¸¸!¸~i
603 Figure 3. An Autoaccommodometer
Figure 4. Examinee's near visual s t a t u s print outed i~ : llll I 1.2 1.2
CYL AXIS
t,tU
U
0.25
0. O0
12.80
15o¢,) ] • SPII I
CYL I -0.50 AXISI
69
-0.25 120
T
12.20
10.80
604 4. CONCLUSIONS The use of our expert system can clearly alert Japanese industrial health care practitioners to the problems in VDT work, particularly in using reading glasses heretofore uniformly prescribed on the basis of chronological age and refraction status. Thus problem of eyestrain or poor near visual acuity among VDT workers can be greatly redressed, allowing an improvement in the quality of working life and daily life for a large number of these workers.
REFERENCES
1. Ministry of Labor in Japan,Guideline for VDT work in view of industrial health. Bulletin 705 (1985) in Japanese. 2. VDT Work Research Group of J a p a n Association of Industrial Health, Jpn. J. Ind. Health, 34 (1992) 80. 3. H. Nakaishi (ed.), Proceedings of the 10th Annual Conference on VDT work, (F) J a p a n Medical Association, Tokyo, 1990. 4. H. Nakaishi and O. Wada, J. Tokyo Ophthalmologists Assoc., 138 (1992) 9. 5. H. Nakaishi, In: Textbook of administration seminar, M. Tachi (ed.), J a p a n International Corporation Association (JICA), Tokyo, 1990, pp.35-54. 6. H. Nakaishi, Safety Science (1995), in press. 7. H.Nakaishi, In: The Paths to Productive Aging, M. Kumashiro (ed.), Taylor & Francis Ltd, Hampshire, 1995, in press.
IV.2 Workstation and Work Environments
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
607
O c u l a r motility o f 7 2 . 0 0 0 V D U operators. Bruno Bagolinia, Fernando Molle ~, Marco Turbati b, Domenico Lepore ~, Luigi Scullica~. aDepartment of Ophthalmology Chairman: Prof. L. Scullica Catholic University of Sacred Heart- Lgo F.Vito 1, 0068 Rome - Italy bASSILS
-
Via Bellotti Bon 14, Rome - Italy
Although visual disturbances related to the use of VDUs are far from being resolved, the present study will allow to evaluate more precisely the relationship between VDU work, some of the symptoms most frequently reported during VDU use and the eye condition. The epidemiological value of the data obtained in a study of this size (73.000) in evaluating the prevalence of some ocular pathology, is obviously enormous. 1.INTRODUCTION In 1989 the ASSILS (Association for Additional Health Insurance of Telecom workers) commissioned a preventive campaign to some Italian ophthalmological clinics. It involved 73.000 employees of Telecom Italy, 30.000 of which had been already examined in a previous study (1986-89). The adoption of an examination protocol, which was the same for all working places, and of a computerized data collection system made it possible to obtain information of relevant epidemiological value and, in particular, to compare the results with that of the previous campaign. In this framework the Department of Ophthalmology of the Catholic University of Rome had the task to study the ocular motility condition of the above mentioned Telecom employees. Oculomotor apparatus is essential for visual comfort and, if altered, can cause tearing, eye burning, eye heaviness and blurred vision(1). 2. METHODS Evaluation of ocular motility was performed with the cover test for distance and near, measuring possible deviation. Particular attention was paid to the standardization of environmental parameters during the examination (working distance, environmental brightness, ect.) in order to have fully homogeneous detection conditions in the visiting place all over Italy. Unfortunately the data collected in the latter investigation are only partially comparable to those of the former one, owing to the different phoria detection method. In particular ocular deviations have been measured by prisms; the Maddox-Wing test, has been used only in a smaller sample to allow comparisons with the data collected during 1986-89 investigation, when the prismatic measurement was not performed. Seven classes of ocular motility have been identified(2): ortophoria (no latent ocular deviation); esophoria (latent deviation greater than +2 A); exophoria (latent deviation greater than -2 A); other types of phoria (latent vertical deviation); esotropia (manifest convergent deviation; suppression, diplopia or anomalous fusion assessed by the Worth test); exotropia (manifest divergent deviation; suppression, diplopia or anomalous fusion
608 assessed by the Worth test); other types of tropia (manifest vertical deviations; suppression, diplopia or anomalous fusion assessed by the Worth test). The ocular motility evaluation and the measurement of possible deviations with prismatic bars (instead of using MaddoxWing as in the former investigation) were performed both for distance and nearness. Stereopsis was evaluated through Lang's stereotest; it was regarded as good (perception of all figures), sufficient (if lor 2 were perceived), or poor(no figure perception). Ocular motility is essential to a good vision quality and if altered may cause, among others, diplopia, blurred vision, as well as lachrymation, eye burning and eye heaviness (1). In particular we identified four symptoms related to visual fatigue and oculo-motor disturbances: tearing~ blurred vision, eye burning and eye heaviness(3). The study investigates the correlation between the clinical data collected during the preventive medicine campaign and these symptoms, reported by patients during the anamnestic interview before the examination and graduated according to occurence frequency (never, rarely and frequently) ~3). Some VDT work parameters were then identified (weekly VDT working hours, length of service at VDT, kind of work) in order to highlight the possible correlations with the clinical data collected. The association between symptoms and the clinical and occupational data have been tested with general association test (Pearsons's 22, and LR test) and with the MantelHaenszel test (checking the assumption of linear association between every single symptom and clinical data). 3.RESULTS The average age of the population is 41,45+ 10, 59 years Table 1 Prevalence of cover test results Cover test
Distant
Near
ortophoria
92.3%
44.3%
esophoria
0.6%
4.1%
exophoria
4.5%
47.3%
other phorias
0.4%
1.4%
esotropia
0.7%
0.8%
exotropia
0.8%
1.1%
other tropias
0.7%
1.0%
Table 1 shows the prevalence of ocular motility classes: the difference detected between vision for distance and nearness is caused by the variation in the AC/A ratio. Regarding the sterescopic vision, 91.8% of the population had a good stereopsis, 3.9% sufficient and 4.3% poor. The four-dot Worth-test performed at distance showed 1.9% diplopic patients, while 0.5% had suppression and 97.6% fusion. Similar values were found for near four-dot
609 Worth-test. The results of the statistical analysis between every single symptom considered (lachrymation, sore eye, eye heaviness and blurred vision) and the clinical parameters for the ocular motility evaluation show a good evaluation. In particular the general association tests (Pearson's ~2 of probability relationship), which investigate the differences between the frequencies observed and those expected in case of no connection between the two characters, show a highly significative connection for the values of the cover test for nearness with all four symptoms. The cover test for distance shows a good association only with blurred vision symptom. The Mantel-Haenszel test, which investigates the possible linear connection between two variables, is significative only for near cover test: the association degree (coefficient of contingency) is very high for blurred vision (0.039) and eye heaviness (0.028). The same tests applied to the anamnestic parameters relating to VDT work do not show relevant associations. 5.CONCLUSION The ocular and visual symptoms we considered are found in 45-90% of VDT workers(a): they are an important part in the so-called ocular discomfort syndrome that is also observed during computer use. Some observations made during the former ASSILS ophthalmological campaign had highlighted that some although latent ocular motility pathologies (heterophorias and exophorias in particular) increase the prevalence of blurred vision, sore eye, lachrymation and eye heaviness in VDT operators °). Heterophorias are certainly an ocular stress factor. The analyses carried out on the latter investigation's data, however, did not show correlations between ocular motility, subjective symptoms and the VDT work parameters considered (lenght of service, duration or kind) We can conclude that ocular motility play an important role in the occurence of the asthenopic symptoms more frequently complained by VDT operators(4). Particularly, exophoria and esophoria seem to be most capable of induce visual stress in this VDT users population. In our opinion, the difficulty in identifying statistically signiticative associations with the VDT work evaluation parameters must shift the studies inside laboratories. Only under these conditions it will be possible to monitor environmental parameters and underline those variables that play such an essential part in the pathogenesis of VDT-induced complaints (gaze position, dampness and microclimate temperatute..). REFERENCES
1. Bagolini B, Rieei B., Molle F., Lepore D. Study on ocular motility in Telephone Company emplyees working with video display units: preliminary conclusions. Boll. Oeul., 68 (Suppl.7) (1989) 49-68. 2. Von Noorden GK. The Near Vision Complex. In: Binocular Vision and Ocular Motility. The CV Mosby Company. St. Louis, Baltimore, Philadelphia, Toronto (1990) 85-100. 3. Yamamoto S. Visual, musculoskeletal and neurophsycological haelth complaints of workers using videodisplay terminal and an occupational health guideline. Jpn. J. Ophthalmol. 31 (1987) 171-183. 4. Jaschinski-Krauza W. Visual strain during VDU work: the effect of viewing distance and dark focus. Ergonomics 25 (1982) 1165-1173.
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
611
The vertical horopter and viewing distance at computer workstations D.R. Ankrum a, E.E. Hansen b, K.J. Nemeth ¢ • Human Factors Research, Nova Office Furniture, Inc., 949 Lake Street, Suite 3-G, Oak Park IL 60301, USA b Department of Technology, College of Engineering, Northern Illinois University, 203 Still Hall, DeKalb, IL 60155, USA
c Center for Ergonomic Research, Department of Psychology, Miami University, 104 Benton Hall, Oxford OH 45056, USA
1. INTRODUCTION The angle of view, otten referred to as the angle of incidence, is the angle formed by the line of sight and the plane of the screen. ISO 9241 [1] and ANSI HFS-100 (1988) [2] both allow the angle of view to be from zero to +/- 40 degrees. The top of the monitor may be tilted forward (negative angle of view) or backward (positive angle of view). Because increasing the angle of view reduces the visual arc of the letters, those restrictions on the angle of view limit geometric foreshortenin~ 1.1 Viewing Distance The closer the viewing distance, the greater the efforts to accommodate and converge [3,4]. Jaschinski-Kruza [5] found less eye strain at a viewing distance of 100 cm than at 50 cm. In a study by Grandjean [6], 75% of the subjects chose viewing distances of greater than 73 cm. In a non-VDT task, Owens and Wolf-Kelly [7] found a positive correlation between inward shifts in the resting point of vergence and subjective eye fatigue. It follows that workstation design should encourage greater viewing distances. 1.2 The Horopter The horopter is the locus of points in space that appear as single images to the observer [8]. Anywhere else in space will appear as double images to the observer. The horopter varies across individuals and with target size, fixation distance and gaze angle. Horizontally, the horopter is curved with the sides coming closer to the observer [9]. The vertical horopter, however, starts somewhere between the viewer's waist and feet and projects outward, intersecting the point of fixation and continuing in a straight line. If an observer fixates on the center of a straight vertical wire in the median plane, both ends of the wire will be seen as double until the wire is tilted backward, with its top farther away from the observer [10].
612
Figure 1. The vertical horopter for fixation at point A.
The development of the human visual system is conditioned by its environment during infancy and early childhood. [11]. When looking at intermediate points on the ground, objects below the point of fixation tend to be progressively closer to the viewer while those that are above are generally farther away. As a result, the lower visual hemifield has developed to be better equipped to see objects nearer than the point of fixation, while the upper visual hemifield is better equipped to see objects that are farther away [ 12].
A vertical VDT screen orientation results in an angle of view which is inconsistent with the developed abilities of the visual system. The characteristics of the vertical horopter predict that computer users with a monitor whose top is closer to the eyes than its bottom (negative angle of view) will experience greater discomfort and reduced performance. A study conducted to evaluate the effects of monitor height and angle on comfort and performance showed significant increases in four measures of discomfort, both postural and visual, when subjects viewed a monitor tilted at a negative angle of view [ 13]. During a pilot study, it was observed that subjects tended to alter their viewing distance as a function of the tilt of the monitor. It was decided to record viewing distance as an additional dependent variable. This paper reports on the relationship between viewing distance and comfort and preference at positive, negative and horopter monitor tilts, and at gaze angles corresponding to both eye level and a low position. Based on geometric foreshortening, userselected viewing distances should vary about the same at equal, but opposite angles of view. Based upon the predictions of the vertical horopter, viewing distances should vary differently as a function of positive and negative monitor tilt.
2. METHOD The subjects were six emmetropic students (refractive correction, if needed) with an average age of 21.5 years (range: 20-24) and an average of 6.1 years (range: 3-10) of computer experience. The experimental task involved comparing an accurate list of names, addresses and phone numbers on hard copy to a list on the screen that contained errors. When they found mistakes, subjects corrected the screen image.
613 There were six experimental conditions involving all combinations of three screen angles and two gaze angles. The three screen angles (measured from the perpendicular to the line of sight) were: "horopter," tilted back 15 degrees at the high condition and 25 degrees at the low condition, more or less coincident with the horopter [14]; "positive," tilted back 40 degrees; and "negative," tilted forward 40 degrees. The gaze angles were "high": top of screen at eye level; and "low": the top of screen 20 degrees below eye level. The center of the three lines of text on the screen was another 8 degrees lower. The viewing distance was initially set at 66 cm, but subjects were free to alter their postures. The characters were 4 mm high, corresponding to a visual angle of 21 minutes at 66 cm. That is within the preferred range of 20 to 22 minutes of arc recommended by ANSI HFS-100 (1988) [2] for tasks where legibility is important. In the positive and negative monitor tilt conditions, the characters subtended a visual angle of 16 minutes, corresponding to the minimum character height required by ANSI HFS-100 (1988) [2] for tasks where legibility is important.
Low Gaze Angle I 75 70
Horopter
55
~'5o 40
Positive
i
i
i
Start Break End Time of Session
Each condition consisted of two 20-minute segments, separated by a 10-minute break. Each subject participated in all six conditions on separate days. The order of conditions was determined by a Latin Square. Eye to screen measurements were taken at three different times during each condition: two minutes after the beginning of the first segment; at the end of the first segment; and at the end of the second segment.
3. RESULTS
I High Gaze Angle I 75.
70,
6oi "{551
Horopter
Positive
40
An earlier article [13] reported that four m e a s u r e s - neck discomfort, upper back discomfort, "tired eyes," and "tired looking at the screen" - were found to be significantly greater in the negative monitor tilt conditions.
i
i
i
Start Break End Time of Session
Figure 2. Mean viewing distance across the session.
The three-way ANOVA was performed to compare the viewing distances assumed in each of the six conditions. Follow-up tests were performed with a Tukey HSD procedure. Reported results are significant at the .05 level. Mean viewing distances are presented in Figure 2.
614 A significant main effect of monitor tilt, (p = .0006), demonstrated that subjects maintained shorter viewing distances for the negative angle, and there were no significant differences between the horopter and positive angles. A significant main effect of gaze angle (p = .0015), showed that subjects chose closer viewing distances in the high gaze angle conditions. There was no significant main effect of time which demonstrated a lack of a significant change in viewing distance over the course of the session.
Table 1 Correlation Coefficients for Viewing Distance and Subjective Measures of Comfort and Preference Subjective Measures
Correlation Coefficients Difficulty Seeing -0.107 Strange Eyes -0.142 Tired Eyes -0.103 Numb Eyes -0.132 Headache -0.109 Neck Discomfort -0.363 " Upper Back Discomfort -0.295 Lower Back Discomfort -0.405" Tired Looking at Screen -0.279 Tired Looking Back & Forth -0.332 Preference Ratin 8 -0.541 b Note. "significant at .05 level, b significant at .01 level
The interactions between time and monitor angle, gaze angle and monitor angle, and time and gaze angle were not significant. The three-way interaction between time, gaze angle and monitor angle was significant (p = .0260). Follow-up analyses found that subjects changed their viewing distances over the course of the session in the high gaze angle, negative monitor tilt condition. Those subjects moved closer to the screen as the session progressed.
In order to determine the relationship between viewing distance and the subjective measures of comfort and preference, Pearson Correlations were performed. As no main effect of viewing distance across time was found, the distance recorded at the end of the session was used in these comparisons. Table 1 lists the results of these analyses. Significant relationships were found between viewing distance, and neck and lower back discomfort (p < .05). Shorter viewing distances were related to increases in reported discomfort. The conditions in which subjects assumed shorter viewing distances were rated less favorably than those conditions in which they assumed longer viewing distance
4. DISCUSSION The results of this study suggest that monitor tilt may play a role in user-selected viewing distances at computer workstations. A vertical horopter which tilts away the from the observer at the top has developed to adapt to a commonly experienced feature of the visual environment. In that environment, objects below a point of visual fixation are usually closer to the observer, while higher objects are usually farther away.
615 In this study, viewing distances were significantly shorter when viewing a monitor at an angle of view opposite to the horopter. There also appears to be a relationship between viewing distance and neck and lower back discomfort. Shorter viewing distances led to greater increases in discomfort in both of those body regions. Because the results of this study appear to concur with the physiological mechanism of the vertical horopter, they suggests that monitor tilts opposite to the horopter may result in shorter viewing distances and greater increases in discomfort. It is inappropriate to consider the angle of view adjustments commonly observed in office environments as reflecting preferred settings due to the constraints of the lighting systems and equipment. In many offices, tilting the monitor back would result in glare from ceiling luminaires. If glare and reflections are not satisfactorily addressed, the potential benefits of a positive-tilted monitor will be lost.
ACKNOWLEDGMENT The authors wish to thank Dr. Walter Makous for introducing them to the concept of the vertical horopter.
REFERENCES [ 1] International Standards Organization. (1992). ISO 9241-3:1992, Ergonomic requirements for office work with visual display terminals (lrDTs)-Part 3: Visual display requirements. [2] Human Factors Society. (1988). American National Standard for Human Factors Engineering of Visual Display Terminal Workstations. Santa Monica, CA: author. [3] Collins, C., OqVleara, D., and Scott, A.B. (1975). Muscle strain during unrestrained human eye movements. Journal of Physiology, 245, 351-369. [4] Fisher, R.F. (1977). The force of contraction of the human ciliary muscle during accommodation. Journal of Physiology, 2 70, 51-74. [5] Jaschinski-Kruza, W. (1988). Visual strain during VDU work: the effect of viewing distance and dark focus. Ergonomics, 31, 10, 1449-1465. [6] Grandjean, E., Hunting, W. and Pidermann, M. (1983). VDT workstation design: preferred settings and their effects. Human Factors, 25, 161-175. [7] Owens, D.A., Wolf-Kelly, K. (1987). Near Work, Visual Fatigue, and Variations of Oculomotor Tonus. Investigative Ophthalmology and Visual Science, 28, 743-749.
616 [8] Tyler, C.W. (1983). Sensory Processing of Binocular Disparity. In Schor, C.M., and Ciuffreda, K.J. (Eds.). Vergence Eye Movements: Basic and Clinical Aspects. Boston: Butterworths. [9] Aguilonius, F. (1613). Opticorum Libri Sex. Antwerp: Plantin. [ 10] von Helmholtz, H. (1866/1925). Treatise on Physiological Optics, volume 3, translated from the 3rd German edition, Southall, J.P.C. (Ed.). (p.429). New York: Optical Society of America. [11] Tychsen, L. (1992). Binocular Vision. In Hart, W.M. (Ed.). Adler's Physiology of the Eye. (pp. 773-853). St.Louis: C.V. Mosby. [12] Breitmeyer, B., Battaglia, F. and Bridge, J. (1977) Existence and implications of a tilted binocular disparity space. Perception, 6:161-164. [13] Ankrum, D.R., Hansen, E.E., and Nemeth, K.J. (in press). The vertical horopter and the angle of view. In Grieco, A., Molteni, G., Occhipinti, E. and Piccoli, B. (Eds.). Work With Display Units '94. Amsterdam: Elsevier. [14] Nakayama, K. (1983). Kinematics of Normal and Strabismic Eyes. In Schor, C.M. and Ciuffreda, K.J. (Eds.). Vergence Eye Movements: Basic and Clinical Aspects. Boston: Butterworths.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
617
R e c o m m e n d a t i o n for V D T workstation design based on analysis of ocular surface area Midori Sotoyamaa, Shin Saito b, Sasitorn Taptagaporn c and Susumu Saito" '
National Institute of Industrial Health, 6-21-1, Nagao, Tama-ku, Kawasaki 214, Japan
b Aloha Mizuho College, 86-1, Haiwa, Hiratobashi-cho, Toyota 470-03, Japan cMinistry of Public Health, Tivanon Rd., Nonthaburi 11000, Thailand
Abstract
Ocular surface area (OSA) is thought to be an informative index of visual ergonomics because OSA size is closely related to visual comfort. This paper presents a comfortable visual display terminal (VDT) workstation design based on an analysis of OSA. By measuring OSA while performing visual tasks with and without a VDT, it was clarified that the OSA was strongly affected by the composition of the VDT workstation, including placement of the display, keyboard, book and so on. It is recommended that the display be set at a lower position to realize a more comfortable workstation with a smaller OSA for the VDT operator.
I. INTRODUCTION In proportion to the increasing use of VDTs (Visual Display Terminals) in many workplaces, large numbers of workers are complaining of eye fatigue or eye strain. One of the most important factors behind these complaints may be the height of the gaze direction in VDT work. A number of studies have suggested that the vertical gaze direction relates to phoria, dark vergence and tear volume [1-3]. These studies have shown objectively that downward gazing is physiologically more comfortable than upward gazing. In addition to these results, as given in our presentation at HCI '93, the gaze direction changes the ocular surface area (OSA) [4]. Upward gaze increases the OSA, and a larger OSA decreases the amount of tear volume [3]. The ocular surface is directly exposed to air and is very sensitive to various irritants such as dust, heat, dryness, air flow and so on. A decrease in tear volume induces the likelihood of eye exposure to the above irritants, and this may cause eye fatigue and eye irritation. Therefore, we consider OSA a useful index for the ergonomic evaluation of visual tasks, workstation design and so on. In this study, we
618 measured the OSA in different kinds of visual tasks, and we here recommend a comfortable workstation design based on our analysis of OSA.
2. METHODS A subject was asked to perform two kinds of tasks. One was work with a VDT, and the other was traditional office work without a VDT. We chose a word processing task with a keyboard and drawing a picture on a CRT (Cathode Ray Tube) display with a mouse as VDT work, and reading a book and drawing a picture on a piece of paper as traditional office work. While the subject was engaged in a task, a frontal image of the eye was recorded on a video tape recorder. We estimated the OSA from the video image of the eye using formula (1), which was revealed by our previous study [4]. y = 3.05 x - 0.39,
r = 0.97
(1)
where x is width of the palpebral fissure in cm, y is ocular surface area in cm2 Data were sampled every one second for two minutes from five minutes atter the beginning of the task.
3. RESULTS Figure 1 shows the averages and standard deviation of OSA for each task. The OSA for VDT tasks was 2.1 + 0.9 cm2 during the word processing task and 2.6 + 0.2 cm2 while drawing a picture with a mouse. The OSA for non-VDT tasks was 1.9 +_0.3 cm2 for reading a book and 2.1 + 0.3 cm2 for drawing a picture. The OSAs while performing VDT tasks were greater than those during traditional office work. Further analyses were carded out to clarify the differences between tasks with and without a VDT. One was an analysis of changes in OSA with time, and the other was an analysis of the frequency distribution of OSA. Figure 2-(a) shows changes in OSA during a word processing task. OSA while viewing the CRT display was 3.2 cm2, and OSA while looking at a keyboard and manuscript was 1.5 cm2. During this task, large changes in OSA were frequent. The average of OSA while drawing a picture on the CRT display with a mouse was 2.6 cm2, the largest value among all the tasks compared. The changes in OSA during traditional office work were very small, as shown in figure 2-(b), which is the OSA while reading a book. The average OSA was 1.9 cm2 and showed little change.
619
3.5 ..~
3.0
¢,.)
2.5
to
2.0
~
1.5
L_
m
1.o
0
0
0.5 0.0
(a)
(b)
(c)
(d)
Figure 1. Ocular surface area while perfonrfing visual tasks (a) Word processing with a VDT (b) Drawing a picture with a VDT (c) Reading a book (d) Drawing a picture
4.0
.o
"~ 3.0
.0
2.0
I lY IA II II//I/AAA l AIti
.o
Lf=_
~
L~.._ _
:3
.o
_~ ].o o.o
!
0
~
!
4o
6'o
~
,oo
120
.o
o
20
40
60
80
100
Time (sec)
(a) Word processing
(b) Reading a book
Figure 2. Change in ocular surface area with time in work with and without a VDT
120
620
Table 1 Frequency distribution of OSA in work with and without a VDT (%) with a VDT without a VDT OSA (cm 2) Word processing 0.0-0.5 0.5- 1.0 1.0- 1.5 1.5- 2.0 2.0 - 2.5 2.5 - 3.0 3.0- 3.5 3.5-4.0
0 3 28 30 0 2 38 0
Drawing
Readin~l
Drawing
0 0 1 1 13 86 0 0
0 0 4 63 32 1 0 0
0 0 1 47 38 13 1 0
Table 1 shows the frequency distribution of OSA for each task. The OSA while performing a word processing task was concentrated at two parts, from 1.0 cm 2 to 2.0 cm 2 and from 3.0 cm2 to 3.5 cm2. While drawing a picture on a CRT, the OSA distribution was highly concentrated at a narrow range, ranging from 2.5 cm2 to 3.0 cm2. The OSA while doing traditional office work without a VDT was concentrated from 1.5 to 2.0 cm2, and the distribution patterns of the two tasks were very similar. The distribution of OSA while working with a VDT was found to be extremely different from that for traditional office work without a VDT.
4. DISCUSSION
The results of OSA analysis with time and frequency distribution clarified the features of work with and without a VDT. During word processing, large changes in OSA frequently occurred, and when drawing with a mouse, OSA was large throughout the duration of the task. The larger OSA while doing VDT work must increase the amount of tear evaporation from the eye surface [5] and decrease the tear volume [6]. We previously confirmed that a large OSA caused by an upward gaze was subjectively evaluated to be more uncomfortable than downward gaze [3]. The decrease in tear volume is very likely to expose the eye to various irritants and induces eye dryness and eye irritation during VDT work. The frequency distributions of OSA during VDT tasks were clearly different from those of traditional office work, as described above. These differences are caused by the composition of the workspace at the office. The subject could easily move a book or paper to the most comfortable or preferred position in the case of traditional office work. In contrast, in VDT work, the subject was constrained by the position of the display and keyboard in the VDT workstation and was forced to maintain a larger OSA. The location of objects being viewed during work strongly affects the OSA. Thus, to realize a comfortable workstation, the consideration that the CRT be set in a lower position is very important.
621 5. CONCLUSION It is concluded from the results of this study that lowering the position of the display would lower gaze direction, reduce OSA and decrease eye discomfort in VDT workers. It is proposed that the display should be set directly on the desk to provide a comfortable VDT workstation, and if possible, a workstation whose display is lower than desk level would be better.
REFERENCES
1. S. Saito, M. Sotoyama, T. Suzuki, Sh. Saito and S. Taptagapom, (1993) Vertical gaze direction and eye movement analysis for a comfortable VDT workstation design, Work with Display Units 92, Ed. by H. Luczak, A. Cakir and G. Cakir, (ELSEVIER, Amsterdam), 110' 114. 2. S. Taptagapom and S. Saito, (1993) Visual comfort in VDT operation: Physiological resting states of the eye, Industrial Health, 31, 13-28. 3. S. Abe, M. Sotoyama, S. Taptagaporn, Sh. Saito, M. B. Villanueva and S. Saito, (1994) Relationship between vertical gaze direction and tear volume. Fourth International Scientific Conference Book of Short Papers, Work with Display Units 94, 1, B6-7. 4. M. Sotoyama, Sh. Saito, S. Taptagapom, T. Suzuki and S. Saito, (1993) Gaze direction and ocular surface area in VDT work, Human-Computer Interaction : Applications and Case Studies, Ed. by M. J. Smith and G. Salvendy, (ELSEVIER, Amsterdam), 750-755. 5. K. Tsubota, (1993) Dry eyes and video display terminals, The New England Journal of Medicine, 8, 584. 6. Y. Yaginuma, H. Yamada and H. Nagai, (1990) Study of the relationship between lacrimation and blink in VDT work. Ergonomics, 33, 799-809.
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawaand H. Mori (Editors) 1995 Elsevier Science B.V.
623
L i g h t i n g a n d V i s u a l E r g o n o m i c s for t h e D i s p l a y S c r e e n Environment M J Perry and P J Littlefair Building Research Establishment (BRE), Garston, Watford, England, WD2 7JR 1. I N T R O D U C ~ O N The rise in the use of computers and their associated display screen equipment (DSE) has been rapid in the developed industrial nations, particularly in office environments. The widespread introduction of DSE into the office environment has been accompanied by problems with the ergonomics and the environment of the DSE work place. It is important to address these problems to create a work place environment for DSE t h a t is comfortable for users and non-users. An important issue raised by the introduction of DSE workstations was how best to light the environment [1]. Electric lighting needs to be arranged to minimize screen reflections and glare. The use of daylight for the DSE environment can address both the ergonomic requirements of the users, and the need for energy efficient use of electric lighting, but needs careful attention at the design stage. The BRE is carrying out a programme of work investigating visual ergonomics of electric lighting and daylighting for DSE. This is described in this paper, with guidance for daylighting of DSE [2,3]. 2. E I ~ E ~ C
LIGHTING I S S U F ~
2.1 LuminMres a n d the visual envirmmmnt One of the major differences between paper based and screen based tasks is the presence of a glass screen on the DSE which acts as a partially reflecting filter between the user and the task on the phosphor screen. Reflections of light fittings on the screen are a potential problem for users, and designers now specify lower brightness luminaires [1] to overcome the problems of reflections. The shift in use from wrap around type luminaires, with light output distributed sideways, upwards and downwards, to low brightness luminaires with very directional downward light output has produced a general change in the luminous environment of office spaces. When using low brightness
624 luminaires, vertical surfaces receive much less illumination, and modelling effects on people and objects in the space can be harsher, particularly where the ceiling and walls have low surface reflectance values. People can find this subjectively unacceptable. Low brightness luminaires may also adversely affect energy consumption. In principle, for a given level and uniformity of illuminance across the horizontal working plane a larger installed load may be needed to achieve the required iUuminance and uniformity if low brightness luminaires are used. So, while DSE installations using low brightness luminaires m a y conform to the relevant recommendations, and the lighting is suitable for screen based tasks, the lighting may be unsuitable for other users carrying out non-screen tasks in the space eg discussions with colleagues. (Screen users are very likely to be non-users at some time during the working day.) The distribution of the light m a y also result in adverse subjective responses about the appearance of the space. These potential negative effects on people in the space may be, in principle, aggravated by the requirement for a higher installed load when using low brightness luminaires. These problems can be ameliorated by including supplementary lighting to increase room surface brightnesses. Individual lighting control, possibly using dimming, can help with the transitions between screen and non-screen tasks. 2.2 DSE screens One of the important findings of a study investigating user responses to DSE was that, for the study sample, few users complained of reflections in their screens. Most of those who did complain of reflections were mainly concerned with the images of windows or of window coverings. Complaints of reflected lighting equipment generally relate to extremes, such as images of high luminance spots above uplighters next to walls. Those using negative contrast screens, where the background is lighter t h a n the characters, suffer far fewer screen reflection problems t h a n those with dark background screens. The common impression t h a t 'white' screens are rapidly overtaking 'dark' screens in commercial offices is exaggerated. Certainly there are increasing numbers of newer 'white' screens in use but there are also m a n y older colour screens of low resolution running old software. There are also m a n y screens t h a t are running software t h a t default to white on blue, or similar compromises. Certainly for the sample use in the study, the colour screens out numbered the mono dark background screens. It is likely to be a n u m b e r of years before the light background screens outnumber dark background screens. In the study there were no adverse subjective responses to jitter or flicker on the screens. This was true for both light background screens and dark background screens. Possibly if more large screen users, or more workers from
625 darker working environments, had been included in the study then some comparative feel for the effects of different screen set ups might have been possible. 2.3 Design issues In the sample of buildings included in this study at least one example of each of the main types of lighting equipment available to light DSE areas was found. There were no distinct user preferences t h a t emerged from the interview data attributable solely to the lighting. For instance although the combined up/downlights and concealed wall washing systems were favourably commented on, this may have been related to their being installed in offices which had a high staff morale and company loyalty and where the office space itself was of an interesting nature. Because of confounding factors it was not possible to determine whether lighting designed for DSE areas using the current guidance would result in more energy usage. On theoretical grounds it would appear so, but this could not be substantiated. An important finding of the study was the absence of integrated design procedures for the interior luminous environment, particularly for DSE. Thus the lighting design for a space is, in the great majority of cases, developed in isolation from consideration of surface colours and reflectance values, and what furniture and other objects may be placed within the space. Changes in the interior environment from those assumed at the design stage will have a profound effect on the lighting levels, energy consumption, and appearance of the space. When planning the building layout, it is important to consider all spaces where DSE is likely to be used, not just offices. For example, factory and workshop spaces often contain DSE, and the reception areas of buildings usually have DSE as well as often having large areas of glazing. If client expectations about the lighting design for a space are to be fulfilled by the final installation it is essential t h a t a well written specification is developed in consultation with the client. Any changes to the environment of the space should be fed back into the design process to establish the effect of t h a t change on the lighting design. The client should be made aware of the changes to establish t h a t they are acceptable. 3. DAYLIGHTING FOR DISPLAY-SCREEN EQUIPMENT 3.1 Background People generally like having daylight in their workplaces, and its use can make buildings more energy efficient. In interiors with DSE, occupants are still very positive about windows and daylight. For example, a German study [4] found t h a t DSE users preferred to be near windows. Daylight can brighten up a room
626 by supplying extra light to vertical surfaces, and windows supply contact with the outside world. For DSE users, an external view can help relax the eyes. However the provision of daylight can lead to problems: Sunlight in the eyes of the DSE user 2. Sunlight reflecting off the screen Reflections of windows in the screen 3. 4. Reflections of interior sunlit patches in the screen Glare from interior sunlit patches 5. 6. Glare from bright patches of sky High illuminances on light coloured surfaces at the workstation making 7. the screen look dark in comparison °
3.2 Building a n d window design In a new building it is possible to control the form of the building and the disposition of glazing to promote a high quality daylit environment. Ideally, the long sides of the building should face north and south. East and west facing glazing should be avoided, because low altitude sunlight can enter and cause glare and screen reflection problems. Screen reflection and glare difficulties tend to be worse in wide, open plan spaces with continuous 'ribbon' glazing. Cellular offices seldom cause problems. Where possible, wide, open plan spaces should be divided into a succession of smaller spaces. Long runs of glazing should also be separated, into a series of smaller windows. Windows on adjacent walls give good daylighting distribution but make it difficult to avoid screen reflections and glare. For this reason, the corners of buildings need special attention in design. Windows solely on opposite walls make it easier to align DSE workstations to avoid glare problems. Glare from windows can be reduced by decreasing the contrast between glazing and the window wall. All the following can help: • a light coloured window wall • light coloured glazing bars • splayed, deep, light coloured reveals • supplementary electric lighting on the window wall Windows should be positioned to give a good view out. In particular, the sill needs to be low enough, especially on the upper floors of tall buildings, to give a view of the horizon and foreground. In a deep building, an atrium can offer some of the characteristics of an external view while bringing in some daylight and sunlight. However, it is important to consider the possibility of sunlight coming through the atrium glazing and reaching adjoining spaces with DSE. In daylit spaces, screen orientation is very important. DSE screens and their users should face parallel to the window wall, not towards or away from it, and they should be at least 2m from the windows. This may require careful planning of workstation positions, and circulation routes. Where windows are
627 holes in the wall r a t h e r t h a n continuous ribbon glazing, workstations could be placed so t h a t the users or their screens face blank areas of outside wall [5]. Appropriate lighting controls are vital to make the most of available daylight. For spaces with DSE the most suitable solution is often localised m a n u a l control, for example using infra- red controllers or luminaire pull cords. Occupants value individual user controlled dimming or step switching, and it will save energy in a room with intensive DSE use. Controls m u s t be easy to u n d e r s t a n d and operate. 3.3 S h a d i n g devices Adjustable shading t h a t the occupants can control is usually the best option. Conventional venetian blinds can be adjusted to retain a view while restricting incoming sunlight and sky glare. For east or west facing windows vertical slat louvre blinds can cut out sunlight while keeping a view out and some incoming diffuse daylight. However, problems can occur if the blind material is translucent. A blind illuminated by sunlight can be very bright, causing reflection in DSE screens and distracting glare. This problem can occur with other non-opaque shading devices when sunlight enters. Translucent roller blinds, net curtains and diffusing glazing can all act as secondary sources of glare and screen reflections. While tinted solar control glazing and window films can reduce sky glare and incoming solar heat, they are not effective at controlling glare from the sun. If sunlight is likely to enter a space with tinted glazing, extra shading will be necessary, usually adjustable blinds. Heavily tinted glazing can make a room, and the view out, look dull. Careful design is required if rooflit spaces contain DSE. Sunlight can come through the rooflights and reach working areas, and rooflights can be reflected in screens. Dome type rooflights with a deep 'collar' are suitable. Northlights (either vertical or sloping) will restrict sunlight entry, but can cause screen reflections if DSE screens face the glazing. Other rooflight types will need shading if they let sunlight reach a DSE workstation or if they are visible at less t h a n 15° - 35 ° to the horizontal (the angle depends on the category of DSE use [1]) from a workstation. 3.4 Innovative daylight systems Innovative daylight systems [6], like the light shelf, mirrored louvres and prismatic glazing, can control and redistribute direct sunlight and improve the uniformity of light within a room. For DSE spaces they are of interest for several reasons: (a) They reduce the high illuminances experienced close to the window. These high illuminances can increase the contrast between the screen and source documents, causing discomfort for DSE users. (b) They can reduce window luminances, decreasing the severity of screen reflections and discomfort glare.
628 (c)
They maintain a reasonable daylight level deep within a space, although BRE research [7] suggests they rarely increase core illuminances compared with an unshaded window. Whatever system is chosen, it needs to work properly for all sun positions. In some cases supplementary blinds will be needed. CONCLUSIONS AND F U R O R
WORK
The use of low brightness luminaires in DSE spaces can lead to an unacceptable luminous environment with dark walls and harsh modelling. Supplementary lighting on room surfaces can help overcome this. Daylighting can bring important benefits to DSE users, but measures must be taken to avoid screen reflections and glare. These include appropriate shading devices and workstation orientation. Innovative daylighting systems can help but need careful design to ensure they can control sunlight at all times of year. The next stage of the project is currently underway and is investigating good practice DSE lighting installations. The study includes a review of the prospects for developing integrated lighting design procedures, and to investigate the effect of DSE lighting on lighting energy consumption. ACKNOWl ,EDGEMENT This paper has drawn on work by Tanya Heasman of System Concepts Limited, Lorraine Gardner, formerly with the Institute for Consumer Ergonomics, Loughborough, and Paul Ruffles for Building Health Consultants. T h e i r contributions are gratefully acknowledged. REFERENCF_~ 1. Chartered Institution of Building Services Engineers. 'Lighting Guide LG3. Areas for visual display terminals' London, CIBSE, 1989. 2. M J Perry and L Gardner, 'Daylighting requirements for display-screen equipment' Building Research Establishment Information Paper IP 14/93. Garston, BRE, 1993. 3. P J Littlefair, 'Daylighting design for display-screen equipment' Building Research Establishment Information Paper, to be published. 4. Cakir A E. 'An investigation on state-of-the-art and future prospects of lighting technology in German office environments'. Berlin, Ergonomics Institute for Social and Occupational Sciences Research Co Ltd. 1991. 5. W Bordass, T Heasman, A Leaman and M Perry, 'Daylight use in openplan offices: the opportunities and the fantasies' Proc CIBSE National Lighting Conference, Cambridge 251-259, 1994. 6. P J Littlefair, 'Innovative daylighting systems' Building Research Establishment Information Paper IP 22/89. Garston, BRE, 1989. 7. M E Aizlewood, 'Innovative daylighting systems: an experimental evaluation' Ltg Res & Technol 25 (4) 141-152, 1993. © British Crown copyright - Building Research Establishment
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
629
Computerised Analysis of Prolonged Seated Posture B Kayis and K Hoang School of Mechanical and Manufacturing Engineering The University of New South Wales, Sydney, 2052Australia
Prolonged seating can cause musculo-skeletal problems in the long term if poor postures are adopted. Three dimensional static model of the body to calculate the intervertebral disc compression at the fifth lumbar disc was built. SAMMIE, a computer aided ergonomics package was used for modelling of two workplace situations and determination of joint centre locations and joint angles. Experimentation was also performed to determine the body-mass distribution on the seats used.
1. INTRODUCTION Prolonged seating can be seen as any period of seating which is undertaken for an extended length of time, up to three hours or more (Shachel et al, 1969). Hockenberry (1980) introduced the term "task seating" defined as seating extended for periods which could exceed one hour in order to perform a certain function. Analysis of prolonged seating posture in the past has been done using various qualitative and quantitative techniques. These techniques include subjective comfort ratings, radiography, electromyography (EMG), height shrinkage measurements, internal pressure measurements and both dynamic and static mechanical modelling. Previous static modelling has been restricted to two-dimensional mechanical models. Photography has been used for joint angle and joint centre location data as inputs to the static models. Armstrong (1986) notes that, rigorous descriptions of posture are difficult since joint axes are in reality not fixed and there are many axes of rotation, joints with small ranges of motion are neglected and joint angles are approximated. Genaidy and Karwowski (1993) analysed the effects of neutral posture deviations on perceived joint discomfort ratings in sitting and standing postures. It was found that lateral bending of the neck is subjectively more stressful than its flexion, extension and rotation. Jones (1969) conducted a fitting trials test and a subjective discomfort analysis on prolonged seating. EMG techniques were applied by Gray et al (1966), Schuldt et al (1986) and Occhipinti et al (1986). A pressure related model of seating discomfort was developed by Shen and Galer (1993). The mechanical aspects of the seated human body was considered by Branton (1969) and Bendix et al (1983). Biomechanical models were also used in order to minimise muscular loads on the sitter (Ekland et al, 1983, Occhipinti et al, 1985, Occhipinti, 1986, Ekland
630
and Corlett, 1986). With the advent of the personal computer, several settings needed for an ergonomic computer workstation were published (McPhee, 1991, Health and Safety Executive Library and Information Services, 1991, Grandjean, 1982, 1988, Halpern and Davis, 1993, Lu and Aghazadeh, 1993), but little study has been done in the area of using a CAD system to generate inputs into a threedimensional static model for seated postures. The main aim of the research was to build a three dimensional static model of the body to calculate the intervertebral disc compression at the fifth lumbar disc using SAMMIE - a computer aided ergonomics package. The first workplace modelled was a computer workstation. Eleven postures were analysed with both the seat pan and back rest varied. The second workplace modelled was an unadjustable easy chair. In this paper, the results obtained from Computer Workstation Modelling is given. 2. METHODOLOGY The SAMMIE ergonomics package enables the rapid determination of joint centre locations and joint angles as input to 3-dimensional static models of prolonged seating postures. However, further experimentation and studies were carried out in order to determine the body mass distribution on the seatpan, backrest and floor of the postures studied, before developing the model. 2.1 Computer Workstation Modelling Before building any static models or manipulating the SAMMIE operator to obtain the desired postures, the workplace must first be modelled. The workplace considered consisted of a chair, a desk and a computer. The choice of workplace components for the model were arbitrarily made.
Eleven postures were chosen for analysis. Each posture first being modelled in SAMMIE, after which the joint centre locations were obtained and input into the three dimensional static models. Of the eleven postures, there are four with the seatpan horizontal, four with the seatpan reclined by five degrees, and three with the seatpan tilted forwards by five degrees. All the modelled postures were created with the operators feet flat on the floor (Figure 1). 2.1.1 Distribution of body mass In order to create a static mechanical model of each prolonged seating posture, the distribution of body mass on the floor, seatpan and backrest (if in use) needs to be determined. An experiment was carried out by sixteen healthy university students (ten men and six women). The results showed that there was a great similarity between body mass distributions for males and females. 2.1.2 Coefficient of static friction for seat cover For those postures in which the seatpan is tilted either forwards or backwards or the backrest is not vertical, the coefficient of static friction for the seat cover is included in the calculations.
631
P~cNum 1
Pomure 7
•
Se~l~n ~,,.~,,~;
•
•
bomm
•
Ul~;pt omc~l~sue
Seamen roamea S o e g n m l k c m m 115 c l e g n m m ==ro
wmx:~
Loi~ng m~m=.
fuu use of
~cwelt
=====::=~
P m •
Iqleuml 2
'
,t,
•
Sea~m Hormx~
•
Backrest v w l x ~
|
Slpm
•Bomre= •
r,,a,nea S = , ~ n m 115 d e g r ~ n m ~
Leaning Dick. p l ' b l i i ~
use
~ Posture S •
Iq:mlltn 3 •
Fcx~w~tmed =empon. -fi clogree=
Seat~an honzcmm
Ba:Smm ncX m U N
BaOvest v w t c u "-';
Trunk erect
8 e r l for.m= pos'~uro
Poetum 10
P m
•
F o r ~ i m t i m ~ seaman -50egrees
•
BaOcll~ ~
•
Ol:mtilor mma0ng for~w~s
=n use
4
•
,S~l~
~
.
B~m~
toO,nero 110 ~ Pollmm 11
Fuji use Of ~ Ixm~vllrcls
P ~
.
•
~lnii'tg "~"~
Focv4ira t,mng seaman. -5 aegnms
•
~
•
Lelr~g ~
105 Oegrees to zero
$
B a c ~ n t 95 Oegrees to zero Erect trurw
P W
~
!
I
.
t k O ~ m ~ a e g n m to zoro
"
BentF(x~wm
i
Figure 1 Postures used for computer workstation modelling
fu=l use of
632
2.1.3 Three-dimensional static model The model was based on a 50th percentile male of a mass 75.2kg (Chaffin and Andersson, 1984). Assuming that the body is divided by a horizontal plane crossing the intervertebral disc between the fifth lumbar disc, L5 and the first sacral disc $1. Two separe models were able to be constructed to calculate the compressive loading on the spine at this level (Figure 2). ~Z
i
o~,~ o
,y
Figure 2 Location of Axis System at the L5/$1 Level For both models, static equilibrium is considered. The moment equation using the triple scalar product was developed in order to determine the resulting moment at the L5 intervertebral disc. From the calculated moment, the force required by the erector spinal muscles to sustain the moment was calculated and the compressive force on the intervertebral disc was found. Main assumptions concerning both Upper and Lower Models are that in a seated typing posture, the body is virtually symmetrical about the sagittal plane, thereby producing no moment about the Y and Z axes. Another assumption was that, all reaction forces to be taken as point Ioadings. 2.1.3.1 Body segment centre of gravity locations and mass data Data on the location of body segment centre of gravities and segment mass was obtained where possible through Dempster's work (Chaffin and Andersson, 1984). Also, the joint centre locations were obtained from SAMMIE and used to obtain the unit vectors.
3. CONCLUSIONS The conclusions drawn for the Upper Model agreed with previous studies. Postures with the operator leaning back using a reclined backrest gave the lowest lumbo-sacral disc loading regardless of the inclination of the seatpan. Posture 4 with the horizontal seatpan resulting in a slightly lower disc load than Posture 5 and 6 with the reclined seatpan. The forward tilting seatpan gave high disc loading but still was preferable to erect or bent over postures by a large margin. Postures with the trunk in an erect posture (Postures 1, 5 and 9) resulted in an almost identical lumbo-sacral disc loading regardless of the tilting of the seatpan forwards or backwards. Forward bending postures (3, 6 and 10) caused the highest iumbosacral disc loading. Tilting the seatpan forwards or backwards will slightly lessen
633
the compression by transferring some of the load to the floor or backrest respectively. Although doing so still gives the worst possible loading on the lumbosacral disc. Slouching or slumping on a chair with a vertical backrest will cause a disc loading almost as high as bending forwards. This is due to the increased moment arm of the backrest reaction which moves up to the top of the backrest. 4. RECOMMENDATIONS
The lower model developed requires refinement particularly in determining the location of reactions on the seatpan. EMG testing may be carried out to find out if there is any correlation between disc compression levels and muscle contractile activity. REFERENCES
1.
Armstrong, TJ, 1986, Upper Extremity Posture: Definition, Measurement and Control, The Ergonomics of Working Postures, Proceedings of the First International Occupational Ergonomics Society Symposium, Yugoslavia, Taylor and Francis, London, pp 44-53. 2. Chaffin, DB, Andersson, GBH, 1984, Occupational Biomechanics, John Wiley and Sons, USA. 3. Genaidy, AM, Karwowski, W, 1993, The Effects of Neutral Posture Deviations on Perceived Joint Discomfort Ratings in Sitting and Standing Postures, Ergonomics, Vol. 36, No.7, 785-792. 4. Grandjean, E, Hunting, W, Nishiyama, K, 1982, Preferred VDT Workstation Settings, Body and Physical Impairments, Journal of Human Ergology, Vol. 11, pp45-53. 5. Grandjean, E, 1988, Fitting the Task to the Man, Taylor and Francis, London. 6. Gray, FE, Hanson, JA, Jones, FP, 1966, Postural Aspects of Neck Muscle Tension, Ergonomics, Vol. 9, 3, pp 245-256. 7. Hockenberry, J, 1980, A Systems Approach to Long Term Task Seating Design - A NATO Symposium Paper, NATO Symposium - Anthropometry and Biomechanics: Theory and Application, England. 8. Halpern, CA, Davis, PJ, 1993, An Evaluation of Workstation Adjustment and Musculoskeletal Discomfort, Proceedings of the Human Factors and Ergonomics Society, 37th Annual Meeting, Melb., pp817-821. 9. Health and Safety Executive Library and Information Services (HSE), 1991, Seating at Work, London. 10. Jones, JC, 1969, Methods and Results of Seating Research, Ergonomics, Vol. 12, 2, pp171-181. 11. Lu, H, Aghazadeh, F, 1993, VDT Positions: Effect on Performance and Comfort, Proceedings of the Human Factors and Ergonomics Society, 37th Annual Meeting, pp397-400. 12. McPhee, B, 1991, Development of Ergonomics Seating, Choosing Chairs The Bottom Line (Seminar), Worksafe Australia.
634
13. Occhipinti, E, Colombini, D, Frigo, C, Pedotti, A, Grieco, A, 1985, Sitting Posture: Analysis of Lumbar Stresses with Upper Limbs Supported, Ergonomics, 28, 9, pp1333-1346. 14. Schuldt, K, Ekholm, J, Harms-Ringdahl, K, Nemeth, G, Arborelius, UP, 1986, Effects of Changes in Sitting Posture on Static Neck and Shoulder Muscle Activity, Ergonomics, 29, 12, pp1525-37. 15. Schen, W, Galer, IAR, 1993, Development of a Pressure Related Assessment Model of Seating Discomfort, Proceedings of theHuman Factors and Ergonomics Society, 37th Annual Meeting, pp831-835. 16. Shakel, B, Chidsey, KD, Shipley, P, 1969, The Assessment of Chair Comfort, Ergonomics, 23, 2, pp269-306.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
635
Indoor air quality evaluation by continuous measurement in computerized offices. A. Ito, M. Takahashi, K. Sakai and K. Kogi The Institute for Science of Labour, 2-8-14 Sugao, Miyamae-ku, Kawasaki 216, Japan 1. I N T R O D U C T I O N The assessment of indoor air quality in computerized offices requires continuous measurement as there is a need to assess it for the total period of computer work [ 1-2]. The cross-sectional space monitoring method commonly used to evaluate indoor air quality does not seem to provide information about the local distribution of air-quality characteristics at the different times [3-4]. Cross-sectional measurements often fail to reveal contain discrepancies arising from varied subjective responses of the occupants in the room and from changing working environments during extended business hours. With a view to examining problems in real work environments interacting with computers, a newly developed method for 24-hour continuous measurement of indoor air quality charactenstics was applied to computerized offices. The purpose was to yield information about detailed fluctuations, including those in extended business hours. The method allowed to monitor air velocity, air temperature, relative humidity, airborne dust concentration, carbon dioxide concentration and noise level. We have applied this procedure to several automated-office buildings in Tokyo, Japan. The results demonstrated the need to look at critically changing office environments in particular during extended business hours. 2. M A T E R I A L S AND M E T H O D S
A combination of continuous monitoring techniques was assembled. The thermal condition monitor, Climomaster (6511, Kanomax) could measure air velocity, air temperature and relative humidity simultaneously by a single probe. The dust concentrations were determined by means of a laser dust monitor (LD-1H, SIBATA) which measured the intensity of light scattered. For carbon dioxide concentration, the portable non-dispersive infra-red carbon dioxide monitor (UM260L, Komyo) was used. The sound level meter (NL-02, RION) indicated the DC output
636 proportional to the sound pressure level. The measuring instruments were compact and easy to handle, and were installed on a desk at the center of the office room studied. The output data from the instruments were transferred and stored in the digital data recorder (DR-F1, TEAC) or directly transferred to a portable computer (PC98LT, NEC). Figure 1 shows the block diagram of the instruments.
I Laser Dust Monitor l C02 Monitor ]
[ Digital Data Recorder ']
Sound Level Meter Thermal Condition Monitor
Air velocity Air temperature Relativehumiditv
~'JLPortable Computer
Figure 1. Blockdiagram of the measuring instruments. 3. RESULTS
Figure 2 shows the results of the measurement which was carried out in a company office building in the summer season. The mean values obtained over every five minutes are shown. The measurements were made from one evening to the following evening. However, the data plotted on the X-axis range from midnight to 12 p.m. for reasons of convenience. Since this building was erected about 25 years ago, the central heating, ventilating and air-conditioning (HVAC) systems had deteriorated with age, and the temperature was around 28 degrees centigrade even during working hours. Moreover, since the HVAC system was switched off at 6 p.m., the occupants had to work under worsening conditions, such as an increased temperature and a higher level of relative humidity. The air velocity data show that they were using electric fans during extended hours after 6 p.m. The noise data indicate that they worked until 9 p.m when the air temperature was well over 30 degrees centigrade. Figure 3 shows the results of measurements taken in another building which was built about 15 years ago. The HVAC system was operated until 9 p.m. in this building and in comparison to the conditions in the previous building, the air temperature and relative humidity were better controlled. Surprisingly, it was found that the occupants worked until 2 a.m., as indicated by the noise data, and it can also be seen from the analysis of dust data that some people smoked until 2a.m.
637
0.3 Air velosity [m/s]
0.3 fAir velosity [m/s]
0.2
0.2
01
0.1
! -
0 , i I , i~ , T ~ ~ , 35
~-~ 35 30
30 25
25
:Air temperature [°C ]
20
~
LI
~
L I
t
~ I~
~1
~
~ I,
,
I,
,I
,,
70
7O 60
50
50
40
40 301-,
,
0.3
1
,
,
i
,
,
J
,
,
J
,
,
i
,
J
,
,
I
,
301-,
,
,
I
,
,
I
,
,
I
,
,
I
,
,
I
,
,
I
,
,
i
,
,
I
~
~
0.3
irbome dust [mg/m 3]
0.2
0.2
0.1 0
....
I
1500fCO 2 [ppm]
15O0fCO 2 [ppm] ]000[
t 0 l~,
70 6O
~ I
J J I J,
I t L I ~ ~ 1 ~,
I,,
I
,
or,
,
oise [dBA
70 6O
50
,
I , ,
I
,
~
I
i,
[d
5O
0
6
12 Time
18
24
0
6
12 Time
18
24
Figure 2. The results of the continuous measurement Figure3. The results of the continuous measurement (22-23 August, 1989). (24-25 August, 1989).
638 Figure 4 shows the results of the measurements conducted in the previously mentioned 20 year-old building during the autumn season. Since the HVAC system was switched off at 6 p.m., a higher dust concentration because of smoking could be clearly demonstrated. Figure 5 shows the results obtained in the newest building, which opened just in April, 1991. The data for carbon dioxide indicated that the office was not adequately ventilated. The ventilation rate may not have been set at high level, because smoking was prohibited in this office. There was a separated smoking area, but, after the HVAC system was switched off, the contaminated air by smoking was able to enter the office area. Figure 6 shows the results for the smoking space where the air was heavily polluted by tobacco products. This was not originally designed for the sole use of smokers, but, rather, as a general rest area, and non-smokers complained of their inconvenience. They do not utilize the space any more. It was revealed that thermal conditions in the buildings were greatly influenced by the efficiency and the operating time of the HVAC systems, It was also found and that air contamination and noise levels fluctuated in response to the activities of the occupants. Tobacco smoking or diversifying operations of office machines also affected the results significantly. Critical environment conditions were often found when occupants were still working in the office after
0.3 fAir velosity
0.3
0.2
0.2
0.1
0.1
0 .... l_~,J ,_,*n-,, t, ,-I', ,~'t'~ :-t, ,u 3 0 ~ Air . _ ~temperature ..35f [°C]
0
, ~ t ~, t , ,
]lll0~~ICO2 [ppm]
~
t , , I
70 70IN I oise [dB 60
40-
4O 301-, , I , , I , , 0 6
i ,-,-r',~~, I,, 12 18 Time
J , , 30I,, 24 0
" ,,
,i ,l~h,I,,',ll,,I 6
Figure 4. The results of the continuousmeasurement (13-14 November, 1989).
12 Time
18
24
639
0.3
0.3
ir velosity [m/s]
0.2
0.2
0.1
0.1
0 35
, i , ~
_
x
.
.
~
~
0 I- , , II-,,~-J 35
Air temperature [°C]
.
30
30
25
25 j ~ l l l l , ~ l l , l , , I , , I , , I , ,
20 70
o
20 70
60
60
50
50
40
40
30 0.3
sity [m/s]
1.2
rborne dust [mg/m 3]
tj
1
0.2
0.8 0.6 0.4 0.2
o.1 1-,
0 1500,
,
I
i
~
I
~-
~o~ppm~ ~ . ~ t / ~~ .
IlS°°f~o~ppm~
I
,
--41 o
01-,
70 60
70
oise [dB
,
I
,
,
I
j
~
I I
j
~
I
,
,
I
,
,
I
,
,
I
,
,
I
.
60
50
50
40
40 0
6
12 Time
18
24
Figure 5. The results of the continuous measurement (16-17 October, 1991).
30
0
6
12 Time
18
24
Figure6. The results of the continuous measurement (17-18 October, 1991).
640 the HVAC system had been switched off. The quality of the indoor air, such as thermal conditions and contamination levels, became worse to a remarkable extent as the work continued in late evening hours. 4. CONCLUSION The continuous measurement of indoor air quality provides various kinds of important information that cannot otherwise be detected. Therefore, the use of continuous measurement techniques should be considered essential and widely applied in various workplaces including computerized offices. Continuous evaluation will contribute to the enhancement of the amenity of working environment for office workers. REFERENCES
1. J.M. Samet and J.D. Spengler (eds.), Indoor Air Pollution, The Johns Hopkins University Press, Baltimore, 1991. 2. R.A. Wadden and P.A. Scheff, Indoor air pollution, John Wiley & Sons, 1983. 3. K. Kimura, K. Shimakage and M. Saito, J. Sci. Labour, 66 (1990) 545. 4. Working Environmental Improvement Office, Ministry of Labour~ Working Environment Measurement System in Japan (Second edition), Japan Association for Working Environment Measurement, Tokyo, 1991.
IV.3 Human Factors in Display Technology
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
643
Effects of ambient lighting conditions on luminance contrast and color gamut of displays with different technologies Satoru K u b o t a a
aDepartment of Industrial Engineering, Faculty of Engineering, Seikei University, Kichijoji, Musashino-shi, Tokyo; E-mail: kubota@ is.seikei.ac.jp
Abstract The effects of the ambient illuminance and the reflections of environmental luminance on contrast ratio and maximum-chromaticity gamut of displays with different technologies are investigated. The results indicate that the LCDs typically maintain much higher luminance contrast and chromatic contrast under condition of high ambient illumination than CRTs or PDPs do. This attribute make LCDs desirable in a variety of conditions including the uses in vehicular applications and use in outside applications. I. INTRODUCTION With the appearance of flat-panel displays (FPDs), the concern of their ergonomic qualities increases. FPDs are used under a wide range of lighting conditions typical of vehicular environments. Hence, the basic visual parameters of the display, such as luminance contrast and color gamut, must be adequate to maintain acceptable image quality under dynamic lighting conditions. The following two factors are independent, and these lighting environments affect in image quality of the display. One is screen illuminance and the other is reflections of environmental luminance. These reflections from a display screen are added to the luminance emitted from both the foreground and background of the display, and reduce both the luminance contrast and the color gamut. In this study, we investigate the influence of screen illuminance and reflections of environmental luminance on contrast ratio and color gamut for different display technologies, FPDs and conventional cathode-ray tube (CRT) displays. A wide range of the two lighting variables was incorporated to evaluate image quality for displays with different technologies under various lighting environments. 2. SAMPLE DISPLAYS Eight different types of displays were evaluated in this study. Each sample display is the typical one in each display technology. Their specifications and reflection coefficients are given in Table 1. These reflection coefficients and emitted display luminances were measured using the methods described in the Part 7 of ISO 9241 [ 1].
644
Table I Specifications of the sample displays
No.
Emitted Display Luminance(cd/m 2) Surface Treatment White Black LeW LeB
Display Type
Diffuse Reflection Coefficient(cd/m2/lx) White Black
qW
qB
Specular Reflection Coefficinet Small-Source Extended-Source ldeg Source 20deg Source Sr-small Sr-extended
1.
Color Transmissive TFF-LCD Matte finish
110.2
0.78
0.0087
0.0081
0.00018
2.
Color Transmissive TFr-LCD Matte finish
86.3
0.72
0.0039
0.0029
0.01662
0.13130
3.
Color AC-PDP
109.0
1.53
0.0496
0.00840
0.08480
4.
Color CRT
150.0
0
0.0216
0.00250
0.00980
5.
Color CRT
Matte finish
150.0
0
0.0320
0.00580
0.04260
6.
Color CRT
Polish
150.0
0
0.0840
0.05250
0.06640
Matte finish Antireflection coating
7. B/W Reflective TFT-LCD
Matte finish
8. B/W Transmissive STN-LCD
Matte finish
0 81.7
6.60
0.02499
0.0577
0.0096
0.00550
0.08200
0.0048
0.0021
0.06690
0.16000
TFT:Thin-FilmTransistor,LCD:Liquid-CrystalDisplay, PDP:PlasmaDisplay Panel,:STN:Super-TwistedNematic 3. CONTRAST RATIO UNDER A WIDE RANGE OF LIGHTING CONDITIONS 3.1. Effects of screen iiluminance on contrast ratio The main effect of ambient illumination striking the display screen is appears as a reflection that sums with the display image, thereby reducing its luminance contrast. The contrast ratio under ambient illumination is given as follows: LeW + E • qW Cr =
(1)
L e B + E . qB
where LeW and LeB are the emitted display luminance for the highest and lowest luminance logical states, respectively. The terms qW and qB are the diffuse reflection coefficients at the highest and lowest luminance logical states, respectively. The term E is the illuminance incident 200100-
,,.,,
O "z:l ,i-a
Fig.1 Contrast ratio as a function of screen illuminance for displays with different technologies. The plotted numbers indicate the displays listed in Table 1.
ra~
t~
88
10
o
g
,./
L)
1 10
100
1000
10000
Screen Illuminance (Ix)
100000
645 in the plane of the screen. Figure 1 shows the contrast-ratio curves for different displays as a function of screen illuminance, obtained from Equation 1 and the data in Table 1. These results indicate that LCDs maintain superior contrast under higher ambient illumination conditions compared with the rest of emissive type displays. Krantz et al. [2] have already mentioned that "this attribute, along with reduced weight, volume, and power requirements, make LCDs especially desirable for use in vehicular applications". 3.2. Relationship between screen illuminance and reflections of environmental luminance to maintain the contrast greater than 3 The contrast ratios of the displays are also influenced by the reflections of environmental luminance (environmental luminance). In other words, the specular reflected luminance sums with the display image, thereby reducing its luminance contrast. The contrast ratio including superimposed specular and diffuse reflected illuminance is given as follows: L e W + E • q W + L i • S r -extended Cr
=
(2) LeB
+ E • q B + L i • Sr_ex~,~ed
where Li=environment luminance or extended-source luminance, St_extended--specular reflection coefficient for extended source, and all other parameters are the same as defined in Equation 1. From the standpoint of legibility, the contrast ratio of the image, including superimposed specular and diffuse reflected luminances, shall be equal to or greater than 3 [3]. Therefore, Equation 2 may be modified as follows to maintain the contrast ratio at 3: 2000010000.444
44~
-6
o~
;555 55~ ~-6--6----6-----¢,.~_ 1000-
o o
il
~.,. ~ ~
"
100
Fig.2 Relationship between screen i l l u m i n a n c e and e n v i r o n m e n t a l luminace to maintain the contrast ratio at 3. The plotted numbers indicate the displays listed in Table 1.
/ 100
1000
10000
Screen Illuminance (Ix)
100000
646 E (qW- 3qB) + LeW- 3LeB Li ~_
(3)
2Sr_extended Figure 2 shows the relationship between screen illuminance and environmental luminance to maintain the contrast ratio at 3, obtained from Equation 3 and the data in Table 1. This Figure characterize the display's luminance and contrast abilities under a wide range of lighting conditions. These results indicate that the lighting-display interaction is dependent on the reflection properties of the display. 4. COLOR GAMUT AND AMBIENT ILLUMINATION The summation of reflected luminances also causes the chromaticities of the image to shift toward that of the reflected light in the manner shown in Figure 3 and Figure 4. Figure 3 compares the displays' color-production abilities when no ambient illumination is reflected from the display screen. Figure 4 also depicts the color-production abilities but with the reflected illumination added. The magnitudes of these effects can be predicted via luminance and color-mixture calculations. These calculations are as follows.
0.6 .
550
0.5~
570
~
55~
590 8
0
0
0-44~I'~J / "'/'
480 /
o. 4.oW.../ 0.1
450 0
470k 4~
460"NK s~en illuminance is 0 Ix
0
. . . .
I
. . . .
0.1
I
450
380 . . . .
0.2
I
/ s c r ~ illuminance is 1000 Ix
. . . .
0.3
I
. . . .
0.4
I
. . . .
0.5
I
. . . .
0.6
0.7
U'
Fig.3 Maximum-chromaticity gamuts of the displays with no ambient illumination. LCD, PDP, and CRT is coded 2, 3 and 5 in Table 1. The assumed emitted luminance is 100cd/ m E o f the white. The gamut of NTSC standard has been included to facilitate comparisons.
0
. . . .
I
. . . .
0.1
I
380 . . . .
0.2
I
. . . .
0.3
I
. . . .
0.4
I
. . . .
0.5
I
. . . .
0.6
0.7
U' Fig.4 Maximum-chromaticity gamuts of the displays with the reflected illumination added. LCD, PDP, and CRT is coded 2, 3 and 5 in Table 1. The assumed emitted luminance is 100cd/m 2 of the white. The gamut of NTSC standard has been included to facilitate comparisons.
647 The diffuse reflected luminance by the ambient illumination is: (4)
Lr = E • q
where E=illuminance incident in the plane of the screen, q=diffuse reflection coefficient. The tristimulus values of the diffuse reflection are: X r = 9u'i • Lr /
(5) (6) (7)
4v'i
Yr = Lr Zr
= 3Lr(4-
u'i) /
4 v ' i - 5Lr
where u'i and v'i are CIE 1976 UCS chromaticity coordinates of the incident illumination. The sum of the tristimulus values of the internally produced color and the diffuse reflection are simply the sum of the tristimulus values of the two sources. Xn = Xo + Xr
(8)
Yn = Yo + Yr
(9)
Zn = Zo + Zr
(1 O)
where Xo,Yo and Zo are the stimulus values of the display color when ambient illumination is absent. Therefore, the chromaticity coordinates of the internally produced color with ambient are: u ' = 4Xn / (Xn + 15Yn + 3Zn) (11) v ' = 9Yn / (Xn + 15Yn + 3Zn) (12)
f,.) r.~ EZ o
1. . . 0.9 I i i[ -I i i i ::i::i::l 0.8- : i i :iiiii: • CRT ~:~, 0.7-"- i i l i ~ : ~ i ! I iii [[ P D P i~i}iiii[[
o~ 0.4-1 ! C ii!i! I <,~ 0 . 3 -
.
i
i I :: i ::iii i iiiil i!! ii i i~~,
i
i!iil]
i i i
i i i: ~
CRT coded 5 in table 1 LCDcoded2 in :tablel
"~"
PDP coded 3 1 i - -..................... in table
~ki~
[ ~
i il
0.2-
i
o.1 0 O f,.)
O1
! i :::~iiii i i :,,::~ 10 100
1000
Screen Illuminance (Ix)
10000
Fit Color gamut as a function of screen illuminance. The assumed emitted luminance is 100cd/m 2 of the white. The incident lightsource is D65.
648 Post et al. [4] already pointed out that "the maximum-chromaticity gamuts for emissive displays are triangles when ambient illumination is absent and hexagons when ambient illumination is present", as shown in Figure 3 and Figure 4. Figure 5 shows the calculated maximum-chromaticity gamuts as a function of screen illuminance for displays different technologies. This figure indicates that the LCD has superior color-production abilities compared with the CRT displays when the screen illuminance is higher than 300 Ix. This advantage of the LCD is attributable to the low diffuse reflectance of the display screen. 5. CONCLUSION Under the conditions of higher ambient illumination, LCDs typically maintain much higher luminance contrast and chromatic contrast than color CRTs or color PDPs do. Thus, these attributes make LCDs desirable for use in a variety of conditions including vehicular applications and use in outside applications.
Acknowledgment This research was supported by Nissan Science Foundation.
REFERENCES 1. ISO/CD 9241 part 7: Ergonomic requirements for office work with visual display terminals (VDTs), Part 7: Display requirements with reflections,1993 2. Krantz,J.H.,Silverstein,L.D., and Yeh,Yei-Yu: Visibility of Transmissive Liquid Crystal Displays under Dynamic Lighting Conditions, Human Factors,34(5),615-632,1992 3. ISO 9241 part 3: Ergonomic requirements for office work with visual display terminals (VDTs), Part 3: Visual display requirements,1992 4. Post,D.L. and Lloyd,C.J.C: Colour display gamuts and ambient illumination, Displays, 15(1),39-43,1994
Symbiosis of Human and Artifact Y. Anzai, K. Ogawaand H. Mori (Editors) 1995 Elsevier Science B.V.
649
Display User Response by Task Lighting/Office Configuration: Implications for Flat Panel Display Users G. Sweitzer* Department of Architecture, Musashi Institute of Technology 1-28-1 Tamazutsumi, Setagaya-ku, Tokyo 158, Japan Abstract Display user response to two task lighting/office configurations is surveyed in situ. Observations, results of a self-reporting questionnaire, and workplane illuminance measurements form the basis for comparisons. Although neither task lighting system is installed according to recommended practice, the fixed system is more widely used while the potentials of the adjustable system remain unexplored. Workplace layout - affecting display location - and individual user differences determine task lighting needs. Hands-on education is suggested as means to help exploit the tacit knowledge of display users. 1. INTRODUCTION Task lighting is often recommended as a means to satisfy a wide range of office display user needs. In practice this range is limited by lighting hardware and user knowledge. For example, fixed task lighting units may be limited to on/off switching while personally-adjustable units may permit changes in source location, intensity, and even colour temperature. Accordingly, fixed task lighting may prove satisfactory for fixed tasks but not for tasks in various locations or, under otherwise varying lighting conditions, such as daylighting. These latter two conditions are increasingly common, as display tasks have now reached every corner of the office workplace (1). Task lighting is also complicated in workplaces where more than one display is used. In established offices, for example, the first installed display may be a "host" Cathode Ray Tube (CRT) that occupies a designated location. In addition, it may be resting on a disc drive, elevated from the workplane, with the screen aligned more or less vertically in order to limit screen reflections of overhead lighting luminaires. This conforms to recommended practice. An additional display, meanwhile, such as a Flat Panel Display (FPD), may occupy horisontal work surfaces once reserved for reading and writing. These units an be easily moved, however, and the (smaller) screens easily tilted to fit user viewing. Assorted documents, meanwhile, may be found at either side of either display, on the work plane. Such conditions complicate viewing display tasks. * Fellow, Japan Society for the Promotion of Science / U S National Science Foundation
650 Accordingly, task lighting can be complicated. How complicated? How are users coping under such conditions? This pilot study does not adequately answer all of these questions. It does however provide a basis for further study. 2. METHOD The method includes selection of the study site and survey population, design and administration of a self-reporting questionnaire, and illuminance measurements, and recorded observations. A description of each follows.
2.1. Study site The study site is a five story-high computer research facility located in a suburb of Tokyo. This location effectively limits both property costs and worker commuting times. Two similarly-appearing building wings, 'A' and 'B', were completed according to the original master plan, in 1985 and 1989 respectively. The wings are joined on the third level via a glass-enclosed bridges that meet at the central cafeteria. The office areas in each wing are primarily open plan. The floor-to-(finish)ceiling height is 2.6m. The ceiling grid is 2' x2 ° and accommodates both 16-cell lay-in "low-brightness" luminaires and supply air diffusers. The subject task lighting/workplace layouts in the two wings meanwhile differ. The 'A' workplaces are composed of back-to-back rows of standard desks, each separated by a desk-height CRT platform. In addition, each pair of rows is separated by partial-height screens that occlude line of sight. A desk-supported fluorescent tube task light is aligned at the back of each desk, just in front of the partial height partition, which is finished with lightreflective fabric. Both the upward and downward lighting components are diffused by acrylic lenses. This configuration is counter to recommended practice, however, as the downward component causes veiling reflections for users per-forming conventional reading and writing tasks or, reading documents while using the display, angled to the right-hand side. Accordingly, the task light of a neighbor behind may be screen-reflected. Nonetheless, the major complaint of the initial users of this layout concerned the common circulation path behind their backs. In response to this criticism, the open plan layout in wing 'B' is designed to provide users more enclosure. The outcome includes paired rows of rectangular workplaces that open onto a common corridor. Separation from adjacent rows is provided by partial-height screens that occlude line-of-sight; lower screens are used elsewhere (see Figure 1). Each enclosure is occupied by either a group manager, at either end of the row, or shared by two engineers. |
r~
,_~_,
z::~
,_
.
'A' wing
'B' wing 4 - - - - - - ~
......
,
Figure 1. Comparison of open plan office layouts in wings A and B
651 Within each shared area are desktop surfaces on three sides, including two angled corners to accommodate one display each. Nonetheless, two to four displays were found at the workplaces surveyed. In addition, above each of the opposing desktops is a partition-mounted, adjustable task luminaire fitted with a fluorescent tube lamp. While its vertical position is fixed, the extent of its (double-arm) horisontal travel is limited by its bracket mounting position. 2 ~ Survey population The survey is limited to 24 graduate engineers, 23 male and one female, aged 31.7 + 3.2 years and, 12 each from wings 'A' and 'B', on floors three and five respectively. Each group is composed of users located at either end of the open plan area. All workplaces, however, are located more t h a n 5m from a window wall, the generally accepted limit of effective window daylighting. Supervisors occupy these areas on each floor. Accordingly, access to daylighting in the workplaces surveyed is considered negligible. 2~. Self-reporting questionnaire Based on observations of the above-described areas, a self-reporting questionnaire was developed. It solicits forced choices concerning importance of and then satisfaction or dissatisfaction with the following factors: 1) office location to external destinations (i.e. home); 2) location of personal office workplace re internal destinations (i.e. cafeteria); 3) office workplace activities; and 4) office workplace environmental conditions, with a focus on lighting. In addition, questions were asked concerning working habits. The questionnaire, in English, was intended to take not more than 20 minutes to complete. The questionnaire was administered by a single m a n a g e r to each of the engineers. The time frame was before lunch on a Wednesday during late February. Sky conditions were clear and sunny, characteristic of winter conditions in Tokyo. All questionnaires were returned by just after lunch.
2.4. I l l u m i n a n c e m e a s u r e m e n t s Following, illuminance measurements were made at each workplace, on the work surface,display screen, keyboard, and document surfaces. 2,5. Recorded observations Finally, observations were recorded at each workplace concerning user working patterns. 3. RESULTS Results are presented for the 24 workplaces studied. Although the number of responses from each group cannot provide for statistically significant comparisons, the frequencies reported here can provide guidance for additional study. 3.1. Office location Office location re home location is considered important and satisfactory by an overwhelming majority of those surveyed (23/22)respectively.
652 Car, train, and bicycle t r a n s p o r t modes used at least once a week were similary rated (13/10), (12/8), and (7/7) respectively, The reported door-to-door morning and evening commuting times were 32+18 minutes and 30+15 minutes respectively, each well below commuting times common within metropolitan Tokyo. Average arrival time was 0904+52 minutes, reflecting the flex-time policy.
3 ~ Office workplace location Ratings of interior destinations (from personal workplaces) meanwhile varied between the two groups. 'A' Users considered the following destinations to be less i m p o r t a n t t h a t 'B' users: cafeteria, toilet, laboratory facilities, meetings rooms, tea/beverage dispensers, and library. The satisfied/importance ratio for these, however, was about the same for the two groups. Office workplace activities Computer u~e was the activity rated as important by most in groups 'A' and 'B', (11/12) respectively; however, the 'A' users were more satisfied (11/9). For the 'A' and 'B' users rating ~ as important (9/6), all were satisfied. Meanwhile, of those reporting printer use to be important (5/5), few were satisfied (2/1). 3.4. E n v i r o n m e n t a l conditions The environmental condition rated as important by most of the users in each group was air conditioning (9/12); of these, few were satisfied (3/2); Work surface ~trea was rated important by more users in group 'B' (6/11), who have more than twice a much surface area as those in group 'A'; of those, more group 'B' users were satisfied (3/10); Personal space, meanwhile, was rated as important by about the same number in each group (8/7); of those, about half of each group were satisfied (4/4) despite the larger and better defined personal areas in group 'B'; File cabinets, located below the work surface in each group, were rated as important (5/8) and, of those, satisfactory (1/3) by 'A' and 'B' users respectively; Likewise, h a n ~ n g shelves, (from the intervening partitions, which are higher in 'A' than 'B';) were rated as important (2/7) and satisfactory (0/5); Meanwhile, window view, which is considered to be distant in either case, was rated as important (2/7) and satisfactory (1/2) respectively.
3.5. Lighting conditions User responses were solicited separately for ceiling- versus task-lighting of reading, writing, and display tasks. Ceiling lighting for reading was rated as important (8/8) and satisfactory (5/6) for users in groups 'A' and 'B', although task lighting was more often always on in the 'B' group; Nevertheless, task lighting for reading was rated as important (7/11.) and satisfactory (6/8); Likewise, ceiling lighting for writing was rated important (6/7) and satisfactory (4/5) by the same users; meanwhile, task lighting for same was rated as important (6/8) and satisfactory (5/7) respectively.
653 Finally, ceiling lighting for (central) display use was rated important (6/9) and satisfactory (5/7); all of those that rated task lighting important (7/1) for display use were satisfied. 3.6. P e r s o n a l information
The last section of the questionnaire asked users about themselves. While all of the users are right-handed, half consider themselves touch-typists. Of the 15 with corrected vision, 12 wear glasses, the remainder contact lenses. Time spent working with display screen tasks versus total time per day is 6:02±2:34 hours versus 9:50+1:06 total hours respectively. Workplace users in groups 'A' and 'B' reported yes to tiredness at the end of the workday as follows: eyes (9/10); neck (5/6); back (5/3); hands (2/3) and legs (3/1). 3.7. m u m i n a n c e m e a s u r e m e n t s Illuminances were measured at the center of the workplane, the (central) display screen, keyboard, and document, as summarized in Table 1.
Table 1. Summary of Measured Illuminaces in (lux) Group 'A' Group 'B' mean ~ mean Workplane 585 290- 850 449 355- 715 Display screen 184 129- 280 175 112- 250 Keyboard 427 249- 610 337 201 - 409 Document 431 125 - 660 416 276- 560 3 ~ Recorded observations Annotated sketches were made at individual workplaces, noting user work organization, storage habits, and personal expressions. 4. DISCUSSION The office location and reported commuting and arrival times indicate that all of the workers surveyed receive daylight exposure before work, year-round. This is considered essential, especially during morning and evening twilight periods, to adjustment for biological rhythms (2). Many of the those surveyed depart from the office after dark, however, year-round. Office location within the building meanwhile establishes access to window view and daylight, from the workplace as well as via circulation routes enroute to other services. Computer use and reading activites are satisfactorily conducted in most of the workplaces surveyed, but printer use is not. This lack of satisfaction may indicate that printer use was not adequately accounted for during design. Likewise, the widespread dissatisfaction with air conditioning, especially cooling, may indicate that the original cooling capacity is being exceeded due to increased equipment loads. Means to limit electric lighting and equipment
654 loads can thereby aid thermal comfort. Daylighting is a proven means to achieve this, given proper glazing selection and electric light controls (3). Concern for surface area and storage reflects the prevailing conditions in many of the workplaces surveyed. Most users have at their workplace more materials than was designed for. The surplus is typically located on, above, or below the workplane. This limits not only distribution of ceiling and task lighting, but user postures that affect task performance. The fixed task lighting in the more confining 'A' workplaces is used more often than the adjustable task lighting is in the more spacious 'B' workplaces. One explanation is that the dividing partitions are lower in the 'B' workplaces, allowing for wider distribution of ceiling lighting. Another explanation is that the adjustable task lamps are usually fixed in position, against the separating partitions. Accordingly, they function much as the fixed task lights in the 'A' workplaces rather than to their potential. In addition, these lamps open only to one side, complicating the provision of recommended sidelighting for workplaces that include corner display locations to the right or left of the user. This is relevant for CAD users that work with large documents. Such complications could compound symptoms associated with tiredness, especially in the eyes. In addition, the time with display tasks reported far exceeds that recommended. The measured illuminances for workplane, central display, keyboard, and document show great ranges. This represents a lack of control as well as a waste of electricity. Education is suggested as a means to empower users to improve and take responsibility for lighting in their workplaces. Following, a followup survey could indicate whether the task lighting should be replaced. ACKNOW[,EDGEMENTS Kazuo Tsuchiya and Ryohji Yoshitake, Human Factors, IBM Japan Ltd.; Dr. Susumu Saito, Japan National Institute of Industrial Health; Dr. Masanori Shukuyu, Associate Professor, Musashi Institute of Technology
0
g
0
Sweitzer, G., (1995) "Daylighting Potentials in Display Office Workplaces: Japan, the US, and Sweden, Proceedings of The Work With Display Units Conference '94, Milan, North-Holland Terman, M., Fairhurst, S., Periman, B., Levitt, H., & McCluney, R., (1986) "Daylight Depreviation and Replenishment: A Psychobiological Problem with a Naturalistic Solution", Proceedings of the 1986 International Daylighting Conference, Ervin Bales & Ross McCluney, Editors, ASHRAE Sweitzer, G. (1993) User-Adjustable Daylighting Controls for Perimeter VDU Office Workplaces, Doctoral Dissertation, Department of Architectural Lighting, The Royal Institute of Technology, Stockholm
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
Computer Workstations and Ergonomic Standards: Issues in Science and Engineering R.E. Granda a and J. Greeson, Jr. b a IBM Corporation, Poughkeepsie, NY, USA b Ergonomic Solutions Incorporated, Raleigh, NC, USA 1. INTRODUCTION The disciplines of Ergonomics and Human Factors - especially as they relate to issues of Health and Safety in human working environments have b e e n receiving unprecedented emphasis in the last several years. Part of this emphasis is traceable to the established programs and planned objectives of the European Union (EU) in the area of social improvement of individuals in the member states. As part of this effort, the European Union promulgated an EC Directive having the force of law in member states with the expressed purpose of 'protecting' the health and safety of workers in office work environments using computer systems with visual display units (VDUs). The EU Council Directive (29/270/EEC) on the minimum safety and health requirements for work with display screen equipment was issued formally on 29 May 1990 by the European Community Council for implementation by member countries by 31 December 1992. It is a significant document which is having repercussions on the conditions of work and workers and the design of computer workstations (Bevan, 1991) not only in the member nations of the European Union (EU) and its associated members but in other industrialized nations throughout the world - foremost being Japan and the United States. The ramifications of this document and associated ergonomic standards which either have been or are being developed are explored in this paper. Major critical standards are reviewed - with an emphasis on front-of-screen characteristics and the unique interaction requirements posed by CRT and flat panel technologies. Difficulties in value specification and measurement protocols will be highlighted. 2. EU COUNCIL DIRECTIVE (29/270/EEC) Council Directive (29/270/EEC) is commonly called the VDU (Visual Display Unit) Directive, and it describes minimum safety and health requirements to be observed in an office working environment with display screen equipment. The VDU Directive consists of twelve Articles and an Annex containing 17 obligations or requirements.
655
656
The articles outline general provisions and employers' obligations, including exclusions (e.g. typewriters of traditional design, 'portable' systems not in prolonged use, calculators, cash registers, etc.). The ANNEX of the VDU Directive identifies three major areas of the workplace for which minimum requirements shall be satisfied. These major areas and specific items covered in each are: • •
•
Equipment: display screen, keyboard, work desk/surface, work chair. Environment: space requirements, lighting, reflections/glare, noise, heat, electromagnetic emissions, humidity. Operator/equipment interface: task suitable, user adaptable, easy to use, performance feedback, application of software ergonomic principles.
The VDU Directive places obligations on employers; these include the analysis of all workstations for safety and health conditions and the remedy of any identified risks. There are also requirements for informing and training employees in ergonomic matters, planning of daily work routines to reduce workload stress, providing eye examinations and eyeglasses if indicated by the examination. Because of the non-specificity in the Directive and in related legislation of member nations, issues of compliance and adherence to this regulation initially caused some degree of uncertainty amongst employers. However, this has been largely cleared up. An increasing number of employers and workers are favoring products which demonstrate compliance to recognized technical standards, as a meaningful and effective way of complying with the requirements of the VDU Directive. The most widely known and accepted set of standards covering this wide range of human factors and ergonomic requirements is ISO 9241 and several related standards under development by the International Organization for Standardization (ISO). To give further emphasis to the validity of this trend, the Office of the European Commission has acknowledged in January of 1994 that conformance to European Norm 29 241-3 (equivalent to ISO 9241-3) will be considered as meeting the relevant clauses of the VDU Directive Annex. 3. I S O 9 2 4 1 - O V E R V I E W
The International Organization for Standardization (ISO) is developing a multi-part standard - ISO 9241: Ergonomic requirements for office work with visual display terminals (VDTs). This standard addresses all of the major items listed in the VDU Directive Annex with the exception of electromagnetic emissions. The standard can be broken down into the following categories: General introduction (Part 1) Guidance on task requirements (Part 2) Equipment requirements (Parts 3-4, 7-9) • Visual displays • Keyboards • Display reflections
657
•
Displayed colors Non-keyboard input devices Environment requirements (Pads 5 - 6 ) • Workstation layout/posture • Environment, physical Software requirements (Pads 10- 17) • Dialogue principles • Usability and measures • Information presentation • User guidance • Menu dialogues • Command dialogues • Direct manipulation dialogues • Form-filling dialogues •
•
•
Those parts dealing with the front-of'screen characteristics of computer displays/monitors, although intended to be 'technology-free' are recognized as dominated by CRT-oriented technology. As ISO 9241-3 achieved international approval, it was recognized that a flat panel technology presented significant issues in tradeoffs and metrology. Therefore, another major standard is being developed to cover the human factors and ergonomic requirements of flat panel displays (ISO 13406-2). This work represents an extremely ambitious attempt to integrate the numerous, and often isolated findings in human factors, visual perception, human vision, optics and color theory into a cohesive set of workstation requirements for the advantage of the worker/user. The rest of this paper will review the major critical human factors/ergonomic requirements that have been specified in the standards - especially those dealing with the front-of-screen characteristics of displays/monitors for both CRT and flat panel technologies. 4. ISO
9241-3: VISUAL
DISPLAY
REQUIREMENTS
The ergonomics of any display system consists of the interaction of the user and three major elements - the equipment, the application and the environment. Change any of the elements and the specific requirements of the other parts of this system may change - significantly. Recognition of this "fact" is underscored throughout ISO 9241 and especially in Part 3 which bases demonstration of compliance on the "configuration of hardware, software and workstation elements" and not on the individual elements even if 'each such element shall be shown by its supplier to comply individually..". ISO 9241-3 contains more than 20 requirements related to front-of-screen characteristics of visual display terminals. These include luminance profiles and contrast, raster modulation, flicker and jitter constraints, orthogonality, uniformity and linearity definitions, character and word formats and viewing considerations. As indicated above, although not explicit, the specific values contained in the requirements are oriented to CRT technology driven displays.
658
To meet the requirements of this standard, involves consideration of a number of engineering and software parameters with subsequent tradeoffs; some of the factors are spot size and pixei pitch ratio, font design, shadow masks, and display luminances. Many of the specifications in ISO 9241-3 are based on research conducted originally with paper and in most cases verified to one degree or another on CRT displays. Hence, we often hear the term paper-like quality in describing visual objectives for presentation of text on the display screen. So for example, the formation of text character sets and their viewing size requirements (16 to 22 minutes of arc) are based on 'paper' research (Tinker, 1944). One of the major requirements of ISO 9241-3 is for flicker. Although specified as a mandatory statement, agreement on measurement was not achieved; as a result, several methods are included as informative sections in the standard. The flicker "problem" illustrates that the working groups in ISO are pressing the state-of-the-art in requirements setting. A mandatory requirement was set without the usual quantitative metric. Doing so may be justified in the case of flicker, but is often unwise. Unintended side effects of such a practice may actually prevent optimization of workplace ergonomics. Requirements based on paper-based and CRT-based research may not transfer without modification to flat panel workstations. 5. ISO 13 406-2: FLAT PANEL DISPLAY ERGONOMIC REQUIREMENTS
ISO 13 406-2 is progressing through the standards process to achieve the status of international standard. The scope of this standard is similar to ISO 9241-3,-7 and -8 (ie. front-of-screen, reflections, color). The same parameters are considered - but value specifications for both requirements and recommendations are being reevaluated and revised, as appropriate to take into consideration the impact of this specific engineering technology upon the visual system of human users. We believe that "flat-panel" workplaces are different from CRT-based display workplaces; certainly the range of environments that flat panels are operating in are significantly more diverse that those of CRTs - and this fact alone changes the equation that we implied in the Introduction. 5.1 Portability Flat panel technology provides the engineering capability to package a device that is suited for portability and movement to a wide range of working environments - changing the physical and ergonomic demands on visual workstations. For example, the illumination of an artificially illuminated office is generally between 200 and 1000 lux. Portability may imply a range of 20 to 10 000 lux. Likewise, the temperature range is extended from 15 - 25 degree C to 5 - 35. In addition, specifications for viewing distances and declination angles need to be re-evaluated. The German DIN 66 234 requires a minimum viewing distance of 500 mm. For flat panels this should be reduced to 350 mm; DIN also restricts the screen angle to 75-95 degrees (ie. approximately perpendicular to the working surface). Clearly, flat panels must be allowed to lay flat on the surface (ie. 0 degrees).
659
5.2 Visual Limitations The light from a display screen comes from several sources - both wanted and unwanted (eg. light reflected from the room). The diffuse (all-directional) portion of the light uniformly reduces contrast. The usually curved CRT screen collects room light from a broader area; the flatter, newer CRTs minimize this effect. Flat panels with very flat screens have the advantage of a smaller collection solid angle, but are constructed of a stack of transparent, or semi-transparent materials, - each differing in index Of reflection and producing a reflection at each interface. The luminance of regular or specular reflection(s) depends (among other things) on the luminous and size of the source that is imaged. At some point, contrast suffers, thus making the repositioning of the panel-eye angle to minimize unwanted reflections very important feature of the technology. 5.3 Anisotropy CRTs and most non-LCD panels are approximately isotropic in both emission and reflection. Most LCD Panels have the property of optical anisotropy. To be fully satisfactory for more than one viewer, a flat panel display would have uniform optical properties over an inclination angle range of 0 to 40 (and best at 60) degrees at any azimuth. This means that two viewers could simultaneously 'see' the information clearly and comfortably. LCD panels have significant optical properties that vary with both azimuth and inclination angle. This produces visual effects relative to luminance and color points and therefore impacts their use in software applications. 5.4 Pixel Limitations All flat panels have discrete pixels. In CRTs, pixels can be set close enough to one another that a relatively uniform visual field exists when all pixels are set to their bright, white state. The 'worst' modulation in this condition in ISO 9241-3 is a modulation of 40% for monochrome and 70% for multicolor. All flat panels exhibit 100%. We believe that the lack of a flat field is important whenever image quality is critical. 5.5 Flicker and Jitter The requirement for flicker for both ISO display standards will most likely be identical - the image shall be free of flicker to at least 90% of the user population. The technology of flat panels presents special challenges to meeting this requirement. LCDs have a relatively long image formation time. As a result, the luminance modulation at the refresh rate of the display is low; because of the interactions between the write times to the display and bias stability, luminance modulation arises and flicker results. This effect is anisotropic and image content sensitive. A very common CRT ergonomic issue is jitter. The trend toward positive image sense (black on white) and brighter images has forced CRT refresh rate away from line synchronism. Images that move at the difference frequency (between the power line and refresh) cannot be tolerated. This effect is much more important to user comfort and usability than flicker. There is no such effect in flat panels.
660
6. DISCUSSION and SUMMARY
Human Factors, as a scientific / technical discipline, has important contributions to make in securing effective working environments using technologically advanced equipment. The nature of these contributions need to be explored and applied carefully - especially in the context of international standards with the goal of convergence on a basic set of ergonomic requirements. The potential impacts on emerging technologies as well as on the future of human factors as an identifiable scientific~technical discipline cannot be over-emphasized. The success of ergonomic standards such as ISO 9241 and ISO 13 406 will be based on the ability to identify the best scientific data, incorporate it in an intelligent, reasonable manner and integrate it with other pertinent requirements. A major challenge lies in distinguishing between solid engineering and scientific data and preliminary observations, principles and theories. It is important to recognize that statistical significance is necessary, but not sufficient to move theoretical principles / constructs from the research literature to the requirement clauses of a standard. REFERENCES
Bevan, Nigel, 1991. Human Aspects in computing: Design and Use of Interactive Systems and Work with Terminals. H.J Bullinger (Ed.) Standards relevant to European Directives for display terminals (pp.533-537) Elsevier Science Publishers B.V. DIN 66 234. German National Standard on Display Workstations. ISO International Standard 9241 - Ergonomic Requirements for Office Work with Visual Display Terminals, Part 3: Visual Display Requirements. (15 July 1992). ISO Committee Draft Standard ISO/CD 13406-2: Ergonomic Requirements for Office Work with Visual Display Units Employing Flat Panel Technology. Reference # ISO/TC 159/SC 4 - N 302 ( 20 January 1995). Tinker, Miles A., Criteria for Determining the readability of type faces. (pp.385-396), Journal of Educational Psychology, 35:7 ( October 1944).
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
661
Measurement of TFT/LCD Flicker for ISO Compliance Ryohji Yoshitake a and Rieko Kataoka b aHuman Factors, Yamato Laboratory, IBM Japan Ltd., 1623-14, Shimo-tsuruma, Yamato-shi, Kanagawa-ken, 242, Japan bNotebook Dev., Yamato Laboratory, IBM Japan Ltd., 1623-14, Shimo-tsuruma, Yamato-shi, Kanagawa-ken, 242, Japan Abstract
Flicker is one of the major concerns in the design of computer display, and there have been many research reports on measuring and predicting flicker in order to provide users with flicker-free displays. Most of the previous studies, however, have been based on CRT displays. In this study, flicker on a thin-film transistor liquid crystal display (TFT/LCD) was measured by using a subjective evaluation method. The purpose of the study was to ascertain whether the TFT /LCD conform to ISO standards on flicker, and to determine the main factors affecting flicker on the TFT/LCD. The results indicate that the TFT/LCD used in the study met the ISO standards at frame frequency as low as 35 Hz when all-pixels-on patterns were displayed. It was found that the common voltage (Vcom) is one of the most important factors affecting flicker on the TFT/LCD, and that the Vcom should be well-balanced to make the TFT/LCD flicker-free. It was also found that the analytic method proposed in ISO/CD 13406-2 was not always appropriate for predicting flicker on the TFT/LCD. 1. Introduction
The use of visual display terminals (VDTs) has introduced the problem of flicker perception into the workplace. The many studies carried out in response to the need for flicker-free display, have proposed various methods for evaluating screen flicker and for predicting whether a display will be flicker-free. Farrell et al. [1-4] have established an analytical method for predicting screen flicker on CRT displays. This method is useful for display engineers, because it allows them to predict the degree of flicker at the beginning of the design phase by calculating some parameters such as the refresh rate, phosphor, and screen luminance. It is also given in an informative annex in ISO 9241-3 [5]. However, it is technology-dependent. Rogowitz [6,7] has developed an empirical technique for measuring perceived flicker on refresh displays, while
662 Chaplin and Freemantle [8] have built an objective method for measuring flicker on VDTs. Both methods are basically technology-independent, and have also proven to be useful tools in display development and product assurance. Most of the measured results, however, have been for CRT displays. Since the late 1980s, a large number of liquid crystal displays (LCDs) have been developed, and their market share has grown remarkably. The retention time in thin-film transistor/LCDs (TFT/LCDs) is much longer than the corresponding phosphor persistence time in most CRT displays. However, it is well known to manufacturers of TFT/LCDs that there are various factors affecting flicker on TFT/LCDs. They have therefore been urgently requested to confirm the assumption that people do not see flicker on TFT/LCDs. We conducted a subjective evaluation to confirm this assumption, and to determine the main factors affecting flicker on TFT/LCDs. Since ISO 9241-3 and ISO/CD 13406-2 [9] require displays to be flicker-free, we used the ISO standard as a criterion for determining whether a display is flicker-free. The Flicker Measurement Technique (FMT) by Rogowitz [6,7] is one of the most systematic methods for determining whether a display is flicker-free, but it is time-consuming and not suitable for measuring large number of displays. In this experiment, a large number of display conditions on a TFT/LCD were shown to many participants, because one of the purposes of this study was to determine the main factors affecting flicker on the TFT/LCD. In addition, the luminance-time function of the TFT/LCD was measured and some calculations were carried out in accordance with a decision method which was proposed in ISO/CD 13406-2. The calculation results were compared with those of subjective evaluation. 2. ISO requirement on flicker ISO 9241-3 states that "the image shall be free of flicker to at least 90% of the user population," and two predicting/measuring methods are proposed in informative annexes A and B. ISO/CD 13406-2 repeats the statement, and proposes a decision method that is an extension of the "analytical techniques for predicting screen flicker" in annex A of ISO 9241-3. In our subjective evaluation, therefore, the criterion for flickerfree display was also set according to the perceptions of 90% of the test subjects.
Table 1. Experimental conditions Frame frequency Displayed pattern
6 2
Displayed gray level Common voltage (Vcom)
3 4
35, 40, 45, 50, 55, 60 Hz General pattern (all pixels on) Stressed pattern (every other sub-pixel on) 8th, 14th, and 16th levels out of 16 A: well-balanced B: 2% shifted from A C: 5% shifted from A D: 18% shifted from A
663 3. Subjective evaluation 3.1 Experimental method
Fifty Japanese people in the 20-39 age range participated in this experiment. The individuals had near-point visual acuities no worse than 0.8. The color TFT/LCD for the IBM ThinkPad 720C, with a diagonal size of 10.4 inches, was used in this experiment. The TFT /LCD was driven by a technique so called "column inversion method". The level of ambient illumination was 250 lx at the surface of the display. A chin rest was used in order to control the viewing conditions. The chin rest was adjusted so that the viewing distance was 500 mm and the eye-height of each subject was leveled at the top-line-height of the displayed image. The screen tilt angle of the display was set perpendicular to the surface of a desk. Table I shows a list of the experimental conditions in which the display was shown to subject in this experiment. There were 144 distinct conditions for each subject. Two display patterns were chosen: the all-pixels-on pattern, which is called the general pattern (GP) in this paper, and the vertical line pattern, called the stressed pattern (SP), in which flicker is liable to appear in the TFT/LCD. In the latter, vertical lines were drawn on every other sub-pixel on the display. Three luminance levels were used in the experiment: the 8th and 14th gray levels (GL) and the maximum (16th) gray level, since it is difficult for LCDs to keep the brightness balanced at the middle gray levels. The luminances of the 16GL and 8GL were 87 c d / m 2 and 34 c d / m 2, respectively in the general pattern. In the case of active-matrix LCDs such as TFT/LCDs, the charge voltage of each frame needs to be carefully-balanced by adjusting the common voltage (Vcom). Since a shift of the Vcom causes flicker, the four levels of voltage settings shown in Table I were employed. lOO ISO criterion (90%)
90 80 --4)
70
..&
>
O~
s
GP16 (Vcom:A)
_.e 60
. . . . -,~---
GP14 (Vcom:A)
~) 50
- - El- -
GP08 (Vcom:A)
!._
-'
40
o
30
.... i""
SP14 (Vcom :A)
u.
20
--
SP08 (Vcom:A)
"
w
.,,.., ..,.,,
I1"-
SP16 (Vcom:A)
10 0 30
.
. . . . ..
_--m
....
•
....
35
III ....
m
40 45 50 55 60 Frequency (Hz) Fig 1. Percentages of subjects who did not perceive flicker at each frame frequency (Vcom:A)
664
loo! 90 A
v ==.
-~-~
ISO criterion (90%)
8070
GP16 (60 Hz)
o
> o m
60
o o
50
••
40
'--'a'v,,-~,,,
0---
GP14 (60 Hz)
• '~•• . ~ Ik.
' O~
•
o
o I,I. °m
30
. . . .
i i - - -
GP16 (35 Hz) GP14 (35 Hz)
20 "'-',':,,~_.__. ---Cl---
10 0 ' 0
GP08 (60 Hz)
..,
I
,
5
i
i
i
10
15
20
GP08 (35 Hz)
Shift of Vcom (%)
Fig 2. Percentages of subjects did not perceive flicker as the Vcom shifted (general pattern)
Before the experiment, the critical flicker frequency (CFF) of each subject was measured. The procedure and equipment used in the CFF measurement were the same as those used by Rogowitz [7]. The mean frequency for all 50 subjects was 48.1 Hz and the standard deviation was 3.5 Hz. Each subject was asked the simple question, "Do you see flicker on the display ?" in all conditions, and was instructed to answer only "Yes" or "No." The subject was allowed to look anywhere on the display, but was told not to move the line of sight outside of the display. The patterns were presented to each subject in random order, for balance. 3.2 R e s u l t s and d i s c u s s i o n s
Since the mean of the CFFs in this experiment was quite significantly higher than in the previous study [6], we decided to judge in this study that the display would conform to the ISO standard if 90% of the subjects participating in the experiment did not perceive flicker in the pattern. In the case of TFT/LCDs, the Vcom should be well balanced in order to minimize the flicker during the manufacturing process. Figure 1 shows the results of the experiment using a display with well-balanced Vcom (A). When the general pattern (GP) is presented, the display meets the ISO requirements at frame frequencies of 35 Hz and higher. Although the stressed pattern (SP) is unlikely to be seen in a typical office, it needs to be recognized that flicker can be seen by 94% of subjects at a frame frequency of 60 Hz at the middle gray levels, even if the Vcom is well-balanced. The effect of Vcom on flicker is shown in Fig.2. In the condition of GP-08GL, a shift of only 2 percent in the Vcom made the display incompliant with the ISO standard at a
665 frame frequency of 60 Hz. This result suggested that fine tuning of the Vcom is essential to make the TFT/LCD flicker-free. 4. Measurement in an Analytic Model ISO 9241-3 and ISO/CD 13406-2 propose analytical methods for predicting flicker that are based on research into human flicker sensitivity [10]. Following the subjective evaluation, we measured the luminance-time functions for most of the patterns used in the experiment. Some of the results of the analytic method did not match those of the subjective evaluation. For example, even though "Eobs" was calculated at 1.4 td in the condition of GP-08GL (Vcom:C), fourteen percent of the subjects perceived flicker at 60 Hz. This problem seems to be due to the temporal instability of the luminance of subpixels. Currently, the sub-pixels of the TFT/LCD used in this study are refreshed (the polarity of the charge voltage is reversed) every other column, that is, the odd columns and even columns are refreshed independently. When the Vcom is well tuned, all subpixels will have almost equal luminance. If the Vcom is not balanced, sub-pixels will have different luminances, but as the TFT/LCD is refreshed, the luminance of the subpixels will appear to be reversed in odd and even columns. The shift of Vcom will result in an increase of the luminance for a column of sub-pixels and a decrease of the luminance for an adjacent column, and this will occur every other column. If the change in luminance is large enough, it may be perceived as flicker by subjects. However, the area average luminance of the TFT/LCD may become so small that it cannot be detected by a photometer. The editor of ISO/CD 13406-2 seems to have noticed some limitations of this analytic method. The result of this study using TFT/ LCDs may show one of the limitations. Instability is often easily noticed near the edge of a screen even when observed with central vision. This result suggests that the luminance-time profile should be measured at the various locations on the TFT/LCD. This phenomenon may be detected if it is measured for various viewing angles, as described in ISO/CD 13406-2. Further study and discussion are required. 5. Conclusion It was confirmed that the flicker of the TFT/LCD used in this study exceeded the requirement of ISO 9241-3 and ISO/CD 13406-2 if the general pattern was displayed. It was also confirmed that there were several factors causing flicker, and that such factors as the displayed pattern, the gray levels, the setting of the Vcom, and the frame frequency were all closely related to flicker perception. Since the displayed pattern and the gray levels depend on the applications, the method of setting the Vcom and frame frequency should be carefully selected to allow the design of flicker-free displays. The results indicated that TFT/LCDs with the general pattern were flicker-free at frame frequencies as low as 35 Hz if the Vcom was tuned correctly, but that if the Vcom was shifted only a little, flicker became clearly perceptible. This suggested that tuning the
666 Vcom should be accorded first priority in eliminating flicker. Though both TFT/LCDs and CRTs are refresh displays, they have different orders of priority for items and factors that must be taken into consideration. In the case of CRT displays, phosphor persistence, refresh frequencies, and maximum luminance are very important. On the other hand, in TFT/LCDs, flicker is mainly caused by shift of the Vcom rather than by the refresh frequency, and should be checked at the medium gray level rather than at the maximum luminance. Flicker perception on TFT / LCDs may be influenced not only by the above-mentioned factors, but also by the retention characteristics of liquid crystal and TFTs, driving methods, frame rate control (FRC) and so on. Further research on these factors is needed. References 1. Farrell, J.E.: An Analytical Method for Predicting Perceived Flicker, Behaviour and Information Technology, 5(4), 349-358, 1986. 2. Farrell, J.E.: Objective Method for Evaluating Screen Flicker. Proceedings of WWDU86, 449-460, 1987. 3. Farrell, J.E.: Predicting Flicker Thresholds for Video Display Terminals. Proceedings of the SID, 28(4), 449-453, 1987. 4. Farrell, J. E., E. J. Casson, C. R. Haynie, and B. L. Benson: Designing Flicker-Free Video Display Terminals. Displays, July, 115-122, 1988. 5. ISO 9241-3: Office Work with Visual Display Terminals (VDTs) - Visual Display Requirements, 1992. 6. Rogowitz, B. E.: Measuring Perceived Flicker on Visual Displays. Ergonomics and Health in Modern Offices. London:Tylor & Francis, pp. 285-293, 1984. 7. Rogowitz, B. E.: A Practical Guide to Flicker Measurement: Using the Flicker-Matching Technique. Behavior and Information Technology, Vol. 5, No. 4, 359-373, 1986. 8. Chaplin R. and R. A. Freemantle: Measurement of Perceived Flicker in VDU Products. Displays, 8(1), 22-28, 1987. 9. ISO/CD 13406-2: Ergonomic Requirements for Work with Visual Display Units Employing Flat Panel Technology. N280 Rev, 1994-08-15. 10. Kelly, D. H.: Visual Responses to Time-Dependent Stimuli. I. Amplitude Sensitivity Measurements. J. Opt. Soc. Amer., 49(4), 422-429, 1961.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
667
A Psychometric Scale of TFT/LCDs with a Few Defecting Sub-pixels Tamura, Tohru a and Gohda, Yuhji b aHuman Factors, Yamato Laboratory, IBM Japan., 1623-14, Shimotsuruma, Yamato-shi, Kanagawa-ken, Japan. bDisplay Technology, Yamato Laboratory, IBM Japan., 1623-14, Shimotsuruma, Yamato-shi, Kanagawa-ken, Japan.
1. Abstract This paper shows that a scaling algorithm of defecting sub-pixels in thin-film transistor / liquidcrystal displays, TFT/LCDs. Some of TFT/LCDs have a small number of defecting sub-pixels, point defect, which do not work. A psychometric scale of the quality of TFT/LCDs which contain a few defecting sub-pixels is derived from the luminance, the color, the location and the number of defecting sub-pixels. The effect of clustering defects is also considered. The score is named Total Score. An experiment was performed to assess the correlation between the Total Score and users' judgment of quality of TFT/LCDs with a few defecting sub-pixels. The results indicated that fairly good correlation between the Total Score and users' judgment would be expected.
2. Introduction Thin-film transistor / liquid-crystal display, TFT/LCD, is one of the promising technologies in LCDs. TFT/LCD has a thin film transistor on every addressable sub-pixel to provide many advanced features, which are high contrast, fast response, good color reproducibility and good screen uniformity.I), 2) The number of sub-pixels on VGA resolution are nearly one million and a small number of sub-pixels which do not work are sometimes contained. If only TFT/LCDs which have no defecting sub-pixel are to be
accepted, tens
of percentage of
TFT/LCDs must be thrown away for these very small defects. This is one of the reasons of preventing to reduce costs. Therefore, manufacturers ship TFT/LCDs which have a few defecting sub-pixels if those TFT/LCDs pass their own inspection standards. The inspection standards should correlate users' judgment of the amount of fault of defecting sub-pixels on
668 TFT/LCDs, however, there may be no such standards. In order to establish inspection standards which correlates users' judgment, many factors such as the luminance, the color, the location and the number of defecting sub-pixels should be considered. And the distance between defecting sub-pixels (clustering) affects users' judgment considerably. Each manufacturer has its own inspection standards, however, those standards may not consider all factors and not be expected good correlation to users' judgment. We propose a scaling algorithm of defects which contains above all factors to derive a score in order to establish a new inspection standard which shows better correlation to users' judgment. 3. Total Score
This section describes the algorithm which derives Total Score from the luminance, the color, the location, the number of defecting sub-pixels and the distance of each defecting sub-pixels. Higher Total Score shows that the defecting sub-pixels are more conspicuous and then worse
quality. Firstly, the luminance score (S L) is calculated by the following equation. S L =alog(L)+b
(1)
Where, L is the luminance of the defecting sub-pixel, a and b are the coefficients which are determined by the color of the defecting sub-pixel. It has been reported that the coefficients are the smallest values when the color is blue and almost same larger values when the color is red or green.3), 4) This equation shows that higher luminance cause higher score. And a blue color defect score is the lowest when the defect has the same luminance as a red or a green defect. Secondly, the location score (Sp) is derived from the distance between the center of the display and the defecting sub-pixel d by the following equation. S e - C 1 0 -ma
(2)
Where, C and D are the coefficients which are determined where the defecting sub-pixel exists. A TFT/LCD is divided into four areas by a horizontal and a vertical line passing the cemer of the TFT/LCD. C and D are determined in each area and being the smallest D of the upper leR area. 3) This means the defecting sub-pixels which locate in the upper left area get higher scores than those in the other areas.
669 Lastly, the defect score S d is calculated by multiplying the luminance score by the location score.
s~ = s~ × s~
(3)
The defect score of every defecting siab-pixel on a TFT/LCD is calculated by following the above procedure. Total Score Ts of a TFT/LCD is calculated by the following equation. Ts - ~ Sai + S c
(4)
Where, S di is the defect score of the i th defect. S c is the cluster score determined by the following equation. Sc - X X % min(Sa,,S, )
(5)
l>j
Where, cij is the function of the distance between the i
th and j th defecting sub-pixel
shown in Fig l. The function of % has the largest value where the distance from 1 mm to 6 mm and smaller values in both the distance is less than 1 mm and more than 6 mm.
1.2 1 0.8 0.6 0.4 0.2 0 0
5
10
15
20
Distance (mm)
Figure 1. The cluster coefficient plotted against the distance between i th and j th defect.
4. Algorithm Assessment This section describes the experiment to assess the correlation between users' judgment and Total Score.
670
4.1. Methods Two TFT/LCDs with a few defecting sub-pixels are presented side by side to the subjects to ask which TFT/LCD is worse cause of the defecting sub-pixels. Total Scores are calculated for both TFT/LCDs. If the Total Score of the TFT/LCD which the subject judged worse is higher than the other, Total Score and users' judgment are the same. If the Total Score is lower, Total Score and users' judgment are different. Total 180 pairs are prepared for the assessment by S/W simulation.
4.2. Experimental Conditions The experimental conditions are summarized in Table 1. The luminance, the color and the location of the defecting sub-pixels are determined randomly. Table 2 shows the number of the testing pairs and difference of Total Scores. Table 1 Experimental Conditions Illuminance Viewing Distance Number of Defecting Sub-pixels Total Number of Test Pairs
Table 2 Difference of Score and Number of Test Pairs Score Difference (Dif) Number of Pairs 0 < Dif< 10 22 10 < Dif< 20 15 20 < Dif < 30 12 30 < Dif < 40 14 40 < Dif < 50 16 50 < Dif < 60 11 60 < Dif < 70 6 70 < Dif < 80 15 80 < Dif < 90 7 90 < Dif < 100 12 100 < Dif< 150 36 150 < Dif 14 Total 180 m
m
Conditions 5001x 500mm 4 to 8 180
671
4.3. Subjects Seven people who use PCs with CRT monitors for daily work took part in the test. All have normal color vision. One of them is female. Their ages are the range from 20s to 40s. Three of them performed the test twice and then total ten trials per one testing pair were tested.
4.4. Results and Discussion The ratios of trials of the same judgment between Total Score and subjects, CORRECT are shown in Table 3 against the difference of the Total Scores. CORRECT is 0.61 in the range of Difbetween 0 to 10 and increases 0.99 where the Difis more than 150. Total CORRECT of all testing pairs is 0.81. This indicates that Total Score and subjects' judgments are the same in 81% of trials. Table 3 Summary of the results. Score Difference (Dif)
CORRECT
STABILITY
ST-COR
0.61 0.71 0.67 0.85 0.82 0.72 0.62 0.83 0.93 0.93 0.93 0.99 0.81
0.80 0.81 0.88 0.88 0.86 0.85 0.83 0.87 0.94 0.93 0.95 0.99 0.89
0.19 0.10 0.21 0.03 0.04 0.13 0.21 0.04 0.01 0.00 0.02 0.00 0.08
0 < Dif< 10 10 < Dif< 20 20 < Dif< 30 30 < Dif < 40 40 < Dif < 50 50 < Dif< 60 60 < Dif < 70 70 < Dif < 80 80 < Dif< 90 90 < Dif < 100 100 < Dif < 150 150 < Dif Total m
To discuss the discrepancy between Total Score and subjects' judgments, STABILITY is introduced by the following equation.
max(XN~(A),XNi(,)) STABILITY =
t
l
1
Where, N i is the number of trials of the i th testing pair. Ni(A) is the number of trials where subjects judged that 'A' defecting TFT/LCD is worse than 'B' and N;(,) is the number of 'B'
672 defecting TFT/LCD is judged worse. Therefore, if ,for example, 'A' defecting TFT/LCD is judged worse in all trails, STABILITY is 1. If 'A' is judged worse in the half number of trials and 'B' is judged worse in the other half trials, STABILITY is 0.5. This STABILITY index may indicates how judgments are stable among the subjects and trials. If the goal is set that "More users judge the TFT/LCD which has larger Total Score is worse than that has smaller Total Score", difference between STABILITY and CORRECT becomes Zero. Therefore, The difference between STABILITY and CORRECT is considered missing rate. This missing rates are also summarized in Table 3 as ST-COR. The Total missing rate is 0.08. This missing rate is much smaller than that if it is assumed that the quality of TFT/LCDs having more number of defecting sub-pixels is worse. Only the number of defecting sub-pixels as the inspection standard may be generally adopted by manufacturers. Therefore, it is shown that Total Score is one of the better scales to qualify TFT/LCDs with a few defecting sub-pixels. 5. Conclusions
Total Score is proposed to scale the quality of TFT/LCDs with a small number of defecting sub-pixels. Total Score consists of the luminance, the color, the location, the number and cluster of the defecting sub-pixels. Total Score is expected to correlates users' judgments fairly well and Total Score is of use as a scale of TFT/LCDs with defecting sub-pixels. References
1) Eugen Munteanu: "Integration of Flat-Panel Displays into Portable Computers", 1994 SID International Symposium, Seminar Lecture Notes F-2, 1994. 2) Mikoshiba Shigeo et al.: " Information Display", Journal of ITE Japan, Vol.48, No7, 770777, 1994. 3) Tamura et al.: "A Psychometric Inspection Method on Point Defect of TFT/LCD", The Japanese Journal of Ergonomics, Vol.29, 444-445, 1993. 4) Tamura: "Psychometric Evaluation of Point Defects on TFT/LCD", ITEC'93, 31-32, 1993.
IV.4 Psychosocial Stress among VDU Workers
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
Research Frameworks of Stress - Impacts of Computerization Computer
Workers
675
a m o n g VDU W o r k e r s and Task Characteristics
of
-
Yuko Fuj i g a k i , Ph. D. Dep. I n f o r m a t i o n S c i e n c e , U n i v e r s i t y of Tokyo, 3 - 8 - 1 , Komaba Meguro, Tokyo, 153 JAPAN
This
paper
psychosocial research
d e s c r i b e d t h e r e s e a r c h frameworks of
stress
questions
computerization
on
characteristics
of
and work c h a r a c t e r i s t i c s were
examined"
stress
a
among
work
on
stress
studies workers.
on
impacts
question
p r o b l e m and a q u e s t i o n
computer
the VDU
on e f f e c t s problems.
task
Conceptual questions
Introduction
When we c o n d u c t t h e r e s e a r c h of s t r e s s two r e s e a r c h q u e s t i o n s . ~Q1. Are t h e r e
The question before state),
among VDU w o r k e r s ,
any i m p a c t s of c o m p u t e r i z a t i o n on s t r e s s
~Q2. Are t h e r e any e f f e c t s stress problems?
of t a s k c h a r a c t e r i s t i c s
there
first question is mainly related to o f f i c e workers. s h o u l d examine t h e time f a c t o r s , c o n s i d e r i n g t h e s t r e s s the and
implementation, long after
just
after
the
On t h e c o n t r a r y , characteristics considers
state).
workers, e n g i n e e r s , managers, cross-sectional approach.
in
task
This state
We have
the second q u e s t i o n deals with the e f f e c t s differences
on
implementation(transition
the implementation(stable
of computer work on s t r e s s the
are
problems?
of c o m p u t e r work
c o n s i d e r c a r e f u l l y t h e d i f f e r e n c e b e t w e e n t h e s t r e s s of t r a n s i t i o n and t h o s e of s t a b l e s t a t e , e x a m i n i n g t h e job m a s t e r y of w o r k e r s .
It
of
of
frameworks and s y n t h e s i s in t h e s t u d y of t h e s e two r e s e a r c h were shown w i t h r e v i e w s of f i n d i n g s of p r e v i o u s s t u d i e s .
1.
on Two
p r o b l e m in t h e s t a b l e characteristics
and o t h e r t e c h n i c i a n s
with
among
to
state of t a s k state. office
computers
in
676 2. Conceptual Frameworks -A Review and SynthesisFigure 1 described the framework-dlfference between the two research questions. The research question-i deals with the change in time longitudinally whereas the research questlon-2 deals with the comparison in the cross-sectlonal approach. R.Q.2. Effect of Task characteristics
transition s t a t e - "'
~ b
,
efore
i
stabie state I
~ ~ d u r i n g <--+ a entation ~
. . .
time
cross-sectional
computerization
Figure I. The framework-difference between the two research questions
2.1. Impact of computerization
This q u e s t i o n should c o n s i d e r the time f a c t o r s , considering the stress state before the implementation, Just after the implementation(transition state), and long after the implementation(stable state}. For example, Jarvenpaa(1992) surveyed the e f f e c t of i m p l e m e n t a t i o n of O f f i c e - A u t o m a t i o n in c o u r t o f f i c e f o r 4 y e a r s study; b e f o r e , just after, two month a f t e r , s i x month a f t e r , and two y e a r s a f t e r the i m p l e m e n t a t i o n . I t d e s c r i b e d the change of j o b - c o n t e n t b e f o r e / a f t e r the i m p l e m e n t a t i o n and showed t h a t s h o r t time mental s t r a i n decreased whereas t h e long time mental s t r a i n i n c r e a s e d a f t e r the i m p l e m e n t a t i o n . Korunka(1992,1993) showed the e f f e c t of implementation of Computer system(CAD, EDP system) b e f o r e , during the i n i t i a l two months and one y e a r a f t e r the implementation. A s i g n i f i c a n t i n c r e a s e in e p i n e p h r i n e and n o r e p i n e p h r l n e was shown in during the i n i t i a l two months and one y e a r after t h e implementation compared with t h a t b e f o r e the i m p l e m e n t a t i o n . Llndstorm(1993) reviewed the four Finish longitudinal studies
677
investigating the e f f e c t of new computer system b e f o r e / a f t e r the implementation. They showed the change p r o c e s s i t s e l f , i t s effects on work c o n t e n t , job-mastery, job r e d e s i g n , and r e o r g a n i z a t i o n of t h e working groups. Showing r e s u l t s of the follow-up s t u d y , Huuhatanen(1993) insisted the importance of mastery of a p p l i c a t i o n , training/learning, and a g i n g problems. Leino(1995) a l s o p o i n t e d out the m a s t e r y problems u s i n g t h e r e s u l t s of 7 - y e a r follow up s t u d y . Based on the above reviews of the s t u d y on impacts of computerization, we can show a s y n t h e s i s of examining p o i n t s f o r t h e s t u d y s u r v e y i n g the f i r s t r e s e a r c h q u e s t i o n as Table 1. These f o u r e x a m i n i n g - p o i n t s l n t e r a c t and c o r r e l a t e each o t h e r .
Table 1. A
synthesis
of examining p o i n t s
in the s t u d y on
impacts
of
computerization 1. Does t h e s t r e s s effect of implementation appear i n s t a n t l y in t r a n s i t i o n s t a t e or c o n t i n u e f o r l o n g e r time? 2. How was t h e j o b - c o n t e n t change? Are t h e r e any j o b - r e d e s i g n ? 3. Consider the job mastery problem, l e a r n i n g / t r a i n i n g problems. 4. Consider the r e o r g a n i z a t i o n of working groups and organizational factors.
2 . 2 . Task c h a r a c t e r i s t i c s
of Computer workers
This q u e s t i o n d e a l s with the e f f e c t s of t a s k c h a r a c t e r i s t i c s of computer work on s t r e s s problem in the s t a b l e s t a t e . I t c o n s i d e r s the differences in t a s k c h a r a c t e r i s t i c s among o f f i c e workers, engineers, managers, and o t h e r t e c h n i c i a n s with computers in c r o s s - s e c t i o n a l approach. For example, Smith(1981} showed t h a t c l e r i c a l VDT operators r e p o r t e d h i g h e r l e v e l s of job s t r e s s and h e a l t h c o m p l a i n t s than d i d professional VDT o p e r a t o r s and i n d i c a t e d the i n t e r a c t i o n between j o b c o n t e n t f a c t o r s and VDT use. Haratani(1995} examined the r e l a t i o n s h i p between job s t r e s s o r s and d e p r e s s i v e symptoms in computer s o f t w a r e e n g i n e e r s and managers. I t s u g g e s t e d t h a t l a c k of i n t r i n s i c rewards and interpersonal c o n f l i c t in the p r o j e c t team may be common r i s k f a c t o r s f o r d e p r e s s i v e symptoms in computer s o f t w a r e e n g i n e e r s and managers. These two s t u d i e s s u g g e s t e d the d i f f e r e n c e of s t r e s s o r s among o p e r a t o r s , e n g i n e e r s and m a n a g e r s ( F u j i g a k i , 1994). Kawakami(1995) examined the characteristics of job-stress among computer workers using a s t a n d a r d i z e d c l a s s i f i c a t i o n system of o c c u p a t i o n and e s t a b l i s h e d job-
678
stress scales. It showed t h a t computer e n g i n e e r s had s i g n i f i c a n t l y h i g h e r j o b - o v e r l o a d s c o r e than computer t e c h n i c i a n s and programmers, and significantly h i g h e r s c o r e s f o r s k i l l use than c l e r k s , w h i l e mean job c o n t r o l s c o r e in computer e n g i n e e r s was v e r y s i m i l a r t o t h a t in c l e r k s . It a l s o i n d i c a t e d by t h e c l u s t e r a n a l y s i s a new c l a s s i f i c a t i o n of t h e c o m p u t e r - r e l a t e d o c c u p a t i o n s based on t he p a t t e r n s of j o b - s t r e s s , suggesting that the c l a s s i f i c a t i o n is r e l a t e d to o c c u p a t i o n a l l i f e cycles and useful in i d e n t i f y i n g m a l a d j u s t m e n t groups in the occupations. These s t u d i e s r e v e a l t h e d i f f e r e n c e of j o b - c o n t e n t f a c t o r s among occupations. On t h e c o n t r a r y , W e s t l a n d e r ( 1 9 9 2 ) r e c o n s i d e r e d t h e job c o n t e n t of VDT work w i t h o u t th e c l a s s i f i c a t i o n of o c c u p a t i o n . They classified t h e VDT work i n t o f o u r c a t e g o r i e s ; data entry, data acquisition, interactive communication and word and t e x t p r o c e s s i n g . Carayon(1993,1995) i n d i c a t e d t h e computer system performance to investigate t h e j o b - c o n t e n t of VDT work more p r e c i s e l y , such a s ; computer r e s p o n s e ti m e , user friendliness, flexibility of computer s y s t e m s , and d e g r e e of computer t r a i n i n g , e t c . Based on t h e above reviews of the s t u d y of t a s k c h a r a c t e r i s t i c s of computer w o r k e r s , we can show a s y n t h e s i s of examining p o i n t s for the s t u d y s u r v e y i n g t h e second r e s e a r c h q u e s t i o n as Table 2.
T a b l e 2. A s y n t h e s i s of examining characteristics of computer workers
points
in
the
study
on
task-
1. D e s c r i b e t h e d i f f e r e n c e of t a s k - c h a r a c t e r i s t i c s among occupation. e.g. clerks, operators, technicians, engineers, managers, e t c . 2. D e s c r i b e t h e d i f f e r e n c e of VDT work. e . g . data-entry-task, data a c q u i s i t i o n , i n t e r a c t i v e communication, and w o r d / t e x t processing. 3. D e s c r i b e t h e d i f f e r e n c e of system p e r f o r m a n c e . 4. C o n s i d e r o r g a n i z a t i o n a l f a c t o r s .
3. The m i x t u r e of t h e two r e s e a r c h q u e s t i o n s In above, we research questions.
d e s c r i b e d t h e c o n c e p t u a l frameworks of F u r t h e r m o r e , t h e r e i s a m i x t u r e of two
q u e s t i o n s which l e a d s to t h e n e x t t h i r d q u e s t i o n . ~Q3. Are t h e r e any d i f f e r e n c e in s t r e s s effects i n d u c e d by t a s k c h a r a c t e r i s t i c s ?
of
t h e two research
computerization
679 There are a few s t u d i e s t h a t answer t h i s new question. gorunka(1993) showed t h a t c l e r i c a l worker showed marked i n c r e a s e of n o r e p i n e p h r i n e with th e i m p l e m e n t a t i o n of new t e c h n o l o g y compared w i t h CAD w o r k e r s . It i s c o n s i d e r e d t h a t CAD worker had h i g h e r b a s e l i n e l e v e l s ( b e f o r e the implementation). I t i n d i c a t e d t h a t t h e r e appeared the d i f f e r e n c e in e f f e c t of i m p l e m e n t a t i o n s i n c e t h e t a s k - c h a r a c t e r i s t i c s of c l e r k workers b e f o r e th e i m p l e m e n t a t i o n d i f f e r e d from t h a t of CAD workers. F u r t h e r m o r e , Leino(1995) p r e s e n t e d t h a t in s p i t e of a g r e a t amount of d a i l y VDT work with growing c o g n i t i v e demands, t h e employees e x p e r i e n c e d fewer p s y c h o l o g i c a l s t r e s s symptoms a f t e r seven y e a r f o l l o w up(stable s t a t e ) than j u s t a f t e r the i m p l e m e n t a t i o n ( t r a n s i t i o n state). On t h e c o n t r a r y , F u j i g a k i ( 1 9 9 1 ) showed t h a t e n g i n e e r s e x p e r i e n c e d h i g h psychological stress caused by i n c r e a s e d w o r k - d e n s i t y when t h e y used h i g h e r s p e e d / f u n c t i o n machine. The comparison of c l e r k s t u d y by Leino and e n g i n e e r s ' s t u d y by F u j i g a k i i n d i c a t e s t h a t t h e work d e n s i t y {amount of d a i l y VDT work) has d i f f e r e n t e f f e c t on t h e s e two k i n d s of w o r k e r s , which a p p e a l s t h e d i f f e r e n c e of c o n t e n t of w o r k - d e n s i t y among t h e s e workers. The d i f f e r e n c e in w o r k - d e n s i t y assumed t o be i n d u ced by t h e d i f f e r e n c e in t a s k - c h a r a c t e r i s t i c s . For f u r t h e r s t u d i e s on the d i f f e r e n c e in s t r e s s effect of i m p l e m e n t a t i o n induced by t a s k c h a r a c t e r i s t i c s , the c l a s s i f i c a t i o n of VDT work shown by W e s t l a n d e r ( 1 9 9 2 ) mentioned above w i l l be u s e f u l . As f o r t h e g e n d e r - d i f f e r e n c e s , Asakura(1995) showed t h e i m p a c t s of office c o m p u t e r i z a t i o n f o c u s i n g on t h e gender d i f f e r e n c e s . It showed t h a t t h e p r o p o r t i o n of workers who e n g a g i n g d a t a - e n t r y t a s k were h i g h e r in female workers than male workers, whereas t h e p r o p o r t i o n of workers who a r e d e a l i n g d a t a b a s e were h i g h e r in male w o r k e r s . I t might be c o n s i d e r e d as a one of th e s t a b l e s t a t e of gender d i f f e r e n c e in computer implementation.
4. Conclusion
This p ap e r showed a review of the r e s e a r c h frameworks and a synthesis of t h e s t u d i e s on impacts of c o m p u t e r i z a t i o n on s t r e s s problem and on e f f e c t s of t a s k c h a r a c t e r i s t i c s of computer work on s t r e s s problems.
680 References hsakura T:1995 The Impact of Computerization on Job Characteristics, Physical and Mental Health of Japanese Office workers: gender Differences, in this Volume. Carayon P.:1993. A Diary Study of Computer Use and Worker Stress. Human ComputerInteraction (ed. by Smith and Salvendy), Elsevier, 715-720. Carayon P:1995. Effect of Computer System Performance and other work Stressors on Strain of Office Workers, in this Volume. Fujlgakl Y:1991. Workload Due to High Speed and High Function Machine. Is the Work-Denslty Increasing? Advances in Human Factors /Ergonomics 18A'human Aspects in Computing, ed. by Bullinger,J. elsevier Sci.Pub. 180-184. FuJlgakl Y, Asakura T, Haratanl T.: 1995. Work Stress and Depressive Symptoms among Japanese Information System Managers. Industrial Health, Voi.32, No.4(in press). Haratani T, Fujigaki Y, Asakura T:1995 Job Stressors and Depressive Symptoms in Japanese Computer Software Engineers and Managers, in this Volume. Huuhatanen P, et.al: 1993. Mastering the Changes in Information Technology: A Follow-up Study of Insurance Tasks. Human Computer Interaction (ed. by Smith and Salvendy), Elsevier, 703-708. Jarvenpaa E'1992. A longitudinal Study of the Implementation of Office Automation, WWDU(International Conf. Work with Display Units)'92 Abstract Book.(ed.by Luzak, H. et.al)F-19 Kawakami N, Roberts C R,Haratanl T:1995. Job-Stress ~ Characteristics of Computer Work in Japan, in this Volume. Korunka C, Bernd K, Andreas W:1992, The conversion to New Technologies. WWDU(International C o n f . Work with Display Unlts)'92 Abstract Book.(ed.by Luzak, H. et.al)F-7 gorunka C, Huemer K H, garetta B'1993. Methodological Aspects of the Longitudinal Studies -Experience from HCI study. Human Computer Interaction (ed. Smith and Salvendy), Elsevier, 709-714 Leino T et.al. The Impact of Computerization on Job Content and Stress: A Seven Year Follow-up in the Insurance Sector, in this Volume. Llndstorm K.:1993. Finnish Longitudinal Studies of Job design and VDT Work. Human Computer Interaction (ed. by Smith and Salvendy), Elsevier, Smith M, Stress in Westlander in Work
697-702. et.al'1981. An Investigation of Health Complaints and Job Video Display Operations. Hum Factors,23(4)387-400. G, Aberg E'1992. Variety in VDT Work:An Issue for assessment Environment Research. Int J Human Computer Interaction,
4 ( 3 ) , 283-301.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
681
The impact of computerization on job content and stress: a seven year follow-up in the insurance sector Tuula Leino, Ph.Lic., Kirsi Ahola, Psych.Lic., Pekka Huuhtanen, Ph.D. and Irja Kandolin, Soc.Sci.Lic. Department of Psychology, Finnish Institute of Occupational Health, Topeliuksenkatu 41 a A, FIN-00250 Helsinki, Finland
I. INTRODUCTION Development of the insurance sector is marked by expanding use of data terminals and microcomputers. Studies mainly in office work have shown that health complaints and stress problems are associated with the intensity of video display terminal (VDT) use. (Smith, 1987; Kalimo & Lepp~inen, 1987; Williams, 1988). At the same time the mastery of computer applications in insurance sector have decreased more often than increased. But the impact of information system on the work content has been different among various occupational groups. (Huuhtanen & Leino, 1992.) The continuous development of data systems is a challenge to the employees. Follow-up studies have been few, however. One follow-up study in the insurance sector revealed that between the late 1970s and the mid-1980s, VDT usage became more demanding and psychologically positive e.g., causing less mental load and strain. (Aronsson 1986). The present follow-up describes changes perceived by employees in insurance when moving over to more advanced data systems. The study was part of an extensive research program, New Information Technology and Work Environment, carried out in 1985-1988 by the Finnish Institute of Occupational Health, The Technical Research Centre of Finland, and the Finnish Work Environment Fund. In the insurance sector the study was continued in 1993 by a questionnaire survey. In this seven-year follow-up, the employees of one insurance company were investigated before the implementation of a new data system, two years after the implementation (transition state), and again after five years of usage (stable stage).~ This study examines the possible changes in the transition and stable stages. The studied aspects were: 1. The amount of time daily spent in VDT work 2. The mastery of applications 3. The assesment of the psychological demands of work and 4. The psychological symptoms
682 2. MATERIALS AND METHODS The study group consisted of employees in one big insurance company with officies throughout Finland. The first survey was conducted before the implementation of the new integrated data system in 1985, the second about a year alter the implementation in 1987, and the third survey five years later in 1993. The overall survey response rates were 75%, 69%, and 65%. The number of the same respondents in each year was 146 (97 were women and 47 men). The average age of the respondents was 38 years in 1985. In 1993 12 respondents were under 35 years old, 49 between 35 and 44 years, and 84 over 45 years old. The occupational groups were customer service workers (n=23), office workers (n=59), data experts (n=4), sales personnel (n=25), and supervisors (n=31). The changes over time were analyzed with one-way ANOVAs with repeated measures using sex, age, and occupation in turn as an independent variable. Repeated measure ANOVAs were also conducted in pairs (1985-1987, 1987-1993) to see whether the change had been continuous or taken place in a particular stage.
3. RESULTS During a seven year follow-up, an increasing number of respondents spent daily more than four hours in VDT work. Also the feelings of sufficient mastery became more general and experiences of excessive fatigue less frequent (Table 1). The working time with VDT increased continuously during the study period. The experiences of excessive fatigue decreased already in the transition stage, but feelings of sufficient mastery were experienced mainly in the stable stage. The sex, age, or occupation of the respondents were not related to these changes. Table 1. Working time with the VDT, mastery of applications, and experience of excessive fatigue as a psychological symptom in 1985, 1987, and 1993 (%) and the significance level over time (n= 146).
Year/Variable
1985
1987
1993
Significance
Working over four hours daily
27
35
49
p < 0.001
Feelings of sufficient mastery
66
63
77
p < 0.01
Experience of excessive fatigue (quite ot~en or continuously)
33
23
19
p < 0.001
683 Workers under 45 years of age felt that the need to make rapid and independent decisions concerning cognitive job demands increased during the transition stage (Table 2). This experience was shared by office workers and sales personnel (significance level for timeoccupation interaction p < 0.01). Table 2. Estimates of cognitive job demands (quite otten or continuously) in three age groups in 1985, 1987, and 1993 (%) and the significance level of the change over time and of the timeage group interaction.
YearNariable
1985
1987
1993
Significance
Ability to make independent decisions under 35 years 35 - 44 years over 45 years
42 57 81
67 77 74
100 84 80
time p< 0.01 time-age p < 0.01
Ability to make rapid decisions under 35 years 35 - 44 years over 45 years
67 53 67
75 69 72
100 76 65
time p< 0.001 time-age p< 0.01
During the study period, the mental load of having too much work was experienced to have decreased by the custom service and office workers, but to have increased by the sales personnel and supervisors (Figure 1). The custom service and office workers reported the change during the stable stage, the supervisors already after the transition stage, and the sales personnel at both stages. In line with the decrease in the experience of excessive fatigue, the experience of nervousness was reported to have decreased, but only by custom service and office workers. This change had also taken place in the transition stage. The supervisors reported no change in nervousness, whereas the sales personnel reported symptoms of nervousness at a steadily increasing rate (Figure 2) as well as of fatigue also (Table 1).
684 % 100-
custom servive --80-
office work
....... sales personnel ....
supervisors
60-
40-
'
--...
20-
O-
!
4
÷
1985
1987
1993
Figure 1. Perception of the mental load: too much work to do (quite often or continuously) in different occupational groups in 1985, 1987, and 1993. The significance level of the change over time was p < 0.01 and of the time vs occupational group interaction was p < 0.01).
% 50
c u s t o m servive --4ff
office w o r k
....... s a l e s p e r s o n n e l ....
supervisors
30-
2ff
fly
1985
1987
1993
Figure 2. Experience of psychological symptoms nervousness (quite often or continuously) in different occupational groups in 1985, 1987, and 1993. The significance level of the change over time was p < 0.001 and of the time vs occupational group interaction was p < 0.05).
685 4. DISCUSSION This study showed that the mastery of data applications had increased along with daily VDT work during a seven-year follow-up. The change in mastery was not yet seen after one year of the implementation, but was evident after a five-year stable stage. Age was not connected to the time in VDT work or to the feelings of sufficient mastery of the applications. The job demands had changed so that now it was necessary for everyone to participate in decision making. The younger employees felt that new apllications had increased the need to make independent and rapid decisions at work. This might be due to the fact that the work tasks of the younger employees had changed somewhat after the implementation, or that the younger employees master the applications better and are able to make independent decisions. The perceived changes were also different among the occupational groups. It seems that in the long run the new data system facilitated the execution of custom service and office work because they experienced a decrease in their work load and nervousness. The decrease may have been due to the greatar time spent with the application daily and the better mastery of tasks beacause of that. The sales personnel seemed to experience a growing amount of pressure. Possible causes for this might be the increasing competition due to the economic recession and lack of time and possibilities to practise with the new applications. These results indicate that the stress response often reported with extensive VDT usage might be related to insufficient mastery of numerous applications, along with great amount of daily VDT work and not the VDT work itself. Thus the stress could be diminished by strengthening the confidence of mastery. Enough time to practise and as long a transition phase as possible should be allowed to active users, because it took several years before the feelings of sufficient mastery increased.
REFERENCES
1. G. Aronson, Work content, stress and health in computer-mediated work. A seven-year follow-up study. Proceedings of the International Scientific Conference on Work with Display Units. National Board of Occupational Safety and Health, Stockholm (1986), 401404. 2. N. Bj0rn-Andersen, The impact of the computer systems in the five banks. In N. Bj0rnAndersen, B. Hedberg, D. Mercer, E. Mumford & A. Sole (eds.), The Impact of Systems Change in Organisations, Sijhoff & Noordhoff, Holland, 1979, 269-282. 3. P. Huuhtanen, New technology, occupational health and stress: Prevention research in Finland. Human Factors in Organizational Design and Management-IV, G. E. Bradley & H. W. Hendrick (Editors), Elsevier Science B.V., (1994), 683-688. 4. P. Huuhtanen & T. Leino, The impact of new technology by occupation and age on work in financial firms: a 2-year follow-up. Int J of Human-Computer Int., 4 (1992):2, 123-142.
686 5. E. Jarvenp~.~i, Mental workload: Research on computer-aided design work and on the implementation of office automation. Report No. 130. Helsinki University of Technology. Industrial Economics and Industrial Psychology, Otaniemi 1991. 6. R. Kalimo & A. Lepp~.nen, Visual display units-psycho-social factors in health. In: M. J. Davidson & C. L. Cooper, Women and information technology, Chichester, John Wiley & Sons Ltd, (1987) 194-224. 7. R. Karasek & T. Theorell, Healthy Work: Stress, Productivity, and the Reconstruction of Working Life, Basic Books, Inc. 1990, USA. 8. K. LindstrOm, Well-being and Computer-Mediated Work of Various Occupational Groups in Banking and Insurance. International Journal of Human-Computer Interaction 3 (1991):4,339-361. 9. M. J. Smith, Mental and physical strain at VDT workstations, Beh. Inform. Technol.6 (1987):3,243-255. 10.T.A. Williams, Computers, work and health: A socio-technical approach, Taylor & Francis, London, 1988.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
687
The Impact o f O f f i c e C o m p u t e r i z a t i o n on J o b C h a r a c t e r i s t i c s , P h y s i c a l and Mental H e a l t h o f J a p a n e s e O f f i c e Workers: 6 e n d e r D i f f e r e n c e . T. As a k u r a D e p a r t m e n t o f H e a l t h and S p o r t s S c i e n c e , T o ky o G a k u g e i U n i v e r s i t y , 4-1-1, N u k u i k i t a - M a c h i , K o g a n e i - S h l , Tokyo 184, J a p a n The h y p o t h e s i z e d p s y c h o s o c i a l mechanism o f t h e i m p a c t o f o f f i c e c o m p u t e r i z a t i o n on w o r k e r s ' h e a l t h was r e e x a m i n e d . C o n c l u s i v e l y , t h e i n f l u e n c e s o f o b j e c t i v e VDT w o r k v a r i a b l e s on h e a l t h i n d i c a t o r s w e r e m e d i a t e d b y j o b c h a r a c t e r i s t i c s , e x c e p t a few o b j e c t i v e VDT w o r k v a r i a b l e s . F u r t h e r , g e n d e r d i f f e r e n c e s in o b j e c t i v e VDT w o r k e n v i r o n m e n t a r e c l a r i f i e d . 1.Purpose Not o n l y i n d i v i d u a l f a c t o r s b u t a l s o p s y c h o s o c i a l VDT w o r k e n v i r o n m e n t w i l l b e i m p o r t a n t , when w o r k e r s ' h e a l t h p r o m o t i o n is c o n s i d e r e d in c o m p u t e r i z e d o f f i c e ( W e s t l a n d e r a n d Aberg,1992, C a r a y o n , 1 9 9 2 , B r a d l e y , 1 9 8 9 , A s a k u r a a n d F u j i g a k i , 1993). One o f t h e p u r p o s e s is t o r e e x a m i n e t h e i m p a c t s o f o f f i c e c o m p u t e r i z a t i o n on j o b c h a r a c t e r i s t i c s , p h y s i c a l and m e n t a l h e a l t h o f o f f i c e w o r k e r s . M o r e o v e r , t h e o t h e r p u r p o s e is t o c l a r i f y g e n d e r d i f f e r e n c e s o f t h e o b j e c t i v e VDT w o r k c o n d i t i o n s , t h e j o b c h a r a c t e r i s t i c s and t h e h e a l t h i n d i c a t o r s . Also, g e n d e r d i f f e r e n c e s o f t h e r e l a t i o n s h i p s b e t w e e n them a r e ex am i n ed . 2 . R e s e a r c h methods and S u b j e c t s A s u r v e y was c o n d u c t e d d u r i n g J a n u a r y and F e b r u a r y in 1994. T h r e e t h o u s a n d c o m p a n i e s w e r e s a m p l e d r a n d o m l y o u t o f more t h a n 30,000 c o m p a n i e s t h a t h a d e m p l o y e d more t h a n 30 w o r k e r s in Tokyo. Almost t e n p e r c e n t in t h e m o t h e r p o p u l a t i o n which c o n s i s t e d o f e l e v e n i n d u s t r i e s w e r e e x t r a c t e d : t h e i n d u s t r y of c o n s t r u c t i o n , manufacturing, e l e c t r i c i t y and gas s u p p l y , t r a n s p o r t a t i o n , telecommunication, wholesale, r e t a i l , finance and i n s u r a n c e , r e a l e s t a t e , i n f o r m a t i o n p r o c e s s i n g service, medical c a r e s e r v i c e . A d d i n g two r e q u e s t s s t a t e d below, q u e s t i o n n a i r e s w e r e a s k e d f o r m a n a g e r i a l s t a f f s t o d e l i v e r t o e l e c t e d f i v e e m p l o y e e s in e a c h company. The f i r s t r e q u e s t was an a r r a n g e m e n t o f s e x r a t i o among s a m p l e s u b j e c t s . Two p e r s o n s o u t o f f i v e w e r e male a n d t h e t h r e e w e r e female. The s e c o n d r e q u e s t was an a r r a n g e m e n t o f t h e a m o u n t o f VDT work among t h e sample. One male in t h e two e l e c t e d male h a v e b e e n w o r k i n g with VDTs. A n o t h e r o n e male n e v e r u s e d a c o m p u t e r f o r h i s j o b . As f o r t h e f e m a l e sample, o n e f e m a l e in t h e t h r e e h a s b e e n a f r e q u e n t u s e r o f VDTs. A n o t h e r o n e o f t e n u s e d t h e VDTs an d t h e r e s t s c a r c e l y e v e r w o r k e d w it h a c o m p u t e r . As a r e s u l t , 32.44 o f 15,000 w o r k e r s r e s p o n d e d t h e q u e s t i o n n a i r e s (male=l,916, female=2,822). D a t a o f VDT u s e r s who
688 Table 1 Backgrounddata of the study sample. Background Sex I terns Categories Male Female 17~24 201(12. 3) 1122(42.6) Age 25~29 392(24. O) 766(29.1) 30~39 541(33. 1) 422(1{}.O) 40~49 499(30. 6) 322(12.2) Managerial 197(12. 2) 15( O.6) & Administrative workers Clerical workers 855(52. 8) 2068(79.2) Tradesman 114(7.1) 38( 1.5) Occupation Sales workers 17( 1.1) 20(0.8) Technical workers 268(16.6) 165(6.3) Professional workers 33(2.0) 72(2.8) Skilled workers 92(5.7) 186(7.1) Service workers 24( 1.5) 28( 1.1) Others 18( 1.1) 19(0.7) Chief of Department 89( 5.5) 9( 0.3) Position in Section chief 312(19. 3) 23( O.9) a Company Chief clerk 236(14. 6) 55( 2. 1) Gmup leader 138( 8. 5) 112( 4. 3) Mere clerk/worker 841(52. O) 2420(92.4) Work with VDT yes 1475(90. 7) 2476(94.6) no 151(9.3) 142( 5. 4) Score of perceived work environment 3. 6+2. 0 3. 2+2. 0 changes due to office computerization 1)Unknown data are excluded for ~ of each item in this paper. 2)F,very item shows a significant difference between gender. Table 2 Working conditions of YDT use workers by sex. - part 1 Working conditions Sex Items Categories score Male Female Training at the time of well or enough (4) 87(5.9) 130(5.3) computer introduction good (3) 336(22.9) 657(26.7) into work places fair (2) 482(32.9) 769(31.3) (NS) no (1) 562(38.3) 902(36.7) 2 (1) 118( 8. 5) 237(10.2) Average days of Vl)T use 3 (2) 108(7.8) 215( 9. 3) in a week 4 (3) 150(10.9) 321(13.9) (p
__~8
(9)
72(5.2)
1)Scores were used for multiple regression analyses.
116(4.8) (continue)
689
w e r e y o u n g e r t h a n 50 y e a r s o l d w e r e u s e d f o r s u b s t a n t i a l
analyses(N-3,951).
3 . R e s u l t s and Discussion. 3.1.Background data T a b l e 1 s h o w e d b a c k g r o u n d d a t a o f t h e sample. Compared w i t h t h e male, r a t i o u n d e r t h e t h i r t y was s i g n i f i c a n t l y h i g h e r in t h e f e m a l e s a m p l e : 71.7~ a nd 36.3~ r e s p e c t i v e l y . A r a t i o o f c l e r i c a l w o r k e r s was h i g h e r in t h e f e m a l e t h a n in t h e male. On t h e o t h e r hand, t h e m a n a g e r i a l and a d m i n i s t r a t i v e w o r k e r s , t h e t r a d e s m a n a n d t h e t e c h n i c a l s t a f f s showed h i g h e r r a t i o s in t h e male r a t h e r t h a n in t h e f e m a l e . Obvious g e n d e r d i f f e r e n c e o f p o s i t i o n in c o m p a n i e s was e x h i b i t e d in t h e T a b l e 1. In f a c t , o v e r 90 p e r c e n t o f t h e f e m a l e was a mere c l e r k . As f o r t h e VDT use, i t f o u n d t h a t a r a t i o o f VDT u s e r s in t h e f e m a l e was h i g h e r r a t h e r t h a n in t h e male. E s p e c i a l l y , t h e s i g n i f i c a n t d i f f e r e n c e o f VDT u s e b e t w e e n t h e g e n d e r was f o u n d s in t h e f o r t i e s ( p ( 0 . 0 1 ) : a l t h o u g h t h e male was 83.0~, t h e f e m a l e was 90.6~;. From t h e s e , i t c o n s i d e r e d t h a t t h e a m o u n t o f VDT w o r k was t r a n s f e r r e d t o t h e f e m a l e d u e t o t h e g e n d e r d i f f e r e n c e s of age and p o s i t i o n . 3 . 2 . O b j e c t i v e VDT w o r k c o n d i t i o n s a n d e n v i r o n m e n t 3.2.1.Perceived changes according to office computerization P e r c e i v e d c h a n g e s o f w o r k c o n d i t i o n s and e n v i r o n m e n t a c c o r d i n g t o o f f i c e c o m p u t e r i z a t i o n w e r e m e a s u r e d u s i n g a s c a l e which c o n s i s t e d o f s i x o r i g i n a l items. The c h a n g e s w e r e m e a s u r e d u s i n g d i c h o t o m i c q u e s t i o n s , s u c h as t o ( 1 ) i n c r e a s i n g t h e number/models of computers, (2)upgrading computers to h i g h e r p e r f o r m a n c e models, ( 3 ) i m p r o v i n g c o m p u t e r n e t w o r k s y s t e m , ( 4 ) i n c r e a s i n g a m o u n t o f j o b n e e d t o do by VDTs, ( 5 ) c r e a t i n g new t a s k / j o b n e e d t o do b y VDTs, ( 6 ) i n c r e a s i n g t h e number o f VDT u s e r s . The c o n s t r u c t v a l i d i t y o f t h e s c a l e was e x a m i n e d b y t h e f a c t o r a n a l y s i s and C r o n b a c h ' s a o f t h i s s c a l e was 0.78. I t f o u n d t h a t t h e male w o r k e r s e x p e r i e n c e d v a r i o u s c h a n g e s r e l a t e d w i t h o f f i c e c o m p u t e r i z a t i o n more t h a n t h e f e m a l e w o r k e r s . 3.2.2.VDT w o r k c o n d i t i o n s a n d e n v i r o n m e n t VDT w o r k c o n d i t i o n s and e n v i r o n m e n t w e r e c o m p a r e d b e t w e e n t h e g e n d e r ( T a b l e 2). I t was showed t h a t t h e male w o r k e d u s i n g VDT more f r e q u e n t l y d u r i n g o n e week. M o r e o v e r o c c a s i o n s o f c o n t i n u o u s VDT w o r k o v e r two h o u r s a p p e a r e d t o b e more f r e q u e n t in t h e male t h a n in t h e female. F u r t h e r m o r e , a v e r a g e h o u r s o f o v e r t i m e w o r k p e r o n e month s h i f t e d t o l o n g e r c a t e g o r i e s in t h e male t h a n in t h e f e m a l e . On t h e o t h e r hand, as f o r t h e f e m a l e t h e a v a i l a b i l i t y o f r e s t d u r i n g VDT work was s i g n i f i c a n t l y p o o r a n d t h e t e m p e r a t u r e in VDT w o r k room was more u n c o m f o r t a b l e . The a b o v e i n d i c a t e d t h a t w o r k o v e r l o a d was c h a r a c t e r i s t i c o f t h e male VDT u s e r s a n d p o o r VDT w o r k e n v i r o n m e n t was c h a r a c t e r i s t i c o f t h e f e m a l e u s e r s . 3.2.3. T y p e s o f VDT j o b A c c o r d i n g t o W e s t l a n d e r G. a n d A b e r g E.(1992), Job c o n t e n t s o f VDT w o r k w e r e c l a s s i f i e d i n t o f o u r t y p e s ( T a b l e 3): (1)word a n d t e x t p r o c e s s i n g i n c l u d i n g g r a p h i c design, (2)data e n t r y , (3)data a c q u i s i t i o n and i n t e r a c t i v e c o m m u n i c a t i o n , and ( 4 ) o t h e r t y p e . E x c e p t f o r d a t a e n t r y , a l l o f t h e J o b t y p e s showed h i g h e r r a t i o s in t h e male r a t h e r t h a n in t h e f e m a l e . C o m p a r i n g t h e n u m b e r o f VDT j o b t y p e s b e t w e e n t h e g e n d e r , i t was c l a r i f i e d t h a t t h e male
690 Table 2 Working conditions of VDT use workers by sex. Working conditions Items
- part 2 -
Sex
Categories score Male Female frequently (4) 511(34.7) 710(28.8) Continuous VI)Tw o r k occasionally (3) 353(24.O) 604(24.5) over 2 hours rarely (2) 368(25.O) 700(28.4) (p
*** p(0. 001 Female
1. Word and text processing, graphic design, 1071(72.9) 1603(65.2)*** CAD/CAM 2. Data entry 996(67. 7) 1910(77.6)*** 3.Data acquisition and interactive communication 813(55.3) 962(39.1)*** (data analysis, using database, E-mail etc. ) 4.Other type of work 399(27. 1) 387(15.7)*** (programming, process control, others) Number of job types performed with one type 386(26.2) 785(31.9) VI)T two types 513(34.9) 1025(41.7) (p(0. 001) three types 421(28.6) 573(23.3) four types 151(10.3) 77( 3. 1) The classifications were in accordance with Westlander G. and Aberg E. (1992). Table 4
GenderDifferences of Job Characteristics and Bealth Indicators JS JP JD Eye Stiffness CES-D
Sex
Male Female
9.9_+2.8 9.6_+3.2 8.2_+2.1 15.2_+5.4 17.4+_6.3 36.6_+8.6 8. 8+-2.8 8. 3-+3. 1 7. 8_+2.2 17. 1-+5. 3 19.9-+6. 1 38. 0_+8. 8
Age S e x , Age
NS
***
NS
NS
*
NS
1)JS-Job Satisfaction, JP=Job Pressure/Work Overload, JD-Job Discretion The higher scores of these scales, the stronger those tendencies become. 2)The higher scores of these scales, the worse health indicators become.
691
u s e r s t e n d t o do more v a r i o u s t y p e s o f j o b s with VDTs. In o t h e r w o r d s , s c o p e o f j o b d o n e b y u s i n g VDTs seems t o be n a r r o w in t h e f e m a l e u s e r s . 3. 3. P e r c e l v e d j o b c h a r a c t e r i s t i c s T h r e e a s p e c t s o f j o b c h a r a c t e r i s t i c s w e r e examined; t h r e e s c a l e s a s s e s s t h e a b i l i t y f u l f i l l m e n t a n d j o b s a t t s f a c t i o n ( J S , 4 items), t h e j o b o v e r l o a d an d j o b p r e s s u r e ( J P , 4 items), a n d t h e j o b d l s c r e t l o n ( J D , 3 items). C r o n b a c h ' s a o f e a c h s c a l e was s u i t a b l e f o r f u r t h e r a n a l y s e s : 0.84, 0.83 an d 0.79 r e s p e c t i v e l y . 3.3.1. G e n d e r d i f f e r e n c e s A l l s c o r e s o f t h e t h r e e m e a s u r e m e n t s in t h e T a b l e 4 w e r e shown h i g h e r In t h e male r a t h e r t h a n in t h e female. A l t h o u g h t h e male w o r k e r s h a d t e n d e n c i e s to e x p e r i e n c e h i g h e r j o b s a t i s f a c t i o n and job d i s c r e t i o n , t h e y seemed t o be e x p o s e d b y more s e r i o u s j o b r a t h e r t h a n t h e female. T h o u g h t h e f e m a l e f e l t lower job pressure, they felt not only lower job satisfaction but also lower j o b d i s c r e t i o n , in c o m p a r i s o n with t h e male. 3 . 3 . 2 . R e l a t i o n s h i p s b e t w e e n o b j e c t i v e work f a c t o r s and j o b c h a r a c t e r i s t i c s R e l a t i o n s h i p s b e t w e e n o b j e c t i v e VDT w o r k f a c t o r s a n d j o b c h a r a c t e r i s t i c s w e r e e x a m i n e d b y m u l t i p l e r e g r e s s i o n a n a l y s e s ( T a b l e 5). I t was f o u n d t h a t t h e p e r c e i v e d w o r k e n v i r o n m e n t c h a n g e s as w e l l as t h e n u m b e r o f VDT j o b t y p e s h a v e t e n d e n c i e s t o make JS and JD e n h a n c e . Th ese f a c t o r s t e n d e d t o make JP i n c r e a s e a t t h e same time. The c h a n g e s a c c o r d i n g t o o f f i c e c o m p u t e r i z a t i o n as w e l l as t h e VDT w o r k h a d b o t h n e g a t i v e i m p a c t s an d p o s i t i v e i m p a c t s on j o b c h a r a c t e r i s t i c s . The b e t t e r t h e a v a i l a b i l i t y o f r e s t time i m p r o v e , t h e h i g h e r s t a t e s o f JS and JD a r e f e l t . On t h e o t h e r hand, JP t e n d s t o b eco m e l o w e r . In view p o i n t o f w o r k s t r e s s , i t p o i n t e d o u t t h a t t h e a v a i l a b i l i t y o f r e s t d u r i n g VDT w o r k was a s i g n i f i c a n t f a c t o r as t h e work s t r e s s b u f f e r . 3 . 4 . R e l a t i o n s h i p s between o b j e c t i v e work v a r i a b l e s , job c h a r a c t e r i s t i c s and health indicators A p a r t fr o m JD, t h e T a b l e 6 showed t h a t b o t h JS an d JP h a d s i g n i f i c a n t r e l a t i o n s h i p s wi t h e y e symptoms(Eye), P h y s i c a l symptoms s u c h as s t l f f n e s s ( S t i f f n e s s ) as w e l l as CES-D. From t h i s , i t f o l l o w e d t h a t t h e i n f l u e n c e o f o b j e c t i v e VDT w o r k v a r i a b l e s on h e a l t h i n d i c a t o r s a r e m e d i a t e d by j o b c h a r a c t e r i s t i c s , e x c e p t a few VDT work v a r i a b l e s . A f t e r t h e s e r e e x a m i n a t i o n , t h e h y p o t h e s i z e d p s y c h o s o c l a l mechanisms o f t h e i m p a c t o f o f f i c e c o m p u t e r i z a t i o n on w o r k e r s ' h e a l t h ( A s a k u r a a n d F u j i g a k i , 1 9 9 2 ) w e r e g e n e r a l l y s u p p o r t e d . I t a l s o f o u n d t h a t t h e a v a i l a b i l i t y o f r e s t time h a d s i g n i f i c a n t d i r e c t r e l a t i o n s h i p s with h e a l t h i n d i c a t o r s . C o n c l u s i v e l y , how t o i m p r o v e t h e a v a i l a b i l i t y o f r e s t time ls i m p o r t a n t t a s k o f t h e h e a l t h p r o m o t i o n in c o m p u t e r i z e d o f f i c e . References 1 . A s a k u r a T. a n d F u j i g a k i Y.(1993). Human-Computer I n t e r a c t i o n : 1 9 B , e d i t e d b y M.J.Smith a n d G . S a l v e n d y , pp982-987, E l s e v i e r . 2 . B r a d l e y G. (1989). C o m p u t e r s and P s y c h o l o g i c a l Work E n v l r o n m e n t , T a y l o r & F r a n c i s , 1989. 3 . C a r a y o n - S a i n f o r t P.(1992). I n t e r n . J. HCI, 4(3), pp.245-261 4. W e s t l a n d e r G a n d A b e r g E. (1992). I n t e r n . J. HCI, 4(3), pp.283-301.
692 Table 5
Standardized Partial correlation Coefficients (B) of Multiple Regression Analyses for Three Job Characteristics. (N) Independent variables
l. Age
JS JP JD male female male female male female (1,211) (1,987) (1,211) (1,992) (1,202) (1,966) .00
.06,
2. 3. 4. 5. 6. 7.
Total years of working with VDT .03 -.02 Average day of VDT use in a week -.11.* -. 05, Average hours of VDT use in a day .06 -.03 continuous VDT work over 2 hours -.06 .04 Availability of rest time .08** .I0,** Training at time of computer .09** .17,** introduction into work places 8. Overtime work .02 .12~ 9. Score of perceived york environment .15,** . 16.** changes due to office computerization I0. Number of job types performed with .06, .05, VDT
Adjusted R2(%)
5%10{0~ ~
.16.**
.03 .05 -.04 .06
.09~c~
.07
-.02 .12~ . O0 -. 02 .09*** -.02 .05,
.08,
.04
.07.* -.06,
-.09** .12,**
-.15.** -.17,** .21,** .19,** . Ol . 50~0~ . 12,**
-. 01
33~
-. Ol
.04
.04,
. 41~0~ -. 02 . 05, .06,
.07** .02
.06**
. II***
28~0~
.06,
9~
1)JS=Job Satisfaction, JP=Job Pressure/Work Overload, JD=Job Discretion The higher scores of these scales, the stronger those tendencies become. 2)Underlined figures were indicated the differences between gender.
3), p
Standardized Partial Correlation Coefficients (B) of lultiple Regression Analyses for Three Health indicators. (N) Independent variables Bye Stiffness CES-I) male female male female male f e m a l e (1, 195) (1,949)
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. ll. 12. 13.
Age -.11.* Total years of working with VDT -.00 Average day of VDT use in a week -.00 Average hours of VDT use in a day -.01 Continuous VDT work over 2 hours .17,** Availability of rest time -.17,** Training at time of computer -.07* introduction into work places Overtime work .05 Score of perceived work environment . 10*** changes due to office computerization Number of job types performed with .04 YDT Job Satisfaction -. 17~*~ .12~c~ Job Pressure/Work Overload Job Discretion .03 Adjusted R2($) 17~
-.12~
(1, 193) (1,943) (1, 197) (1,942)
-.01
-.04
-.03
.01
-.08*
.04
-.05
.03
.01
.03
.03
.10,**
.06
.05 .04
-. 03 .06
.13,** .07* -.11,** -.10,** -.12,** -.01 -.05* -.10.** -.07** .02 -. 02
.06*
. O1
.04
.04
.04
.02
.05*
.02
.02
-. 07*
. O0
15~t~
-. 02
-. 05* .06* -. 04 -. 06** -. 02
.03
-. 15~c~ .16~c~ .02
.07*
-. 12,** -. 03
-. 1 5 , ~ -. 15~c~ -. 38*** -. 28*** .15~c~ . 19~c~ . 15~c~ . 19~c~ .03 .04 -. 04 -. 05*
12~,
10~
2~-~*~.-~ 15~c~
1)Eye:Number of eye symptoms. Stiffness:Limb and Physical symptoms such as stiffness, tiresome and pain. CES-I):Center for Epidemiologic Studies-Depression The higher scores of these scales, t h e worse the health becomes. 2)Underlined figures were indicated the differences between gender. 3)* P
Symbiosis of Human and Artifact Y. Anzai, K. Ogawaand H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
693
Effect of computer system performance and other work stressors on strain of office workers * Pascale CARAYON Department of Industrial Engineering, University of Wisconsin-Madison, 1513 University Avenue, Madison WI 53706, USA
This study examines the effect of computer system performance and job stressors (demands, support, content, and organizational and task control) on worker strain in a population of 171 office workers in a public service organization. Results show that computer system performance had mainly an indirect effect on worker strain. Computer problems had an indirect effect on strain via job demands and support. Duration of computer problems had an indirect effect on strain via job demands. Computer resources had an indirect effect on both indicators of strain via task control, and a direct effect only on mood disturbances. Results of this study partially confirm previous studies (Carayon-Sainfort, 1992; Asakura and Fujigaki, 1993) that show an indirect effect of computer system performance on strain via job stressors.
1. INTRODUCTION Computerization of offices has led to dramatic changes in the way work is performed, designed and monitored. Research has shown that some changes have resulted in negative consequences for the workers (Smith, 1987; Carayon, 1994; Carayon and Lim, 1994). In particular, computer system performance has been identified as a critical stressor for office workers who are "intensive" computer users (Carayon-Sainfort, 1992; Asakura and Fujigaki, 1993). This study focuses on the effect of computer system performance and other job stressors on the strain of office workers. It is a follow-up to the study performed by Carayon-Sainfort (1992) where a model of the effect of computer system performance on job design and strain was tested. This study expands and tests this model in a population of 171 office workers in a public service organization. Figure 1 shows the model of the effect of computer system performance and job stressors on worker strain tested in this study. The model assumes that computer system performance has an indirect effect on worker strain: computer system performance can affect job stressors which, in turn, can influence worker strain. The job stressors selected for this study have been shown to be the most important job characteristics that predict worker strain in a range of occupations and organizations. Funding for this study was provided by the National Science Foundation (No. IRI-9109566).
694
COMPUTER SYSTEM PERFORMANCE: - computer problems - duration of computer problems - computer resources
JOB STRESSORS: - job demands - job support - job content - organizational control - task control
WORKER STRAIN: - mood disturbances - health
Figure 1 - Model of the Effect of Computer System Performance and Job Stressors on Worker Strain
2. M E T H O D S 2.1 S t u d y d e s i g n
Computer users from one division of a public service organization in the U.S. were asked to participate in the study. The employer provided a list of employee names within one division of the organization. All employees selected to participate in the study were extensive users of computer and were performing a range of clerical tasks. Questionnaires were handed out to the employees directly by the research team. The employees were asked to put their questionnaire into sealed envelops that were then collected by a member of the research team. 2.2 S a m p l e
One-hundred-seventy-one office workers agreed to participate in the study (response rate=85%). These workers perform a wide range of clerical tasks and spend an average of 7.2 hours per day working on a computer. The majority of the study participants (86%) were women. The average age was 43 years (S.D.=9 years), the average tenure with the employer was 14 years (S.D.=8 years), and the average experience with one's current position was 6 years (S.D.=5 years). 2.3 M e a s u r e m e n t
The main data-collection method was a questionnaire survey that includes questions used in previous studies of VDT office workers (Smith et al., 1981; Carayon-Sainfort, 1992). The measures of computer system performance included the following: computer response time, computer breakdown, user friendliness of computer interface, flexibility of computer system, degree of computer training, relationship with computer support staff, and computer-related documentation. Several aspects of job design were measured: quantitative workload, meaningfulness, social support from supervisor, management and colleagues, and different facets and levels of job control. Measures of worker strain included shortterm responses, such as mood disturbances (Profile of Mood States scale, McNair et al., 1971), and long-term responses, such as anxiety and distress (NIOSH health checklist, Smith et al., 1981; Sainfort and Carayon, 1994).
695 A set of three factor analysis was performed on each of the three groups of variables, i.e. computer system performance, job design and worker strain, to reduce the number of variables used in the statistical analyses. In the analysis of computer system performance three factors emerged: (1) frequency and bother of computerrelated problems such as response time and computer breakdown, (2) duration of computer-related problems, and (3) computer-related resources (e.g., computer training). The factor analysis of job design factors or job stressors yielded four factors: (1) job demands, (2) job support, (3) job content, (4) organizational control, and (5) task or instrumental control. The analysis of worker strain yielded two factors: (1) an index of health problems derived from the NIOSH health checklist (e.g., anxiety, nervousness, and musculoskeletal, gastrointestinal and respiratory problems), and (2) an index of total mood disturbances. For more details on the measures contact the author.
2.4 Statistical analyses First, Pearson correlations were computed to examine the relationship between computer system performance, job design and worker strain. Second, two sets of regression analysis were performed. The first set of regression analyses examined the influence of computer system performance on job design, while the second set examines the influence of both computer system performance and job design on worker strain. These regression analyses can help us compare the direct and indirect effects of computer system performance on worker strain. 3. RESULTS The Pearson correlations between the study variables are shown in Table 1. With regard to the relationship between computer system performance and job stressors, computer problems was related to increased job demands, job content and organizational control, whereas computer resources was related to increased job support and task control. Duration of computer problems was also related to increased job demands. With regard to the relationship between computer system performance and worker strain, a high level of mood disturbances was related to high computer problems and low computer resources, and poor health was related to computer problems. In terms of the relationship between job stressors and worker strain, job content and organizational control were not related to any of the two measures of worker strain. High job demands and low job support were related to both measures of worker strain, whereas low task control was related only to mood disturbances. Table 2 shows the results of the two sets of regression analysis. The first set of regression analysis examines the influence of computer system performance on job stressors. The three measures of computer system performance explain a significant proportion of the variance of 3 measures of job stressor, i.e. job demands (adjusted R2=19%), job support (adjusted R2=9%) and task control (adjusted R2=5%). On one hand, computer problems and duration of computer problems were related to high
696 job demands. On the other hand, computer resources was related to high job support and task control. The second set of regression analysis examines the influence of both computer system performance and job stressors on worker strain. Both measures of worker strain were significantly influenced by computer system performance and job stressors. The amount of adjusted R2 explained was 25% for mood disturbances and 12% for health. Both job demands and job support were significant contributors to mood disturbances and health. In addition, computer resources was also a significant contributor of mood disturbances.
Table 1. Pearson Correlations between the Study Variables JOB STRESSORS Org. Task Demands Support Content control control .36*** -.05 .17" .23** -.07 Comp. pbs. Duration of .23** -.06 .00 .04 .00 pbs. Comp. .08 -.32*** .05 .01 -.27** resources -.02 .17" .23** -.27*** Demands -.02 --.07 .37*** .15 Support .17" -.07 -.22** .16" Content .23** .37*** .22** .31"** Org. control -.27*** .15 .16" .31"** -Task control
STRAIN Mood .21"
Health .20*
.06
.07
.36***
.11
.35***
.32***
-.33*** .01 -.07 -.25**
,.18" .03 .03 -.03
* p<.05, ** p<.01, ***p<.001
Table 2. Regression Analyses (Beta-coefficients, adjusted R2, and p-value) DEPENDENT VARIABLES Org. Task INDEPEND. control Mood VARIABLES Demands Support Content control .05 .14 .25** .02 .05 Comp. pbs. .37*** Duration of -.14 -.03 -.03 -.09 -.06 pbs. .23** Comp. .21" -.34*** -.01 -.08 -.26** resources .00 .29** Demands -.26** Support -.07 Content -.05 Org. control -.09 Task control 9% 0% 3% 5% 25% adj. R2 19% ** * *** p-value *** * p<.05, ** p<.01, ***p<.001
Health .11 -.09 -.06 .32** -.26** -.06 -.03 .02 12%
697 4. DISCUSSION The results of this study show that two measures of computer system performance (i.e. computer-related problems and duration of computer problems) have an indirect effect on worker strain, and that a third measure of computer system performance (i.e. computer resources) has both direct and indirect effects on worker strain. The correlational.analysis shows that computer-related problems is significantly related to high mood disturbances (Pearson correlation=.21*) and poor health (Pearson correlation=.20*). One set of regression analysis examines the combined influence of computer system performance and job stressors on worker strain. This analysis shows that computer-related problems iss not a predictor of either measure of worker strain. Even though computer-related problems is correlated to worker strain, when combined with job stressors, its influence on strain becomes non-significant. This result indicates that the effect of computer system performance on worker strain may be mediated by job stressors. The regression analysis of job stressors as dependent variables indicates that computer-problems is a significant contributor to job demands. Job demands, in turn, is a significant contributor to mood disturbances and health. Therefore, it is plausible that the indirect effect of computer-related problems on strain may be mediated by job demands. That is, computer-related problems is related to high job demands which, in turn, is related to high worker strain. A similar result was found for duration of computer-related problems. The results are slightly different for another measure of computer system performance, that is computer resources. Computer resources is significantly correlated with mood disturbances (Pearson correlation=.36***) and is a significant contributor to mood disturbances in the regression analysis (beta-coefficient=.21*). This result shows that computer resources has a direct effect on mood disturbances. Employees who report high computer resources report low levels of mood disturbances. Computer resources is also related to job support (beta-coefficient=.34***) which, in turn, is related to mood disturbances (beta-coefficient=-.26**) and health (beta-coefficient=-.26**). These results show that computer resources seem to also have an indirect effect on mood disturbances and health via job support. It was expected that other job stressors could also play a mediating role between computer system performance and strain. The study results show that, on the contrary, one job stressor (i.e. task control) has an indirect effect on strain via computer system performance. Task control is correlated with mood disturbances (Pearson correlation=-.25**), but is not a significant contributor to mood disturbances in the regression analysis. Task control is also correlated with computer resources (Pearson correlation=-.27**). Therefore, it is plausible that task control has an indirect effect on worker strain via the mediating role of computer resources. Task control may be an instrument that employees use to gain computer resources which, in turn, are used to reduce their levels of strain. A similar model of the effect of job control on strain via other job design factors has been tested by Carayon et al. (1993). The study results partially support the model of indirect effects of computer system performance on worker strain via job stressors. Computer,related problems, such as computer breakdown and slowdown, seem to have an indirect on worker strain, whereas the effect of computer-related resources, such as training, support
698 from computer staff and documentation, seems to be both direct and indirect. This study has explored various dimensions of computer system performance, some of which have been studied before (see for example the study of computer-related problems by Carayon-Sainfort, 1992), while others have not been studied in relation to worker strain. Further research is necessary to pursue our understanding of the effect of various dimensions of computer system performance on worker strain. The study results show the way to several different strategies for reducing strain in computerized offices. It is not only important to improve the design of the jobs (e.g., increasing job support and job content), but also to improve computer system performance or to reduce the negative impact of computer system performance. REFERENCES
Asakura, T. and Fujigaki, Y. (1993). The impact of computer technology on job characteristics and workers' health. In Human-Computer Interaction: Applications and Case Studies, edited by M.J. Smith and G. Salvendy, Elsevier, Amsterdam, The Netherlands, pp.982-987. Carayon-Sainfort, P. (1992). The use of computer in offices: Impact on task characteristics and worker stress. The International Journal of Human-Computer Interaction, 4(3): 245-261. Carayon, P. (1994). Automation and the design of work: Stress problems and research needs, to be published as a chapter in Stress in New Occupations, edited by R.A. Roe and T. Gaillard. Carayon, P. and Lim, S.-Y. (1994). Stress in automated offices. In The Encyclopedia of Library and Information Science, edited by A. Kent, Marcel Dekker, New York, vol.53, supplement 16, pp.314-354. Carayon, P., Jarvenpaa, E. and Hajnal, C. (1993). Effect of job control on the design of jobs and stress among computer users. In M.J. Smith and G. Salvendy (Eds.) HumanComputer Interaction: Applications and Case Studies. Amsterdam, The Netherlands: Elsevier 1993, pp.863-868. McNair, D.M., Lorr, M. and Droppleman, L.F. (1971). EITS Manual for the Profile of Mood States. Educational and Industrial Testing Service, San Diego, CA. Sainfort, P. and Carayon, P. (1994). Self-assessment of VDT operator health: Validity analysis of a health checklist. International Journal of Human-Computer Interaction, 6(3): 235-252. Smith, M.J. (1987). Mental and physical strain at VDT workstations. Behaviour and Information Technology, 6(3): 243-255. Smith, M.J., Cohen, B.G.F., Stammerjohn, V. and Happ, A. (1981). An investigation of health complaints and job stress in video display operations. Human Factors, 23: 387-400.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
699
Job stressors and depressive s y m p t o m s in Japanese c o m p u t e r software engineers and managers* Takashi Haratani a, Yuko Fujigaki b and Takashi Asakura c National Institute of Industrial Health, Ministry of Labour, 6-21-1 Nagao, Tama-ku, Kawasaki 214, Japan b
Department of Information Science, College of Arts and Sciences, University of Tokyo, 3-8-1 Komaba, Meguro-ku, Tokyo 153, Japan
c Department of Health and Sports Science, Tokyo Gakugei University, 4-1-1 Nukuikita-machi, Koganei, Tokyo 184, Japan
To examine the relationships between job stressors and depressive symptoms in Japanese computer software engineers and managers, a cross-sectional mailed survey was conducted in 1991. Eight subjective job stressor scales were constructed based on a factor analysis and content of items. Depressive symptoms were measured using the Center for Epidemiological Studies Depression Scales (CES-D) as stress response. In this study, 1,694 software engineers and 296 managers were analyzed. Engineers reported significantly higher lack of job control, lack of intrinsic rewards, and ambiguity of career development, lower job overload and change of computer technology than managers. Managers worked longer hours, but used VDUs shorter hours than engineers. Engineers had significantly higher CES-D scores than managers. Multiple regression analysis of each group revealed that lack of intrinsic rewards showed the most significant predictor of CES-D scores. Interpersonal conflict in the project team and lack of control were common significant stressors for each group after controlling for confounding variables. In managers, job overload and changes of computer technology showed significantly associated with depressive symptoms. These results suggested that reduction of such job stressors might improve mental health of software engineers and managers.
1. INTRODUCTION Stress has become one of the most serious health issues of the twentieth century and numerous occupations have received special attention in research and the literature on job stress [1,2]. According to the vast increase in number of computer software engineers, job * This research was supported by the Grant-in-Aid for Scientific Research from the Ministry of Education, Science and Culture in Japan (No. 03229112, 04211113).
700 stress problems for those have been emerging [3]. A lot of job stressors of software engineers have been reported such as rewards, human-resource development, or role-ambiguity [4,5] as well as communication problems [6,7], technical difficulties [8,9], and ambiguity in specification [ 10-13 ]. Weiss [ 14] investigated information systems managers and pointed out their main stressors were role ambiguity, career development, and organizational structure and climate. Depressive symptoms have been employed in industrial setting as a useful measure of the mental health and well-being [15]. Many studies have shown that job stressors are associated with depressive symptoms among workers [16]. However, few study has investigated job stressors and depressive symptoms in software engineers and their managers. The purpose of this study is to examine the relationships between job stressors and depressive symptoms in Japanese computer software engineers and managers.
2. SUBJECTS AND METHODS 2.1. Subjects A cross-sectional mailed survey was conducted of members of "Software Manager Association," "Japanese Federation of Electrical Machine Workers' Union," and "Federation of Telecommunications, Electric Information and Allied Workers" in 1991. Each subject was asked to complete a self-administered questionnaire that evaluated job stressors and depressive symptoms. Questionnaires were distributed for 2,393 engineers and 452 managers. Among them 1,997 software engineers and 300 managers responded the survey. The response rate was 83% for the engineers, and 66% for the managers. 2.2. Measures Subjective job stressor items were designed based on previous studies [4,12,15-17] and our
Tables 1. Measures of subjective job stressors and depressive symptoms
Measures Subjective job stressors Job overload Lack of job control Lack of intrinsic rewards Ambiguity of career development Interpersonal conflict in the project team Difficulty in communication with users Change of computer technology Inadequate computer equipments Depressive symptoms CES-D
Number of items
Range
Cronbach's alpha
4 3 3 3 4 3 3 2
4-16 3-12 3-12 3-12 4-16 3-12 3-12 2-8
83 81 74 76 75 66 .56 .43
20
0 - 60
.88
701 interviews to 26 managers who belong to 10 companies. These items contained specific stressors items for software development work as well as generic stressor items such as job overload, job control. Eight subjective job stressor measures shown in Table 1 were constructed based on a factor analysis and content of 33 stressor items. Cronbach's alpha reliability coefficients of measures ranged 0.43 to 0.88. Five measures of eight subjective job stressors had reliability of 0.74 or above. Depressive symptoms were measured using the Center for Epidemiological Studies Depression Scales (CES-D) as stress response [18,19]. It is designed to measure the current level of depressive symptomatology in the general population. The Japanese version of the CES-D has acceptable reliability and validity [20]. In addition to these measures, objective job stressors were assessed by hours of overtime in previous month and average hours using Visual Display Units (VDUs) at work per day. Age, marital status, education, and years of experience in current job were asked as demographic variables. 2.3. Data Analysis Because managers were almost male, we analyzed only male data (1,694 engineer and 296 managers) in this study to compare between engineers and managers excluding gender effects. Table 2. Comparisons of job stressors and depressive symptoms between software engineers and managers Engineers n=1602-1694 Mean SD Subjective job stressors Job overload Lack of job control Lack of intrinsic rewards Ambiguity of career development Interpersonal conflict in the project team Difficulty in communication with users Change of computer technology Inadequate computer equipments Objective job stressors Hours of overwork in previous month Average hours using VDUs at work per day Depressive symptoms CES-D scores Demographic variables Age Years of experience
Managers n=290-296 Mean SD
t-test
9.9 7.3 8.4 8.5 8.5 6.9 6.8 3.8
3.0 2.1 2.0 2.3 2.7 2.0 2.1 1.6
10.8 6.6 7.4 7.1 8.6 7.1 7.1 3.8
2.4 2.0 1.8 2.0 2.2 1.8 1.8 1.6
40.5 4.0
28.1 2.4
46.8 2.4
28.3 2.0
p<.01 p<.001
18.3
8.8
14.8
7.2
p<.OO1
29.8 3.8
6.1 1.8
41.4 6.5
5.1 1.5
p<.O01 p<.O01
p<.OO1 p<.OO1 p<.OO1 p<. O01 n.s. n.s.
p<.05 n.s.
702 To compare measures of job stressors and depressive symptoms between two groups, t-tests was conducted. Multiple regression analysis of each group including the 8 subjective job stressors, 2 objective stressors and 4 demographic variables was performed to examine the effects of each job stressor on CES-D scores with controlling possible confounding variables. The SPSS-X computer programs were used for these analyses.
3. RESULTS Table 2 shows comparisons of measures in the study between engineers and managers. Managers reported significantly higher job overload and change of computer technology than engineers. Engineers reported significantly higher lack of job control, lack of intrinsic rewards, ambiguity of career development than managers. Managers worked longer hours, but used VDUs shorter hours than engineers. Engineers had significantlyhigher CES-D scores than managers. Table 3. Effects of job stressors and demographic variables on CES-D scores Engineers n=1288 Pearson R Beta Subjective job stressors Job overload Lack of job control Lack of intrinsic rewards Ambiguity of career development Interpersonal conflict in the project team Difficulty in communication with users Change of computer technology Inadequate computer equipments Objective job stressors Hours of overwork in previous month Average hours using VDUs at work per day Demographic variables Age Marital status (single/married) Education (college or higher/lower) Years of experience
.22 .28 35 28 31 21 20 16
*** *** *** *** *** *** *** ***
.12 *** .12 ** -.10"** .12"** -.08 ** -.09 **
10"** 12"** 25 *** 09 ** 13 *** 07 ** 07"* 00
.31 *** .33 *** .42 *** .27 *** .33 *** .22 ** .30"** .17 **
.04 .02
.12 .12
-.05 .08" -.07 ** -.05
Multiple R = .52 R square = .27
* p<.05, **p<.O1, ***p<.O01
Managers n=244 Pearson R Beta
.03 .18 ** -.08 -.03
.22"** .12" .31 *** .05 .14" .00 .15 * -.04 .02 .09 .08 .21 *** .03 -.03
Multiple R = .62 R square = .39
703 The results of multiple regression analyses including job stressors and demographic variables are shown in Table 3. Lack of intrinsic rewards showed the highest and significant Pearson correlation coefficient and beta both in engineers and in managers. Interpersonal conflict in the project team and lack of control were common significant stressors both two groups. In managers, job overload and changes of computer technology showed strongly and significantly associated with depressive symptoms.
4. DISCUSSION Multiple regression analysis of each groups revealed that lack of intrinsic rewards showed the most significant predictor of CES-D scores. Interpersonal conflict in the project team and lack of control were common significant stressors for each group after controlling for confounding variables. In managers, job overload and changes of computer technology showed significantly associated with depressive symptoms. These associations do not necessarily mean causal relationships, because this study was a cross-sectional questionnaire survey. However, these results suggested that lack of intrinsic rewards, interpersonal conflict in the project team, and lack of control could be common risk factors for depressive symptoms in Japanese computer software engineers and managers. Therefore, reduction of such job stressors might improve mental health of software engineers and managers. Software engineers had significantly more depressive symptoms and higher lack of job control, lack of intrinsic rewards, and ambiguity of career development than managers. Software engineers should be informed of the meaning of their job and use their skills or abilities in order to reduce lack of intrinsic rewards. Their controllability should be increased and they need to be clearly informed of career development. Managers are key persons for engineers' stress problem, as management of personnel and production affects engineers. Software project teams consisted of several or dozens of members of engineers. Interpersonal conflict may emerge not only among engineers but also between engineers and managers. Communication is very important in software developing work. Software managers must deal with task allocation among engineers, management on time schedule of development process, and confirmation of the progress in making products before deadline. In software development work, there are great individual differences of abilities of workers. An able engineer's productivity is more than ten times as high as that of a non-able engineer [21]. In addition to individual difference in software developing ability, lack of systematic job assignment plans was pointed out [22]. Managers are required to assign a suitable job to each member's ability. Improvement of management method is needed. Time management is one method to improve management of software projects. However, tight time management could cause serious time pressure for software engineers [11-13]. It will sacrifice the quality of product and engineers job satisfaction [23]. Therefore, time management should be useful for engineers and managers. In this study, managers worked more overtime than engineers, felt more overload and had to keep up with change of computer technology, and overload and change of computer technology had strong association with depressive symptoms of managers. Though they manage their subordinates, they must manage themselves to prevent and promote their health.
704 As change of computer technology is rapid, educational opportunities should be provided not only for engineers but for managers. To promote mental health of software engineer and managers, stress management training for them in the worksite are needed. It will reinforce stress resilience of workers. Reduction of job stressors also should be emphasized to prevent mental health of workers. Further longitudinal or intervention studies are required to clarify the effect of job stressors on mental health of software workers.
REFERENCES
.
7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23.
International Labour Office. Conditions of Work Digest 1992;11(2). International Labour Office. World Labour Report 1993;6:65-76. Brod C. Technostress. Massachusetts: Addison-Wesley, 1981. Lo MW. SIGCHI Bulletin 1987;18:25-9. Saleh SD, Desai K. IEEE T Eng Manage 1986;33:6-11. Ivancevich JM, Napier HA, Wetherbe JC. Commun ACM 1983;26:800-6. Ivancevich JM, Napier HA, Wetherbe JC. Inform Manage 1985;9:77-85. Keenan A, Newton TJ. J Occup Psychol 1987;60:133-145. Zavala A. In: Human-Computer Interaction. Amsterdam: Elsevier, 1984;365-70. Fujigaki Y. In: Berliguet L, Berthelete D, eds. Work with Display Unit 89, Amsterdam: Elsevier, 1990; 395-402. Fujigaki Y. In: Noro K, Brown O, eds. Human Factors in Organizational design and Management-3, Amsterdam: Elsevier, 1990;255-258. Fujigaki Y. Occupational Stress of Software Engineers. Kanagawa: The Institute for Science of Labour, 1992. Fujigaki Y. Am Programmer 1993;6(7):33-8. Weiss M. MIS Quart 1983;3:29-43. Cooper CL, Davidson M. In: Kalimo R., E1-Batawi MA., Cooper CL, eds. Psychosocial factors at Work and their relation to health, Geneva: WHO, 1987;99-111. Kawakami N, Haratani T, Araki S. Scand J Work Environ Health 1992;18:195-200. Donovan R. Social Casework 1987;68:259-66. Radloff LS. Appl Psychol Measurement 1977;1:385-401. Marsella AJ, Hirschfeld RMA, Katz MM, eds. The Measurement of Depression. New York:Guilford Press 1987. Shima S, Shikano T, Kitamura T, Asai M. New Self-Rating Scale for Depression. Jpn J Clin Psychiat 1985; 27:717-23. Newman RC, et al. Inform Manage 1987; 13" 171-8. Totsuka H, Nakamura K, Umezawa T. Software Industry in Japan, Tokyo: University of Tokyo Press, 1990. DeMarco T, Lister T. Peopleware; Productive Projects and Teams. New York: Dorset House Publishing, 1987.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
705
Job Stress Characteristics of Computer Work in Japan N. Kawakami', C. R. Roberts b and T. Haratani c "Department of Public Health, Gifu University School of Medicine, 40 Tsukasamachi, Gifu 500, Japan bDepartment of Psychiatry and Behavioral Sciences, University of Texas School of Medicine, P.O.Box 20706, Houston, TX 77225, U.S.A. CNational Institute of Industrial Health, Ministry of Labour, 6-21-1 Tama-ku, Kawasaki 214, Japan
I. INTRODUCTION
Job stress has been recognized as a major cause of health problems at work [1]. Also recent technological development might change job descriptions and occupational class structures in the industry. From both medical and sociological viewpoints, it is important to assess the job stress characteristics of computer-related occupations (e.g., computer engineers, computer technicians, programmers) and other workers who engaged in work with computer equipments. Karasek [2] reported from the 1972-77 surveys in the U.S. that computer engineers had higher job demands as well as higher decision-latitude (control) over job, suggesting that they are "active" job. On the other hand, recent studies [3, 4] have pointed out that computer engineers, as well as operators of computer equipments, perceived higher job overload, lower control over job and higher psychological distress, although some studies did not [5]. Since these studies used empirically developed scales of job stress, it is needed to compare the job stress characteristics of these occupations/jobs using wellstandardized and theoretically-based measures of job stress. We conducted a survey of employees in a computer company in Japan using a standardized classification system of occupation and established job-stress scales. The purpose of the study are 1) to know the job stress characteristics of computer-related occupations in Japan, such as computer engineers, computer technicians and programmers, 2) to classify these computer-related occupations further based on the patterns of job stress, and 3) to compare the job stress characteristics among clerical workers and machine operators using computer equipments with different frequencies.
2. SUBJECI'S AND METHODS 2.1. Subjects We surveyed a random sample (n=500) from 2,500 employees in one factory and all
706 employees (n=2,700) of another factory of a computer company in Japan using a mailed questionnaire. A total of 2,314 (72%) returned the questionnaire. Data from 1,552 male and 262 female respondents (57% of the initial target population) who completed the questionnaire were analyzed. 2.2. Classification of occupations
The subjects were asked to briefly describe "kind of their work" and their "most important duties" in the questionnaire. Occupation was classified according to the 1980 U.S. Census Classification of Industries and Occupation and categorized as follows: three computer-related occupations, i.e., computer engineers (code 055), computer technicians (code 213), programmers (code 229); and six other occupations, i.e., managers, other professionals, other technicians, clerical workers, machine operators and transportation/others. The number of subjects by occupational groups was shown in table 1. Table 1. Sex- and occupational distribution of the subjects Occupation Computer-related occupations: A. Computer-engineers B. Computer-technicians C. Computer programmers Other occupations: D. Clerical workers E. Managers F. Other professionals H. Other technicians G. Machine operators I. Transportation/others
Male
Female
292 222 18
15 11 7
239 140 26 35 540 40
165 3 1
4 49 7
2.3. Measurement of job stress
Job-stress of individual workers was assessed using ten job stress scales developed in the U.S., i.e., two scales of job overload (Quinn et al.'s and Caplan et al.'s) [6, 7], role ambiguity and role conflict [8], job future ambiguity [7], job control [9], skill utilization [7], social support from supervisors and coworkers [10], as well as job centrality [11]. Although the two scales of job overload highly correlated (r=0.62), the Quinn et al.'s scale mainly assesses the speed and time pressures, while the Caplan et al.'s rather focuses on a longer-term, projectbased job overload. The Japanese translations of these scales were prepared by the authors and checked through a back-translation procedure. These job scales showed high internal consistency reliability (Cronbach's alpha) in Japan, ranging 0.64-0.93, which were very similar to those observed in a sample of the U.S. employees. 2.4. Other variables
The subjects were asked about the frequency of their use of computer equipments, such as
707
computer terminals, PCs and computer-controlled production machines, at work. Clerical workers and machine operators were further classified on the basis of their frequency of use of computer-related equipments, i.e., high-frequent (75% or more of work), moderate (25-50% of work) and low-frequent (almost none) users. The Center for Epidemiologic Studies Depression (CES-D) scale was used to assess levels of psychological distress.
45 40
MALES
E
O
-
¢-
._o 8
o
t_
tO
35
-
gl C
A
"
~Ii
D H
c~
o 30
E
_
MALES
O
B
N
-
~
B
H
7
25 l e
20
I
13
35 30 o
I
I
14 15 16 Job Overload (Caplan et al.)
i 17
9
D
B
I
I
10 11 Supervisor Support
12 E
-
FEMALES
• "
QeG
_
A
o
25 -
o ~J ..Q .--} o
I
10
FEMALES A
L,-
¢,.
6
I
•
N
•C 20
D
6 -
B
O
-
GO
4 -
15 -
I
10 c
10 15 Job Overload (Caplan et al.)
20
4
6 8 10 Supervisor Support
Figure 1. Comaprison of age- and education-adjusted mean scores of job stress among computer engineers (A), computer technicians (B), programmers (C) and other occupations (D-I) on the basis of a combination of job overload (Caplan et al.) and job control or supervisor support and skill utilization. See table 1 for more detail about abberevated labels for occupations. Size of each circle is proportionate to the number of subjects in the occupation. Arrow indicates that the circle is under others.
12
708 2.5. Dala analysis Scores of the ten job stress scales were compared among the nine occupations by sex controlling for age and education (ANCOVA). Male computer engineers and computer technicians were subjected to a cluster analysis to obtain a further possible classification of the computer-related occupations on the basis of the ten job-stressscale scores (Ward's method). The validity of the obtained classification was examined by comparing age, overtime hours per month and the CES-D score among the cluster groups. The ten job stress scales scores were also compared by the groups classified on the basis of the frequency of use of computer equipments in clerical workers or machine operators by sex, controlling for age and education (ANCOVA). These analyses were conducted using the SAS system on a PC.
3. RESULTS In males, computer engineers had significantly higher Caplan et al.'s score of job overload than computer technicians and clerks (figure 1, p<0.05); they had significantly higher job control than computer technicians, programmers and machine operators (p<0.05), while mean job control score in computer engineers was verysimilar to that in clerical workers and was significantly lower than that in managers (p<0.05); and they had significantly higher scores of skill utilization than clerical workers and machine operators (p<0.05). Computer technicians and programmers had significantly lower job control than clerical workers, while computer technicians had higher job control than machine operators (p<0.05). In females, computer engineers and technicians had significantly higher supervisor support than other technicians (p<0.05). In the cluster analysis of computer engineers and computer technicians (table 2), pseudo F and t 2 statistics indicated eight possible clusters: (1) "Very high demand" group with higher overload and longer overtime, (2) "higher leader" group with the highest control/support scores, (3) "middle leader" group with the second highest control and younger age than the group 2, (4) "low demand/younger", (5) "medium demand/intermediate age" and (6) "high role conflict/middle-aged" core groups, (7) "high future/role ambiguity" group, and (8) a small number of maladjusted young group with lower overload, higher future/role ambiguity and lower control, skill use and support. The group (1), (5), (7) and (8) showed higher CES-D depression scores than other groups. The group (2) and (3) consisted of relatively higher proportion of computer-engineers. However, in general, the clusters did not agree with the classification of engineers/technicians. Male machine operators using computer equipments moderately (25-50%) in their work had higher job overload and job control than high- and low-frequent users, after controlling for age and education (data not shown, p<0.05). In female clerical workers and machine operators, moderate users had lower role ambiguity, higher skill utilization and higher supervisor support than high- and low-frequent users, after controlling for age and education (data not shown, p<0.05).
709 Table 2. Cluster analysis of male computer engineers and technicians based on ten job stress scales: comparison of job stress and other variables among eight extracted clusters. %of engineers
Job stress characteristics of the cluster group*,** #
N(%) O1 0 2 RC RA FA JC SU SS CS CT
Mean Mean Age-adjusted age over- depression (yrs) time (CES-D) (hrs/ score too.)
1
88(17)
H
H
H
M
L
M
M
M
M
M
66
34
55
21.6
2
41(8)
H
H
M
L
L
H
H
H
H
H
71
40
47
12.5
3
66(13)
M
M
M
L
L
H
H
H
H
H
71
37
42
13.1
4
50(10)
L
L
L
M
M
M
M
M
M
L
40
27
44
16.1
5 100(19)
M
M
M
M
M
M
M
M
M
M
48
30
51
23.0
6 103(20)
M
M
H
M
M
M
M
M
M
M
56
35
42
18.5
7
54(11)
M
M
M
H
H
L
L
L
L
M
46
30
41
29.2
8
12(2)
L
L
L
H
H
L
L
L
L
L
58
25
54
28.4
* O1, job overload (Quinn et al.); 02, job overload (Caplan et al.); RC, role conflict; RA, role ambiguity; FA, future ambiguity; JC, job control; SU, skill utilization; SS, supervisor support; CS, coworkers support; CT, job centrality. ** H, the first and second highest groups; M, intermediate groups; L, the first and second lowest groups. # Cluster number.
4. DISCUSSION Our study indicated that male computer engineers had higher job overload and higher skill utilization. However, job control in computer engineers and computer technicians were similar to or even lower than that for clerical workers in males. A previous U.S. study [2] failed to separately measure job control and skill utilization of computer engineers because it used a composite scale of these two dimensions of job stress. Our findings suggest that computer engineers and computer technicians are not higher class occupations in terms of decision authority or control over job in Japan. Relatively lower job control compared with higher levels of job overload in these occupations characterizes them as a "higher strain" job, in which more adverse health effects have been reported [12]. In future research, more attention should be made on job control, as well as job overload, in computer engineers and technicians. The cluster analysis suggests a new classification of computer-related occupations based on the patterns of job-stress. The proposed classification was supported by differences in mean age and levels of depression, suggesting that the classification is related to occupational
710 life cycles and is useful in identifying maladjustment groups in the computer-related occupations. Job-stress characteristics of computer-rdated occupations reported in previous studies [2-5], such as higher demands, future ambiguity and lower workplace support, might be not general characteristics of the occupations, but those of heterogenous groups in the occupations. A heterogeneity of computer engineers/technicians also has been pointed out by using the job analysis [3]. A future study is needed to clarify whether the classification is supported by other classification systems of computer-rdated occupations, such as job analysis and social class. Our findings also indicated that use of computer equipments in 25-50% of their work is associated with higher job control in male machine operators and with less role ambiguity, higher skill utilization and supervisor support in female clerical workers and machine operators. While high-frequent user of VDT or other computer equipments perceived more job stress including lower job control [4], low-frequent use of these equipments might be associated with passive and isolated work situation. Our findings suggest that 25-50% use of computer equipments is best for clerical workers and machine operators in terms of prevention/reduction of job stress. No clear conclusion could be drawn from our study conceming job-stress among female computer engineers and technicians because of a small number of the subjects. Organizational or individual factors specific to Japanese culture might limit the generalization of the present findings to other countries. Our study indicates that job stress research on computerrelated occupations and workers using computer equipments is promising. Future crosscultural research in Japan and other countries is needed to compare the job stress characteristics of these occupations among male and female workers using standardized measures of job stress.
REFERENCES
1. International Labour Office, World Labour Report 1993, ILO, Geneva, 1993. 2. R.A. Karasek, Int. J. Health Serv., 19 (1989) 481. 3. Y. Fujigaki, Occupational Stress of Software Engineers (in Japanese), Institute for Science of Labour, Kanagawa, Japan, 1992. 4. R. Alcalay and R.J. Pasick, Soc. Sci. Med. 17 (1983) 1075. 5. S. Ezoe, et al., Environ. Res., 63 (1993) 148. 6. R. Quinn, et al., Survey of Working Conditions. U.S. Government Printing Office, Washington, D.C., 1971. 7. R.D. Caplan et al., Job Demand and Worker Health, NIOSH, Washington, D.C., 1975. 8. J.R. Rizzo et al., Adm. Sci. Q. 15 (1970) 150. 9. D.B. Greenberger, Personal Control at Work: Its Conceptualization and Measurement. Technical Report 1-1-14. University of Wisconsin-Madison, 1981. 10. J.S. House et al., Occupational Stress and the Mental and Physical Health of Factory Workers, University of Michigan ISR: Ann Arbor, 1980. 11. T.M. Lodahl and M. Kejner. J. Appl. Psychol. 49 (1965) 24. 12. R.A. Karasek and T. Theordl, Healthy Work, Basic Books, New York, 1990.
IV.5 Input Devices
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
713
An integrated haptographical user interface using a force feedback mouse A.J. Kelley a, T. Higuchi a, and S.E. Salcudean b aDepartment of Precision Machinery Engineering, University of Tokyo 7-3-1 Hongo, Bunkyo-ku, Tokyo, Japan 113 bDepartment of Electrical Engineering, University of British Columbia 2356 Main Mall, Vancouver, British Columbia, Canada V6T 1ZA A novel input-output device utilizing force feedback has been developed for use in everyday point-and-pick functions. Compared with a previous prototype, the significant improvements in motion range were made. The integration of a dedicated microcontroller subsystem with a Macintosh host is presented. An outline of a new haptographical user interface is given. Initial selection time comparisons between the force feedback device and a regular mouse are provided. 1. INTRODUCTION In the early eighties, a significant milestone in human-computer interaction was reached when the first mouse-style point-and-pick devices were commercialized for use with graphical userinterfaces (GUIs). Since then, much research has focused on interface software in order to improve graphical presentation and functionality. However, it can be argued that there have been relatively few advances in interface hardware. Indeed, even though the standard variety of mice and trackballs have been complemented with several novel and more compact devices, the fact remains true that currently available devices are only passive tools which do not provide direct feedback. In 1991, an attempt to change this situation was initiated at the University of British Columbia through the development of a novel force feedback input/output device, known as the MagicMouse prototype (Kelley and Salcudean, 1993 and 1994). This technology allows GUI objects to be augmented with realistic active and passive mechanical properties that can be perceived by the user. Experience with the original prototype, suggested that if the user's sense of touch could be extended to the user-inte~ace "desktop" in this way, measurable increases in user performance, speed, and efficiency at lower levels of mental and physical stress might be possible. The fundamental idea of such an application was also considered by Hannaford et al (1989) based on work in force feedback interfaces for teleoperation but no implementation was provided. The goals of the work presented in this paper were to seamlessly incorporate a new MagicMouse electromechanical design into an integrated haptographical user interface (HUI) on a Macintosh TM platform with NuBus TM slots and perform some initial human factors experiments. 2. ELECTROMECHANICAL DESIGN Recently, there has been a flurry of research involving haptic computer interfaces using force feedback. Unfortunately, most systems are largely inappropriate for everyday computer interaction because they were designed for other high-end (expensive) applications which may require more than two degrees-of-freedom (DOF), large motion ranges, and heavy computation. In addition to MagicMouse, the AT&T (Schmult and Jebens, 1993) and Sandpaper (Minsky et al., 1990) joysticks are the only known devices that are practically applicable to HUIs. However, we consider the joystick implementation to suffer from two drawbacks: (1) if used in rate-control mode, the position-velocity-position mapping process the user must perform is taxing and difficult (2) if used in absolute positioning mode, implementation of high resolution end-point position sensing is difficult. Instead, for MagicMouse we have pursued a planar design which has a more natural position mapping to the display and has potentially sleeker styling and better ergonomics. The electromechanical design of this version of MagicMouse was completely revised from the
714
original. The important issues addressed were to: (i) improve the motion range from 17x17 mm 2 to 40x40 mm 2 (ii) minimize the overall footprint of the device to the order of a typical mouse pad; (iii) improve position sensing resolution; (iv) incorporate an ergonomically comfortable handle; (v) add a vertical tactile feedback element to the click-button in the handle; and (vi) pursue low cost alternatives in the kinematic design.
Figure 1. MagicMouse and an Apple mouse A photograph of MagicMouse and an Apple TM mouse are shown in Figure 1. The lower portion of the device remains stationary O~l the desktop. The handle above translates with low friction in two DOFs and can also rotate the vertical axis. Force feedback is achieved using a planar, directdrive electromagnetic actuation scheme with high frequency response. High resolution encoders are employed for position sensing. The top of the handle has a plastic leaf spring that rests above a single click-button similar to those found in regular mice. In addition, however, the leaf spring can also be driven upward with a direct-drive voice coil actuator enabling tactile signals to be transmitted to the user's fingertips. Design details are discussed in Kelley et al. (1995) and Kelley (1995). 3. SYSTEM INTEGRATION AND C H A R A C T E R I Z A T I O N Following the path taken with the original prototype, control of MagicMouse is performed using a dedicated microcontroller subsystem. This choice is based on the fact that human haptic senses require substantially higher "refresh" rates exceeding 1 kHz to create a quality illusion of smooth continuity. Because of this, the Macintosh host cannot be part of a real-time control loop for the haptic presentation of the GUI. System software or toolbox routines which functionally define the user-interface have to operate quickly and imperceptibly behind the higher level applications code. Since the user-interface is really only window-dressing for other tasks, its implementation cannot and should not dominate CPU cycles for the sake of force feedback. Thus, the system integration, seen conceptually in Figure 3, was chosen. The microcontroller and Macintosh are connected through a high bandwidth NuBus interface which provides a means for the microcontroller to send cursor position and click-button information to the Macintosh and for the Macintosh to send supervisory control commands to the microcontroller. The microcontroller subsystem is based around an Intel TM N87C196KC20 embedded microcontroller with 32 Kbytes of external 16-bit RAM. At the electromechanical interface, encoder signals are fed through two 16-bit quadrature signal counters. The translational actuators are bi-directionally driven from an external supply to two H-bridge configured power MOSFET circuits which are controlled by PWM signals. The tactile element is driven unidirectionally by a single power MOSFET, also controlled by a PWM signal. At the NuBus interface, a 2 Kbyte 8-bit dualport RAM is the communication bridge between the microcontroller and the Macintosh. Table 1 outlines the characteristics of the MagicMouse system. In the microcontroller, sampling and force update rates exceeding 2 kHz have been implemented. For the haptic augmentations discussed in the following section, sense-to-actuation latencies of 100-250 Its are typical.
715
o0 haptic feedback
Figure 3. Conceptual illustration of system
Table 1 Summary, of Ma~icMouse S~,stem Characteristics Dimensions Length 20.5 cm Width 18.6 cm Base Height 4 cm Mass Total 2.5 kg Movin~ Components 230 g Translational Maximum Force +6.2 N Stage Force Resolution 24 mN Static Friction 0.5 N Motion Range 40x40 mm 2 Sensor Resolution 1 ~m Position Freq. Resp. 135 Hz Force Freq. Resp. < 4 kHz Tactile Max Deflection 7 mm Element Max Deflection Force 1.6 N Force Freq. Resp. < 3.5 kHz
4. D E V E L O P M E N T OF A H A P T O G R A P H I C A L USER I N T E R F A C E The dominant goal in our HUI development was to take an existing GUI and provide a haptic enhancement. For it to be useful, it is obvious that any out-of-the-box programs which use the standard components of the GUI should, by definition, automatically exhibit the corresponding haptic functionality. On the Macintosh, this means that we must be able to intercept particular calls made by applications to the Toolbox in order to have real-time knowledge of modalities and changes in the visual representation. In practice, this is done by patching Toolbox trap routines with special HUI code which updates the microcontroller through supervisory commands. In our implementation, this patching process is performed at startup time as a system extension. Device driver routines have been created so that the haptic patch routines can communicate with the microcontroller through the standard Device Manager interface. At the lowest level, the device driver delivers information to the microcontroller via the dualport RAM using a first-in first-out mailbox. When the microcontroller is not busy with the sense-control-actuation process, it removes the information and updates its haptic representation accordingly. In the opposite direction, the microcontroller sends cursor position and button information to the Macintosh using another mailbox which is monitored during the display's vertical blanking operation.
4.1. Specific Implementations Whether MagicMouse is used with a large or small display, the cursor positioning resolution remains constant to maintain the absolute sizes of HUI objects from one system to another. To accommodate positioning on a large 1152x870 pixel display, a nominal resolution of 32 ~tm/pixel was selected. If a smaller display is used, this means that there would be a logically smaller MagicMouse workspace. Mechanical bounds are implemented by the emulation of hard walls at the display edge using a discrete-time PD control model with a braking pulse similar to that described in Salcudean and Vlaar (1994). Stiffnesses of 11600 N/m, damping of 47 N.s/m, and braking pulses of 188 N.s/m proved to be very effective. Based on ideas discussed in Kelley and Salcudean (1994), haptic augmentations to the Macintosh GUI were made to objects in the following Toolbox Managers: • Window Manager: Compliance control is used to attract the mouse to center axis of the title bar and also to the centers of close, zoom, and grow boxes. Negative velocity control is used to let the user stop at one of these objects more easily, yet not strongly enough to impede volitional motion past the object in a disruptive way. When the cursor enters a close box for example, a tactile signal is also sent to the user. During grow or drag operations, velocity control is used to create a viscous drag feel. • Menu Manager: Compliance control and tactile feedback are used between items in the menu bar and in menus to create the feel of small detents as the cursor travels across them. Not only is the mouse helped away from ambiguous positions near an item edge, but the user can easily find a particular menu item without looking. The motion of the mouse is bound at the bottom (and top) of a menu, allowing rapid vertical motions without fear of overshoot the end.
716
•
• •
C o n t r o l M a n a g e r : Standard radio button, check box, push button, and scrollbar objects were given centering compliance, damping, and tactile attributes to help aid their selection. When a scrollbar is actively being dragged, velocity feedback is used to create a drag feel along the axis and stiff compliance control is used to prevent the cursor from overshooting the ends of the scrollbar and from "falling off" laterally. Nonstandard "MagicButton" controls were created for use as short-cuts to common operations. These are haptic objects with graphic a graphic image resembling the profile of a button. When pressed, the user feels a spring compression and finally a click-like release at the end of the button's travel. List Manager: Detent emulation like that used with menus are used for list items. D i a l o g M a n a g e r : When a modal dialog appears on the screen, the user is given very little visual feedback concerning the fact that the dialog must be attended to first before any other operations will be permitted. The use of a "gravitational" force to pull the mouse into the dialog box can be an effective tool of informing the user of this modality.
5. H U M A N F A C T O R S Although it is possible to measure the performance of one device against another in a controlled experiment, the results do not always reflect user preferences. For example, studies show that mice are significantly faster than trackballs, yet the latter are still popular (Foley et al., 1984). Similarly, although studies have shown the use of gains and thresholds in mice actually degrades performance, such functionality is widely used (Trankle and Deutschmann, 1991; Jellinek and Card, 1990). In both cases, it is conjectured that qualities other than positioning speed measures are important, i.e. stationary operation or less desktop usage. Despite this situation, we believe that a comparison study between a regular Apple mouse and MagicMouse is a reasonable starting point for force feedback evaluation. The test presented here is based on a Fitts' style reciprocal tapping test (Fitts, 1954) and the technique used by Jellinek and Card (1990). In a first experiment, fourteen experienced computer mouse users were recruited to take part in a "ribbon" session, which is a study of selection times between two vertical ribbon-like strips on the display. The subjects were asked to select the ribbons in altemation as quickly as possible for the duration of one set, or 20 repetitions at a particular ribbon separation distance., An entire trial involved a total of 9 sets at a variety of distances presented in random order. Between sets, the subjects were able to rest and begin the next set on their own volition. Subjects were told to emphasize speed of selection, while keeping errors to a minimum. Three trials were performed in order using: (1) an Apple mouse, (2) MagicMouse without force feedback, (3) MagicMouse with force feedback. Users were encouraged to practice freely with an individual device before starting its trial; however, most chose to start after only a few practice selections. The widths of the ribbons were 16 pixels. With force feedback, a compliance of 2250 N/m (to a maximum of 0.68 N) towards the axis of the ribbon and a damping coefficient of 58 N.s/m were applied when the cursor was within the bounds of the target. This gave the ribbons the same attributes as those of window title bars in the HUI implementation. In a second experiment, twelve of the previous participants performed a "radio button" session, identical to the previous session except radio button controls were used as targets instead of ribbons. Additionally, testing with MagicMouse was only performed with force feedback. Although the radio buttons have circular graphics, they are defined by a 16x16 pixel regions that are sensitive to selection. The haptic parameters are the same as with the ribbon test except compliance control is used both vertically and horizontally towards the radio button center. The mean selection time results, given graphically in Figure 5, show linear relationships with distance. Thus, for a given device condition, it is useful to consider the point at zero distance being representative of the minimum achievable selection time, which is comprised of the time to verify that the cursor is within the target and the time to press the click-button. Also, we can interpret the inverse slope of the line to be the average execution velocity of the pointing portion of the task. A summary of these and other statistical data are presented in Table 2. From the ribbon trial data, we can see that the minimum selection times for both the Apple mouse and MagicMouse without force feedback are essentially equal as would be expected. However, the execution velocity of MagicMouse without force feedback is lower. Two reasons for this might be that (1) the users were unaccustomed to the 32 ktm/pixel resolution, which is much finer than the 292 ~trn/pixel, no gain resolution of the Apple mouse and (2) high stiction - i.e. static friction that is greater than dynamic friction - make fine adjustments from a resting position difficult. This indicates that a slightly larger motion range and a tuning of friction characteristics
717
0
1600
5
MagicMouse Handle Motion Amplitude (mm) 10 15 20 25 30
35
..
i ....
~ ....
~ ....
~ ....
~ ....
~ .... .
O
o ...... , . . . .
1200
...-"
.-
. . , ..... , i
..O'"
.
. . ~ " i i i .~,,] ~.- .... '.
ram.-"
...... , ..... ° ....
9 ....... . o•- " .... i - ~ ...... °..O."oo
i1
..--
~
.
jd
I
0
800
.... ~--.... i--• .... 0----o--
400
0
200
400
Ribbons - Apple Mouse " Ribbons- MagicMouse (without force) Ribbons- MagicMouse (with force) Radio Buttons - Apple Mouse Radio Buttons - MagicMouse (with force) 600
800
1000
1200
Pointer Motion - Horizontal Amplitude (Pixels) Figure 5. Mean selection times versus distance for ribbon and radio button tests Table 2 Minimum selection times, execution velocities, and statistical data minimum execution velocity regression mean standard time ~ m s ) ~pixels/ms) coefficient R deviation ~ms) Ribbons Apple Mouse 817 0.50 0.992 291 MagicMouse without ff 823 0.62 0.988 390 Ma~icMouse with ff 544 0.31 0.980 193 Radio Apple Mouse 938 0.58 0.977 347 Buttons Ma~icMouse with ff 561 . 0.58 0.980 219
error rate 7.2 % 6.2 % 5.0 % 2.4 % 2.2 %
might be important in future device developments. The data does show that there are significant decreases in mean selection time when using force feedback. This is characterized by both increased velocity and decreased m i n i m u m time. The reason behind the velocity increase is that subjects were able to move their hands quickly without fear of overshoot because the damping would dissipate most of the m o m e n t u m if the handle is grasped lightly. The reason for a decreased m i n i m u m time may be justified by the fact that the verification stimulus from one's haptic sense is more acute than the visual one. For the radio button data, a constant overall improvement in selection time with force feedback is seen. In this case, the velocities are equal because the likelihood of hitting the target with a single motion - and being stopped by damping - is low due to its size. As seen in Table 2, the average standard deviation of mean selection time across the various target distances is lowest with force feedback for both the ribbon and radio button trials. If we compare the two data sessions together, it is seen that the minimum times for both cases of force feedback are essentially equivalent. On the contrary, the minimum times for both cases using the Apple mouse "are different. From this, it can be inferred that the visual verification of positions inside the small radio button is more difficult than for the ribbon. Also, it might be proposed that not only is the haptic verification more salient than the visual one, but the "information" the user receives is largely not geometry dependent in these cases. Because the user feels essentially the same compliance and damping, the sizes of the objects has little effect on the m i n i m u m selection time. More thorough testing would have to be done to verify these hypotheses. In qualitative terms, these first-time M a g i c M o u s e users generally expressed unrestrained
718 enthusiasm for force feedback technology. The device's smooth motion, not subject to x-y coupling or slippage common with ball mice, was also favourably commented on. However, some subjects suggested that the tactile element might become irritating with prolonged use. In this regard, a refined design and more testing will be necessary. The most interesting result came through comments made by every subject during the first session. Typically, the Apple mouse trials were characterized by initial excitement which faded quickly as the subjects realized the task was more mentally and physically demanding than they had expected. Complaints of cramped hands and strained eyes prevalent until the beginning of the trial with force feedback. At this point there was a startling change - participants all began to voice there surprise (and relief!) at how cognitively and physically easy the task became. Even though these results are favourable, it is recognized that the testing conditions do not realistically reflect typical mouse operations - repetitive tasks rarely occur, and a user generally does not devote his entire concentration to the point-and-click process. Normally, a user's mental capacity would be shared with other thoughts and his visual acuity reduced by multiple distractions. In this type of real life situation, it is expected MagicMouse and a haptographical user-interface would bring an even greater benefit. 6. CONCLUSIONS The goal of seamlessly interfacing a new MagicMouse design into a system including a Macintosh host was reached. By adding extensions to the Macintosh system, fully-integrated haptic enhancements to the existing GUI were made. The use of force feedback to quantitatively improve reciprocal point-and-click task execution times was demonstrated. Perhaps more importantly, the qualitative reaction from volunteer users was of excitement and expectation. In the future, along with work to improve the pointing device, research aimed at making a more complete haptographical user interface will be necessary. Psychophysical studies will be increasingly needed to guide the designer down the most fruitful path.
REFERENCES Kelley, A.J., and Salcudean, S.E., 1994, .On the Development of a Force-Feedback Mouse and its Integration into a Graphical User Interface, Radcliffe, C.J. (ed.), Dynamic Systems and Control: Volume 1, DSC-55(1), ASME: 287-294. Kelley, A.J., and Salcudean, S.E., 1993, MagicMouse: Tactile and Kinesthetic Feedback in the Human-Computer Interface using an Electromagnetically Actuated Input/Output Device. Submitted for publication in the IEEE Transactions on Robotics and Automation, January, conditionally accepted for publication. Kelley, A.J., Higuchi, T., and Salcudean, S.E., 1995, Development of an Improved Force Feedback Computer Mouse, Submitted for oral presentation at SICE '95 in Sapporo, July 26-28. Kelley, A.J., 1995, Integration of a Force Feedback Mouse into a Haptographical User-lnterface, Master's Thesis, University of Tokyo. Hannaford, B. and Szakaly, Z., 1989, Computer Input Device with Force Feedback, Jet Propulsion Laboratory Invention Report, NPO-17520/7034, November. Salcudean, S.E., and Vlaar, T.D., 1994, On the Emulation of Stiff Walls and Static Friction with a Magnetically Levitated Input/Output Device, Radcliffe, C.J. (ed.), Dynamic Systems and Control: Volume 1, DSC-55(1), ASME: 303-309. Minsky, M., Ouh-Young, M., Steele, O., Brooks, F.P. Jr., and Behensky, M., 1990, Feeling and Seeing: Issues in Force Display, Computer Graphics, 24(2), August: 235-243. Schmult, B., and Jebens, R., 1993, Application Areas for a Force-Feedback Joystick, Kazerooni, H., Colgate, J.E., and Adelstein, B.D. (ed.), Advances in Robotics, Mechatronics, and Haptic Interfaces, DSC-49, ASME: 47-54. Fitts, P.M., 1954, The Information Capacity of the Human Motor System in Controlling the Amplitude of Movement, Journal of Experimental Psychology, 47(6), June: 381-391. Foley, J.D., Wallace, V.L, and Chan, P., 1984, The Human Factors of Computer Graphics Interaction Techniques, IEEE Computer Graphics and Applications, 4(11): 13-48. Jellinek, H.D. and Card, S.K., 1990, Powermice and User Performance, Proceedings of the CHI 90' Conference on Human Factors in Computing Systems, ACM, April: 213-221. Tr~inkle, U., and Deutschmann, D., 1991, Factors Influencing Speed and Precision of Cursor Positioning using a Mouse, Ergonomics, 34(2): 161-174.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
Discussion
on Method
719
for Predicting Targets in Pointing by Mouse
Atsuo MURATA*, *Department of Computer Sciences, Faculty of Information Sciences, Hiroshima City University, 151-5, Numata-cho, Asaminami-ku, Hiroshima, 731-31, Japan e-mail • [email protected] In this paper, the method to predict a target which a user is about to point with a mouse on the basis of the trajectory of the mouse cursor was proposed. The effects of the interval between targets, the sampling interval and the number of selection of targets on the point time and the prediction accuracy were investigated. The pointing with no prediction mode was also conducted. As a result, the prediction method 1 that regarded the target which was selected continuously 5 times as the candidate target was found to be proper from the viewpoint of the prediction accuracy. On the other hand, the prediction method 2 that caluculated the angle between the cursor movement vector and the vector which connected the current cursor position and the center of each target, and determined the minimum cumulation value as the candidate was proper in that the point time was shorter than the method 1. The optimal condition of the prediction method 1 was st=4ticks and d _~ 30dots.The optimal condition of the prediction method 2 from the viewpoint of the prediction accuracy was n=6, 8, 10. 1.Introduction An important element in the design of the human-computer system is the method of pointing by which the users indicate their selection to the computer with input devices. The role of input devices such as a mouse and a touchscreen is becoming more and more important in the humancomputer system. It is not too much to say that the performance in the human-computer system is affected by the input devices. Until now, studies on the usability of the input devices are tried [1-3] . Almost all of these studies compare the point time, the point accuracy and so on among a few input devices, and predict the performance, for example, by Fitts' law. As a method to improve the performance of the point operation by a mouse, the method to predict the target which the user is about to point and automatically move the mouse cursor to the target immediately after the completion of the prediction is strongly recommended. If the point system with such a prediction mode is equipped with the human-computer system, the point time is sure to be shortened. Moreover, the space to operate the mouse is reduced by this method. However, there are little studies that try to improve the performance of the point operation with a mouse by predicting the target which the user is pointing. In this paper, the method to predict the target which the user is about to point with a mouse on the basis of the trajectory of the mouse cursor was proposed. The effects of the sampling time, the number of sampling of the mouse cursor, the interval between targets and the position of targets on the point time and the prediction accuracy were considered.
2.Method for Predicting Target Three types of methods were proposed. Two of them are based on the cursor movement vector of the mouse cursor. By the cursor movement vector, we mean the vector, the initial point and the terminal point of which correspond to the mouse cursor at the position one sampling time before and that at the present position, respectively. 2.1 Prediction Methodl In the prediction method1, the angle between the cursor movement vector and the vector which is composed of the mouse cursor at present and the center of each target was calculated to
720 find the minimum angle. The target, the angle of which corresponded to the minimum angle continuously five times, is selected as the prediction target (candidate) . In the prediction methodl, the following four kinds of the sampling time st of the trajectory of the mouse cursor were taken as an experimental parameter" 3,4, 5 and 6ticks. One tick corresponds to about 1/60 second. The caluculation of the angle in prediction methods 1 and 2 for predicting the target is explained in Fig.1. 2.2 Prediction Method2 In the prediction method2, the sampling time of the trajectory of the mouse cursor was fixed to 4 ticks. Here, the angle is calculated in the same manner as the prediction methodl (See Fig.l) . The angle is accumulated n times. The number of sampling of the mouse cursor n was taken as an experimental parameter. The following four kinds of n were selected • n=4, 6, 8,10. The target with the minimum cumulative angle is determined as the prediction target. 2.3 Prediction Method3 In Fig.2, the process to predict the target in the method3 is summarized. The prediction method3 is based on the prediction area which consists of a fan-shaped area with a center angle of 0 . The center of the fun corresponds to the present position of a mouse cursor. In the prediction method3, whether each target is contained within the fan with the center angle of 0 is calculated continuously ten times. The number of cases when each target is contained within the fun is summed up. The target with the maximum number was selected as the prediction target. Four kinds of 0 were selected : 0 =30, 40, 50, 60degree.
3.M©thod The performance (the point time and the prediction accuracy) of three methods above was compared. Moreover, the performance of these three methods was compared with the performance of the control condition under which the mouse is pointed with no prediction mode, and targets are not predicted at all.
Target1
Target2
Target3
Target4
Target5
°m
vector
\
\
I
\
~MI~\
~
/
I
e.l
~ / "~\e~ /
/
/
/ / J
/
~
~ ~
J
~
L MOA--
~
0~,
/--
LMOC=e~ / MOD=
L
cursor
att, 1~lk'-'
Mouse cursor at t i - 1"-%
...
.
ijj.)
c_o.ntinuously 5?imcs
O~
__LMOE= O~
Prodict!onmethod2 ............
Mouse
.
[mln
'
,,.-
,., "' "..........
sampling
~dtllll3Ull~
'
,., "
I m in " '
L
Oj
Fig.1 Calculation of angle in proposed methodsl and 2 for predicting target.
J
721 3.1 Subjects Twelve male undergraduates in the age range of 22-24 years old participated in the experiment. All subjects have experience of using a mouse of a Macintosh computer at least more than six months. 3.2 Equipment The experiment was conducted using a Macintosh computer Ouadra7O0 with a 13inch AppleColor high-resolution RGB monitor and a Macintosh mouse. The mouse controlled the movement of the cursor with a C/D (control/display) ratio of the second fastest setting for the mouse sensitivity on the Macintosh control panel. This setting corresponds to the C/D ratio of about 0.53. The pointing system was programmed using a C language (THINK C version5.0) . The software for the pointing system has a data recording function. The data record contained (1) the time from the appearance of targets until the completion of the movement to the predicted target, and (2) numbers of the target to be pointed and the predicted target. Under the control, the data recorded were (1) the time from the appearance of targets until the arrival of the cursor to the target and the click of the mouse botton, and (2) numbers of the target prespecified by the computer and the target which user pointed. 3.3 Procedure Four experiments (prediction methodsl, 2, 3 and control) were conducted. In all experimental tasks, the number of targets was fixed to five. The five targets were numbered consecutively from left to right. For example, the targets on the left end position and the right end position were numbered 1 and 5, respectively. The size of the target was fixed to the 30 x 30 dots square. The target to be pointed was predetermined randomly. The interval between targets d and the position of the target were also considered as experimental parameters, and five kinds of d were selected • d=10, 20, 30, 40, 50dots. In the prediction methodl, 2 and 3, each subject performed four trials to each combination of d (5 levels) , the position of targets (5 levels) and st (n, 0 ) (5 levels) .In other words, each subject pointed 400 times in each prediction method. In the control, each subject performed four point operation to each combination of d (5 levels) and the position of targets (5 levels) . Each subject performed 100 pointing trials. In the prediction methodl, the effects of the sampling interval on the point time and the prediction accuracy were investigated. In the prediction method2, the effects of the number of
,.roetlo ---r.eO / ,.roet -i
Movement mouse cu I Prediction
area
at
Prediotlon
t
,.roet,
,.roe
Mouse c u r s o r a t t,.2 (Target1 is selected.) Mouse cursor a t t i+1 (Target1 is selected.) -Movemlent vector of mouse cursor a t 1~
I+1
area
at
t
Mouse (Target
cursor 1 and
at t, 2 is selected.)
Fig.2 Method for predicting target by prediction area.
722
selection of targets on the pointing time and the prediction accuracy were investigated. In the prediction method3, the effects of angle 0 on the performance were investigated.
4.Results
In Fig.3, st (sampling interval) × d (interval between targets ) interaction for the point time is demonstrated. The n × d interaction is shown in Fig.4. Fig.5 compares the point time among 13 experimental conditions. The st × d interaction for the prediction accuracy is demonstrated in Fig.6. In Fig.7, the comparison of the prediction accuracy among 12 experimental conditions is demonstrated. The effects of the position of targets on the prediction accuracy in the prediction method2 is summarized in Fig.8. Here, the value of d is fixed to 40 dots. 5.Discussion The point time increases linearly with the sampling interval of a mouse cursor (See Fig.3) . Moreover, we can observe that the distance between targets d has little effects on the point time. The point time for st=3ticks was the shortest of all conditions of st. Fig.4 demonstrates that the point time in the prediction method2 does not differ among four conditions of n and among five conditions of d. It seems that n and d have little effects on the point time. Insofar as the method3 is concerned, the parameters d and 0 had no remarkable effects on the point time. From Fig.5, 1.2 I
1.1
o
E ._
/ ~
o
.~ t-
0.8:
"3
0.7
ll.
j
j
0.9
f,-/ --,p-
d=10
~
d=20
I
0.6
~
,l~/'"
d=40
--#---
•
4
5
Sampling interval
IIII =
~ .
ticks
.
,, o
.
0.76 0.75 0.74
==
!• !
'='
" " 0.72 0.71
0.98
.
o>, 0 . 9 6
d=20
.
"o,, -
conditions
~
== == == == 10 10 10 10 o
o
o
o
0
d =30
~
d=40
#
4
6
8
. . ~ 7 7 /" ~"-/"
~
-
+
~=~o
=
d=30
,
d=40
#
d=50
o 0.92
d=50
c 0.9 -"~ 0 . 8 8 •lo p 0.86 ix. 0 . 8 4
0.7
Experimental
.
= ;
d=lO
,
0.73
ix.
~, =
~"
4,
-
"= =-
~= = ~, ~
Fig.5 Comparison of pointing time among all experimental conditions.
0.77
g =a
"I II
o
6
Fig.3 Comparison of pointing time among st.
ce •,=
ilililll I
ix. 0.2
d=50
I
0.4
......
0.6
d=30
• ~
0.5
E
I
10
N u m b e r of s e l e c t i o n
/
/
/
0.82 0.8
3
4
Sampling interval
Fig.4 Comparison of pointing time among n.
5
6
ticks
Fig.6 Comparison of prediction accuracy among st.
723 it can be observed that the point time for st=3ticks in the prediction methodl is remarkably shorter than that of other conditions' The point time for st=3ticks was reduced by about 44% as compared to that of the control. It is clear that the point time of each condition in the prediction method2 and prediction method3 is nearly equal, and n or 0 has no effects on the point time. Concerning the prediction methodl, the point time when st is more than 4 ticks is larger than that of other prediction methods. Especially, the point time for st=6ticks is larger than that of the control. The point time of the prediction method2 and the prediction method3 was reduced by about 26% relative to the control. From Fig.6, it is observed that the prediction accuracy tends to increase with the sampling interval st for each d. What is different from the point time is that the prediction accuracy is affected by the distance between targets d. The prediction accuracy for d=10dots is lower than that for other conditions of d. With respect to the method2, the prediction accuracy tended to increase with n for each d. Unlike the point time, the distance between targets d had an effect on the prediction accuracy. The prediction accuracy for d=10dots was lower than that for other conditions of d. As for the method3, the prediction accuracy decreased with /9 . The distance between targets d in the prediction method3 had more remarkable effects on the prediction accuracy than that in other two prediction methods. The prediction accuracy for d=20 and 30dots was lower than that for d=40 and 50dots. The predicition accuracy of the prediction methodl was found to be the highest of all three prediction methods for each d. The prediction accuracy for d=10dots is lower than that for other conditions of d. This is true for all prediction methods. When the value of d is equal to 10dots, it is laOt proper as a condition of the target prediction system. This also means that when the targets with the same size as that in this paper are arranged horizontally so that d is less than 20dots, it is infered that this arrangement is inadequate from the viewpoint of the prediction accuracy. If we assume that the size of targets is the same as that in this experiment, we can consider that the number of targets must be at most 8 (400dots/ (30dots+10dots × 2)) . In Fig.7, the prediction accuracy among 12 experimental conditions is compared. When st equals 3ticks, the prediction accuracy (about 0.915) was lower than that of other conditions of
0.9
.
0.7
'
~o.ei
t0.9 0.8
o~
?~
o= =.
-
o:
~o.5!
o t~
=o0.4i
0.6 0.5
~
~o.al °-
0.7
0.2
i
0.1
i
~
0.4 o.a 0.2
o . . . .
~
"~,
~
. . . .
cn
o)
II ~
II rn
Experimental
'=' ~ ~= ~= ~= ~= 0 03 II CI)
• 'q' II (i)
~ U3 II Ci)
r,D II ill
0.1
1
2
3
4
5
Position of target
conditions
Fig.7 Comparison of prediction accuracy among all experimental conditions.
Fig.8 Effects of position of targets on prediction accuracy (d=40dots, Method2) .
724 st. As for the prediction method2, the prediction accuracy increased with n. However, the prediction accuracy for n=8, 10 was lower than that for st=4,5,6ticks in the prediction methodl. The prediction accuracy of the method3 tended to decrease with 0 . The value of the prediction accuracy of all conditions in the method3 was lower than 0.9. This suggests that the prediction method3 is not proper from the viewpoint of the prediction accuracy. The effects of the position of targets are discussed. The prediction accuracy of the positions 2 and 4 in the methodl tended to be lower for all conditions of sampling intervals. Moreover, the prediction accuracy for st=3ticks tended to be lower. The prediction accuracy did not differ among four conditions of st when targets are arrange to the positions 1, 3 and 5. In other words, the st had little effects on the prediction accuracy for the positions 1, 3 and 5. On the other hand, as far as the positions 2 and 4 are concerned, the st had effects on the prediction accuracy, and the prediction accuracy differed among conditions of st. Concerning the prediction method2, the similar results to the prediction methodl was obtained. From Fig.8, it is clear that the prediction accuracy for the positions 2 and 4 is lower than that for other positions. For the positions 1, 3 and 5, n has little effects on the prediction accuracy. Even in the prediction method3, the similar tendency to the prediction methods 1 and 2 was observed. As for the positions 2 and 4, the prediction accuracy tended to be lower than that of other positions. Except the positions 2 and 4, the prediction accuracy did not differ among conditions of 0 . To improve the prediction accuracy at these positions 2 and 4, special algorithm must be added to each prediction method. With the present prediction method, it is difficult to satisfy good performance for both the point time and the prediction accuracy. Therefore, the trade-off between the speed (point time) and the prediction accuracy is an important matter. If the importance is put to the prediction accuracy, the prediction method 1 is selected at the cost of the point time. If the speed is required, the prediction method 2 or the prediction method 3 will be selected at the sacrfice of the prediction accuracy. However, in the concept of predicting targets which the user is about to point by the mouse, it seems that the accuracy must be paid more attention, because the false prediction is meanless, and increases the job of operators. In the future, therefore, the prediction method superior in both point time and prediction accuracy will be developed. The number of targets and the size of targets are considered as experimental parameters, and its effects on the performance will be discussed. The method to predict targets in point operation by a mouse need to be put to practical use in the human-computer interface to improve performance of users.
Rcfcrcnecs 1.B.W.Epps • "Comparison of six cursor control devices based on Fitts' law models", Proceedings of the 30th Annual Meeting of the Human Factors Society, pp.327-331, (1986) . 2.S.K.Card, W.K.English and B.J.Burr" "Evaluation of Mouse, Rate-Controlled Isometric Joystick, Step Keys, and Text Keys for Text Selection on a CRT, Ergonomics, 21 (8) , pp.601613, (1978). 3.S.K.Card, T.P.Moran and A.Newell ""The keystroke-level model for user performance time with interactive systems", Communications of the ACM, 23 (8) , pp.123-136, (1980) .
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) 1995 Elsevier Science B.V.
725
The D i f f e r e n c e of I n f o r m a t i o n I n p u t M e t h o d on P s y c h o - p h y s i o l o g i c a l r e a c t i o n of VDT work. Takao OHKUBO* M i c h i y o s h i AOKI* M i t s u g u SAWA** Moritoshi IKEDA** Keun Sang Park* * C o l l e g e of I n d u s t r i a l T e c h n o l o g y , N i h o n U n i v e r s i t y **Railway Technical Research Institute ABSTRACT One of m o s t i m p o r t a n t f a c t o r s t h a t we s h o u l d t a k e i n t o c o n s i d e r a t i o n w h e n we * e s t i m a t e .
The o u t p u t of t h e s y s t e m is t h a t m a n h a s a l e a r n i n g a b i l i t y in
r e l a t i o n to t h e d i f f e r e n c e of I n f o r m a t i o n I n p u t m e t h o d .
In the w i d e r field of
E r g o n o m i c s a t r e m e n d o u s n u m b e r of s t u d i e s h a v e b e e n done to i n v e s t i g a t e the r e l a t i o n s h i p output)and
b e t w e e n VDT i n p u t m e t h o d s ( i . e . , t h e
physiological
response
of p e o p l e
who
are
v a r i e t i e s of w o r k due to d e g r e e of o p e r a t i n g d i f f i c u l t y . authors' operating
study
are
related
to t h e
relationship
s k i l l s of a c o m p e n s a t o r y
e f f i c i e n c y of w o r k engaged
between
visual-manual
in m a n y
The r e s u l t of t h e
tracking
the
change
task
of
and the
c h a n g e in t h e i r p s y c h o - p h y s i o l o g i c a l r e a c t i o n s u n d e r o p e r a t i n g c o n d i t i o n s . E x p e r i m e n t s w e r e c a r r i e d o u t u s i n g five m a l e s t u d e n t s , t o d e t e r m i n e t h e learning
patterns
and
responses
of
them.
The
response
experiments
show t h a t t h e g r a d u a l a c q u i r i n g of o p e r a t i n g
through
increasing
number
reactions.
The
an
physiological
r e f l e x , a n d o t h e r all p a r a m e t e r s
of p r a c t i c e s
heart
are
revealed
rate,respiration
of
these
skills gained in
psycho-
rate,galvanic
skin
show t h e s i g n i f i c a n t d i f f e r e n c e due to t h e
d e g r e e of d i f f i c u l t y in o p e r a t i o n a n d some of t h e m i n d i c a t e t h e i r h i g h e s t l e v e l in t h e i r h i g h e s t l e v e l in t h e i r f i r s t p r a c t i c e a n d a g r a d u a l d e c r e a s e to t h e i r f i n a l l e v e l s as p r a c t i c e p r o c e e d . 1.Introduction A l t h o u g h m a n y m e t h o d s h a v e b e e n u s e d to e s t i m a t e t h e r e l a t i o n s h i p s between factors
the that
abilities must
be
of
men
considered
and in
machine,the the
design
principal of
any
or
basic
man-machine
s y s t e m m a y be l i s t e d as follows: 1)Those p a r t s of t h e s y s t e m are b e s t a l l o c a t e d to m e n or to m a c h i n e .
726 2)How to a r r a n g e m a n a n d m a c h i n e to g e t t h e o p t i m u m s y s t e m o u t p u t , both q u a n t i t a t i v e l y and q u a n t i t a t i v e l y . 3)Dose t h e s y s t e m b u i l d up to give a w e l l - i n t e g r a t e d as the whole? One of t h e m o s t i m p o r t a n t f a c t o r s to be c o n s i d e r e d in s y s t e m d e s i g n is t h e f a c t t h a t m a n h a s a b i l i t y to l e a r n . with practice
His skill at his t a s k will i m p r o v e
a n d at t h e s a m e t i m e he can r e a d i l y a d a p t to u n f o r e s e e n
c h a n g e s in t h e s y s t e m .
In o r d e r to i n v e s t i g a t e t h e o p e r a t i o n a l r e l a t i o n s h i p
b e t w e e n a t c o n t r o l w o r k s on VDT u n d e r t h e two d i f f e r e n t e x p e r i m e n t a l c o n d i t i o n s , t h e f o l l o w i n g i t e m s m u s t be s t u d i e d . 1)The o p e r a t o r ' s a b i l i t y to c o n t r o l due to t h e two d i f f e r e n t c o n t r o l t y p e s of t h e VDT work. 2)The
operator's
decision
making
abilities,experience
and
other
psychological and physiological reactions. The r e l a t i o n s h i p b e t w e e n o p e r a t i n g s k i l l s a n d p a t t e r n s on c o n t r o l movements
has
been
studied
by
many
institutes
in
the
world.
V a r i a t i o n s in c o n t r o l o p e r a t i o n i n v o l v e , o f c o u r s e , v a r i a t i o n s in t h e s t r e s s of w o r k as well as in skill in c o n t r o l l i n g t h e s y s t e m .
S t u d y of t h e s t r e s s e f f e c t
of o p e r a t i n g t h e s y s t e m h a s b e e n m a d e by a n u m b e r i n v e s t i g a t o r s in J a p a n u s i n g p a r a m e t e r s such as h e a r t r a t e , r e p i r a t i o n r a t e , r e a c t i o n t i m e , s u b j e c t i v e a s s e s s m e n t , e l e c t r o - e n c e p h a l o g r a m a n d blood p r e s s u r e . These
emotional
response
are
well i n d i c a t e
by such c h a n g e s
in t h e
a u t o n o m i c n e r v o u s s y s t e m , t h u s it is p o s s i b l e , t h o u g h d i f f i c u l t to g u a g e t h e i r m a g n i t u d e a n d from t h i s to d e r i v e a m e a s u r e of t h e c h a n g e in m e n t a l t e n s i o n t h a t are involved.
However,this doesn't mean that any simple relatoinship
exists b e t w e e n these reactions and the emotional factor,of which they are indicative.
At b e s t it can be said t h a t only a r o u g h e s t i m a t e of t h e e m o t i o n s
are revealed.
We h a v e a l r e a d y n o t e d t h a t o n e ' s skill at t h e t a s k will also
alter with experience.
In the e a r l y s t a g e of l e a r n i n g , t h e v a r i o u s limb a n d
body
a necessary
movements
that
to
p e r f o r m e d w i t h o u t c o - o r d i n a t e d skill.
the
task
will
be
independently
But, w i t h p r a c t i c e a n d l e a r n i n g , t h e
e x e c u t i o n of s u c h m o v e m e n t s will t e n d to b e c o m e m o r e s k i l l e d u n t i l , a t l a s t , t h e y a r e c a r r i e d o u t a l m o s t w i t h o u t c o n s c i o u s e f f o r t a n d m a y h a v e come conditional reflexes.
S u c h p h y s i c a l s k i l l s r e s u l t in o p t i m u m e c o n o m y of
body m o v e m e n t to g e t h e r w i t h a r e d u c t i o n in t h e s t r e s s e s i m p o s e d by t h e task. The
following
relationship
experiments
between
the
are
acquisition
aimed of
at
skill
investigating through
the
continuous
p r a c t i c e a n d t h e d i f f e r e n c e of c o n t r o l of t h e w o r k a n d t h e v a r i o u s m e n t a l a n d the
difference
of
control
of
the
physiological reaction experienced.
work
and
the
various
mental
and
727 2 . T h e m e t h o d of t h e s t u d y . A s e r i e s of t h e two e x p e r i m e n t s was c a r r i e d o u t w h i l e u s i n g t o t a l l y five healthy
university
male
psycho-physiological compensatory operation
students
visual-manual
control
a g e d f r om
variations
tracking
of s p a c e s h i p s
17 to 23 to e v a l u a t e
during on
a
task
the
operations
which
is
screen
under
reffered the
two
their of
a
as
the
different
control system i.e.,a joystick control with a hand only and the both stick and
a p e d a l c o n t r o l w i t h a h a n d a n d a foot on t h e s a m e
of joy
task.
The
m a i n t a s k e m p l o y e d w a s a s i n g l e d e g r e e of f r e e d o m
compensatory tracking
t a s k t h a t w a s t h e s a m e as t h e e x p e r i m e n t s l a s t y e a r .
The s u b j e c t w as s e a t e d
i n f r o n t of a 1 . 1 8 m * l . 1 8 m s c r e e n t h a t was 4.55m far f r o m t h e s u b j e c t s a n d t r i e d to a t t e m p t
to n u l l t h e t r a c k i n g e r r o r by t h e two d i f f e r e n t t r a c k i n g
m e t h o d s , n a m e l y one of t h e n is t h e m o t i o n of a h a n d s t i c k c o n t r o l c o u p l e d to t h e v e h i c l e d y n a m i c s w a s 1/s.
The t a r g e t was m o v e d on t h e s c r e e n in e v e r y
directions randomly and freely.
In a d d i t i o n , t h e
s u b j e c t s w as g i v e n t h e
s u b - t a s k in w h i c h t h e s u b j e c t h a d to a r e d l i g h t s i g n a l a p p e a r e d on t h e s c r e e n as for t h e m a i n t a s k w i t h a p r e - a s s i g n e d key i n f r o n t of him.
A continuos
m a i n m a n u a l t r a c k i n g c o n t r o l w o r k l a s t e d t w e n t y m i n u t e s at a t i m e a n d two t i m e s of a d a y ( A . M 9 : 0 0 ~ 1 3 : 0 0 were
inserted
more
than
experimental
conditions
measurements
of
skin
an
and P . M 1 3 : 0 0 ~ 1 6 : 0 0 ) . hour
were
psycho-physiological
resistence,frequencies,reaction
during
although
between
the
experimental
the
carefully
two
The r e s t p e r i o d s sessions.
design.
performance,heart
time
were
sessions.
measured The
results
As
All t h e for
the
rate,galvanic continuously the
subjects
p e r f o r m e d w e r e s h o w n to t h e i r i m m e d i a t e l y a f t e r a s e s s i o n f i n i s h e d . Three
minutes
experiment.
r e s t w i t h e y e s c l o s e d w e r e done b e f o r e a n d a f t e r
the
The e n v i r o n m e n t a l c o n d i t i o n s for t h e s t u d y w e r e 2 1 . 1 _ + 1 . 7 c a s
t e m p e r a t u r e , 6 4 . 5 + 7.7% as h u m i d i t y a n d 750Lx as i l l u m i n a t i n g l e v e l on t h e c o n t r o l d es k . 3.Result and discussion C h a n g e s in t h e o b s e r v e d v a l u e s of H . R . , R . R . , E y e B l i n k i n g a n d G . S . R c l e a r l y i n d i c a t e t h e t e n d e n c y for s u b j e c t i v e s t r e s s e s to d e c r e a s e w i t h t h e d i f f e r e n c e of t h e s y s t e m b u t n o t s i g n i f i c a n t d i f f e r e n c e due to i n c r e a s e of t h e practice. F i g . 1 s h o w s m e a n v a l u e s for h e a r t r a t e , e y e b l i n k i n g a n d g a l v a n i c s k i n reflex. F r o m t h e s e , t h e f o l l o w i n g c o n c l u s i o n s c o u l d be d r a w n .
728
=®1 ~ ~
i
~....
i
J
i':]" "~
94
©
..,.=
illlilll
-=
i
Illl I I I I I l l l J i l l I l J l I l l I I I I I I I I I I I I I l l
IW] ...............................................................
|
OLEVEE end f£DAL (N-S)
|
÷ JOTSTICX (a-4)
Fig.3 T~a.ean P. . . .
,, T,.=klu= E.... "V.,latioa, in tha botk Laver. Pedal and J o l l t l c k d u r l n | 20 minutes at mack 10 experimental 8sssloms.
IIIlllllllllll
I ,., i,., l,.d I,,, I,,, I,,, I,,, I ,,, I,- I .,,,I
Tab.1 Analysis
of
variance
of
Heart
Rate
(N=5)
0 LEVEE and PEDAL +JOYSTICr
.. •-.
(N-S).
"." F'lg.1 ¢=
160
80 . . . . . . .
(a=4)
Source ,can V a r t a t l . . . . Pursuit Trackin|
..........................
t 1=--,~ ~,
...............................
f
HE,It,El
and GSD f o r
the
Difficulty Trials Error Total
"....................................
......~
. . . . . ~ . . . . "-e'. ~ " • :.: • • • "; ~" . + : . ~ ~
...............................................................
!
i
i
!
i
i
i
OLEVED and PEDAL + JoIrsTICK (a,4)
so or Purselt'Trackln= Error
In
(a,5)
t~, koth Le,er.-
and J o ; s t i c k f o r d u r l n 8 30 m i n u t e s a t t a c k iO e x p e r i m e n t a l sessions.
Pedal
1 9 9 19
Source
OF
Difficulty Trials Error Total
1 9 9 19
,
20-
Fig.2
HF
F-Ratio
1.57E÷02 1.57E+02 11.55.* 3.67E+02 4.07E÷01 2.99 1.22E+02 1.36E÷01 0.65E÷03
SS
HF
F-EatEn
.................... , ...................
""~:'~. . . . . . +:.:.:.:.:.": - ' ~ - ~
i
SS
TaJx2Analysis of variance of Respiratory Rate (N=5)
~-: . . . . . . . . . . . . . . . . . . . . . .""-r . ......... . . . . . ..4-"::-:,:-:-j--........ -
i
[IF
VisUal
Experiment.
i
,
7.85E+00 7.85E+00 2.57E+01 2.86E+00 7.38E+00 8.20E-01 0.41E+02
9.57, 3.48*
729
( 1 ) H e a r t r a t e on e a c h e x p e r i m e n t a l day shows no c l e a r d e c r e a s e as t h e p r a c t i c e p r o c e e d s , b u t shows h i g h e r level in case of the c o n t r o l s y s t e m w i t h l e v e r and p e d a l to the l a s t day of the e x p e r i m e n t . Level of m e a n r e s t i n g h e a r t r a t e , t h a t s u b j e c t s s h o w e d before and a f t e r each s e s s i o n on e a c h day u n d e r two c o n d i t i o n s s t a y e d more or less at the same level t o w a r d s the e n d of the experiment. h o w e v e r , i t can be seen t h a t t h e r e are i n v e r s i o n r e v e r s a l s of the level of the r e s t i n g h e a r t r a t e m e a s u r e d before and a f t e r in the f i r s t five t r i a l s c o m p a r e d to the o t h e r d a y s of the e x p e r i m e n t . ( 2 ) M e a n r e s p i r a t o r y r a t e for five s u b j e c t s i n d i c a t e s no c h a n g e as t h e day proceed,but
significant
difference
between
the
two
control
systems.
N a m e l y , t h e m e a n R.R. level of the s u b j e c t s for the l e v e r and p e d a l c o n t r o l is c l e a r l y h i g h e r t h a n the j o y s t i c k c o n t r o l u n t i l the f o u r t h t r i a l . ( 3 ) M e a n eye b l i n k i n g for five s u b j e c t s shows t h a t the d i f f e r e n c e b e t w e e n the
two e x p e r i m e n t a l c o n d i t i o n s b e c o m e s less as the d a y s p r o c e e d
u n t i l the f o u r t h t r i a l s b u t b i g g e r a f t e r the s e v e n t h t r i a l . Eye b l i n k i n g v a r i a t i o n s are h i g h e r u n d e r the o p e r a t i n g c o n d i t i o n w i t h j o y s t i c k t h r o u g h the e x p e r i m e n t a l t r i a l . ( 4 ) M e a n g a l v a n i c skin r e f l e x s e e m s t h a t no d i f f e r e n c e e x i s t s b e t w e e n the two c o n d i t i o n s u n t i l the fifth t r i a l b u t a s l i g h t d i f f e r e n c e a f t e r t h e n . Fig.2 and Fig.3 show the r e s u l t s of t h e m e a n t r a c k i n g e r r o r r a t e a n d the v a r i a t i o n s for five s u b j e c t s a g a i n s t the m o v i n g t a r g e t . (1)A g r a d u a l and f a i r l y c o n s t a n t d e c r e a s e in t r a c k i n g e r r o r t h r o u g h o u t t h e experiment. Over a l l , i t can be said t h a t a m a r k e d i n c r e a s e in skill was g e n e r a l l y s h o w n by the f i r s t to the fifth of the t r a c k i n g work w i t h l e v e r a n d p e d a l c o n t r o l . As for the a n a l y s i s of the r e s u l t s o b t a i n e d from the e x p e r i m e n t s , t h e f o l l o w i n g c o n c l u s i o n s could be drawn,('~b.1: and Tab.0,) (1)It a p p e a r s t h a t t h e r e are s t r o n g m u t u a l r e l a t i o n s skills b e t w e e n i n c r e a s e of such f a c t o r s as h e a r t r a t e , r e s p i r a t o r y r a t e e l e c t r o o c u l o g r a m and g a l v a n i c skin r e s p o n s e and t h e d e g r e e of d i f f i c u l t y in t r a c k i n g skills.
T r i a l s and
d a y s c o n s i d e r e d as v a r i a b l e s are n e g a t i v e l y c o r r e l a t e d w i t h r e s p i r a t o r y r a t e only. 4.Conclusions In
conclusion,it
can
be
observed
from
a
series
of
two
different
e x p e r i m e n t a l c o n d i t i o n s t h a t no s i g n i f i c a n t d e c r e a s e or i n c r e a s e of p s y c h o -
730 p h y s i o l o g i c a l r e a c t i o n s d u r i n g the course of e x p e r i m e n t s in t e r m s t r i a l s and days w h e r e a s clear d i f f e r e n c e were o b s e r v e d on a l m o s t all r e a c t i o n s due to the d e g r e e of d i f f i c u l t y in o p e r a t i n g s y s t e m s . F r o m the r e s u l t s of d e t a i l e d s t a t i s t i c a l a n a l y s i s , t h i s was also c o n f i r m e d . The m o s t i m p o r t a n t , basic and effective way of solvency the d i f f i c u l t y in o p e r a t i n g such t a s k is to build up the o p t i m u m s y s t e m in a d d i t i o n to build up the o p t i m u m t r a i n i n g s y s t e m s . 5.References ( 1 ) M . O s h i m a , " T h e s t u d y of F a t i g u . " ; T o k y o Dobun shoin, 1960 ( 2 ) T . O k u b o , " L e a r n i n g p r o c e s s e s on m e n t a l and p h y s i c a l r e s p o n s e s a b o u t d r i v i n g . " ; E r g o n o m i c s Vol,8,1972 ( 3 ) H . A k a s h i & H . H a s h i m o t o , " L e a r n i n g c h a r a c t e r i s t i c s of o p e r a t o r in manm a c h i n s y s t e m . " ; E r g o n o m i c s Vol,8,1972 ( 4 ) E l w y n E d w a r d s & F r a n k P Lees,"The
Human
Operator
in
Process
C o n t r o l . " ; T a y l o r and F r a n c i s Ltd.,1974 ( 5 ) E w a l d E . S e l k u r t , " B a s i c P h y s i o l o g y for the H e a l t h S c i e n c e s . " ; L i t t l e , B r o w n and C o m p a n y , 1987 (6)H.E.Petersen & W.Schneider,"Human
Computer
Communications
in
h e a l t h c a r e . " ; n o r t h - H o l l a n d , 1985 (7)David J O B o r n e , " E r g o n o m i c s at w o r k . " ; J o h n Wiley and Sons L t d . , 1 9 8 2 (8)Peter
H
Lindsay
and
D6nald
A
Norman,"An
P s y c h o l o g y . " ; A c a d e m i c P r e s s Inc.,1974 (9)K.Hashimoto,K.Kogi & E.Grandjean."Methodology A s s e s s m e n t . " ; T a y l o r and F r a n c i s Ltd.,1969
in
Introduction Human
to
Fatigue
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
731
R o t a t i n g objects using dials A.Imamiya and T.Sakamoto Department of Electrical Engineering and Computer Science, Yamanshi University, Takeda 4-3-11,Kofu, Japan 400
1. INTRODUCTION This paper describes two experiments to study perceptual judgements of alignment and rotational motor operations in the context of interactive computer graphics. Although there have been several human factor studies of pointing devices such as a mouse, or trackball[ 1 ], we can not find a human factor study of valuator logical input device, such as a dial. The user of an interactive graphics often uses dial devices to rotate three dimensional object in order to obtain an understanding of the shape of the object. This research represents the second in a series of experiments designed to investigate the human factors of rotation tasks in the interactive computer graphics[ 2 ]. The primary object of study is to construct an empirically-based cognitive theory of human-computer interaction in this domain. In our previous study[ 2 ], we presented the effect of graphical representations on the process and performance in mental rotation task. In this paper we describe two experiments designed to develop a predictive model of time to rotate an object on the screen using dials, and find types of rotational operation and errors. First, we measure mean operation times v.s. the difference angle between two images. Secondly, we consider errors for perceptual judgements, and the predictive equations of the time to perform the rotation action in the four performance types. 2. METHOD 2.1. Device The control dial is perhaps the simplest of the valuators. Dials are sensitive rotating potentiometers. Subjects use three dials for activating rotation functions,and one for quitting the operation in a trial. A dial device sat on the table to the fight of keyboard. Each dial except QUIT dial corresponds to a coordinate axis,and each rotational operation is performed about the coordinate axis.
732 2.2. Subjects Ten men, all undergraduates at Computer Science majors of Yamanashi University,served as subjects in each experiment. 2.3. Procedure Subjects were seated at a computer terminal with a 19 inches display, a keyboard, and a dial device for rotating an image on the screen. In each trial, a pair of images of geometric figures, called the ten-blocks was displayed on the screen. An fixed, or standard figure was shown on the left side, and a test, or target figure was shown on the fight side of the screen(Figure 1). In the experiments,subjects are presented with a number of pairs of images of geometric figures called ten-blocks (Figure 2). The views of the ten-blocks are from various directions in three dimensional space,and the difference between the view angle of standard and target figures. The subjects are asked to rotate a target image into complete alignment with the standard by rotations about an axis using dials. Subjects were given approximately 5minutes practical rounds. In the main experiments ,then a number of pairs of images were shown to each subject. All trials were recorded as data. Dial data consists of the name of rotation axis,x,y,or,z, and the rotated angle, and the values of the dial data were sampled by the computer in 1/128 second cycles. Video data consists of the video record of the subject's operational behavior and verbal reports(think aloud reports), and the images on the screen. Experiment 2A: Each subject was compelled to use a designated dial for the rotation task. The experiment consisted of the average 160 pairs of images for each subject. Experiment 2B: Each subject can select a dial from three alternatives for rotational operation. The experiment consisted of the average 58 pairs of images for each subject.
yi
Y x
X
Y Z
I
!,
I
target
standard
,l
•
o'@ o'@ o'@ @©
J
Figure 1. Stimulus, dials and a screen.
Figure 2. Ten-blocks.
733
3. RESULTS Analysis of data from the timed task performance phase of the experiment was broken down into the following sections. First, a misunderstanding or error analysis was performed on all responses, that is, we considered error for perceptual judgments. Secondly, the sequence of rotation were analyzed over all trial. The subject's behavior consists of three steps; moving hand from the home position to a dial, rotating the dial, and back to the home position. The results were analyzed in terms of relative angles of rotation and rotational operation time and the total average rotational operation times. Finally, we found that a rotation task consisted of three phases; the initial(estimating the relative angles and orientation), the middle(coarse tuning), and the last(fine tuning) rotation for adjustment. We concentrated to analyze the predicted equations for the tuning phases.
3.1. Errors When subject has judged the view of figures to be same, the difference of angles between a standard and a target figures was analyzed (Figure 3 for the experiment 2A). Two singular points for y coordinate axis were for the mirror image figures, and a singular point for z coordinate axis was for the point symmetry image figures. Table 1 summarizes the number of errors over two experiments. According to verbal reports, some subjects, who particularly focused on a connection of blocks, tended to make errors in judgment. 3.2. Rotational operation times Rotational operation times were analyzed after first removing all incorrectly operated trials. Figure 4 plots rotation angles as a function of rotational times for the experiment 2A overall, and Figure 5 shows the data plotted for the experiment 2B.
180
°
Remainder of angles
.1
90
. . . . . . . . . .
°
1l
all
'
T
±
_
X
I
+
Y
Z
Dials Figure 3. The difference angles at the end of a trial(Experiment 2A)
734 Table 1. The number of errors number of trials Experiment 2A practice rounds main experiment Experiment 2B practice rounds main experiment
number of errors
1080 569
6 29
540 139
6 4
The equation for Figure 4 as determined by regression analysis is: Tr = 0 . 0 0 5 0 + 0.071, where O is the rotated average angles.Similarly, the equation for Figure 5 is • Tr - 0.005 O + 0.079.
I
! 8
8 Q
~
• ! k'..
,'."
'
•. !"~":;,'? • ~
.~
.,,..'.."
.... ''
-
.
i
:i', ~.
o
0.2
0.4
0.6
0.8
1.0
second
Figure 4. Rotated angles and times(Experiment 2A). 3.3.
The
total
.~' .'.~.:'-""
.
0
0.0
6
'; ,:~2.."..':. "'. • .~.~".,_...:....:
a~
/ 0.0
i
i
I
i
i
0.2
0.4
0.6
o.a
1.0
second
Figure 5. Rotated angles and times (Experiment 2B).
times
To predict the average total operational time Tt in a trial, we assumed a linear model Tt = Ts + cTr, where Ts is the times for selecting a dial, and c is the average number of selected axes for a trial. Since a dial was designated in the Experiment 2A, Ts=0, and c=21.5. From the analysis of dial data, Ts-0.532, and c-9.167 in the Experiment 2B. Consequently the predicted time Tt=5.1 seconds, and the actual time was 6.2seconds in the Experiment 2A,. In the Experiment 2B, the predicted time Tt=3.8seconds, and the actual time was 4.1seconds.
735
3.4. Rotation Phases
Figure 6 shows that the rotational operation for a trial consists of three phases: In the initial phase, according to the analysis of verbal data, subjects tried to estimate the relative angle and orientation, while rotating the dial slowly. In the coarse tuning phase, subjects coarsely rotate a dial to make the relative angle to be zero based on the predicted angle. In the fine tuning phase, subjects rotated a dial slowly and finely. 3.5 T h e m o d e l of tuning phases
We developed an effective predictive model of angle rotated in the coarse and the fine tuning phases. From the experiment data, we can assume the rotation action to be the damped oscillation : The difference of angles A R between the standard and target images at the time T is AR=R0exp(-//T)cos( to T), where R 0 is the initial difference of angles, ,~ and to are constants determined by operation types as follows; Four types of times vs. rotated angles and parameter values are shown in Figure 7.
/ ~
~
Initiation / c o a r s e tuning fine tuning
/ //~"
degree
j/
initiation coarse tuning / finetuning
160 120 °
80°,.,~
40-
~xo
<
.,d~'~,vJ
0t - 40
Z,5~,,, 5.0 Rotation time '
I sec
I sec
0 1
-40 -1
Experiment 2A Figure 6. Typical examples of rotation action.
4
Rotation time
8
.... - ~ - - - - - z
Experiment 2B
Subjects tended to rotate a dial in the type A at the beginning of trials. As trials were performed ,subjects then changed the operation to type B followed by type C or type D. 4. CONCLUSIONS According to the analysis of data in the controlled experiments, we proposed two formulas for rotating a dial, one was the predictive linear model of the rotational operation times, and another was the predictive model of rotated angles and times for four types of the rotational operation. We also found the transition among rotational operation types in trials by reason of subjects' learning effect.
736
REFERENCES [ 1] Shneiderman,B. • Designing the user interface, Addison Wesley, 1992 [2] Imamiya, A. and Sakamoto,T. • The effect of graphical representations on the process and performance in mental task, WWDU'92,1992 pp G52-54
%
• v,,,q
"o
"~
<
!
!
0.0
0.2
i
!
0.4
!
0.6
<
i
0.8
1.0
'!
0.0
1.2
Rotation time
!
!
0.2
0.4
'l
Rotation time
sec (a)Type A(I-1.3, ~0"1.5:A--2. 581
0.6
i
I
!
0.8
1.0
1.2
sec
(b)Type B(I-1. 6, ~-3. 5:A---16.1)
% Q
O tO
"
't,
? °1,14
<
o
< o.o
0.2
Rotation time
0.4
o.6
o.a
$0¢ (c)Type C(1=1.8,~=5.2'A=14.4)
1.o
o i
!
!
!
I
!
!
0.0
0.1
0.2
0.3
0.4
0.5
0.6
Rotation time
see
(d)Type D(I=3.4, ~=2.9:A=4.47)
Figure 7. The damped oscillation models for rotating a dial.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
737
A n e w integrated s y s t e m to assess the a m o u n t of i n f o r m a t i o n of pointing devices for m o t o r - d i s a b l e d person Toshiyasu Yamamoto ('), Tetsuya Yamashina (':, Jyunichi ohshima ('~, and Masafumi Ide c"~ ('~Rehabilitation R&D Department, Toyama Prefectural Koshi Rehabilitation Hospital, 36, Shimo-Iino, Toyama, 931, Japan c,~Spinal Injuries Center, Labor Walfare Corporation, 550-4, Ikisu, Iizuka, 820, Japan This paper describes a new system to quantitatively assess the interface control of the input/output devices for the motor-disabled person. A integrated system has been developed, simultaneously to measure for the interface devices and the kinematics of upper extremity. Kinematic analysis is preliminarily introduced to explain a relation between the motor task and its compensatory movement. From a viewpoint of information theory, the amount of information is discussed to describe an quantitative transmitted measure in a specified task. 1. INTRODUCTION Recently though some of the interface devices has been developed for the disabled and the elderly, they are practically selected and served without any quantitative assessment at work place and at home. It is important to consider and estimate a training condition for the residual functions at the stage of rehabilitation program, and the activities of daily livings through the whole environmental process. The final object of this study is to discuss with a method to assess the amount of information of the redisual functions, and to make fit with the amount of information for handling the input/output interface of the electronic devices in the indoor work places, etc. It will be necessary to assess the physically residual functions to control some of the devices: keyboard, joystick, mouse/track ball, push/touch switch, pressure switch, respiratory switch, joint angular switch, voice recognition switch, etc. Here w e introduce a new integrated system to measure the kinematc data for the interface devices, and the upper extremities. And it will be preliminarily discussed about some of the analytical results of the execution errors in pointing the targets, to assess the man-machine interface, sepecially in the computer system. 2. INTERFACE SYSTEM AND TI-IE AMOUNT OF INFORMATION As a measure to express the fitness for the interface system of command control, the following three kinds of indices are introduced, analogous to information theory; (1) A part of funding for this study is provided by Japan Foundation for Aging and Health, supproted by the Ministry of Health and Walfare.
738 multiple access channel(channels), (2) information density(bits/sec), (3) information capacity(bits). Multiple access channel is defined as the number of information access, used for command control. Information density was introduced by Fitts[1]. The amount of information of human motor system is discribed as the variables of \\ x analog measure of successive responses. It is infered as the mean information transmission rate. In 1980, Sakkits defined information capacity as the resolution of angular motion of the Fig.1 The body coordinates system by the joint[2]. This attractive approach has electromagnetic position sensors in pointing a been also applied widely, because it can target, with touch point sensor in the space be explained with no connection with coordinates(O-XYZ) difference in the physical measures. Thougu these methods are based on the error analysis of a motor task, it also 3 dim RS232C Personal can lend insight into the principles, positoin sensoi which the movement of the upper computer extremities is organized and controlled. Some of the researchers has studied to Input/output I experimentarily understand the invariant Personal properties according to the kinematic representation of a motor task in pointing, ~A/D] computer etc. It was often useful to describe a characteristic mechanism of the ip li sensorimotor transformation[3]. In this preliminary study, we have eximined the usefulness of information Fig. 2 The integrated measurement system for theory approach to estimate the human input/output interface of the electronic devices performance in reciprocally and discretely pointing a target. This approach may be able to use a new measure of assessment of the residual functions for a better communication.
I
i
3. THE INTEGRATED MEASURE-MENT SYSTEM First we have developed such an integrated system as shown in Fig. 114]. This is composed of 1) 3 dimensional electro-magnetic positoin sensor(EMPS, POLHEMUS) to sense the motion of the upper extremity, and 2) color LCD (14 inch ) with touch position sensor (TPS). Now this system has been applied for estimating the other devices, as shown in Fig. 2, which has some of the interfaces :mouse, A/D converter, etc. Here the reciprocal and discrete pointings will be experimentally discussed.
739
W
< //c-.L Liquid crystal display
Fig. 3 2 dimensional setup of experimental display for reciprocal and discrete motion S=starting point, P(i)=pointing target, W=targel width, L=movement distance between 2 targets. In reciprocal pointing, after holding the hand, the experiment starts on pointing S. P(i) and P(i+l) are shown on the display during a session. In discrete movement, for each trial, after holding the hand, the subject points a target after appearing for 0.5 sec (regulated with the level of motol impairment: 0.5~2.0 sec ). on the 8 directions of randamized position.
The 4 sensorsof EMPS has been fitted on acromion, distal and post. region of humerus, ext. retinaculum, and dorsum manus. The pointing co-ordinates are directly picked up by the touch positoin sensor. In reciprocal pointing, its experimental technique is almost the same as Fitts. Here it is extended into such a 2 dimensional surface as CRT display. In discrete pointing, as an example of thehorizontal direction, the total distance 24.5 cm is divided in 49 sections. W = 0.5, 1.0, 2.0 cmD, and L = 16, 8, 4 cm. The subject was indicated to point a target as fast and accurate as possible, and not to adjust pointing at the end of target. The subjects are 14; normal = 6, SCI = 4, CP = 4. 4. KINEMATICS OF THE UPPER EXTREMITY Here only the results of normal and SCI subjects are shown in Fig. 4. There are few of kinematic motion analysis of the upper extremity, especially on the joint movement of pronation/supination. The normal shows a smooth coordinated motin pattern with the smaller varialbes (s.d.). He mainly controls a motion round the shoulder, and may adjust the joint angles of the wrist, for a precise pointing. The SCI patient is C5 paralysis. His left hand is of a weak flexor, pronator/spinator (PR/SP) of the forearm, which moves with lighter load. The fight hand is severely motorimpaired. His shoulder scarecely gained movement because of difficulty in trunk control. The upper arm has a similar motion as the normal, but a longer and delayed motion in the vicinity of reciprocal pointing. He usually has a large PR/SP of the elbow joint, which has a faster response at the beginning of a pointing motion. This compensatory movement may be related to set a direction, together with extension/flexion of the elbow joint. Lastly we point out that the forearm movement is irregular because of weak muscle force, frequent muscle fatigue, and limited range of motion (ROM), though the variables of the velocity of the hand become larger by adding another session. It may lead to the larger pointing errors. 5. M O V E M E N T TIME AND POINTING E R R O R Here we tested how compensatory movement affects the pointing accuracy at the end of the arm. At first, the displacement and the velocity(and s. d.) of the stylus pen are shown in
740 (a)
Normal, 1st session
(b)
SCI, 1st session
Shoulder
Shoulder RIGHT~L~ EFT ,
(CM) LEFT~ .5
(CM) RIGHT--)LEFT--)RIGHT 1.5i ......... 1
0.5 0 -0.~
). -0.
-1.5
-1
f,
, VT v,T
-2 0
0.5
0
1 (SEC)
0.5
1
1.5
2
(SEC)
Upper ann
Upper arm (]DEG) 20
(DEG) 20
10
( D E G ) ...... , ~
40
10
0
0
-10 -20 0
0.5
, 0
l
PR
20
O'
-10
(c)Forearm, 5th session, SCI
0.5
, EX,
,I
1
2
1.5
-20 0
0.5
1
(SEC)
(d)Forearm, lOth session, SCI
Forearm
Forearm (DEG) 20 ..............................................................................
[~G)
"~-,.
1.5 (SEC)
,...........
~G)
.........
10 0
-10 . . . . . . . . . ..., . _ , , " 7 ~
-10
-20
-4O
-20 0
0.5
1
(SEC)
0
0.5
1
1.5
2
(SEC)
t 0
0.5
1
1.5
(SEC)
Fig. 4 An example of the motion patterns of the upper extremity in reciprocal pointing The normal and SCI subject put long oponens wrist hand orthosis to lock the wrist joint. Movement distance = 16cm, target width - 1.0cm*, # of sessions = 10, # of reciprocal motion for each session = 25 for the normal, 10 for the SCI patient. Each pattern is shown in the curves of the mean * s.d., FB = forward/backward, LM=lateral/medial, VT-vertical displacement;PR=pronation(+)/spination, AD=abduction/adduction(+), EX = extension(+) /flexion direction
741 The SCI (N=10),
The normal (N=25) (cm)
16
(om)
(¢m/8~)) >Left < > R i g h t
light<
1st trial (om/s /
25 ~~2~01[5L1e~/ ~
E80 60 v
40"~ ~I0
E
®8 o D
~4
.~
20 ~
20
;~ 5
0
0.2 0.4 0.6 0.8
40
0
0
0.5
1
T ime (see)
The SCI (N=10),
The SCI(N=I 0), 10th trial
~25
80
~=20
60~
Ol 0 -~ 2
40 "~ 20 ~ o 0
0.5
1
T ime (see)
5th trial
i15
0
1.5
25 ~20 ~15
t
]80
~10 0
1.5
0
T ime (sec)
0.5
1
1.5
T ime (sec)
Fig. 5 Displacement of position and velocity of the hand in a cycle of reciprocal pointing Fig. 5, in which the wrist is free. The displacement is calculated as the distance from the operating hand. The normal shows a smooth curve and a smaller s. d. with a constant speed in pointing a target. On one hand, the SCI has a larger s. d. at the accelerating and decelerating phase. A motion pattern has a tendency of the continuous changes with the muscular force not enough to control a movement precisely (similar to the above). Secondly, the distribution area of pointing (DA) is calculated by estimating the eliptic area, on 2 dimensional normal distribution hypothesis (95% c.i.). A typical movement time (MT) and DA are
6
........... o i \
......................... ,
,,
5 "LRi\:'
:
~4 ~, 3 ~ 2 .-~ -1 ,T, 0
i ::\ I~,CP
o
16 8 4 Pointing d i stance (cm)
.
,
'1
16 8 4 Pointingdistance(an)
Fig. 6 Movement time and the pointing errors" RL = from fight to left, LR = from left to fight, in reciprocal pointing,NO, SCI, CP = the normal, and SCI, and CP
742 shown. In the normal, DA area is less than lcm*, and MT is not well-correlated with the movement distance (L). In the CP, ( quadriplegia, with athetosis), MT has a tendency to decrease with smaller L, but has comparatively larger variances. DA is not correlated with L, and has difference in pointing between the right and left targets, because of difficulty in trunk positioning control. Next, SCI has a remarkable tendency in dependence on a limitted ROM, that is, a large difference of DA between the right and left targets, and a shorter MT with a smaller L. These chracteristics of MT and DA are very useful to estimate one of the main parameters for the characteristic responses of motion control at the end point. 6. INFORMATION ON POINTING RESOLUTION It is useful to test the point resolution for 5 quantitative digital measure of the residual +~C0) function. Here a discrete pointing is used, .__. and its setting is described before. ~ 4 --I-~CC) In Fig. 5, an example of the experimental .....A ..........SCl . results is shown. In the normal, the--= transmitted information is 4, with the eyes ~ 3 CP open, and 3 bits with the eyes closed. The ~.~ CP subject has a similar accuracy with the = 2 normal with the closed. In the case of SCI, it ~ was a little better than CP. I I I I These show that it is possible for the 1 0 10 20 30 40 50 direct pointing motion to apply for Sakkits' s Targets (N) method. For the future work, it may be made use of clarifying the cognitive Fig. 7 Resolution in discrete pointing: spatial-motor process, comparing with data O, C = with the eyes open, and closed obtained under different behabioral manipulation. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A
q-.
REFERENCES 1)Fitts, P. M.: The information capacity of the human motor system in controlling the amplitude of movement, J. Exp Psychol, 47, 381-391, 1954 2)Sakkits, B., etal: The information transmitted at final position in visually triggered forearm movements, Biol Cybern, 46, 111-119, 1983 3)Soechting, J. F., Flanders, M.:Sensorimotor Representation for pointing to targets in three-dimensional space, J. Neurophysiol., 62(2), 582-594, 1989 4)T. Yamamoto, Measurement system to assess input control of computer for difficulty in upper extremity, Proc. of 16th Anuual International Conf., IEEE EMBS, Baltimore, 1994
IV.6 Musculoskeletal, Postural, Visual, and Psychosocial Outcomes Resulting from Ergonomics and Optometrical Intervention
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawaand H. Mori (Editors) 1995 Elsevier Science B.V.
Musculoskeletal, postural, visual, and psychosocial outcomes resulting from ergonomic and optometric intervention
A. Aarrs l, G. Horgen 2, M. Thoresen 3, Bugajska4, A., Wolska4, A., Danuta4,R., WiderszalBazyl4, M., Konarska4, M J. Dainoff s , B.G.F. Cohen6 , M.H. Dainoff 7 AlcateI-STK, Oslo, Norway. 2 Kongsberg College of Engineering, Kongsberg, Norway, 3. MEDSTAT, Lillestrom, Norway, 4 Central Institute for Labour Protection, Warsaw, Poland, s Miami University, Oxford, Ohio, USA, 6. Internal Revenue Service, Washington, D.C., USA, 7 M. Dainoff Associates, Cincinnati, Ohio, USA.
1. INTRODUCTION The MEPS project "Musculoskeletal, Eyestrain, Psychosocial Stress" represents an unprecedented example of international multidisciplinary cooperation and coordination, the objective of which is to examine the effects of various kinds of ergonomic interventions, including corrective lenses, on a combination of musculoskeletal, postural, and psychosocial outcomes. These studies have been conducted in several different countries. Each country has utilized the same standardized research protocol, but ergonomic interventions are individually designed. Preliminary results of this research were presented at WWDU '94 in Milan. At that meeting, the focus was univariate statistical analysis of data from individual countries. The present paper will focus on cross-country comparisons of relationships between predictor and health outcome variables.
2. METHOD 2.1. Design The basic research design, repeated in each country, consisted of four components: Pre-Test, Intervention, One Month Post-Test, and One Year Post-Test. A research protocol was developed which was repeated for each test component. The protocol consisted of a series of standardized measures carried out on each of the participating subjects. 2.2. Research Protocol The protocol was composed of standardized measurements of musculoskeletal load, demographic and psychosocial factors, medical and optometric status, and ergonomi¢ factors. Musculoskeletal load was assessed using a computerized field-portable apparatus
745
746 (Physiometer) which allowed simultaneously measurement of EMG of right and left trapezius, and postural angles of head, neck, and trunk. These measurements were carried out during a 45 minute period of data entry at the worksite. Ergonomic analyses of the workstation included workstation dimensions and luminance/illuminance measures. Demographic and psychosocial questions--including attitudes toward work, family and economic situation, and symptoms of psychological stress--were obtained by a combination of interview and questionnaire. A similar approach was used to obtain subjective ergonomic assessments of subjects' workstations. Finally, each subject received professional medical and opthalmological examinations.
3. ANALYSIS STRATEGY For this presentation, results will be presented from the baseline Pre-Test data only. Comparisons will be among the three national data sets presently entered into the centralized data base--those from Norway, Poland, and the U.S. (Swedish results will be presented in a separate paper.) The following groups of variables drawn from the protocol have been identified for further analysis: a.) Health Outcome Measures (subjective judgments of pain) b.) Physical Examination e.). EMG Measures (taken during sample work period) d.) Postural Measures (taken during sample work period) e.) Optometric Examination f.) Psychosocial Measures (questionnaire and interview) Variables from groups c-f will be considered as exposing (etiologic) factors which will be used to predict health outcome measures from groups a and b. EMG variables will be related to neck and shoulder pain. These variables include measures of static load-- values of Maximum Voluntary Contraction (%MVC) at the 10th percentile of the cumulative distribution, work pauses--duration and number of periods below 1 and 2 %MVC, and median load--values of%MVC at the 50th percentile of the cumulative distribution. Postural angles will be related to neck, shoulder, and lower back pain. Extent of average (median) values of back flexion will be related to extent of lower back pain. Other postural variables will be related to shoulder and neck pain. These include static head
747 position, head extension, static sideways head rotation, upper arm extension, flexion, and abduction. In addition, duration of fixation time on the display screen, derived from the postural data, will be related to visual discomfort and headache. Of the components of the physical examination, the isometric test (maximum contraction against resistance for 15 sees), extent of range of sideways head motion, and number of trigger points will be related to neck and shoulder pain. The outcome ofPhalen's test (complete wrist flexion for 30-60 sees) will be related to forearm/hand pain. Finally, extent of smoking will be related to lower back pain. Of the components of the optometric examination, existing correction--if any--will be related to eye discomfort but also to head, neck and shoulder discomfort. Lastly, from the psychosocial questionnaire, reports of psychological problems, feelings of tenseness, and sleep problems will be related to all of the health outcomes.
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
A METHOD TO CONSIDER ERGONQMIC CONDITIONS AT VDT WORKPLACES Annika Johansson and Houshang Shahnavaz Department of Human Work Science, Division of Industrial Ergonomics, Luleh University, S-971 87 LuleL Sweden In 1989, an international project was initiated known as MEPS (Musculoskeletal, Visual, and Psychosocial Stress in VDT Operators in Optimised Environment). The aim of the MEPS project was to minimise stress amongst VDT operators by creating optimum working conditions through intervention measures at individual workstations. Professionals representing different disciplines were involved in this study to investigate possible factors causing musculoskeletal complaints, visual discomfort and psychosocial stress among VDT users. This paper describes the ergonomic part of the study including the methods that were developed and used to examine prevailing working conditions and how the data was analysed. B ACKGROUND TO THE ERGONOMIC INVESTIGATION Two different questionnaires were developed for the ergonomics part of this international project 1. One was completed by the VDT operator and the other by the ergonomic experts. Operators having years of experience in their work are regarded as valuable resources for identifying work-related problems in their work environment, since they are facing the work environment daily. However, experts can contribute by identifying problems that have long term effects. It is also true to say that some operators can get used to a certain type of problem and are for that reason unable to identify it as a problem. In such cases expert evaluations are a vital contribution (Noro, 1991). Both questionnaires have undergone thorough revisions at the meetings held with the international project members. The reason for these revisions has been to obtain questionnaires that are applicable in all countries from a practical, financial, and scientific point of view. The questionnaires have been designed so that they are easy to complete without being time consuming and will still gain good insight into the prevailing situation and allow for international comparisons. The ergonomic investigation consists of both objective and subjective evaluation measures, "objective" in terms of experts evaluating the work environment through various measures and "subjective" through operators self-reported ratings. While evaluating a workstation, whether it concerns musculoskeletal, psychosocial, or visual problems, it is important to collect information from both experts and the actual users and to relate it to prevailing problems. This means that objective as well as subjective estimations of the working conditions are necessary to provide a comprehensive description of prevailing circumstances. The objective data obtained from these examinations are seen as complementary to the subjective information gathered. The self-reported ratings are valuable in themselves because of their subjectivity, since they take into account individual differences and preferences. These questionnaires were developed in conjunction with Prof. M Dainoff, Miami University, OH, USA.
749
750 THE ERGONOMIC PART OF THE STUDY Workstation equipment, whether it concerns the chair or the VDT screen, should be fully adjustable in order to satisfy individual differences and different tasks. Our bodies are made for movements, and prolonged sitting in the same posture may cause discomfort. It is of vital importance that operators are able to change postures during work and that the workstations allow this change and do not force the body into an unwanted position. Adjustment possibilities should be convenient and designed to encourage use. Adjustments should also be easy to handle, require minimal effort to use, and allow postural changes without imposing inconvenience to the user. In the MEPS study the operators were asked to evaluate the usability of the VDT (e.g. adjustment possibilities, readability, and clarity), furniture (e.g. adjustment possibilities, comfort, and working space) as well as ergonomic aspects of their work environment such as climate, lighting conditions, and noise level. Method of data analysis: Development of combination codes The questionnaire for the operators' subjective estimations was constructed so that the respondents were asked either to place a mark on a visual analogue scale (VAS) to indicate their opinion or to choose among different alternatives (Johansson, 1992). These methods were decided upon agreement with statisticians involved in this international project. The purpose was to find standardised forms for recording the data in order to simplify a multi-disciplinary analysis. The rating scale chosen is a 100 mm line, where the anchor points are described with words. This is the most common technique used by ergonomists, especially because it is easy to use, particularly for the respondents (Sinclair, 1991). Specific measurements were made by an expert in those cases where a condition in itself was not clear and did not allow for correct estimation. The lighting conditions and available height and angular adjustment possibilities are examples of conditions that were measured (Johansson, 1992). Data, both from the experts' and the operators' questionnaires, were summarised in order to provide ergonomic explanations. All visual analogue scales were divided into seven equal parts in order to simplify the analysis. .......
7
' 1
' 2
' 3
' 4
'
5
7
'
6
7
The first three parts (1, 2, and 3) are regarded as not acceptable, needing some immediate interventions. Parts 4 and 5 are regarded as acceptable conditions, needing some potential improvements. The last parts of the scale (6 and 7) are regarded as good with no improvements required. Responses to different questions that are related to a specific issue are combined into suitable "combination codes" and given a new set of numbers that indicate the condition of a certain feature of the workstation, e.g. not acceptable, acceptable, or good. The following are examples of combination codes that were developed: • Working chair adjustment possibilities • Working chair adjustment usability • VDT screen/support adjustment possibilities • VDT screen/support usability • VDT keyboard adjustment possibilities • VDT display adjustment possibilities • Screen readability • Working space usability Below are some examples of combination codes, how these were developed, and the outcome of these combinations.
751
Combination codes for Working Chair Adjustment Possibilities How good are the adjustment possibilities of the working chair? Are they satisfactory from both experts' and operators' point of view? This can be answered by a combination of the following questions from operators' questionnaire (CRF IIc) and experts' questionnaire (CRF IIIc) (for further information see Johansson (1992)). CRF IIc 2.How good is your chair regarding the possibility of adjusting the height? 1
7
Very poor
Excellent
3. How good is your chair regarding the possibility of seat tilt adjustments so as to allow you to lean forward while working? 17 Very poor Excellent 4. How good is your chair regarding the possibility of seat tilt adjustments so as to allow you to lean backward while working? 1
7
Very poor
Excellent
5. How good is your chair regarding the possibility of making adjustments while seated? 1
7
Very poor
Excellent
CRF IIIc 6. Height adjustments; seatpan, armrests, and lumbar support. 7. Angular adjustments; seat, back. 14b.Force requirement 1
7
Very high
Very low
Out of these questions, question 2 and 5 in CRF IIc and questions 6 and 14b in CRF IIIc have higher priority than the rest of the questions because these requirements are more important to accomplish.
Operator's evaluation on adjustability of working chair Questions 2 and 5 (CRF IIc) are combined to find out how good the possibility of adjusting the height of the chair is and if it is easy to adjust the chair from a sitting position. All possible combinations of these two questions are shown below.These combinations are given a new set of numbers (1-6) as combination code OI. CRF IIc 2
CRF IIc 5
-
2:1,2or3 2:1,2or3
o
o o o + + +
2:1, 2 or3 2:4 or5 2:4 or5 2:4 or5 2:6 or7 2:6 or7 2:6 o r 7
5:1,2or3
5:4 o r 5 + 5:6 or 7 - 5:1,2or3 o 5:4 o r 5 + 5:6 o r 7 - 5:1,2or3 o 5:4 o r 5 + 5:6 o r 7
Combination code O I 1 2
3 2 4 5 3 5 6
These combinations can also be shown as in figure 1.
752
Combination code 0 1
Figure I
CRF Uc 2
Not
Not acceptable
Acceptable
Good
2:1m2~3
2:4t5
2:6~7
5: 1,2,3
0
1 CRF lie 5
Acc
5:4,5
-
0
°
3
O0
- +
+0 4
S
O+
3
+
2
2 Good 5:6,7
o
++
5
6
There is only one combination (combination 6) that indicates that the chair has good possibilities from a operator's point of view for height adjustments and for making adjustments while seated. Combinations 4 and 5 indicate acceptable conditions, i. e. needing some potential improvements.
Expert's evaluation on adjustability of working chair A judgement of the chair's height adjustments is done by assigning a special code based on the height adjustment possibilities given in question 6 in CRF IIIc. Code Bad adjustment possibilities 1 Acceptable adjustment possibilities 2 Good adjustment possibilitites 3 If the chair has good adjustment possibilities within a suitable range, compared with international recommendations and standards, then the chair is given code 3 for height adjustments. An "acceptable" chair might have a suitable range for seat height adjustment but not good adjustment possibilities for the armrest. Questions 6 and 14b from CRF IIIc are combined to combination code E I to provide experts' evaluations on chair adjustability. CRF IIIc 6
CRF IIIc 14b
-
6:1
-
14b:1,2or3
Combination code E I 1
o o o + + +
6:1 6:1 6:2 6:2 6:2 6:3 6:3 6:3
o + o + o +
14b:4or5 14b:6or7 14b:l,2or3 14b:4or5 14b:6or7 14b:l, 2 o r 3 14b:4or5 14b:6or7
2 3 2 4 5 3 5 6
Only combination 6 indicates good adjustment possibilities from an expert's point of view. Combinations 4 and 5 indicate acceptable adjustment possibilities and acceptable force for adjusting.
Operator's evaluation of seat tilt Is the adjustability regarding the possibility of seat tilt backward and forward satisfactory from the operator's point of view? To find out whether the seat tilt is satisfactory or not, we have combined questions 3 and 4 in CRF IIc. These combinations are listed under "combination code O II".
753 CRF IIc 3 (forward) -
3:1,2or3 3:1,2or3 3:1,2or3
o o o + + +
3:4 or5 3:4 or5 3:4 or5 3:6 or7 3:6 or7 3:6or7
Combination code O I I
CRF IIc 4 (backward) o +
4:1,2or3 4:4 or5
1 2 3
4:6 o r 7 - 4:1,2or3 o 4:4 o r 5 + 4:6 o r 7 - 4:1,2or3 o 4:4 o r 5 + 4:6or7
2 4 5 3 5 6
There is only one combination, combination 6, that indicates good seat tilt backward and forward. Combinations 4 and 5 indicate acceptable possibilities of seat tilt.
Expert's evaluation of angular adjustments A judgement of the chair's angular adjustments is done by assigning a special code based on the angular adjustment possibilities given in CRF IIIc, question 7. Code Bad adjustment possibilities 1 Acceptable adjustment possibilities 2 Good adjustment possibilitites 3 If the chair has good adjustment possibilities within a suitable range, compared with recommendations and standards, then the chair is given code 3 for angular adjustments. An "acceptable" chair might have a suitable range for for the seat adjustments but not for the backrest.
Expert's evaluation in combination with operator's evaluation of angular adjustments Combination code O II is combined with the expert's evaluation of angular adjustments of seatpan and backrest, CRF IIIC 7, to develop a combination code which indicates operator's and expert's assessment of the angular adjustments of the chair. CRF IIIc 7
Combination code O I I
Combination code O/E 1
-
7:1 7:1 7:1 7:1 7:1 7:1
1 2 3 4 5 6
1 1 1 2 2 3
o o o o o o + + + + + +
7:2 7:2 7:2 7:2 7:2 7:2 7:3 7:3 7:3 7:3 7:3 7:3
1 2 3 4 5 6 1 2 3 4 5 6
2 2 2 4 4 5 3 3 3 5 5 6
Combination 6 indicates good adjustment possibilities from both experts' and operators' points of view. Combinations 4 and 5 indicate acceptable possibilities.
754 Results of combination codes Good height adjustments and having the possibility of making adjustments while seated, are considered as important features of a functional working chair. The working chair should also have good possibilities for angular adjustments to allow a comfortable working posture while leaning forward or backward. The operators' evaluations of the adjustability of their working chairs are summarised in table 1. The results given are based on a survey among 25 VDT operators involved in full-time data-entry work at the Swedish Post (Johansson, 1992; Westlander et al., 1992).
Table I
Operators' evaluations of adjustability of working chair, [n=25] (Combination code 0 I) Adjusting the height
Adjustments while seated
Not acceptable
Acceptable
Good
12 %
16 %
24 %
Acceptable
0
12 %
24 %
Good
0
4%
8%
Not acceptable
Table 1 indicates that only eight percent of the operators evaluated the adjustability of their working chairs as good (please compare with figure 1). The possibility for making adjustments while seated has gained a much lower rating. The overall judgement from an expert's point of view, of the same working chairs, was that the chairs had acceptable force requirements for adjustments and acceptable height adjustments, i. e. the chairs were satisfactory but could be better to encourage use of the adjustments. The experts' evaluations are based on established knowledge, recommendations, and standards in the ergonomic area. The combination of experts' and operators' evaluations of angular adjustments is displayed in table 2. This combination indicates the operators' attitudes to the adjustment possibilities and measures and current values from the experts' points of view.
Table 2
Experts' and operators' evaluations of angular adjustments [n=25] (Combinations code O/E I) Operators' evaluations of angular adjustments
Experts' evaluations of angular adjustments
Not acceptable
Acceptable
Good
Not acceptable
12 %
4%
0%
Acceptable
52 %
20 %
8%
0
0%
0%
Good
The experts have judged the angular adjustments of the chairs acceptable. The angular adjustments were within acceptable ranges compared with available recommendations and standards. Obviously, most of the operators do not find the angular adjustments as acceptable. More than 60 % have evaluated the possibility for adjustments as not acceptable. It is hard to know how the operator actually has assessed the situation and arrived at an answer, for example, whether ease of
755 adjustment and force required for adjustment are included in the answer. The subjective evaluation in these types of questions give another dimension to the actual values, since the figures in themselves do not very well describe the attitudes of an individual. CONCLUDING REMARK While evaluating VDT workstations, it is of vital importance to involve both experts and operators. "Objective" and "subjective" measurements are both necessary for the evaluation of human working conditions. No method is better than the other, since these methods gather different kinds of information regarding the conditions and should be seen as complementary to each other. The "subjective" evaluation, i. e. the actual user's assessment of a certain condition, is the main importance to make a true and valid evaluation of a prevailing situation. Conditions concerning a certain comfort level are difficult to assess by the use of "objective" measurements only. The "objective", i. e. the expert's evaluation, contributes to the situation by giving a description of the prevailing conditions based on well-known and accepted knowledge and evaluation methods. The ergonomic methods developed for data analysis in this international MEPS project have given valid information by taking into account operators' as well as experts' evaluations, and these results can act as a platform for improvements of VDT workplaces. Furthermore, the method of combining different aspects of an ergonomic issue into a specific combination code can provide comprehensive information on a specific issue. ACKNOWLEDGEMENT Sincere appreciation and gratitude for their enthusiastic participation are given to all our colleagues in the international MEPS project and to the operators and management at the workplaces studied. Financial support was received for this project from the Swedish Working Environment Fund (project no: 90-0127,91-0925). REFERENCES Johansson, A. (1992). Synthesis of experts' and users' knowledgefor ergonomic improvements of VDU workplaces. Licentiate thesis, 1992:26L. Lule~. University of Technology, Lule~. Noro, K. (1991). Methods and people. In K. Noro & A. Imada (Eds.) Participatory Ergonomics. London:Taylor&Francis Ltd. Sinclair, M. A. (1991). Subjective assessment. In J. R. Wilson & E. N. Corlett (Eds.)
Evaluation of Human Work: A Practical Ergonomics Methodology. London:Taylor&Francis Ltd. Westlander, G., Shahnavaz, H., Johansson, A., and Viitasara, E. (1992). An b~tervention-oriented Study of VDT Operators in Routinized Work. Research report 1992:32. Solna: National Institute of Occupational Health. In Swedish, summary in English.
This Page Intentionally Left Blank
W.7
Physiological Measurements 1
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
759
Task.related musculoskeletal disorders in computerized office work P. Sepp~il~i Finnish Institute of Occupational Health, Topeliuksenkatu 41 a A, FIN-00250 Helsinki, Finland
Usage of computers in various tasks related to real estate and construction affairs was examined in a municipal organizational unit. 104 employees, representing different occupational groups, answered a questionnaire on musculoskeletal disorders. The frequency of reported symptoms was associated with gender, occupational gategory and age.
1. INTRODUCTION Experiences in the use and implementation of computerized technology were examined in the municipal sector (1). This paper describes the experiences from an organizational unit which deals with real estate and construction affairs. The study is being conducted stepwise. In the first phase, an extensive questionnaire was administered to chart the extent of the use of different computerized equipment and application software, as well as the opinions on training, usability of software, stress and strain, etc. in different user groups. 314 employees answered the first questionnaire. The respondents represented the following job categories: 1) administrative jobs 2) planning jobs, 3) clerical jobs, 4) draftsmen, 5) supervisory jobs 6) others. The first questionnaire study showed that the attitude toward computerization was positive, in general. Criticism was expressed, however, toward implementation, especially as regards participation and training. Furthermore, the problems were experienced differently in different job categories, as well as in different age groups. The draftsmen had clearly more problems than the other occupational goups. Also, criticism toward various aspects of implementation increased systematically with age. Prompted by the findings of the first study, a continuing study was launched in a unit in which numerous different kinds of work tasks were being done, relating to the preparing of maps needed in town planning and road construction. The main emphasis in the second part was on musculoskeletal problems.
760 2. MATERIAL AND METHOD A questionnaire was delivered to the whole personnel of the department under study (130 in total). 80 % returned the questionnaire. 63 % of the respondents were women. The age of the subjects varied from 21-62 (mean 41.8 and standard deviation 10.2) years. The average work experience in their current or in comparable tasks was 13 years. 53 % of the respondents had an intermediate school education and 30 % had completed secondary school. A look at the professional education showed that about 50 % had been trained in a vocational school or had attended specific courses, one quarter had completed a technical school and 8 % had a university level education. The questionnaire used was constructed on the basis of the questionnaire developed in the context of the prevention program on work-related musculoskeletal disorders carried out in the Finnish Institute of Occupational Health (2).
3. RESULTS 3.1. Work tasks and equipment used According to their job rifles, the respondents were divided into the following categories (the percentage of women in each category is given in parentheses): 1) draftsmen (96 %), 2) clerical workers (100 %), 3) technicians (26 %), 4) chiefs and foremen (17 %) and 5) others (38 %). The occupational categories differed clearly according to gender, Nearly all draftsmen and clerical workers were women, whereas men constituted the majority in the other job categories. Because job rifles do not necessarily tell a great deal about the characteristics of the work, more detailed questions were posed in the questionnaire. Table 1 shows the extent to which different types of basic tasks, conducted by means of computerized equipment, were performed among the respondents.
Table 1 Prevalence of different tasks in the study group (%)
Word processing Drawing Calculations Updating registers Browsing registers Digitizing
41 37 31 35 52 27
761 Different types of visual display units, such as terminals, PCs with and without a mouse, general purpose, as well as special digitizing work stations were used for performing work tasks. The job contents and use of different equipment differed between the men and women. Women, more than men, were engaged for long periods doing word-processing, drawing, digitizing, and updating registers.
Table 2 Neck-shoulder and upper etremity pain experienced during the past 12 months, according to gender (%) Symptom
Men n=35-37
Women n=63-64
Pain radiating from the neck to the upper extremity > 7 days >30 days longest period >14 days hinders daily activities >30 days
31 22 14 11 20 6
56 39 20 8 38 6
Other neck-shoulder pain > 7 days >30 days longest period >14 days hinders daily activities >30 days
76 32 11 5 39 3
78 56 19 8 52 5
Shoulder pain > 7 days >30 days longest period >14 days hinders daily activities >30 days
40 22 14 11 30 8
43 21 11 11 30 6
Pain in forearm and hand > 7 days >30 days longest period >14 days hinders daily activities >30 days
20 8 5 8 16 5
42 17 8 6 31 5
762
3.2. Pain experienced during past 12 months, according to gender and job title It can be seen from Table 2 that women had had neck-shoulder pain more often than men, and the pain had hindered the daily activities of women more often than of men. Table 3 Neck-shoulder and upper extremity pain experienced during the past 12 months, according to job title (%)
Symptom
DraftsClerical TechChiefs & men workers nicians foremen n=25-26 23-24 n=19 n=17-18
Others
Pain radiating from the neck to the upper extremity 8-30 days >30 days
12 15
25 33
11 5
12 12
17 17
Other neck-shoulder pain 8-30 days >30 days
28 28
33 17
32 5
33 6
33 17
Shoulder pain 8-30 days >30 days
8 12
9 13
5 11
6 11
15 15
Pain in forearm and hand 8-30 days >30 days
4 12
8 8
5 5
0 0
15 8
n=12-13
Draftsmen and clerical workers had experienced more neck-shoulder pain than the other groups (Table 3). Especially clerical workers reported pain radiating from the neck to the upper extremity, whereas draftsmen experienced mostly general neck-shoulder pain.
763 3.3. P a i n e x p e r i e n c e d d u r i n g p a s t 12 m o n t h s , a c c o r d i n g to age The employees aged over 40 years reported more pain in all categories, except unspecified neck-shou]der pain, which the younger respondents have experienced more often (Table 4).
Table 4 Neck-shoulder and upper extremity pain experienced during the past 12 months, according to age (%) Aee (vears) - 30 n=13-15
31-40 41-50 50n=28-29 n=33-34 n=23-24
Pain radiating from the neck to the upper extremity > 7 days > 30 days longest period > 14 days hinders daily acitivities > 30 days
40 20 13 0 20 0
39 21 7 0 14 4
48 42 21 15 39 6
58 42 29 17 48 13
Other neck-shoulder pain > 7 days > 30 days longest period > 14 days hinders daily activities > 30 days
85 31 8 0 29 0
79 52 10 0 45 0
74 56 21 18 50 6
71 33 21 4 54 8
Shoulder pain > 7 days > 30 days longest period > 14 days hinders daily activities > 30 days
15 8 0 0 8 0
38 10 3 3 17 3
35 18 15 9 29 6
67 46 25 30 58 17
Pain in forearm and hand > 7 days > 30 days longest period > 14 days hinders daily activities > 30 days
29 14 7 7 21 7
24 7 0 0 14 0
35 12 6 6 26 3
42 25 17 17 37 13
Symptom
764 4. DISCUSSION AND CONCLUSIONS On the basis of the questionnaire data we can conclude that the jobs are divided into "women's jobs" and "men's jobs". Women did long periods wordprocessing, digitizing and updating of registers. Men were more often in supervisory positions, and their daily task profile was more variable than that of the women. Typically, men performed different tasks in rather short periods, such as calculation, drawing and browsing registers. These differences in job content seem to be one explanation for the finding that women had more neck-shoulder pain than men. However, we know from previous studies that women tend to report their pains more than men (2-3). It is also typical that gender and job content are confounded in many branches. Consequently, it is difficult to show, especially in a small s_Rmple like this, how much the reported symptoms depend on work tasks and on working conditions. On the other hand, studies on computerized office work have revealed that prolonged work with VDUs is associated with the risk of neck-shoulder pain (3). Thus it is very probable that the symptoms reported in this study are work-related. In addition to individual physical fitness and recreational activities, remedies should be sought by analyzing work organization, job content and workplace design, and by searching for solutions to the problems through better organization of work and workplace design. These last-mentioned activities have already been started in cooperation with the personnel. Efforts for developing equipment and working conditions, as well as the organization of work, become especially important because of the aging of the employees.
REFERENCES
1. Hukki K, Sepp~il~i P. Utilization of users' experiences in the introduction of information technology: a study in a large municipal organization. In: Ilmarinen J, editor. Aging and work. Proceedings of the International Scientific Symposium on Aging and Work; 28-30 May 1992. Helsinki: Finnish Institute of Occupational Health, 1993:170-176. 2. Viikari-Juntura E, R i i h i ~ H, Takala EP, Rauas S, Lepp~inen A, Malmivaara A, et al. Factors predicting pain in the neck, shoulders, and upper limbs in forestry work. Ty8 ja ihminen 1993;7:344-350.(in Finnish with English summary) 3. Sauter S, Hales T, Bernard B, Fine L, Petersen M, Putz-Anderson V, et al. Summary of two NIOSH field studies of musculoskeletal disorders and VDT work among telecommunications and newspaper workers. In: Luczak H, Cakir A, Cskir G, editors. Work With Display Units 92. Amsterdam: North-holland, 1993: 229-234.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
765
Analysis of mental workload during the work with computers using R-R intervals time series Kiyoko Yokoyamaa, Masanori Moyoshib, Yosaku Watanabe c, Takayoshi Yoshiokac, Isao Kawamura~and Kazuyuki Takatab aNagoya Municipal Women's College, 2-1-10 Kitachikusa, Chikusaku, Nagoyashi 464, Japan bDaido Institute of Technology,2-21 Daidocho, Minamiku, Nagoyashi 457, Japan CToyota National College of Technology, 2-1 Eiseicho Toyotashi 471, Japan This paper describes the analysis methods and results of mental workload during computer work. R-R intervals time series were used for the analysis. The heart rate, coefficient of variation of R-R intervals, power spectral density function and impulse response function deriving from autoregressive model were used as parameters of heart rate variability. Computer workload was characterized by comparing other mental and physical workloads to computer ones. Results are also described with regard to changes of the computer workloads during 75 minutes work.
1. Introduction Several methods using heart rate variability have been used to evaluate mental workload[ 1,2]. And there are a few cases using heart rate variability in analysis of mental workload during the work with computer[3 ]. However there are few studies on cardiovascular system identification in respected of the impulse response function to evaluate parameter for the workload with computers. This study aims to compare the mental computer workloads with other mental ones or physical ones using heart rate variability and the cardiovascular system identification. And it aims to analyze the changes of the heart rate variability during the period of computer workloads. Impulse response function is used as cardiovascular system identification parameter. We propose a new estimating method concerning impulse response function. In this method, the autoregressive model of R-R intervals time series deriving from ECG is used to estimate the impulse response function. 2. Experimental protocol Two tasks were set to the subjects as a computer workload. One is the work of word processing using word processor software on the personal computer. Another is the adding up two the two-digits decimal numbers on CRT screen of personal computer. The answer was keyed in from the keyboard. The diagonal of the CRT screen was 14 inches. The white characters were displayed on the black screen. The subject keyed in the answer. Next question was displayed at same time. The subject couldn't know the answer is correct or not. The number of the answers and correct answers was recorded on the magnetic disk automatically.
766 The time of the computer workload was 75 minutes. Subjects of this experiment were male students in college. Next four kinds of workload were reference workloads to the computer workload. 1) Pedaling a bicycle ergometer 2) Sorting of five different sizes using a coin sorting tester. 3) Adding up two the two-digits decimal numbers printed on a paper and writing the answer on the same paper. 4) Daily working without special physical or mental workloads. A time of duration of each workload was 75 minutes. One subject did all workloads at the similar time on different days. Heart rate variability and impulse response function were calculated from the R-R intervals time series.
3. Analysis The electrocardiograms were digitized at 1,000 samples/sec by an MD converter and all R-R intervals were measured with a fast peak detection algorithm at an accuracy of lms. We used an autoregressive model to estimate the power spectral densities function(P(f)) of the R-R interval time series. The model order was chosen as the one that minimized Akaike's final prediction error of ignore of merit[4]. (1)
(t) = ~1 a(k) R (t - k) + Z(t) where /~ (t) = R(t)-
Z R(i)/n p(D = o 2 l1 - ,~ a(k)exp(-jZ~fk)1-2 k=l i=l
(-0.5 < f < 0.5)
(2)
where t7 2 is the variance of the residual Z(t) in equation (1) and f is the frequency. The autoregressive model parameters {a(k)lk=l,2,...,p} relate to the order of the mental workload. The dynamics of the cardiovascular system to a shock is called an impulse response function h(x), which can be calculated by using equation (4), under an initial condition (3). /~(-p) = R(-p + 1) = ... =/~(-1) = 0 R(O) = 1 and Z(t) = 0 ^
h(,) = 0
( , < 0)
h0:) - 1
(1; = 0)
h@)
k~=la(k)h(k-1;)
(1: > 0)
(3)
(4)
It is considered that the response shows the regulation mechanism of cardiovascular system to an impulse. The regulated time, the time appeared the first peak and the value of the first peak were calculated from the impulse response function. We calculated the heart rate and coefficients of variation of R-R intervals (CVr-r) during the workload every 3 minute.
4. Results The workloads of adding up work with a computer and 4 kinds of works were compared. Fig. 1 shows mean heart rates in work time 78 minutes. It shows the average of 3 experiments to carry out each work in one subject. Workloads were the adding up the 2 numbers displayed on the screen of computer, adding up 2 numbers primed paper, coin sorting work, exercise load by bicycle ergometer and spending time in room freely. The abscissa shows progress time
767 of work start. Mean heart rates becomes very large in exercise load by bicycle ergometer. In other four kinds of workloads, mean heart rates is the lowest when does not give load. Level of degree the workloads is the smallest when does not give load. On adding up work with a computer, mean heart rates is higher than the coin sorting work that is the simplicity work or spending free time without workloads. It is considered that workloads degree is large in the work of the adding up 2 numbers displayed the screen of computer. But the mean heart rates decrease with time progress by degrees. Heart rates decrease with time progress is not found in the work of adding up 2 numbers printed paper. Fig.2 shows change the CVr-r for in 3 minutes interval in work time 78 minutes. It shows an average of 3 experiments it as Fig. 1. The abscissa shows progress time of work start. Works were it to a case of Fig. 1. It shows the value different from exercise load by bicycle ergometer and other 4 kinds of works. The change of CVr-r in the work with computer is similar to the case spending free time. It is difficult to concentrate to work and to continue the strain in a case of work with computers in comparison of a case of adding up numbers printed on paper and adding up numbers displayed computer screen, It is compared the work efficiency with adding up work with a computer and without computers. The performance of work in 78 minutes is 1120 answers, 970 answers, and 1552 answers in case of using computer. In case of not using computers, it is 1196 answers, 1080 answers, and 1211 answers. A correct rate is 95.2 %, 9 6 . 1 % and 96.4% in case of using computer. In case of not using computers, it is 99.6% ,99.7% and 99.6%. These show that concentration can't continue in work with computers.
130I/ 120
16 -
---" _
-
-,'"" 12
.=- 110 ed
-I I !
v
100
t.
d. 8 -~" 4
81) 70' 0
--'. ~ ~ . 15
3O
45
60
75
time(rain) computer calc sort ergo dai ly
Fig.1 Change of the mean heart rate measured by 3 minutes intervals. These values were averages of three experiments by one subject.
'"
I
".~.-.~.~--,-.~...~:.-:-
.... ' .... ' .... ' .... ' .... '
5
1
I
0
1
. . . .
~
~
~
I , . 1 , I
15
~
. . . .
30
I
45
. . . .
I
. . . .
60
I
75
time,in) c0mputercalcs0rterg0daily
Fig.2 Change of the CVr-r measured by 3 minutes intervals. These values were averages of three experiments by one subject.
768 The workloads with computer and other works were compared by the impulse response function that is an evaluation parameter of cardiovascular system. Fig.3 is the result compared impulse response functions in 5 kinds of work condition described in 2. The autoregressive model was applied to the R-R interval time series for 3 minutes atter 30 minutes from the work start. It is assumed that a virtual impulse was added to cardiovascular system in 0 of abscissa. The abscissa shows time progress by heart beats. The ordinate has R-R interval of moment when stimulation was given to with 1 and shows a change of R-R intervals in relative value. What become value 0 show that it did not receive influence of impulse stimulation. In comparison of other cases and on computer work, some different characteristics were appeared. It is not seen it 4 beats rhythm. The regulated time is short. The overshoot is found. The 4 beats rhythm is thought about with a period by respiration arrhythmia. In computer work, this rhythm and 10 beats rhythm by blood pressure arrhythmia not being found, and it is thought that the rhythms were disappeared by the mental and physical noise. Furthermore, shortness of the regulated time is regarded that cardiovascular system reacts to outside stimulation sensitively. These show that mental workloads degree is large. 26 impulse response functions were derived from the one work whose work time 78 minutes devided each 3 minutes. The average of these 26 impulse response functions is shown Fig. 4. These functions are similar to the function described Fig.3. Impulse response function had a similar characteristic through 78 minutes. The outstanding characteristic in computer work is that the disappearance of the rhythm, the existence of the overshoot and the shortness of the regulated time. It shows that the mental workload with computer is different from other mental work. Next, we describe experimental results for a change of mental workload for continuation time of computer work degree. Fig.5 shows a change of mean heartrates in computer work. Mean heartrates measured 3 minutes intervals. The solid line shows adding up work using a computer described in 2. The dotted line shows word-processing work with word processor software of personal computer. Adding work and word processing work were done by different subjects. Accordingly, the difference of mean heartrates in the two works is the individual difference. Among 3 times adding up work, heart rates did decrease with time progress in heart rate high 2 cases. Fig.6 shows the change of CVr-r. It measured in 3 minutes interval in the same experiment in Fig.5. The solid line is adding up work. The dotted line is word processing work. Change period is about 10-20 minutes. It is thought that this is equivalent with the continuance time of concentration or strain. Fig.7 shows the change of the regulated time of impulse response function in 3 minutes interval. The regulated time is defined as the time that a value of impulse response function dose not exceed +0.1 and-0.1. The average of the work with a computer in 3 times experiments is shown. The abscissa represents work progress time. The ordinate is the regulated time. The conspicuous change for progress time does not show in 78 minutes work. The conspicuous change of mental workload that appeared in heart rate variability was not shown continuation time of computer work in 75-90 minutes from results of Fig.5 to Fig.7.
5.Summary In this study, we analyzed mental workload with computer using heart rate variability. We compared mental workload concerning computer to other mental works and physical work load. We evaluated it by heart rate variability parameters and the impulse response function
769
1.6
~
-
1.6r-
1.2
1.2
0.8
.
0.8
I t
1"J
0.4
~.t~.::. ~\ x • • m,:,';
.~
', --.,..
~
. - - - -.
\x.,,'~. ,;-,.:... :7 ....... "-.-'-.'-.".
0.4
o|
..-,4
............................... 0 5 I0 15 20 25 30 time (beats)
-0.4
_ 0 . 4 ,
14
90
12
t-.
70
~n v
, .........
15
,
20
I
I
10
~ so
v
, ....
10
.....
25
30
Fig.4 Averages of the impulse response functions deriving from five kinds of works.
100
60
, ....
5
computer ca 1c sort ergo dai 1y
Fig.3 Impulse response functions deriving from the five kinds of the workloads. It is derived from the R-R interval time series for 3 minutes at 30 minutes after start of the work.
~
....
time (beats)
computereal c s0rt ergo dai 1y
..
0
8
-,I/
,, •
~,~tv
F"
~ .;~ V\,'.I
6 ~'~
--
4-
......................
0
15
30
45
60
,
,
75
90
time(min) calewp Fig.5 Change of the mean heartrate during computer works. Computer works are adding up work and word processing work.
I
0
i , ,
,I
. . . .
15
I
. . . .
I,
,,,
I
. . . .
30 45 60 time(rain)
I
. . . .
75
1
90
calcwp Fig6 Change of the CVr-r during computer works Computer works are adding up work and word processing work
770
30
10
0
" ' l ' ' l l l
0
15
....
l | l l | l | , , | l
45 ti~ef~itO
30
60
75
Fig.7 Change of the regulated time of the impulse response functions deriving from the R-R intervals time series measured in computer work. Computer work is the adding up work. The average of the three experiments are plotted.
estimated from the time series of R-R intervals. In computer working time, cardiovascular system responded to mental and physical stress. These stresses are estimated from the impulse response function. The concentration or strain cannot be continuous in computer work. And, continuation time of computer work was degree for less than 90 minutes, the conspicuous change during mental workload did not appear to heart rate variability. From the CVr-r and the work performance, the extent of stress and concentration to the work were similar, but work efficiency was not so high. Mental workload with computer distinguished from another mental workload using impulse response functions. Impulse response function deriving from the R-R intervals in computer work showed that the cardiovascular system is very sensitive against the stimulation during working. REFERENCES
1.G.Mulder and L.J.M.Mulder : Information processing and cardiovascular control, Psychophysiology, 18, 4, 392-402 (1981) 2.A.Murata : Experimental discussion on measurement of mental workload -Evaluation of mental workload by HRV measures, IEICE Trans. of Fundamentals of Electronics, Communications and Computer Sciences, E77-A, 2, 409-416 (1994) 3.National Research Council : Video Displays, Work, and Vision, National Academy Press(1983) 4.Akaike H.: Fitting autoregressive models for prediction, Ann. Stat. Math, 21, 234-247 (1969)
Symbiosisof Human and Artifact Y. Anzai, K. Ogawaand H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
771
A s s e s s m e n t of m e n t a l workload based on a model of autonomic regulations on the cardiovascular system* Mieko Ohsuga, Hiromi Terashita, Futomi Shimono and Mamiko Toda Mitsubishi Electric Corp. Central Research Lab. 8-1-1, Tsukaguchi-Honmachi, Amagasaki, Hyogo, 662, JAPAN 1. I N T R O D U C T I O N Quantitative assessment of mental workload (MWL) is helpful in improving the usability of computer systems, the working environment, and work schedule management. We have been investigating the changes in autonomic indices such as heart rate, blood pressure and their variabilities induced by MWL. We found that the multidimensional use of these indices is useful to assess MWL. However, most of the indices are multiply regulated by the autonomic nervous system and influenced by other indices through feedback loops. So if careful considerations are not made, their changes which are not induced directly by MWL would be misunderstood. For that reason, we introduced a physiological model of autonomic regulations on the cardiovascular and respiratory systems to obtain new measures more directly related to MWL, such as sympathetic tone, vagal tone and baroreflex gains. 2. M O D E L S E L E C T I O N A N D S I M U L A T I O N
Referring to the qualitative model by Mulder, G [1] shown in Figure 1, we surveyed various quantitative models in literature. After some investigation, the model proposed by Luczak, H [2] was selected since the model describes most of the blocks and paths in the qualitative model. The simulations were executed on the engineering workstation using the general purpose simulation tool (MATLAB+SIMULINK). The fifth order Runge-Kutta method was used. We investigated the simplified version of Luczak's model (Figure 2) and the model with some modifications, which we made by comparing the model performance to our own experimental data.
*The present study is supported by MITI's Project on "Human m e a s u r e m e n t application technology".
sensory
772
system I I respirat°ry
INTRATHORACIC PRESSURE i-
RESPIRATORY RHYTHM
I
respiratory control center
T
I
+TVENOUS
| VOLUME
VASOMOTOR TONE ~
/TOTALREsISTANcEPERIPHERAL
Ihigher centerl I,:ariovacular.---..---.-.---,--, SYMPATHETICTONE ~ ! i control PARASYMPATHETICTONE ~L ~ I center
i
.,
SYMPATHETICTONE I bar°recept°rsl
STROKE I "-" ' VOLUME ICARDIAC . ,~I~.IOUTPUT , , ~
heart~ HEARTRATE BLOOD PRESSURE
Figure 1. Qualitative model by Mulder,G(1980)[1].
d tpS I J ! I "lt7"~l "!/I I
PR l+t4S)(2-tsS
Figure 2. Block diagram of the model revised from Luczak,H(1975)[2].
3. DATA ACQUISITION Eighty male adults aged from 22 to 53 years participated in the experiment. They were screened for good health and no medication. Instantaneous heart rate (HR) was derived from an electrocardiogram and mean arterial blood pressure (MBP) was measured beat by beat at the left thumb using FINAPRES. And impedance cardiogram was also used to estimate stroke volume (SV). Changes in respiratory volume (Resp) was measured using a mask.
773 Experimental conditions were controlled respiration (fixed rate), several maneuvers for autonomic functions (holding breath, standing up, Valsalva, cold pressor) and mental tasks (mental arithmetic, color matching, tracking). The physiological indices obtained beat by beat were converted into equi-interval data by spline interpolation. Furthermore, the magnitude of the respiratory frequency component (RF; respiratory frequency +_0.05Hz) and mid frequency component (MF; 0.078-0.137Hz) were quantified via spectral analysis using FFT.
4. MODEL PERFORMANCE COMPARED TO EXPERIMENTAL DATA Model performance was compared to the experimental data in the following four aspects.
4.1 Mid frequency component(MF) Experimental data showed the instability in MF appearance and large individual differences in both magnitude and stability. As for the model, MF's were able to be simulated by the oscillations generated in the feedback loops and the changes in the values of two feedback gains explained the instability in MF's. 4.2 Respiratory frequency component(RF) The magnitudes of RF's in variabilities of HR, MBP, SV and respiration were investigated. Log RF's were plotted vs. log respiratory frequency for the nine conditions of fixed rate respiration (5,6,10,12,15,20,24,30cpm). In the experiment, respiratory volume was not controlled but the subjects were instructed to make it constant at each condition. The magnitudes of RF's in variabilities of HR, MBP, and respiratory volume decreased at a constant slope in the log-log space at respiration rates faster than 8cpm. The median value of the slope normalized by RF of Resp was -0.6 for HR variability and-1.0 for that of MBP. The result for HR was comparable to those in previous literature [3]. The RF of SV variability showed no frequency dependent phenomenon. In the original model, the sinusoidal inputs with constant amplitude were added to simulate the respiratory influences on the intrathoracic pressure. In this condition, the slope for RF in HR variability was -0.8 and that for MBP was -3.0. The former was adequate. However, the latter was too steep compared to the experimental data. In order to improve the frequency characteristics of RF in MBP variability, the respiratory input to SV was added and the transfer function of the arterial system was adjusted with care not to change the closed loop delay in order to remain the proper MF frequency. After these modifications, the slope of RF in MBP variability also showed the value about -1. The phase characteristics of RF in HR and MBP variability to respiration were similar to the experimental data in the both models.
774
4.3 R e s p o n s e s to m a n e u v e r s for a u t o n o m i c f u n c t i o n s Time series in HR, MBP, SV and respiration were investigated for the conditions of holding breath, standing up (ST), Valsalva and cold pressor (CP). Using the experimental data, the global mean at each index over time was calculated to capture the characteristics of the response for each condition. In the model, the simulated changes in respiration, SV (for ST and Valsalva) or total peripheral resistance (only for CP) were introduced. The simulation results of HR and MBP showed similar time response compared to the experimental data and changes in the feedback gains produced variations which explained the individual differences such as the MF increase or luck of increase in the standing condition. Figure 3 shows some examples of the results. Simulated Changes in Resp.(mmHg), SV(ml), TPR(mmHg/I) a)l°° /
.
.
.
.
I
050, ~ .~ ., ,. .
~
Results for MBP(mmHg) and HR(bpm) 12o[
"
110L~ 1001 ~"
~
80 t H R "
b)1-5 . .................
'
,
"
"
olHR ,
,
, t
li
, ",O,p",
1
~oo~ ' ~ , ' , , ~ ~-c----~ ~"p,."Yr'~ ('x ~\] 90
' '\]
8O
d)1_~58o ~;i~;;~;.i/~]~ 12o~ :jJ~~~~~~;~Ai:
100110
-501 0
. . 40. . 60 20
90 ~0 800 80 (sec)
. . . . 20
40
60
80 (sec)
Figure 3. Examples of the simulation results for holding breath(a), Valsalva(b), standing up(c) and cold pressor(d) 4.4 R e s p o n s e s to m e n t a l w o r k l o a d Differences in mean values of the indices during the mental tasks compared to the previous baseline periods were investigated. In the experimental data, the task dependent characteristic patterns were observed. For example, HR acceleration and MBP elevation were especially large under mental arithmetic, and the acceleration in the respiration rate and the decrease in its amplitude were dominant under color matching and tracking tasks, which also were found to lead to the reduction in RF's of HR and MBP variabilities. For all tasks, although changes in MF of MBP variability were inconsistent, MF of HR variability
775 generally decreased, which suggested the reduction of the baroreflex sensitivity. We also found that the individual differences in the changes of RF's largely depended on the respiratory changes. The simulation variables were the respiratory frequency and amplitude, two feedback gains (k3, k7) and the sympathetic and vagal tone( A Fs, A Fv). More than one hundred combinations of their values were tested and the results showed various patterns of changes in HR, MBP and their RF's and MF's, which explained the divergence of the experimental observations. 5. ESTIMATION OF AUTONOMIC PARAMETERS We also introduced a method using neural networks for the estimation of the autonomic parameters such as feedback gains (k3, k7 in Figure 2) and autonomic tone (A Fv, A Fs in Figure 2). They are considered to be directly influenced by mental workload but unable to be measured directly. Each network was composed of six inputs, two middle layers with 15 and 10 components respectively, and four outputs (Figure 4). The inputs were mean heart rate, mean blood pressure, RF and MF of heart rate variability and those of blood pressure variability. The outputs represented the above mentioned model parameters. During the training phase, the network weights were adjusted using the back propagation method. The training data sets were obtained by simulating every combination of five levels of the four model parameters. Further, the training was executed for each combination of the several values of stroke volume, respiratory frequency and respiratory amplitude creating an array of networks. Testing the networks with simulation data, the estimated levels of parameters agreed with the targets for more than three-fourths of the data sets used for training and more than one half of those not used for training. In the application phase, the six indices for inputs of the network and the three indices for the selection of the network will be obtained from an actual human subject by physiological measurement and the selected network weights will be utilized to estimate the parameters for him. Training phase
..............................................................................
! • • - ~ meanHR ! K3-~| ~-~HRV RF I K7 --~| ~-I~HRV-MF A F V ~ M o d e l ~-~meanl~lBP !! FS-~i ~MBPV ~ !
~ ( nputj,,7.. ~..~O~utputs ~ - ~Of~ ~ _ Rx
,r- ~ ' ~ ' O ~ r . I~, ~ . ' , . ~ . O I r • ~W ' ~[ ~ " - ~ - o l l • A; Ill ~ ' , . . . o ~ k " #
~
~
-~A FS
- ' ~ R V ' - M F j II "r" ,--~mean'MBP- II Selection of networks /I~-'~MBP--i---~MBPV RF ~l I / ~BPV-MF ..a I A --~ICG -~SV / \---~ Resp ~__~,,~ RSPF Application
I
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
K3 K7
÷ ~
~-~AF'~
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
phase .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Figure 4. The method to estimate autonomic parameters
776 6. D E V E L O P M E N T OF THE REALTIME ESTIMATION SYSTEM
We also developed a personal computer based system which realizes the realtime estimation of the autonomic parameters by the proposed method. The developed system also collects subjective assessment data using voice input for which the subject is prompted when some typical patterns of the estimated parameters are detected. The timing signals of the detection are also written on two video tapes which record the subject and the work system. They provide information on the work situation, the subject's work behavior and also his facial expression. The environmental parameters such as temperature, humidity, air flow, noise level, and lighting are also measured and recorded. We can investigate the relations among the subject's physiological state, the subjective state, and the work parameters after the experiments. For subject
For experimenter
Figure 5. The developed system 7. CONCLUSION It is concluded that the performance of the introduced model of the cardiovascular system was satisfactory, and further study using the model would be beneficial to understanding the autonomic responses to MWL. The proposed method to estimate the autonomic parameters should be evaluated by further experiments. We believe that the developed system will provide a promising research method to investigate how work stressors cause the physiological and subjective strain and how we can control the level of stress in the work place. REFERENCES
1. G. Mulder, The heart of mental effort, Thesis. University of Groningen, 1980. 2. H. Luczak and F. Raschke, Biol. Cybern., 18, (1975) 1. 3. J.A. Hirsch and B. Bishop, Am. J. Physiol., 241, (1981) H620.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
777
E x p e r i m e n t a l Study on R - R Intervals of Heart R a t e by Wavelet Analysis S. Kishino*, M. Katoh* and Y. Hayashi* *Department of Industrial Engineering, Musashi Institute of Technology 1-28-1 Tamazutsumi, Setagaya-ku, Tokyo 158, Japan
1. I N T R O D U C T I O N The analysis on heart rate variability has been widely studied as representative indexes for stress visualization [11, [2], [31, [41. R-R interval (RRI) time series of heart rate are one of these indexes and expected for the measure of the short term mental workload. Fourier analysis and auto-regressive model have been applied to the spectral analysis on RRIs [51, [6]. Compared with Fourier analysis, wavelet analysis recently developed has the advantage of higher resolution in low band frequency and resolution in time domain. So it was applied for various fields such as signal understanding, acoustic-phonetic recognition, and obtained some excellent results [7]. In this paper, the actual RRIs data analyzed by using wavelet-packet method.
2. E X P E R I M E N T A T I O N
For the model of mental workload, an experiment was conducted for a male student with following three conditions: • REST : Masking eyes and sitting in a comfortable position on a chair in a laboratory. Lying his face on the table is admissible but sleeping is not admissible. • CALCulating : Adding two digits printed on the paper as fast as he can. He is also required the accuracy. This situation is expected to cause mental workloaded circumstances. • MOTionless : Masking eyes and sitting on a chair in a laboratory. He is forced to keep his first posture. The irritation can be caused by this situation.
778
Measurements of RRI with electrocardiogram of these conditions were separated in three days. A test subject was asked to control his living circumstances. The temperature, the humidity and the noise level of the laboratory were not actively controlled during the experiment. Before the measurement RRI
of each day, a 15 minutes rest was kept for relaxation.
data was analyzed directly without interpolating to regularize intervals. The first
512 beats were analyzed.
Obtained
i00 ,
sec.
three kinds of RRI
200 ,
300 ,
data are shown
as Fig. I.
400 ,
1.2
500 , REST
--
-
1.0 !
O.8 mean
0.6 1
• 9.52 sec. ,
I 1.2 -
,
,
,
I
I --
I
-
,
I~
CALC.~ J
1.0
0.8 an
0.6
I
8.92 see.
,
,
,
I
I
|
i
I
MOTI, ~
1.2
-
1.0 0.8 0.6 sec.
i00
200
300
400
500
beat Figure 1. Analysed RRI data under three conditions.
3. A N A L Y S I S Wavelet analysis were performed by using a packaged software "Wavelet Packet Laboratory for Windows" [8]. Seventeen wavelet packets (B18, C06, C12, C18, C24, C30, D02,
779
D04, D06, D08, DI0, DI2, DI4, D16, DI8, D20 and V24) and two local trigonometric functions are proposed in this software to transform the signal. B I8 is a Beylkin filter, C06 through C30 are Coifman filters, D02 through D20 are Daubechies filters and V24 is a Vaiyadanathan filter. RRI data was transformed by 17 wavelet packets. The results of wavelet transform are able to display as three dimensional charts. Fig. 2 shows the result of REST transformed by B I8. From the observation of Fig. 2, it is recognized that the peak seemed to the adjusting temperature fluctuations (ATF) appears from the beginning of measurement. But the peak seemed to the blood pressure fluctuations (BPF) is not recognized in the first half of the measurement. Looking at the details of Fig. 2, the 300th spectral diagram of Fig. 2 was shown in Fig. 3. The vertical and horizontal axes of Fig. 3 are amplitude of power spectrum (dB) and frequency (cycle/beat) respectively. e-- ATF <--" BPF
amplitude ao ~ . . . ~ . ~ ~ 20 -
~
~~,,-.j~-~~ ~
-~,.~
~
N
~
~ ~
-..~
~F~4Wliltpq~
~
cycle/uea~
o.45
o.5
400
o
Figure 2. Three dimensional representation of wavelet transforms : REST, B18. In Fig. 3, fluctuations seemed to RSA, BPF and ATF are clearly distinguished. One more fluctuation can be recognized between ATF and BPF. Table 1 to table 3 show the results of transforms. In these tables a the non-numerical fields means that no peaks are recognized for corresponding fluctuations because of the reason written there. Wavelet packets which do not recognize all of three peaks are not listed on these tables. The units of fl'equency and amplitude of these tables are the same as Fig. 3.
780
amplitude f ATF dB
f
BPF
/ asa 0.1
1
I
,
0"01°
l
,
[
0.i
I
i
0.3
I).2
Ill
II
i
0.4
cycle/beat Figure 3. The 300th spectral diagram of Fig. 2. Table 1. Wavelet transforms for REST. wavelet B18
fi'om 0.035
ATF to 0.039
amp. 0.577
from 0.078
BPF to 0.086
amp. 0.465
from 0.320
RSA to 0.328
amp. 0.168
C06 C12 C18
0.035 0.035 0.035
0.043 0.039 0.039
0.231 0.564 0.546
0.082 0.086 0.066
0.086 0.090 0.074
0.205 0.280 0.388
0.320 0.316 0.324
0.324 0.324 0.328
0.119 0.179 0.174
C24
0.035
0.039
0.518
0.078
0.086
0.135
small
C30
0.035
0.039
0.485
0.086
0.090
0.165
small
D02
0.031
0.039
0.590
0.074
0.078
0.293
D04
0.035
0.039
0.311
0.074
0.082
0.147
D06
0.035
0.039
0.572
0.074
0.078
0.328
D08 D10
slope 0.020
D12
0.035
valley 0.667
small
0.309
0.324
0.278
small slope 0.305
0.312
0.264
0.074
0.078
0.275
0.277
0.309
0.239
0.070
0.102
0.174
0.309
0.313
0.187
D14
0.035
0.039
0.547
0.066
0.082
0.443
0.320
0.324
0.119
D18
0.020
0.035
0.639
0.082
0.098
0.380
0.313
0.320
0.102
D20
0.031
0.039
0.694
0.078
0.082
0.378
0.320
0.324
0.143
V24
0.035
0.039
0.598
0.074
0.082
0.444
0.316
0.320
0.134
781 Table 2. Wavelet transforms for CALC. ATF to amp. 0.035 0.621 0.031 0.857 valley
fi'om 0.086 0.074 0.078
BPF to 0.094 0.078 0.082
amp. 0.437 0.506 0.326
0.031 slope
0.440
0.090 0.074
0.094 0.078
0.220 0.371
0.031 slope
0.621
0.094
0.098 slope
0.501
0.031
0.035
0.491
0.086
0.090
0.516
0.020
0.027 small slope 0.063
0.422
slope valley small 0.086 0.102 0.665
wavelet B18 C06 C12
fi'om 0.031 0.027
C18 C24
0.000
C30 D02
0.027
D04 D06 D08 D10 D12 D14 D16 D18 D20 V24
0.000 0.000
0.000
slope 0.031 slope slope 0.031
0.449 0.299
0.607
0.094 0.082 0.082 0.086 0.082
0.098 0.086 0.090 0.090 0.090
0.405 0.486 0.476 0.188 0.580
fi'om 0.328 0.336
RSA to 0.344 0.340 slope
amp. 0.129 0.356
0.316
slope 0.332
0.467
0.332
small 0.336
0.227
slope 0.332 0.336 0.328 0.320 0.328 0.336 0.324
slope 0.336 0.340 0.336 0.328 0.332 0.340 0.328 slope
0.234 0.312 0.364 0.239 0.120 1.185 0.222
Table 3. Wavelet transforms for M OTI. wavelet B18 C06 C12 C18 C24 C30
ATF to amp. small 0.027 0.035 0.217 0.020 0.027 0.148 0.020 0.027 0.165 0.020 0.023 0.161 valley
D02
slope
fi'om
BPF to amp. small valley small slope slope 0.074 0.082 0.216 from
0.074
0.090
0.157 0.164
D06
slope
0.074
0.078
D08 V24
small 0.027
0.074
0.090 0.176 valley
0.023
0.223
from 0.324
RSA to 0.355 small small slope valley valley
amp. 0.116
small valley 0.340
0.344 slope
0.113
782 4. C O N C L U S I O N S From the analysis of REST, the transformed peak seemed to the ATF is distinguished with BPF except D08, D12 and D16. Three peaks seemed to the ATF, BPF and RSA are also distinguished in transform of B18, C06, C12, C18, D02, D10, D14, D18, D20 and V24. For the CALC, many transforms still distinguish the ATF from BPF. The two digits calculation may not cause mental workloads for a college student. Comparing the results of CALC with the REST for each transform, the ATF peak moves to the left (longer cycle) and has relatively large value. The BPF peak has not remarkable tendency compared to the ATF peak. Perhaps, the 4th fluctuation between ATF and BPF (see Fig. 3) has some effect on these peaks. The RSA peak moves right (shorter cycle) and has relatively small value. For the MOTI, all peaks have less amplitude and large widths, so RRI fluctuations seems irregular. The similar results described above are obtained from other student's data. Wavelet-packet method seems to be effective for RRIs analysis. It is necessary to characterize the practical meanings of each packet for RRI analysis, conducting further experiments with severe conditions.
REFERENCES
1. Murata A., Measurement of mental workload by heart rate variability indexes, The Japanese Journal of Ergonomics, 28(1992)91 2. S. Akselrod et al., Power spectrum analysis of heart rate fluctuation: A quantitative probe of beat-to-beat cardiovascular control, Science 213(1981)220 3. G. Mulder and W. R. E. H. Mulder, Mental load and the nleasurement of heart rate variability, Ergonomics,16(1973)61 4. W. Rohmert et al., Heart rate variability and workload measurement, Ergonomics, 16(1973)33 5. B. W. Hyndman and J. R. Gregory, Spectral analysis of sinus arrhythmia during mental loading, Ergonomics, 18(1975)255 6. N. Egelund, Spectral analysis of heart rate variability as an indicator of driver fatigue, Ergonomics, 25(1982)663 7. G. Kaiser, A friendly guide to wavelets, Birkh/iuser, Boston, 1994 8. Wavelets and Adapted Waveform Analysis : Ronald R. Coifman and M. Victor Wickerhauser, A K Peters, Ltd., 1994
IV.8 Physiological Measurements 2
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
785
CFF Values for Stress Caused by VDT Work and Relationship Among Analysis of Uric Properties MASAHARU TAKEDAO), YOSHIO HAYASHI(2), KAORU SUZUKIO) (1)AND(2) Department of Industrial Engineering, Musashi Institute of Technology, l-28-1 Tamazutsumi Setagaya-ku Tokyo, Japan
(3) Information Processing Course, Tokyo Vocational Training College, 2-32-1, Ogawanishimachi, Kodaira City, Tokyo, Japan
Introduction The VDT occupation has been infiltrated into all kinds of human activities. Especially in industry, the WHO and ILO prescrive various criteria in view of labor hygiene problems in order to avoid occupational diseases, which belong to the ophthalmology. The Ministry of Labor in Japan supervises administratively in order to make enterprises set up an inside organization managing the labor hygiene, establish the criteria for the management of the labor hygiene concerning the VDT occupation, and enforce the health management mainly by the measure of the occupational environment and industrial doctors. These labor hygiene activities consist of prevention, diagonasis and necessary measures. However, they have not been proposed yet, because it is difficult for workers to measure and judge objectively their daily fatigue changes. This study reveals the fact that the CFF (Critical Flicker Fusion Frequency) makes the judgement quicker and easier. Recently, however, lwasaki x) have suggested that changes of the CFF values can judge the effectiveness in measuring only asthenopia. It seems to be difficult to separate asthenopia and fatigue centralis. If the CFF values can judge only asthenopia, we cannot but change the evaluation of the CFF values. Therefore, we conducted both occupational experiments on the CRT and on paper with the occupation of continuous one-digit additions (Kreapelin's test) as the work load. Then we analyzed catechoramine as the measurement of the CFF value and as the evaluation of physiological stress.
Purpose These experiments use the occupation of continuous one-digit addition (Kreapelin's test) as the work load and occupational conditions are divided into one on the CRT, which is considered to be likely to result in asthenopia, and one on paper, which is considered to be unlikely to result in it, changes of the CFF values and catecholamine are considered the index of fatigue centralis. Then we suppose that the CFF values are effective as the index including fatigue centralis as well as asthenopia if there is no difference between two conditions. In addition, if the end effects of work performance per unit (one minute) of the work load show the increase of the occupation resulting from l~eration feeling from the work load, we regard it as the relief of fatigue centralis. We verify these supposition by means of the experiments.
786
Method of experiments 1) Equipment and urinalysis a) Flicker apparatus and stimulation method Evaluation of CFF value is largely affected by the factors such as individual differences, stimulation method rate of increase or decrease in flicker frequencies, qualification and skillfulness of the subjects, time required for measurement and time of measurement. Even in the same subject, CFF values change according to the differences in inner coordination x). Thus, CFF values are not often appropriate as evaluation index for fatigue change. The present method to measure CFF was for the subjects to operate an attenuator of the flicker apparatus. In order that the subjects might discriminate the CFF values, operating angles of the attenuator was set rough, and was controlled by a computer near at 35Hz blinking frequency, to enable fine adjustment of the attenuator.
b) Urinalysis Following matters were taken care of, during the collection of urine. • Individual subjects were exposed to work load for a constant period (including intervals and time). Urines were collected at constant intervals, after the start of loading of work lords. • It is possible to estimate concentration of catecholamines in urines, since the concentration of urinary catecholamines can be corrected by urinary creatinine and specific gravity, even if the collected urines are dilute or concentrated ones. Therefore, the time from the start to the end of loading of work load was made constant, and the time required for the test was shortened as far as possible. • Considering the half decay time of catecholamines in urines, it was desirable that the urine would be collected within one minute after cessation of loading of work loads. But, it was impossible to collect urine within one minute. Thus, the intervals from the start of collection of urine were made constant. • Diuretic may be available for shortening the collection time of urine. But the use of diuretic was avoided, because some diuretic often induce neurological side effects. • Conditions related with five senses (mainly, visual power, color sense, sense of color brightness, sense of smell, sense of hearing, sense of touch, sense of taste, sensation to temperature and humidity) and factors affecting to working postures were made constant as far as possible. • Collected urines were treated by a series of treatments, and were stored in a cooling box including dry ice. Thus, the urine samples were quickly frozen, and were maintained under frozen conditions as far as possible. Those urines were analyzed by specialists, using modern pioneering techniques.
2) Subjects Eight male adults, who were consistent with the following conditions were selected as subjects. • Normal adults with normal sense of sight, between 20-22 year old • Adults whose health-conditions were checked
787
• Person who can type keys of computer board without seeing a
START
board • Adults who are good at Kraepelin's test • Adults who are accustomed
Measurement of the CFF Collecting urine for urinalysis outside the exoerimental room
to CFF measurement by repeated
I P Collect urine Note 1) Urine is collected at the 1st. 6th and 12th time. qote 2"1 The CFF is measured after
performance n=ll? NO
3) Experimental method Kraepelin's test was selected
Kraepeling test for 10 minutes
case A (VDT)
as work load due to the following
n=12?
Case B (paper) YES
reasons. • W o r k i n g p r o c e d u r e s are easy to understand • Work performance is easily quantitated
with
work
Fig. 1 Flowchart of the work load experiment
amounts fractionated per minute, and with error amounts • Degree of difficulty is equivalent during each time (minute) • Master of work performance is easy • Stress is revoked due to monotonous work (To intensify the function of stressor, unit price per a correct answer for the work was set, and high marks were given to the correct sensors. It is conceivable that eagerness to work may be stimulated by serving high marks.) • Working process can be depicted as working curve (Initial effects, intermediate, stable effects, terminal effects) • Characters of the subject can be classified into a giving-best type or a normal type. Experimental procedures are shown in Fig.1. The subjects of Case A participated in work to exhibit a number in one figures on CRT, total adjacent figures, and input a number at one decimals by ten keys. The figures on a screen were changed every one minute. The work was performed for 10 minutes, and repeated ten times. The subject of Case B did the same procedures on a paper instead of using a TV (a paper corresponds to a screen of TV).
Results Work performance of the subjects in Case A and Case B was expressed as average work amounts per person per working unit (10 min's working). The results are shown in Fig.2, with 95 % confidence interval
788
value.
........................................................................................
1.00
The work amounts of the subjects in
0.98
Case B were increased as much as 20 %
0.96
n~-8
0.94
higher than those in Case A, showing o
that the working procedure on a paper is
0.92
CRT
o 0.90 tm 0.88
easier than on CRT. Since the subjects in Case A and Case B were accustomed to
(,.)
the working procedures during initial 20
0.86 I
I
I
I
8
9
10
11
0.84
minutes, the work amounts increased ex-
1
tensively. Urination after the 5th time of
sam work
work enabled the rest of the subjects, re-
2
3
4
5
6
7
12 ARe,
Time of measuring (Times)
work
Fig.2 Average change rate of CFF obtained
suiting in the increase in work amounts
by work on CRT or paper
at the 6th time of work in both Case A and Case B, by about 4%. The changes in CFF values which correspond to the work performance were calculated: CFF change rate = CFF values at Xth time of work--CFF values at the 1st time of work The values are shown in Fig.3, with 95% confidence interval value. CFF values at the 6th time were decreased by 8.6% in Case A (the subjects involved in CRT operation) compared with those at the 1st time. CFF values at the 9th time were decreased by 10.1% in Case A, compared with those at the 1st time, and showed the Showest. Slight recovery of CFF values was indicated Upper limit I
1000
95% confidence interval
Upper limit'~
M~an? C R T
M o ~ on a p a p e r
Lower limitX
i
i
n=8
Lower limit~
900
.
i
!
i
i
.
.
.
,
.
!
.
.
~r
'.
800
~
:
i
i
!
-
!
i
i
:
700 600 500 400 1
2
3
4
5
6
7
8
Time of measuring (times) Fig.3 Work amounts in the experiment using CRT and paper
9
10
789
at the resting time between the works. CFF values at the 6th time were also decreased by 5.1% in Case B (the subjects involved in operation on a paper), compared with those at the 1st time. Slight recovery of CFF values was also indicated in Case B at the resting time between the works. The curve of CFF in Case A was different of that in Case B, i.e., the decrease rate of CFF in Case B was slower than that in Case A. But, both curves closed to each others at the bottom values of CFF. The correlation of CFF values with work amounts and catecholamines were shown in Fig.4 for Case A, and in Fig.5 for Case B.
The changes in CFF values, noradrenaline and dopamine exhibited similar trends. Since the subjects were probably exited before work, due to eagerness to operate it was conceivable that secretion of noradrenaline and dopamine (which distribute in sympathetic nerves and brain, and act as nerve transducing n=8
*~ 36 34 33
"~-~... d
"l 750
e J
1st
6th
CFF ~
J
650 ~ 1 600 .~ ~ 550 121h ~
33
250 i 200 150 100 ,
,
6th
o~o
12th
Adrenaline I 2500 2000
~,.~. 36 35 ~ 34 33
1000 ~ 500 1st
I-'-
CFF ~"Noradrenaline I
30 j~ 25 20
6th ~12th
CFF ~
38
350 "~ 300 ,~,
1st
1st ' I~
400 ~,
~,,.,~.
~ i
34
Work mounts
38
36 35 34 33
** 36
6th
12th
CFr-'-Oopammo
I
Fig.4 Analysis catecholamines ( test using a CRT )
O1
38
37 35 34 33
o,.-
38
850 800
~
750 700 650
36
' ' 550 1st 6th 12~ [-4t-CFF--*- Work amounts
33
~"~"~i~,.,.~,~
36
250 200 150
35
i
1st
I
i
6th
12111
CFF ~ - Adrenaline I 2500 1500 1000
~36
"~ 35
oo
33
1~
~ 6~
~ 12~
[- ~ CFF --~ N o - - i n ©
0 ]
=i
35
38
~
n=8
5oo~ 33
1st
i
6th
i
121h
0
l---cFr---r~~o ]
Fig.5 Analysis catecholamines ( test using a paper )
790
substances) was increased. According to gradual adaptation and increase of fatigues, the secretion of these hormones decreased. Adrenaline, which is an adrenal medullary hormone, showed a maximal value in the urines obtained at the termination of 5th work. However, it was inclined to decrease thereafter. In Case B, catecholamines such as adrenaline, noradrenaline, and dopamine were maximally secreted in urines before the start of work, and were inclined to decrease thereafter. The amounts of catecholamines secreted in urines were higher at daytime, and lower during night. Daily amounts of catecholamines of Japanese secreted in urines are estimated to be: l~20[xg/day for urinary adrenaline 30 ~ 120~tg/day for urinary noradrenaline 1 0 0 ~ 1000~tg/day for urinary dopamine , though daily variation of the values is known. In our results, the presumed secreted amounts of catecholamines were: 65~90~tg/day for urinary adrenaline 142~619~tg/day for noradrenaline 1016~4000~g/day for urinary dopamine This results suggests that fatigue change due to work load was reflected to these values.
CONCLUSION From the results in Case A and Case B, the following conclusions were obtained. • Excitement before work due to eagerness to operate and stress factor produced in the subjects might induce extreme increase in secretion of catecholamines of adrenal glands. Thus, the secreted amounts of catecholamines at the initial stage were consistent with the CFF values. • According to the prolongation of work loads, the secretion of catecholamines was slightly decreased, and the change rate of CFF was decreased, showing that fatigue is induced. • Change rate of CFF values was lower in Case A than in Case B. The difference in work load between Case A and Case B corresponds to the difference in operations using a CRT screen and a paper. Thus, asthenopia may be reflected to these differences. But, the minimal CFF values in Case A and Case B closed to each others, i.e., 10.1% in Case A and 7.3% in Case B. Therefore, it may be reasonable that central fatigue is involved in the decrease in CFF values. From these results, CFF values were considered to be useful as an index of asthenopia and central fatigues.
REFERENCE 1) IWASAKI: Changes in CFF Values and that physiological meaning during the experimental visual task with CRT display screen, Japanese Journal of Ergonomics, Vol.26, No.4, 181-184, 1990
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
791
Development of a New Hand-Grasp Measurement System Yoshikazu Seki, Sigeru Sato, Makoto Shimojo, and Akihiko Takahashi Human-Environment System Department, National Institute of Bioscience and Human-Technology 1-1 Higashi, Tsukuba, Ibaraki 305, Japan 1. INTRODUCTION The establishment of methods for designing friendly man-machine interfaces requires the development of techniques for evaluating the ease of use of the devices on which they operate. It is therefore necessary to analyze hand movement, although comprehensive studies of hand movement remain to be organized [1]. One of the reasons for this has been the lack of standardized instrumentation in measuring grasping pressure in finger operation for general purpose applications [2]. Sato et al.[2] developed a device which can measure the distribution of grasping pressure which they called the Sensor Glove. This device enabled the distribution of grasping pressures between fingers and a held object to be measured for generalpurpose applications. Comprehensive studies of hand movement may require, in addition to the distribution of grasping pressure, the position and angles of the hands, angles of each finger joint, and image data in order to know determine hand operation. The Sensor Glove alone is not enough, because it measures only the distribution of grasping pressure. We developed a system for synchronously recording and reproducing data from combined position-angle sensors, joint-angle sensors, video, and the Sensor Glove. Our findings are summarized in the sections which follow. 2. SYSTEM O V E R V I E W
The system(shown in the basic system of Fig. 1) includes the 3 SPACE Isotrak (manufactured by Polhemus Co.), used to measure hand position and angle through magnetic sensors. It has a measurement accuracy of 0.6 cm each in position and 1.5 degree in angle for each position (3 degrees of freedom) and angle (3 degrees of freedom) within a 70 cm radius of the source coil at its center. The Data Glove (manufactured by VPL) is used for measuring finger joint angles through optical fiber sensors with a resolution of approximately 1 degree for a total of 10 joints -- the MP and IP joints of the thumb(the 1st finger) and the MP and PIP joints of the index finger, middle finger, ring finger, and little finger (the 2nd - 5th fingers).
792 The Sensor Glove[2] is used for measuring the distribution of grasping pressures at 81 points in the palm and onthe inner surfaces of fingers using pressure sensors of pressure-sensitive electroconductive rubber (Fig. 2). A video camera (manufactured by Sony: Hi8 NTSC) and a video deck (manufactured by Matsushita: AG7355 SVHS) are used for Image recording in 30 frame/s. Data obtained by 3 SPACE, Data Glove, and Sensor Glove -- i.e., measured data -is sent to a computer (NEC: PC9821 AP2), temporarily stored on RAM disk, then collectively stored on a magneto-optical (MO) disk. The sampling rate is about 100 ms and the maximum continuous recording time of the system is about 7 minutes in the case that the RAM disk is I M byte. An RS232C (9600 baud) interfaces the 3 SPACE, Data Glove, and computer. Thirty-two-channel DIO is used as control signals and a 32-channel A/D converter is used in data acquisition. Image data from the video camera is sent to the video deck and recorded on video tape. An RS232C (9600 baud) interfaces between the video deck and computer to controll the video system. The system is capable of exchanging control signals through RS232C with another experimental system (shown in the application system of Fig. 1), and can also be used in experiments requiring an instrumentation and control system for the object being analyzed, for example, in the evaluation of the feeling in automobile lever, handle, and door knob operation. The software source program is written in Turbo C (Borland) and the execution file runs under Micro- Soft _BA__SLO__LS__y_ST_E__M)---~
(MS) DOS.
COMPUTER - -
mo
! ~
~ ~11
SENSORGLOVE ~ v ~ '
ch#l [
RS232C0 ~ | ch#2I~
3SPACE
!Iiiiiii;ii;iiii!iiili •
==~,' APPLICATIONSYSTEM, /
INTERFACE ~
,!...................
EXPERIMENTAL
....
Fig. 1 Systemdiagram.
;iiiiii!iii!ili!iiii iN
:~igi!ii:igi:iiiii!i.:ii!.:.:i:.i.:#iigii~igii:i.:i.i:i:.:i:. ~:~!", .:,ii',',i',iiiiiiiiiiiliiiiiiiii',iiiii',ii!iiii',iiiii~ • N :!ii!i:ii!:!i:i!i::i!i!!!::!Eiiiiiii::i!i!ii' i::!il :!!!iiii.~::i~ ""::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: i~!,i"~i!i!i!ii!!::!:ii!!ii!!!!!i!~!i!i!iii!ii!i~!i!!!i~ ~!!!i!!!!!.~.
iiiiiiiiiiiiiiiiiiiiiiiiiiiii!iiiiiiiii!i!iii! /
Fig. 2
The Sensor Glove.
793 3. SYSTEM FUNCTION The system has the following features: (1) Distribution of hand grasping pressure, position, and angle. These can be recorded synchronously with angles of each finger and image data on operating conditions(Recording Mode). (2) Data can be reproduced synchronously, in real time, on monitors (Playing mode). (3) The system can be used in combination with other experimental systems. (4) System operation is based on a graphic user Interface (GUI) using a mouse.
3.1. Recording Mode Recording mode is literally a mode for recording experimental data. Measured data from various instruments is recorded simultaneously with the synchronization signal from the video deck, concretely speaking, the image frame number or counter. In playing mode, image data and instrumentation data are reproduced synchronously. Data is temporarily stored on a high-speed-access RAM disk, then transferred and stored on a large-capacity MO disk once measurement is finished.
3.2. Playing Mode In reproduction mode, the system synchronously reproduces all data recorded in real time, on monitors. Image data is displayed on a video monitor and instrumentation data on a computer CRT (basic system, Fig. 1). The distribution of grasping pressure obtained by the Sensor Glove is displayed at left on the CRT display in 8-scale grey scale. Finger joint angles and hand position and angle from the Data Glove and 3 SPACE are displayed on the right side of the display using a simple line frame model simulating the form of a hand. (Fig. 3)
! ~; !f _.E ~C~I~;~:'~ i
I
Fig. 3
( (.~:..:,"
~: ,3(:-~: -":5: ~.!~'!.
. I
One example of the CRT monitor display of the measured data in the playing mode.
794 3.3. On Interfaces A GUI using a mouse serves as the system's user interface. This alleviates the workload on the researcher during experiments. As previously mentioned, multiple types of data are handled simultaneously and stored in different media, including RAM disk, MO disk, and video. If data had to be managed separately during operation, the workload on the researcher would be tremendous. The system was designed, however, so that data management is handled automatically by a computer and the researcher is free to handle data by operating a few GUI buttons.
4. EXAMPLES OF APPLICATION Examples of system applications include the following: In an example of image data (Fig. 4 (a)) and instrumentation data (Fig. 4 (b)) in which a perpendicular wooden stick 34 mm in diameter is grasped, the situation of the hands is confirmed by image data. The bending of finger joints which cannot be explicitly confirmed from image data alone because the fingers are hidden object and the distribution of holding pressures which cannot be obtained from image data can be observed from the instrumentation data displayed on the CRT of the computer. The angles of finger joints and hand position and angle can be observed from different direction by using rotation in display monitoring. Data processing software we developed for the system enables instrumentation data stored on the MO disk to be evaluated in different ways. Changes over time can be seen in the distribution of grasping pressure during the experiment in Fig. 4 (Fig. 5). The area indicated by the black circle shows the magnitude of pressure at that point. 0 ms shows the time a hand touches the operating object. In this experiment, the first contact begins at the base of the 2nd finger and the center of the palm. The next contact occurs at the base of the 2nd finger and the 3rd finger, at points between the MP and PIP joints, then further contact develops at the finger tip and finally extends in the direction of the little finger. i:.~:!~:~:!;!E7:~:~°~!!:~:!~E~!!:~i:~2!~!!~i:!:!~:i~g~:i:~i!:!:i;~:!:~i:~!i~iiiiii~i~ii!ii~i~i~i~i!i!~!i~!~!~!i!!~i!~!~!~!i!~!!!~!~i~!iiiHii~iii~ii~ ............................................... ~ ~ ~ ........~ ~.~.~.~ ii
!!!iii!!!
lli
.!!!!!!!!~!! !!!!!!~!!!!!!!!!!!!!!
ii~i!i,liiii~iii
~'~
|'1"
"1- t
|-I.
-|:1:1:1-
l:l:
:1:[
"F t •
[ii
I-I.
l~
i [ I " :"1-'1"
nU I~~ili'iiiillliiii!,iii,i I'~l~l~l~l~ "
I ~
I~.
:.
lii
i'1'
:':"': ...........
:. :::::::::::::::::::::::::: .'~-~-:--'"-~--~'
~i
_ .o:.~..~.. V:II::.~.:~V':
'
"':.:~
. . . . . - " . . . . . . . . . . . . |:.~. . . . . . . . . .
"~i~: =
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ~
~
r---.:
r ....
~
: .....
~ ........ ~ : ~ ......
~
~ ~:-:-~..'-.".-~:.'..~:~,:.!!!!~:=::~!~!:-:.::.:.!~:::.:~!! , :.:.:.iii:.:.:.:.ii!:.~::.!!!~:.~:.!!ii!ii!!iii~!~iiii!!!!'.-.-~ !:.!:.~:.!i:.~:.!:. :i:.:.:.:.:.i~:.i~i:.::.i:.:.:.i:.i:.i:.::.:.::.~::.:.:.:.i ::.:.~:i:.i:.::.:.:.:.::.:.~.:.~:.:.:.~:.:.i~:.~ ~ilii
(a) Fig. 4
(b)
One example of (a) the visual data displayed on the video monitor, and (b) the measured data displayed on the CRT monitor.
795 The correlation between the M P joint angle of the 2nd finger and these angles for other fingers for the experiment in Fig. 5 (Fig. 6) enables us to observe that a higher degree of synchronization in finger m o v e m e n t with that of the 2nd finger is in the sequence of the 3rd finger, the 4th finger, and the 5th finger. The 1 st and 2nd finger work almost without synchronization. Currently, we are working to establish a new standard for classifying grasping. Several proposals for classification have been made based on previous studies[3] and have tended to concern the researcher's subjective method of classification based on the shape of hands. We are attempting to establish an objective classification standard based on physical grasping data obtained by this system. Currently, we have been trying to establish an a u t o m a t i c classification system which a u t o m a t e s the classification of grasping, through data obtained, based on the six classifications of graspings (Fig. 7).
5. SUMMARY
We have developed a system which can synchronously record and reproduce data of four different types: hand position and angle, individual finger joint angles, image data on hand operation, and the distribution of grasping pressure, combining position and angle sensors, finger joint angle sensors, video, and the Sensor Glove. Since this system can synchronously record multiple physical variables involved in the hand and finger operation (recording mode), synchronous reproduction (playing mode) and/or the multiphase evaluation of data (data processing software), previously not possible, have been made possible. We expect this system to contribute to the establishment of a method for designing comfortable man-machine interfaces. !~i i)~E~i!~EE il
+:::+iEi+iE i+.:~i)
•=-++::_~+::+_.:
o
)............
+...+.- ++..-+--+
• •--"+-:=::++
ms
.
.
.
)...).,0,~.~+.+ .
)
)
)
)
• v+--=-+ -+ ) " •
" I •
• ......
°i ..............
Fig. 5
o)
+.+.. + . . + +...... +........+........+....... +
....- ......
"
.
).4.-- j.m.) . . !---t--t--.w--)
+...).,J,,,,.m.)
......+~+ :
F+--.--+--~-+
. •
O O
....+=.++:
+.............+1 I ""+1 +" .......:..........
•
+ o)
+-4--.Jm.+ , , , r
+
+
' •
)
• . . . . . . . . . .......
+
t
+
+
I •
.~.........
+-..+.m.m.+ • + ) ) + ) .....+++ :-~-+~-~-+++ +~+++.!ii
+ .......+'"1+"' ~. ~ .....
•
F o)
3000ms
°+
,.......................+......
)-4.o. +-m! i. . . . ~___.;.__~__.~ + i ) i +
)-+-:-=~-:-+t
..................+......... 2500ms
:
1500ms
+...+..m,,-m.+
. . . .
+.
• .....
............................ T+I............... 1000ms
• ~............+'1 +I" ~ .......... +
2000ms
)...)o+..+.)
500ms
• 4...~.W.-) . . . . . . . °o
)...)..-+.+,,..)
++.
) "
0
l:::+:: :+:6:1: :+
• ....+~."+ +-'-+~-+-; ....+++) :• +-i+.-++-' ....::++ ""• i )+....................... .... + + -+ ++ + + + +--i-) + + . +• +• ):+ ! o i ......q ........ " . . ° L......./!Y ...... • . -i .....~q.~ -++- .....
• .+.+++..++ii :)
.i.......... . . .;). ............ .
•
+...+.--+...+)
+..... t..... t..... .+---)
,it . . . . . . . . .......
.
e
' ............ + +
"
i o+
+.......
~ .....
e ! ....... .......
3500ms
Time variation of the manipulation force distribution on the hand surface.
796
Correlation coefficient = 0.980
Correlation coefficient - 0.823
~rrelation coefficient - 0.914 x/2
r/2
x/2
~:~ :'%
• .::. . ..... ,..-.• •~,..*...
• -~:a--
..(::'Y" .. ..
e:
;/2
The MP joint angle of the 2nd finger
il
'...- t~'"
:~:,
•
:..~,~-..-., .-" : ~ j ~ -
",.5""
~,." ,;:..,.--.,.
i
~n
0!
0
x/2
The MP joint angle of the 2rid finger
The MP joint angle of the 2rid finger
(b)
(c)
(a) Correlation coefficient - 0.515
=/2
Fig. 6 ,
• i" ...:..=-..,
,., ~-;.-.;..'- ..:.~~
•
.........
~.¢ i~4~" :*: 0
.- . . . . .
=... • .
i "x/2
Correlation between the MP joint angle of the 2nd finger and (a) the MP joint angle of the 3rd finger. (b) the MP joint angle of the 4th finger. (c) the MP joint angle of the 5th finger. (d) the MP joint angle of the 1st finger.
The MP ioint angle of the 2nd finger
(d)
I COMPUTER
II~
f~y,i~ p,..... h] I1[ [ of criterions for classification
'C classificatton results" ' " " ~ i i il
I
Hilt I t
il
!
::
Fig. 7
-~
J
DISC RIM' NATOR ~I'---S-E-NSQ-"---G-LOV-E' -'~
I I
l~q
~
..................... ...........................................
D,T, GLOVE
...............................
The automatic grasping classification system (under development).
REFERENCES 1. Toshihiko Amemiya, An Essay Conceming the Frame for the Finger Manipulation Skill, Japanese j. of Ergonomics, Vol.24 No.6 (1988) 353. 2. Sigeru Sato, Sensor Glove, J. Robotics and Mechatronics, Vol.4 No. 1 (1992) 87. 3. C.L.MacKenzie and T.Iberall, Grasping Hand, Advances in Psychology 104, NorthHolland, Amsterdam, 1994.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
On a S i m p l e M e t h o d Kaoru
to M e a s u r e
797
the
I n t e n s i t y of K e y s t r o k e s
Suzuki
T o k y u P o l y t e c h n i c College, 2-32-1, K o d a i r a - s h i , T o k y o 187 J a p a n
Ogawanishimachi,
1. I N T R O D U C T I O N In r e c e n t years, m u s c u l o s k e l e t a l d i s c o m f o r t r e l a t e d to V D T work has been frequently reported. Because keyboards are c o m m o n l y u s e d as input devices, d i s c o m f o r t has b e e n r e p o r t e d in the h a n d s a n d w r i s t s [I]. In s t u d i e s a n d a p p l i c a t i o n s to r e d u c e this d i s c o m f o r t , it w o u l d be e f f e c t i v e to m e a s u r e the i n t e n s i t y of e a c h k e y s t r o k e . Moreover, a f e e d b a c k s y s t e m b a s e d on that intensity measurement might increase the performance of V D T operators.
2.
METHOD
OF
MEASUREMENT
L o a d c e l l s are c o m m o n l y e m p l o y e d as force sensors. H o w e v e r , the cost, i n c l u d i n g s t r a i n a m p l i f i e r , is too h i g h to a t t a c h one to e a c h key. T h e y are a l s o d i f f i c u l t to a t t a c h to the u n d e r s i d e of the keys. A n a l t e r n a t i v e m e t h o d of m e a s u r i n g k e y i n g i n t e n s i t y is to a t t a c h e l e c t r o d e s to t h e m u s c l e s involved in f i n g e r movement. Electromyographic (EMG) a c t i v i t y w o u l d v a r y w i t h the force g e n e r a t e d b y the m u s c l e s . That m e t h o d is not s u i t a b l e for widespread applications. Moreover, EMG activity from deeper m u s c l e s w i l l not be r e g i s t e r e d f r o m e l e c t r o d e s a t t a c h e d to the s u r f a c e of the skin. The present study proposes an alternative method for measuring keystroke intensity. A s e a c h k e y is d e p r e s s e d , a v i b r a t i o n c o r r e s p o n d i n g to the k e y s t r o k e s p r e a d s t h r o u g h o u t the keyboard. That vibration can be detected by a piezoelectric accelerometer ( a c c e l e r a t i o n sensor) a t t a c h e d to the k e y b o a r d . The o u t p u t of the s e n s o r c a n be a m p l i f i e d by a c h a r g e a m p l i f i e r a n d s t o r e d in a c o m p u t e r t h r o u g h the u s e of a n a l o g - t o - d i g i t a l (A/D) c o n v e r t e r .
3 . SYSTEM
FOR
EVALUATION
To e v a l u a t e the e f f e c t i v e n e s s of t h i s method, the o u t p u t of the sensor must be compared with a measure known to b e a r e l i a b l e i n d i c a t o r of e x e r t i o n , s u c h as t h e E M G . T h e m u s c l e w h i c h f l e x e s the r i n g finger lies c l o s e to the s u r f a c e of the skin a n d is e a s y to t a k e EMG m e a s u r e m e n t s from. W h e n a key is
798
L~
Figure
1.
Overview
of the
system.
d e p r e s s e d b y the r i n g finger, a c o m p a r i s o n c a n be m a d e b e t w e e n the w a v e f o r m s g e n e r a t e d b y the p i e z o e l e c t r i c s e n s o r a n d the E M G a c t i v i t y . It w a s f o u n d that b o t h the EMG r e a d i n g s a n d the s e n s o r waveform t e n d e d to f l u c t u a t e widely when the force used to depress the key was high. As has been established, the integration of r e c t i f i e d electromyogram (IEMG) per u n i t time correlates with the force generated by the muscle without fatigue. On the o t h e r hand, the m a x i m u m a c c e l e r a t i o n s w i n g for e a c h k e y s t r o k e o c c u r s w h e n the k e y b o t t o m s . A m e a s u r e m e n t s y s t e m e m p l o y i n g a p e r s o n a l c o m p u t e r (PC) w a s developed to c o m p a r e the IEMG and the maximum acceleration swing. Figure 1 s h o w s an o v e r v i e w of the system. T h e PC w a s e q u i p p e d w i t h an A / D c o n v e r t e r a n d an e x t e r n a l , 1 0 - k e y pad. The
]
-:~_-75E>_EF) EEi[:=--:E: i!_: (L- -:~_~-~-~-~:: $-: E:_E
- 1
0
I
(sec.)
a c c e p t a n c e of contact closure F i g u r e 2. The 1 0 - k e y p a d and the accelerometer.
F i g u r e 3. A n e x a m p l e of t h e a c c e l e r a t i o n a n d EMG w a v e f o r m s . (upper : a c c e l e r a t i o n , lower : EMG. The w a v e f o r m s b e t w e e n d a s h e d l i n e s are r e c o r d e d . )
799
output of a b i o e l e c t r i c amplifier was connected to the AiD c o n v e r t e r , a n d its i n p u t w a s c o n n c e t e d to t h e E M G e l e c t r o d e s . A piezoelectric accelerometer was attached to t h e 1 0 - k e y p a d as s h o w n in F i g u r e 2, a n d its o u t p u t w a s a l s o c o n n e n t e d to t h e A / D converter via a charge amplifier. When the subject depressed a k e y on t h e k e y p a d w i t h t h e r i g h t r i n g f i n g e r , the EMG signal from the right forearm and the signal from the sensor on the keypad were simultaneously r e c o r d e d b y a s o f t w a r e r u n n i n g on t h e PC. Both the EMG and acceleration signals began registering 300 milliseconds before key-contact closure was registered by the s o f t w a r e , a n d w e r e r e c o r d e d for 600 m i l l i s e c o n d s (approximately 300 m i l l i s e c o n d s after key-contact release). See f i g u r e 3. T h e IEMG for each keystroke was calculated as an average of the absolutized EMG over that 600 milliseconds. The maximum acceleration swing for each keystroke was calculated as the difference between positive and negative peak values for t h e s a m e 600 m i l l i s e c o n d interval.
4.
EXPERIMENT
For t h e study, four s u b j e c t s d e p r e s s e d keys on the 10-key p a d u s i n g t h e r i n g f i n g e r on t h e r i g h t hand. I n s t r u c t i o n s were p r e s e n t e d o n a C R T d i s p l a y . T h e y i n d i c a t e d , in r a n d o m o r d e r , t h e k e y to b e d e p r e s s e d ( n u m b e r s 0 t h r o u g h 9, c a r r i a g e r e t u r n , b a c k space, p e r i o d , a n d . , + , - , / , a n d =) , s u b j e c t i v e intensity (hard, medium, or soft), and subjective touch ( c r i s p or s t i c k y ) . The subject depressed e a c h k e y f i v e t i m e s for a l l c o m b i n a t i o n s of intensity and touch.
Scattergram Split By: INTEN 20 18 16 O
14 ..-..
~ <
~ D ~ t~,," O "-'
12
)
o o
0 hard [] medium Z~ soft
8 6 4 2 0
-2 0
.02
.04
.06
.08
.1
.12
.14
IEMG(VS/S) AMAX(G) = -.283 + 161.725 * IEMG(VS/S); R^2 = .902
F i g u r e 4. C o r r e l a t i o n b e t w e e n t h e I E M G p e r s e c o n d and the maximum acceleration s w i n g . ( s u b j e c t A)
800
Cell Bar Chart
~
.9
,,u..8
~
1 . 6
rr
.3 .2
.1 0
subj_A
Figure
5. S u m m a r y
sub _B
subi_C
subj_D
of the c o r r e l a t i o n
coefficients.
5. R E S U L T S Figure 4 shows the correlation between the IEMG and the maximum acceleration swing for subject A (r=0.95). Figure 5 s u m m a r i z e s the c o r r e l a t i o n c o e f f i c i e n t s o b t a i n e d from all four subjects (r=0.90 on average). The e f f e c t i v e n e s s of the m e t h o d was g e n e r a l l y confirmed. B e c a u s e of the i n s t r u c t i o n s c o n c e r n i n g s u b j e c t i v e intensity, b o t h the IEMG and the m a x i m u m a c c e l e r a t i o n swing v a r i e d e n o u g h to c o n f i r m the c o r r e l a t i o n b e t w e e n them. The e f f e c t s of the i n s t r u c t i o n s c o n c e r n i n g s u b j e c t i v e touch were u n c l e a r and not c o n s i s t e n t a m o n g subjects. F i g u r e 6 shows the data for subject A, split by the v a r i o u s keys, for c a l c u l a t i o n of c o r r e l a t i o n c e f f i c i e n t s and r e g r e s s i o n lines. The c o r r e l a t i o n c o e f f i c i e n t s and the r e g r e s s i o n lines v a r i e d s l i g h t l y w i t h the keys on the 10-key pad. However, a c c o r d i n g to the g e n e r a l aim of this study, the v a r i a n c e s s h o u l d be acceptable.
6. C O N C L U S I O N The importance of o b s e r v i n g human behavior for the i m p r o v e m e n t of m a n - m a c h i n e i n t e r f a c e d e s i g n has b e e n e m p h a s i z e d by m a n y r e s e a r c h e r s [2]. The m e t h o d d e v e l o p e d in this a r t i c l e can be e a s i l y a d a p t e d to a s y s t e m w h i c h c o u l d w a r n o p e r a t o r s w h e n e x c e s s i v e forces are u s e d d u r i n g k e y b o a r d operation. The author is now p l a n n i n g an e x p e r i m e n t to d e t e r m i n e the effect of an i n t e n s i t y f e e d b a c k s y s t e m on p e r f o r m a n c e .
801 0 []
Scattergram Split By: CHAR
key_* key_+ key_-
20
key_. 18 16
•
÷
key_/
•
X
key_O
O n
14
key_l •
12
+
key_2 key_3
10
t
8
key_4 key_5
6
D
key_6
4
[]
key_7
2
V
key_8 key_9
0 -2 0
.02
AMAX(G) = AMAX(G) AMAX(G) AMAX(G) = AMAX(G) AMAX(G) AMAX(G) = AMAX(G) AMAX(G) AMAX(G) AMAX(G) AMAX(G) AMAX(G) AMAX(G)
-
AMAX(G)
-
AMAX(G)
-
AMAX(G) AMAX(G) =
.04
.06 .08 IEMG(VS/S)
.I
.1 P,
-
e
key_
.14
0
key_
D
key_=
-.652 + 162.364 * IEMG(VS/S); R^P, - .977 (key_*) -.99 + 191.393 * IEMG(VS/S); R^2 - .96 (key_+) -.I 53 + 134.624 * IEMG(VS/S); R^P, = .986 (key_-) -.671 + 198.167 * IEMG(VS/S); RAP- = .806 (key_.) -.44 + 140.662 * IEMG(VS/S); R^2 = .957 (key_/) -.P,42 + 177.126 * IEMG(VS/S); R^2 = .82 (key_O) -.694 + 204.165 * IEMG(VS/S); R^P, = .927 ( k e y _ l ) -1.1 53 + 20:3.898 * IEMG(VS/S); R^2 = .898 (key_p,) -.764 + 198.635 * IEMG(VS/S); RAP-- .936 (key_3) -.85 + 175.1 56 * IEMG(VS/S); R^2 - .927 (key_4) -I .I 94 + 201.03P, * IEMG(VS/S); R^2 - .944 (key_5) -.849 + 192.364 * IEMG(VS/S); RAP_- .971 (key_6) -.478 + 153.829 * IEMG(VS/S); RAP_= .957 (key_7) -.506 + 148.764 * IEMG(VS/S); RAP-- .972 (key_8) -.346 + 140.934 * IEMG(VS/S); RAP-- .955 (key_9) -.841 + 1 52.997 * IEMG(VS/S); RAP,- 3 7 3 (key_) -.457 + 209.987 * IEMG(VS/S); RAP,- .798 (key_) -.686 + 210.445 * IEMG(VS/S); RAP-= .926 (key_=)
F i g u r e 6. The c o r r e l a t i o n s and the r e g r e s s i o n lines. (split by the v a r i o u s keys, subject A)
REFERENCES 1.
D.E. Legrande, VDT R e p e t i t i v e M o t i o n H e a l t h C o n c e r n s in the U.S. T e l e c o m m u n i c a t i o n s Industry, In- M . J . S m i t h a n d G. S a l v e n d y (eds.), H u m a n - C o m p u t e r Interaction- A p p l i c a t i o n s and Case Studies, Elsevier, 1993, pp. Y 8 0 - Y 8 5 .
2.
Proceedings of Quality of Life,
Summit Meeting on Osaka, Japan, Feb.,
Human 1995.
Engineering
for
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
803
A support system for handwriting for the blind using a virtual auditory screen * Kazunori Itoh, Yoshihiro Inagaki, Yoshimichi Yonezawa and Masami Hashimoto Department of Information Engineering, Faculty of Engineering, Shinshu University, 500 Wakasato Nagano, 380 Japan
In order to help the acquired blind with handwriting, we already proposed the sound support system which converted a pen position into that of a point sound image by using the control of sound lateralization. Handwriting patterns can be expressed in a virtual auditory screen reproduced on headphones. This paper deals with the construction of a new virtual auditory screen based on the experimental factors of sound location. The possibility of handwriting and drawing is examined by using a new support system. 1. INTRODUCTION The population of the blind is about 330,000 in our country and the rate of the aged rises every year. Recently, the number of adventitious blind is increased by traffic accidents and diseases, such as cataract and diabetic retinopathy. It is difficult for the aged acquired blind to learn Braille. However, they still want to read printed type, write letters and draw pictures as sighted people do. When a sighted person writes characters and draws figures, he unconsciously uses the visual feedback. If he lost the feedback, he could not write characters properly formed in the correct position on the paper. Accordingly, the visually handicapped persons may be able to write characters by using a sound feedback of information which helps them to write correctly. There have been a number ofattempts to use the auditory sense for the perception of pictorial images. Some early work is restricted to the transmission of sound codes of characters[ 1]. After a few years, an audio display device was developed by using the sound localizing signals[2],[3]. We also have studied the support system for handwriting by using this method[4]. In this paper, we try to improve our support system which helps an acquired blind to write characters and to draw figures by using a new spatial auditory display. Handwriting patterns can be expressed on a virtual auditory screen perceived with headphones listening. In this system, a pen position on the tablet is converted into that of a point sound image which is synthesized by the factor of sound. A new virtual auditory screen is designed by the experimental factors of sound image location. We also examine the possibility of handwriting and drawing by using a new support system. *This work was partly supported by Grant-in-Aid for General ScientificResearchfrom the Ministry of Education, Science and Culture of Japan.
804 2. SYNTHESIZED FACTOR OF A VIRTUAL AUDITORY SCREEN The location factors of the horizontal plane are interaural level differences and interaural time differences between both ears[5]. It is not clear what elements contribute to the localization for the median plane. Sound reflection and scattering on the pinna are important cues of location. Moreover, the sound of low frequency has the tendency perceiving the image to a low position, whereas for the sound of high frequency the situation is reversed. The computational simulation of spatial sound images was investigated by the head related transfer function (HRTF)[6],[7]. However, this simulation has not accomplished the synthesis of an effective sound image and a complex digital signal processing is needed. Figure 1 shows a virtual auditory screen which is composed of 100 x 100 point sound images synthesized by factors of sound location[8]. A horizontal axis of the screen is composed by combination of interaural level differences and interaural time differences between fight and left ears. Maximum values of these factors are set at 14dB in level difference and 0.6ms in time difference. They are equal to zero in the center of the screen. A vertical axis of the screen is composed by a psychological factor due to frequency alteration. Each display point will be on the harmonious scale based on the mel scale of pitch whose lowest frequency is 400Hz and highest one is 2000Hz.
Virtual auditory screen Horizontal
.
. '
Up
...............................................
• .....
.
. '
. '
. '
"
I"
[ i i .....................
! ...........................
1600
1400 .o~ .
.
.
.
.
.
.
i .......................... ' ................. ii......i
Center
(D
1200 "~ 0
1000
¢o ~; G)
800
Down
600
................................................
I
"",
0.4
"',,,,
|
0.6
0.8 1.0
Frequency of display tone (kHz)
'"....
Left
A
E ¢
Center \
Right "...
co" "10
0.6
14 ~
0.3
7
0 C
~
C I.._
.m
"~ 0 E = -0,3 -0.6 C
.
.
.
.
.
.
.
.
0
"~
-7 ~ •
|
2.0
- 1 4 ~= e" m
Figure 1. A virtual auditory screen of the support system.
400
805 Interaural level difference (dB) - 1 4 - 1 2 - 1 0 - 8 -6 -4 -2 0 2 4 6 8 10 12 14 I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
Right
15 E E
..................................................................
iii
....
....
¢--
.o 03 O
Center ,
rl
tone
T T .~x'l',~''" "" 1" •....zx..... 2 . O k H z - . ~1-...~..~•...1.}...-- ....................... - - + - O.4kHz
-15
I
i
I
I
I
I
I
-0.6-0.5-0.4-0.3-0.2-0.1 Interaural time
t
I
I
I
......... Left I
!
0 0.1 0.2 0.3 0.4 0.5 0.6 difference (ms)
Figure 2. Relation between the combination factors of horizontal sound location and perceived position. 0.4
I
--E
Frequency 0.7
I
of
I
display 1.0
tone
(kHz) 2.0
I
I
............................................................................................ Up
15
l-.,,.,.
ca.o
~
0
- 1 5
................~ ~ ~ . ~ - - -
I [
Center
................
Down
..............................................................................
1
500
. . . .
I
.
.
.
1000
.
.
I
,
1500
Mel s c a l e Figure 3. Relation between the frequency of display tone and perceived position. Then, we examined that the relation between point sound images synthesized by above factors and positions of graduated line on the paper. We added differences in each element between the input of both ears and subjects pointed out the position from which they felt the sound was coming. The result of the horizontal axis of the screen shown in Figure 2, Figure 3. shows the result of vertical axis of the screen. It is clear that the relation between the sound elements and the perceived positions are linear and smooth. These characteristics are appropriate for our purpose.
806
3. DESIGN OF THE SUPPORT SYSTEM FOR HANDWRITING Figure 4 shows a block diagram of the support system for handwriting. Some characters and figures are handwritten in the arbitrary sized window set in tablet. The tablet is made of a sensitive panel which acts as a position sensor for a wireless drawing pen. Each window on the tablet corresponds to a virtual auditory screen of the system. When we start a handwriting on the tablet (WACOM SD-420A), two dimensional data of handwritten positions are transferred from the tablet to the personal computer system (NEC PC9801BX2). These writing positions are successively converted into point sound images on a virtual auditory screen. The signal output from the FM sound board (NEEDS TN-F3FM) is converted into a right and left signals by the interaural time difference controller with the BBD delay unit and the multiplying digital to analog converter which makes interaural level difference. Finally, brief tones of 30ms display duration and 10ms silent duration are amplified and fed to cordless headphones (SONY MDRIF610).
Infrared rays STEREO TRANSMITTER SONY:TMR-IF33
HEADPHONES MDR-IF610K •E S S
AMPLIFIER ONKYO:A-812XG
MULTIPLYING D/A CONVERTER Level Difference
i~ Left J
~Ri~
BBD DELAY SYSTEM Time D i f f e r e n c e
TABLET
RS-232C
r........_ _ _ . . _ . r - . , I
PERSONAL
,
J |
COMPUTER NEC
: PIO BOARD i
FM SOUNDBOARD NEEDS :TN-F3FM
Figure 4. Block diagram of the support system for handwriting.
807 4. E X P E R I M E N T S ON HANDWRITING When we write characters, we have to recognize a starting position of handwriting character. We first tried the experiment of handwritten point. We gave a side length of window of 30mm for handwriting on the tablet. This is the same size as the writing frame in a notebook used by pupils to first learn the Japanese characters in the elementary school. Handwritten points are examined under three kinds of condition, namely, with visual feedback, without visual feedback and without sound feedback, with sound feedback. The cardboard of the thickness of about 2mm was pasted around the window on the tablet. We ordered subject to write nine lattice points in the window shown in Figure 5. The sighted subject wearing eye masks, listen to the sound imaging signals at about 70dB SPL. The subjects are three 20-year-old students with good hearing. The accuracy of the starting position in three kinds of condition are shown Figure 5(a),(b),(c) by the average value of each point and the standard deviation in the horizontal and vertical direction as the example of subject. If there is neither visual feedback nor sound feedback, the difference between the indicated point and handwritten point is large. However, subjects can be handwritten a near position when sound feedback exist. Figure 6 shows the average of search time until the point is handwritten under three various conditions for three subjects. Under the condition of sound feedback, it takes about 3.5 times longer compared with the condition of visual feedback. We also evaluate a virtual auditory screen from the handwriting characteristics of lines, figures and characters for the adventitious blind. We found that they wrote lines and simple characters well. Finally, we examined what happened when the acquired blind drew several circles and arbitrary patterns laterally on the tablet. Figure 7 (a) and (b) show the examples of drawing these patterns. These patterns are well balanced in shape and located in the correct position on the window.
.... 0--, ...... O! ........ 0 ........ .....
•o
.... 0 " : ...... Oi ........ 0 ......... .
.
.
.
.
.
0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0.:
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.........
.
.
0
.
.
.
6.
.
.
.
.......
.
.
.
.
.
.........
.
.
.
.
.
.
.
.
.
.
.
: ! ......... i......... .......... ~......... O ~
•
(a) With visual feedback
(b) Without visual feedback (c) With sound feedback and without sound feedback
Figure 5. Accuracy of handwritten points.
808 A
20
I
=¢ - - 1 5
I
..~ . . . . . . . . . . . . "~_. . . . . . .
-13
10 "6
== 5
--~-- Subject B -43-- Subject C
' With
Oisual feedback
I
I
Without visual feedback With sound feedback and without sound feedback
Figure 6. Search time of the handwritten points under various condition of feedback.
(a) Circles
I 0 0
(b) Arbitrary patterns
O
Q
0
Figure 7. Example of handwiting with sound feedback. (Size of window 30mm x 30mm)
5. CONCLUSION We propose a new sound support system for handwriting characters and drawing figures by using a virtual auditory screen. As the experimental results, it is shown that this system is helpful when the adventitious blind write characters and draw figures. It is necessary to improve the handwriting speed and the accuracy of a virtual auditory screen. The authors would like to thank Mr. Yoshihiro Imazeki and Mr. Hiromasa Kaneko for their helpful discussions.
REFERENCES 1. 2. 3. 4. 5. 6. 7. 8.
M.P. Beddoes and C.Y. Suen, IEEE Trans.Biomed.Eng., BME-18, 2 (1971) 85. R.M. Fish, IEEE Trans.Biomed.Eng., BME-23, 2 (1976) 144. Y. Yonezawa and K. Itoh, Electronics & Communication in Japan, 60, 10 (1977) 98. K. Itoh and Y. Yonezawa, J.Microcomputer Applications, 13, 2 (1990) 177. J. Blauert, Spatial Hearing, MIT Press, Cambridge, MA, 1983. F.L. Wighman and D.J. Kistler, J.Acoust.Soc.Am., 85 (1989) 858. K. Crispien, W. Wiirz and G. Weber, Proceedings of 4th ICCHP, Wien (1994) 144. K. Itoh and Y. Yonezawa, Proceedings of 1st World Congress on Technology, 4 (1991) 230.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
809
A S y s t e m for 3D M o t i o n and Position Estimation of Hand from Monocular Image Sequence Yoshio IWAI, Yasushi YAGI, and Masahiko YACHIDA a aDepartment of Systems Engineering, Osaka University, Toyonaka, Osaka 560, JAPAN Human interface has recently gained increasing importance for human-machine communication such as virtual reality. It is useful for us to employ gestures or communication as a human-computer interface. This paper describes a model based system that estimates the motion and the position of the hand from the monocular image sequence. 1. I N T R O D U C T I O N Sensing of human hand motion is very important for human-computer interactive applications such as virtual reality, gesture recognition, and communication. Current commercial methods require a user to wear wired gloves with a magnetic sensor[i]. Using wired gloves, the direction of the hand can be easily estimated; however, the estimation accuracy of the direction is not satisfactory to us because the resolution of the sensors of the wired gloves is poor, and it is uncomfortable for the user to wear wired gloves. On the other hand, a vision system is suitable for human-computer interaction since it is passive sensing and the orientation of the hand can be estimated without being noticed by the user. The existing methods to estimate hand motion by a vision system is classified into two method, namely, "feature-based" and "model-based". A "feature-based" method relies on the extraction of a set of discrete object features, such as points and lines located in successive images, whose image coordinates are used as data to estimate object motion[2]. This method can estimate the motion of the hand speedily and effectively. It is, however, difficult to track features while they are occluded. In the model based method, it is assumed that the 3-D geometrical model of a hand is given a priori[3]. The location and the motion of the hand can be estimated by matching the input image and image model generated from the given 3-D model. Such application to real-time problems as human-computer interaction is not easy owing to long computational time. In this paper, we propose a system for estimating motion and location of the hand from a monocular image sequence by combining both the "feature-based" and "model-based" methods. 2. M O D E L S A N D P R O B L E M S T A T E M E N T 2.1. C A M E R A M O D E L We use the 3D camera-centered coordinate system O c - XcY~Z~, aligned with the image coordinate system OI - XIYI and fix the origin Oc at the camera center and image plane is on a level of f, where f is the focal length of the camera. We assumed that the focal length f, the aspect ratio a, and the image center (icx, icy) T of the camera system are
810 calibrated in advance. Thus, the point Pc - (xc, Yc, Zc)T at camera-centered coordinate system Oc-XcY~Zc is projected onto the image point P1 at (xi, yI) T under the perspective projection as the equation (1).
(x,) Y1
I zc \ Yc /
\icyj
(1)
2.2. O B J E C T M O D E L We consider an object-centered coordinate system O o - XoYoZo whose origin Oo is fixed at the bottom of the palm. We define the position of Pc(t) at time t on the cameracentered coordinate system and the position of Po(t) ,which corresponds to Pc, on the object-centered coordinate system. The relation between Pc(t) and Po(t) is given by Pc(t) = R(t). Po + T(t)
(2)
where R(t) and T(t) are the 3 × 3 rotational matrix and translational components at time t between these coordinate systems, respectively. Next, we represent a human hand by the following parameters.
Poi(t) lji
Wji rfi, Ofi
the location of the feature point i in the object-centered coordinate frame at time t. the length of the j-th link of the finger i. the width of the j-th link of the finger i. the location of the tip rE and the bottom OF of the finger i in the objectcentered coordinate frame.
2.3. P R O B L E M S T A T E M E N T To overcome the scaling ambiguity, our system makes a shape model of user's hand from the input image at a known initial position. This is a reasonable approach for the human interface not to restrict the users. We assume that the initial position of the user's hand W(0) and R(0) are given in advance. Po~; lj~; W/i;rFi; OF~, and Cj~(0) are measured by the Initial Analyzer of this system when the user places his hand at a certain distance from a camera. After obtaining these initial values, the system estimates the following parameters at each succeeding frame. W(t), R(t)
dOj(t) dW(t) dt ' dt
Cji(t)
the position of the hand at time t. the motion of the hand at time t. the angle of the joint j of the finger i at time t.
3. O U T L I N E O F T H E S Y S T E M The system consists of four parts: Initial Analyzer, Feature Extractor, Motion Estimator, and Position Estimator. We explain the flow of the system and these four parts' roles in the following section. First, the user places a hand on the initial position parallel to the image plane (cf. the 1st frame in Fig. 1). The Initial Analyzer measures the initial parameter of the hand
811 and makes finger shape model from the first input image. After the Initial Analyzer successfully ends, The Feature Extractor finds feature points on the hand from an image sequence and traces these feature points and makes their trajectories. The Motion Estimator and the Position Estimator cooperate to compute motion and position of a hand. The Motion Estimator calculates the motion of the hand from trajectories of feature points and prior positions of feature points estimated by the Position Estimator. The position of the hand is calculated from the motion and prior position. The estimated position, however, has an observational error. Therefore, our system estimate again a more precise position using the Position Estimator. 4. G E T T I N G H A N D M O D E L A N D I N I T I A L P A R A M E T E R S In this chapter, we explain the method for the acquisition of the hand shape model and the initial parameters by the Initial Analyzer. 4.1. F I N D I N G H A N D R E G I O N A N D F E A T U R E P O I N T S We find the hand region by subtracting the background image from the input image and by binarizing. We extract the feature points of the hand from the binarized image by secondary differentiating along the contour of the hand region. 4.2. E S T I M A T I O N O F F I N G E R P A R A M E T E R S At first frame, the user's hand must be placed at initial position. We measure finger parameters as follows. 1. Find the X axis of the object coordinate system. The moment axis of the binarized hand region is regard as the X axis of the object coordinate system. At that time, the direction of the X axis is not determined. 2. Make tip-bottom pairs of feature points. First, We make a pair of one convex feature point and two concave feature points along the contour of the hand. We assume that the middle point between two concave feature points as the bottom of the finger and the convex feature as the tip of the finger. We then determine the length and the direction of this pair. 3. Extraction of the finger. In the initial state, we can assume that the user's fingers are parallel. Therefore, we find out groups of the tip-bottom pairs which are parallel to each other. Next, we find the finger group of which the variance of the angle is the smallest of the groups previously found. 4. Find the thumb. We regard the tip-bottom pair as the thumb which is near the finger group and the middle point projected to X axis is the nearest from the centroid of the hand region. 5. Determination of each finger Then the tip-bottom pairs in the finger group which is near the thumb is considered as the fore finger, the middle finger, the ring finger, the little finger in that order. 5. F E A T U R E T R A C K I N G ( F e a t u r e E x t r a c t o r ) Feature Extractor calculates the similarity of the two features between two frames and then makes the correspondence of the feature and the most similar feature point. When the similarity is greater than the given threshold value, such feature points are treated as
812 disappeared. The similarity of a feature point is defined as follows:
]I(r~ + x, t)
conf- ~
_,+z,, + x, t + At)l I(,i,p,~d
-
(3)
xEw
where w is the region of the feature, r it is the position of the feature i at time t on the image coordinate system, and ri,v,~d is the predict position. The next position of the feature is predicted from linear combination of the corresponded position and the previous predict position as the following equation: t 1 t 1 rt+l i,r~d - ric--onf + ri,r~d(1 - c--onf)"
(4)
6. M O T I O N A N D P O S I T I O N E S T I M A T I O N OF P A L M ( M o t i o n E s t i m a t o r ) To estimate the motion from a monocular image sequence, we need five feature points on a rigid object. However, it is difficult to find and track five points continuously. Therefore, we use a hand model and a finger model for simplifying the estimation problem of motion and position. We can estimate the motion and the position from only three feature points on the hand. To estimate the motion of hand, the system uses the equation (5), which is derived from equations (1) and (2), AX = B
dot dO2 dO3dT. dT~ dT~ x -( A-
alj =
{(xi(t + A t ) - xi(t)).a/f ~ zc(t)
dr' dr' dt dt ' dt ' dt )' (all \a21
a12 a13 1 0 a22 a23 0 1
OR(t)po.t.() { O R ( t ) 00j
•
00j
(5)
B - \ (y,(t + A t ) - y , ( t ) ) / f ] ~Xt
-xi(t +At).a/f~ -yi(t +At)/f ] Po(t)
z
-xi(t)
'
a2j =
OR(t) Po(t) ] 00j
~
OR(t)Po(t) l 00j
z
.yi(t).
As input data actually has observational error, this system uses the weighted least square method to estimate the motion parameters precisely. The position of hand is given by the following equations, T(t + At) - T ( t ) +
~dtT . A t '
i-1 O,(t + ~Xt) - O,(t) + dOi.At ~t
2, 3.
7. P O S I T I O N E S T I M A T I O N OF F I N G E R S ( P o s i t i o n E s t i m a t o r ) The degree of freedom of finger is too high to estimate the finger motion analytically. We use, accordingly, the energy minimization method for finger position estimation by using model matching. The energy cost function Etotal consists of the feature position constraint EF, the contour constraint Ec, and the finger position constraint FB. 7.1. E V A L U A T E F U N C T I O N S Let r i be the position of each feature point i on the image coordinate system and mi be the position, which corresponds to ri, on the model coordinate system. The feature position constraint is defined as follows:
EF -
1 ~n
~(ri-
Prr~)TKs(ri - Prr~)
(6)
813 where EF is the weight matrix, and P is the projection function from the model coordinate system to the image coordinate system. Above equation (6) is minimal when the projected position of the model point and the position of the feature point are equal. The contour constraint indicates if the contour of the model is matched with the edge extracted from the real image. Let n(dc) be the normal vector on the location dc on the contour c generated from the shape model. The contour constraint is expressed by the following equation:
1 0 I(dc)12dc Ec = --~ ~ l On(dc) G .
(7)
where L(= fc dc) is the total length of the contour. This equation is minimal when edges actually exist on the generated contour. The finger constraint binds the fingers not to move freely. The finger constraint is given by the following equation:
1
Es---~~{(ri--rj)-(r
° - r °)
}T
.KB{(ri-rj)-(r
o
°-rj)}
(8)
where r~ is the bottom position of the finger i, and r i0 is the bottom position, estimated by the Initial Analyzer, of the finger i at time 0. 8. E X P E R I M E N T A L R E S U L T S We show an experimental result on a real image sequence. We use 40 frames as shown in Fig. 1. The left-top image is the 1st frame and the right-bottom image is the 40th frame. The result image generated from the estimated shape model and the estimated position are shown in Fig. 2. (The angle of the camera is slightly changed from the angle of the real camera.)
:::
Figure 1. Snap shots of the input images" the 1st, 10th, 20th, 31th, 33th ,39th frame
814
:i!!:I~, ~:::i. .i:. "..,.
....I
....!:|
::|
1
Figure 2. Result images generated from estimated shape and position: the 1st, 10th, 20th, 31th, 33th, 39th frame 9. D I S C U S S I O N A N D C O N C L U S I O N We described our hand motion and position estimation system, and its implementations. Our system can stably estimate the motion and position of the hand from the monocular image sequence by using the shape model and feature point tracking. The high degrees of freedom of the hand model bring about the long computation time to minimize the energy cost function. For the reason given above, we used the motion information estimated from the trajectory of the feature points to reduce the search space of the hand model. The accuracy of the feature tracking affects the accuracy of the motion estimation of the hand. It is, however, difficult to track the feature points owing to the occlusion and the weakness of the feature points. Therefore, we used the weighted least square method to estimate the motion of the hand. This method largely contributed to increase the accuracy of the motion estimation. We will apply this system to the real time application and develop the hand gesture recognition system in the future work. REFERENCES 1. Tomoichi TAKAHASHI and Fumio KISHINO. A hand gesture recognition method and its application. The Trans. of IEICE D-II, J73-D-II(12):1895-1992, December 1990. 2. Roberto Cipolla, Yasukazu Okamoto, and Yoshinori Kuno. Qualitative visual interpretation of 3d hand gestures using motion parallax. In MVA '92 IAPR Workshop on Machine Vision Applications, December 1992. Tokyo. 3. Norihisa FUJII and Toshimichi MORIWAKI. A study on motion measurement of fingers based on image analysis. The Japanese Journal of Ergonomics, 27(3):151-157, June 1991. 4. Yoshio IWAI, Yasuhi YAGI, and Masahiko YACHIDA. 3-d motion and position estimation of hand from monocular image sequence. In Meeting on Image Recognition and Understanding (MIRU'9~), pages II-207-II-214, Tokyo, July 1994. IPSJ.
IV.9 Physiological Measurements 3
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
817
A case study on evaluation method for VDT work load using with face skin temperatures Yoshinori HORIE Dept. of IE & Management, College of Industrial Tech., Nihon University 1-2-1, Izumi, Narashino, Chiba, 275 JAPAN Abstract Many methods such as Oxygen Consumption, Heart Rate, Electro-myogram, CFF value and etc., are now thought to be effective to evaluate physiological load of VDT workers. Measuring surface temperature of body with Thermal Video System has already been established as a useful index for evaluating physiological load of the workers. This study deals with measuring various parts of face skin temperatures represented by nose and cheek, CFF value and Heart Rate of VDT workers, to skin for their validity. Through comparing each of them. The correlation coefficient between each items were also obtained. The results proved the index of face skin temperature to be the most effective index for evaluating VDT work load. In conclusion, the correlation between nose skin surface temperatures and CFF values was significantly highly while the correlation between cheek skin surface temperatures and CFF values was not so clear as the relation between nose CFF. This was also confirmed through statistical analysis. From these results, it can be concluded that TVS could be one of the useful, reliable and easy to measure method to evaluate VDT work load. 1.INTRODUCTION
There are many methods to evaluate worker's psycho-physiological work load. As for evaluation of neuro-sensory work load for VDT workers, CFF value, Heart Rate, EMG, Subjective Symptoms of fatigue feelings and etc., have been frequently used as effective indexes. When we use them, the most reliable methods should undoubtedly be selected depending on the type of work.
818
The most important thing when we estimate work load is to obtain effective and highly reliable data with reliable measuring methods. In addition, workers can work trader normal working condition and never allow measuring items affecting on work load. Body temperatures is one of the objective and physiological indexes for workers, and has been thought to be the effective index ~ u s e of high correlation coefficiency existing between body skin surface temperature of each part of body. This study mainly deals with measuring face skin temperatures of VDT workers by Thermal Video System(TVS), if we can purpose it as a useful index to evaluate VDT work load.
2.METHODS A series of experiments was carded out with using VDT worker's face skin surface temperature recorded by TVS. The main task given to subjects in the experiment was to search and add a pair of randomized 400 one digit numbers from 3 to 9. They were told to input only the last digit of the added number into the computer using the ten keys with the fight hand. Ten healthy male students were used as the subjects. All subjects were fight handed and their visual and musculoskeltal functions were normal. They were prohibited from smoking during the experimental days. Prior to the experiments, all subjects were asked to take rest under seated posture for 30 minutes acclimatize themselves to the thermal conditions during experiments to in the experimental room where the room temperature was set up at 20,0 to 22,0 ° C, with humidity of between 55 to 60 %, and the air current from 0,1 to 0,3 meter per second. Immediately after the rest, all subjects had to continue the main task for continuously 60 minutes. During each experimental session, thermo-graphy of the subjects were measured at every 5 minutes of totally 13 times, while CFF value were measured at every 10 minutes of totally 7 times. After completion of work, subjects took 10 minutes rest and then, were measured thermo-graph and CFF. Heart Rate variation was measured all through the experimental sessions. The TVS equipment used for the experiment was produced by the Nippon Avionics Co.,Ltd. and called the TVS-2200 series. The subjects' skin surface temperatures were measured b y t h e TVS with a resolution of 0,01 ° C, and with condition of radiation rate which was equivalent to E=0,98.
819
3.RESULTS Variation rate for each measurement shown in Fig.1 was based upon the mean value at rest before starting experiment. As mentioned earlier face temperature, CFF value and Heart Rate were measured. Correlation coefficient of each measurement item was calculated at every 10 minutes and was shown in Table 1. Average face surface temperatures were obtained from both at the nose and cheek part. 1
o
0
5
-
-
"
"
1.04 1.03-,_. ---&...
1.021.01 o om * m
>.
~ . o
....
...& ....
.,~.-....~
t
o ....
o ....
.,,,& \
..~.__,.._~ /
o . . . .
o ....
<~ . . . .
e---.~
1
0.99-
""
0.98-
~EL.. ""
.,4 ........ + ....... + ' " . ' +. ....... . . + ....... . +'" "
0.97 -
.....
".
~ f f . T
~B
.÷ ....... + ....... + ....... +...
.....~"
,..b,,t,B
0.960.95
1 0
I---
5
10
I
15
"
-
I
!
i
i
213
25
313
35
'l
413
~
l
45
513
'
Time(min)
!
J'
55
613
'I'
713
~
0 °.,i
t= ~. F1
CFF value
Nose Temp.
~
Cheek Temp.
/~
HR "~
~=
Fig.1 Each measurement items variation rate (n=10)
t
Fig.2 Thermo-graphy on before VDT work
Fig.3 Thermo-graphy on after 5 min. VDT work
820
Table 1. Correlation coefficient between each measurement items Time(min)
Nose Temp.- CFF
0 10 20 30 40 50 60 70
0.72, -0.76, -0.66, -0.74, -0.83 ** -0.87 ** -0.73 , -0.51
Time(min)
Cheek Temp.- CFF
0 10 20 30 40 50 60 70
-0.38 -0.55 -0.46 -0.45 -0.83 ** -0.75, -0.60 -0.42
Nose Temp.- HR
0.49 O.54 O. 7 4 , O. 7 2 , 0.77, 0.66, O. 80 ** O.64
CFF- HR
-0.19 -0.33 -0.54 -0.59 -0.57 -0.57 -0.53 -0.01
Cheek Temp.- HR
-0.03 -0.03 0.33 0.37 0.37 0.53 0.56 0.33 ,'P~_O.05 **'p~0.01
Nose skin temperature shows quick decrease for 5 minutes after starting work, and then, continuous increase for next 35 minutes, then re-decreased, and finally re-increased again at the last working session before finishing the all experiments. Fig.2 and 3 showed the thermo-graph of subjects before some and after 5 minutes of each work. Nose temperature clearly decreased about 2" C during five minutes work. The significant decrease of nose skin temperature immediately after starting work were caused by the load of main task that leads to decrease of cerebral cortex activity. On the other hand the temperature of peripheral parts of nose were influenced by temperatures in the experimental room. Re-decreasing phenomenon of temperature for 35 minutes after work starting were mainly due to the last spurt or work stress. Cheek skin temperatures were significantly and continuously increased during experiments, but variation rate was recognized less than 1 % . From this results therefore, evaluation with cheek skin temperature is not so efficient for estimating work load of this kind as nose temperature. CFF values increased for 20 minutes immediately after work starting, and then showed continuous decrease. Together with the deterioration of the cerebral cortex activity that accompanig with increase of monotony feelings caused by the neuro-sensory mental work, i.e. VDT task.
821
Heart Rate varied rather bigger 15 minutes after working starting, and then stayed constantly within small domain. It recovered to the variation level before the experiment immediately after experiment. From the results of correlation coefficient between each measurement items, it can be observed that significant correlation existing between nose temperatures and CFF values, and Heart Rate, but not between cheek temperatures and CFF values and Heart Rate, These results showed that the face skin temperatures showed significant difference depend on measuring spot. There was slight minus correlation coefficient observed between CFF values and heart rate. 4.CONCLUSION In conclusion, the correlation between nose temperatures and physiological functions such as CFF values and Heart Rate were validated. As for the correlation between the cheek temperatures, and CFF values and Heart Rate, it was not so clearly observed. Data relating to variation nose skin temperatures along with VDT work load is more useful than that of cheek skin temperatures. It can also be concluded that using TVS should be one of the effective evaluation methods to estimate VDT work load of workers. REFERENCE
1. K.Atsumi, Handbook of Medical Thermography, Nakayama-shoten, Tokyo, 1984 2, T.Funakawa, Modem Medical Technology 9, Clinical Physiology, Igakushoin, Tokyo, 1980 3. K.Hashimoto, Physiological meaning of the critical flicker frequency (CFF) and some problems in their measurement --Theory and practice of the flicker test--, Japanese Journal of Industrial Health, 5(6), 3-16, 1963 4. Y.Horie, A study on the evaluation of simple work load by a thermal video system, Towards Human Work, Ed. M.Kumashiro & E.D.Megow, Taylor & Francis, 1991 5. T.Miura, Handbook of Occupational Health (3rd edition), The Institute for Science of Labour, Kawasaki, 1974
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
MEASUREMENT DURING
VDT
OF
WORK
LOAD
823
USING
BRAIN
POTENTIALS
TASKS
Akihiro Yagi & Mika Ogata* Department of Psychology, Kwansei Gakuin University, Nishinomiya, Hyogo, 662, Japan 1.1NTRODUCTION The event related brain potential (ERP) changes with visual attention or work load. However, application of visual ERP to assessment of work load is quite limited, because eye movements are restricted in measurement of ERP. Theauthor, etal.[1] and Isreal, etal. [2] have used the auditory ERP as a measure of the secondary task in the dual task method for assessment of visual work load. A single processing capacity of attention should be hypothesized in the dual task method. However, the visual task and the auditory task do not always have a common spare capacity. Therefore, the direct visual ERP should be required to measure the work load. During a visual task, eyes usually move. An EOG eye movement record shows a step-like pattern consisting of saccades and fixation pauses. The saccade occurs about three times a second. Information on the visual object is sent up from the retina to the brain during the fixation pause. When EEGs time-locked to onset of fixation EE6 pauses were averaged, the eye fixation related potential (EFRP) can be obtained (Fig. 1). We developed the system to detect the EFRP [3]. EFRP consists of several components. Some components of EFRP change with properties of the stimulus object; e.g. the spatial frequency, the contour and the brightness of the stimulus [4]. The wave form of EFRP might change with the differences of the screen "1" V" of the CRT. The other components change with information processing load; e. g. the signal detection [5] and the language processing [6]. Therefore, there is a possibility to apply EFRP to assessment of Fig.1. Detection of EFRP. visual environments or work load. The purpose of Averaging EEGs at onset of fixation this study was to examine the variation of EFRP (i.e." offset of saccades). during VDT tasks. £
L
1
*Present position, Mika Ogata, Lighting Research Laboratory,CorporateResearch Division, Matsushita Electric Industrial Co. Ltd. This research is supported by the grant of The Illuminating Engineering Institute of Japan (No.3Af02).
824 2. METHOD
Subjects were "eight students with normal vision aged from 20 to 27 years old. They all have had an experience of computer operation by using a mouse. The distance from the subject to the CRT was about 60 cm. The subject was instructed to draw a same pattern at the fight side on the CRT as the stimulus figure presented at the left side without using "copy command". The task was performed with operating a mouse by using a software for drawing (Just System, HANAKO). The stimulus figures were 20 types of random polygons with 20-26 comers. The average visual angle of figures was about 6 x 6 . When a figure was completed, a new figure in the file was displayed for the next task. 10 figures were given for one task condition. F_ach task continued for 60 min. Each subject was assigned to two task conditions in which the screens of the CRT were positive; black lines on the white background (P condition) and negative; vice versa (N condition). The order of conditions was counter balanced. The duration of each condition was 50 min. A rest period between tasks was 60 min. Brain waves (EEG)referred to linked ears were recorded from Cz, Pz and Oz. EEGs were amplified with high gain differential amplifiers at a low frequency time constant 2.0 s and a high frequency cutoff at 50 Hz. The horizontal and the vertical electro-oculograms (EOG) were measured for eye movements. EEGs and EOGs were recorded on a magnetic tape. Data of EEGs and EOGs were divided into five blocks (every i 0 rain ; 0, 10, 20, 30 and 40 rain) in order to compare the data in course o f time. The data for 150 s from the beginning of each block were digitized at a sampling rate of 200Hz. The 500 ms EEG epochs without artifact, beginning 100 ms before the onset of the fixation pauses (i.e. offset of saccades) were averaged in order to obtain the fixation related potentials.
3-i -
N EOG
,
Cz D-,
P
[.
I
|
~
,
I
~
.... I
,
I
,
|
EOG r-
,..
~...~.~.:,
,
I
•
•
!
u/ |
| ...v
i
,
~
~-,
Z L~.' ~i
v/_
aUV7 .
2BiB Time (ms)
4BB
'r-I
''
~~..-.~7/
~
.~
l
:
',',
'
p,. - J ,,,,~ .
auVl 1 B
CZ
' ,'
*
]
:
J
-
~.T~-']
] A-J
..,
. . . .
,
.
l
I
'
~
0
,
~
l l .......
200 Time (ms)
Fig. 2. Grand averaged EFRP at five blocks in the positive (P) and the negative conditions (N).
,
-~--I
'
40B
825
3. RESULTS
Fig. 2 shows the grand averaged potentials across 8 subjects in five blocks for two task conditions. The top traces are grand averaged EOGs. After the moment of eye fixation ( the time zero ; 0 ), the patterns of EOGs were flat. The results indicated that there was no artifact during the fixation pause. Positive components peaking around 80 ms from onset of fixation pauses appeared at Oz and Pz under the P condition. The component is called the lambda response. On the other hand, the components observed at about 80 ms were very small under the N condition. The positive peak appeared at about 200 ms. Fig. 3 shows EFRPs in the P and the N conditions superimposed in order to compare in more detail. The amplitudes of the potentials in the N condition were lower than in the P condition. Mean amplitudes of the potential with latency from 50 ms to 150 ms were quantified and analyzed by ANOVA of Task x Electrode Cite x Block. The main effects of task conditions (F(1,7)= 15.12, p < .006), and of electrode cites (F(2,14)=10.43, p< .0017) were significant. The interaction between the tasks and the electrode cites (F(2,14)=31.15, p< .0001) was significant. There was no significant difference in the potentials in five blocks. The mean amplitudes of components for each block (10 min) were computed to analyze the variations in course of time. On the whole, no systematic changes in amplitudes and latencies of the components were observed among blocks. There were large inter-individual differences in the patterns of the variation of the amplitude in course of time. The wave forms of EFRP at Oz in the N condition could be classified into two groups by inspection in more detail. Fig. 4 shows potentials averaged separately from three subjects with the negative shift and those from five subjects with the positive peak in the N condition. Although five of eight subjects showed the double positive peaks at about 80 ms and 150 ms in the N condition, three subject showed the negative shift at about 100 ms and the positive component at about 200 ms. Wave forms at Oz and Pz showed large differences P N........ between two groups. On the other hand, the wave forms at Cz showed littledifferencesbetweentwo groups. EOG~-!' ~' ~ " : The test of subjective symptoms on fatigue, the subjects felt fatigued by the tasks. The scores in complaint of fatigue were higher in the N condition than in the P Cz ~ ' f ~ ~ ~-', condition. 4. Discussion
When a subject performed a visual task in the dark situation, we found, in a recent research [7], that latency of the positive component (the lambda response) delayed more than that in the bright situation. The delay of the component to the negative screen (in the N condition) would be caused by the darkness of the stimulus. It would reflect the activities of specific cells on the retina under dark adaptation. The early components of EFRP might reflect the property of the stimulus in the environment. Therefore, the potential might be useful as an index for the assessment of a screen of a display.
I-. I ,
Pz i o A .
I
X, B v
I..'..I
..... 2,BB
4BB
Time (ms)
Fig. 3. Grand averaged EFRP in the positive (P) and the negative conditions (N)
826 In the present study, there was no systematic variation or decrement in EFRP among O---q 1 - - - 2 - - 3 - - 4 - 0--; l - - - 2 " 3--4-blocks. Although the pattern of E0G i---~] . - i , ~....... t ~-7 ~ / i : I : i , / • ,, the variation in amplitude seemed /v-, to related to the arousal level, ,~ , '~ .,,.,. , , , , ~ .,......... ~.~.~ , .~ k,--, - ~ '~'~' ~ , , . - .,.~~-.~, , --.--., .-_ ,.-, ,.u:.-, '--'"~the further research should be Cz ,~, "',.," required for the conclusion. The subjects reported that they were enjoying the tasks and felt no decrement of the visual function Pz ;,.:"J--~.,~: ~.._-.~_,~ k~m~. , , , ~ during the task. The EFRP could be observed as long as the subjects perceived the stimulus object clearly. The result ~ ~~ ' ~":'"-'~ ' - " J ' ' ' ~ < showed that the EFRP reflected Oz the processing of visual information. The test of the subjective a u v T . ..... ,, 7, I ..... ...... symptoms on fatigue indicated 0 20B 0 200 4B that the subject felt fatigued by the tasks. The values were Time (ms) Time (ms) measured after the task, not during the task. In this study, we Fig.4. Grand averaged EFRP in the N condition. could not find the relationship A. Three subjects with negative shift, between EFRP and fatigue of the B. Five subjects with positive shift. subjects. However, if the task continued longer period, EFRP might change more clearly. This research indicated that EFRP would be applicable as an index of the work load in the VDT task.
A
B
x~5~
4~0
REFERENCES 1. Yagi,A. & Ohtani,A., Bulletin of Industrial Products Research Institute, 7 (1975) 43-47. 2. Isreal, J.B., Wickens, C.D. Chesney, G.L., & Donchin, E., Human Factors, 22 (1980) 211-224. 3. Konishi, H. & Yagi,A., Information Processing Research (Kwansei Gakuin University), 9 (1994) 19-24. 4. Yagi, A.,(eds. Tsutsui,S. & Shirakura, K.), Biobehavioral Self-Regulation in the East and the West, Springer Verlag (In press). 5. Yagi, A., Electroencephalography and Clinical Neurophysiology, 52 (1981) 604-610. 6. Yagi, A., Kita, K. & Katayama, J., Supplement to Psychophysiology (1992) $75. 7. Y agi, A., Imanishi,S., Konishi, H., Akashi, Y. & Kanaya, S., Ergonomics (In press).
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
827
The R e l a t i o n s h i p b e t w e e n H u m a n M e n t a l V a r i a t i o n and Stereoscopic I m a g e s ....... EEG Approach ...............
Sakae Yamamoto a, Shigeaki Matsuokab, Sumio Yanoc aDepartment of Management Science, Dokkyo University, Souka, Saitama 340, Japan bDepartment of Neurosurgery, University of Occupational and Environmental Health, Japan, Yahatahigashiku, Kitakyushu, Fukuoka 807, Japan ~ATR Laboratories, Kyoto, Japan Stereoscopic imaging TV is expected to be the next generation TV. This kind of TV is based on the artificial stereoscopic images in which there are great differences in perception in comparison between 2D and 3D TV. The relationship between spatiotemporal analysis of EEG activity and the differences between 2D and 3D TV watching were studied. The results obtained indicate that in 3D TV, more characteristics EEG changes were revealed, such as the frontal midline theta activity, caused by heighten concentration in comparison with 2D TV. 1. Introduction In recent years, attention has been shifted from a conventional two-dimensional image representation to stereoscopic image representation. Stereoscopic images can more clearly express feelings of solidity, jumpout, and mass, and can become closer to actual images than conventional two-dimensional images. A stereoscopic image is artificially constructed by introducing a left and right image through the left and right eye respectively. The brain fuses the two images into a single image. Various methods are developed to introduce different signals into each eye in a stereoscopic imaging system, including the polarizing filter method and the liquid crystal shutter method.
2. Purpose The purpose of this study is to present the subject with stereoscopic TV images produced by the polarizing filter method and the liquid crystal method and to investigate how the EEG of the subject changes when the subject fixates on the presented stereoscopic stimulus. Investigation of the cerebral activity during image fusion amounts to study of information processing in the image fusion process. The cerebral activity can be grasped by observation of brain waves as physiological reaction. Special emphasis is placed on the relationship between the appearance and the position of brain waves as measured from the scalp.
828
3. Experiment-1. Liquid Crystal Method 3.1 Stereoscopic Stimuli and Two-Dimensional EEG The subjects were presented with random arrangements of about 10 chessmen each and were asked to count the chessmen in each frame presented. 3.2 Subjects Six subjects, 20 to 21 years old, participated in the experiment. 3.3 Experimental Procedure The frame presentation time is given in 3 minutes. The three experimental conditions, or 3D, 2D and spectacles worn, and 2D and spectacles not worn, for EEG recording were randomized for the subjects. The EEG was recorded during the experiment. Each subject remained still with their eyes closed for 5 min. before the start of the experimental task, and the EEG of the subject in this posture was recorded and used as control. 3.4 EEG Recording Procedure The EEG was recorded with the International 10-20 system as shown to the left of Figure 1.
/
JII II I I
~
.
•
3
Ik If ,Z
Figure 1. The location of electrodes and he edited parts of EEG 3.5 Analysis of EEG The theta wave was analyzed in 3.6 to 7.8 Hz frequency bands, because recent investigations have indicated that the theta wave has a close relation to mental activity. 3.6 Experimental Results and Discussion The derived brain waves were analyzed on a Nihon Kohden Model ATAC-3400 two-dimensional brain wave analyzer. The brain waves were subjected to fast Fourier transformation and linear interpolation to identify 25 cerebrum parts ( the nodes in the middle of Figure 1). The potentials in the frequency bands of the 25 cerebrum parts were determined accordingly. For better legibility of variations in the brain waves, the 25 cerebrum parts were reduced to nine cerebrum parts as shown to the fight of Figure 1, and the analysis was made based on the average potential in each cerebrum part. The analysis of variance (two-way classification) was performed on the data edited in
829
the nine cerebrum parts. The following data model was assumed: Yijk = ~ 4" a i Jr ~j Jr ( a~)ij Jr eij k
where i indicates the experimental condition and j represents the edited recording part(see Figure 1) and k refers to the subjects. The results of this analysis of variance are given in Table 1. The analysis of variance identified significant differences in the theta wave between the experimental conditions and between the cerebrum parts. An attempt was made to extract the features of the experimental conditions and cerebrum parts, based on the results of analysis of variance. Table 1. Results of ANOVA 2 in the Theta wave. FACTOR
S.S.
D.F.
M.S.
F-VALUE 13.93 ***
A
32.05
3
177.35
B
237.78
8
29.72
2.33
95.20
24
3.97
0.31
E
2291.13
180
12.73
T
3156.15
215
A*B
A
: The effect of the experimental conditions
B
: The effect of the location of head
A*B
**
(p<0.01) (p<0.05)
: The effect of the interaction of factor A and B
E
:Error
T
:Total
The ratio of the brain waves before the start of the experiment to the brain waves under each experimental condition was taken to normalize the data. The mean and standard deviation (SD) of all nine cerebrum parts were obtained under each experimental condition. In order to classify the brain wave potential, we took the boundary of Mean ___SD, Mean_+ 2SD. Theresults of the analysis are shown in Figure 2. THETA WAVE
2-D spectales not worn
2-D spectacles worn
3-D spectacles worn
Figure 2. The different of location of EEG potential.
830 Under the 2D and spectacles not worn condition, the occipital potential is lower than the control potential, and the fronto-bitemporal potential is higher than the control potential. Under the 3D condition, the fronto-central and parietal potentials are higher than the control potential. When the 2D and 3D conditions are compared, the theta was higher in the fronto-central and parietal areas under the 3D condition.
4. Experiment-2 Polarizing Filter Method 4.1 Experimental System The two TV monitors were positioned at fight angles to each other, and images from the two TV monitors were displayed on the same TV screen through a half mirror. The subject watched the images on the TV screen with polarizing filter spectacles. The TV screen measured 50 in. diagonal and was placed at a height of 90 cm and 2300 mm in front of the subject. The images measured 83 cm per side and subtended 21 o at the eyes of the subject. The subject sat on a chair, and the height of the chair was adjusted to bring the sitting eye height of the subject to the center of the TV screen.
4.2 Stereoscopic Stimuli and Two-Dimensional EEG The following four types of images were prepared: 1)A bowling pin is being thrown, 2)A street organ is being moved, 3) A sea turtle is swimming, 4) Sea gulls are flying (freeze or still frames). Frames 1) are those of motion and emphasize stripeless. Frames 2) are those of slow motion. Frames 3) emphasize parallax (depth). Frames 4) allow the subject to count the flying sea gulls. The distance of the jumped image of the stereoscopic image is created on the TV screen. The maximum and minimum templets of each stimulus are as follows: Image Pin Sea gulls Sea turtle Organ
Jumped 28.7 47 106 0
distance (cm) 89.6 89.6 135 28.7
Angle (deg) 2 2 3.3 0 -
3.2 2.2 4.2 2
Conventional 2D frames are used as reference frames for each type of images listed above.
4.3 Experimental Procedure S -~ X--,. A -,. D, --~ C -~ B' -~ A' -~ C -~ D' -~ B -~ S A to D correspond to stimuli 1) to 4), respectively, and A' to D' correspond to the 2D reference frames for stimuli 1) to 4), respectively. S refers to the condition in which the subject sits still with the eyes closed and is used as control. X is a frame presented to the subject for about 5 min. to accustom the subject to the stereoscopic images. The eight frames A to D and A' to D' are presented at random.
4.4 EEG Recording Procedure Recording methods are the same of Experiment-1.
831
4.5 Subjects Nine male and female subjects, 20 to 21 years old, participated in the experiment. 4.6 Experimental Results and Discussion Analysis of variance (two-way classification) was performed on the data edited in the nine cerebrum parts. The results of this analysis of variance are given in Table 2. The analysis of variance identified significant differences in the theta waves between the displayed stimuli and between the cerebrum parts and in the stimulus-2D/3D interaction. We continued the analysis with the same methods for the Liquid Crystal experiments.
Table 2. Results of analysis of variance. FACTOR
S.S.
D.F.
M.S.
F-VALUE
499.84
3
166.61
3.80 ***
B
26.87
1
26.87
0.61
C
3442.74
8
430.34
9.81 ***
(p<0.01 )
A*B
372.91
3
124.30
2.83 **
(p<0.05)
B*C
18.33
A
A*C 394.70 A*B*C 162.55 E 25264.78 T 30182.71
8
2.29
0.05
24 24 576 647
16.45 6.77 43.86
0.38 0.15
(p
A • The effect of the experimental conditions B • The effect of 2D and 3D C • The effect of the location of head
The 3D sea turtle and organ frames produced higher potentials in the fronto-central and parietal parts than in the other parts. The 2D and 3D pin frames produced high potentials in the fronto-central part, and the 3D pin frames produced low potentials on the left side of the occipital part. The sea gull frames produced high potentials in the fronto-central part. The results of analysis of variance showed significant differences in the theta waves between the stimuli and between the cerebrum parts. Especially under the influence of the stimulus-cerebrum part interaction, the sea turtle and organ frames produced high potentials from the fronto-central to the occipital part. The high theta waves appear in the frontal part to the occipital part when the jumpout (depth) of images is under 30 cm and above 106 cm. Five of the nine subjects showed theta waves in the frontal-midline region in the background EEG. This tendency is likely to develop when people concentrate their attention on mental tasks (Schacter 1977; Yamamoto and Matsuoka, 1990).
832
Theta Wave
TURTLE
PIN
Figure 3.
3D
2D
3D
2D
3D
2D
3D
2D
Difference of location of EEG potential in theta wave
The jumpout of images is considered to affect the appearance of theta waves in the EEG. Concentration on the image frames in particular varies with the jumpout of the images. When we watch stereoscopic images of limited jumpout, we will suffer from a visual load. The experimental results of this study show that the jumpout of image frames should not be too small nor too large. It is suggested that images with a jumpout distance of under 30 cm or above 90 cm increases the visual load on the viewer and that images with a jumpout distance of 30 to 90 cm do not increase the visual load on the viewer. 5. Conclusion In the EEG of the subjects who watched stereoscopic TV images, theta waves preferentially appeared in the fronto-central to occipital regions. This finding agreed with the EEG characteristic observed when concentration heightens. The EEG characteristic is related to the jumpout of stereoscopic images. The subjects demonstrated an increase in their concentration for images of excessive jumpout and images of no jumpout. References Schacter, D.L.: EEG theta waves and psychological phenomena: A review and analysis.
Biological Psychology, 1977, 5, 47-82. Takemura, H.: Development of camcorder for taking stereoscopic images. Eizo Joho ("Video Information" in Japanese), 1988, 1, 61-65. Yamamoto, S. and Matsuoka, S • Topographic EEG study of visual display terminal (VDT) performance with special reference to frontal midline theta waves. Brain Topography, Vol. 2, No. 4, 1990, pp. 257-267. Yamamoto,S.,Matsuoka,S.,Yano,S.,:TOPOGRAPHIC EEG STUDY OF STEREOSCOPIC IMAGING TV(3D TV),DOKKYO KEIZAI,1992,Vol.59,pp. 164-176
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
833
64-channel EEG Measurement System - Applying to Stress Measurement Shin'ichi F U K U Z U M I NEC San-ei Instruments, Ltd. t 1-57, Tenjin-cho, Kodaira-city, Tokyo 187, Japan TEL: +81-423-42-6405, FAX: +81-423-42-6404, E-maihfukuzumi@j oke.CL.nec.co.j p
Summary The objectives of this study are to develop a 64-channel EEG data measurement and analysis system and to investigate the possibility of applying this system to stress measurement. The present system includes stressor generator, a 64-channel topography mapping method, an equivalent current dipole localization method and an electrode position measurement for the dipole localization. By this system, an equivalent current dipole for visually evoked potentials was localized within the visual cortex. It was found that this system allows us to measure the stress evoked by a mixture of auditory and visual stimuli.
1
Objectives
Stress is one of the most i m p o r t a n t problems to solve in modern society. It is difficult to solve a variety of the problems simultaneously because stress is arisen by many factors [1]. Firstly, the author defined stress, and studied methods for quantifying stress by physiological measures. In this study, the stress is defined as a change in human physiological and psychological tension condition. The tension condition is controlled by the autonomic nervous system which consists of hypothalamus and peripheral nervous systems. Activities of the peripheral nervous system can be monitored by finger pulse volume^[4] . To analyze the stress problems, it is necessary to measure not only peripheral nervous systems but also hypothalamus nervous systems, because the hypothalamus nervous systems control the peripheral ones. | Present division and address: Inforraation Technology Research Laboratories, NEC Corporation I-I, Miyazaki 4-chome, Miyamae-ku, Kawasaki, Kanag~wa 216, Japan Te]:-[-81-44-856-2155, Fax:~-81-44-856-2239
834 Activities of deeper cortices such as the hypothalamus could be visualized by a topographic mapping method and an equivalent current dipole localization methods from multi-channel EEG data [2][3]. Therefore, it becomes necessary to develop computer systems where these methods are implemented. The objectives of this study are to develop a computer system for measuring the multi-channel EEG data, especially 64 channel data and for analyzing the EEG data by the above methods, and to investigate the possibility of applying this system to stress measurement.
2
System
development
In this research, three systems were developed. First is a stressor generation system, second is a 64-channel topography mapping system, and the last is an equivalent current dipole localization system. 2.1
Stressor generation system
I
s o or I Finger Pulse Volume
Polygraph I
[Flash "
i-~
"
IIEEGAmplifier [I
G -!A DP 1100
Figure 1: Apparatus of the Stress Generation System
Auditory Stimuli Flash Stimuli
3-,-4~e
111
LLL
LILIL
1 Trigger
Figure 2: Stimuli appearance of stress-evoked stimuli To evoke stress which is defined as a change in human physiological and psychological tension condition, the stressor generation system is developed [5]. This system generates a mixture Of auditory and visual stimuli. The apparatus of this system is shown in Figure
835 1. The auditory stimulus interval is between three and four seconds. The visual stimulus is sometimes displayed between the auditory stimuli. In case that the auditory stimulus frequency between the visual stimuli is high, the subject feels irritation or tension [6]. The condition of stimuli appearance for stressor is shown in Figure 2. 2.2
64-channel t o p o g r a p h y m a p p i n g s y s t e m
To study EEG response by stress-evoked stimulus, the 64-channel topography mapping system works on Signal Processor DPll00. Evoked potentials were measured by 64 electrodes positioned on the grid points of subject's scalp. Figure 3 shows the configuration of 64-channel electrodes. In this figure, open circles mean no electrode. Each potential between electrodes was calculated by a spline interpolation function.
A
B )
•
FP1
kJ
F7
•
IF
..~F5
..,~
,. ~
qlF
~lF
A • 51 F
F5
13 A h 11 •
hT35 F
• L lIF
T5
•
F
qF
L
D ,,F
IIF
W
J
C k,/
• • q F
tB
• ,~ FC3 • W
CP5
• I)5
,IlL IF
CP3
H ()
.. F1
L/ • F
A&
FC1
LI& qlF
CP1
t ~M
. . , ol
t"'~
pol
F2
IF
d&
F4
IF
PC2
d &
F6
I •
~
• •
T24
FC6 • •
d
'q •
'q •
•
AlL qF
AL
CPz
Pz
ah
It, CP2
A qF
LIL
P2
qF
pOz
IF
N
d&
• •
qF
AL
t' ~,
,IlL
Ak
IF
•q F
qw
LL pl
A L 1)O3
dl ' q rk ~ ) ~
LVpV2 hFpF4
1:~
qr
5IF
k
dlL
dL 1~
L • T57
Fz
F
',~'
IF
qllF
• •
,,F
:F
IF
,ill IF
E
~,,J
d&
f'"~
AlL
CP4
1)4
IF
PO2
IF
o~
• IL qF
L h
dh
•
ILCP6
hP6
IF
PO4
• F
O
• IF
d •
d •
T46
• •
T68
( K
• •
02
Figure 3' Configuration of 64ch electrodes
r'X
P
r
Q
836 2.3
Dipole localization s y s t e m
GP-IB
DPll00
EWS4800
MD
HD
Figure 4: Apparatus of the dipole localization system Lastly, to study active condition of the cortex, the dipole localization system is developed (see Figure 4). This system works on Workstation EWS4800. Using each electrode position data and potential data at each electrode, current dipoles are localized.
3
Experiments, results and discussion
3.1
Evaluation of stressor generation s y s t e m ~
~
I
~
~~ Finger Pulse Volume
!ii iiiiiiiiiittf ilill............illiil i....i!i ~
~
~
~
~
~
T
~
.
.
.
.
.
.
~
Auditory stimulus
..
"~
EOG(Lu)
..-..............-.-~.-~.........~.......................... :..i.........".... '.......:..i:.......-....--...... .....................................................
.
.
.
.
.
.
.
.
.
.
.
.
.
EOG(Ru)
: .............................................................................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.
:
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..................................................................................~-?-~.: ?.~~
.
EOG(R1)
Figure 5: Relationship among auditory stimuli, visual stimuli and finger pulse volume The evaluation of stressor generation system was carried out.
The amplitude of the
finger pulse volume is known to reflect physiological and psychological tension condition. Therefore, to study the relationship between the mixture stimuli and the finger pulse volume is useful for evaluating this system. The result of this experiment is shown in Figure 5.
837 It was observed that the amplitude of finger pulse volume is decreasing with the interval of visual stimulus. From this, it followed that this system is useful as a stressor. In the next experiment, the visual stimulus after the auditory stimuli have displayed five times, is used as a trigger stimulus. 3.2
R e l a t i o n s h i p b e t w e e n stress and brain function
Figure 6: Topography mapping results: (left)64ch, (right)16ch
Estimated Electrode Positions With Fitting Sphere
D
,
°
°, ~,
°
Figure 7: Dipole localization results In the second experiment, the 64-channel EEG measurement ~nd the current dipole localization are carried out by the stressor generation system. The measured EEG data are
838 averaged for the topography mapping. This 64-channel topography mapping is compared with 16-channel one [7]. Figure 6 shows the results of topography mapping comparison between 64-channel and 16-channel. The 64-channel topography mapping could detect a detailed change in EEG amplitude which could not be shown by 16-channel topography mapping. The result of the dipole localization shows that a current dipole, which are marked by a large full circle in Figure 7, for the visual evoked potentials could be located within the visual cortex, which meets with physiological findings. This system might be used for localizing equivalent current dipoles for the stress-evoked potentials. 4
Conclusion
The 64-channel topography mapping could detect a detailed change in EEG amplitude by the stressor, and the dipole localization system might be used for localizing equivalent current dipoles for multi-channel evoked potentials such as the stress-evoked ones. It was concluded that 64-channel EEG measurement system, 64-channel topography mapping system and dipole localization system are found to be possible of applying to measure stress by auditory and visual stimuli. The present study was supported by MITI's Project on "Human Sensory Measurement Application Technology". References [1] Smith, M. J., Cohen, B. G. F. and Stammerjohn, L. W.: An Investigation of Health Complaints and Job Stress in Video Display Operations, Human Factors, Vol.23, pp387-400, 1981. [2] De Munck, J. C.: A mathematical and physical interpretation of the electromagnetic field of the brain, PhD-thesis University of Amsterdam, Amsterdam, 1989. [3] Souffiet, L., Toussaint, M., Luthringer, E., Gresser, J., Minot, E. and Macher, J.P.: A statistical evaluation of the main interpolation methods applied to 3-dimensional EEG mapping, Electroencephalography and clinical Neurophysiology,Vol.79, pp393-402, 1991. [4] Kamijo, K. and Kenmochi, A.: A Virtual Eeality System Using Physiological Data -Application to Virtual Sports CAI-, Advances in Human Factors/Ergonomics, 19B, Human-Computer Interaction: Software and Hardware Interface, pp675-680, Elsevier, 1993. [5] Fukuzumi, S.: Study on stressor by biological information, Japanese Journal of EEG and EMG, Vol.22, No.2, ppll8, 1994 (In Japanese). [6] Fukuzumi, S.: Study on stressor by biological information, Japanese Journal of Ergonomics, Vol.30, Supplement~ pp242-243, 1994 (In Japanese). [7] Fukuzumi, S.: 64-channelTopography Mapping of Stress-evoked Potentials, The proceedings of the 24th Conference on Japan Society of EEG and EMG, pp246, 1994 (In Japanese).
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
839
ANALYSIS OF BRAIN ACTIVITY F O R HCI M a r i k o F u j i k a k e F u n a d a - a n d S a t o k i P. N i n o m i j a b
~ D e p a r t m e n t of
M a n a g e m e n t , H a k u o h Univ.,
1117, Daigyoji, O y a m a - s h i , Tochigi, 326, J A P A N b F a c u l t y of Science a n d E n g i n e e r i n g , A o y a m a G a k u i n Univ. 6-16-1 C h i t o s e d a i , S e t a g a y a k u , Tokyo, 157, J A P A N 1.INTRODUCTION HCI (Human-Computer Interaction) is a filed which treats a kind of communication methods between computer systems and human systems controlled by brain. If we wish to have easier, more comfortable and collecter interaction with computer systems and make clear that we feel what is easy and what is comfortable. There are, at least, three approaches to make the characteristics clear: one is to consider human systems as a black box and to be only connected with the characteristics about inputs (like indications on a CRT display) and outputs (behavior) of human systems. The second approach is an analysis of brain activity under tasks of HCI because a brain is a CPU of human system and controls the total behavior of the human system. The third approach is an analysis both of them; response characteristics of human systems and activity of brain. Since true nature of a human system determi,es what is suitable to HCI method, the second approach should be tried at first. From this point of view, we made our purpose to take out brain-level characteristics of human systems under a simple HCI task. In order to attain the purpose, we use electroencephalograms (EEGs), especially event-related potentials (ERPs). Because EEGsshow us an functional aspect of brain activity, furthermore ERPs are direct responses of brain to the given task. Through discussions about the results of analysis, we consider a course of future HCI and propose a method to realize more suitable and comfortable HCI.
2.METHODS
2.1.Object data Object data are electroencephalograms (EEGs) when five normal students were doing a simple
840
BI~,
.-
• "
•
"
,m
. ~ _ _
the most abnormal Five kinds of facial contours used for stimuli.
the most normal Fig. 1
task under the following conditions; (D stimulus : 5 different kinds of facial contours from normal to abnormal (Fig.l). ~) frequency of stimulus: I time by 4 seconds (0.25Hz). ~) length of display: for 2 seconds. ~) numbers of stimuli:250 times. The five kinds of stimuli are included in 50 times each. displayed order of stimuli: at random. task: push a key "1" if a subject judges a displayed stimulus is normal, and push a key "2~ when the subject decides the stimulus is abnormal. ~) EEGs: 16 channels measured by 10-20 electrode system and monopolar. (~) sampling frequency for analogue to digital converter : 1KHz. 2.2.Analytical methods When a task defined above is given to a subject, two kinds of event related potentials appear on EEGs: one is a visual evoked potential (VEP) and the other is an event related potentials (ERP) in a narrow sense. We made object data Ploo of VEPs and Psooof ERPs, which appear at about 100 msec and 300 msec after a stimulus is given. VEPs and ERPs are usually measured by an averaged method because poorness of the signal-to-noise ratios of these potentials, but we made a kind of adaptive filter to measure the potentials by s single stimulus(I). In order to grasp changes of VEPs
and ERPs, a single
stimulus measurement is required. We used the adaptive filter to measure single stimulus VEPs and ERPs. The analytical methods are followings; EEGs under task are converted from analogue to digital (A/D) with 1KHz. ~) The digital EEGs
are filtered by the adaptive filter whose cut off frequency is
appropriate to measure P~ooand Psoo(1). ® Latencies and
amplitudes of Ploo and P3oo are detected in the filtered data.
The detected latencies and amplitudes are drawn in a (latency, amplitude)-space. The detected latencies (latency, amplitude,
and
time).
amplitude
are drawn in a three dimensional space
841
Fig.2 An example of original data. The waves are EEGs when the m a t abnormal stimuli are given. The numbers of 1 ~ 16 mean positions of channels.
Fig.3 The filtered data of Fig.2. • " candidates of Ploo O • candidates of P ~ .
.=~:~I~. ...,,,~E~~~
ii ampl
lude
• ' •
Fig.4 An example of changes of latencies amplitudes. Stimuli: the most abnormal ones. Channel: the 4th.
.
'
'
-'atenc~ 51818
msec
Fig.5 An example of 3 dimensional expression of Ploo and P3oo. The data are the same as ones in Fig.4.
3.RESULTS Fig.2 is an example of our original data; A/D converted EEGs when the most abnormal stimulus is given. The figure is one, seeing from top of head, and the numbers in the figure represent the channel numbers. Noises with higher frequencies are included and the wave in the 14th channel includes large amplitude noise. Fig.3 is the filtered data of Fig.2 The higher frequencies noises are decayed, and peaks of Ploo (0) and P3oo(O) are detected. Fig.4 is an example of changes of latencies and amplitudes during 50 times stimuli. The used stimuli are the most abnormal one in Fig. 1. The horizontal line is latency, and the vertical line is amplitudes. This figure represents a kind of characteristics of human systems; the same input (stimuli) dose not cause the same response. In another words, the system is always fluctuating.
842 100
0 :
I
I
:
0
I
,
10@
200
I
,
I
,
200
300
I
,
I
,
30@
400
I
,
I
:
400
500 msec :
I
0
-I
100
h
I
500 msec
L
.
,
•
0
:
I
200
I
~..
I
,
10e
,
I
,
200
300
I
,
I
,
300
400
I
500 msec
I
'
,
-I
H
400
500 msec
Fig.6 An example of 50 original data when the most abnormal stimuli were given.The first wave in left side is EEGs when the first stimulus was given. The last wave in right side is EEG when the 50th stimulus was given. 0
I
0
:
108
I
,
I
,
100
208
I
,
I
,
200
,
I
:
300
Fig.7
380
I
480
I
,
I
;
400
I
580 msec ,
13
I
I
500 msec
I
13
:
180
I
,
I
,
100
200
I
,
I
,
200
380
I
,
I
,
300
408
I
,
I
:
400
I
50@ msec ,
I
I
50@ msec
The filtered data of those in Fig.6. • : candidate of Ploo. C) : candidate of P3oo.
Fig.5 is an example of the three dimensional expression oflatencies, amplitudes and time. The data in this figure is the same of Fig.4. The fluctuation oflatencies and amplitudes of Ploo and P3oodraw large and small spiral in the space. This is one of characteristics of the fluctuation.
843
1.8
"
1
g
..................................................
I.(~
o.8
~ ~ iiiiiiiiiiiiiiiii!iiiiiiiiiiiiiiiiiiiiiiiiiiiiiii~. 8
0.6
0 6
~.~.
"
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
0.6
~ ~..iiiiii..iiii......... i.i.......... o.4
,I,l ,I i ,,I,IIIIi 2i I:,Ii, i.ii , +1111 ,lUll IIIUlI,II nulUUll IIIIIIII!
II "],.i,l. i,
o lUll,llll lUlllllllllUn IIII @
1.@
2@
3@
4@ times
(a) Uariance of data
~
1~
28
3@
48 times
(b) S i g n a l - t o - n o i s e r a t i o of data
Fig.8 An example of dynamic changes of variance of data and signal-to-noise ratios. 4.DISCUSSIONS W e consider and discuss about the characteristics of h u m a n systems. Fig.6 is an example of measured E E G s when the most abnormal stimuli in Fig.1 were given. The waves in the figure do not resemble mutually. Fig.7 is the filtered data of fig.& The marks Q and O mean that position of the candidates of P1oo and P3oo The changes of positions of each peak also represent the fluctuations of brain activity. To find out the reasons of the fluctuations, we defined a signal-to-noise ratio as a ratio of variance of original data to variance of true P1oo and P3oo. But true P1oo and P3oo can not obtain, we use the 50 times averaged wave in Fig.7. The calculated signal-to-noiseratios are in Fig.8. Changes of signal-to-noise ratios are comparatively large, and the variance of the signal-tonoise ratios is one of factors coursing the fluctuations. In order to decrease the variance of signal-to-noise ratios, we use a moving average method to the data in Fig.7. Fig.9 is the results of the moving average with 9 data. The waves in Fig.9 decrease variance of each wave and tendency including the data is obtained. From these results of analysis, the characteristicbehavior of brain which is a C P U of h u m a n system, is to change dynamically. Even if a same stimulus (input) is given to a h u m a n system, the same response (output) are not almost obtained. This fact characterizes activitiesof h u m a n systems and the spiral changing determine responses of h u m a n systems. Since the results of the analysis show that what h u m a n systems feel more comfortable is not to disturb the originalfluctuation, and to follow the fluctuation, we consider that the fluctuation
844
0
,
•
0
,-
100
100
'
I
200
,
200
I
300
300
:
I
40
:
400
500 ms
-I
500 msec
I
0
500 msec
,
I
100
,
I
200
,
I
300
,
I
400
:
I
500 mseo
Fig9. The smoothed data of those in Fig.7. A number of data for smoothing is 9.
on HCI is a keyword for future courses of HCI. There are probably lots of methods to accomplish such fluctuation. One of concrete methods to realize the idea is that we make response time of computer system changeable following the response time of human system. And another method is to make the size of displayed character and color changeable. Other environmental factors like sound, light, etc., are possible to realize fluctuating HCI.
5.CONCLUSIONS From the above analysis and discussions, the conclusions of this paper are followings; (1) We analyzed single stimulus recording Ploo (VEP) and P3oo (ERP) under a simple task of judgment. (2) Both of potentials Ploo and P3oochange dynamically drawing large and small spiral loci in a (latency, amplitude, time)-space. (3) Fluctuation is one of keywords of future HCI. And in order to realize such a HCI, we propose "a fluctuation interaction method" which can follow the changes of brain activities oh human beings. REFERENCES (1)Mariko Fujikake Funada, Satoki P Ninomija, Kenji Suginome; "Evaluation of mental stress by single measurement of ERPs "; the Japanese journal of Ergonomics, in contributing.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
845
Detection of the Event Related Brain Potential and Its Application to Communication Aids. Takashi Kawakami, Michio Inoue*, Yasuhiro Kobayashi*, and Kenji Nakashima** Tottori SANYO Electric Co., Ltd. *Department of Information and Knowledge Engineering, Faculty of Engineering, Tottori University. **Institute of Neurological Sciences, Faculty of Medicine, Tottori University.
This paper first introduces handicaps of ALS patients and appeals the necessity of CA equipment for them. Then, its description is assigned to explain the property of ERP deflection and a method of detecting it. Further, this paper referrs to use the ERP deflection for CA equipment.
I. Introduction There live sick persons who are damaged in their physical abilities, though retaining their intelligent activities. ALS (Amyotrophic Lateral Sclerosis) is an incurable disease with progressive changes led to a miserable situation where its patients cannot speak even a word in spite of still maintaining his healthy senses and mental capacity. Communication Aid ( CA ) is just the equipment that provide facilities for communication between such a handicapped person and us. Recently, various type of CAs has been developed [1] - [3]. Meager communication can be undertaken as a sequence of affirmative response for our inquiry one by one. Our inquiry comprises a number of complaints and requests in a hospital life. Thus, the patient has only to send a sign YES if he finds out the agreeable suggestion. In our previous type of CA, a partner of communication was requested to shut his eyes as his affirmative response. For its detection, electronic sensors have been used. A strain-gage varies sensitively its resistance corresponding to its mechanical distortion.
846 Unfortunately, any physical motion in serious cases becomes too weak to be detected. Since there still remains his mental capacity, responses for visual stimuli will be seen in his brain wave as an evidence of mental activity. Patient's ideas can be understand. This is our motivation of the present work. Its details are given as follows.
Grounded electrode Conduction electrode
A1
A2
Standard electrode Fig. 1 The disposition of electrodes Electroencephalogram is the record of the timedependent voltage difference between Pz (or Cz) and A 1, A2.
Fig. 2 Example of the brain waves ( at the conduction electrode Pz )
2. The brain waves and ERP variation
The brain wave is an electronic potential variation corresponding to the cerebral activity, and measured at the set up of the grounded electrode Fpz, the conduction electrodes Cz, Pz, and the standard electrodes A1, A2. Their disposition is shown in Fig. 1. An electroencephalogram is the records of the brain waves with respect to time. The brain waves consist of various vibrations and deviations. Among them, there exist deflections as the cerebral responses for an external stimulus. These deflections, called Event Related brain Potential (ERP), are classified as N100, P200, N200, P300 etc., according to their polarities and their incubation periods. The P300 deflection is the positive variation that appears at most 600 milliseconds behind than the time when a target stimulus is applied. Here, a target means an object of thinking. The receiver concentrate his attention on the perceived event to clarify its attributes and to evaluate its effects and influence. The time-lag is interpreted as the information processing. This
847 predicts that we can know what a partner in conversation has much interest in, and which is the object of his consideration. In fact, the P300 deflection is too little to be detectable on a straightforward measure. Fig. 3 shows electroencephalogram for external stimuli [A], [B], [C]. ERP deflections are hiding under the electroencephalogram. As a possible step, we can apply a method of synchronous addition to drag out the P300 deflection. That is, recording of the brain waves is repeated at a rate of at most 1 second per lap, and the records on every lap is piled up one another, as if all the records started at the same o'clock. According to our experimental investigation, with at most 10 times of repetition of the synchronous addition, random wavelets are cancels with one another and regulated ones become emerged from noises. As seen Fig. 4, the P300 deflection for a target stimulus [B] is emphasized by means of repetition of synchronous addition.
Fig. 3 Electroencephalograms for defferent external stimuli [A], [B], [C], where [B]: target stimulus [A], [C]: non-target stimulus, respectively.
) Fig. 4 ERP deflections after repetition
[A]
of synchronous addition .,-
0
V
•
v
400
T IME ( m s e c )
848
3. Application to CA. Remember here that a companion to talk to us loses entirely his physical power of oral speaking, though retaining his cerebral activities. In other words, we cannot expect any visible response from him, even if we throw some questions to him. We are now ready to perceive his psychological response within his brain waves. With this clue to go upon, we can understand his ideas. Our conversation is forwarded in a style of the answers vs. questions. Our topics are arranged by conceiving our companion's daily life in hospital. A list of our questions is shown in Table 1. Example of ICONS is shown in Fig. 5. Table 1. Examples of questions to patients (1) Inquiry of the patient's requests CHANGE the POSTURE
(2) Inquiry of the ill conditions
?
STIFLING
WIPE the SWEAT
9
CHILLY
CLEAR my PHLEGM
?
STUFFY
CHANGE the DIAPER
?
HUNGRY
LOOK into RESPIRATOR ?
THIRSTY
RUB down the BACK
?
PAINFUL
MASSAGE my WAIST
?
ITCHY
WIPE off SALIVA
9
NAUSEOUS
WET my LIPS
9
DIZZY
v
m
"!"',7"
Fig.5 Questions in a Package form
H$
849 Since our companion has no means to tell us his complaints or requests, we have to anticipate his appeal and to ask for his agreement. These questions are displayed on the face of a CRT as telops. If necessary, we can exhibit icons instead of literal expression. Further, if he want to write something, he request to choose characters necessary for composition. Thus, an architecture of the CA consists of (1) electroencephalograph as a psychological data acquisition unit, (2) a data processing unit for detecting the P300 deflection, (3) a CRT display unit for a patient and that for physicians or nurses, and (4) another Devices. ( See Fig. 6 ) Display unit for nurse Display unit for patient
Voice synthesizer ~ Electroencephalograph
Printer
~
~
I
I
Infrared beam Nurse Call
(D Processing unit Fig. 6 Configuration of the CA In general speaking, a person with long experience in his work can relieve his psychological tension as much as getting skillful with his operating. Besides this, repetitive exercises yield instinctive reactions against external stimuli without consideration. People had nothing to do, when they heard of boy's calling "wolves come on". Such affairs may happen in the CA base on the P300 deflection. It is because psychological relaxation dulls the P300 deflection, making us have difficulty in finding out our companion's affirmative response. If missing his answer, we are no longer possible to continue our
850 conversation. Some contrivance is needed so that concentration of his attention does never be disturbed. On the other hand, it is important to disturb his learning in the scheme of answers vs. questions. Random throwing of questions is an appropriate scheme in this CA system. For speeding up the tempo of conversation, we would like to shorten the period of the exhibition and to cut down the time of repetition of the synchronous addition. According to our experimental investigation, we can repeat question at intervals of 500 milliseconds. 4. Conclusion
This paper has reported the fact that there live sick person who cannot speak even a word due to loss of physical capacity, though retaining intelligent activity. For understanding their ideas, this paper has proposed a communication aid based on a scheme of answer vs. questions. A companion's ideas in our conversation can be confirmed by the P300 deflection within his brain waves. Thereby, we can obtain useful facilities for communication between serious sick persons, through never fluent. References
[1] T. Tokunaga, M. Inoue, Y. Kobayashi, N. Kanou, K. Inoue: "Design of a communication Aid for a Patient with Amyotrophic Lateral Sclerosis", IEICE report, CAS 87-26, pp. 1-8, 1987, ( In Japanese ) [2] M. Inoue, Y. Kobayashi, N. Kanou, K. Inoue: "A Method of Word Processing for a patient with Amyotrophic Lateral Sclerosis", Trans. oflPSJ, vol. 33, No. 5, pp. 645-651, 1992, ( In Japanese ) [3] N. Kanou, M. Inoue, Y. Kobayashi, S. Inoue, K. Inoue :"Detection of Winking for Communication Aids", IPSJ SIG Notes, vol. 94, No. 74, pp. 9-11, 1994, ( In Japanese ) [4] e. g., L. A. Farwell and E. Donchin: " Talking Off the Top of Your Head : Toward a Mental Prosthesis Utilizing Event-Related Potentials", Electroencephalography and Clinical Neurophysiology, vol.70, pp. 510-523 (1988) [5] Y. Oishi, M. Inoue, Y. Kobayashi, N. Kanou, K. Nakashima, T. Kawakami: "A Communication System Utilizing Event Related Brain Potentials ", 48th Annual meeting of lPSJ, 1-409, 1994, ( In Japanese ) [6] N. V. Thakor: "Adapting filtering of evoked potentials", IEEE Trans. Biomed. Eng, vol. 34, No. 1, pp. 6-12, Jan 1987 [7] S. Nishida, M. Nakamura & H. Shibasaki: "Method for single-trial recording of somatosensory evoked potentilas", J. Biomed. Eng., vol. 15, pp. 257-263, May 1993
IV. 10 Organizational and Psychological Aspects
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
853
A basic experimental study on mental workload for human cognitive work at man-machine interface H. Yoshikawa, H. Shimoda*, O. Wakamori** and Y. Nagai*** Institute of Atomic Energy, Kyoto University, Gokasho, Uji-shi, Kyoto-fu, 611 Japan
A model VDT experiment simplifying actual HCI situation at MMI, was conducted, where two subjects participated in experiment to solve the same cognitive task in competition. The experimental parameters were (i) different kinds of cognitive task, and (ii) cycle time of information display, to see the MWL influence on various biocybernetic signals. A special procssing unit for eye camera was developed and used for measuring subjects' eye movement characteristics. Regarding analysis, total information needed for problem solving was taken as anchoring measure for MWL, and the applicability of biocybernetic method for MWL estimation was evaluated from two aspects : (i) efficiency of visual information acquisition, and (ii) difficulty of inner cognitive process to solve problem, both in time pressure situation. It resulted in that eye movement were correlated to (i) , while heart rate, (ii) . 1. I N T R O D U C T I O N As regards to discussions on harmony between human and machine, studies on mental workolad for scaling workload of human cognitive works at man-machine interface have been made from various aspects [1] . In the present study, the authors assumed that the cognitive behavior of process operators who perform process monitoring and control through man-machine interface as "data-driven, realtime, online problem solving activity " or "understand and judge state transition rules of process based on the information sequentially presented on VDT interface". Then they conducted a model laboratory experiment which very much simplified the real situation, although the situation setting was the same type. Basically, the mental workload (hereafter "MWL") has miroscopic changes during the course of problem solving. But in the present study, it is assumed that the MWL is the integration of this change for the whole time span until problem solving. The resultant MWL is assumed to be given by the ratio of A/B, where A is total presented information quantity required to solve the
*Present Address : ShimadzuCo.ltd., Kyoto-shi, Kyoto-fu, Japan **Present Address : Toshiba Corporation, Kawasaki-shi, Kanagawa-ken,Japan ***Present Address : Sharp Corporation, Tenri-shi, Narak-ken,Japan
854 imposed task, and B is the minimum information quantity originally sufficient for solving the problem by ideal solving strategy. Since B is constant if the cognitive task is clearly defined, therefore A becomes the experimental index which reflects the actual level of MWL (A~) . In the present study, the information quantity A is taken as anchoring measure of MWL to anlyze the experiment, where two experimental parameters to influence problem solving process are (i) cycle time of presenting information on VDT and (ii) different nature of cognitive tasks imposed to the subject. Concerning the relation of biocybernetic measures with MWL, we especially noticed eye movement and heart rate, and the data analysis were conducted to investigate whether or not any relationships between their characteristics and the anchoring measure of MWL. 2. OVERVIEW OF EXPERIMENT
2.1. Material for Cognitive Task Experiment The material used for cognitive tasks is a state transition model of three-input, three-state as shown in Figure 1. Three figures of (S), z~ and [-7 in Figure 1 are "states", while key numbers 1, 2 and 3 and the arrows attached to them are "input keys" which bring figure change according to the arrow direction. Using this state transition model, three kinds of cognitive task were prepared and given to the subject who will solve the problem automatically dislayed on VDT screen by personal computer. There are two types of cognitive task, namely (i) "learning problem" to realize the rule of this state transition model, and (ii) "selection problem" to distinguish a right one from a set of given alternatives. 2.2. Protocols from Subject In the experiment, the subject were asked to wear eye camera and to attach many electrodes for polygraph to measure various kind of biocybermetic signals. Those biocybernetic signals were automatically recorded by computer. The data on the problems given automatically to subject by computer, and subject's responses from touch panel and keyboard were also automatically recorded as operation records. Concerning verbal report, two kinds of verbal protocol were collected from the subjects, think-aloud protocol during the execution of each cognitive task, and retrospective report after the experiment.
1 nitial
Key••re 0
A
~
1 0 I I A
//
"0
State transition model
z
AAO
3
IIAll
Transition rules
Figure 1. Illustration of state transition model used for cognitive task experiment.
855
2.3. Major Features in Experimental Program The major features are summarized below concerning experimental conditions and procedure : (1) Experiment setting to the subject is "competitive experiment ", where two subjects tried to solve the same cognitive task competitively. This is to assure their efforts and attitude to solve the problem more seriously. (2) Nature of problem solving is "passive monitoring", i.e., he/she was asked to solve problem by observing sequential changes of state transition automatically displayed on VDT by constant cycle time. (3) The sequence of automatic change was designed beforehand so that the problem difficulty to the subject can be reflected on the observed number of display presentations until problem solving. (4) The cycle time for automatic key change was set to three group with the order of 3 seconds, 2 seconds and 1 second. The slowest cycle time ones given in the first phase was to familiarize subjects to the problem. (5) Retrospective report were collected from subject by using the same format sheet by which major differences among different subjects can be compared on problem solving strategy and impression of problem difficulty. (6) Each subject took the same experiment two times by changing days and the partner. This aims at mitigating more or less inevitable factors such as physilogical change, experience, and to reduce common tendencies caused by preset experimental parameters. 2.4. Experimental Method There are three experiment series in the present study. According to the type of cognitive
Own d a t a
Automatically changing
Your Current Prob.No
CycleJ, N o. 15
Partner'sdata
~
v_
Partner's Current Prob.No
I No.5 I
I No.7 I
Your RightAnswer Rate
Partner's RightAnswer Rate
180%1
I 73%1
Tie'erY°u'Pa n0e° ner m I 2 II I I
3
L
Time After stert
L
Select one of the three belowwhen you understand. Sample1
£,
@_----Do
Sample2
I Sample%
N
@=%
Figure 2. Example of VDT display for Experiment Series B2.
856 tasks, both B2 and B3 are "selection problem", while B4, "learning problem". There are multiple numbers of test for the three experimental series, and for each series, two subjects start at the same time, solve the problems of each test in turn, and the series ends when both finish all tests. The example of VDT display is shown in Figure 2 for B2 experiment. In Figure 2, when either of key number 1, 2 and 3 at the "Automatic changing part" is selected, the figure in the center part changes according to state transition model. Also as shown in Figure 2, progress of test numbers and right answer rate for both subjects are always displayed on the screen, and beep sound and change of display background color tells the subject when the competitor finishes all tests. The subjects who participated in the experiment were P, Q and R, all of them students in psychology course. 3. DATA ANALYSIS 3.1. Records of Performance The operation records of all the sujects P, Q and R, were summarized for experiments B2, B3 and B4, with respect to (i) total numbers of display presentation until problem solving and (ii) right answer rate. An example is shown in Figure 3, for subject Q. The results are depicted on two graphs for each subject" one for the first time and the other for the second time of the three experiment series. 3.2. Reduction of Anchoring Measure of MWL As seen in Figure 3, the variations of total numbers of display presentation until problem solving against experimantal parameters (differences in cycle time and nature of cognitive tasks) have the same trends for all subjects, notwithstanding the difference in experiment days : they increase with acceleration of cycle time, and for the difference of experiments, B4>B3>B2.
First Time(Subject Q)
I; "
200
--Cycle No. I .... Rightscore/
Second Time (Subject Q)
I ~ Z6 t I "~ ~ 200
~~Cycle No~._-" I e-- ~,/~.... RightscoreII i100 ~ l "~
180 160 140 120
,o
:::F
,o
40
~I
40
20
40
20
100 80 60 40 20 0
2O
I/3 I/2
n
i " Cycle of display (see"I)
O
•
I/3 I/2
I 0 Cycle of display (secI)
Figure 3. Number of display presentation and right answer rate versus cycle of display for all experiments (Subject Q) .
857 Since "total number of display presentation" is equivalent to "total information quantity needed until problem solving", this will be used as anchoring measure of MWL, and will be used for correlating it to the responses of various biocybemetic measures.
3.3. Interpretation of Eye Movement Characteristics Based on (i) a biologically supported knowledge that "there is no perception by visual system during saccade motion" and (ii) a hypothesis that "there is a correlation between omission rate of presented information and eye blink proportion "deduced from preceding study [2] , effective information input rate denoted as FM (a measure related with efficiency of visual information aquisition) was estimated by using eye movement data obtained by eye camera with its data processing unit. The FM was calculated by the following equation FM=Fx
(1-Sf)
where S f = saccade fraction, and F -- proportion of eye blink time interval distribution which is slower than presentation cycle. Figure 4 shows the calculated dependence of F and FM for subject Q , on cycle time for all experiments B2, B3 and B4. As seen in Figure 4, there is a tendency that the FM tends to decrease as the display cycle becomes faster. This is in good agreement with the observed trend in Figure 3 that the number of presentations until problem solving increases as the display cycle becomes faster, and further it well corresponds with Q's impression that "it becomes hard to watch as cycle becomes faster". However, there is no significant difference in the FM as to the difference in cognitive task. The FM of Q is better than those of others, and moreover his FM is not degradated as cycle becomes faster. This seems to indicate the reason of his better performance than the others, already at the input stage of visual information acquisition.
3.4. Heart Rate Characteristics Heart rate is said to be governed by the opposing regulations b e t w e e n s y n p a t h e t i c and parasynpathetic nerve systems, and it is a well admitted fact that heart rate is influenced by cognitive activity. In view of this, analyses were conducted on average heart rate (m) and the standard deviation (o') during the time span of experiments B2, B3 and B4. The results of subject Q are given in Figure 5, where the two graghs show the variation of m and o" as fuction of display cycle. The avarage values of m and o" over
F 1.0
0.5
FM
Subject Q K
1.0
0.5
Exp. O B2 A B3 × B4
FM-F --I ]
I I
i
Cycle of display (sec-I ) Figure 4. Calculated dependence of F and F M on cycle of display for all experiments B2, B3 and B4 (Subject Q) .
858
the whole time spans of conducting each experiment, were also calculated as M and 5"., and the obtained values of M, Z and Y./M are tabulated in Figure 5. We can see from this table, that the value orders of Y. and Y./M are in good agreement with that of anchoring measure of MWL, that is, Bd>B3>B2, although the difference between B3 and B2 are very minor in value. Concerning display cycle effect to heart rate, there are no consistent trends seen in Figure 6 for m and (7 v e r s u s d i s p l a y cycle relationship. As a conclusion of the results mentioned above, we may reduce a Subject Q] hypothesis that 5". and Z / M would be a usable measure to distinguish MWL M X Z/M Exp. (beats/min) (beats/min) levels by different nature of cognitive task, although premature at this phase 99.4 4.05 0.041 B2 of study. B3 95.2 4.12 0.043 134
90.2
6.84
0.076
O ]32 A B3 XlM 7 6 s 0 4
100
O~
O
O B2 A B3 - - O x tM
4. CONCLUSION As a conclusion from a rather simplified basic laboratory experiment conducted for m e a s u r i n g human cognitive characteristics at MMI, we found out the significance of usage of multiple biocybemetic measurement for online, realtime estimation of various cognitives-related MWL characteristics of HCI at MMI, although the tested biocybernetics instruments should be further improved to the practical application for real MMI problems. REFERENCES
1. B.H.Kantowitz : Mental Workload ; in Human Factors Psychology (ed. P.A. Hancock) , North-Holland, (1987) 81. 2. H.Yoshikawa, H.Shimoda, Y.Nagai and S.Kojima,Trans. Inst. Sys. Cont. & Info. Eng., 3 (9.) (1990) 261. (in Japanese) .
90
80
I
I
1
1
5-
-fi-
I
1 Cycle of display (sec -1)
Figure 5. Average heart rate (m) and standard deviation ((7) during experiments B2, B3 and B4 (Subject Q) .
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
859
W o r k f l o w T e c h n o l o g y B a s e d Project M a n a g e m e n t Carlos K. H. Leung, Heloisa Martins Shih, Mitchell M. Tseng Department of Industrial Engineering, Hong Kong University of Science and Technology, Clearwater Bay, Kowloon, Hong Kong
1. INTRODUCTION Tough international competition and the trend to market globalization lead to shorter product life cycles, increasing complexity of production / services systems, and more decentralized decision making in the organizations. These changes result in faster evolution of technology, companies, markets and society. To cope with the dynamic changes and huge associated costs, companies and government are initiating more and increasingly complex projects. Therefore more robust software support is needed for the planning, monitoring, and control of these large and complex projects. Presently, many different types of project management software are available [1, 2], from PC- to mainframe-based, from general purpose to problem specific, each of them having its own strengths and weaknesses. The existing systems support a combination of various functions (e.g., Gantt chart, network diagram, PERT/CPM, resource allocation), and some even integrate spreadsheet functions to assist in estimation. The support is mainly related to the planning phase of the projects, being mostly inadequate in the functions of monitoring and control, in communication between different groups, and in information flow. We propose a workflow technology-based project management system, utilizing existing project management software functions for planning, coupled to a new workflow technologybased system for communication, coordination, and control. Workfiow [3] is an emerging network-based technology in information systems. Workflow systems are designed to assist groups of people in carrying out work procedures through a more effective utilization of the organizational knowledge about resource requirements and flow of work. Workfiow is reputed to be extremely powerful in specifying, executing, monitoring, and coordinating the flow of work within a distributed environment. Therefore workflow technology was selected to support the communication, coordination, and monitoring functions of project management. In order to integrate the planning functions supported by project management software and the monitoring and control functions supported by workflow, we map the elements in the project management software into workflow. To realize this mapping, we introduce a systematic approach utilizing Petri net (PN) [4] as a common modeling tool.
860 2. PETRI NET MODELING In order to establish the communication between classic project management elements, (e.g., tasks, activities, resources, precedences) and workflow elements (e.g., process, activities, roles and actors), we utilize a Petri net-based model. The model includes all the elements that are normally present in existing project management software, as well as elements that are normally not tackled in that software group (e.g., information furnishing resources like reports and specifications). The mapping of the Activity-On-Nodes (AON) network of the project into PN is quite straightforward. The AON elements are mapped through the association of tasks to PN transitions, of events to PN places, of precedence relations to PN net arcs, and of conditions to firing rules of Petri net. Figure l a shows the AON diagram of a Materials Selection Project example modified from [5]. Figure lb shows the corresponding Petri net model for the same project example. The Petri net model obtained through the mapping of the AON network represents the top management level of the project. In order to monitor the project during its production (implementation) phase, detailed monitoring of each of the tasks is essential. As shown in Figure 1, a task of the project network is mapped into a expandable Petri net transition. The expandable transition is stored as a workflow module in the database containing the information about resources and activities. Classical Petri net places and transitions are not enough to realize the modeling of the workflow module. Extensions to the classical Petri net have been proposed by us [6] to cope
A-Select Materials
~
A-SelectMaterials B-ObtainMaterials ms2,
B- Obtain Materials E-Experiment
\
START/ G-Build/I'''~jD" Debug x~tJ- Final Report
Q
"0""~r'x
0
-'0 ~ - umont i n :
STAR"
,0
~(~1 '~ms3-I
~
msl6
H-Theoretical Study /
G. Study Literature
O Activity
~
~ I-TheoreticalReport ',
Literature ~Precedence Arc
b)Petri net of the MSP example.
a)AON network diagram for the MSP example.
A- SelectMaterials ...................... Chemical Material Junior Engineer --.-
~~" ~
Data
Mt~ante°~Saels/4O~ , ~ . ~ \
....
Information Materials Selected
ro
Technical Supporter
Project Pr°lx)sal
Chemical Sr. Engineer
Start
c) Petri net graph of the Select Materials module.
Figure 1. Mapping of AON network to Petri net with expandable transitions.
861 with the special features of some workflow elements. The XOR transition and AND transition have enhanced the power of Petri net to model both decision-making (e.g. transition Approval in Figure l c) and non-decision-making activities (e.g. transition Choose materials in Figure lc). Role places (e.g. place Chemical Junior Engineer in Figure l c) represent resources that execute the work in the activity; deliverable places (e.g. place Project Proposal in Figure l c) represent the resources that furnish information for the activity. The workflow module proposed in this paper is modeled within the framework of these extensions to the classical Petri net. All the workflow elements of the modeled task (e.g., activities, roles, and actors) have corresponding Petri net elements (e.g., transitions, places, and tokens). This approach to the conception and representation of the project has two advantages. First, it is very close to how the project manager and task leaders think about the project; second, it forms the internal representation of the structure of workflow, which can be efficiently executed by the workflow system. A detailed description of the mapping can be found in [6].
3. SYSTEM F R A M E W O R K
The framework of the proposed system, as shown in Figure 2, has a number of software layers: the database, internal control, object mapper, application programming interface, and user interface. The database, internal control, and object mapper layers are the main focus of this End-User
I .roectWorOow4} t Management User Interfac
ser Interfac
User Interface
@
@
Workflow Management Inte~ace
Project Management Intei'face
API
@ # [ Object Mapper ]
# ] Internal Control ] ~ [ Figure 2. System framework.
Database
862 research; the application programming interface and user interface are to be mainly provided by existing project management and workflow software. Therefore, the interface with users and the typical functions of project management planning and workflow technology are not object of this research; they are considered already existing. An existing project management software is used to serve as a planning tool for project managers, including for example Work Breakdown Structure, Project network, Scheduling, Gantt chart. It also provides a higher level view of the project m one that can be easily understood by project managers. Instead of replacing the project management software, we are enhancing it with better support. The main purpose of integrating existing workflow software into our system is to have a workflow environment which can provide the project environment with different types of communication: • communication with end-user through enhanced interface • an information transfer communication protocol under distributed environment • communication with the central database • communication with other independent applications, apart from workflow. However, up to now there is no fully developed workflow software already released in the market which can support these needs.
3.1. The Database Layer The lower layer contains the data structures for storing the information of the project, its state, and the workflow modules, resources, and routines associated with them. The data structures have been designed with a large number of links between the different components so that very little time is consumed in searching through different lists. 3.2. The Internal Control Layer This layer acts as a server providing the execution algorithms of the proposed system. The algorithm used to execute the proposed system is similar to that used for the execution of a Petri net. It is fairly simple and straight forward. To begin with, when a place is marked, the status of that place becomes "true" in the database. Next, the transition that has that place as an input place is found and triggered. This is done by tallying the "ToTransition" of the marked place. The transition that is triggered and has all conditions satisfied is enabled and is placed in the database called "Progressing." However, unenabled transitions will remain in the "Waiting" state, waiting for the state of other status places, role places representing executor resources, or deliverable places representing information furnishing resources that are inputs to that transition to change. When the state of any input place changes, that transition will be triggered again. Therefore, a detailed picture of the present state of the Petri net is derived. Firing a transition includes updating the marking of its input places: "unmarking" the status places, and removing a number of tokens (corresponding to the power arc) from the role places; the deliverable places remain marked since they can be concurrently accessed by multiple users. Updating is realized in the database and reflected in the Petri net graph. When a completion message is feedback from the user, the system will recognize it as the firing is completed. Therefore, all the output places of the AND transition (classical Petri net transition) representing non-decision-making activity are marked: the status places and deliverable places receive one token each, while their state in the database becomes "true"; the role places receive the number of tokens corresponding to their power arc. For XOR
863 transitions representing decision-making activity, only output places satisfied by the outcome of the transitions are marked. After the complete execution of the selected transition, the databases representing the marked places and enabled transitions are updated. New transitions are triggered and the cycle is repeated. Instead of overriding the control of existing workflow software, our internal control acts as a transparent link, bonding the workflow and project management software together. Our internal control is an extension of the original workflow control, translating the control of projects into workfiow-like control, so both project management software and workfiow can share a common database.
3.3. Object Mapper Layer The object mapper provides access to information stored in the database by mapping different information to different user interfaces. Besides accessing the database, the presentation of information from the internal control layer to users under different user interfaces also needs the object mapper to do the mapping.
3.4. Application Programming Interface Layer (API) The Application Programming Interface layer (API) is built on top of the object mapper and is a utility library consisting of project management and workflow service routines. These routines are mainly functions for planning supported by existing project management software and for tracking, control and communication supported by a to-be-selected workflow software. Some routines may also be developed during implementation of the proposed system, if necessary.
3.5. User Interface Layer The user interface layer is the top l a y e r - where users will be interacting with the system. It consists of two parts: the project management user interface and the workflow user interface. These two interfaces are intended to be supported by existing project management software and workflow software. The project management interface is accessible only by the project manager and task leaders. This interface is used to plan the project. From the project management user interface, all utilities of project management in the API layer are supported. Project managers and task leaders can do simple monitoring within this interface, utilizing for example Gantt chart. The workflow user interface supports all users including the project manager, task leaders and team members. This user interface supports the detailed tracking and controlling of the project. Communication is also provided under this interface. Project managers and task leaders will further enhance the project using this interface. Team members will receive and send work information under this interface. All workflow utilities in the API layer are provided under this interface.
4. CONCLUSION The proposed system is an integration of all components included in the system framework. The project management interface and API are supported bycommon project management software, and the workflow management interface and API are supported by
864 workflow software. The object mapper, internal control and data structure are discussed in this paper as the major issue of the proposed system. We envision that the application of the proposed system will greatly modify the way and the composition of work of project managers, activity leaders and team members, because • changes in project (e.g. activity delays, design modification) and their consequences will be made known automatically to all related personnel; • all information related to the activity to be performed is routed and made available to the actor or performer when it is needed, increasing the productivity of work; • sensitive information is protected with different usage privileges for security, so that only entitled users can read, modify, or delete the information; • activities, tasks and privileges are linked to roles (e.g. designer, administrator) and not to persons (John, Mary), so that, when any person is not available, the new person assigned to the roles can proceed with minimal disruption; • routine tasks (e.g. processing a purchase order, processing a request form) are automatically routed and classified by the system; this way project manager and activity leaders have more time to dedicate to ad hoc situations demanding their expertise; • human-system interaction is better established through a user friendly interface and a strong, reliable support for all project phases; • better human-human interaction is established through better project communication and information flow, enhancing the team synergy. The proposed system offers a reliable information flow which provides strong support for communication, coordination, and control of the project. The structures of past projects utilizing the system are reusable due to the modularity support of the system. The proved strengths of existing project management software are incorporated. Moreover, the proposed system aims to bring people closer, to minimize conflict areas, and to support better decisions, thereby increasing the efficiency of the whole project.
REFERENCES
[1] [2]
[3] [4]
[5] [61
Avraham Stub, Jonathan F. Bard, and Shlomo Globerson. Project Management: Engineering, Technology, and Implementation, Prentice-Hall, Inc., 1994. Lois ZeUs, Managing Software Projects: Selecting and Using PC-Based Project Management Systems, QED Information Sciences, Inc., 1990. Clarence A. Ellis and Gary J. Nutt. "Modeling and Enactment of Workflow System", Application and Theory of Petri Nets, Springer-Verlag, 1993. James L. Peterson, Petri Net Theory and the Modeling of Systems, Prentice-Hall 1981. Milton D. Rosenau, Successful Project Management: A Step-by-Step Approach with Practical Examples, Van Nostrand Reinhold, 1992. Carlos K. H. Leung, "Workflow Technology Based Project Management", MPhil dissertation, Hong Kong University of Science and Technology, Hong Kong, 1995.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Moil (Editors) © 1995 Elsevier Science B.V. All rights reserved.
865
Involving workers in the transformation of w o r k organizations: Problems and tools Irene Odgaard General Workers' Union in Denmark Nyropsgade 30 DK- 1790 Copenhagen, Denmark Fax: (+45) 3397 1460 Phone: (+45) 3314 2140 Email: iodgaard @inet.uni-c.dk The emerging business environment of modern manufacturing is characterized by a new regime of market demands where factors such as the ability to adapt to changes, handle shorter through-put times, etc. become increasingly dominant. Manufacturing enterprises attempt to handle this challenge by reducing the number of levels in the organizational hierarchy, by changing the management style towards 'management by objectives', and by enhancing the responsibility and competence of operators in the executive functions of the organization. New management techniques tend to involve concepts such as KAIZEN (continuous improvement), flexibility, multi-skilling, and teamwork. This general trend makes the motivation of blue-collar workers a crucial factor. Accordingly, the dialogue between management and the involved workers concerning job design and the support provided for coping with the new tasks and responsibilities becomes imperative. This dialogue is very much supported by many trade unions. Perceiving prospects of achieving their own goals, Danish trade unions have traditionally had a positive, cooperative stance in relation to industrial growth, and they have, since the middle of this century, signed agreements on cooperation on the development of production with the Employer's Federation. The first of these agreements was signed in 1947. When the agreement on industrial collaboration was revised in 1970, it stated that "delegating responsibility and authority to the individual employee or group of employees in the widest feasible range is necessary in order to make it possible for employees to have a voice in the shaping of their own working situation and to exercise influence upon the process of decision making in the enterprises."l
The unions in Denmark have a fight to negotiate working conditions with the objective of industrial improvement and greater job satisfaction amongst employees. Since end of the Eighties, a concept has been evolving in the unions which aims at work designs that enable individuals to develop in their working life m in sharp contrast to the Danish
1
Dansk Arbejdsgiverforening og Landsorganisationen i Danmark (1970): Samarbejdsaftalen. Aftale om samarbejde og samarbejdsudvalg i virksomhederne. Vejledning og kommentarer, p. 4.
866 versions of systematic or scientific management developed after the second world war, concentrating on time and motion studies and more efficient exploitation of labor power. The core of the old 'Tayloristic' paradigm in industrial organization is the concept of labor as a passive factor of production. Planning and doing should be strictly separated. Other principles of the paradigm are: rigid standardization of working methods (avoiding 'rule-of-thumb' based on workers experience), fragmentation of work (jobs should be limited to the smallest possible task range), and consequently the minimization of job-learning time. All of it with the aim of doing away with slow working and of extracting the largest possible output from each worker and each machine. The concept of the unions in Denmark of 'work design for human development' has been inspired by different sources. In Germany, since the Seventies, projects have been launched in the program "Humanisierung der Arbeit". An important issue in Germany has been to secure the acceptance of new technology by workers due to the influential role of the Enterprise Council (Betriebsrat) in connection with development of the company. This has led to the development of the concept of "anthropocentric production systems" according to which technological systems are designed according to aims of a social as well as an economical and technical nature, are important. It has been demonstrated, that it is possible to design and build control systems, enhancing human skills and using labor as a creative, active force of production. Another source of inspiration has come from Sweden, where the Metal Workers Union in the Eighties developed a union strategy on production systems, introducing group work as a central element, and demanding, that jobs included tasks such as planning and control. Amongst the different types of interests and arguments raised to promote the strategy where the rising numbers of injuries from monotonous working conditions in the Swedish factories. Also, due to the high level of employment in the Eighties, employers faced difficulties in attracting and preserving their labor force, if they could only offer monotonous jobs. In Denmark, employers did not face problems like workers fights to veto new technology, or high labor turnover and absenteeism, but nevertheless they faced problems which could form the basis of an alliance for union efforts to restructure jobs and organization in the process of technological development. During the Eighties, several projects for improving the adaptability of manufacturing systems had been carded out in cooperation between Danish employers' federations and trade unions. The focal points of the projects had c h a n g e d - from technical to organizational and human resource issues. The importance of education, broader job scope, and delegating responsibility and authority to the individual employee or group of employees had been emphasized, but many companies admitted to having been too absorbed in implementing advanced machinery and too negligent regarding the necessary development of the organization. An survey of Danish Industry in 1990 showed that industry's two most serious problems were: • Problems related to organization of work and delegating responsibility. • The company over-estimated workforce skills in implementing new technology. Motivated and educated people were now recognized as one of the critical management resources - - but it seemed as if earlier management philosophy had made it a scarce resource.
867
The fragmentation of work and the minimization of job learning time had created a poor environment for workers creativity. This has been realized by one of the largest Danish industrial manufacturers. Since the mid 1980s, this company has attached great importance to the development of the organization: the employees' qualifications and motivation, the composition of the work and the way the work is coordinated. In the words of a director: Our philosophy from the mid Eighties was to concentrate on technology and investments in rationalization. It was necessary then but it turned out not to be enough. You also have to motivate the employees; there must be a balance. In the mid 1980's there was a shift in orientation and a far more pronounced focusing on the "soft" sides of the technological development such as forms of management, wage systems, qualifications, and the motivation of the employees. It had previously been attempted to develop the contents and work organization but at that time much more "from above" and on a different basis: We have previously tried working with autonomous groups but it was a failure in the 1970s. Neither the qualifications nor the cooperation relations were ripe. Actually, nobody wanted the employees to take the responsibility. It was in everybody's best interest if you "left your brain outside". Among the trade union representatives at the company there was at the same time an ongoing process which was to result in a far more active participation in and influence on the development of the company. From 1987 and onwards a number of courses, seminars and meetings took place with the participation of an external consultant. The discussions which took place at the Local Union's seminars, meetings and in the club committees led to a ripening of thoughts concerning a development of the work organization which would give the individual member more influence. In 1993 the Local Union formulated a number of "visions" with corresponding objectives, goals, strategies, and action plans. In the section on "Personnel Policy" it says: Vision for Personnel Policy
Strive for job security through stable employment and a positive tone and the employee's influence on his own goals and development. Create a framework in which the employee has the opportunity through training and commitment to the work to attain more responsibility and influence on the planning and implementation of the work according to the objective of the company. And to improve the individual's quality of life through activities which increase the employee's awareness of ergonomics, health and well being at work. (...) Objective To establish conditions which create a more open dialogue between the management and the employees. More delegation of jobs and responsibility, so that all employees play a part, not only in performing their jobs, but also in planning, preparing and controlling them. The efforts of the Local Union and the management are thus almost identical regarding the development of the work organization. Management wants a "flat organization", focused on the needs of the customers, with a quick reactivity, a short time from the order is placed until it is
868 fulfilled. This was the development which the management initiated in 1985. A manager put it like this: You can say that the goal is to make the employees so independent that they can keep themselves busy. How far should we go in regard to what other tasks they can carry out? I believe that using information technology they will also be able to confirm orders, plan, set quality standards, ensure delivery. The development of the production groups is seen by the Managing Director as a natural element in a development which is also supported by other efforts to make the employees take on greater responsibility for the development of the company: For instance the speed-up-programme, which was initiated just after Whitsun. We tell the employees: "It is no longer enough to say "we have already suggested that". Now it is also your responsibility to carry it out and follow up on the suggestion." This is the first time that the company officially from the top management has asked the employees to take on responsibility to improve first their own area of work, then their department's and finally other conditions which influence their work. According to the Production Manager, the reason is: that we believe that there is an enormous potential for improvement not only here in Denmark but also abroad in our other companies (...) Formalizing this structure and asking the employees to take on responsibility is not something we have said before. We used to say that it is the responsibility of the management or others to develop ideas. It is decided to start eleven production groups at the company. Although it is decided to start the "easiest" groups first, and a training course is offered, it is not an easy process. In the training course, the group is introduced to the objectives which the m a n a g e m e n t wishes rendered visible to the group: A production should show improved competitiveness through a positive development, employee involvement/competence, productivity, quality, time of delivery and costs. And it is the objective in the production group to develop the job contents for the individual m it must benefit both parties, not only one party. (Factory Manager) It is thus stressed that the objectives stem from mutual interests. The Factory Manager presents proposals for items to be discussed in this connection: • Communication between the group and the foreman m how should it take place? • Development of qualifications. • How to avoid "informal leaders"? • Handling of times of delivery. • Follow-up on the group's objectives m shall the leaders intervene? When? • Information to the group on work tasks. •
Handling of conflicts.
• How do we handle the need for capacity adjustment? What if your capacity goes up. That is a very good thing and then we need more people and we will pick out the ones we like, the good ones, those who can earn us some bonus - - and then what happens, when it is the other way around, who is to leave the group, how are you going to handle that? (Factory Manager Bjarne Neig~ard) • Handling of absenteeism and flexible working hours.
869 The Factory Manager suggests that the meetings which are necessary when the employees themselves are responsible for coordination etc. are somehow formalized. And he stresses the point that the agenda is open: you can bring up other items than the ones suggested, if the group feels the need to do so. In the course of the discussions which follow the presentation from management, the group finds out which resources m professional and personal m it contains in relation to the external objectives and framework. A set of "internal rules" is worked out and it is decided which areas can be covered by the group's own resources and which objectives should be set. On the last day an action plan is made, symbolized by a drawing on a poster: a tree with a number of "bubbles" in which the group indicates where it stands on quality, time of production etc. Finally, the poster is signed by all participants m including management ~ in order to render the change to another attitude and status as visible and symbolic as possible. In other words, the groups can discuss their way to ambition levels which vary and keep their own pace in the development. "Production groups" can mean very different types of tasks, depending on the group's attitude. This has been specifically stressed by the Local Union, as they have pointed to the great danger of "losing people" if you progress too rapidly. The product groups' independence generally consists in their ability to autonomously contact the quality department, repairmen, and sub-managers. In order to perform their tasks, most groups have brief morning meetings (5-10 minutes), brief weekly meetings with the participation of the shifts, foremen and sub-managers and finally longer "big meetings" (a couple of hours or more) as the need arises to solve various problems. Apart from that, the groups do not have new tools for planning. But even though work tasks have not changed much yet, it definitely is not easy. There is a difference in the indicated, positive attitudes to the objectives of the change towards production groups and to the actual groups as they are experienced by the employees: One thing is theory, another practice: "it was worse in practice than we expected", "in practice it is hard". In the words of the foreman: "It was very easy to just fill in the bubbles in the tree ~ but it is different in real life, when you return. We thought that after a month, we could do everything ourselves. But we could not." As previously mentioned, it was decided to start the "easiest" groups first. Therefore the difficulty of control varied between the different groups. This probably results in different terms for the productivity development as well. The last group to start "first had to find out how to control the production", as the foreman said. The first group thus had it easier and maybe this was a contributing factor to their clearly higher bonus ~ while this was not as clear for the later groups, some had even experienced a decline in bonus. Perhaps because they spent more time on the meetings which were now necessary to handle the tasks of control and coordination m meetings which are not held outside the bonus time. (Another factor is the fact that the task of converting cannot immediately be taken over by the operators, without a financial disadvantage.) Various difficulties in connection with the existing wage system clearly "scored highest" on the list of problems experienced by the production group members.
870 And the wage system is also at the bottom of a number of other problems: difficulties with cooperation among the shifts, lack of rotation and a too high rate of working. Concerning job satisfaction and working environment, the development is, as can be seen, complex. For a time the rate of working certainly increased to an extent which probably worsened the working environment. But there has also been a higher degree of control over time: the opportunity to plan your own work. People have been very satisfied with this part. The production responsibility has increased and so have the requirements for problem solving. How this influences the working environment depends on the preparedness of the individual, but the development provides a potential for jobs which live up to the objective of the Local Union to increase the access to influence personal and professional development. All in all, many difficulties were encountered in this transformation process. From this case, we can point to the following issues as problems which may be encountered generally: 1. The quality of the skills and tools available to employees with respect to handling the management and coordination their own tasks were not developed enough. The more complex the task of management and coordination, the more apparent the lack of decision and coordination support systems suitable for this category of users. 2.
3.
The conditions for team formation. On the one hand, management wants the team to assume the tasks and responsibilities that previously were at a higher level in the organizational hierarchy within a short time span. On the other hand, team formation is a process where the internal relations (ability to make demands, reject demands, and accept differences in opinion, style and performance) as well as the ability to take wider interests and goals into account in the daily activities develops over time and depends strongly on the background and experience of the participants. The contradiction between a traditional notion of efficiency by means of specialization on the one hand, and on the other hand the required increase in flexibility which may be impeded by the pursuance of local efficiency, especially when the particular wage system in the particular setting encourages efficiency at the cost of flexibility.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
871
Emotional W o r k L o a d : Its Operationalization, Measurement, and Consideration in the Design o f H u m a n - C o m p u t e r Interfaces Irwin C. Marin, Ph.D., Principal Scientist* The USER Institute of The Emblematics Corporation, Pittsburgh Applied Research Center, 1180 William Pitt Way, Pittsburgh, PA. 15238 USA ABSTRACT Mental Work Load and its physiological effects have been studied in a variety of environments using measures such as changes in heart rate, in pupil dilation, in galvanic skin response and so on. Physical Work Load which exceeds a person's capacity has been of continuing interest since the earliest days of medical/human factors research (such as the pioneering stress effect studies of Hans Selye). However the interactions between the physical and mental stressors manifested in the form of Emotional Work Load have neither been studied nor measured with the same precision for a number of reasons: • the difficulty of operationalizing the concept of emotional behavior • until recently, the lack of understanding of non-linear behavioral dynamics and emergent phenomena • the lack of precise tools to study the phenomena in its full multivariate form. This has left an important gap in the theory, understanding, and management of Work Load required to improve the performance of the current generation of intelligent computer interfaces and the design of their successors It is the purpose of this paper to briefly define the concept of emotional workload, delineate and operationalize some of the key concepts and examine their measurement, and from this perspective, suggest approaches for tackling the problem of designing "tunable" Human Computer Interfaces including those used by health care provider teams delivering health care services remotely via utilization of Telemedical Technology 1. INTRODUCTION Mental Workload has been studied extensively. Physical Workload also have been studied in the context of the mental workload. However physical factors are usually not separated from mental factors and further interact often non-linearly to create emotional workload effects often manifested which have non-negligible consequences on task performance especially * email: [email protected]
872 in critical mentally and physically demanding and stressful tasks where performance decrements cannot be tolerated. These deleterious effects appear to be especially present in tasks where automation and the intense use of computers are involved. It is therefore necessary to develop ways to study this important yet non-linear factor and incorporate it into the development of next generation Human-Computer Interfaces. Before this can done systematically the concept of emotional workload must be formalized and methods be suggested for study. 2. PRECURSORS OF THE CONCEPT OF EMOTIONAL W O R K LOAD Consider a system as a set of objects (people, processors etc.) called M coupled (connected) into a network by a set of constraints we will call system relations R so that we have a network we can denote by the symbol <M,R>. Borrowing concepts from physics let us say a system will perform "work" on some object in its environment if it can be considered to create an impetus (Force) which when applied to the object moves it the through some "distance" in the space considered (sensory or symbolic with sufficient structure). Such constructs fit the usual definitions of work in applications if we connect the "goal attainment" in a particular metric space, sensory or symbolic, with displacements in the goal space. This is nothing new to behavioral sciences for such approaches date back to the group dynamics school pioneered by K. Lewin. We can associate measures of motion (momenta), the derivatives (force and thrust), and the integral (Energy) with the motion in sensory or symbolic space in the usual sense as the terms are used in Mechanics. In addition, with all the measures of motion we can associate and examine measures of order, or better-- disorder, measured using the notions of information and entropy coming from the more recent understanding in thermodynamics and information theory. Applying these concepts, we get "e-motion" as entropized and possibly erratic motion. [1] Definition: Emotional Behavior is energetically perturbated motion. 2.2. Characteristic Work Patterns If we follow the concepts found in mechanical and electrical engineering, "load" on a system can be defined in terms of force. Furthermore a system can be assessed as having capacity measures to "bear a load" and not deviate in performance (the creation of a specific effect). Put in other terms, "load" is a measure of the capability of a system to produce (effect a change in its environment by the expenditure of energy, modification of entropy, or regulation of its exchanges (economics)) specific behavior trajectory [effect/expenditure] edges which it can exhibit with sufficient regularity given a set of contingencies. A characteristic work pattern or envelope for the system is defined as the associated capacity to effect, change, modify objects and move them through a distance in a space and thereby to perform work on them.
This work characteristic interacts with the load characteristic patterns so that the interaction (linkage with load) may interfere with the operational characteristic of the systems. The property we can associate with the envelope or limit within which these systems operate is also studied by mechanical engineers. It is the concept of elastic limit in which a
873 force applied to the system causes a restorative effect to occur (as is done with a spring) up to the elastic limit even under a load with the effects of decrements in velocity, acceleration, displacement. The system can also perform work of a temporary nature (a system transient). If the load characteristics go beyond this system elastic limit, there will be an "overload" condition and permanent deformation of material as is seen in plasticity for engineers, growth for biological systems, and learning in behavior systems. If the load goes beyond plasticity conditions there is loss of integrity separation and possible explosion. We can talk about system strain in this light. In the other direction, where we have compression, we can get overload and loss of elasticity by stress (compression), reduction of time and space available is a case in point. For example, engineers use the notion of elasticity by comparing stress to strain loading use an index called Young's Modulus of Elasticity as measure of cycle properties, especially where there are dynamic oscillations (strain-strain pressure waves). These concepts of are general, a la virtual work, and can thus be utilized to consider both sensory and symbolic aspects of a task (an imperative or purposeful act to seek a goal and have it interfered with by overload, underload, or oscillation between both. Hence stresses and strains caused by explosive and implosive forces which result in overloads to physical (sensory-motor) as well as mental (perceptual, cognitive) aspects results in interactions (interfaces) which produce perturbations in the motions (e-motion effects) which coincide with the way emotional effects arise in behavioral systems. Thus overloads in emotion coincide with e-motions overload which have as their roots overload in physical (sensory-motor) and mental perceptual-cognitive (symbolic) disturbances.
2.2 Emotional WorkLoad From biophysical behavior science and technology we can coin the term "emotional workload" of a system with given work capability under conditions of capacity in communication with a constraint combination (loading). "Overloading" then is to exceed capacity and to generate e-motions which are realized as entropized experiences (feelings) and entropized expression (emotions) which are some times erratic and sometime creative and lead to growth. Transient decrements in performance (work performed) followed by longtime improvement or vice versa. But how do we know and predict, the conditions and ultimately manage them to produce work improvement? This requires we formalize these intuitive notions. Definition: The forms of communication of emotional behavior are experiences and expressions. The experiences of emotional behavior in human beings will be called feelings. The expressions of emotional behavior will be referred to as emotions. The emotional communication interface contains both feelings and emotions. 2.3 Affect Linkages The Theory of Affect Linkages is a model to describe some the structures and processes of how an organism or organizations orients to its environment to satisfy its needs and facilitate its goals In its initial form it defines the attitudinal construct, affect link, in terms of transformation of the marginal degree of goal facilitation attributed as being provided by the object of orientation to the assessor which is computed by comparing what is perceived as
874 being actually provided with what was the expectation learned from past experience. The result is then updated as the event in the environment unfolds. In addition to the value aspect of the construct there is a belief or cognitive aspect (which associates certainty the attribution). Hence the concept of orientation = (value, belief) is a fuzzy concept which is modelled mathematically via complex functions. The construct is operationalized by expressing it in normalized form, which allows it to process measurements of monitored physical and symbolic events. The structure of affect linkages are modelled by a scaling geometry and the dynamics determined in both deterministic and stochastic form. The array approach developed is extended to consider individual, group, and organizational processes. A simple mathematical expression developed to describe stable affect states is as follows using a fixed reinforcement schedule is &j(t) = Log[1-ka(t)/1-ke(t)] ,
(1)
where Aij(t) = affect link from party i to party j; ka(t)j i = actual degree of goal facilitation from j to i; and k~(t)ji= expected degree of goal facilitation. Bij(t)= Belief in the affect attribution Aij(t), and
(2)
Oij(t) : orientation of i to j : (Aij(t), Bij(t)).
(3)
These can be generalized to allow the consideration of both spatial and temporal effects. The affect link can be made to directly correspond to e-motion and thus be a valid measure of entropy-energy interactions and emotions which is an operational construct. How does affect link or other e-motional measures relate to actual emotional behavior? It is conjectured that the emotional behavior is engaged in as exchange which is continually taking place in which emotion is composed or synthesized from the interaction of its entropic (thoughts) and energetic (actions) antecedents. This is the process of emotional experience (perceived as feelings (mood)). Simultaneously there is a decomposition going on in which emotions decompose into entropy (dissipation as noise + signal (neg-entropy=information)) and energy (action) which together are manifested as emotional expression (emotions). These states coexist and compete for dominance in an introversion-extroversion process with stabilization being personality orientation and the transients being "letting off steam" (energy) and/or noise (entropy-information). 3. QUESTIONS AND ANSWERS ABOUT OF EMOTIONAL W O R K L O A D Q. Is there a functional significance to emotional behavior ? A. If emotional behavior is shown to be chaotic behavior it serves as way to allow rapid adaptation to chaotic environmental conditions. Q. Are there limits on the adaptive capability of emotional behavior. A. Yes. If the acceleration of change is too rapid there is an overload and apparent fixation and disorientation of the system. This suggests an upper limit on adaptive capability of emotional behavior. Similarly the studies of conflict, curiosity and arousal show that there is also a lower
875 limit in the form of boredom due to sensory underload. Note in cases of either boredom or overload there are many studies on mental and physical overload to indicate that emotions may function as a safety valve to prevent system destruction from inner commotion which if unchecked would lead to system implosion (excess stress) or explosion (excess strain). Hence emotional behavior may be considered to have functional significance as an operational safety valve. Physiological study with endorphins (associated with crying, etc), adrenlin secretion (with fear,) and nor-adrenalin (with fight) adds further strength to these contentions. Q. How could these perturbations in emotional behavior be analyzed in our framework? A. Emotional loading can be decomposed in energetic and entropic (or emblematic) loads (over, critical, under) and their interfaces Note that according to this qualitative analysis there are four 1st order cases of emotionally imbalanced loadings which could be analyzed by defining dynamics.
Loading Energy Entropy Emotion
Under-Loaded Depressed Information Overload Depressed and Overwhelmed
Critically-Loaded
Over-Loaded
Normal Activitation
(Manic) Hyperactive
Normal Attention
Information Underload
Emotionally Balanced
Depressed and Bored
Table 1 First Order Component Loading: Comparative Capacity vs. Type Q. How can the dynamics of emotional imbalances be computed? If in each category the behavior exhibited in under-critical-overdamping convolutional equations can be written to simulate the dynamics changes in time ,where emotional behavior can be considered as the convolution of energetic and entropic components. 4. CONCLUSION In this paper, it is suggested that the concept of emotional overload be modelled by the use of physical concepts incorporating energy-entropy interactions. These concepts can then be formalized using the author's affect linkage theory and applied to the design of a more advanced "friendlier, satisfying and tunable" Human-Computer Interface which can be utilized to developing large distributed and differentiated friendly HCI which can facilitate the performance of remote teams as found in remote telemedical applications.
REFERENCES [ 1] Marin, I.C. and J. Sappington "A Machine with "Feelings": Emotional Behavior, Proc. World Neural Nets Conference (1994)
System Models of
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
877
The psychological impact of computerised production feedback systems" A comparative study of the U.K. subsidaries of U.S. and Japanese multinational companies C. Oswick and D. Grant The Management Centre, King's College, University of London, Kensington Campus, Campden Hill Road, London W8 7AH, United Kingdom
1. SYNOPSIS This study examines the extent to which the use of computerised systems of feedback in manufacturing environments affects employee attitudes and behaviour. Using samples taken from four diverse organisations, data were gathered via; documentary sources, interviews, observational techniques, and group sessions. The most significant finding of the study is that the immediacy and comprehensiveness of employee feedback provided by sophisticated computer-based systems was found to have a detrimental impact upon attitudes and work performance. In contrast, feedback systems using less sophisticated technology and placing a greater emphasis on interpersonal communication had a more positive psychological impact.
2. INTRODUCTION The primary objective of this paper is to compare mad contrast differing approaches to the computerised monitoring and feedback of production performance data. In particular, the interface between the teclmology (computerised feedback system) and the end user (the production employee) is explored in terms of the attitudinal and behavioural implications. It is posited that independant feedback variables (comprehensiveness, immediacy, frequency, unit of measurement, and media) have a direct, and significant, impact upon the employee's psychological orientation towards work (i.e., job satisfaction,
878
motivation, and commitment). Much of the mainstream management literature extols the virtues of comprehensive feedback systems [1]. Indeed, Hackman and Oldham [2] suggest that feedback is one of the most important variables in determining what they refer to as the motivating potential score (MPS) of a job. Although several authors have highlighted the need for feedback to be meaningful and immediate [3,4], the contingent nature of the complex relationship between feedback and work perfonnance is still not fully tmderstood. This study seeks to establish the extent to which a firm linkage exists between feedback mechanisms and employee attitudes and behaviour. It is hoped that identifying the nature of the interplay between these factors will aid the design of optimal feedback tectmology which maximises the potential for socio-technical congruence, and therefore improved productivity.
3. METHOD 3.1. The Sample The research is based upon data gathered from the U.K. subsidaries of four large multinational companies. All of the companies are drawn from the manufacturing sector. Two of the corporations represented are U.S. owned (company A and company B), while the other two are Japanese owned (company C and company D). Company A is a large-scale producer of male grooming products. Company B manufactures photographic goods and materials. Company C produces a particular electrical component essential to the manufacture of most consumer electrical goods. Company D manufactures consumer electrical products. A brief outline of the feedback system employed in each firm is provided in Table 1. 3.2. Data Collection Data on the nature and operation of the computerised feedback systems were gathered using several methods, namely: secondary data sources; non-participant observation, and; informal interviews with operators, managers and system experts. Insights into the psychological impact of the systems were provided via; documentary sources, group feedback sessions, and participant observation. An extensive progranune of data collection was undertaken, i.e.; 202 employees completed attitudinal questionnaires, 22 employees drawn from various organisational strata were interviewed, 18 days were spent observing and shadowing operations, and a series of 47 group feedback sessions (consisting of
879
373 production employees) were held. The use of documentary sources involved the analysis of a variety of records (including; production figures, down-time, labour turnover, and absenteeism). Direct exposure to the effects of the feedback systems were provided during brief periods of participant observation where production tasks were undertaken and operatives were shadowed. The group feedback sessions contained between 5-15 employees and offered a forum for teams, and work units, to engage in frank and open discussion of their feedback system in a candid and non-threatening enviromnent.
Table 1 - A Summary of the Characteristics of the Feedback Systems Employed in Companies A, B, C, & D. SOPHISTICATION UNITOF FREQUENCY OF TECHNOLOGY MEASUREMENT & IMMEDIACY OF FEEDBACK
METHOD OF FEEDBACK
COMPANY A
High
Group, S h i f t , Department
Intermittent and Self Determined
Remote VDU
COMPANY B
Very High
Group, S h i f t , Department
Continuousand Instantaneous
Shopfloor Light Screen Display
COMPANY C
Low
Individual and Group
Daily (individual) Weekly (group)
Daily = Verbally (via supervisor) Weekly = Display Charts (public)
COMPANY D
Moderate
Group, Shift, Department
Daily, Weekly and Delayed
Display Charts (via computer printout)
4. RESULTS
The nature of the computer system used to monitor and feedback production performance was found to be the major determinmat of other facets of the feedback process. For example, the sophisticated hardware used in Company B enabled ongoing immediate feedback to be provided directly to workers via terminals located on the shopfloor. By contrast, the 'low-tech' system operated in Company C resulted in a delay in production feedback. A strong negative
880 correlation was identified between sophisticated feedback technology and high levels of the employee satisfaction and motivation. Indirect corroboration for this finding was provided by the secondary data gathered - higher productivity, lower absenteeism, less down-time, fewer breakdowns and more favourable labour ttLmover rates, were identified for the two companies (C & D) with less sophisticated feedback mechanisms, than their 'hi-tech' counterparts (A & B). Direct shopfloor observation and the group feedback sessions provided insights into several tmderlying reasons for the aforementioned finding. In particular, the immediate access to performance provided by hi-tech feedback systems caused several problems. Some workers admitted that having concurrent, rather retrospective feedback, enabled them to regulate their performance to ensure that they met, but did not exceed, the minimum production targets. Paradoxically, other workers indicated that continuous feedback spurred them on to work harder - this lead to intra-team problems due to an inevitable conflict between those wanting to maximise production and those seeking to minimise it. In addition, high levels of inter-team competition were reported at companies A and B. Unfortunately, the form of competition which resulted tended to be dysfunctional. Almost two thirds of the production teams at these companies admitted attempting to outperform neighbouring teams by unsanctioned means. This involved manipulating perfonnance data by nmning unloaded machines at excessive speeds for short periods which increased the efficiency rating displayed on the feedback system, but did not enhance production and in the longer term also damaged the machinery. In more extreme forms inter-team competition involved restricting the performance of adjoining teams by; witholding infonnation, hoarding spares, and in some instances even sabotage. The intra and inter-team problems associated with concurrent feedback were a source of demotivation, dissatisfaction and disharmony. The unhealthy forms of competition outlined for Companies A and B were not reported by respondents in Companies C and D. Teams at these companies focused on self improvement in order to out perform the others - what management at company C tenned "healthy competition". This could be attributed to the retrospective nature of the feedback as a consequence of using unsophisticated teclmology. These factors reduced the emotional impact of feedback on employees, and limited the scope for dysfunctional action because it related to past, rather than present, perfonnance. The method of feedback used in the two companies with sophisticated computer systems also caused motivational problems. The use of hi-tech media for presenting feedback was perceived by employees as 'cold' and impersonal.
881
However, at the companies with less sophisticated systems the daily feedback provided by supervisors and/or team leaders was described as being more 'human' and interactive. From a managerial viewpoint the use of direct interpersonal feedback also allowed greater control to be exercised because of the inherent flexibility of the system, i.e. factors such as; the timing of feedback, the opportunity to interpret rather than merely report results, and the scope for positive or negative delivery.
5. DISCUSSION & CONCLUSIONS The results presented above indicate that the two Japanese companies (C and D) have the more effective feedback systems. Nevertheless, it is not possible to draw any generalisable inferences about U.S. and Japanese sudsidaries per se. Instead, the emphasis in the following discussion of the findings is centred upon teclmological differences between the companies, rather than the international dimension. It is possible to think of computerised feedback systems in terms of the form of control they offer. Donnelly et al [5] have identified three forms of production control, namely; preliminary, concurrent and feedback. Preliminary control takes place prior to operations and includes activities such as the inspection of raw materials and the training of operators; concurrent control occurs during the production process, mid; feedback control is concerned with outcomes (i.e. overall results and deviations from set standards). The sophisticated computer systems used in colnpanies A and B are designed to assist management in all three of these areas of control- whereas the systems in operation in companies C and D place emphasis on feedback control. The immediacy and complexity of the concurrent feedback provided by the more advanced systems offers valuable data which aids the design, plamling and monitoring of production operations. However, as was demonstrated earlier, it is the immediate access to this information which allows production workers to manipulate it in ways that are detrimental to output. The nature of the interface between employees and the computer systems within companies A and B also appears to be problematical. The feedback is direct from the computer to the worker, rather thma via a supervisor or manager. Consequently, raw, rather than screened or interpreted, data is fed back and this limits the scope for interpersonal interaction with superiors. As a result, psychologically reassuring aspects of feedback such as praise and recognition are circumvented while management also lose an important opportunity to assert
882
control over operatives who are performing poorly. The primary conclusion of this study is not that companies A and B should opt for less sophisticated systems similar to those in use in companies C and D. Instead, the way in which the technology is utilised needs to be reconsidered. The difficulties experienced in companies A and B appear to stem from their original design and implementation. In terms of human-computer interaction (HCI) it would seem that interface with managers, as systems users, has been fully considered at the designa stage of introducing the new technology. However, the psychological impact upon workers, who are effectively the end-users of the system, has been somewhat neglected. This lack of foresight is not a new phenomenon, there is a considerable volume of HCI-related research which identifies the general problems which arise due to a tendancy to concentrate upon the teclmical requirements of system and failing to consider the psychological aspects of systems design [6-8]. ~
REFERENCES
1. F. Luthans, Organizational Behaviour, 5th Edition, McGraw-Hill, New York, 1988. 2. J.R. Hacklnan and G.R. Oldham, Work Redesign, Addison-Wesley, California, 1980. 3. D.M. Prue and J.A. Fairbank, "Performance Feedback in Organizational Behaviour Management: A Review", J. Org. Behaviour Mgt., Spring (1981). 4. D.A. Nadler, "The Effects of Feedback on Task Group Behaviour: A Review of the Experimental Research", Org. Behaviour and Human Perf., June (1979). 5. J. Dolmelly, J. Gibson and J. Ivancevich, Fundamentals of Management, Business Publications, Texas, 1981. 6. J.A. Hughes, I. Somerville, R. Bentley and D. Randall, "Designing with Etlmography" Making Work Visible", Interacting With Computers, 5 (1993). 7. K. Davids and R. Martin, "Shopfloor Attitudes towards Advanced Manufacturing Teclmology" The Changing Focus of Industrial Conflict?", Interacting With Computers, 4 (1992). 8. F.D. Davis, "User Acceptance of Information Teclmology: System Characteristics, User Perceptions and Behavioral Impact", Int. J. of ManMachine Studies, 38 (1993).
IV.11 HCI Standard
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) 1995 Elsevier Science B.V.
Human-Computer Interaction Standards Nigel Bevan NPL Usability Services, National Physical Laboratory, Teddington, Middx, TWl 1 0LW, UK [email protected] 1. DIFFERENT APPROACHES TO STANDARDS FOR HCI
It is often assumed that a standard means a precise specification. Such standards have brought benefits in many fields, eg: bolts which screw into nuts, ATMs which can read credit cards, and compilers which can read programming languages. Some HCI standards are also of this type: many design guides provide a detailed specification of the nature of the user interface. Although standard user interfaces provide the benefit of consistency, they become out of date as technology changes, and are usually only appropriate for limited types of users and tasks (Bevan and Holdaway, 1993). Thus most work on international standards for HCI has not been about precise specification, but instead has concentrated on the principles which need to be applied in order to produce an interface which meets user and task needs. These standards broadly fall into two categories. One is a "top-down" approach which is concerned with usability as a broad quality objective: the ability to use a product for its intended purpose. The other is a product-oriented "bottom-up" view which is concerned with aspects of the interface which make a system easier to use. The broad quality view originates from human factors, and standards of this type are applicable in the broad context of design and quality objectives. The product-oriented view concentrates on the design of specific attributes, and relates more closely to the needs of the interface designer and the role of usability in software engineering (see Bevan, 1995). Section 4 explains how standards can be used to provide a means of meeting the requirements for the operator-computer interface in the European Directive on Display Screen Equipment. 1.1 Usability as a quality objective These standards relate to usability as a high level quality objective, and usability is defined in this way in ISO 9241-11" Usability: the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of Use.
Standards of this type can be used to support the following activities: • specification of overall quality and usability requirements and evaluation against these requirements (ISO 9241-11 and ISO/IEC 14598-1) • incorporation of usability into a quality system (ISO 9241-11) • incorporation of usability into the design process (ISO/IEC 13407) Section 2 describes these standards. 1.2 Product-oriented standards In the product-oriented view, usability is seen as one relatively independent contribution to software quality, and is defined in this way in ISO/IEC 9126: Usability: a set of attributes of software which bear on the effort needed for use and on the individual assessment of such use by a stated or implied set of users. Section 3 describes standards which deal with usability in terms of attributes which must be designed into a software product to make it easy to use:
885
886 ISO 9241: Ergonomics requirements for office work with visual display terminals: Part 10, 12-17: dialogue design •ISO/IEC, 10741.t!~Dialogue interaction - Cursor control for text editing ISO/IEC 11581 Icon symbols and functions ISO/IEC 9126 Software product evaluation - Quality characteristics and guidelines for their use These standards can be used in the following ways: • To specify details of the appearance and behaviour of the user interface • To provide detailed guidance on the design of user interfaces • To provide criteria for the evaluation of user interfaces However the attributes which a product requires for usability depend on the nature of the user, task and environment. A product has no intrinsic usability, only a capability to be used in a particular context. ISO 9241-11 can be used to help understand the context in which particular attributes may be required. 2. USABILITY AS A HIGH LEVEL QUALITY OBJECTIVE 2.1 ISO 9241-11 Guidance on Usability
The objective of designing and evaluating for usability is to enable users to achieve goals and meet needs in a particular context of use. ISO 9241-11 explains how usability can be specified and evaluated in terms of user performance and satisfaction. User performance is measured by the extent to which the intended goals of use are achieved (effectiveness) and the resources such as time, money or mental effort that have to be expended to achieve the intended goals (efficiency). Satisfaction is measured by the extent to which the user finds the use of the product acceptable. ISO 9241-11 also emphasises that usability is dependent on the context of use and that the level of usability achieved will depend on the specific circumstances in which a product is used. The context of use consists of the users, tasks, equipment (hardware, software and materials), and the physical and organisational environments which may all influence the usability of a product (see Figure 1).
intended objectives
....
,,oa,s ) & I
usability: extent to which goals are achieved with effectiveness, efficiency and satisfaction I
Y
equipment. ) ,
,,,
C env'onment) Context of use
•
~ : . : ~ : ~ . ~ : ~ ~ ' ~ .
C e,,ect,veness)
,,,,
outcome of interaction
C
efficiency ')
C satis'action) Usabilitymeasures Figure 1 Usability framework
887 ISO 9241-11 was developed in close conjunction with the MUSIC project. The user-based MUSIC methods and tools provide a practical implementation of the principles of the standard. The Usability Context Analysis Guide (Macleod 1994) provides a procedure for documenting the context of use and context of evaluation. The Performance Measurement Method (Bevan and Macleod, 1994) provides a reliable and repeatable method for measuring effectiveness and efficiency and diagnosing usability problems. SUMI (Kirakowski, 1995) enables different aspects of user-perceived usability to be measured and areas of difficulty to be pin-pointed. Cognitive workload can be measured (Wiethoff et al 1993) as a means of predicting over- or under-loading of the user. 2.2 Quality Systems and ISO 9001
Dealing with usability as part of a quality system for design and development of products, as specified in ISO 9001, involves the systematic identification of requirements for usability, including usability measures and verifiable descriptions of the context of use. These provide design targets which can be the basis for verification of the resulting design. ISO 9001 specifies what is required for a quality system. A quality system is a documented set of procedures intended to ensure that a product will meet initially stated requirements. A quality system is a desirable (though not sufficient) condition for achieving quality of the end product. ISO 9241-11 describes how the usability of a product can be defined, documented and verified as part of a quality system which conforms to ISO 9001 (Figure 2). The overall context of use should be identified, usability requirements should be specified, usability issues should be monitored during development, and the usability achieved should be evaluated. Activities
I
Documents/Outputs
identify context of use
.-I " I,
specification of context of use
select usability measures criteria and context
..1 "l
usability specification
evaluate usability redesign product
.=1 statement of compliance with "! criteria .J
improved product
"1 Figure 2 Quality Plan Overall context of use: Information about the characteristics of users, their goals and tasks and the environments in which the tasks are carried out provides important information for use in the specification of overall product requirements, prior to development of specific usability requirements. Usability requirements: Prior to development of a custom system, the purchasing organisation should specify the usability requirements which the system must meet and against which acceptance testing may be carded out. Specific contexts in which usability is to be measured should be identified, measures of effectiveness, efficiency and satisfaction selected, and acceptance criteria based on these measures established.
888
Monitor usability: At various stages during the development process the developer should measure the usability achieved against these targets. This information enables objective decisions to be taken about the need for design changes to enhance usability, and about tradeoffs which may be appropriate between usability and other requirements. Usability evaluation: The characteristics of the context in which a product is likely to be used need to be identified. To ensure the validity of test resuks the users, tasks and environments used for the evaluation should match the real context of use as closely as possible. 2.3 Quality of use
ISO 9241-11 introduces the concept of a work system, consisting of users, equipment, tasks and a physical and social environment, for the purpose of achieving particular goals. Measures of user performance and satisfaction assess the quality of the work system in use, and, when a product is the focus of concern, these measures provide information about the usability of that product in the particular context of use provided by the rest of the work system. ISO 9241-11 defines the quality of a work system in use as: Quality of a work system in use: the extent to which specified goals can be achieved with effectiveness, efficiency and satisfaction in a specified work system. The difference between usability and the quality of a work system in use is a matter of focus. When usability is evaluated, the focus is on improving a product while the other components of the work system (user, task, equipment, and environment) are treated as given. If the aim is to improve the quality of the overall work system in use, any part of the work system may be the subject of design or evaluation. For example it may be appropriate to consider the amount of user training to be provided, changes in lighting, or re-organisation of the task. In this case the element which is the object of design or evaluation is considered to be subject to potential variation, while the other elements of the work system are treated as fixed. 2.4 Software quality evaluation
ISO 8402 (Quality Vocabulary) defines quality as: Quality: the totality of characteristics of an entity that bear on its ability to satisfy stated and implied needs. This defines quality in terms of the characteristics of a product. To the extent that user needs are well-def'lned and common to the intended users it implies that quality is an inherent attribute of the product. However, if different groups of users have different needs, then they may require different characteristics for a product to have quality. ISO/IEC 14598-1 (Information Technology- Evaluation of Software Products- General guide) distinguishes between the concept of quality as an inherent characteristic of the product, and quality of use: Quality of use: the extent to which an entity satisfies stated and implied needs when used under stated conditions The ultimate objective of software quality evaluation is to ensure that the product provides quality of use - that it meets the needs of the users. (Users may include operators, recipients of the results of the software, or maintainers of software.) This definition of quality of use is very similar to the definitions of usability and the quality of a work system in use in ISO 9241-11. The only difference is that ISO 9241-11 specifically defines the needs in terms of user performance and satisfaction, and the stated conditions in terms of users, goals and environments. Internal software quality attributes (such as the functionality, usability and efficiency attributes defined in ISO/IEC 9126) can be used as indicators to estimate f'mal software quality. The specific internal attributes which are relevant to final quality of use will depend on the intended conditions of use - for an interactive product this will depend on the needs of the eventual end users and the tasks.
889 2.5 Human-centred design To achieve the overall objective of usability and quality of use requires a human-centred approach to design. This is the subject of a standard under development: ISO 13407 (Humancentred design process for interactive systems). This standard is expected to cover topics including: planning the usability process, incorporating human-centred design activities in interactive system development processes, and assessing the benefits of human-centred design. 3. DESIGNING FOR USABILITY: PRODUCT-ORIENTED STANDARDS
Usable products may be designed by incorporating product features and attributes known to benefit users in particular contexts of use. ISO 9241 provides requirements and recommendations relating to the attributes of the hardware, software and environment which contribute to usability, and the ergonomic principles underlying them. The following parts of ISO 9241 and other standards deal with attributes of the software: • ISO 9241-10: Dialogue principles. This part of ISO 9241 deals with general ergonomic principles which apply to the design of dialogues between humans and information systems: suitability for the task, suitability for learning, suitability for individualisation, conformity with user expectations, self descriptiveness, controllability, and error tolerance • ISO 9241-12: Presentation of information. This part of ISO 9241 contains specific recommendations for presenting and representing information on visual displays. It includes guidance on ways of representing complex information using alphanumeric and graphical/symbolic codes, screen layout, and design as well as the use of windows. • ISO 9241-13: User guidance: This part provides recommendations for the design and evaluation of user guidance attributes of software user interfaces including Prompts, Feedback, Status, On-line Help and Error Management. • ISO 9241-14: Menu dialogues. This part provides recommendations for the ergonomic design of menus used in user-computer dialogues. The recommendations cover menu structure, navigation, option selection and execution, and menu presentation (by various techniques including windowing, panels, buttons, fields, etc.). Part 14 is intended to be used by both designers and evaluators of menus (however, its focus is primarily towards the designer). • ISO 9241-15: Command language dialogue. This part provides recommendations for the ergonomic design of command languages used in user-computer dialogues. The recommendations cover command language structure and syntax, command representations, input and output considerations, and feedback and help. Part 15 is intended to be used by both designers and evaluators of command dialogues, but the focus is primarily towards the designer. • ISO 9241-16: Direct manipulation dialogues. This part provides recommendations for the ergonomic design of direct manipulation dialogues, and includes the manipulation of objects, and the design of metaphors, objects and attributes. It covers those aspects of "Graphical User Interfaces" which are directly manipulated, and not covered by other parts of ISO 9241. Part 16 is intended to be used by both designers and evaluators of command dialogues, but the focus is primarily towards the designer. • ISO 9241-17: Form-filling dialogues. This part provides recommendations for the ergonomic design of form filling dialogues. The recommendations cover form structure and output considerations, input considerations, and form navigation. Part 17 is intended to be used by both designers and evaluators of command dialogues, but the focus is primarily towards the designer. • ISO/IEC 10741-1 Dialogue interaction - Cursor control f o r text editing. The standard specifies how the cursor should move on the screen in response to the use of cursor control keys. • ISO/IEC 11581 Icon symbols and functions - Part 1: General. This part contains a framework for the development and design of icons, including general requirements and recommendations applicable to all icons.
890 • ISO/IEC 11581 Icon symbols and functions - Part 2: Object icons. This part contains requirements and recommendations for icons that represent functions by association with an object, and that can be moved and opened. It also contains specifications for the function and appearance of 20 icons. Before designing appropriate usability attributes into the software following the guidance and requirements of the standards listed above, a software designer needs to identify the anticipated users, tasks and environments using ISO 9241-11. However, using attributes which conform to these standards cannot guarantee that a product reaches a required level of usability, as these standards do not provide an exhaustive specification of how to apply the general principles that make a product usable.
4. EUROPEAN DISPLAY SCREENS DIRECTIVE The European Directive on Display Screen Equipment (EEC 1990, Bevan 1991) is primarily concerned with the physical working environment and working conditions, but also includes requirements that: • Software must be suitable for the task. • Software must be easy to use and where appropriate adaptable to the user's level of knowledge or experience. • Systems must display information in a format and at a pace which are adapted to users • The principles of software ergonomics must be applied. This applies to software used as part of new workstations immediately, and all workstations from 1997. Conformance with usability standards provides one means to ensure compliance with the Directive. The minimum requirements of the Directive are similar, but not identical to, the requirements of the relevant parts of ISO 9241 which are in much greater detail. In particular, ISO 9241-10 contains the main principles of software ergonomics. In general, the standards contain broader requirements than the Directive, as the Directive is concerned only with health and safety, while the standards are also concerned with the effectiveness and efficiency of users. It would have been simpler if the Directive had made direct reference to standards rather than containing its own requirements. However, not all the standards are complete, and the contents of standards are agreed by experts in national standards bodies, while the contents of the Directive are approved at a political level in the European Commission. REFERENCES
Bevan N (1991) Standards relevant to European Directives for display terminals. In: Bullinger HJ (ed): Proceedings of the 4th International Conference on Human Computer Interaction, Stuttgart, September 1991. Elsevier. Bevan N (1995) Usability is quality of use. In: Anzai & Ogawa (eds) Proceedings of the 6th International Conference on Human Computer Interaction, Yokohama, July 1995. Elsevier. Bevan N and Holdaway K (1993) User needs for user system interaction standards. In User needs for information technology standards, Evans, Meek and Walker (eds), Butterworth Heinemann. Bevan N and Macleod M (1994) Usability measurement in context. Behaviour and Information Technology, 13, 132-145. CEC (1990) Minimum safety and health requirements for work with display screen equipment Directive (90/270/EEC) Official Journal of the European Communities No L 156, 21/6/90. Houwing E.M., Wiethoff M., and Arnold A.G. (1994). Introduction to cognitive workload measurement. Delft University of Technology (WIT Lab). Kirakowski J (1995) The software usability measurement inventory: background and usage. In: P Jordan et al, Usability Evaluation in Industry. Taylor & Frances, UK (in press). Macleod, M (1994) Usability in Context: Improving Quality of Use. In: G Bradley et al (eds.) Human Factors in Organizational Design and Management - IV. Elsevier/North Holland.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) 1995 Elsevier Science B.V.
891
The applicability o f the ISO user interface standards. Frederik Dehlholm Datacentralen Oldenburg Alle 1 DK 2630 Taastrup Denmark E-mail: [email protected]. dk
The Parts 10 to 17 of ISO 9241 are user interface standards. Part 10 was agreed upon in 1994 and will be issued in the first half of 1995. Three of the other parts are out for final voting in the beginning of 1995, and the last four parts will follow in 1996 and 1997. Part 10 contains seven general dialogue principles and examples of their application. The Parts 12 to 17 contain specific rules for the dialogue and for the design and layout of the screen. As a member of the ISO group that makes these standards, Datacentralen has been already been able to incorporate these standards in our usability work. (Datacentralen is a Danish large software house that develops business information systems for many different platforms).
The Seven dialogue principles. In our company some discussions have taken place about the necessity of incorporating Part 10 into our usability work. Some regarded the seven dialogue principles as a good introduction and as good general guidelines for our usability work. On the other hand, the quality assurance team found the principles too general. They thought it would be impossible to check whether a specific user interface complied with these principles. The conclusion of the discussion was that the seven dialogue principles were incorporated in the company styleguide. In the company styleguide the seven dialogue principles serve as an introduction, but are at the same time mentioned as principles that ought to be followed. The seven principles are quoted with some guidelines on how to apply the principles. These guidelines are the guidelines of the ISO standard relevant to the kind of administrative applications made by Datacentralen.
The company styleguide. The purpose of the company styleguide is to help the systems developers make usable and beautiful systems and at the same time to make sure that the systems have the corporate look and feel and follow the de facto standard for the platforms, that the systems are made for. The general contents in the seven dialogue principles have not been disputed, but one of the guidelines for the third principle, that the user should be in control, created a lot of debate. The guideline says that the waste-paper basket should not be emptied until the user wants it.
892
The arguments against this guideline were: • it is too difficult to implement • it is a part of the case tools that we use • it is a feature in either Windows or OS/2 the waste-paper basket will become too full because the user forgets to empty it. The conclusion of the discussion was been to include the guideline not only as a specification of the general principle, but also as a part of the corporate look of the user interface.
Propaganda for the ISO 9241 standard
We have made a lot of propaganda for ISO 9241: • We have had several articles on ISO 9241 in the Danish ComputerWorld • We have made a short introduction to the standard in the company styleguide • We have presented the standard at some GUI-seminars for all the systems developers in the company • We have used one hour in the company GUI training course to go through the Parts 10 to 17.
Incorporation of the Parts 11 to 17.
The drafts of the Parts 11 to 17 contain a lot of specific rules for the user interface. We did not want to include all these rules in the company styleguide because that would make the company styleguide document much too comprehensive. We wanted to make the company styleguide as small as possible. So we went through all the rules in the drafts of the Parts 11 to 17 and included only those that: • are relevant for the kind of Administrative Systems that are made at Datacentralen and at the same time • are not included in CUA or Windows
893 • are not included in CUA or Windows and at the same time • are not self-evident for the systems developers at Datacentralen, considering that almost all the system developers have been through a three-days course in usability
The rules of the Parts 11 to 17 have not caused much debate. The issues in the company styleguide that have given cause to most debate have been: which elements to use in designing the corporate image in the user interface. Many parts of the company participated in the development the company styleguide. This was crucial for both the quality and the acceptance of the standard. The standard was also reviewed by all the existing GUI projects in the company, but it turned out not to be enough. The persons writing the company GUI guides for the different GUI tools used in our company should also have participated in the group, developing the company GUI guide. To be a reviewer of the guide was not enough to make them feel committed to the guide. This means that there are a few arias in which the GUI guides for the GUI tools deviated from the company GUI guide. This is now being corrected.
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) 1995 Elsevier Science B.V.
895
Application of e r g o n o m i c standards to the EC Directive on requirements for display screen equipment. Henrik H o p f f Tele D a n m a r k Ltd. E r g o n o m i c Office 21 N o e r r e g a d e DK-0900 Copenhagen C Abstract The European Union is eager to promote the use of standards in the legal requirements for work with display screen equipment. This paper describes the difficulties in applying standardisation to the law-making process. It is shown to what extent standards are applicable and in which cases other methods are necessary. The paper demonstrates a case story of how a large Danish company has managed to apply successfully the parts 3 and 10 of the ISO 9241 standard in their implementation of the requirements of the EC Directive 90/270/EEC.
1. INTRODUCTION In the EC, the efforts towards obtaining a single market is regulated by using directives. The directives act as a way of harmonising the legislation of the individual member countries. One such directive is the minimum safety and health requirements for work with display screen equipment (90/270/EEC). The Directive was published in 1990. Apart from other directives, the directives concerning worker's safety and health are so-called minimum directives. This implies that the individual member countries can increase the requirements during the national implementation of the Directive. The Directive aims at the employers who are responsible for the worker's safety and health. In the Directive on the minimum safety and health requirements for work with display screen equipment (90/270/EEC) no references to international standards are found. 1.1. CEN Standards However in 1988, SOGITS (Senior Officials Group Information Technologies Standardisation) which is formed by the European Commission, moved towards making use of the what was to be the ISO 9241 standard. SOGITS mandated CEN, the European Commitee for Standardisation to convert the first four parts of the ISO 9241 standard into CEN standards. In this way, the European Commission wanted to provide a link between the Directive and the ongoing standardisation work. In future a revised directive might have a reference to the 9241 standards.
896 2. DIFFICULTIES IN USING ERGONOMIC STANDARDS Some European member countries inclusive of Denmark, recognize a difficulty in regulating protection of worker's safety and health through applying standards to the legislation. In the Danish community, the legislation on worker's safety and health is provided by the authorities in a close cooperation with employers and workers. The use of standards will affect the influence of the employers and workers. With the standards the participants will have less possibility to influence the national legislation.
3. PRACTICAL USE OF THE ISO 9241 STANDARD The Danish legislation on worker's safety and health demands that all major companies form an in-house security organization. This organization consists of representatives of the company management and workers. The purpose of the organisation is to control that equipment and work procedures meet the legal requirements. Tele Denmark Ltd. is the national telephone operator with at present three million subscribers. The company has participated in the national and international preparation of the ISO 9241 standard. Due to this work, it has been possible to perform an early implementation of the standard in the company. Three different approaches have been taken to make use of the standard.
3.1. Checklists In order to provide the security organisation with checklists, three different checklists have been developed for hardware, software and office equipment. The checklists contain the recommendations of the parts concerning display unit and the general dialogue principles. In addition the checklists incluse the recommendations of the Swedish MPR 1990 and TCO '92 ergonomic requirements. All suppliers of office equipment are asked to fill in the relevant checklists and send them to the members of the security organisation. The company can then use the checklists to evaluate whether the products meet the requirements of that particular part of the company.
3.2. Project Management Handbook In order to make sure that new software products conform to usability principles, the Part 10 Dialogue principles are built into the handbook. The handbook follows the traditional model of development. At the beginning of a project, the prospective userorganisation is notified in order to let the security organisation provide the ergonomic requirements. In the design of the user interface, the ergonomic recommendations are applied. The quality control must go through the initial design and make sure that all deviations from the standard are reported to the security organisation. In the near future, usability testing will be provided in order to make sure that those usability aspects not supported by the recommendations in the standard will be investigated and documented.
3.3. Company Styleguide The requirements in Part 10 Dialogue Principles are used to design the layout of user
897 interfaces and dialogue principles, as well as the principles of the IBM SAA/CUA styleguide. The Company Styleguide provides the developers with a tool to make consistent interfaces building on ergonomic principles.
4. E X P E R I E N C E GAINED F R O M T H E I M P L E M E N T A T I O N The main difficulty in implementing the standard is the fact that the use of standards may obscure the tradition of heuristic testing of user interfaces. The ergonomic recommendations sometimes override individual user requirements derived from either usability testing or from the initial user requirements. Many users and developers are reluctant to rely on ergonomic standards and prefer to rely on results from usability testing and experience gained from similar user interfaces. In this case the standard is used as a design guide where applicable. The recommendations are followed only when they fit into the developer's mental model of the user dialogue. I see this conflict as a natural consequence of the role of ergonomic standards in the Danish community. Protection of worker's safety and health relies on individual inspection and action and not on the use of standards.
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) 1995 Elsevier Science B.V.
899
S t r u c t u r e d H u m a n Interface V a l i d a t i o n T e c h n i q u e - S H I V A Jtirgen Ziegler, Fraunhofer Institute IAO Michael Burmester, Fraunhofer Institute IAO tel: +49 711 970 2334 fax: +49 711 970 2300 [email protected] [email protected]
The paper describes a new evaluation approach based on a walkthrough method combined with the task and navigation model of the user interface software to be evaluated. The user interface is evaluated according to the requirements of ISO 9241.
1. N E E D F O R U S A B I L I T Y E V A L U A T I O N More and more workplaces are equipped with visual display terminals. It is estimated that by the year 2000 90% of all employees in industrialised countries will use VDTs in order to perform their tasks (F~ihnrich, 1987). This leads to the situation that on one hand health and safety and on the other hand effectiveness and efficiency is required for VDT workplaces. Since January 1, 1993 the Council of the European Union issued the Directive 90/270/EEC "on the minimum safety and health requirements for work with display screen equipment". The governments of the member States of the European Union must converse this directive into national law. With this the European directive is valid and must be applied to the workplaces with display screen equipment. The EU-Directive demands very high safety and health standards for work being carried out on computer systems. Workplaces must be analysed and it must be ensured that the VDT workplaces exclude heavy physical and mental strain. Furthermore, it is required to involve end users in the design and development phases of workplaces employing computer systems. In five short paragraphs the EU-directive lays down special requirements concerning manmachine interfaces: The software must be better adapted to improve the execution of jobs. It must also be easy-to-use, flexible, user-oriented and able to present information about currently running processes in the system. Besides this, the most basic ergonomic standards must be employed. The most agreed approach is to interpret these requirements on the basis of the ISO 9241. Part 10 "Dialogue Principles" of ISO 9241 (1994) is already agreed as an international standard concerning the software ergonomic quality of user interfaces. After that, software must fulfil the following dialogue principles: suitability for the task, self-descriptiveness, controllability, conformity with user expectations, error tolerance, suitability for individualisation, and suitability for learning. Taking these facts into account, it is obvious for software developing organisations that efficient and effective means for user interface design and evaluation are required. 2.
EVALUATION
METHODS
A lot of user interface evaluation methods are available. Two main classes can be differentiated. One group of the evaluation methods are focused on users of a particular system.
900 The other category of the evaluation methods are designed to support ergonomic experts in evaluating user interfaces. Heuristic evaluation and cognitive walkthrough are well known expert evaluation approaches.
2.1.
Expert Evaluation
Heuristic evaluation is based on a systematically and theory driven assessment of user interfaces carried out by software ergonomic experts (Nielsen, 1992). A large variety of procedures for expert evaluation can be found. Some expert methods are highly structured providing a lot of instructions for the evaluator. An example for a highly structured and voluminous expert evaluation method is EVADIS II (Reiterer & Oppermann, 1993). Other expert evaluation procedures provide only some guidelines. For example, heuristic evaluation can be carried out by assessing each screen according to a four level user interface model (VDI, 1990; GtJrner, 1994). Following this approach, the user interface is analysed on the levels: •
Task level (What are the tasks? Which goals do users have?)
•
Functional level (Does the software provide all functions needed to perform the tasks?)
•
Syntactic level (Is the meaning of labels and icons clear? Does the system provide clear feedback?)
•
In- and Output level (Are the dialogue elements correctly chosen? Are colours, grouping of elements etc. according to human factors requirements?)
Cognitive walkthrough is another approach of expert evaluation. It is based on the human action theory (Norman, 1986). The user interface evaluation is structured by the tasks of the users. A task is defined by a goal which has to be achieved and actions to be performed in order to reach the goal. The expert evaluates the user interface according to the human action cycle (Polson, Lewis, Rieman & Wharton, 1992): •
Goals
•
Action plan
•
Execution of actions
•
Evaluation of feedback
•
Revision of the goals
•
Continuation of the cycle
2.2.
User Testing
User testing or empirical evaluation is based on the analysis of user behaviour during the use of the software to be evaluated. A pool of several data collection and data analysis methods is available to run user trials: logfile analysis, video protocolling, verbal protocolling, task completion time, number of errors, calculation of unproductive time etc.
2.3.
Comparison of Expert Evaluation and User Testing
The following table shows a comparison of expert evaluation and user testing methods according to the criteria effectiveness and expenditure of the evaluation method. For software development especially in small software companies the factor expenditure is a central cost factor.
901
Table 1: Comparison of Expert Evaluation and User Testing
User Testing
Expert Evaluation
Effectiveness
70% of errors in user interface design can be detected by running user trials with 5 users (Nielsen & Landauer, 1993)
1 Expert is able to detect between 30% and 50% of user interface problems, 5 experts are able to detect up to 95% of design errors (Molich & Nielsen, 1990)
Expenditure
very high: : Acquisition of users, planning of trials, data collection methods, data analysis, interpretation and generation of design alternatives;
low: Little preparation for heuristic evaluation is needed Cognitive walktrough requires the definition of the user tasks
At least two evaluators are needed to perform user trials The effectiveness of both approaches is nearly similar, but a big difference can be found in the costs of the two approaches. Karat, Campbell & Fliegel (1992) calculated 160 hours total time for user testing and 94 hours total for walkthrough performed by one expert. This is a significant difference. The proportion of significant problem areas detected in the software per hour evaluation work is better for empirical evaluation (4.0 hours to detect a significant problem) than for individual walkthrough (5.2 hours to detect a significant problem). In contradiction to this, Nielsen & Landauer (1993) showed that expert evaluation is more effective than empirical evaluation, especially if the experts are experienced with the tasks environment of the system to be evaluated. The reason for the different results is due to the different level of expertise between the two experiments. Nielsen & Landauer (1993) run the expert evaluation with human factors experts and Karat et al. (1992) running their trials with experienced computer users as experts. Taking into account the good effectiveness and cost-efficiency of expert evaluation, a well structured evaluation approach which is closely related to ISO 9241 seems to be promissing. 3.
TASK
AND
OBJECT
ORIENTED
DEVELOPMENT
METHOD
In order to be efficient and effective in user interface evaluation it is necessary to construct evaluation methods according to user interface development methods and the development process. In the following, a methodology for development and design of object oriented and graphical user interfaces (GUIs) is described (Beck, Janssen, Weisbecker and Ziegler, 1994). The method follows an iterative development approach which includes object oriented task analysis and modelling, as well as navigation and dialogue modelling combined with prototyping and modelling techniques from software engineering (Ziegler & Janssen, 1995). The user interface development method is strictly user oriented. The iterative development process consists of four phases: •
Analysis of user tasks
•
Definition of the essential task model
•
Conceptual user interface model
•
Design of the user interface
During the analysis of user tasks, the main focus is on getting information concerning the user behaviour, the time structure of task sequences and objects needed to perform the tasks. Data collection can be done by using interview techniques, user observation (e.g. event sampling) or scenario technique or any other suitable technique (Kirwan & Ainsworth, 1992).
902 From these data an object model and a task model will be generated. The object model consists of the objects needed and their attributes. The task model covers the dynamic relations between the objects. The combination of the object and task aspects is called essential task model. The structure of the task model is represented by data flow diagrams, linked to the object model analogous to Entity-Relationship diagrams in Structured Analysis (Yourdon, 1989). The conceptual user interface model is constructed on the basis of the object and task model. For the representation of objects user views are defined. A view is a part of the overall object model and has two aspects. The logical aspect defines the elements of the object model belonging to the view. The physical aspect defines the screen appearance, the association of GUI interaction objects, and the layout. The distinction of conceptual objects and views is similar to the MVC model (Krasner & Pope, 1989). On the navigation and dialogue control side, two layers are distinguished. The coarse grain dialogue defines the sequencing of views and the call of object methods in response to user input. The fine grain dialogue defines the dynamics of view attributes, e.g. the change of the sensitivity of menu choices according to the state of the dialogue. After having defined the view and navigation model, the design of the user interface can be started. During this phase the dialogue elements will be selected. The selection can be supported by rapid prototyping tools providing the dialogue elements. Normally, these dialogue elements are designed according to an industrial standard (e.g. MS Windows, 1991 or CUA, 1991). Furthermore, the visual information presentation will be performed in this phase. With the first prototype the evaluation of the user interface can be started. After evaluating the user interface the refinement of the user interface will start with the next iteration.
Task Analysis (e.g. UserObservation,ScenarioTechnique) y
User particiga tion Object Model
Task Model
I I Iterative Development Approach I I Prototyping
View Definition
Navigation Structure
Selection of Dialogue Objects
Visual Information Design
I I Evaluation
903 Figure 1. Methodology for task and object oriented development of graphical user interfaces 4.
NAVIGATION
MODEL
AS BASIS
FOR
EVALUATION
The Structured Human Interface Validation Technique SHIVA uses a model of the navigational structure of the system for systematically assessing the usability of the system. As described in the software development method, the navigation model describes how the user can move between the different parts of the user interface. The elements of the navigation model are so-called views which provide a specific perspective on the data object to be manipulated. Views typically correspond to collections of interaction and display objects shown in a single window or dialogue box or represented by iconic objects on a desktop. In order to perform a certain task, the user has to navigate to a particular view which supports the task at hand, or has to go through a sequence of views to accomplish the different subtasks associated with the main goal. The navigation model is thus an important abstraction of the actual interaction with the system which describes the conceptual architecture of the user interface. It can be represented in a suitable fashion in a graphical form either e. g. as specific Petri Nets (Janssen 1993) or as a network of predefined view types which are depicted as different pictograms (Ziegler 1995). An example of the latter modelling approach as part of a larger system is shown in Fig. 2.
leon
Icon Suppliers
On:leB
I
I I I I I I
L
Search Supplier
Products
Search Oxler
I I I i
I
kiste
kiste
Suppliers
Orders
l
Suppler Data
I..,
1~' I"
, I Order Data
Figure 2. Example of a navigational structure showing a part of a larger system for a purchasing department. The view supplier data can be reached by starting from a supplier icon, leading either over a search dialogue box if there are many suppliers or a supplier list. The supplier data view can also be accessed over the orders which have been issued to a particular supplier. The navigation model abstracts from the specific techniques and conditions for making a transition from one view to another as well as from the specific contents of the different views. These aspects will be evaluated later on the basis of the actual system, a prototype or additional specifications. 5.
TASK
SCENARIOS
Independently from elaborating the navigation model, a set of realistic task scenarios is developed, preferably by or in close co-operation with the prospective users of the system. These task scenarios can either be provided in a free textual form or as a more formalised description using a suitable task modelling technique. The scenarios should cover the relevant
904 and typical tasks of the user and should address all major functionalities of the system to be evaluated. Scenarios represent composite tasks consisting of a set of subtasks. Scenarios can be organised around a single object to be manipulated (,,change customer address data") or around semantically related objects (,,check the status of a certain customer a see whether there are any orders which have not yet been delivered"). The object relations are typically documented in an object or entity-relation model. It is not necessary that the scenarios cover all possible dialogue paths through the system as a check concerning changing task goals will be performed during the evaluation walkthrough. 6.
PERFORMING
THE
EVALUATION
The evaluation of the system is done on the basis of a walkthrough mechanism which is applied in two cycles. In the first cycle, the evaluator walks through all the different views of the system, asking a set of standard questions which are derived from overall usability principles such as ISO 9241-10. A first set of questions has been developed and is currently being further analysed. The questions relate either to the specific view under scrutiny or to the navigational transitions which are possible from this particular view. Questions concerning the contents of the view currently analysed are (related ISO criteria are shown in brackets): •
Which task/tasks can be performed within this view (task adequacy)
•
Are the interaction objects selected suitable for the user's task (task adequacy)?
•
Is the information presented understandable to the user (self-descriptiveness, leamability)?
•
Is help and support information available and adequate (self-descriptiveness)?
•
Is the layout and the display of information clear and easy to perceive (ISO 9241-12, Information Presentation)?
In addition, the reviewer can note all possible other usability problems related to this view (cf. heuristic usability evaluation, see Nielsen & Mack, 1994). The second set of questions relates to the transitions leading from the current view: •
Are the transitions to other views of the same object completely supported (task adequacy, suitability for individualisation)?
•
Are the transitions to semantically related objects completely supported (task adequacy, controllability)?
•
Is it transparent to the user which other views of the same or related objects can be reached from the current one (controllability, learnability)?
•
Are the interaction techniques for triggering the transitions suitable (conformance with user expectations, learnability)?
In the second cycle, the evaluation focuses on the tasks scenarios provided. The reviewer walks through the scenarios, analysing the sequence of views needed for performing the particular scenarios. This yields results concerning the number of different views needed for performing the single tasks. In each step, the reviewer should ask a set of what-if questions concerning possibly changing task goals. These questions address issues like: •
What if the user wants to perform a different operation on the same object?
•
What if the user wants to perform the operation on a set of objects instead of a single object?
•
What if the user wants to see or manipulate a related object?
The assumption underlying these questions is that those tasks the user wants to perform cannot be completely enumerated in an efficient manner for information systems of realistic size.
905 It is therefore necessary to explore the task space through what-if questions while going through the different steps of a defined scenario.
7. I N T E G R A T I O N OF THE W A L K T H R O U G H R E S U L T S The two walkthrough cycles need not necessarily be performed in the sequence described above. However, the results must be integrated in order to get a coherent picture of the system's usability. The first walkthrough cycle yields information related to each single view in a local perspective, whereas the second cycle allows to check the appropriateness of the system for defined task or task sequence in a global perspective. The transitions evaluated in the first cycle may be used for answering the what-if questions addressed in the second cycle. Cycle 1 helps to determine which tasks the system supports at all, while cycle 2 checks whether this support is appropriate for a given set of scenarios. Currently, various forms of integrating, analysing and documenting the results of the walktroughs are being analysed such as tables showing which views are needed for which tasks. While such specific techniques are still under development, the method has already been used in a couple of test assessments. There is evidence that it can provide valuable support for an expert evaluator of a system by structuring the evaluation process. Until now, it is an open issue, however, whether such techniques might also be used by non-experts or end users.
8. R E F E R E N C E S 1. Beck, A., Janssen, C., Weisbecker, A. & Ziegler, J., Integrating Object-Oriented and Graphical User Interface Design. Proceeding of the SE/HCI Workshop, Sorrento, Italy, May 16-17, 1994 2. F~ihnrich, K.-P. (ed.). Software-Ergonomie. Mtinchen: OldenbourgVerlag, 1987 3. G6mer, C., Vorgehenssystematik zum Prototyping graphisch-interaktiver Audio/VideoSchnittstellen, Berlin: Springer, 1994. 4. IBM:SAA/CUA (System Application Architecture-Common User Access) (1991). Guide to User Interface Design (IBM-Nr. SC34-4289-0). 5. ISO 9241, Ergonomic requirements for office work with visual display terminals (VDTs) Part 10: Dialogue principles, 1994. 6. ISO 9241, Ergonomic requirements for office work with visual display terminals (VDTs) Part 12: Presentation of Information, 1994. 7. Janssen, C., Dialognetze zur Beschriebung von Dialogabl~iufen in graphisch-interaktiven Systemen. In: K.-H. R6diger (ed.), Software-Ergonomie ,93 Von der Benutzeroberfl~iche zur Arbeitsgestaltung. Stuttgart: Teubner, 1993. 8. Karat, C.-M., Campbell, R. & Fliegel, T. (1992). Comparision of empirical testing and walkthrough methods in user interface evaluation. Proceedings of CHI '92, 397-404,1992. 9. Kirwan, B. & Ainsworth, L.K. (eds.), A guide to task analysis.London:.Taylor & Francis, 1992 10. Krasner, G.E. & Pope, S.T., A Cookbook, for Using the Model-View-Controller User Interface Paradigm in Smalltalk-80. Journal of Object-Oriented Programming, 1 (3), 26-49, 1989. 11. Microsoft Corporation (1991). The Windows Interface; An Application Design Guide. Redmond, Washington: Microsoft Press. 12. Molich, R. & Nielsen, J., Improving a Human-Computer Dialogue. Communications of the ACM, Vol. 33, 3, 1990. 13. Nielsen, J., Finding usabilty problems through heuristic evaluation. Proceedings of CHI '92, 373-380, 1992. 14. Nielsen, J & Landauer, T.K., A mathematical model of the finding of usability problems. Proceedings of INTERCHI '93, 206-213, 1993. 15. Nielsen, J. & Mack, R. (Eds., 1994): Usability Inspection Methods. New York: John Wiley
906 16. Normen, D.A., Cognitive Engineering. In: D.A. Normen & D.A. Draper (eds.), UserCentered System Design: New Perspective on Human-Computer-Interaction. Lawrence Erlbaum: Hilldale, 1986. 17. Poison, P.G., Lewis, C., Rieman, J. & Wharton, C., Cognitive walkthrough: a method for theory-based evaluation of user interfaces. International Journal of Man-Machine Studies, 36, 741-773, 1992. 18. Reiterer, H. & Oppermann, R., Evaluation of user interfaces: EVADIS II -- a comprehnsive evaluation approach. Behaviour and Information Technology, 12 (3), 137-148, 1993. 19. Software-Ergonomie in der Btirokommunikation (Software Ergonomics in Office Communication). VDI-Richtlinie 5005. Berlin Beuth, 1990 20. Yourdon, E. Modern Structured Analysis. Engewood Cliffs: Prentice Hall, 1989. 21. Ziegler, J & Janssen, C., Aufgabenbezogene Dialogstrukturen ftir Informationssysteme. In H.-D. B6cker (ed.), Software-Ergonomie ,95 Mensch - Computer - Interaktion, Anwendungsgebiete lemen voneinander, Stuttgart: Teubner, 1995 22. Ziegler, J., Objektorientierter Entwurf graphisch-interaktiver Informationssysteme. Tech. Rep. Fraunhofer-Institut IAO, Stuttgart (in German), 1995
V. Interface for Physically Challenged
This Page Intentionally Left Blank
V.1 Interface for Physically Challenged
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
911
Composition of M e s s a g e s on Winking by AI.S Patients Naoyuki Kanou ", Michio Inoue b and Yasuhiro Kobayashi b Electronic Control Engineering, Yon~agoNational College of Technology Hikona, Yonago, 683 JAPAN b Department of Information and Knowledge Engineering Faculty of Engineering, Tottori University Koyama, Tottori, 680 JAPAN This paper is concerned with a development of a Communication Aid (CA) for Amyotrophic Lateral Sclerosis (ALS) patients who lose the physical ability of communication. A CA is just the equipment that provides the facilities for conversation with them. In our CA, inquiries are arranged in a matrix form, and thrown to patients by means of crosswise scanning. If the patients find out the desired one, among the exhibited ones, they are requested to send a sign YES, by winking before a CCD camera. It is because winking is a most simple motion which can be done by a little physical power. As a matter of fact, wink happens also due to physiological demands. Hereupon it becomes necessary to discriminate the intentional wink from the unintentional wink clearly. This paper proposes a method of remote sensing of intentional winking as the sign of affirmation by a Time-Delayed Neural Network(TDNN), and describes how to compose their messages on their winking. 1. I N T R O D U C T I O N Patients with ALS are damaged in their physical capacity seriously so much as they cannot speak even a word, though retaining their mental capacity. Communication Aid (CA) is just the equipment that provide facilities for conversation with them. In detail speaking, the CA is requested to furnish the functions for interpreting their ideas and those for its literal expression. Their ideas can be barely informed to us by a simple motion on their meager physical power [1], [7]. As their possible motions, ALS patients can move their eyelids, or can turn their eyes intentionally. Let us consider that the ALS patient, a partner through our conversation, is requested to send a sign, if he finds out an affaumble point among our questions to him. On the other hand, we shall throw our questions and watch his affirmative response. If we can perceive his weak response from him, we can converse with him, though merger in the tempo on conversation [3]. To detect his physical motions, a strain-gage sensor and an eye-camera are usable.
912 In practice, the former is attached on his eyelids, detecting his wink. The later is set up at his spectacle frame, tracing his visual focus. Unfortunately, these sensors are not welcomed in spite of their excellent performance. Frankly speaking, any sensors of the attached-type are obstructive. It is because attachment of materials and their wiring before eyes at a close quarter give a feeling of psychological depression. Moreover, performance test is needed whenever these sensors were reset. In this paper, remote sensing technology using a TV camera and a time delayed neural computation are applied to detect the patient' winks and to discriminate their motivations which is due to consciousness or not. Its detail description is given as follows. 2. PERCEPTION OF HIS SIGN Conversation with the ALS patient is carded out in a form of Answers To Questions. His answer (affirmation only) is dispatched by shutting his eyelids intentionally. If missing his signs, we cannot understand his ideas In the present issue of our CA, a TV camera is focused to the ALS patient's face, taking his pictures at a rate of 60 frames per second. Each frame consists of 256H × 256W pixels with each 64 gradations of luminous intensity. These graphic informations are interpreted as 64H × 64W pixels with each 256 luminous gradations. To emphasize the edge line of the picture, its vertical differences are calculated. It is because pixels with each steepest descents of their luminance satisfy the edge lines of the monochromatic graph. For speeding up the data processing without deteriorating its accuracy, we can localize luminous informations enclosing the patient eye. Further, we can thin out the frames. At last, luminous informations are send to a Time-Delayed Neural Network (TDNN) [5], at a rate of 15 frames, each of which consists of 24W x 16H pixels. 3. TIME-DELAYED N E U R A L NETWORK TDNN consists of 3 layers. The first layer has 24 x 16 neurons receiving the luminous informations of the frame, consisting 24W x 16H pixels, and delivering them to the second layer (network). The second layer comprises 16 neurons, recognizing the either state of eyelids. Luminous informations as the input variables of the 1st layer are refreshed at every frame and the outputs of the 2nd layer are held until a sequence of processing for 16 frames are finished. The third layer, consists of a single neuron, recognizing time-dependency of the two states of eyelids, and discriminating the intentional wink from the unintentionals in a real-time operation by processing continuous sixteen frames without employing high -performance hardware. Fig. 1 shows the configuration of TDNN. 4. DISCRIMINATION OF T H E INTENTIONAL WINK The ALS patient, a partner of our conversation is requested to shut his eyelids, as his affinuative response to our inquiry. Sometimes, however, he shuts his eyelids unintentionally as an instinctive motion due to his biological demands. Such an unintentional motion makes us misunderstand his ideas. Motivations of his physical motion must be investigated.
913 1st layer
2nd layer
16th ! . , . ~ . . ~ ~ frame
foo
ith
;~"~-'---"~,O
frame [..'~'...
"-
/layer
2. = =
2nd . . . . . . . . frame ._, ~ : __ -- -1st t : z = ~ --- --. frame :~=====&._ ~ ---Fig. 1. Time- Delayed Neural Network.
ON
~ I ~ I ~ I I~'~'~'~1~1~1~1~1~1~1~1~1~1~1~1~1
OFF
I~I~I~I~I~I~I~I~I~I~I~I~I~I~I~I~I I~I~I~I~I~'~'~'~ ~,~,~,~I~I~IGI~IGI~I~I~I
OFFI~i~1~1~1~1~1~1~1~1~1~1~!~1~1~1~1 ON ~,~,~I~I~I~I~I~I~IGIGI~I~IGIGI~I [~1~i~ ~1~1~ O F F ~,~,~1~,~,~1~1~1~1~1~1~1~1~1~1~1 OFFI~1~1~1~1~!~1~1~1~1~1~1~1~1~1~1~1 ~ ~,~,~,~,~,~,~1~1 OFFI~1~1~1~~,~,~,~1~1~,~,~,~,~,~,~ Fig. 2. The series of frames for training.
Our careful observation drag out the fact that different motivations allow different duration of winking. In detail speaking, it takes approximately less than 0. 2 second in an unintentional wink. Contrary to this, an intentional wink requires much more time than the unintentionals. In this CA system, TDNN is taught to assume that if the scene of closed eyes is observed through continuous 10 frames, then the motion is performed intentionally. This criterion is determined by considering the frequency of our inquiry and the promptitude of the patient's response. Thus, the standard luminous patterns of the open eyes and the closed eyes are provided as the teaching materials. These are shown in Fig. 2. Classification of the received luminous informations is implemented by means of the neural computation. The weight for the neurons of the second layer are determined as the deference in luminous informations between two patterns, opened eye and closed eye. The weights for the neurons of the third layer are soon converged to certain values through training. These weights are commonly used for processing luminous informations of the following frames. The name of TDNN results from the architecture that input informations are refreshed at every frame, though maintaining the common weights [6]. 5. APPLICATION TO CA For talking to patients with serious ALS, we can not expect so much. A possible gesticulation is to use wink as a sign of their affnTmtive response to our inquiry. As mentioned above, intentional winks have been discriminated from the unintentionals. This provides a facility for conversation with them. In this CA, our inquiry is thrown to him about their compliances or their requests by imagining some scenes in hospital. These examples are tabulated in Table 1. In practice, these are exhibited in a form of telops on the CRT display unit. The patients have only to wink when they agree with the meanings of the extffoited telops. This CA can assist literal expression of their ideas. In this application, The patient specifies desired characters one by one among the syllabary shown in Fig. 3 and spell them along the grammar. Our inquiry is thrown by means of crosswise scanning with a constant speed. As for the composition, this CA permits a short approach where some idioms conducted by the first two characters
914
that the patient specifies. If the patient finds out a desired idiom, he can economize on his physical efforts to specify necessary characters. It is shown in Fig. 4. [2], [4]. An example of the AI~ patient's letters in Japanese are shown in Fig. 5. He spends about 40 minutes on this composition.
~. -t ~: ~ 9 ~
g ~.gv~ry
•
zaei}
z
(a) Column F¢'X/ is first selected by the column scanning.
~
b 4
, ?
~
6 t11~
05
~--T-
07 ~
~v,m
Fig. 4. Candidate words conducted by two characters.
Table I. An example of questions. .
.
.
.
.
.
.
.
.
.
(b) Character F(S._J is then selected by the row scanning. Fig. 3. A crosswise scanning for s e lecting the character F(S.J .
~~
?
~
?
~:~
?
¢~v, ~
? ?
~, ~)p ~
? ? ?
(SOMETHING TO EAT ?) SOMETHING TO DRINK ?) FEVERRISH 7) CHILLY ?) HARD BREATH ?) NAUSEOUS ?) ITCHY ?) SWEATY HOT WATER 7) COLD WATER ?) TEA ?) TOWEL ?) VENTILATION ?)
I
Translation: I have despaired of my future and feel deeply vexed in my life, cursing my domn every day. Thank you very much for your kindness of providing this facilities. This can dispel my gloom before a half year. My irritation ~ quieL I can talk to my family and friends, and moreover, I can write to them. They are deeply moved with lit~al correspondence. Wonderful !
Fig. 5. An example of the ALS patient's letters.
915
6. DISCUSSIONS 6. 1. Feature of our Time-Delayed Neural Computation The TDNN has an excellent attribute, such that it is able to classify the eye states by tracing their time-dependent variations on a common hardware. In practice, the neurons of the third layer pushes out results of training, that is the time dependency in the outputs of the second layer. The neurons of the second layer output the value of nearly One or Zero corresponding to the state of eyelid (Opened or Closed). The neuron of the third layer is trained to remember the series of One or Zero, these are transition of the state of the eyelid. These are shown in Fig. 6. Fig. 7 shows the weights of the 3rd layer. By setting the weight of the second layer as the difference between the opened eye and the closed eye, the neurons of the second layer can discriminate the states of the eyelid.
1
~
1
The neuronsof the 2nd layer
g 0' .~_~ [-.
0
(a) 2nd layer (b) 3rd layer Fig. 6. The independence of the 3rd layer from 2nd layer.
Fig. 7. The weights of the 3rd layer.
Table 2. Discrimination in composition. (each testee)
Table 3. Discrimination composition. (each distance)
distance testees
15cm A
B
C
testees
D
For successful 205 205 207 205 inten- discrimination tional failured 3 3 14 winking discrimination 5 For misuninten 2 4 7 3 -tional discrimination motion The focal distance" 4.7mm
distance
13
15cm 25cm 35cm
For successful inten- discrimination 207 tional failured 10 winking discrimination For uninten mis2 -tional discrimination motion The focal distance" 4.7mm
201 203 0
0
8
0
916 6. 2. Experiments by composition The authors test our CA through the experiments by composition. First, Five testees composed a short messages. Table 2 shows the result. The intentional wink could be recognized with high probability. There were two wrong cases in recognition. The one was caused by missing an intentional wink. The other was due to misjudgment as a intentional wink in spite of no sign. Secondary, a testee composed the messages at several distance form the TV camera. Table 3 shows this results. This was almost in agreement with the result of the fast experiment. Thereby, it is not necessary to set the position of the TV camera exactly. 7. CONCLUSION Intentional winks have been discriminated from the unintentionals. Its mathematical basis is on the difference in the duration of eye's motion. In a practical view, this achievement results from a remote-sensing technology in combination with a time-delayed neural computation. Upon this, we can understand the ALS patient's ideas by perceiving his wink as an affirmative response to our inquiry. This type of the communication aids can assist literal expression of his ideas. REFERENCES [1] Perry A. R., Gawel M. and Rose F. C., "Communication aids in patients with motor neurone disease", British Medical Journal, vol. 282, pp. 1690-1692, 1981. [2] Heckathome C. W. and Childress D. S., "Applying Anticipatory Text Selection in a Writing Aid for People with Severe Motor Impairment", IEEE MICRO, vol. 3 No. 3, pp. 17- 23, 1983. [3] T. Tokunaga, M. Inoue, Y. Kobayashi, N. Kanou, K. Inoue "Design of a Commu -nication Aid for a patient with Amyotrophic Lateral Sclerosis", IEICE report CAS 87-2 6, pp. 1- 8, 1987(inJapanese). [4] M. Inoue, Y. Kobayashi, N. Kanou, K. Inoue, "A Method of Word Processing for a Patient with Amyotrophic Lateral Sclerosis", Trans. of IPSJ, Vol. 33, No. 5, pp. 6 45- 651, 1992(inJapanese). [5] Y. Miyata, 'Neural Networks for Temporal Processing", Tranr~ of IEEJ, vol. 113, No. 6, pp. 372-377, 1993(inJapanese). [6] N. Kanou, M. Inoue, Y. Kobayashi, S. Inoue, K. Inoue "Detection of Winking for Communication Aids", IPSJ SIG Notes, Vol. 94, No. 74 pp. 9 - 14, 1994(inJapanese). [7] Wolfgang L. Zagler, Geoffrey Busby, Roland R. Wagner, "Computers for Handicapped Persons", Springer- Verlag.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
917
Development of Language Training System for Developmentally Handicapped Children K. Itohaand K. litaka b aDept, of Computer Science, Tokyo Polytechnic College 2-32 Ogawanishi-machi, Kodaira-shi, Tokyo 187, Japan bResearch Institute for the Education of Exceptional Children Tokyo Gakugei University 4-101 Nukiikita-machi, Koganei-shi, Tokyi 187, Japan A language training system with a personal computer is developed for developmentally handicapped children. They see action images presented on the display of the personal computer and learn action words with this system. They have a difficult problem to solve when they learn words. For consideration of the problem, the system has an improved method of presenting actions as follows. For each of the action words, the system presents action images from an actor's viewpoint and an observer's viewpoint in the form of a line-drawn animation and a movie by using multiple examples. The new language training system is applied to developmentally handicapped children. The effectiveness of the system in helping them to learn the action words is evaluated. The results obtained are that with each of the presentation methods, the correct answer rate in the post-learning test is higher than in the pre-learning test and that there are significant differences between the pre-learning and post-learning test results. 1. Introduction This study is concerned with the development of a language training system for developmentally handicapped children 1.2. The system presents to developmentally handicapped children a movie and an animation depicting an action word on the display of a personal computer and helps them to learn the action word. When a previous language training system was experimentally used on developmentally handicapped children, they were seen enjoying their learning experience.
918
Here is reported a new language training system developed with an improved method of presenting actions to developmentally handicapped children. They find it difficult to generalize action words when they learn them. Each action word is presented in the form of a movie and a line-drawn animation from two viewpoints, or actor's viewpoint and an observer's viewpoint, by using multiple examples. The new language training system is applied to developmentally handicapped children, and their test performance is compared before and after learning. evaluated.
The effectiveness of the system in helping them to learn action words is The action words "Cut" and "Tear" were selected from among those
action words that are considered difficult for developmentally handicapped children to distinguish (Morioka et al., 19903). 2. Language training system The language training system is divided into learning and testing phases. The examples presented to subjects are given in Table 1. In the learning phase, three examples are presented for each of the action words "Cut" and "Tear". In the testing phase, an additional example is also presented. Since the system presents images from the actor's viewpoint and the observer's viewpoint in a movie and an animation, four problems can be presented in different ways for one example. In other words, 24 (2 x 3x 4) problems are presented in the learning phase, and 32 (2 x 4 x 4 I problems are presented in the testing phase. Table 1. Examples presented on display Learning software Testing software .3ut
Cut a carrot with a kitchen knife. Cut fingernails with a pair of clippers. Cut a wooden plank with a handsaw.
Tear (Tear a bag.) Tear paper. Tear packaging paper. Tear a sheet of the calender.
Cut a carrot with a kitchen knife. Cut fingernails with a pair of clippers. Cut a wooden plank with a handsaw. Cut a cake with a knife. (Tear a bag.) Tear packaging paper. Tear a sheet of the calender. Tear an envelope.
2.1. Hardware configuration
A Macintosh Ilfx personal computer is used in the language training system. The monitor is equipped with a touch panel so that developmentally handicapped children can learn the presented action words by touching the screen.
919 Sound and images are introduced through a digitizer and used to create software on HyperCard.
A movie is a digital movie entered from a videotape. An
animation is a kind of "flipping" cartons created from three line-drawn pictures. 2.2. Software configuration The configuration of the learning software is shown in Figure 1.
Since a
subject learns within 20 minutes at a time, 24 problems are divided into two groups, and 12 problems are given in one experiment.
When the subject
touches the problem display screen, a movie or an animation is started.
After the
movie or animation is finished, a question like "What is he doing?" is asked of the subject.
If the subject gives a correct answer, a picture and sound are presented
to encourage him or her to learn more.
If the subject does not give a correct
answer, the spelling and the pronunciation of the correct action word are presented.
The problem is given repeatedly until the subject can give the correct
answer. The configuration of the test software is shown in Figure 2. are given in one experiment.
Sixteen problems
The reaction time and answer type (correct
answer, wrong answer1, wrong answer2, no-reaction) are noted.
Wrong Voice ans. "~ I Start ! ~_~Letters LJAmovleor I I Pr°bleml1 ~ A ananimationl
picture,Ill
[Problem2 ! IProblem3 1 ! Pr°blem4
I Correct .j.]_. ans v [ " Sound i
I
1I I Pr°blem11 I I [Problem12 !
- -Reaction • timer Start ! -.L{ Correct ans. J--Problem1 -~ Wrong ans.1F i [Problem2 I 1 -~ No reaction F JProblem3,.J IProb,e'm4! ~-{ Wr°ng ans-2 i"I 1!
l ro 'e S ! I
Fig 1. The configuration of
Fig 2. The configuration of
the learning software
the test
software
3.Results and discussion Using the newly developed language training system, pre-learning tests were conducted on 12 developmentally handicapped children.
The subjects range
from 7 to 12 in chronological age and from 5 to 9 in vocabulary age. Subjects N and Y who had to learn action words were selected. the presented action words and were tested after learning.
They learned
The chronological
920
age is 7 for subject N and 9 for subject Y. The vocabulary age is 6 for subject N and 5 for subject Y. The way the subjects participated in the experiments was videotaped.
3.1. Pre-learning test The pre-learning test results arranged as classified for the action words "Cut" and
1oo, 80 "Tear" are shown in Figure 3. Wrong 6c answer1 in Figure 4 means that the subjects 4( mistook "Cut" for "Tear" or "Tear" for "Cut." 2( Wrong answer2 in Figure 4 means that they C had any other answer. The rate of correct
answer is nearly 70% for "Cut" but less than 40 % for Tear." There are significant differences between the results for "Cut" and "Tear." The rate of wrong answer1 for
Fear i ~11k3 I ~ a ~ . , L l ~ . , l l
I
Wrong2 Fig 3. The pre-learning test results arranged as classified for "Cut" and "Tear".
"Tear" is higher than for "Cut." The subjects mistook "Tear" for "Cut" more often than "Cut" for "Tear."
The subjects exhibited
results similar to those of the preliminary study where still pictures were presente~. The videotape shows, however, that they uttered much more words than in the preliminary study. In the preliminary study, most of the subjects gave only answers, in this study, when motion pictures were presented, the subjects uttered cries and talked about the motion pictures by touching them on the display screen. When sound was presented, the subjects said "Big sound!" as well as "Gohgoh" or "Bribri" to imitate the sound. The pre-learning test results arranged as classified for the examples are shown in Figure 4.
The rate of correct answer for the example "Cut a carrot with a
kitchen knife" is more than for the other examples "Cut .... " The example is often presented to the subjects in a picture-word matching vocabulary test. more familiar to them.
It may be
The rate of wrong answer2 for the example "Tear
packaging bag" is higher than for the other examples "Tear .... ". This is because many subjects answered "Open packaging bag."
There are significant
differences between the results for the example and for the other examples "Tear .... " In the language training system, the subjects were made to decide action words only when action images were presented. The subjects found it difficult to decide the action word for the example without knowing the situation where the action was actually performed. The results for the example are thus excluded from the pre-learning and post-learning test results.
921
/ ~ \
(%)
(%/
1 o(
1 oo
8 6 4 r,
"-r-A
_
_
80 6(: 4( 2t
4
8
Co No reaction No-reaction Wrong2 Wrong2 Fig 4. The pre-learning test results arranged as classified for the examples. 1. Cut a cake with a knife.
5. Tear a bag.
2. Cut a carrot with a kitchen knife.
6. Tear an envelop.
3. Cut fingernails with a pair of clippers. 7. Tear packaging paper. 4. Cut a wooden plank with a handsaw. 8. Tear a sheet of the calender. 3.2. Learning When the two subjects did not give correct answers, they were not only presented with action images, but also made to imitate the actions to learn the action words. The videotape shows that subject N was pleased with the pictures and sound presented to encourage him to learn and wanted to have them. 3.3. Post-learning test The results of the pre-learning and (%) post-learning tests of the two subjects loo are shown in Figure 5. In the 8o 6o pre-learning test, the rate of wrong answer2 is 70%, and the subjects
In the
4c 2( (
post-learning test, the rate of correct
Cc
made many wrong answers.
answer is about 90%. There are
significant differences between the pre-learning and post-learning test
~ost-learning -learning
......... "Wrong2 Fig 5. The pre-learning and post-learning
results, suggesting the learning effect.
test results for two subjects
Some examples were presented in the learning phase, and some were not (Table 1).
The test results were arranged as separated for the former and the
latter. The rate of correct answer for each of the two groups in the post-learning test is higher than in the pre-learning test.
There are significant differences
between the pre-learning and post-learning test results for the two groups.
The
922
increased percentage of correct answers for the examples not presented in the learning phase indicates the possibility of generalization occurring. The language training system presents examples by four different methods. The different presentation methods produced no significant differences between the pre-learning and post-learning test results, but one subject produced different test results depending on the presentation methods employed. In a future study, more subjects will be employed to study individual differences between them. 4. Summary A language training system with a personal computer is developed for developmentally handicapped children. The system is improved in the method of presenting actions to the developmentally handicapped children who find it difficult to generalize action words when they learn them. The new language training system to learn the action words "Cut" and "Tear" is built and applied to two subjects. Their test performance is compared before and after learning. The correct answer rate in the post-learning test is much higher than in the pre-learning test. The learning effect is thus suggested. The correct answer rate for the examples not presented in the learning phase is also increased. The possibility of generalization occurring is indicated as a result. Individual differences between the subjects will be studied in the future. Acknowledgements We would like to express gratitude to the pupils and teachers in a special class for developmentally handicapped children at the K elementary school. References 1. K. Itoh, K. litaka, The 8th Congress of the Japan Educational Technology Society, (1992), 262-263. 2. K. Itoh, K. litaka, The 9 th Rehabilitation Engineering Conference, (1994), 341-344. 3. N. Morioka, K. litaka, Master"s thesis submitted to Tokyo Gakugei University in fiscal 1990.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
INTERACT: disabilities
An
interface
builder
923
facilitating
access
to
users
with
C. Stephanidis and Y. Mitsopoulos Institute of Computer Science, Foundation for Research and Technology - Hellas, Science and Technology Park of Crete, P.O. Box 1385, Heraklion GR-71110, Crete, Greece
This paper describes INTERACT 1, a tool for the specification of interaction dialogues and the construction of user interfaces appropriate for various categories of users, including people with disabilities. INTERACT builds on the notion of separating an application in two functional components, namely the application functional core and the user interface component, thus allowing the provision of multiple user interfaces and supporting the "highlevel" design of the interaction dialogue, i.e. independently from the presentation details and operational constraints of a particular technological platform.
1. INTRODUCTION Recent efforts addressing the problems of accessibility to graphics based applications by people with disabilities have mainly addressed and explored two alternatives. The first, involves the enrichment of interaction objects of existing toolkits. The second concentrates on the development and embodiment of new toolkits into new User Interface architectures and tools [3]. Following the first approach, INTERACT is a user interface construction tool that aims towards the enrichment of interaction objects of existing User Interface toolkits. While INTERACT exhibits the majority of the characteristics of other state-of-the-art User Interface Builders [2],[4],[5], it also facilitates the development of graphics based applications for disabled users through the provision of enhanced user interface customisation possibilities. More specifically, INTERACT supports different interaction styles through the utilisation of alternative human and computer interface channels and media; the audio, visual and haptic modalities can be selected by the user interface designer, taking into consideration the characteristics of the target user grpup and the scope of the particular application.
1Part of this R&D work has been carried out in the context of the IPSNI-II (R2009) project, which is partially funded by the RACE II Programme of the Commission of the European Union (DG XIII). The partners of this consortium are: Consiglio Nazionale delle Ricerche (IROE-CNR), Italy; Centro Studi e Laboratori Telecommunicazioni (CSELT), Italy; Institute for Rehabilitation Research (IRV), The Netherlands; Dundee University Microcomputer Centre, UK; Katholieke Universiteit Leuven, Belgium; Institute of Computer Science - FORTH, Greece; Technical Research Centre of Finland (VTI'), Finland.
924 2. DESIGN AND DEVELOPMENT WITH INTERACT The underlying concept of INTERACT is that the enhancement of existing toolkits with additional "look and feel" styles can facilitate the development of graphics based applications accessible by a wide user population, including people with disabilities. In addition to the standard I/O communication channels supported by existing toolkits for developing graphics based applications, INTERACT introduces additional features regarding: the media and modalities used during interaction, - the employed interaction techniques, the utilised I/O devices, navigation in the user interface, - feedback provided to the user, etc. -
-
-
-
~-to-Servlce
~
action Dialogue
~
i zeauZta
in...
Abstract User-to-Service Interaction Dialogue /
is. " t |
Presentation Independent Interaction
Obkct
~o'? ~
.... ~-I
:!
'.... '
User Interface
I
.=1
' !l
is..
Technology Platform Independent
Interaction Techniques and I/O Devioes
=,: •: •. :! :: ::
Presentation Dependent Interaction Object
* .... VO Devices Spe#~
Independent ::
1st S t a g e 2nd Stage ,,,.........o..........,,,.,..,.=.=.=..=........==....=.. ==.. .... ...................... : ................=.,...................................o..=..,.=.,,,--...=..,,,
Figure 1.
The stages in the user interface construction process using INTERACT.
For instance, in order to support the development of graphics-based applications accessible by blind users, INTERACT enhances the graphical objects with additional attributes (e.g. presentation in auditory or tactile form) and provides facilities for switching between the applications, as well as facilities for the exploration of the graphical objects of the various applications. User interface construction with INTERACT takes place in a two stage process (see Figure 1). During the first stage, the designer interactively constructs the dialogue by means of a hierarchical (tree) structure of abstract interaction objects. The second stage involves the "binding °' of the abstract interaction dialogue to produce a specific User Interface. Both the abstract interaction dialogue and the target user interface have interaction objects as building blocks. The abstract dialogue consists of interaction objects that possess only semantic / functional properties (i.e. abstract interaction objects). The end user interface consists of physical instantiations of the abstract interaction objects which inherit the semantics of their abstract correspondents while conveying the presentation, manipulation, control and feedback characteristics of the target technology platform (e.g. OSF Motif). For example, at the first stage of the design process (i.e. abstract level), an interaction object could be a Menu (i.e.
925
10I~ sI I de'.15.1 ms~aGe.17o1
/bleed= ~) Buttons
I button_24.4
button_ 18_3 button_9_2 button_B_1
(ffecter:. ~) Shells "
-i
lshell_4_3 shell_3_2 shell_2_1
Speech Options (~) SpeechOff
Interface Type (~ Cruphlcsl When: [~
Left mouse button pressed
Action: (~ HI~
@,r~J:
Technology Platform (~ MOTIF
View Connections: (~) Buttons
I button.~4_4 (left mouse button pressed/-> shell_4_3 (Show) button_18_3 (left mouse but'ton pressed)-> shell_q_3 (Hide)
CCreateOraw ,~rea) Ccreate o~..... ~)
Figure 2.
A snapshot of the working environment in INTERACT.
abstract selector), which can be mapped, at the physical level, to an Option List, Pop Up Menu, etc. The design of the user interface, both at the abstract and physical levels, is performed by "easy-to-use" interactive facilities; thus, non-computer experts, human factors specialists, rehabilitation experts, etc, could easily and efficiently design graphics-based applications accessible by users with disabilities. At present, INTERACT addresses the needs and requirements of blind, low vision and motor-impaired users through the provision of appropriate interaction techniques and the utilisation of suitable input and output devices for each user category. In this context, the following interface styles are supported: - tactile based user interfaces, - speech based user interfaces, - combined speech and tactile based user interfaces, - large size widgets, - large size widgets with auditory cues, - large size widgets and speech.
926 3. SYSTEM ARCHITECTURE OF INTERACT The main parts of the system (shown in Figure 3) are the following: a) The user interface component INTERACT provides "easy-to-use" interactive facilities for the design of graphical user interfaces. Currently, the implementation is based on the XView toolkit. One of the key subcomponents is the graphical browser which provides the dialogue designer with "standard" browsing facilities as well as management facilities for the dialogue trees (e.g. hierarchical structure of interaction objects). Type checking facilities are also supported (e.g. a menu interaction object should not contain valuator or command interaction objects as options). The depicted dialogue tree is "translated" into an internal representation and "saved" in the internal dialogue representation module. b) The internal dialogue representation module (IDRM) This module is responsible for the mapping of a particular dialogue tree to a corresponding high level dialogue structure with embedded dialogue sequencing and control schemes; it "transforms" the dialogue designer's conceptual model, which is represented graphically in the dialogue tree, into the corresponding abstract dialogue specification which is still presentation independent. The internal dialogue representation module cooperates with the code generation and attribute modification components that are responsible for matching and translating the already designed interaction dialogue to a specific technology platform with a specific I/O configuration and a specific set of interaction techniques that are suitable for the target user group. c) The attribute modification component (AMC) The attribute modification component provides online (i.e. interactive) restructuring of the interaction dialogue by altering its internal representation, as well as facilities for the modification of the attributes of the particular interaction objects. The attribute modification component communicates with the library modules either for the modification of presentational attributes of the particular interaction objects or for the modification of manipulation/control and feedback attributes. d) The code generation component (CGC) Taking into account the target technological platform that the dialogue designer wishes to use for the realisation of the end user interface (e.g. OSF Motif), the code generation component "maps" the internal dialogue representation to a target software module, using relevant I/O device related functions and interaction techniques.
e) The library module The library module consists of three "sublibraries", namely the technology platforms library, the I/0 device related functions library and the interaction techniques library. More specifically: - The technology platforms library includes the required software modules for the attribute modification/code generation components. Archetypes of the various technology platform specific interaction objects, such as OSF MOTIF, MS Windows [ 1] (i.e. widgets, controls, etc) are included in this module to be used during the realisation of the abstract dialogue for a specific technological platform and the generation of the target software module.
927
INTERACTUser InterfaceComponent Graphical Browser and Management
Tree
m
n
"% .... , r
Attribute Modification Component (AMC)
INTERNAL , DIALOGUE REPRESANTATION ODULE I (MDRM)
~ ~' •
Code Generation Component
'
(CGC)
---!i
i Library Technology Platform Related Libraries
Figure 3.
Modules I/O D e v i c e s Related Functions
Target User Group Interaction Techniques
Runtime architecture of INTERACT.
- The 1/(9 device related functions library provides the "interfacing" of the particular interface code with the utilised I/O devices and facilitates handling of I/O events at run time. - The interaction techniques library mainly includes the supported interaction techniques for the particular target user groups; it is used both during the realisation / online attribute modification and code generation processes. The library modules have been implemented in such a way so that new functions can be incorporated; no global recompilation of already existing modules is required.
4. DISCUSSION AND FUTURE WORK
Preliminary tests with different user groups have confirmed the practical value of INTERACT. More systematic user trials are foreseen. Work currently under way seeks to augment the supported user interaction styles through the further exploitation of the haptic and audio channels. In addition, design assistance is foreseen by means of a module that provides the user of INTERACT with suggestions for lexical aspects of the User Interface such as the presentation of interface objects with respect to the user group under consideration. Future work envisages the automated production of "log files" supplementary to the
928 generated software modules. These "log files" can be used by the designer in the early design phases in order to improve the usability of the interfaces under development.
REFERENCES
1. R. Chimera, Evaluation of Platform Independent User Interface Builders, Proceedings of the 10th Annual Symposium and Open House of the Human-Computer Interaction Laboratory, Center for Automation Research, University of Maryland, June 1993. 2. B. Myers and D. Olsen Jr, User Interface Tools, Tutorial 36 Notes, CHI'94 Conference on Human Factors in Computing Systems, 1994. 3. C. Stephanidis, A. Savidis and D. Akoumianakis, Tools for User Interfaces for all. Paper to appear in the Proceedings of the 2nd TIDE Congress, La Villette, Paris, April 26-28, 1995. 4. U. Thakkar, G. Perlman and D. Miller, Evaluation of the NeXT Interface Builder for Prototyping a Smart Telephone, SIGCHI Bulletin, January 1990. 5. OpenWindows Developer's Guide 3.0.1, User's Guide, Sun Microsystems, 1993.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
929
Supporting Blind and Sighted User Collaboration through Dual User Interfaces using the HOMER System A. Savidis and C. Stephanidis Institute of Computer Science, Foundation for Research and Technology - Hellas, Science and Technology Park of Crete, P.O. Box 1385, GR-71110 Heraklion, Crete, Greece The emergence of Graphical User Interfaces has introduced additional problems regarding the accessibility of computer systems by blind people. The implications include restricted opportunities for computer-based collaboration between blind and sighted users in a working environment. Currently, accessibility to graphical User Interfaces by blind users is enabled through systems which reproduce the lexical structure of User Interfaces (i.e. interaction objects and their relationships) in a non-visual form; such systems introduce visually oriented concepts in the context of non-visual interaction. The concept of Dual User Interfaces has been defined as a more efficient and effective approach to address the accessibility and collaboration problems. A User Interface Management System, called HOMER, has been developed for the construction of Dual User Interfaces 1. 1. INTRODUCTION The emergence of Graphical User Interfaces has initially excluded accessibility to computer systems by blind users. However, special purpose software has been developed for reproducing the lexical level of interaction (i.e. presentation and dialogue aspects of interaction objects, hierarchical structure, etc) in a non-visual form. Such methods are characterized as adaptation oriented methods and are usually tightly coupled to the target graphical environment (e.g. MS-WINDOWS TM, Macintosh TM, PRESENTATION MANAGER TM, X WINDOWING SYSTEM). It is argued that, even though these methods have offered partial solutions to the problem, there are a number of outstanding issues to be resolved: (i) All adaptation oriented approaches achieve dialogue reproduction based on lexical level information. No knowledge concerning the application semantics and the particular application domain can be extracted and, consequently, the semantics of the visual application are not taken into consideration during non-visual reproduction. (ii) Reproduction of the dialogue in a non-visual form is based on the visual dialogue structure. However, such a dialogue structure is constructed in accordance to the specific needs of sighted users. Moreover, with adaptation oriented approaches fixed dialogue decisions are implicitly taken
1 This work has been partially funded by the TIDE Programme of the Commission of European Union (DG XIII), under the project GUIB-II (TP 215). The partners of the GUIB consortiumare: IROE-CNR, Italy; Institute of Computer Science-FORTH, Greece; Vrije Universiteit Brussel, Belgium; Department of Computer Science-FUB, Germany;Institute of Telecommunications-TUB,Germany;IFI, Universityof Stuttgart, Germany; V'Iq', Finland; RNIB, England; F. H. Papenmeier GmbH&Co, KG, Germany.
930 for all non-visual User Interfaces. Consequently, there is no support for non-visual interface design; this contrasts one of the main principles of good User Interface design. (iii) Considering that the trend of User Interface software technology is towards interaction methods aiming to enable maximum exploitation of the human visual information processing capability, such as virtual reality and 3D representations (i.e. visual reality), it is expected that the employment of adaptation oriented techniques will become unrealistic or meaningless in the future. (iv) Finally, there are no methods or tools available at present which enable the development of non-visual User Interfaces. If the implementation of a dedicated non-visual application is required, interface developers are faced with the problem of lack of tools. In order to effectively support the socio-economic integration of blind users and to prevent possible future segregation in their working environment, it is argued that there is a need for more powerful User Interfaces than the adapted versions of existing visual User Interfaces. In this context, the concept of Dual User Interfaces has been defined and is characterized by the following properties: (i) it is concurrently accessible by blind and sighted users in order to enable collaboration; (ii) the (possibly different) visual and non-visual metaphors of interaction meet the specific (different) needs of sighted and blind users respectively; (iii) the (possibly different) visual and non-visual syntactic and lexical structures meet the specific (different) needs of sighted and blind users respectively, thus, the non-visual dialogue design may be different from the visual interface design; (iv) at any point in time, the same internal (semantic) functionality should be made accessible to both user groups through the visual and non-visual "faces" of the Dual User Interface (i.e. the What You Can Do Is What I Can Do principle); (v) at any point in time, the same semantic information should be made accessible through the visual and non-visual "faces" of the Dual User Interface (i.e. a type of What You "See" Is What I "See" for semantic information). An interface development system has been designed and implemented, called HOMER [3], which falls in the domain of User Interface Management Systems, and supports the construction of: (a) dedicated visual User Interfaces, (b) dedicated non-visual User Interfaces and (c) Dual User Interfaces. The HOMER system provides a high-level language for Dual User Interface specification, the HOMER language, and translates such a specification into a C++ implementation. Also, the HOMER system provides facilities for integrating different visual and non-visual lexical technologies (i.e. interface toolkits). 2. THE PRINCIPLE OF DUALITY IN INTERACTION One of the basic requirements of Dual User Interfaces is to facilitate concurrent access by blind and sighted users (i.e. collaboration). There are a number of significant dimensions which concern concurrency of interaction with respect to the visual and non-visual faces of a Dual User Interface, as is shown in Table 1. The A dimension is related to concepts employed for communication between the two user groups (i.e. discussion during collaboration) and can be either lexical (e.g. "this menu, this button"), syntactic (e.g. "I am doing editing") and semantic (e.g. " the recording volume has changed"). The B dimension concerns the metaphors of interaction for the visual and non-visual environments which can be either identical (e.g. Desk-top and Desk-top) or different (e.g. Desk-top for visual and Rooms [4] for non-visual). The C dimension characterizes the freedom that blind and sighted users have on performing actions independently of each other. More specifically, the dialogue control can be either synchronous (i.e. the visual and the non-visual dialogues always pass from the same "states" by progressing in strict parallelism), semisynchronous (i.e. users have
931 freedom to focus on different tasks; however, at certain points synchronicity is imposed) and asynchronous (i.e. users may focus on different tasks, interacting on different interaction objects and performing different actions). The D dimension characterizes the flexibility on the physical structure of the visual and non-visual faces of a Dual User Interface. It is possible to have identical structure (e.g. same object classes and instances, same layout, same object hierarchies), similar structure (e.g. some of the interface components and object classes are different, however, the overall physical similarity is evident) and totally different structure (e.g. the hierarchical organization of objects is completely different involving different classes of objects - no physical similarity can be identified). The E dimension concerns the flexibility of applying a different dialogue design for each environment. The dialogue structure can be either identical (e.g. same decomposition of user tasks) or different (e.g. different organization of tasks for blind and sighted users). Finally, the F dimension concerns the type of collaboration which can be either local (i.e. users work on the same machine and are physically close to each other) or remote (i.e. users work on different machines - distant collaboration).
Concepts for communication between users {Lexical, Syntactic, Semantic } B~
Metaphor relationship between visual and non-visual faces {Identical, Different }
C -_
Dialogue control between visual and non-visual faces {Synchronous, Semisynchronous, Asynchronous } Lexical structure between visual and non-visual faces {Identical, Similar, Different}
E~
Dialogue structure between visual and non-visual faces {Identical, Different }
F~
Collaboration type {Local, Remote }
I
Table 1: Dimensions of concurrent visual and non-visual interaction. The properties related to the concurrency of interaction for Dual User Interfaces can be defined by any 6-tuple from the set A x B x C x D x E x F. In comparison, the concurrency supported by adaptation oriented approaches has properties which belong to the following set: A x {Identical} x {Synchronous} x I1dentical, Similar} x {Identical} x {Local}. It is evident that apart from the theoretical and practical drawbacks from which adaptation oriented approaches suffer (as previously mentioned), the type of collaboration supported is considerably restricted. In Figure 1, the two modes of collaboration between blind and sighted users are illustrated for Dual User Interfaces generated by the HOMER system. The nonvisual technology that has been integrated in the HOMER system is the COMONKIT toolkit [4]. This toolkit complies with the purpose-designed Rooms metaphor [4] and supports non-
932 visual interaction based on speech, braille and non-speech audio output, while keyboard is used for input. The visual lexical technology that has been integrated is the Athena widget set of the X WINDOWING SYSTEM. * Local c o l l a b o r a t i o n m o d e
* Remote collaboration mode
! i i
m£ghted umI~
network
Braille
~
keyboard , audio' "]i a'='ll~mm=n=~
mouse
•/
/
blind u80r
keyboard
Figure 1: Collaboration modes for Dual User Interfaces developed using the HOMER system.
3. AN OVERVIEW OF THE HOMER SYSTEM The HOMER system enables the development of Dual User Interfaces by reducing implementation to specification. It provides a powerful specification language [3] which encompasses many innovative features with respect to existing UIMS languages. It supports metaphor independent interaction objects (called virtual interaction objects in HOMER terminology) which stand one level of abstraction above platform independent interaction objects. The HOMER language by itself provides no built-in interaction objects, but supplies mechanisms for introducing objects both at the physical (i.e. metaphor-specific or platformspecific level) and at the virtual level (i.e. pure abstract level). The dialogue control model, which is based on agents and has some relationships with the PAC model [2], is reflected in the language by explicitly introducing the notion of agent constructs (called dialogue agents in HOMER). Event handlers are supported and can be attached to interaction objects. The technology integration interface model, which is based on interaction objects and Input/ Output events, constitutes an important extension of existing technology integration methods [1]. The HOMER language enables distinction between visual and non-visual physical constructs through the qualifiers visual and nonvisual respectively. The lexical technologies are integrated in the HOMER system by means of implementing technology servers. It should be noted that, at run-time, the visual and the non-visual technology servers can be both handled in parallel. In Figure 2, the utilisation of the HOMER system is outlined. A proposed design methodology is discussed later on. The HOMER system will transform a Dual interface specification to a C++ implementation. The HOMER language provides powerful notational methods for the specification of the application interface by combining transactions on a
933 shared space with message channels. The run-time architecture of Dual User Interfaces developed with the HOMER system is based on the Dual run-time model [3] which constitutes an appropriate enhancement and extension to the Arch model [5] in order to fulfil the requirements of Dual User Interfaces.
O tilt| ues=g-"-Rl--'ner
sighted user needs % '
~
.
blinduser~ / ~ 0
t11t
needs ~
~
"
~
designer
d
g
sighted user
Dual User
Interface
5
~nput
d HOMER L ~ ~ r .
blind user
Figure 2: Using the HOMER system to develop Dual User Interfaces. 4. DUAL USER INTERFACE DESIGN M E T H O D O L O G Y The goal of the Dual User Interface design process is to preserve two fundamental principles regarding the relationship of the visual and non-visual interfaces: (i) independent but integrated, where there should be no interference or conflict of user requirements between the two environments, and (ii) similar but different, which signifies the similarity of the two faces at an abstract level. It is this similarity on which the design process is based. A decomposition of the design process in a number of distinct phases has been defined. Firstly, identification of the semantic services which are provided by the functional core is carried out. Since in many situations it is primarily more important to focus on the type of data "exported" by the functional core and secondly to identify the operations which can be applied on them, it has been decided that in the first phase both the identification of the functional services (i.e. supported semantic functions) and of the data / information structures will be performed in parallel and at the same level of priority. Formally, this is called the semantic services and semantic data
identification phase. The purpose of the User Interface is to realize methods of accessing internal functionality and applying it interactively on the internal information structures. Hence, the second task of the design process is to identify such "manipulation methods" for the internal structures. Such methods are to be made interactively available to blind and sighted users, hence, in one sense
934 they formulate a high-level description of what the user has to accomplish (i.e. user tasks). Thus, this part of the design process has been named abstract task identification phase. Next, for each abstract task it is necessary to identify the way in which each user will carry out specific actions. The key issue during this phase, where the transition from abstract to concrete is realized, is to ensure that the semantic information which is made accessible to one user group is also made accessible to the other in the same qualitative manner (i.e. the What You Can Do Is What I Can Do principle). Such rules must be clearly stated in this design phase so that it is always well defined when consistency of representation should be applied and on which information structures. The above phase concerns the description of user specific actions (user task identification phase). Finally, the design of the physical aspects of the visual and non-visual faces is to be carried out by addressing issues related to the physical appearance. This is called the physical design phase. 5. DISCUSSION AND CONCLUSIONS Existing approaches addressing the problem of accessibility of Graphical User Interfaces by blind people are based on adaptations of visual interactive applications at the lexical level. However, these methods are associated with significant theoretical and practical problems. To avoid segregation of blind users in their working environment, it is critical to efficiently and effectively facilitate computer-based collaboration with sighted users. To address these problems, the concept of Dual User Interfaces has been defined and a User Interface Management System for developing Dual User Interfaces, called HOMER, has been constructed. Using the HOMER system, a number of experimental interactive applications with Dual User Interfaces have been built like a payroll management system, an application providing description of graphical pictures annotated with text and a personal organizer, while preliminary trials have demonstrated the practical value of the adopted approach. The Dual User Interfaces generated by the HOMER system may run either in local collaboration mode or in remote collaboration mode. The HOMER system realizes an efficient approach for eliminating accessibility and collaboration problems from the development phase (i.e. proactive approach) in contrast to the more restrictive adaptation oriented approaches which necessarily attempt to "react" to problems arising from new technological developments (i.e. a reactive approach). REFERENCES 1. CMU/SEI-91-UG-8, Guide to Adding Toolkits, Serpent User's Guide, 1991. 2. J. Coutaz, Architecture Models for Interactive Software: failures and trends, in G. Cockton (ed.), Engineering for Human-Computer Interaction, North-Holland, 137-151, 1990. 3. A. Savidis and C. Stephanidis, Developing Dual User Interfaces for Integrating Blind and Sighted Users: the HOMER UIMS, to appear in the Proceedings of the CHI '95 Conference on Human Factors in Computing Systems, Denver, Colorado, May 7-11, 1995. 4. A. Savidis and C. Stephanidis, Building non-visual interaction through the development of the Rooms metaphor, to appear in the Conference Companion of the RCHI '95 conference on Human Factors in Computing Systems, Denver, Colorado, May 7-11, 1995. 5. The UIMS tool developers workshop, A Metamodel for the run-time architecture of interactive systems, SIGCHI Bull 24(1), 32-37, 1992.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
935
D e v e l o p m e n t of Human-oriented Information systems - Learning with mentally handicapped people Yasuko Kaminuma Department of Information Systems, Faculty of Informatics, Teikyo University of Technology, 2289, Uruido, Ichihara, Chiba, 290-01, Japan
ABSTRACT Recently, it became increasingly necessary to obtain information systems professionals who can take into consideration the harmony between technologies and human beings/ societies. How can these information systems professionals be obtained? Can they be educated in universities? To study these problems, we tried to utilize action research. In this paper, we present information systems education report in rehabilitative training schools for mentally handicapped people as one example of practices in university education.
1. INTRODUCTION In recent years, the environment of the facilities or resources relating to information systems has been changed. Accompanied with this, variation appears also in human activities in relation to computers. System engineers should be aware how they should do requirement analysis of the information systems reflecting situation changes, and should comprehend correctly what the optimum requirement specifications are. On the other hand, information system handlers also should acquire the ability to allow themselves to analyze information and actually utilize the result of the information analysis. For that purpose, university undergraduates who are to be engaged in the work of information systems are required to cultivate the power of concept for expressing themselves. In this paper, a report is made with the examples of the information system development exercises in rehabilitative training schools for mentally-handicapped people as practices in university education.
2. DEVELOPMENT OF HUMAN-ORIENTED INFORMATION SYSTEM
Downsizing and networking are now in progress, and development environment of information systems has been changed. As a result of this, dispersion of information resources is also in progress. Confronted with such a situation, improvement of user
936 interfaces is in requirement and betterment of the development technology of the information systems is furthermore required. The theme of the development project has been expanded from settled business to un-settled one, and moreover attention is paid also to the aid of a variety of social problems. On the other hand, autonomous dispersion of the systems has compelled their end users to seek enhancement of knowledge and technology. Furthermore the operability of the information systems has become an objective of evaluation, and importance has been attached to the object orientation or multi-media technology. In this paper, mechanism of information systems composed of human and mechanical systems concerning collection, processing, transfer, and utilization required for human activities is dealt with as an objective. An information systems is a system which assembles, stores, processes and delivers information relevant to an organization ( o r to society ), in such a way that the information is accessible and useful to those who wish to use it. An information systems is human activities system which may or may not involve computer systems. It is important for system developers to explain a problem concerning resources such as budget, manpower, terms, etc., a problematic point regarding planning or administration of developing projects, etc. However concurrently with this, both developers and users should agree with themselves on the matter what the contents of the information system to be developed are. This is the most difficult problem in the system development. To comprehend a problem on the matter of development, it is one of the most important assignments to investigate what the education that universities can perform is. We have made investigation with respect to the existing techniques or methodologies that can be applied effectively under new environment. This has induced us to do action researches[I-2] as a method to do requirement analysis of the existing information system, to build up a prototype, and to estimate whether the requirement analysis has been made correctly or not. In accordance with this method, problematic points or required matters have been made clarified under interference by entering the users' problem region. Accordingly the method in question directly drives analyzers to be involved into action processes, and it might be permissible to state that the method is quite an adequate one for dissolving such a kind of problems where importance is attached to human activities. That is to say, it becomes easy to add new settlement of problem affairs, to re-construct a theory, or to combine plural situations by the circumstance that analyzers are involved to a situation where theoretically correct tests can be done.
3. FUNDAMENTAL EDUCATION OF INFORMATION SYSTEM D E V E L O P M E N T PROSECUTABLE IN UNIVERSITIES In future for the students who are to be engaged in development of information systems, problem-finding ability, problem-analyzing capability, and problem-dissolving expertise together with performance to execute decision making are believed to be required.
937 A method is available in analysis work of an information system for educating a process for formation of agreement where human and human correspond to each other by means of computer simulation. The method is however almost devoid of effect in a beginners' course, because correspondence of human vs. machine and correspondence of human vs. human have so many portions intrinsically different from each other. In this stage, an action research has been intended to be planned in non-profitmaking system in order to acquire the ability for problem finding and requirement analyzing in an information system. The plan aims at allowing students specializing information system technology to cultivate the fundamentals for becoming system engineers. The purpose of this education resides with the fact that students are required to obtain knowledge necessary for planning/developing of information systems by comprehending the information systems to be developed and the roles and assignments of the people who are engaged in the systems. Students are guided in a direction of explaining problematic points of the information systems they are going to deal with, concurrently by learning required knowledge in the individual processes of the system development. The efforts for the explanation of the problems are made through a series of the processes to allow a prototype to be built up by securing targets of discovery of the problems existing in the real world and by doing requirement analysis. As the problem region to be experienced, the environment where practical utilization of information systems is behind schedule will probably be a desirous choice. This is because the students are obliged to undergo a training to dissolve the problems they themselves have discovered without just mimicking other people's actions. In this exercise, processes of the students' actual experiments are videoed as far as possible and are practically utilized as the materials to investigate the students' thinking processes. With this, it becomes possible to check to see what the students accepted are, whether judgment is correctly made, whether there is any ambiguousness incomprehension, whether other interpretation can be made, etc.
4. PRACTICAL LEARNING OF ANALYSIS/DESIGN IN INFORMATION SYSTEMS Here is the description concerning the exercises for problem finding and requirements analyzing conducted by entering the field of self-establishment education for heavydegree mentally-handicapped people. The learning environment referred to above is explained in Figure 1. Field A is a facility for the mentally-handicapped people. In the field the mentally-handicapped people are provided with instruction from instructors for daily-life activities, who are experts of the welfare performances. Field B is a university. Students collect a variety of knowledge concerning information technology through activities including learning in classes. With interference by securing themselves in Field A, the students observe mentally-handicapped people's conducts/actions. The students and the teachers possess the view for the world as analyzers, whereas the mentally-handicapped people and the instructors share the view for the world as users.
938 C is an interface between the users and the analyzers, i.e. human vs. human. With the cooperation among the people possessing the individual types of view for the world, analysis of information systems is made. The students intend to improve the level of the analysis by repeating the analysis of the systems, the acquisition of the knowledge, and the collection of the information as is shown with the arrow (a). With the repetition of this operation, the world view of the individual persons are gradually changed. The systems are directed towards procedure D, when the agreement between a user and an analyzer is obtained. The students execute design attaching importance to a human-computer interface. At that time, the products obtained from prototyping is exposed to users' evaluation. To the extent that the users' satisfaction is attained, the analysis and design are repeated as shown in the arrow mark (b).
iew
ANALYSIS C human-human / ~ interface analyzers' "~ view
action ~ ~ (/ ~dents, .
/
teach~
DESIGN human-computer interface D
(b)
(a) Figure 1. The Learning Environment
The practice referred to above was made in accordance with the action research. To the collection and analysis of the information a grounded theory[3] is applied, whereas SSADM[4] is applied to the analysis/design of the system. The cooperators to these practices are comprised of about 60 mentally-handicapped people in their 20s or 30s, almost all of whom make their living together in a housing complex established subordinately to the facility. We, the authors of this paper, have made some preparations in obtaining such cooperation. The matter to which the most earnest attention should be paid among them was never to give stress to the mentallyhandicapped people. Especially with the influence produced by introducing a computer to the facility, investigation was deliberately conducted for about one year. For example, problems including such a matter whether the mentally-handicapped people can be familiar with equipment and apparatus are examined, by inviting to laboratories
939 in the university those handicapped who shows interest in machines or by allowing them to touch the equipment and apparatus. On the other hand, the students themselves are willing enough to visit from time to time the facilities to deepen their comprehension for the mentally-handicapped people. For a social aid system, importance was placed upon the concept in purport: (1) The system should not be the one for coercion. (2) The system should be the one acceptable for the people to receive the aid without any resistive feeling. Also to avoid the coercion caused by the developers' erroneous understanding, actions were started with observation of the students who are for a settled duration of a period with the mentally-handicapped people and their tutors to experience the daily life of such people in special training and watch actual state of their life. It remains to be seen whether there exists a system useful as a measure for the aid of life. If there exists such a one, what is it like? Supposing that the system is developed, how is the order of its priority? With regard to such doubtfulness, the students conceive plural system plans with several problem conceptions kept in mind in parallel with observation and detection. An idea selected among the plans mentioned above enjoys a considerable degree of evaluation by the mentally-handicapped people by performing analysis/design and prototyping. The evaluation is done straightforwardly and quantitatively in an easily understandable manner. For example, checking can be made with the items shown below. Are the mentally-handicapped people interested in the matter? Are the mentally-handicapped willing to touch it? Are they desirous to have a try? Are they apt to be fired in the course? Are they utterly reluctant to see it? Are they eager to use it repeatedly many times? As a result of acquisition of the evaluation, the students improve the matter that have found to be problematic points and repeat the operation until they become good enough to be accepted by the mentally-handicapped. The compuer-aided systems developed by the students until today through this practice are as shown below. (1) Computer-aided system for purchase training This is the system to allow the mentally-handicapped to train purchase in a manner of especially discerning types of money, calculation of prices, selecting changes, etc. The most difficult term for the mentally-handicapped is such comprehension concerning money. (2) Computer-Aided system for homecoming training Those mentally-handicapped who live in the residences can avail themselves of the system of returning to their native places several times every year. The system is for the learning aid enabling the disabled to return home by themselves individually. (3) Computer-Aided system for promoting mutual understanding This is the aid system to enable those people who suffer from such double handicap as of the trouble stemming from deaf ears in addition to the intellectual disability to carry on conversations using a computer.
940 (4) Computer-Aided system for fabric making Fabric making is the work to fabricate picture patterns onto mats and the other materials similar to them by referring to the fundamental designs. Mentally-handicapped people always have hard time in recognizing something objective and then getting to the next conduct. Accordingly trainings to enable the mentally-handecapped to do fundamental actions become necessary. The system in question is for the purpose of helping the learning.
5. CONCLUSIONS The systems were made in the laboratories and the prototypes completed concurrently with securing the systems were brought into the facility to be subjected to evaluation. The mentaly-handicapped people were looking forward to undergoing system tests. This encourage the students to willingly get to their work, and as a result they are directed to solution of the problems. Autonomous dispersion environment progresses, and reducing the information system developed under complicated facility/resource environment into a specific type becomes furthermore difficult. The users participating in project conferences are diversified, and level difference among knowledge and technology are more and more expanded. Although agreement formation of users and developers is very difficult, it is effective to visualize indistinct user requirement by using end-user thought or multi-media information. We, the authors of this paper, have been successful in heightening work efficiency of visualization by re-utilizing the constructed prototype.
REFERENCES
1. G.Mansell, Action Research in information systems development, Jurnal of Information Systems, No.1 (1991) 29. 2. Y.Kaminuma, Training for the development of piblic information systems, User Oriented Information Systems Simposium, IPSJ (1993) 101. 3. W.C.Chenitz and J.M.Swanson,From Practice to Grounded Theory,Addison-Wesley,1986. 4. G.Cutts, Structured Systems Analysis and Design Methodology, BlackweU Scientific Publication, 1991
VI. Social Aspects, Management and Work
This Page Intentionally Left Blank
VI.1 InformationTechnology
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
945
Personal Information Appliances Peter J. Thomas, John F. Meech and Robert D. Macredie Centre for Personal Information Management, Faculty of Computer Studies and Mathematics, University of the West of England, Bristol, Coldharbour Lane, Bristol, BS16 1QY, UK 1. INTRODUCTION The range of information management activities which personal computing devices are intended to support include (a) Storage of local information created or manipulated by users (b) Retrieval of local and non-local information (c) Integration of multiple sources of local and non-local information (d) Decision-making by integrating stored, retrieved and integrated information (e) Communication with other users and devices The activities which comprise what may be termed 'personal information management' [1, 2, 3, 4] require not only the use of various technologies, media and modalities but their integration. In this paper we look at the characteristics of personal information appliances, the range of activities which they support, and review families of appliances and their application domains with examples from a current project. 2. PERSONAL INFORMATION APPLIANCES Before looking at specific research issues and topics, we will describe the motivation for research and development in personal information appliances by looking at three areas: key development issues, user experience, and enabling
technology developments.
2.1 Key Development Issues The development of personal information appliances is centred on the following key issues: (a) Convergence. Technologies such as multimedia, broadband networks, personal computing and telephony are converging into a situation where networks provide more than transport of information, and applications and data are embedded in high-speed information networks.
946 (b) Integration. These technolo~es require integration in way which creates a seamless 'information space' of families of information appliances. This suggests t h a t radically new technologies may not be required to support personal information management. (c) Management. Traditionally, the concept of 'information management' has been emphasised in corporate settings. 'Information Management' can also be extended to all users' activities, and suggests a focus on issues such as information properties (depth and breadth, for example), and the possibilities for users to sort, categorise, personalise and share information across activities and tasks.
2.2 User Experience In terms of users' experience of integrated personal information appliances the following characteristics are central: (a) Transparency. Access to information is transparent, and appliances are interchangeable. (b) Appropriate information provision. The focus is not only on the bandwidth of information or simply multimedia delivery, but on the provision of information which is appropriate to the task at hand. (c) Tailorable enduser interfaces. Interfaces to personal information appliances are personalisable and can be driven by the contextual features of use and the content of the information itself. (d) Multimodal interfaces. User interfaces use several forms of presentation not simply multimedia in the commonly-accepted sense.
2.3 Enabling Technology Developments Finally we can specify enabling technology developments which support the development of personal information appliances: (a) Communications and Network Technologies. Networks, infrastructures and equipment; the integration of networks and services; mobile broadband services; personal communications networks. (b) H u m a n Interfaces. The design of information displays; multimodal interfaces; the use of 'digital documents'; video/voice integration; small and innovative display devices. (c) Information Engineering. Support for mobile professionals and knowledge/information workers; integrated office information systems; and techniques for managing 'interpersonal information management' [6]. 3. A P P L I C A T I O N DOMAINS AND APPLIANCE FAMILIES We suggest t h a t for personal information appliances there is a string relationship not between application domain and particular appliance but
947 between domains and families of information appliances. Families of integrated personal information appliances include familiar technologies and appliances but also radically new ones. Figure (1) provides a mapping of the space of families of personal information appliances into application domains, and the overlapping relationships between application domains and families of appliances. HQ_m___e___S_yst¢_ms
...........................................
Site~building Systems
[ .........
Office Systems
Mobile Systems
Figure 1: Personal Information Appliances (adapted from [4])
We are currently using this mapping to drive the design and development of families of appliances and specific members of those families. The following sections look at application domain (site/building systems), appliance family (telephony appliances) and user interface technology (agent-based appliances).
3.1 Application Domain: Site And Building Systems The development of integrated personal information appliances suggests an integration of 'bits and buildings' and the integration of information appliances into sites and buildings themselves. Here information management facilities can be embedded into buildings via distributed appliances which interface with other appliances.
3.2 Appliance Family: Telephony Appliances Telephony systems are still relatively simple and reactive information appliances. The use of voicemai! - ubiquitous within many organisations - call management or fax-switching, are familiar services which can be enhanced by a perspective which emphasises integrated personal information appliances. The requirement for transparency suggests that telephone handsets can be access points to information management functions.
948
3.3 Interface Technology: Agent-Based Interfaces The concept of agency in terms of managing information may best be thought of as an enabling functionality. In these terms an agent may be thought of providing a special facility such as scheduling meetings to a particular user's preferences, or performing other functions in much the same way as a h u m a n might. These agent roles may be viewed as functioning at the information level: they manage information within the context of h u m a n understanding. Such agents are controllable, minimal, 'information management support agents'. 4. P E R S O N A L O F F I C E S U P P O R T SYSTEMS To illustrate work in personal information appliances, we describe an ongoing project on the development of Personal Office Support (POS) Systems which unite the observations we have made so far with the complexities of supporting information management in a work setting.
4.1 S u p p o r t i n g Office-Based Collaboration Communication, the management of information and the scheduling of time are central to the working relationship between individuals. In particular, the relationship between office-based collaborating professionals [6], such as managers and their personal assistants, is based on the smooth management of communication, information exchange and time-management. Although there are numerous examples of 'productivity' software such systems are essentially ad hoc, based on little detailed investigation of the complexity of the tasks to be supported. We suggest t h a t a great deal of leverage to be obtained by understanding thoroughly the nature of the relationships between office-based collaborating professionals, the nature of the information which is necessary to those relationships, the key information management tasks to be supported and the ways in which those relationships and tasks are changed by the nature of different user interface modes. In these working relationships information can come from diverse sources: face-to-face interaction with co-workers, from interactive devices such as telephones or from non-interactive media such as fax, email, letter and memo. POS systems [7, 8] recognise that adequate support for the work of collaborating office professionals requires an understanding of both the 'personal' aspects of information management and the 'collaborative' aspects. The key feature of this approach, in line with our approach to the development of personal information appliances, is the integration of the diverse media necessary to support an individual's work. Our approach to providing support for these personal information management activities is to develop information appliances which provide (a) high levels of connectivity (b) a narrow range of core information management functions (c) support for specific users through information content and provision without personal ownership of the appliance and which (d) serve as ways of locating individuals' activities (as opposed to individuals) within the larger social and organisational settings in which they work.
4.2 R e s e a r c h Studies We have used various approaches to investigate the complexities of the relationship between office-based collaborating professionals such as managers
949 and personal assistants, and in the of design prototype POS systems. Initially small-scale multi-modal studies of working offices were used to arrive at taxonomies of salient events using analysis of video-recorded materials. The studies included videotaping of the interactions between office-based collaborating professionals, the use of technologies (both paper-based and electronic), and the ways in which information was managed in the office setting. Results from initial observational studies of office-based professionals suggested that the salient issues which personal office support systems would need to address were those of (a) communication of short, content-intensive messages (b) support for shared understanding of a medium of information exchange such as a shared view of a diary and (c) support for storage and retrieval of longer, less content-intensive messages in the form of electronic documents. These issues taken up to design a series of multimodal prototypes which incorporated penbased interaction, voice messaging, computer-telephony integration and shared visual workspaces. Further studies of these prototypes suggested that the complexities of the work of office-based collaborative professionals such as managers and assistants could enhanced by the provision of low-functionality robust communications technologies designed around a central 'working medium' such as a shared diary representation. 4.3 P r o t o t y p e
Development
These studies were carried out in parallel with the incremental prototyping of a POS system using paper and software prototypes and a set of scenarios which were intended to reflect accurately possible use of the prototype. Finally, a commercially-available collaborative work system [9] was re-developed to provide a prototype POS system. This system was evaluated using both traditional usability studies and observational and scenario-based studies. The POS system consists of a small gesture, voice and pen-enabled LCD colour display and associated technology which manages telephone, fax, and email and allows a user to indicate their availability status in a time-management system. Pairs of appliances allow users to indicate their availability, interruptability and unavailability to interruption and therefore represent an enhancement of traditional network-based diaries or schedulers. The results of this project suggest that the problems of designing POS systems are more complex from those of simple 'personal systems' but share many of the same concerns.. In particular, collaborating office professionals see such systems as 'highconsequence'- crucial for the support of working relationships where technology is a mediating factor. The further value of the project, which is being taken up in current projects, is to provide a base to explore a number of research themes such as (a) the human interface design issues for visual displays of time and the use of colour, resolution, icons and symbols (b) multimodal interface issues - integration of pen, voice, speech and gesture (c) integration of information appliances - the ways in which networks of ubiquitous computing devices can be managed (d) technology issues in developing a software architecture which can integrate and manage telephony, fax, email and voicemail, for example, and (e) the social and organisational issues in supporting office work.
950 REFERENCES
1, Thomas, P. J. (1995) (ed.). Mobile Communication and Collaborative Technology. (London: Alfred Waller/Unicorn Seminars series). 2. Thomas, P. J. and Meech J. F. (1994). Personal Information Management: Applying HCI Techniques to Develop Usable Personal Technology. Proceedings of HCI'94, Glasgow, August 1994. 3. Thomas, P. J. and Meech, J. F. (1994). Personal Information Management: developing usable personal systems. In the Proceedings of OZCHI'94, Australian annual conference on Computer-Human Interaction, Melbourne, November 1995. 4. Thomas, P. J., Meech, J. F. and Macredie, R. D. (1995). Personal Information management using integrated information appliances. To appear in Vince, J., and Jones, H. (eds.) Digital Media and Electronic publishing, book edition of BCS Graphics and Displays Group Conference (to appear, 1995). 5. Thomas, P. J., Meech, J. F. and Macredie, R. D. (1995). Integrated Information Appliances and Added-Value Information Services. In the proceedings of HFT'95 15th International Symposium on Human Factors in Telecommunications, Melbourne Australia, March 6-10, 1995. 6. Frohlich, D. (1995). Interpersonal information management. In Thomas, P. J. (ed.). Mobile Communication and Collaborative Technology. (London: Alfred Waller/Unicorn Seminars). 7. Fleuriot, C., Lees, D. Y. Macredie, R. D., Thomas, P. J. and Meech, J. F. (1995) Interface Engineering in an Office Information Appliance. To appear in the Proceedings of CHI'95 Human Factors in Computing Conference, Denver, May 1995. 8. Thomas, P. J., Meech, J. F. and Macredie, R. D. (1995). Managing OfficeBased Services using an Integrated Information Appliance. In the proceedings of H F T ' 9 5 15th I n t e r n a t i o n a l S y m p o s i u m on H u m a n Factors in Telecommunications, Melbourne Australia, March 6-10, 1995.* 9. O'Conaill, B., Geelhoed, E. and Toft, P. (1994) Deskslate: a shared workspace for telephone partners. Hewlett-Packard Technical Document, HP Laboratories, Bristol, UK.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
951
Efficient Development of Organisations and Information Technology- A Design Approach Jan Gulliksen a'b ([email protected]), Mats Lind a ([email protected]), Magnus Lif a'b ([email protected]) and Bengt Sandblad a'b ([email protected]) a Uppsala University, Center for Human-Computer Studies, L~igerhyddv. 18, S-752 37 Uppsala, Sweden. b Uppsala University, Dept. of Technology, Systems and Control Group, POBox 27, S-75103 Uppsala, Sweden.
In this paper a framework for the entire process of organisation and information system development is discussed, focusing especially on design issues. Our definition of the process of design in human-computer interaction is the process of creating a formal description by appearance and functionality of an information system. This design is based on both formal and informal descriptions of interaction requirements as a result of a work analysis process. The analysis phase is separate from the design phase. According to the definition of design, as a specification into a formal language, it can never completely describe all requirements. We distinguish four, different, consecutive gaps of communication in the design process. In each of these gaps some information about the actual work situation can be lost. These gaps constitute severe obstacles in the process of developing efficient and usable information systems for specific work situations. Development models covering the entire process of design might bridge, or at least narrow, these gaps. Some main characteristics of such models are presented. 1. B A C K G R O U N D Developing information systems for complex tasks in working life is not trivial. Skills in software development and cognitive psychology in combination with an insight into the enduser's work situation are required. Most methods for development of information technology support are a mixed process of organisational development, task analysis, data modelling, application development and interface design. They emphasise certain aspects of the development process but lack other. Disadvantages with such methods: • • • • •
Difficult to plan and control development - increased development times and costs Unfriendly, fragmentary, inconsistent, unchangeable user interfaces are often the result Inferior user participation Problems with personal competence and knowledge, due to complexity Increased work load and personal conflicts due to unclear roles of responsibility and insufficient competence
Several competencies are involved in the organisational and information technology development process. Human-factors experts analyse the user and his task, software engineers perform system development etc. An analysis of expectations identify goals for the work situation of tomorrow, organisational developers analyse implications of general future work methodological changes, trying to predict the effects of modem computer support, reengineering measures and knowledge development. Task analysts establish what information is used and derive together with designers and programmers a datamodel. The amount of extra work involved can be big due to unclear roles and competencies. A methodology for user interface design and development could significantly reduce the amount of work required. A designer is often introduced as a communication link between work analysts and software developers. He is often a software engineer with special interest in formatting and layout but often without formal training in graphics design. This designer often performs design as an
952 artistic leap, sometimes referred to as innovative design. Assembled information and knowledge of the work, result in a prototype with more or less functionality. Problems in the design process can refer to problems in communicating and translating one semiformal notation (e. g. a report from the task analysis) into another formal notation (e.g. a MacroMind prototype, or a mock up design on paper). The prototype can seldom be re-used in software development and these prototypes seldom illuminate all relevant aspects encountered in the task analysis, since prototyping tools are not developed to communicate design decisions. 2. ORGANISATIONAL ASPECTS AND MODELS OF WORK There is a need for a new paradigm of system development, focusing on development of organisations and the work activities as such. This means that methods for analysis, requirement specifications, system design and construction, must include all relevant aspects simultaneously, and not focus only on technical support systems. It is obvious that couplings between e.g. work organisation and information handling in organisations are very strong. A certain work organisation defines certain information support requirements. An information system is normally developed with a specific work organisation in mind, reflected in e.g. datamodel and functional specifications. Changing the work organisation without changing the information system will often create severe informational problems for workers in the new organisation. In administrative and service work situations, the main focus of our studies, it is important to base information technology development on models of work, considering all aspects; competence for performing a specific work task, organisational impacts due to automatisation of tasks, information needs and possibilities of sharing large information sets, work flow and supervision, evaluation and efficiency. We are currently developing work models for the case handling domain [ 1]. Development based on such models can contribute to the improvement of work efficiency, user satisfaction, competence, work environment etc. When designing computer support it is necessary to question the present work organisation, to fully utilise the potential of technological support. We need to develop information systems for tomorrow's organisations, not for today's. Often, in a development project, a task analysis is performed, focusing only on information aspects of the work. Information related problems are identified and a suitable computer application solution is specified, resulting in false conclusions and inappropriate requirement specifications. Information problems, identified by the task analysis and experienced by professionals in a work setting, can be symptoms of more fundamental work environment problems. E. g. organisational problems, lack of personnel or other resources, lacking competence or inefficient support systems, will result in information handling problems. Developing computer systems can eliminate symptoms but will not solve, rather conserve, the original problems.
Figure 1. The OTM-model describes the process of simultaneously considering organisational aspects (O), human, social and comPetence aspects (M) and technological aspects (T). A new approach to task analysis, considering all aspects of a work situation, will result in more correct requirement specifications. Close relations between work organisation, information handling and competence must be considered simultaneously [c.f. fig. 1]. An important part of task analysis is the expectation analysis, where different staff categories can specify expectations
953 regarding work organisation, work procedures and support systems. Such expectations are an alternate, and more efficient, way of specifying goals. In a 'learning organisation', organisational development is a continuous process integrated in the work. Goals are formulated, compared to the actual work situation, development plans specified, effectuated and evaluated, in a continuous feedback process [c.f. fig 2]. This 'learning organisation' can support development of organisations, information technology and competencies. Goals and
[ ,
-
Generalgoals e.g.: efficiency
I "usability ,
~
.
-
! i
,
good work en~ronment n
0 J" activities
actual state t~i1e
'now'
Figure 2. Continuous development of organisation, competence and information systems in a 'learning' organisation. The development activities can be seen in more detail in figure 3. Describing and analysing work requires detailed domain knowledge, which is why an efficient user participation is essential. Organisation, work contents, activities, communication and information aspects must be described. Problems and expectations of different staff categories are identified and possible solutions analysed. Requirement specifications for development should be formulated in activity terms, covering information and organisation aspects simultaneously. Requirement specifications in this way describe how potential end-users want to organise and perform their work activities and environment. Expectation analysis can reveal conflicting goals, which can be handled only if explicitly documented. An experimental model for system development assumes professionals participating actively. A preliminary requirement specification in activity terms is interpreted by computer experts into a requirement specification in 'computer' terms. A rapid prototype is developed and tested. Again computer experts interpret and evaluate data etc. and the prototype gradually transforms into the final system. Parallel requirement specifications allow users to participate and even obtain a leading role in development. Prerequisites for this development model to be feasible are e.g.: an organisational decision to work according to an experimental model and assign reasonable resources, competence for this kind of work both with domain and computer experts and finally, efficient rapid prototyping tools. 3. O R G A N I S A T I O N AND I N F O R M A T I O N T E C H N O L O G Y D E V E L O P M E N T Above mentioned disadvantages might be overcome by defining a structure for the entire process of organisational and information technology development, consisting of separate but sequentially related parts [c. f. fig. 3]. Organisational d e v e l o p m e n t includes e.g. specification of goals, relations, roles, competencies and work processes. The organisational model is the description of this, the basis for specification of user requirements, both concerning functionality and human-computer interaction. Information analysis establishes, using this organisational model, what data, are needed, by whom, when and where, to perform the work. The resulting application model is both a datamodel and a collection of rules or methods, depending on the nature of the database. Analysis of information utilisation focuses upon certain aspects of how information entities are used in specific work situations, and, especially, factors affecting cognitive load [2]. It assumes existing application and organisational models and the result is
954 called a work model. User interface d e s i g n is the interpretation of the work model into an interface model. Construction is the production of applications (database and presentations user interface and functionality) based on the application and interface models. A ( ~ ~ . , Applicati°n3 development
I
4
~rgani-f sa~lonm ~ s a t i o n a l ~ develop-[ w-]-model I v
mation l---~catio n ] analysis I J m o d e ~ / ,..~l i
q
:...................
utilisation analysis
, struction
['-I Activities- work processes (analysis, design, construction) /7
Descriptions- resulting formalised models
Figure 3. Iterative experimental system development; including consecutive activities and the descriptions they produce. Feedback and possibilities for participatory design are also illustrated. 4. M E T H O D S
FOR DESIGN - BRIDGING THE DESIGN GAPS
We need to enhance the communicating and interpreting aids for transfer of knowledge between the different areas which have been subject to analysis. This communication is to some extent a question of design. The concept of design in human computer interaction can be defined as the creation of a formal description (e. g. program code, formal language) of appearance and functionality based on partly informal (to a large extent) results of an analysis. Four different types of design, corresponding to the formalisation of the description models are defined. The design of the organisation model is a question of being innovative and imaginative in predicting future changes of work organisation and impacts from information technology and relating this to the overall goals of the organisation. The design of the application model is a characterisation of the objects and methods of work. The design of the work model is a process of formalising the outcome of the analysis of information utilisation. And finally the design of the interface model is the formalisation of results of the decisions made during the user interface design process, as e. g. a prototype. As information about the actual work is lost when designing formalised descriptions, communication gaps in design occur. Today's methods are 'leaps' over these gaps, referred to as 'innovative design'. Methodologies for design can bridge or narrow these gaps, improve knowledge communication, make development more efficient and economic, prevent unnecessary work and produce usable interfaces. • Gap 1 - refers to problems when formalising the design of the user interface model, the appearance, functionality and behaviour of the intended human computer interface, in an interpretable way for software development. Prototyping with existing design tools never illuminate all relevant aspects of work, it shows a minimum of functionality and dynamics and it seldom produces formal specifications that are reusable in construction. Our group is currently developing an interface emulating tool that can interpret a formalised design description file, together with an existing database, into a functioning application. The description file describes important aspects of layout and relations between interface elements at different levels. Prototyping is performed in real terminology with a dedicated tool. • Gap 2 - refers to problems when formalising results from the analysis of information utilisation. In the final specification of formalised, viable methods for analysis of information utilisation, a language for the specification of the work model is defined. Through the domain specific design methodology [3] the interpretation of the work model is enhanced. It is
955 recommended that the same person performs the analysis of information utilisation and the user interface design due to the loss of information in every transfer of complex knowledge. • Gap 3 - refers to the problems when formalising the result of the information analysis, the application model, as a formal design of the datamodel (objects and methods) along with behaviour and functionality. Object oriented methodologies for application modelling are preferred, where relations to interface elements and concepts of end-user's work situations are straight forward [4]. The domain specific design methodology emphasise the importance of having datamodels that mirror the actual work situation, which also improves possibilities for efficient user participation in the iterative development process. • Gap 4 - refers to the problems of specifying the organisational model, through a formal description that can be adapted, understood and efficiently introduced in the entire application development process. The concept of 'learning organisations' is central. 5. ANALYSIS OF I N F O R M A T I O N U T I L I S A T I O N Analysis of information utilisation focus upon how information entities encountered in the information analysis are being used in a specific work situation. An existing application model (an implemented, functioning, object-oriented datamodel containing data and methods to specify the behaviour) and an organisational model, can be assumed. The datamodel puts limits on the design space. If the analysis of information utilisation demands changes, the design of the user interface can be enhanced, if the datamodel is specified in a modifiable way. We are working on methods for analysis of information utilisation and a taxonomy for specification of the work model, the outcome of the analysis of information utilisation. Central to the creation of efficient user-interfaces is to minimise cognitive load, imposed upon the user when performing work related tasks. To do this, we have argued elsewhere [2], the user's tasks must be analysed in terms of which decisions they make. Examples of the kinds of decisions we refer to are: rejecting or accepting an application, sending or not sending a form to someone else for consultation etc. From when the task is started and until a decision is reached, the cognitive resources of the user are heavily strained and the manipulation of the interface should be kept at a minimum. In order to make such a design, the work model must include: (a) a list of tasks performed, as defined by decisions that are made, for each type of worker, (b) a list of data, in terms of the datamodel, that could possibly be used in performing each task, (c) a list of actions needed to manifest each decision, (d) a list of naturally occurring "work-situations", defining sets of tasks which usually, or possibly, are performed concurrently. This work model can then be used by the interface designer in the creation of suitable interface elements e. g. "screen documents" or forms, or larger entities such as "workspaces" or "rooms"[5] and the dialogue needed to use these to perform the tasks. 6. D E S I G N DECISIONS AND P R O T O T Y P I N G Documentation of design decisions, and the reasons behind them, are seldom performed, but when redesigning the interface, during experimental development, the reasons for design decisions must be known, clearly specified and easily understood if they are to be evaluated. The Design Rationale approach [6, 7] is a framework for documenting design decisions, a semi-formal notation of different design options and explicit representations or reasons for choosing among those options. The main concepts are QOCs, Questions - highlight key issues in the design, Options - possible answers to the questions and Criteria-reasons for or against possible options. QOCs are mainly identified by recording design sessions. Inventors of the design rationale approach argue that the methodology provides a theoretical framework for design. Our opinion is that design rationales solve some problems on documentation of design decisions, describing decisions that have been made, but give no guidance for which decision to make in a certain context. The rationale is therefore not a methodology for interface design.
956 The usability engineering approach [8, 9] to interface design focus on the evaluation of an interface. The design must fulfil utility and usability criteria. The focus is on the definition of these criteria, and on methods for evaluation and testing. Methods of this nature are also important in a more complete design methodology, but lack support for the design decision process. Basing design work on a domain specific style guide [3] can minimise the distance from the style guide to interface design for a specific application. Normally, a style guide is of a general nature with very limited design support for applications in a specific work domain. A style guide on a higher level, including domain knowledge, is much more detailed and can efficiently support the design process. Important parts are composite interface elements corresponding to more complex information structures of the domain. However, a more structural approach to the design process is needed, even here. We are currently formulating a methodology, in which the representation of design decisions is immediate and with a minimal loss of information. 7. D I S C U S S I O N A framework for organisational and information technology development is stressed in this paper by defining design in HCI as the creation of a formal description of appearance and functionality, based on partly informal results of analyses. Due to the design gaps occurring from the communication of design decisions, information about the actual work situations is lost. These gaps can be bridged or at least narrowed through the definition of methodologies for design. Extensive research work has been performed on definitions of methods for analysis of information utilisation, domain specific design and interface modelling tools. Future research aims at increasing the granularity of methods for the entire process of information system development. By a further analysis of the design process, the possibilities for including knowledge on users in general, on work, design and software development, are increased. Domain specific design increases the possibilities for efficient user participation. Well defined roles and channels for easy cooperation eliminates unnecessary work and facilitates the preserving of knowledge gained from different analyses. Decreased development times and lower costs as well as improved human-computer interaction is the result. REFERENCES
1.
Gulliksen, J. & Sandbiad, B. (in press) Domain specific design of user interfaces - Case handling and data entry problems. In David Benyon & Phillipe Palanque (eds.) Critical
issues in User Interface Systems Engineering, Springer Verlag London Limited. 2.
Nygren, E., Johnson, M., Lind, M. and Sandblad, B. (1992) The art of the obvious.
Proceedings of CHI'92, Monterey, California, May 1992. 3.
Gulliksen, J. and Sandblad, B. (in press) Domain specific design of user interfaces. Int.
Journal of Human-Computer Interaction, Ablex Publ. Corp., Norwood, New Jersey. 4.
Rumbaugh, J., Blaha, M., Premerlani, W., Eddy, F. & Lorensen, W. oriented modeling and design. Englewood Cliffs: Prentice-HalL
(1991) Object-
5.
Card, S. K. & Henderson, A. (1987). A multiple Virtual-Workspace Interface to Support User Task Switching. Proceedings of CHI+G11987, Toronto, Canada.
6.
MacLean, A., Young, R.M., Bellotti, V.M.E. and Moran T.P. (1991) Questions, options and criteria: Elements of design space analysis. Human-Computer Interaction, 6, 201-250.
7.
McKerlie, D. and MacLean, A. (1993) QOC in Action: Using Design Rationale to Support Design. INTERCHI'93 video program, Amsterdam: A CM.
8.
Nielsen, J. (1993). Usability Engineering. Academic Press, Inc. San Diego.
9.
Nielsen, J. & Mack, R. L. (1994). Usability Inspection Methods. John Wiley & Sons Inc.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawaand H. Mori (Editors) © 1995Elsevier ScienceB.V. All rights reserved.
957
I n t e g r a t i o n of People, T e c h n o l o g y a n d O r g a n i z a t i o n : The European Approach Christina Kirsch, Peter Troxler, Eberhard Ulich Work & Organizational Psychology Unit (IfAP) Swiss Federal Institute of Technology ETH, Ztirich, Switzerland
This paper presents the general outline of new method, HITOP-D, considering the integration and joint optimization of people, technology, and organization. This method is based on the existing american methods HITOP and ACTION. It takes in account the specific european industrial context. In an iterating process the preliminary design of a project is assessed according a list of criteria of the four aspects people, technology, organization, and task design. Incongruencies are solved through a fit analysis and redesigning the original project. The performance of HITOP-D will be empirically evaluated.
1. I n t e g r a t i o n of People, Technology, and Organization Designers of processes an structures in industrial contexts are confronted with the growing need of adressing multiple aspects of a design project as there are task design, organizational design, people-related questions, and technical design. The work psychological point of view is that only the joint optimization of technology, qualification and organizational structures can lead to successful implementation of processes and structures (Ulich, 1994). In the government funded CIM Action Program in Switzerland the holistic conception of computer integrated manufacturing, encompassing the three aspects - people, technology, organization- is referred to as the MTO-Model ("Man - Technology- Organization". Ulich, 1993). The focal point of the MTO approach is the functional division between man and machine. Work design ie. the task allocation in man-machine and man-man systems is the basic question in analysis and design of production systems. One instrument that furthers the proposed joint optimization of industrial projects is HITOP (Majchrzak et al., 1991). It primarily addresses the design of sociotechnical systems implementing new technologies. The use of the HITOP method for the analysis and design process reduces the probability that important aspects are neglected. The fit or congruency of the three aspects people, technology and organization is evaluated according to various sociotechnical criteria. Misfit or incongruency is resolved in mutual discussion by the HITOP Team.
2. F r o m HITOpTM-91 to HITOP-D In the original HITOP manual (Majchrzak et al. 1991), which has been developped in the late 1980s, the implementation of advanced manufacturing technology was the predetermined starting-point for performing the analysis.
958 Many of the problems implementing new technologies emerge indeed considering organizational or human resources aspects (Strohm et al., 1993). F u r t h e r e d by the recession of the early 1990s productivity enhancement programs in organizations started to change from technology ,ie., automation oriented strategies with high investment risks to strategies dealing with organization or personnel features, eg., implementing team work in production islands, or establishing a Total Quality Management (TQM) system. Especially small and m e d i u m sized e n t e r p r i s e s are moreoften confronted w i t h organizational changes or changes in human ressources. These are the reasons why in European industry there is a strong need to go beyond the limitations of the originial HITOP manual. The revised HITOP-D version consequently treats each aspect with the same emphasis. HITOP-D can be used to realize the joint optimization of personnel, organizational or technology projects. With this approach, HITOP-D is more open than ACTION which is an analysis program based on HITOP that was developped by Ann Majrchzak (Majchrzak & Finley 1994). I m p l e m e n t i n g new technology like modern computer systems, artificial intelligence, and multimedia systems are highly sensitive to organizational and personnel issues. The application of HITOP-D in these cases is still a key success factor t h a t g u a r a n t e e s positive effects considering integration of people, technology, and organization. According to the basic concept of the socio-technical system approach the central focus is neither on personnel, on technology nor on organization, but on the fit or congruency of the various factors that constitute the production system. HITOP-D offers the option to start at any chosen entry point - personnel, technology or organization.
2.1. P r e l i m i n a r y Steps for a HITOP-D Analysis The first steps of a HITOP-D analysis consist defining the goals and restrictions of the innovation project analyzes and describes the actual situation before changes are implemented. This analysis is based on the MTO-Analysis developped in the project GRIPS (Ulich & Strohm, 1995). As in HITOP-91 it is proposed to work with HITOP-D in a team. To provide the necessary integration of the different views on an innovation project the future users, the various departments, and the different interest groups whithin the organization should be represented in the HITOP team. The team is lead by one or two HITOP facilitators. 2.2. E n t e r i n g the HITOP-D Analysis HITOP-D offers for each aspect to be considered - personnel, technology, organization, task design - a set of criteria to describe the relevant characteristics of the project. Each criterion is presented with a brief definition and guidelines for assessment. The assessment expresses how problematic each criterion has to be considered rating it from 1 (higly problematic) to 5 (no problems expected). A set of questions allows the users to collect the relevant information. Definition and assessment guidelines are clarified by additional practical examples. Additionally the HITOP users have to determine for each criterion whether it describes an unchangeable or a changeable characteristic of the existing situation in order to develop the conclusions for the design of the sociotechnical system. The next step of the HITOP-D analysis is to determine the desired entry point and reference point of the respective project and to describe and assess the
959 Critical Features (CF) of the project in terms of sociotechnical criteria for each aspect, i.e., task, personnel, organisation, and technology. This description uses the same criteria mentioned before.
Peopleo 1 Technologyo Organizationo "~" A1 Task Designo
People Technology Organization Task Design
People1 Technology 1 Organization1 Task Design l
~
(i)
These first steps acomplished the HITOP analysis has come to a preliminary description of the innovation project as expressed in (1) where the first vector indexed 0 represents the existing situation, the last vector index 1 - the intended future situation, and the delta vector the planned change to the existing situation. 2.3. A s s e s s i n g the Interference B e t w e e n Different Aspects The goal of the HITOP analysis is to achieve the fit of the four aspects - people, technology, organization, and task design- for the future situation. For this purpose, the interference between the four aspects of the suggested preliminary future situation - the fit- has to be analyzed.
///4
people
S
technology
~,
v
organization
Figure 1: Interrelationship matrices between the four aspects of the HITOP-D analysis - people, technology, organization and task design The fit is analyzed evaluating the impact every aspect has on each of the remaining other three aspects. HITOP uses six impact matrices shown in figure 1 to operationalize this step. For each matrix HITOP provides guidelines-i.e. questionnaires, examples-which mutual impacts have to be expected comparing two aspects. The impact analysis ends up identifying critical aspect criteria for the suggested future solution. f fi
Peoplel 1 Technologyl Organizationl ~ 1 Task Design1
(2)
960 Usually the preliminary future design will not satisfy the proposed fit criteria, cf. (2). The next steps of the HITOP analysis will have to deal with the redesign of the intended future situation in order to achieve the fit between people, technology, organization, and task design. 3. R e d e f i n i n g t h e F u t u r e S i t u a t i o n I t e r a t i n g t h e H I T O P - D P r o c e s s
The discussions during the preliminary description of the future situation and especially the first fit analysis usually produce plenty of ideas and questions how the project could be changed to get better results according to the criteria applied. In the following iterative steps these ideas are systematically integrated redefining the intended future situation and reassessing the fit. People0 Technology0 Organization0 Task Design0
+An
People Technology Organization Task Design
f Technologyn Peoplen 1 fi Organizationn = 1 Task Designn
____)
Peoplen 1 Technologyn Organizationn Task Designn
(3)
(4)
After several iterations there will be found a future situation with a high degree of fit or congruency of the four aspects people, technology, organization, and task design. The new delta vector contains the necessary changes to the existing situation. From this basis an action program can be derived. The whole HITOP process will take - depending on the scope of the project three to five days including three or four iterations. 4. E v a l u a t i n g t h e H I T O P - D M e t h o d
The hypothesis is, that the use of the HITOP method in design projects assures that the relevant aspects are taken into account, whereas projects not using HITOP-D tend to neglect relevant PTO (people technology organization) aspects. Numerous instruments, methodologies, and tools are available, claiming to improve performance and efficiency of production design and implementation of new technologies. Most of these instruments just claim to have an effect without giving any empirical evidence for objectively attained improvements. Thatfore an empirical evaluation of the pilot studies is conducted to determine: (a) The impact of HITOP-D on the design process: process evaluation (b) The impact of HITOP-D on the outcomes of the design process: cost-benefit analysis, product evaluation and goal attainment scaling. 4.1. H I T O P - D O u t c o m e s E v a l u a t i o n
With a first i n s t r u m e n t - the HITOP-D pre-evaluation survey- the project structure, input (costs, duration) and goals of the project are analyzed at the beginning of the project. At the end of the project goal attainment, problems,
961 positive effects and general satisfaction with the HITOP-D method are analyzed with a similar instrument - the HITOP-D post-evaluation survey. Comparing the results from the two datasets to datasets of other projects not supported with HITOP-D it can be determined how powerful an instrument HITOP-D is and clear evidence can be given what performance improvements are possible with the use of HITOP-D. 4.2. H I T O P - D P r o c e s s E v a l u a t i o n
Additionally during the design process a diary survey is conducted as sort of a brief longitudinal study. After each project-meeting the participants complete a questionnaire concerning the current task of the project meeting, method, problems and satisfaction within the project. We investigate on how the HITOPD projects proceede, what kind of turning points and problems occur. From the results of these pilot studies we will be able to develop project management and facilitator guidelines making HITOP-D even more user friendly and easy to use. References
Majchrzak, Ann et al. (1991). Reference Manual for Performing the HITOP TM Analysis. Ann Arbor: Industrial Technology Institute. Majchrzak, Ann, Finley, L. (1994). Extending the Concept of Fit to a Theory of Sociotechnical Tradeoffs. Paper presented at the Fourth International Conference on Management of Technology, Miami, FL, March 1994. Strohm et al. (1993). Integrierte Produktion: Arbeitspsychologische Konzepte und e m p r i r i s c h e Befunde. In: G. C y r a n e k & E. Ulich (Eds.). C I M H e r a u s f o r d e r u n g an Mensch, Technik, Organisation. (pp. 192-140). Schriftenreihe Mensch-Technik-Organisation (Ed. E. Ulich), Band 1. Zfirich: Verlag der Fachvereine; Stuttgart: Teubner. Ulich, E. (1993). C I M - eine integrative Gestaltungsaufgabe im Spannungsfeld von Mensch, Technik und Organisation. In: G. Cyranek & E. Ulich (Eds.). CIM - H e r a u s f o r d e r u n g an Mensch, Technik, Organisation. pp. 29-43. Schriftenreihe Mensch-Technik-Organisation (Ed. E. Ulich), Band 1. Zfirich: Verlag der Fachvereine; Stuttgart: Teubner. Ulich, E. (1994). Arbeitspsychologie, 3rd edition. Zfirich: Verlag der Fachvereine; Stuttgart: Sch~iffer-Poeschel. Strohm, O. & Ulich, E. (Eds.) (1995). Ganzheitliche Betriebsanalyse unter Berficksichtigung von Mensch, Technik und Organisation. Vorgehen und Methoden einer Mehr-Ebenen-Analyse. Schriftenreihe Mensch-TechnikOrganisation (Ed. E. Ulich), Band 10. Zfirich: Verlag der Fachvereine; Stuttgart: Teubner.
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
963
Dynamic changes of human systems under a simple task of HCI Mariko Fujikake F u n a d a , , Satoshi SUztlki b, Takao Tanaka b, Yusuke Yazu b, Kyoko Idogawa c, Chieko Hukuda,, Satoki P. Ninomija b
a Department of Management, Hakuoh Univ., 1117, Daigyoji, Oyamashi, Tochigi, 326, Japan bFaculty of Science and Engineering, Aoyama Gakuin Univ., 6-16-1, Chitosedai, Setagayaku, Tokyo, 157, Japan ~Seitoku Univ., Matudo, Chiba, Japan
1. INTRODUCTION Since HCI (Human-Computer Interaction) is a field which treats a kind of communication methods between computer systems and human systems controlled by brains, true characteristics of human systems should be made clear in order to devise a suitable HCI method. Three approaches are considered to get near the truth. The first one is to study about an active brain under HCI task, because a brain is a CPU of human system. The second approach is to study about relations between states of human systems and input-output (I/O) responses of the systems. In this case, states of human systems are activity of brain, cardiac rhythm, etc., inputs mean a indication displayed on a CRT or message of computer voice, etc., and outputs are behavior of human beings to the influence of computer systems. The third approach is to study about only I/O responses of human systems treating a human system as a black box. The order of these approaches are reasonable, because a brain controls and determined all behavior of a human system, and total states of human systems determine I/O responses of the systems. W? have done about analysis of brain under a HCI task and report the results by another representation of this conference(l). In this paper, our situation is at the second approach., and our purpose is to analyze both of states of human systems brain and I/O responses of human systems under a simple HCI task. We selected grouped a waves of EEGs, cardiac rhythms, and numbers of winking to keep watch states of human systems, because grouped a waves appear when human beings fall down or falling down to low awake conditionsa) and cardiac rhythm and numbers of winking are reported that they are deeply concerned with levels of mental stress or awakeness of human beings. And we made a simple HCI task which is to input the same character as that displayed on a CRT display and defined several kinds of variables to show efficiency of I/O
964 responses. From the results of analysis about these variables, we take out the characteristics of human systems about the relations between states of the systems and I/O responses, and then consider the reasonable nature to have suitable future HCI methods.
2.METHODS 2.1.Object data Object data are measured from five normal male students under the following task; (1) task: Push the same key as a numeric character displayed on a CRT display as fast as possible. Displayed numeric character is one of 1"-~9, and the displayed sequence is at random. The displayed time of next character is 0.5 msec after a correct answer is inputted. This task is continued for one hour. We defined the following variables to analyze the relation between brain activity and I/O responses of human system; (2) Defined variables: N: numbers of displayed numeric characters for one mi,utes E: a ratio of numbers of error input to total input numbers for one minute. CR: mean of periods from the time when a character is displayed to the time when a correct answer is inputted during one minutes. R: mean of response time from the time when a character is displayed to the time when the first character is inputted during one minute. Az time length when grouped a waves appear for I minutes. C: cardiac rhythm for one minute. W: numbers of winking for one minute. LW: numbers of winking whose continuous time is more than or equal to 0.5 seconds.
2.2.Analytical methods We analyze the variables defined above by the following methods; (1) representation of variables as time series data. (2) calculations of mutually correlation among variables. (3) calculation of difference of variables and repetition of the same calculation ten times. (4) smoothing by moving average method. (5) drawing graphs in a three dimensional space constructed by the three kinds of variables.
3.RESULTS Fig. 1 is an example of time series representation of measured variables. The horizontal axis is time and vertical line is values of defined variables. The changes of variables include short and
965 0
10
I
20
I
30
I
40
I
50
I
I
min !
0
10
20
:
:
I
I
!
t
30
40
I
I
I
I
50 I
min t
N
E CR
A
A
H
H
L
I
0
10
I
I
I
20
30
40
I
50
4
min
Fig.1 An example oforiginal data. N~LW: variables defined in the section 2.1.
I
I
0 10 20 30 40 50 min Fig.2 An exampleoforiginal data. N ~ L W : variables defined in the section 2.1. 0
10
0
10
I
I
20 I
30 I
40 I
50 I
min 4
,.IT,Tll, TIrTI N
E
CR
R
Tiiltll!
A
H
W
LW
Fig.3 An example of mutually correlation. N~LAW: variables defined in the section2.1. The data are the same as ones used in Fig. 1.
I
I
I
I
I
20
30
40
50
I
min
Fig.4 An example of difference calculated by 10 times repetition. The data are the same as ones used in Fig. 1. comparatively large frequencies. Fig.2 is another example of time series of measured variables. An amount of a waves in Fig2 is fewer than that included in Fig.1. Amounts of grouped a waves are different among individuals. Then, all our data are divided into these two types; one includes a
966
0
10
I
20
I
30
I
40
I
50
I
I
I
I
E
E
CR
CR
20
10
0
mi n
I
40
30
I
I
I
50 I
m
in I
/ fl
t.W
.._~. I
-',_AI~"
J
~
I 10
0
~
I
I
20
I
30
I
50
40
I
I m in
I
0
I
10
I
20
I
30
40
I
50
I
mi n
Fig.5 The smoothed data of Fig.1 with 11 data. Fig.6 The smoothed data of Fig.2 with 11 data.
i. ...... •
!.!
l i,
,I,ID,.,,,,,..,,,"
•
•
!
I
,~
6
?
,
........
"
'[f,IUi
: 'I" ....................... '.... .... i'"
8
I
I
I
r 6566.
9
1~
11 seo '64 ~ 0.0
Fig.7 Changes of grouped a waves"A", cardiac rhythm "H", and response time "R". The height of line represents the length of response time. The data in Fig.5 are used.
'........if:; ........ ,'
0 ""~
" 0.5
I
H
I
:/ ~C
Fig.8 Changes of grouped ~ waves "A", cardiac rhythm "H", and response time "R'. The height of line is the length of response time. The data in Fig.6 are used.
large amount of a waves and the other is a few. Fig.3 is an example of mutually correlation among variables about the data of Fig.1. Both of horizontal and vertical axes are corresponding to the defined variables. The variables "N~, "CR", and "C" have large correlation among them. Fig.4 is an example of difference when a calculation is repeated ten times. The horizontal axis is time, and the vertical axis is values of difference of respective variables. The used data are the
967
33' 30
25
20
5
Fig.9
6
7
8
9
10
11 sec
Changes of grouped a waves "A", winking "W", and response time "R". The height of line is proportional to value of response time "R".
same as ones of Fig. 1. Resemble patters appear in "It", "A', and '~r' variables. The relations among "It", "A', and "W~ are deeper than the other relations among variables. Fig.5 and Fig.6 are examples of smoothing the data of Fig.1 and Fig.2. The tendencies included in these data become clearer. Fig.7 and Fig.8 are graphs in a space constructed variables "A", "IF', and "It". The horizontal axis is variable "A", the depth of the figure corresponds to "IF', and vertical axis is "It". The dotted line is a locus of variables "A" and "H" following time. The loci are not the same and the tendencies are also different. But the changes of loci are not at random, and they seem that there are some rules. Fig.9 is a graph in ("A', "'~r', "R"")-space. 4.CONSIDERATIONS Our purpose of this paper is to make clear the relation between states of human systems and I/O responses of the systems. The defined variables are divided into two kinds of classes. One is to indicate the characteristics of I/O responses, and the another is to show the states of human system. The variables "N', "E", "CR', and "It" are the former ones, and "A', ""H", "WTM, and '%W~ are the latter ones. Since the variables '%T', "CR" and "R" have great mutual correlation, we select "It" to indicate the character of I/O responses and consider the relations among "It" and other variables of states of human systems. One of major characteristics of human system are
that the states are unstable and are
dynamically changing. And another character is large difference among individuals. But the
968 changes do not occur at random, and there are some tendencies or rules observed in Fig.7 ~ Fig.9. Since the variable "It" has greater mutually correlation to those variables ;".4,", "I-F, and "W~ in smoothed data like Fig.5 and Fig.6, we select "A', "I-I', and 'T( ~ to represent states of human systems. Though selected variables are moving in the space defined by those variables, the movements are not at random and have some tendencies. The tendencies are concerned with the variable "It". If we cut the space defined by "A" and "W~ or "IF variables like Fig. 7~Fig.9 following the values of "It", we make a function from (""A", "H""), or (""A", "W") space to "R" space. The function has a faculty combining states of human systems and I/O responses of the system and represent the relations between states of human systems and I/O responses. The changes of states of human systems and existence of such functions indicate that there are two possible approaches to design a method suitable to HCI; one is to keep the state in the faster response area, and the other is that response of computer corresponds to fluctuating state of human system. The former HCI method is suitable to important and significant HCI work, and the latter is to realize a comfortable HCI. In each case, a flexible or fluctuating mech~niam, which is following or checking the change of states of human systems, is required to future HCI. In order to realize
the flexible and fluctuating HCI,
studies about characters of human systems are
important.
5.CONCLUSIONS From the analyses and considerations, the conclusions of this paper are followings; (1) We measured I/O responses of human system, states of human systems like amounts of grouped a waves, cardiac rhythms and winking under a simple HCI task. (2) The all measured data are changing dynamically. (3) It is possible to represent the response of human systems as a function of factors indicating states of human systems. (4) Amounts of grouped a waves,
cardiac rhythms, and numbers of winking are one of the good
factors to determine the states of human systems. (5) A results of (4) show the importance of studies about human states to design a suitable method to future HCI. (6) A flexibility or fluctuation is one of important characteristics of future HCI.
REFERENCES (1)Mariko F. Funada, Satoki P. Ninomija, et.sl.:Analysis of brain activity for HCI, HCI in Yokohama, 1995 in printing. (2)Chiek Hukuda, et. al : A study about shifting time to low awakening condition on Monotonous VDT works, HCI in Yokohama, 1995 in printing.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawaand H. Mori (Editors) © 1995Elsevier Science B.V. All rights reserved.
969
Temporal organisation of h u m a n centred systems V.A. Chernomoretsa and S. V. Kirpichb aInstitute of Cybernetics, Ukrainian Academy of Sciences 252028 Kiev, UKRAINE bInstitute of Engineering Cybernetics, Academy of Sciences of Belarus 220012 Minsk, B E A U S * Abstract: In solving problems on analysis and synthesis of complex systems, in particular decision making in these systems, it becomes often necessary to consider the integrity (systematization) criterion, thereby yielding the emergent effect. Here the integrity criterion is a defined ratio of components or processes in a system, the relationships between which are characterised as organisation, harmonium, subordination to some proportion [1-5]. For example, it is known that at a visual perception of an object the perception strategies are oriented to the vision psycho physiology regularities that allow for the Fibonacci numbers. The present paper considers the strategy of the integral behaviour of man making discrete activity in time. Optimising the behaviour consists in subdividing a time interval of a discrete action (i.e., quantum) into sub intervals (operating period and activity slack). In this case, subdividing the time interval objectively obeys the Fibonacci series. Such a conclusion, grounded theoretically and proved experimentally, gives the possibility to model and optimise strategies of the man's activity in accordance with his psycho physiological characteristics by using the integrity criterion.
1. INTRODUCTION Decision making under conditions of uncertainly is a rather common kind of the information management man's activity in complex systems, viz., in man-machine systems, human-centred systems, etc. The present work is devoted to investigation of such kind of activity concerned with processing of discrete signals by a man-operator and analysis of * Mailing address: Nikiforov Str. 7-160, 220141 Minsk, BELARUS Fax: 7 (0172) 34 15 54; E-mail: [email protected] or kirs%avtlab.itmo.minsk.b~sr.eu.net
970 the management activity quality in the course of control and maintenance of a complex system. The input data for a man-operator under such conditions of the information management activity contain different uncertainties p e r t a i n i n g , for instance, to problems, methods or data. Such lack of information is either objective or due to economic restrictions. On making the decisions under the conditions of information uncertainty (or lack) it is, as a rule, presumed that a man fills a gap in the data with his personal experience, qualification, current situation, etc. in mind and with use of the integrity principle. The strategies of decision making by a man and temporal parameters of his activity are the goal of the present research. 2. S T R A T E G I E S OF TEMPORAL ORGANISATION OF SYSTEMS We will consider the strategies of man-operator's activity with regard to (a) external limitations on the time of signal processing; (b) duration of signal processing; (c) individual psycho physiological data and motivations of a man, i.e. the signal repetition period T, the processing time x of such signals. Now we pass to the dependence x - f(T). A decision is made by a man-operator within the limits train < T < tmax, where tmin(tmax) is the minimum (maximum) time interval for the man to perform any kind of activity. The time required for a man to take a decision during processing of the i-th discrete signal is Ti = ' r i + R i , where Ti is the i-th signal repetition period; x is the time of the i-th signal processing; Ri is the activity slack after termination of processing of the i-th signal before starting to process the next (i + 1)-th signal. Analysis of the relations ai - Ti/Ri and bi - Ri/~i reveals that a - f(T, R) is a monotones-increasing function while b - f(R, x) is a monotones-decreasing $ one. Then for an a r b i t r a r y fixed Ti there exists the unique value of xi at which ai - bi. In this case the following equality Ti
Ri
(1)
$
is valid, whence it follows that at xi - xi the activity slack R~ is Ri =
1+~2
, , xi = ( 1 . 6 1 8 . . . ) ~ = a xi
971 In view of the fact t h a t for numbers of the type (x the relation +affia 2
1
is fulfilled, t h e n ¢i = a-2T at a21;min _
(2)
Thus, the locus ~* of Eq. (I) m a y be described by Eq. (2).
3. EXPERIMENTAL RESULTS Experimental studies of the man-operator activities in a man-machine system concerned with processing of numerical data arrays are made in the assumption that some arithmetic operations with integers are to be performed. In the experiment, the signal repetition period T changed linearity. To evaluate the quality of the man-operator's activity, we used the dispersion D(~) of duration of signal processing in each array of numerical data. Such a quality criterion allowed unique interpretation of the obtained result in the experiment within the limits of the temporal range of ~. To calculate Eq. (1), we used the expressions 1
a
=
N
Ti.
1
~ZRi'
b
=
i=1
N
Ri
~ Z ~-T
(3)
i=1
The dispersion D(¢) - f(b) was at its m i n i m u m at b - 1.618... for the all tested man-operators. In this case, the m i n i m u m n u m b e r of various errors (corrections, omission of a signal, calculation errors, etc.) in the man-operat o r ' s activity was observed. Also, the dispersion D(¢) - Drain determined the induced deviations of • from its optimal value. The dynamic range from ¢min to ¢opt for a concrete tested man-operator was defined as a regular proportion Cmax
¢opt
-
1;opt
=
1.618
...
Cmin
Thus, it is established t h a t the time intervals of signal processing at the limiting ~min, optimal ~opt and natural ~max rates are interrelated by the geometric proportion with the base b - 1.618... (see Eq. (3)). In the general case, the temporal characteristics of the activity in the form of values of the function ~ - t(T) at some singular points Tn-1, Tn and Tn+l specified the character of their interdependence in the form of the r e c u r r e n t equation
972 • n+l
-~n-1
+ ~n
known as the Fibonacci series. 4. CONCLUSIONS To sum up, the experiments conducted have revealed that optimisation of the temporal parameters of man's activity is accomplished in the same proportions independently of a man or a kind of his activity. The observed strategies of man's activity rest upon the unified behavioural patterns which may be considered as the natural man's functional-structural strategy instinctively employed in processing of digital information and in the general case for decision making with regard for an influence of temporal parameters on the quality of man's activity. Based on the result of these studies, first published in 1986 by one of the present authors [1, 2], the following objective regularity of the information management man's activity is formulated: duration of processing of discrete specified values at limited, optimal, and natural rates is described by the Fibonacci series. REFERENCES:
1. V.A. Chernomorets, Some Feature of the Influence of External Limitations on the Parameters of Msm-Operator's Activity (in Russian), Cybernetics and Computer Engineering. Vol. 70, USSR, Kiev (1986). 2. V.A. Chernomorets, Man's Strategies in Formation of Subjective Limitations on Manual Control Parameters (in Russian), Cybernetics and Computer Engineering. Vol. 72, USSR, Kiev (1986). 3. S. V. Kirpich, Structural Inv~ismts in the Problem of Designing and Quality Assursmce of Systems. In: Proc. of the 1 l t h European Meeting on Cybernetics and Systems Research, Austria, Vienna. Cybernetics and Systems'92 (R. Trappl, Ed.) World Scientific Publishing Co, Singapore, (1992), 623-630. 4. S. V. Kirpich, The Technique of Analysis of Systems Using Integrity Criterion. In: Book of Abstracts of the 9th Int. Conf. on Mathematical and Computer Modelling, USA, Berkeley (1993). 5. S.V. Kirpich, OrgsJ~isation~ Design on the Base of Structural Harmonization Criterion. In: 4th Int. Symp. Humsm Factors in Orgs~_isation Design and Management, Workshop Papers, Block 1, Session 40rganisational Design, Sweden, Stockholm (1994).
VI.2 Job Design
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
975
J O B S A T I S F A C T I O N IN T H E C O M P U T E R - A S S I S T E D W O R K E N V I R O N M E N T ANDREW A. M O G A J I DEPT. OF PSYCHOLOGY UNIVERSITY OF LAGOS, LAGOS, NIGERIA.
ABSTRACT The rapid rate of industrialization has made the Nigerian economy to embrace the advent of computer technology. It is a common belief that computerization of activities will alleviate some of the human problems associated with the storing and retrieval of information in the work environment. Hence, this study tried to identify those factors within the work environment that tend to imfluence computer users' satisfaction. Responses were obtained from 550 employees including 440 men and 110 women randomly selected from different organizations in Lagos, Nigeria. The criteria for the choice of this sample, were based on the fact that the subjects must be using computer to accomplish task in their places of work. Based on their responses, subjects were classified into groups according to occupational status (e.g. managers, supervisors and junior workers) and sex. All other variables were regressed upon each of the last two variables. Utilizing the analysis of variance, the results showed that there were differences among the classified groups as to their satisfaction with the use of computer. However, significant differences were obtained for the sex and occupational groups. The findings of the study were interpreted and discussed as having socio-technical implications for technical change, management information system and organizational development.
INTRODUCTION We are in the computer age where tile use of computer to accomplish any task, name it, is in vogue. The level of technological development in Nigeria coupled with the rapid rate of industrialization, has made nearly all organizations in the private and the public sectors to go computerized. It is therefore the focus of this paper to identify which of the factors in the work environment that leads to users' satisfaction. Moreover, the paper will also identify which of the following factors: ability utilization, achievement, activity, advancement, authority, company policies and practices, compensation, coworkers, creativity, independence, moral values, recognition, responsibility, security, social service, social status, supervision-human relations, supervision-technical, variety and working conditions; that affects or relates with the user's intrinsic, extrinsic and overall satisfaction. However, it should be noted that measuring and analyzing computer user satisfaction, is motivated by management's desire to improve tile productivity of information systems (Bailey & Pearson, 1983). And researchers like Hanes & Kriebel (1978) and Morris (1978) have recognized that productivity in computer services means that there are both efficiently supplied and effectively utilized data processing outputs. It is further argued that computer utilization is directly connected to the user community's sense of satisfaction with those services. Cyert & March (1963) argued that the daily environment of the organization continually imposes upon nmnagers, the need for information. The success at meeting those needs either reinforces or frustrates the user's sense of satisfaction with that source. However, the literature contains a number of attempts at measuring user satisfaction. In each case, the users were asked to evaluate their computer services relative to a sense of satisfaction. For example, Debons, Ramage & Orien (1978) developed a list of 10 items affecting satisfaction out of which is the work environment. It is therefore necessary to study the relationship between the use of computer in line with some personal and organizational characteristics of the work environment; to determine how users cope with the demand and stress associated with the computer use. Swanson (1974) empirically found high correlation in a query environment between the user's appreciation for the system and the utilization of his outputs. Moreover, Neunmnn & Segev (1980) showed that there is a low correlation between bank branch users" reaction to satisfaction factors and their organization's
976
performance. So also, Lucas (1975) had shown a weak relationship between the economic performance of sales personnel and their information system utilization. But Powers & Dickson (1973) concluded that user satisfaction is the most critical in measuring computer system success and failure. But none of the reported studies made use of a standard measure of satisfaction. However, it can be argued that user satisfaction is related with infommtion system utilization and systems success. At any rate, an accepted measure of user satisfaction is clearly needed. Hence, the clear need is for a definition of satisfaction which contains a complete and valid set of factors and an instrument which measures not only the user's reaction to each factor but why the respondent reacted as he did.
Conceptual Model While seeking a model of computer user satisfaction, it is natural to turn to the efforts of psychologists who study satisfaction in its larger sense (e.g. Churchill, Ford & Walker, 1974; Cross, 1973; Schwab & Cummings, 1973; Smith, Kendall & Hulin, 1969). The literature generally agreed that satisfaction in a given situation is the sum of one's feelings or attitudes toward a variety of factors affecting that situation. Hence, satisfaction is the sum of the user's weiglRed reactions to a set of factors. This model suggests that satisfaction is the sum of one's positive and negative reactions to a set of factors. But the implementation of this model centers on two different requirements. The requirements according to Bailey & Pearson (1983), are; firstly, the set of factors comprising the domain of satisfaction must be identified. Secondly, a vehicle for scaling an individuaFs reaction to those factors must also be identified. That is why the present study has decided to make use of standardized instruments such as the Minnesota Satisfaction Questionnaire (MSQ) and the Job Descriptive Index (JDI), to meet up with the two above-mentioned requirements. However, Frese (1989) argued within an industrial psychology framework that the user should be in control of the computer use by suggesting the use of engineering approach to explain that three areas of research such as training, stress effects and optimization of interaction, should be substantiated to ascertain their consequences in performance and well-being. It is on this basis that the researcher is using the contingency theory such as the socio-technical model, to explain the relationship between the social system and the technical system. It is noted that the computer is a technical system and its use will suitably describe the kind of relationship being referred to, here. From tiffs, it is clear that the technical and human factors interact to influence job performance and that technology and the social system are in mutual interaction with each other and that each determines the other. Chalykoff & Kochan (1989) contended that computer-aided monitoring is likely to become more prevalent to the understanding of contemporary employee responses such as employee-level job satisfaction. The results of their study showed that employees' affective responses to monitoring are influenced by adherence to systematic feedback and that variables like clarity of rating criteria, supervisory expertise, and supervisory consideration behaviour are significantly related to satisfaction with monitoring. Aronson (1989) studied 132 individuals with frequent computer use to find out that changes in qualification demands anmng office workers, have been brought about by the process of computerization in the electronic industry. The results showed group differences in work group cohesion and possibilities for contact with work mates. These studies showed that the most computer-aided monitoring is a mediator of other managerial practices that are expected to influence job satisfaction. However, the results were discussed in relation to possibilities for personal growth, social-relationship and work-place learning and participation. Aronson, Dallner & Aborg (1994) examined the consequences of computerization for different types of computer users mid tried to identify the winners and losers in psycho-social terms. They found that work conditions vary considerably, both between groups of users and between men and women. They reported that the use of the computer affects everyone within a company and that the consequences vary between occupational groups. The analyses of results showed that there were significant differences between the different groups of users with respect to time spent on the computer. Majority of the subjects answered that they considered the computer equipment to be an aid in carrying out their work. A large majority of about 86 - 91% of the subjects also considered that they had good relations with their immediate superior coworkers. There was a weak tendency for people who had worked longest at a computer terminal to report poorer relations with their colleagues at work, With respect to relations with job supervisors and managers, there was no indication that time spent per day on the computer, was of significance for any of the groups. However, the groups were similar with respect to gender distribution and time spent at a computer terminal, to permit comparisons to be made. But there was no
977
statistically significant difference between the genders when tile analysis was performed by job category with the use of chi-square test. From all indications, it can be assumed that organizational tenure, educational qualification, gender, supervision, coworkers and other perceived characteristics of the work environment such as the working conditions, would relate with job satisfaction to affect the use of the computer. It is on the basis of this assumption that the following hypotheses will be tested: 1. There will be significant intercorrelations among the variables of interest, to yield job satisfaction in relation to the use of computer. 2. There will be significant differences between the sexes as to their satisfaction with the use of computer. 3. There will be significant differences amongthe occupational groups as to their satisfaction with the use of computer. METHODOLOGY Subjects: 550 subjects were randomly selected from different organizations in Lagos, Nigeria. The subjects comprised 180 managers, 170 supervisors and 200 junior workers. In all, 80% were men ~vhile women constituted 20% of the sample. They were all drawn from organizations such as bmfl~s, manufacturing and engineering companies. Their average age was 23.07 years. Their average years of formal education was 12.27 years. This means that all the subjects were educated and have at least West African School Certificate. They have been in their present employment where they have been using computer to accomplish tasks for the average of 4.07 years at the time of this study. At any rate, all the subjects were part-time students pursuing either the diploma or degree programmes at the University of Lagos. Materials: The following instruments were used for data collection: (I) The Biographical hlformation Questionnaire (BIQ) was designed by the investigator to obtain information about subjects' demographic and socio-econolnic cllaracteristics (e.g. age, sex, educational back~ound, tenure etc.) The Job Descriptive Index (JDI) designed by Smith, Kendall & Hulin (1969) was used to measure five (ii) aspects of job satisfaction: the work, supervision and coworkers' subscales which contain 18 items each, while the pay and promotion subscales have 9 items each. The response categories used are: "Yes" for affirmative and positively worded items; "NO" for negatively worded items while "7" is denoted for items for which the respondent is not certain. The authors reported an average corrected reliability coefficient for the five scales of .79 for split-half estimates of internal consistency. Higher internal consistency reliabilities were found for each of the scales: work (.84), pay (.80), promotion (.80, supervision (.87) and coworkers (. 88). The JDI possesses discriminant and convergent validities and studies clearly show that it has a moderate degree of subscales' inter correlation (e.g. Schrieshein & Kinicki, 1984, p.25). The correlations were reported to have a modal tendency in the .30s and low .40s, although some range as low as .08 and as high as .76. The JDI is also said to have a predictive validity concerning withdrawal behaviour like absenteeism and turnover. The Minnesota Satisfaction Questionnaire (MSQ) developed by Weiss, Davis, England & Lofquist (1967) (iii) having 20 items, was used to measure Intrinsic Satisfaction (12 items), Extrinsic Satisfaction (6 items) and General Satisfaction (the whole 20 items). The items were scored on a 5 point Likert-type scaling system from Very Dissatisfied = 1 through Not Decided = 3 to Very Satisfied = 5. Apart from the summation of scores to derive scores for the three major subscales, ratings for each of the items, represent the score for each of the scales measuring computer user variables. Weiss et al. (1967) presented normative data for a range of occupational groups. They reported Hoyt internal reliability coefficient for the subscales and overall scales for a number of samples as : Intrinsic Satisfaction (rsmgs of 0,84 to 0.Ol ); Extrinsic Satisfaction (0.77 to 0.82) and General Satisfaction (from 0.87 to 0.92). Test-retest reliability of 0.89 ~vas reported across one-week (for 75 employees attending night school). Wanous (1974) reported a correlation of 0.71 between General Satisfaction scores and the sum of the 5 JDI subscales.
978
Procedure: The subjects were tested in the classroom while receiving lectures for their part-time diploma and degree programmes at the University of Lagos in Nigeria. The criteria for the choice of the sample, were based on the fact that the subjects must be using computer to accomplish tasks in their work-places. Based on their responses, subjects were classified into groups according to occupational position and sex. Each of the variables/scales was scored differently. But some sub-scales made use of the same scoring technique. It should be noted that there are no right or wrong answers. For the JDI, there is a scoring key for scoring the items. In the case of the MSQ, scoring dimension is by summing up the scores for each of the subscales. The possible range of scores is between 20 and 100 for General Satisfaction; 12 and 60 for Intrinsic Satisfaction and between 6 and 30 for Extrinsic Satisfaction. The higher the score, the more satisfied m~ individual is, on each of the subscales in both cases of JDI and MSQ. However, the score of each of the items in the MSQ, represents the scores of the Other 20 scales which were used to measure the respondents'satisfaction on the job (in the use of computer). The SPSS program was used to analyze test-scores. Analysis of test scores: An examination of tile correlation coefficients between silnilar and dissilnilar scales across the instruments, was performed through the regression analysis. But tile analysis of variance was used to find separate and interactive effects among the variables. RESULTS All tile variables from Age, Education, Computer users' Satisfaction, reasons for this satisfaction, Work, Supervision, Coworkers - (human relations), Pay, Promotion, Motivation, Intrinsic Satisfaction, Extrinsic Satisfaction, Overall Satisfaction, Ability Utilization, Achievement, Activity, Advancement, Authority, Company policies and practices, Compensation, Coworkers - (technical), Creativity, Independence, Moral Values, Recognition, Responsibility, Security, Social Service, Social Status, Supervision-human relations, Supervisiontechnical, Variety and Working Conditions; were taken as the independent variables and regressed upon sex mid occupational groups that were regarded as dependent variables. The relationship between all the independent variables and each of the dependent variables, is generally classified as R. The statistical tests operated include correlations, intercorrelations and multivariate analysis. The summary of results is shown in the tables below. A matrix of correlation coefficients between the measures of all variables of interest, is shown in table 1 below.
,_.,"
~ ~ ~ ~
"
,_
•
.i
If)
~
~= (D
T-
~ i~
._~
(N
:
C~I .,r-
,-
O
("wI
O
w-
O o r-. o L
_m O (.3
o"
I.O tO II Z a/
> o
...q
~v-V o o
"a
¥1
~g_r--® .~.~ ~ O
~,
,
R3
(2)
(.O
~ ~ ~'..8E~,~',~o ~
~ ~ ~ ~ ' ~ o ~ .
); F~~
~
,-~
~'{R~~i
.. !~o~FI~.'(" ~P'
o~-
~0;0)
~ ~ ~ 8 ~ ~ ~ ~ ~ ~ , ~ ~ i ~ , ~ s ~
~~ •-'.
979
980
Table 1 above, shows the nmtrix of correlations between scores derived from measures of variables of interest in this study. Table 2: The Summary of the Multiple Regression of all Variables on Sex and Its bmalysis of Variance. Multiple R = .80154 R Square = .64247 Adjusted R Square = .08063 Standard Error = .38707
Analysis of Variance Ree;vession Residual
DF 33 21
Sum of Square 5.65370 3.14630
Mean Square
Obtained F
Table F
.17132 .14982
1.14350'
.3801
Note: * Significant P < .01 Table 2 above shows that the relationship between sex and all other variables, is significant at .01 level of significance. Table 3" The Summary of the Multiple Regression of all Variables on Occupational position and Its Analysis of Variance. Multiple R R Square Adjusted R Square Standard Error Analysis of Variance Regression Residual
DF 33 21
=
.82239
= = =
.67633 .16770 .76457
Sum of Square 25.65127 12.27600
Mean Square
Obtained F
Table F
.77731 .58457
1.32971 *
.2492
Note: * Significant P < .01 Table 3 above shows that the relationship between occupational position and all the variables, is significant at.01 level of significance.
DISCUSSION
The results in table 1 shows that all measures of the variables correlate with each other. But, a careful look at the table, reveals the significance of intercorrelations between measures at both. 01 and.001 levels of significance. Out of 335 correlations computed, about 144, were significant meaning that 43% of the correlations, were significant. It is very easy to deduce from the table which of the measuressrelates with the other~to yield job satisfaction. The convergent and discriminant validities of measures can also be easily computed from the recorded correlation coefficients. This can be achieved by comparing the agreement between similar variables measured in different ways, with that of dissimilar variables measured in the same way or vice versa. Looking at table 1 very carefully, it is seen that when all the variables were regressed on Overall Satisfaction, all the correlations with the exception of its relation to compensation (i.e. 19 out of 20 correlations), were significant at either.01 or.001 levels of significance meaning that hypothesis 1 is confirmed. This shows
981
that job satisfaction can be derived from the interaction between some personal variables and organizational factors within the work environment. The results show that compensation is not necessarily a motivating factor. In the case of hypothesis 2, the results in table 2, show that when all the variables were regressed on sex, R=. 80 was obtained, the scores were further subjected to the analysis of variance and F = 1.14 was obtained. The result is sig~aificant at. 01 level of sigjfificance thus, confirming hypothesis 2 which suggests that there will be sexdifferences in job satisfaction and the use of computer. The results did not support the finding of Aronson et al. (1994) which did not show a significant difference between the genders. The reason for this trend of results, may be due to the variance in the work environment of the selected organizations and the ability of workers to adapt to psycho-social work conditions which depends on the time spent, using the computer. With respect to hypothesis 3, the results in table 3, show that when the relationship anmng the occupational groups mad all the variables was sought, R# .82 was obtained. Subjecting the scores further to an analysis of variance, the obtained F = I. 33, was found to be significant at .01 level of significance. This means that hypothesis 3 is confirmed as there is a significant difference among the three occupational groups of managers, supervisors and junior workers. The difference mnong the groups may be as a result of the length of period for which they have been using computer to accomplish tasks. The results further support the finding of Aronson et al. (1994) who suggested that time spent on the computer may cause significant differences among groups of users. The findings of Chalykaff & Koclmn (1989) that computer-aided monitoring can enhance the relationship between supervision and satisfaction, may be adduced as one of the reasons for the results of the present study. CONCLUSION Data ffoln this study have shoval that there are significant intercorrelations between measures of variables of interest. There was also a significant difference between male and female computer users. The difference among the managers, supervisors and junior workers, was also significant. Generally, the trend of results for this study is not surprising as maturation affects individual's capability to adapt to situations and to learn new things such as ill the use of computer. Although, the results showed that the use of computer leads to job satisfaction between sexes and among occupational groups but further analysis is needed to determine the interaction between the two factors. The generated data can also be factor-analyzed to determine which of the variables are either personal factors or organizational factors. Hence, the results have implications for computer training as some of those factors call be identified practically during training. This means that the findings of this study can be interpreted as having socio-techincal implications for technical change, management information system and organizational development. REFERENCES Aronson, G. (1989) Changed qualification demand in computer-mediated work. Applied Psychology: An International Review. 3__88,(1), 57 - 71. Aronson, G; Dallner, M. & Aborg, C. (1994). Winners and losers from computerization; A study of the psychosocial work conditions and health of Swedish state employees, lnterlmtional Journal of HmnanColnputer Interaction, 6, (1), 17 - 35. Bailey, J.E. & Pearson, S.W. (1983). Development of a tool for measuring and analyzing computer user satisfaction. Management Science, 29, (5), 530 - 545. Chalykoff, J. & Kochan, T.A. (1989). Colnputer-aided monitoring: Its ilffluence on employee job satisfaction and turnover. Personnel Psychology, 42, 807 - 829. Churchill, G.A; Ford, N.M. & Walker, O.C. (1974). Measuring the job satisfaction of industrial Saleslnen. Journal of Marketing Research, l__Ll(3), 254 - 260. Cross, D. (1973). The Worker Opinion Survey: A Measure of Shop-Floor Satisfaction. Occupational PsvcholoKv, ( 3 - 4 ) , 193- 208. Cyert, R. & Marcia, J. (1963). A belmvioral Theory_ of tile Firm. Englewood Cliffs, N.J." Prentice Hall, Inc. Debons, A; Ramage, W. & Orien, J. (1978). Effectiveness Model of Productivity, In L.F, l-lanes & C. H. Kriebel (eds-)p Research on Productivity Measurement Systems for Administrative Services: Computing and Information Services. 2, NSF Grant APR- 20546.
982
Frese, M. (1989). Humm~ Computer Interaction within an Industrial Psychology Framework. Applied Psychology: hal International Review, 38, (1), 29 - 44. Hanes; L.F. & Kriebel, C.H. (1978). "Research on Productivity Measurement Systems for Administrative Services: Computing and hfformation Services', vols. I & II, Pittsburgh, Pa, Westinghouse Electric Corporation, Research and Development Center. Lucas, H.C. (1975). Performance and the Use of an hfformation System. Management Science, 21, (8), 908 -919. Morris, W. T. (1978), The Development of Productivity Measurement Systems for Administrative Computing and hfformation Services Ohio State University. Neumann, S. & Segev, E.(1980). Evaluate Your Inf0rnmtion System. Journal of Systems Management, 31. Powers, R.F. & Dickson, G.W. (1973). MIS Project Management: Myths, Opinions and Reality. California Management Review, 15, (3), 147 - 156. Schein, E.H. (1990). Organizational Psycholog~y. Englewood Cliffs, N. J: Prentice Hall Inc. Schwab, D.P. & Cummings, L.L. (1973). Theories of Performance and Satisfaction: A Review. In W.E. Scoot & L.L. Cummings (eds.), Readings in Organizational Behaviour and human Performance. Homewood: Richard D. Irwin, Inc. Smith, P.C; Kendall, L.M. & Hulin, C.L. (1969). The Measurement of Satisfaction in Work and Retirement: A Strate~,y for the study of Attitudes. Chicago: Rand -McNally. Swanson, E.B. (1974). Management hfformation Systems: Appreciation and Involvement. Management Science, 21, (2), 178 - 188.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
A Study on S h i f t i n g Time to Low A w a k e n i n g C o n d i t i o n s on M o n o t o n o u s V D T W o r k s Chieko FUKUDA', Satoshi SUZUKI", Takao TANAKA", Keiko KASAMATSU'" , Yusuke YAZU", Mariko Fujikake FUNADA', Kyoko IDOGAWA'", Satoki P.NINOMIJA'" *Hakuoh Univ., Tochigi, Japan, **Aoyama Gakuin Univ., Tokyo, Japan, ***Seitoku Univ., Chiba, Japan 1. INTRODUCTION We are now in a computer age when everyone has his own computer and operates without any training. Then a man-machine-interface which is adjusted to a human being is an essential requirement. Now the main interface method is using VDT. Input devices of VDT can be roughly classfied into two groups. One is analogous such as a mouse and a track ball, the other is digital such as a keyboad. The influence of VDT works especially fatigue has been discussed from various angles. And many interfaces were improved for the purpose of decreasing this fatigue. We have been concerned about workers' reactions to extended periods of computer interaction. We have studied them especially from a viewpoint of EEG changes which directly bespeaks physical changes. This time, we research three kinds of monotonous VDT work which have different interface. During repeated monotonous work which needs less physical exercise, most of workers shift to low awakening condition. Then we research the characteristic of shifting time to this condition. In the previous studies, it's obvious that we can use appearance of grouped a waves as the indicator, of low awakening level on which brain a c t i v i t y goes down. Of course, in these a p p e a r a n c e s there are many i n d i v i d u a l d i f f e r e n c e s of time and quantity. But it's p r o v e d by the experiment most of subjects who were not trained to privent drowsiness and had a fully sleep have shifted low awakening condition between 20 and 30 minutes later. When the subjects continued the same monotonous work,they were shifting to low awakening condition regardless of different interfaces. 2. METHODS 2.1.
Subject Normal 5 male students and 2 female students aged 20 to 23. The day before those experiment they had at least six hours' sleep. 2.2. Work a. Experiment 1
983
984 A data entry of one-digit random number (from 1 to 9) displaied on a screen, using a keyboad. They type the numbers as fast as possible. 0.5 seconds later from responding correctly the next o n e - d i g i t number is displaied. If the response is incorrect~system is waiting until the correct number are inputted. b. Experiment 2 A couple of two-digit random numbers (from 10 to 88) are displaied, the subject adds them in his head and enters the answer using a keyboad. They types the numbers as fast as possible. 0.5 seconds later from responding correctly the next two-digit numbers are displaied. If the response is incorrect) system is waiting until the correct number is inputted. c. Experiment 3 A driving simulation using a steering wheel. There is a imitative car and in front of it there is a screen. On the screen a moving point is displaied. In the car there is a wheel for moving the cursor on the screen. The subject makes an effort to fit the cursor to the moving point on the screen using the wheel. Our experiment grade for this work is a distance betweem the cursor and the movineg point. 2.3. Records (~ 100 times per second we record the following data In experiment 1 or 2 • a digit on screen and an inputted one In experiment 3 • a distance between the cursor and the moving point (~) Electroencephalogram(C3-O~) Sampling frequency is 100Hz. (~) Electrocardiogram Sampling frequency is 100Hz. (~ Their facial expressions during the experiments taped on a video tape. 2.4. Calculated variables • Task performance In experiment 1 or 2: (~ Numbers of displaied digits per minute (~) A ratio of numbers of incorrect input to those of total input (~) Length of time between a display of digit and the correct answer (~ Length of time between a display of digit and the first response if it's correct or not. In experiment 3: (~) The mean distance per minute. • Physical changes In all experiments: O Grouped a waves appearance time. 0 The mean of heart rate variables per minute. (~) Numbers of blink per minute. (~) Numbers of blink whose duration times is more than 0.5 seconds. 3. RESULTS There are three types of human computer interactions (HCI). The first, using a k e y b o a d digital type input devices they continue monotonous works. The second, though using the same type devices the tasks
985
are more complex. The third, using a steering wheel analogous type one they continue monotonous work. In the previous studies, we recognized appearance of grouped a waves as the indicator of low awakening conditions. In the Fig.1. we marked ' i ' w h e n grouped a waves appear every 2 s e c o n d s. That's a distribution of g r o u p e d a w a v e s . T h e n , we e x p e c t e d t h r e e experiments show us a differnce pattern of shift to 1o w aw ak ening c o n d i t i o n s but less d i f f e r e n c e we can find. In all e x p e r i m e n t 10 m i n u t e s later from the beginning grouped a waves appeared. About 20 minutes later appearance time of grouped a waves per minute was longer. We cannot avoid s h i f t i n g l o w a w a k e n i n g c o n d i t i o n d u r i n g e x t e n d e d p e r i o d s of computer interactions. F i n a l l y , w h i c h of t h o s e i n t e r a c t i o n t h e y u s e d is iaot so directly i n f l u e n c e to a w a k e n i n g conditions. Rather than that, the factors of c h a n g i n g a w a k e n i n g c o n d i t i o n are subject's p h y s i c a l c o n d i t i o n or their motivation. And avoiding using one kind of interface we must consider about combination of task. In this f i g u e r c h a r a c t e r i s t i c of g r o u p e d a waves appearance is not linear but fluctuate.
E×periment l
subject a
!
i
b Iii
c
d e
I IIIII I
i Iii
i!
i
!
II il
! i~
!1
•
8
IlllU]
I!
! II
a,ilnn
III
I
II
II
illlllli
i lli
I Iniillllilillllll
lllilll
fiBRil
Ii
Ii l llllll
! I
illllnilllllllmllminmliililUlillllnil
3'0
a'O
!
i
I
I
I
I
I!1
il III
!1 iinllllllll
Ii111 I I ! i IIii111
30
I!
li
III
I
! I!1
2O
10
mln.
Distribution
of
grouped
a
waves
shows us the interval in which grouped
horizontal
axis:
a
waves appeared.
time
Experiment 1. : Data entry of one digit random number. Experiment 2. : Addition of 2 digit random numbers. Experiment 3. : steering simulation This figure shows us less differances between three experiments, but shifting to low awakening coditions
I
ii
II
Iii!!11
aO mln.
il at li IH II il
0
II
2O
E×per/ment3
a
I!1111 I'i I
lmmll
10
subject
I
Iil
mln.
lllliil i IIii
nl
I llllliillllmllllHIllilllli
2'0
Hill !1111 l!
0
Fig.1.
a
Experiment2
I II
I
b
I In
ill
!! iilll
l'O subject
I
IllUlalllllHlilililli i III!
i
I
fluctuate.
986
subject
d
at
experiment
1.
sec/ i -; i i ....:: ~.~..,:~:...................... i.................................................................................... •........................................ i.: .................................. i ......................................... T"i i 11-. i i. i ,, ,7 ! ~ ~III
10 iJ;:~ ........................................ i "I:: (I ,t............ i :
..... <1~ ......................................................... , , ' ~ Illi ........ ~...................[Ij
! I:,, --- ,-ii subject
b
at
experiment
A5 m m
2.
:.....-.i..................................................................................................................................................................................................... 7 -'"
i
•
;-"
!
s2c/ i ,~' ~2¢ ........................ i........................................................................................................ i.i., ~, F ~{
~i
~r
i i
i
......ii...'.<..iiii.....ii."III ....
g
i,
%.."J................................................................... ;7"'"t...............::
z..,:-
i ~
.,.
~,,"
{I
"
i
/
~'''"
....
.............
iiJll" iii iii iiii
..
............................................................................................................... L:::II
Fig.2.
Three
dimensional
graph
i
............. ~...............ill .................................
IS
30
of
grouped
a
waves
and
rain
correct
time. horizontal axis: time vertical axis
: the mean time of correct response
depth
: the sum of grouped
~r
waves appearance time per minute
In this graph the higher points locate in the distance and the lower points locate near here. Correlation correct response time with grouped a
waves
appearnce time is large. Awakening levels go down not directly but repeatedly go up and down, finally become drowsy. Correct response times follow that appearnc%as a result task performance decreases.
response
987
Next we consider about relation between awakening condition's change and task performance. Grouped a waves appearance time and task performace, we display those values at 3 dimensional graph. In the Fig.2. we make time series graph about subject d at experiment 1 and subject b at experiment 2. Horizontal axis is time, vertical axis the mean time of c o r r e c t response, and depth is the sum of grouped a waves appearance per minute. In this graph the h i g h e r p o i n t s l o c a t e in the distance and the lower points locate near here. Correlation correct response time with grouped a waves appeamce time is large. Awakening levels go down not directly but repeatedly go up and down and finally become drowsy. Correct response times follow that appearnce, as a result task performance decreases. In the Fig.1. and Fig.2. we can find dynamical changes in human systems. In Fig.3. those changes are discripted by frequency response of grouped a waves appearnce interval (Left) and one of correct response time interval (Right). In those figuers fluctuation is recognised. Fitting regression line, the left g r a p h ' s c o e t f i c i e n t larger than the right one. Even if brain activity changes are large, human behavior is change not so directly. It means that human system is very stable. 10 ~
10 7 rouped a waves
106
10~
£
correct r e s p o n s e t i m e
fd
I0 L
0~,
I(#.
10
lY
1C l
10
102
'3
IO/2&OOHz
frequency
Fig.3.
10
102
l O}8a.OHz
frequency
Fluctuation
L e f t - grouped a waves appearnce interval Right" correct response time interval Fitting regression line, the left graph's coefficient larger than the right one. Even if brain activity changes are large, human behavior directly. It means that human system is very stable.
changers
not so
988 4. DISCUSSIONS Experiment 3 this work is "an analogous behavior which is more continual and tends to shift to low awakening level. Experiment 1 this work is a digital behavior, it's rhythmical and need a little judgment. B u t they get used to this work then shift to l o w awakening level. In experiment 2 they need ability of calculate which is a high-level brain activity. They feel not only monotonous but also fatigue and shift to low awakening level. In term of the appearance time of grouped a waves there are few difference between three types of interaction. Anyway during monotonous work which recieves less stimulation from outside and needs less physical exercise we confirm that EEGs which reflect physical condition directly change from 20 to 30 minutes later and awakening level go down objectly. They cannot privent drowsiness. There are two reasons why we shift to low awakening level. They are monotonousness and fatigue. Now, our concepts of developing a man-machine-interface are easy, simple and standard. But our 3 experiments show that they are not so suitable for us. On the premise of those human charactors we must design interface which is not only easy and simple but also has a certain kind of variety and physical exercise. Now, in the case of data entry they don't take batch processing but using a hand-held-computer they enter datas when and where datas created. It's very desirable style. 5. CONCLUSIONS Our conclusions of this paper are following 1) During three kinds of: monotonous work which recieves less stimulation from outside and needs less physical exercise we confirm that EEGs change from 20 to 30 minutes later and awakening level go down objectly. Few difference between those methods of interaction are recognized. 2) Awakening levels go down not directly but repeatedly go up and down and finally become drowsy. 3) The same monotonous work continue, human cannot prevent drowsines, when human computer interaction are designed, we must have the viewpoints which is not only easy and simple but also has a certain kind of variety and physical exercise. And consider of c o m b i n a t i o n of task and rest~ we must prevent monotonous. REFERENCES 1. M.F.Funada, Y.yazu, et al.: On a sensor to detect the point just before the dozing off state is reached, Sensors and Actuators, Vol.43, No.l-3, 145-152,1994. 2. C.Fukuda, M.F.Funada, et al.: Evaluating Dynamic Changes of Driver's Awakening Level by Grouped a waves, The 16th Annual International Conference of IEEE EMBS, Vol.2 1318-1319, 1994. 3. M.F.Funada, S.Suzuki, et al.: Dynamic Changes of Human Systems under a Simple Task of HCI, 6th International Conference on HCI in Tokyo
1995.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
989
C o m p l e m e n t a r y allocation of functions in a u t o m a t e d w o r k systems G. Grote, S. Weik, T. Wafter and M. ZOlch Work and Organizational Psychology Unit, Swiss Federal Institute of Technology, ETHZentrum, CH - 8092 ZUrich 1. INTRODUCTION Determining the degree of automation, which includes the allocation of functions between human operator and technical system, is a central step in the design of work systems. It has important consequences for the specification of technical requirements, for the design of the jobs of future operators of the system, and for the efficiency, quality, and safety of the production process. When following new engineering concepts like simultaneous or concurrent engineering, function allocation decisions in relation to the production process are of particular importance. In "traditional" engineering, more time was allowed for optimizing the production process (cf. e.g. Susman, 1992), which also left more time for compensating for inadequate or missing decisions regarding the allocation of function. The more intertwined and rapid product and process development cycles become, the less opportunity there is for correcting design failures or inadequacies. Naturally, the "right first time" requirement concerns function allocation as well. What is needed is the prospective design of operator jobs and technical systems based on allocation decisions which take into account technical, organizational, and people considerations. However, as a number of authors have pointed out, to date these decisions are often made "unconsciously rather than by deliberation" (Price, 1985, p. 33). A second argument for the importance of function allocation decisions stems from the increasing need for manufacturing flexibility. The allocation of functions will significantly affect flexibility, not only because machines are still - despite the development of flexible manufacturing technologies - inflexible by comparison, but also because functions can be allocated in a way that renders it very difficult or even impossible for the human operator to use his or her flexibility. Two other important issues in this context are the opportunities for the development and maintenance of practical production skills provided by a given allocation of functions and the shift generally required in more automated systems from practical to theoretical systems knowledge. How both of these issues are dealt with in terms of technical and job design as well as qualification of the operator will again strongly influence the degree to which human potential can be employed. New engineering concepts, even more so thantraditional strategies, call for an integrative approach to technical, job, and organizational design, with function allocation being at the core of design decisions regarding the production process, potentially also the product itself. Such an approach has been described as work-oriented (cf. Ulich, 1994). After briefly looking at existing methods for making allocation decisions, we will present our own approach to these issues. 2. EXISTING METHODS FOR FUNCTION ALLOCATION A strategy still frequently adopted in industrial practice for deciding the allocation of functions is the "leftover principle" which starts with the allocation of functions to the technical system only, i.e. attempting to fully automate a given process. Whatever functions
990 cannot be automated are then allocated to the human operator. These functions do not necessarily form a meaningful job, they might even be impossible to carry out, as Bainbridge (1982, p. 130-131) has argued convincingly: "... the automatic control system has been put in because it can do the job better than the operator, but yet the operator is being asked to monitor whetherit is workingeffectively.... if the decisions can be fully specified then a computer can make them more quickly, taking into account more decisions and using more accurately specified criteria than a human operator can. There is therefore no way in which the human operator can check in real-time that the computer is following its rules correctly." This quotation also illustrates the problems associated with a second approach to function allocation, the "comparison principle". Based on lists comparing characteristics of humans and machines (e.g. Fitts, 1951), functions are allocated to the one who supposedly can perform that function better. With increasing technical possibilities, these lists change as more and more functions can be performed better by machines, leaving monitoring and dealing with deviations and disturbances to the operator, which brings us back to Bainbridge's example of producing impossible jobs. Jordan (1963) pointed out thirty years ago, that humans and machines are fundamentally different and therefore cannot be compared and assigned functions accordingly. Instead, they are to be seen as complementing each other in performing a task. Function allocation should allow for the support and development of human strengths and compensation of his or her weaknesses by the technical system in order to secure efficiency of production as well as humane working conditions. This approach may lead to a design strategy which deliberately does not use the full potential of technology for automating a given process in order to create a meaningful and challenging job for the human operator, thereby furthering the overall effectiveness of the work system. The focus of most recent attempts is to operationalize this concept of complementarity. However, looking at existing methods and instruments for the complementary allocation of functions, one is at first sight confronted with a multitude of criteria for complementarity, such as flexible and dynamic allocation, transparency, operator control, low coupling, and operative flexibility, just to name some frequently mentioned criteria (e.g. Bailey, 1989; B6hle & Rose, 1992; Corbett, 1985; Henning & Marks, 1986; Kraiss, 1989; Price, 1985). There have also been some attempts to operationalize these criteria, but to date neither a fixed set of criteria, comparable to software usability criteria, nor generally accepted methods for their measurement exist. As an overriding principle, it is widely recognized that current and future users should participate in system design in order to capitalize on their knowledge and to incorporate their needs and concerns into the design process (e.g. Ulich & Grote, in press). Finally, the link between function allocation and job design needs to be pointed out. Generally, two approaches can be distinguished, the first being more frequently applied: (1) different options for function allocation are developed and, among other things, evaluated with respect to the quality of the resulting jobs or (2) jobs are designed that conform with criteria for intrinsically motivating work and functions are allocated accordingly. These approaches will be discussed in more detail and in comparison with the approach chosen in KOMPASS at the end of this article. 3. THE KOMPASS APPROACH TO FUNCTION ALLOCATION KOMPASS is a research project concerned with developing guidelines for the complementary design of production systems. The project is part of a larger cooperative research effort in the field of advanced manufacturing systems, with a number of engineering disciplines involved in the design of highly integrated production technology. The guidelines are intended for design teams in industrial settings, supporting them in following an integrative approach to systems design, with a special focus on decisions regarding function allocation. The current emphasis of the instrument is on the design of work systems in piece production, but the general principles embedded in the suggested procedure
991 are applicable to other types of work systems as well. The instrument will be in the form of written guidelines, leading a design team through the following four design steps: (1) Identification of furthering and hindering preconditions for complementary design: Automation philosophy of project team; openess for work-oriented design; potential for creating or strengthening such openess. (2) Sociotechnical analysis of the existing work system: Analysis of technical, organizational, and social conditions and problems; evaluation of existing jobs with respect to chances and risks of the planned changes; basis for defining project objectives (including the possibility of aiming at a purely organizational solution). (3) Development of options for the allocation of functions between humans and machines as well as between humans: Based on an allocation heuristic, options are developed in cooperation with current and future system operators. (4) Evaluation of options based on work-oriented criteria for function allocation, resulting jobs, and sociotechnical design. A participatory approach to the design process is envisioned, that is, future system operators should be included in the design decisions as early as possible. However, participation and the use of expert design criteria have to be balanced. If a design is based on a fixed set of very specific criteria, participation has no meaning anymore. On the other hand, the use of participation as an excuse for not defining clear-cut, more general design criteria is to be avoided. One way of achieving this is to specify the steps and tasks within a criterion-guided design process where participation is required (Grote, 1994).
3.1. Criteria for function allocation, job design and sociotechnical design Existing criteria for the complementary allocation of functions as well as for job and organizational design were compiled and a set chosen for the initial exemplary analyses of work tasks in automated production. Based on the results of these analyses, the following criteria were selected for further analyses including tests of reliability and validity: • First level of analysis and design - The sociotechnical system (adapted from Emery, 1978; Susman, 1976; Ulich, 1994): Completeness of task of organizational unit; independence of the organizational unit; task interdependence, polyvalence of the operators; autonomy of work groups; boundary regulation by supervisors; • Second level of analysis and design - The individual work task (adapted from Dunckel et al., 1993; Hacker, 1986; Oesterreich & Volpert, 1986; Ulich, 1994): Completeness of individual tasks; planning and decision making requirements; task variety; cooperation/communication requirements; autonomy; opportunities for learning and personal development; absence of hindrances. The criteria for the third level of analysis and design, the human-machine system, are presented in more detail in Table 1 as most operationalizations were developed within the project whereas the criteria on the other levels of analysis were adapted from existing instruments. The complete set of criteria was tested for reliability and validity by means of 30 analyses of pairs of identical jobs mostly in the machine industries. Usually two operators working with the identical technical systems on different shifts were chosen and questioned and observed by two different investigators. The operators were also asked to fill out a questionnaire containing measures for job perception and persoanl well-being as well as a diary of all production disturbances and their management during one week. The analysis of the obtained data has not been completed yet. However, first qualitative results of the analyses show that the criteria cannot be treated as independent. Especially, the relationship between coupling and process transparency needs to be investigated further. Usually, research stresses the necessity of loosly coupled systems in order to provide higher levels of autonomy for the operator. With high levels of automation, loose coupling may result in the loss of process transparency, however. It is argued that not loose coupling, but dynamic coupling is to be aimed at, that is giving operators the possibility to influence the amount of coupling, e.g. by means of programming processes in different ways or flexibly allocating functions to the technical system or themselves.
992 Table 1 KOMPASS criteria for the level of the human-machine system Third Level of Analysis and Design: The Human-Machine System (adapted from Clegg et al., 1989; Corbett, 1985; Kraiss, 1989) Coupling • Closenessof formal coupling between operatorand machine/Influenceof operator on degree of coupling: Extent to which the technical system determinesthe activity within a human-machine system, including when, how and where it has to happen as well as by whom and by which means • Closenessof cognitive coupling based on level of required cognitiveeffort and influence of operator on the amount of effort Transparency / Proximity to the Process • Possibilitiesfor developing a mental model of the production process • Proximityof the operator to the actual process (analogous and digital feedback) Decision Authority • Extentof decision competenceover controlling the production process (manual, automatically supported/limited, manually limited/approvedand automatic) Flexibility • Possibilityfor flexible/dynamicfunction allocation • Periodof time (long-term,medium-term,short-term)which is necessary to achieve flexible allocation • Adaptivity/adaptability(decision on changes in function allocation by system and/or operator) Technical Linkage • Dependenceof the observed technical system on other technical systems,e.g. with respect to data transfer
3.2. The use of the citeria in a design process The starting point is a systematic analysis of the work system for which automation is considered. In order to support the description of the functions to be performed by the system and their evaluation by means of the KOMPASS criteria, a standard set of functions was defined. Examples of these functions are design workpiece, improve existing production process, plan work, program machine, prepare production resources, operate technical system, check workpieces, maintain and repair technical system. For each function performed in the existing work system, the following questions are asked: What is the content of the function? What has to be done by the human-machine system? What are the objectives or the products of this function? Which parts of the function are taken over by the operator, the machine or both? Are there other persons involved in the fulfillment of the function? The functions and the task resulting for the human operator can then be assessed by the criteria presented above. To evaluate scenarios representing different forms of function allocation for a planned system, the KOMPASS-criteria can be used in the same way as for tasks in existing sociotechnical systems. The design team has to systematically determine in advance all the functions which would be affected by the aimed at technological change (for an example see Grote, Weik, W~ifler & Z61ch, in press). Concerning the development of options for the allocation of functions, a heuristic has been outlined and tried out in some pilot cases, but will have to be tested further in ongoing automation projects (Grote, 1994; W~ifler, Weik & Grote, 1995). The basic idea of the heuristic is to support the design team in following an integrated approach to system design by first developing a shared understanding of the system objectives and the contributions of technical, organizational and human aspects to successful system performance. Based on this understanding, options for function allocation are determined and evaluated by means of the KOMPASS criteria. In order to stress the link between function allocation and job design, each function is analyzed with respect to four aspects: decision requirements, type of activity
993 (planning, executing, controlling), importance for transparency of the production process, and automation potential (including the classification "must not be automated" on the basis of job design considerations). The characterization of functions in this way allows for a rough allocation into three "slots": (a) allocation to human alone (based mainly on the automation potential), (b) allocation to machine alone (functions that contain no decision requirements, are not important for process transparency, and technically can be automated), and (c) allocation to human and machine (functions that contain decision requirements and planning and controlling elements, and/or are important for process tranparency). The functions in the third slot then have to be analyzed in more detail and allocation options developed for them, taking into account different organizational options (e.g. programming on the shopfloor versus in a separate organizational unit). For all functions, decision authority has to be determined, meaning the degree to which operator and technical system can influence the execution of a function (Kraiss, 1989). The different design options are then evaluated - in as much as that is possible in prospective design - on the basis of all KOMPASS criteria. Participation of operators having worked in the "old" production system and of future system operators is crucial during the development and evaluation of design options. 4. CONCLUSIONS Having outlined the potential use of the KOMPASS criteria and heuristic, the question arises of how this particular approach differs from comparable approaches and what its advantages and disadvantages are. Two kinds of approaches which deal explicitly with the question of function allocation between human and machine in relation to job design were distinguished in section 2. Both of them can be described as work-oriented or human-centred, and both basically follow the sociotechnical tradition. Nevertheless they differ in some crucial points. The first approach - of which the KABA-Instrument (Dunckel et al., 1993) is a prime example - strives explicitly for creating humane working tasks and conditions. It therefore contrasts humans and machines to ensure that only allocations are made which promote the operators in their personal development, i.e. provide them with a broader scope of action. The second approach, followed by Clegg et al. (1989) and Clegg (1993), also aims at appropriate job design for making work tasks intrinsically motivating, but it also considers a variety of other technological, environmental, task-related and people-related characteristics. The user of their instrument is guided by these characteristics and a number of specific questions to develop and evaluate scenarios for different allocations of functions in a particular work system. The tool is basically made for moderating concrete design processes, the deduction of more general and also empirically testable criteria for evaluating human-machine systems is not intended. In both approaches, no specific guidelines for the design of the technical system are given, making it difficult for engineers to deduce practical requirements for the systems they develop. The question of whether psychological requirements concerning work tasks, like decision making and planning requirements, scope of action, task variety, or opportunities for learning could be translated into concrete demands on technical systems has to date mainly been discussed in the field of software ergonomics. As far as the allocation of functions in automated productions systems is concerned, a number of such criteria have been proposed, but their operationalization, their intercorrelations and the effects they have on the complementarity of a production system have - to our knowledge - not yet been analyzed empirically. KOMPASS attempts to improve the quality of design decisions by providing empirically validated criteria for integrated work system design as well as a moderation aid for the design process. Two additional characteristics of the KOMPASS guidelines further this aim structuring the description of productions tasks in order to make them comparable and providing the multifunctional design teams with criteria more directly addressing technical requirements. However, the criteria are not to be understood in the sense of "quick fix" leading to one straight forward design solution, but as a heuristic, which enables the design
994 team to - as early as possible in the design process, which we regard as particularly relevant to new engineering concepts - reflect and if necessary correct their decisions about distributing functions between humans and machines from an integrative work psychological point of view. Future experience with the application of the suggested criteria and design heuristic will show to what degree these aspirations can actually be met.
REFERENCES Bailey, R.W. (1989). Human performance engineering (2nd ed.). London: Prentice-Hall International. Bainbridge, L. (1982). Ironies of automation. In G. Johannsen & J.E. Rijnsdorp (Eds.), Analysis, design and evaluation of man-machine systems (pp. 129-135). Oxford: Pergamon Press. Brhle, F. & Rose, H. (1992). Technik und Erfahrung. Arbeit in hochautomatisierten Systemen. Miinchen: Campus. Clegg, C. (1993). Tool for the allocation of system tasks between humans and computers. Unpublished manuscript, MRC/ESRC Social and Applied Psychology Unit, University of Sheffield. Clegg, C., Ravden, S., Corbett, M. & Johnson, G. (1989). Allocating functions in computer integrated manufacturing: a review and a new method. Behaviour & Information Technology, 8, 175-190. Corbett, J.M. (1985). Prospective work design of a human-centred CNC lathe. Behaviour & Information Technology, 4, 201-214. Dunckel, H., Volpert, W., Zrlch, M., Kreutner, U., Pleiss, C. & Hennes, K. (1993). Leitfaden zur Kontrastiven Aufgabenanalyse und-gestaltung bei Biiro- und Verwaltungst/itigkeiten. Das KABAVerfahren. Ziirich: Verlag der Fachvereine; Stuttgart: Teubner. Emery, F.E. (1978). Characteristics of socio-technical systems. In F. Emery (Ed.), The emergence of a new paradigm (pp. 38-86). Canberra, Australian National University. Fitts, P.M. (Ed.) (1951). Human engineering for an effective air-navigation and traffic-control system. Washington, DC, NRC. Grote, G. (1994). A participatory approach to the complementary design of highly automated work systems. In G. Bradley & H.W. Hendrick (Eds.), Human factors in organizational design and management- IV(pp. 115-120). Amsterdam: Elsevier. Grote, G., Weik, S., W~ifler, T. & Zrlch, M. (in press). Criteria for the complementary allocation of functions in automated work systems and their use in simultaneous engineering projects. International Journal of Industrial Ergonomics. Hacker, W. (1986). Arbeitspsychologie. Bern: Huber. Henning, K. & Marks, S. 81986). Inhalte menschlicher Arbeit in automatisierten Anlagen. In R. Hackstein, F.-J. Heeg & F.v. Below (Eds.), Arbeitsorganisation und Neue Technologien (pp. 215244). Berlin: Springer. Jordan, N. (1963). Allocation of functions between man and machines in automated systems. Journal of Applied Psychology, 47, 161-165. Kraiss, K.-F. (1989). Autorit/its- und Aufgabenteilung Mensch-Rechner in Leitwarten. In Gottlieb Daimler- und Karl Benz-Stiftung (Hrsg.), 2. Internationales Kolloquium Leitwarten (S. 55-67). Krln: Verlag TOV Rheinland. Oesterreich, R. & Volpert, W. (1986). Task analysis for work design on the basis of action regulation theory. Ergonomics and Industrial Democracy, 7, 503-527. Price, H.E. (1985). The allocation of functions in systems. Human Factors, 27, 33-45. Susman, G.I. (1976). Autonomy at work. A sociotechnical analysis of participative management. New York: Praeger. Susman, G.I. (Ed.) (1992). Integrating design and manufacturing for competitive advantage. New York: Oxford University Press. Ulich, E. (1994). Arbeitspsychologie (3rd Ed). Ziirich: Verlag der Fachvereine; Stuttgart: Poeschel. Ulich, E. & Grote, G. (in press). Work organization. In ILO Encyclopaedia of Occupational Health and Safety, 4th Ed. Volpert, W. (1990). Welche Arbeit ist gut fiir den Menschen? Notizen zum Thema Menschenbild und Arbeitsgestaltung. In F. Frei & I. Udris (Hrsg.), Das Bild der Arbeit (pp. 23-40). Bern: Huber. W~ifler, T., Weik, S. & Grote, G. (1995), Complementary design of an automated metal sheet bending cell. An exampel of interdisciplinary cooperation beetween technicians and work psychologists. Paper to be presented at the seventh European Congress on Work and Organizational Psychology in Gy6r (Hungary), April, 19 - 22, 1995.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
995
From Taylorism to Tailorability Supporting organizations with tailorable software and object orientation Helge Kahler Institute for Computer Science III, University of Bonn Roemerstrasse 164, D-53117 Bonn, Germany Tel: (+49)0228/550-221, e-mail: [email protected]
Abstract With markets globalizing and customers" demands specializing organizations worldwide need to change. To reach the necessary flexibility of information technology one approach is tailorability, i. e. users are enabled to adjust software to their needs. Some examples for tailorability are given, and its potential benefits and shortcomings are discussed. Software development plays an important role for establishing tailorability, and object oriented methods can be helpful in this context.
1. Intro The former successful tayloristic work model that divided labour to increase efficiency is now a big obstacle for modern business with its need to react quickly to environmental changes. The conviction that there is one best way for an organization to run and that organizations work like machines which follow the linear principles of cause and result is replaced by the idea of an organization as a constantly moving social network that keeps adapting to environmantal changes (Paetau 1994). Some of he newer literature on organization introduced new concepts to overcome the rigidity of taylorism and payed tribute to a less linear understanding of organizations. The suggestions include "reengineering the corporation" (Hammer & Champy 1994) and building "virtual" (Davidow & Malone 1993), "fractal" (Wamecke 1993), or "object oriented" (Klotz 1993) organizations. All of them differ in their main focus, but all stress the importance of information technology while skipping its particular role and shape in a posttayloristic setting. That this must not be neglected in any organization we learned in a project finished recently in which we developed software ergonomical design principles for groupware (cf. (Herrmann et al. 1993)). The principles were influenced by interviews we conducted and that supported our hypothesis that the usage of networks which are usually designed for communication and co-operation also raises conflicts of interests between different users. An example for this is the ambiguity of visibility in a network of cooperating people: Whereas it can be of importance to one person to see what other people in a team work on it might be felt to be unwanted control by the latter. In a situation like this no single optimal solution will be available. Instead a common stable solution for all participants must be found. Moreover, the system should be able
996 to handle ad-hoc-negotiations about conflicts by means of a negotiation function (cf. (Wulf 1993)). In order to reach the organizational flexibility demanded by newer organizational theory as well as by the software ergonomical need to adjust systems to individuals and groups several measures and concepts are promising. One of them is tailorability.
2. Tailorability Software for a modern organization should be tailorable. This means that users can adjust it to their special needs by themselves. Ideally there will be different levels of adjustment for different needs and qualifications (cf. (Henderson & Kyng 1991)). Thus, adjusting the software might mean that a person places icons or a toolbar on the screen whereever she or he wants, that the input device can be chosen (keyboard, mouse, voice etc.), but also that people make highly sophisticated configurations in a system to support their work as a team e. g. by defining who's substituting whom at which occasion in a workflow process. Also end user programming can be used to tailor software. Relevant dimensions of the tailoring process are who the initiatiator, the actor, and the affected persons are, what its subject is (user, task, organizational context), what its goal is, or when, for how long, and for what parts of the system it is made (cf. (Haaks 1992)). Some examples of collaborative work practise can highlight different aspects of tailoring software in a collaborative work setting. • When introducing a new ISDN telphony system in an organization several configurations can be made. Some of them concern technical aspects, others may result from the organizational structure and again others relate to privacy aspects. Thus, one part of the configuration adopts the telephony software to the standards of the local telephone company while another part sets the rules for call forwarding or prevents any unauthorized person to activate the microphone of one of the telephones from a remote place. Usually these configurations affect all of the people using the telephone system, and they are rarely changed. They are made before the first use of the system. The telephone users are seldom asked to participate in the configuration process or even informed. This is a classical and rather rigid form of tailoring. • With SHARE Greenberg (1991) proposes a layered architecture for a conference tool with the possibility to use one of a set of "personizable floor control policies", i. e. rules for deciding who's turn it is next to speak/write in a computer mediated conference. Such a rule could state that every person that wants to say somthing can just "grab" the turn by interrupting the speaking person, or that the speaking person has to explicitly stop speaking before anybody else can start. Also there could be a chairperson who picks the next speaker from the group who raise their hands (shown by an icon on the screen). Other floor control policies can be defined. The decision for a special policy is made before every single session. It remains open if only the person initiating a meeting or the whole group can decide about a meeting's floor control policy. • A closer look at organizational aspects of tailoring is taken in some papers dealing with the sharing of customization files (Mackay 1990; Nardi 1993 Chapter 6; Trigg & Bedker 1994). While the software that is dealt with is not necessarily groupware, i. e. used by people to collaborate, there are still interesting observations about collaborative work
997 practice made. The main point of interest here is that people individualize software they use for their daily work and share these individualized custom files with others who feel that the files are useful for them. The customization may include some macros or lengthy blocks of standard text that different people need to write to fulfill legal or other requirements. To have others benefit from the customization files "translators" (Mackay 1990) are needed who make custom files accessible and talk to people about them, thus "translating" between the system developers and the end users. They should be domain experts as well as interested in computers and willing and able to communicate and help people. The translators often emerge from a group in a "natural" way. They can be beneficial for an organization in supporting the process of tailoring software to an organization's needs. Some positive experiences with translators ("gardeners", as Nardi (1993) calls people who are rewarded or paid for being translators) have been made. They have shown that gardeners can be a source of high quality support. Other field studies have mentioned that the process of sharing custom files has become more systemized in the course of the time (Trigg & BCdker 1994). • Malone et al. (1992) describe a system that allows end users to tailor software on a level closer to system development then just setting parameters. Their Oval System is a "radically tailorable tool for cooperative work" where "radically tailorable" means that the tool is meant to enable end users to create different applications by modifying a working system. This is done by combining and modifying objects, views, agents, and links which provide an elementary tailoring language. While, in fact, the idea of end users designing the application that suits them best is intriguing, the question remains how many users will be able to handle the more advanced features of this complex system. A very important aspect of tailoring in a work group setting has not yet been widely discussed. None of the examples mentioned involved a group of users jointly tailoring their groupware system to the group's needs. This will be of increasing importance in the future since more and more groupware systems will be installed in post-tayloristic organizations. Having a group tailor their groupware system raises questions about how this can be done. Similar to the technical (cf. (Wulf 1995)) and organizational means to deal with conflicts among groupware users while they use the system, ways have to be found to moderate different interests in a groupware tailoring process. The tailoring examples above highlight some of the potential benefits and shortcomings of tailoring software. Generally tailorable software can enhance the flexibility of an organization by enabling technical adjustments to organizational needs. Thus, for an organization tailoring can be a vital part of the management of change and the heading towards a learning organization. By actively supporting the tailoring process e. g. by gardeners spreading custom files or by work groups discussing group-relevant aspects of tailoring the self-organizational potential of work groups in a post-tayloristic organization can be set free. Also organistional support by gardeners or regular tailoring meetings can be helpful to provide structure to the tailoring process and thus keeping the organization's technical infrastructure from turning into a thicket of incompatible individual solutions or being lost in space between system planets tailored by groups. On the other hand tailoring software is not inevitably beneficial. There is the danger of a tailoring overhead since the time and effort needed for tailoring is lost - at least at a In:st glance- for the primary work task. Gardeners might be difficult to find, since they need a
998 double qualification finding themselves on the border of system development and everyday work. Also, there is a danger of getting too unfamiliar with the work settings when a gardener works for a longer time on the developer's side of the border. Moreover, the gardener system provides another layer between system developers and end users. While this is intended to improve the communication between these groups it might just increase the organization's hierarchical overhead. Organizations face the danger of running into a qualification problem not only for gardeners but for all personell. Tailoring software requires technical and social skills, particularly in a collaborative work environment, that not every organization member might have or be willing or able to aquire. Moreover, software ergonomical design principles particularly for groupware and work psychological demands are in danger of being disrespected by unskilled tailorers. This is a dilemma software ergonomy faces by pleading for tailorability to ensure that a system fits the users" needs on one side while on the other side struggling with the potential negative effects of letting the final look and functionality of a system slip out of the hands of developers and putting it into the responsibility of people tailoring within an organization. Finally, allowing for more internal dynamics as a result of continuously tailoring software might be a problem for some managers fearing to lose control. Deciding on how a system needs to be to fulfill the expectations of all or most of the people working with it at least to a certain extent doesn't get easier when the decision is made within an organization rather than by external system developers. But the chances are better to find a way of tailoring the system to support the organization and its members well by discussing the special needs and trying out what works good and what doesn't.
3. Software Development Tailorability is a feature of software. Therefore, software development is a relevant area to look at. Moreover, software development is affected by the post-tayloristic organization models directly. Developing software for a special organization must no longer be document driven and follow the rigid top-down waterfall model, i. e. consist of predefined disjunct phases with written milestones to document the state of a project before the next phase may be started. To overcome the waterfall model several approaches have been made. Two of the most promising are EOS (Evolutionary Object Oriented System Development) (Hesse & Weltz 1994) and STEPS (Software Technology for Evolutionary Participative System Development) (for an introduction cf. (Floyd et al. 1989)). Both consider software development to be evolutionary and stress the necessity to continuously coordinate the areas of development and usage since system development always includes the design of workplaces. The particular strength of EOS is the usage of object orientation (see chapter below) which might be a good technical basis to realize tailorable systems. STEPS relies on user participation where developers and users exchange their views of the system-to-be or the last revision to ensure that the software fits the users" current and future needs. Thus, part of the tailoring before the continuous usage of the software is done by system developers together with end users. This procedure for the two groups to work together in evolutionary development seems to be a good social approach to tailoring.
4. Remarks on Object Orientation There are different relations between the computers science's concept of object orientation and post-tayloristic organizations. Klotz (1993) has used the object oriented image of independently
999 operating objects communicating through clearly defined interfaces and being provided with all necessary resources to describe how a modem organization should work and called it "object oriented organization". It is yet an open question if this is not overstretching the comparison with the object oriented approach. But still object orientation can be of great value in designing and using tailorable software. Since people tend to think in objects rather than functions the interaction in a participatory design process between developers and users on what the work and the software are about is made a lot easier with the gap between their different notions narrowing. On the other hand object orientation supports an evolutionary process by allowing easy prototyping and changing of modules. Basic to object orientation are modularity by data encapsulation, inheritance of object features, and the polymorphism that frees objects that send a message from knowing the receiving object's properties. Each of them can be very helpful for tailorability, e. g.: • encapsulation ensures lean and clearly defined interfaces, so parts of the software can relatively easy be changed, removed, or added without risking a decrease of system stability; • inheritance particularly supports working in a group where features of a configuration object of the work group can be passed on to the group members" configuration objects without them having to configure each and every of their work environments manually whenever the group configuration changes; • polymorphism is needed in tailored work settings to ensure that changing one object doesn't make it necessary to change the methods of other objects. These features have shown to be valuable for prototyping and will be of use to build tailorable software since many of the requirements for prototyping and tailorability are common. Moreover, for tailorable software a layered architecture seems to be useful, where each layer is responsible for one aspect of tailorability, e. g. the user interface or building and using macros. To keep these layers independent and stable to the frequent changes resulting from tailoring object orientation can provide the methods. 5 . Extro
Tailorability can be part of the solution to the problems that organizations face in a fast changing market. On the other hand it imposes more work on an organization since it takes part of the responsibility for the adequate realization of the software from the system developers. After all, building tailorable software and working with it is a process of deregulation with the chances and dangers depending on an organization's ability to use the potential of tailorability. More work must be directed on the tailoring practice for different settings, including the development of gardening models, concepts for tailoring as a group process, and the evaluation of tailoring activities. Moreover, technical advancement towards architectures for tailorable software must be made. Object oriented concepts might be helpful here. Finally, it is crucial to understand that organizational and technical flexibility and change are intertwined and must be dealt with jointly (cf. (Rohde & Wulf 1995) for suggestions for an integrated organization and technology development).
1000 References
Davidow, W. H.; Malone, M. S.: Das virtuelle Unternehmen. Der Kunde als Koproduzent. Frankfurt am Main und New York 1993. Campus Floyd, C.; Reisin, F.-M.; Schmidt, G.: STEPS to Software Development with Users. In: Ghezzi, C.; McDermid, J. A. (eds.): ESEC'89 - 2nd European Software Engineering Conference. pp. 48- 64 Greenberg, S.: Personizable groupware: Accomodating individual roles and group differences. In: Proceedings of the Second European Conference on Computer Supported Cooperative Work 1991. pp. 17-32 Haaks, D.: AnpaBbare Informationssysteme - Auf dem Weg zu aufgaben- und benutzerorientierter Systemgestaltung und Funktionalit~it. G6ttingen und Stuttgart 1992. Verlag fiir Angewandte Psychologie Hammer, M.; Champy, J.: Reengineering the Corporation - a Manifesto for Business Revolution. New York 1994. Harper Business Henderson, A.; Kyng, M.: There's No Place Like Home: Continuing Design in Use. In: Greenbaum, J.; Kyng, M. (eds.): Design at Work. Cooperative Design of Computer Systems. Hillsdale, NJ 1991. pp. 219-240 Herrmann, T.; Wulf, V.; Hartmann, A.: Requirements for a Human-centered Design of Groupware. In: Proceedings of the 12th Interdisciplinary Workshop on Informatics and Psychology. Hesse, W.; Weltz, F.: Projektmanagement fiir evolutionfire Software-Entwicklung. In: Information Management 3/1994. pp. 20-32 Klotz, U.: Vom Taylorismus zur Objektorientierung. In: Scharfenberg, H. (ed.): Strukturwandel in Management und Organisation. Baden-Baden 1993. FBO-Vedag. pp. 161-199 Mackay, W. E." Patterns of Sharing Customizable Software. In: Proceedings of the CSCW "90. pp. 209-221 Malone, T. W.; Lai, K.-Y.; Fry, C.: Experiments with Oval: A Radically Tailorable Tool for Cooperative Work. In: CSCW "92 Proceedings. pp. 289-297 Nardi, B. M.: A Small Matter of Programming. Cambridge, Massachusetts 1993. MIT Press Paetau, M.: Configurative Technology: Adaption to Social Systems Dynamism. In: Oppermann, R. (ed.): Adaptive User Support - Ergonomic Design of Manually and Automatically Adaptable Software. Hillsdale, N.J. 1994. Lawrence Earlbaum. pp. 172207 Rohde, M.; Wulf, V.: Introducing a Telecooperative CAD-System - The concept of Integrated Organization and Technology Development. In: In: Proceedings of the International Conference on Human Computer Interaction (HCI) 1995 Trigg, R.; B¢dker, S." From Implementation to Design: Tailoring and the Emergence of Systematization in CSCW. In: Proceedings of the CSCW "94. pp. 45-54 Warnecke, H.-J.: Revolution der Unternehmenskultur. Berlin 1993. Springer Wulf, V.: Negotiability: A Metafunction to SuPport Personable Groupware. In: Proceedings of the International Conference on Human Computer Interaction (HCI) 1993, 19B. pp. 985990 Wulf, V.: Metafunctions in a Groupware: A Technical Support to Handle Conflicts among Users. In: Proceedings of the International Conference on Human Computer Interaction (ncI) 1995
VI.3 The Esprit Project 8162 QUALIT, Quality Assessment of Living with Information Technology
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
1003
Human oriented management of Change. A conceptual model Federico Butera Professor, University of Rome 'La Sapienza'; President, Istituto RSO via Boccaccio 24, Milano, Italia In the 70's and 80's the discussion on the social effects of automation brought to the conclusion that information technology has little deterministic effects on Quality of Working Life (QWL). Positive or negative effects on QWL mainly depend on the combination with the various dimensions of the socio-technical system (organization, people, processes, management rules, etc.). Improving QWL is still a great challenge in modern organizations based upon IT. Face to previous static and defensive models, Qualit give a new framework, based on the active participation of people in protecting one's own integrity, and on the dynamic contribution to the performances of the socio-technical systems, intoducing the complementary key concept of empowerment of people. Thus, people should not be just protected, but should become enabled (i.e., get the power) to actively defend and develop one's own integrity and quality of life through various ways: getting more understanding and knowledge, emotional stability, clear roles, social integration, and personal identity, in order to choose how to cope with external threats. The person should keep the control on working processes, instead of being controlled by the organization and the technology. Modern socio-technical systems should be, and sometimes are, built on open professional roles of empowered people (i.e., small firms in the firm), which have in large part the workplace within. A process of individual growth and empowerment is the objective of the Change Management Process Framework of Qualit in a context of re-engineering or continuos improvement of the socio-technical system. In network organizations empowerment means to improve communication and co-operation with remote people.
1.
PEOPLE AT THE CENTER OF DEVELOPMENT TECHNOLOGY AND ORGANIZATIONS: HOW?
OF NEW
In the last few years, in western world, governments, managers, union leaders and scholars expressed concern for bringing h u m a n being at the center of economic and technological development in the organizations: living and working conditions m u s t be preserved and improved vis-a-vis to new technologies. E m p o w e r m e n t of people is often advocated by managers, public administrators, and union leaders, stating t h a t h u m a n resources are central for the life of firms and public administrations. In the productive world as we know it, only m e a s u r a b l e variables affect m a n a g e m e n t actions: money, quality, volumes, technical breakdowns, and m a n y others. As far as Quality of Working Life (QWL) and empowerment are not easily conceptualised and measured, they are rarely included in the set p a r a m e t e r s for programs of new design and improvement: as a result they remain so far little more than a suggestive buzz word.
1004 2.
T E C H N O L O G I C A L C H A N G E A N D QWL
The Quality of working Life movement of the 70's In the 70's the International Council for Quality of Working Life (Davis and Cherns 1973), focussed on QWL among incoming great organizational changes. They considered QWL more that working conditions. It implies both objectives and measurable conditions (as cardiac pulse) and subjective and not easily measurable conditions (as stress associated with organizational charge). The whole of working life was taken into consideration. The key issues were: 1. The economic compatibility between competitiveness and improvement of QWL. Workers' pathologies and poor QWL were still affecting production quality and operating costs, limiting chances for better variance control and innovation. 2. A second issue was the understanding of the complexity of the variables affecting working conditions and the required planned processes of change. QWL approaches challenged economic and technological determinism, all "one cause-one effect" explanations. A systemic approach was adopted. New p r o c e s s e s of action research were envisaged, implying co-operation among disciplines, participation of people concerned and control over the entire process of change. 3. The definition and measurement of Quality of Working Life itself. To be mentioned the model developed by the first Conference on Quality of Working Life, describing QWL with five fundamental dimensions of individual integrity (integrity of body, of mind, of self, of professional roles, of social roles). The presented idea of joint optimization of economic, technical and h u m a n performances, inspired some innovative designs (Amhedabad, Shell, Volvo, Olivetti, US Steel, Aluminum Company of Canada, Dalmine, TRW, etc), but scarce research and poor management of changes affected the QWL movement.
Social effects of automation and IT uptake in the '80s In the '80s, IT in factories and in offices was capable of overcoming most problems associated with QWL issues and removing from men heavy, repetitive, unmeaningful tasks. IT have enabled managers, professionals and normal workers to take decisions faster and better, to better control the work processes and to communicate with each other. But IT uptake also met some problems. The literature has studied many cases of "technical success but business failures": costly automation of poorly re-engineered processes; lack of consistency with the organizational structure, inadequate working roles, insufficient training for the new process-control responsibilities; problems of implementation, etc. New QWL problems also arose with the IT introduction. IT absorbs an increasing amount of conversion activities, both of material and information, so that jobs of unskilled and semiskilled blue and white collar workers are faded away. Also for people remaining at work new problems arose with IT. Anxiety and despair were often experienced by the elder or female working population about their employment since they felt they could not be needed anymore. Symptoms of eye desease, mental fatigue and stress have been witnessed by semiskilled and skilled workers operating at IT work stations. Shift work has been widened in
1005 particular in factory automation, causing the well known problems concerning with workers' familiar life. Uncertainties about the real content of new jobs, stress coming from not getting enough training, changing p a t t e r n of tasks, fluctuation of activity needed, unclear allocation of authority and responsibility, alteration of group communication, the pressures and tensions with clients for the front office operators and so on. Even workers performing new and more complex and responsible jobs in the factory and in the office- as the "process operator" or the "computer assisted front office operator" - are still living some uncertainties in their professional and human identity and sometimes they share with the others eye strain and mental overcharge associated with Video Display Units.
IT, QWL A N D E M P O W E R M E N T IN THE 90'S: AIMS AND NATURE OF THE QUALIT P R O J E C T The attention by IT users and suppliers have drastically shifted in the 90's from IT uptake to programs for getting the best strategic use of technology. IT is not only crucial for decreasing costs, but for improving the competitiveness of operations and to enrich and diversify products and services. In this scenario, competencies, self control, creativeness, participation of people, in one word professional empowerment is required for the strategic use of IT. The speed and the width of innovation is unprecedented, the risks for people in such a radically changing environment is huge. Now everybody states that persons are central for business. For this being real, persons should be powerful enough and not heavily dependent upon the changing environment of the organizations and technology: they develop conditions for improving their own quality of working life (personal empowerment). The idea to preserve the human capital of companies is more necessary than before, but not enough. It is required to develop conditions for any person to be, when possible, actors of competitive advantage of the organizations, in innovation, continuos improvement, impact in the market, quality, cost reduction, social and technical integration, in alliance with the organization. They need to be empowered for creating their own strength to cope with uncertain tasks and environments. Technologists and IT specialists, (which the QWL projects and IT uptake programs tried to influence) are not alone roles. New designers took responsibility for development of IT: they are in most cases interfunctional, multilevel groups of line managers, specialists (strategic planning, cost control, product designers, process experts, IT specialists, organization experts, human resources specialists), workers. In most European countries it is frequent the association of Unions playing a proactive role. New programs should identify and re-engineer strong and weak connections between business objectives, technical objectives, QWL objectives, helping the engagement of people in the accomplishment of strategies of the organization and improving the global empowerment (professional and human) of real persons.
1006 As a consequence, any program for developing harmoniously IT, organization, professions and improved quality of working life and empowerment of people should not only include m e a s u r e m e n t and evaluation techniques but also methodologies of change management to help various typologies of groups of innovation and improvement to actually take place. THE QUALIT P R O J E C T
Qualit is an Esprit Project developed by a Consortium composed by Cap Gemini Innovation, Fiat, Istituto RSO, IPK (Institute for Production Design), SID (Danish Union Confederation), University of Dublin, University of Siena whose aim is to provide practical help to designers, managers, union leaders in introducing Information Technologies achieving at the same time business and technical excellence, as well as improved Quality of Working Life and empowerment of the people involved. Assumed that competitive advantage of organizations and improvement of QWL shall converge in the European new private and public organizations, assumed that new technologies and new organizations require motivated, integral and empowered people, Qualit aims to: support the empowerment of h u m a n Actors and the m a n a g e m e n t of organisational change associated with IT introduction or use - To increase productivity or quality of products of an organisation by orienting the use and/or the introduction of IT to QWL implications, people empowerment and organisational factors To disseminate and make easily available case studies, models and usable knowledge To start an autonomous action research program - To provide user-friendly tools which help to support these objectives and which give their users a common language helping them to dialogue together To incite final users to actively and positively participate to all the different phases of their project -
-
-
-
Components of QUALIT project are: A philosophy for design, implementation and continuos improvement Sociotechnical system giving the organization success, good degree of QWL and ' empowerment to people. - Conceptual device including operationalized concepts; typologies; metrics; - A model of change management fully supported with tools; - A set of tools for helping groups of people working on joint design; Focused hypothesis of virtuous and vicious correlation among elements of the socio-technical system and the Quality of Working Life dimensions; - A diagnostic tool; - Consulting guideline; - Training package; - Library of cases; -
-
1007
4.
C O N C E P T S OF T H E Q U A L I T P R O J E C T
Qualit project is based upon two interrelated set of concepts: the socio-technical system and the Quality of Working life dimensions The concept of socio-technical system is based upon the dimensions represented in fig.1. The dimensions of quality of working life identified are reported in fig 2. The quality of life model
The socio-technical system
r[_Physi callife (Body)
Processes~~. life(Mintivd)e] [ Technology ] [ Cogni
Social System
1 resources
rules
"x
[:d::~ RBotera
Organizatio
/
[ Pr°:;s~S:°na' 1
Fig. 1 The socio-technical system
fProfessional] life(Workrole)
[ liEm°ti °nalJ'~~'~ fe(Psyche) ~
iFstideti:~{~(~tera
~'~I (S°cla~f?e t )]
r Refl vlife e t~~ (Selef-ictidenti y) '~"
[
Fig. 2 The Quality of life model
Empowerment should be defined as the process through which an individual or a group of individuals improve his or her ° individual and social power to recognise, protect, develop his or her integrity of body, mind, psyche, profession, social integration, self, and • the ability to act individually and in co-operation with others in order to control work processes, to positively influence the structures and to improve the performances of a socio-technical system, due to his or her joint physical sanity and strength, level of understanding and competence, emotional stability, professional mastering, social integration, self confidence • the "being a person able and willingly to properly use the technological tools: it is associated to the degree of the person's maturity, hopefully supported by the inner ability to develop the integrity of body, mind, psyche, profession, social integration, and also to cope with eventual limitations in some of the mentioned sphere.
Real empowerment (not the fashionable empty buzzword frequently used in the managerial jargon)becomes the focus key of t h e Qualit Project because improvement of performances of socio-technical systems and the Quality of Working Life could and should be positively affected by the visibility, degree and the speed of processes of empowerment of people and also because empowerment is a way for individuals and groups to partially self-manage his or her own Quality of Life and to become a person making good use of the IT power.
1008 The person using IT has never been so powerful, but on one side not all people is able to manage such power and not all people have real access to IT. ITechnological power should be regulated by strong social and ethical values and norms, to be put into practice by mature, empowered persons. How to describe, measure the mentioned phenomena? And moreover how to develop socio-technical systems and programs for improving Quality of Working Life and developing the empowerment of persons? How to develop programs of improvements of QWL and empowerment which could beneficially affect from technical and economic standpoint the structure and the functioning of the SocioTechnical- System? It seems impossible so far to have a predictive model able to foresee correctly the impact of new technology projects, both upon new socio-technical design or upon Quality of Working Life and even less upon the empowerment processes. But designing and making improvement taking into account QWL dimensions or trying to improve QWL and empowerment is still a great challenge and it can be done. We indicate here two ways for providing technology designers and sociotechnical practitioners tools for predicting positive results for QWL and empowerment.
1. THE WORK ECOLOGY APPROACH The mentioned QWL dimensions are influenced by a variety of set of factors from the socio-technical system. We call these set of factors "work ecology" that is the relationship between the individual at work and its environment. IT is linked with economics, organization and management criteria in the
"strategic ecology"of work. Technology has direct influences upon physical and operational universes of work. Integrity of body and integrity of mind are directly affected by screens, keyboards, response times, software, etc. This is the technical ecology of work. Technology has a range of indirect influences.. Since IT affects the procedures, decision making and communications,professional integrity is affected by what we call organizational ecology of work. QWL is also affected by institutional protection, social cohesion, values, etc. We call it social ecology of work. It does not determine QWL but it is a contextual variable responsible for worsening or improving it. The intellectual, emotional and moral ability of each individual to cope with challanging living and working conditions is an important factor which has an impact on risk and can be termed the individual ecology of work. Except the integrity of body, all the dimensions could be protected or developed on the basis of the degree of empowerment of the individual. In short Qualit users can find: a) Requirements for the different ecologies of work in order to develop QWL and empowerment: ergonomic requirements for technology, organizational requirements, human resources rules for integrity of social life. b) Recommendations and good solutions for technology, organization, and human resources program for QWL.
1009 Requirements and recommendations are provided when architectural or detailed design should be evaluated or performed (see below). 2. THE C H A N G E M A N A G E M E N T F R A M E W O R K A P P R O A C H
The change management model includes the previous model, providing tools to support groups of experts in factories and offices in managing the change process oriented to improve the QWL and the empowerment of people. Three are the components of such an approach: a) Identification and application of solutions of STS associated with the development of QWL and people empowerment. E.g. Network socio-technical systems; Self regulated groups; Professional roles of knowledge workers and "process roles" of white and blue collar workers; IT supporting professions and communication; b) A stepwise recoursive managed process of change oriented to QWL and empowerment. It jointly affects processes, technology, organization and people empowerment which may help in joint design of factories, offices, services etc. (new design, restructuring, re-engineering) and in continuos improvement. Company plans, design and improvement should not be sequential top-down events, but it is possible to start anywhere: pilot design, programs of continuos improvement which may accumulate enough learning in the organization and in people to refurnish higher level company-wide changes. The main steps of Qualit change management process are: - Goal and problem setting and diagnosis: why introducing IT, or designing or improving STS. Expected results; what should be improved; weak and strong points; - Organization of the project: who, when and how will do what. - Examinations of options, Architectural design and Feasibility analysis: consideration of the existing stock of solutions (organization, roles, technology, staff rules, training programs etc.); principles, philosophy, architectural schetch (technology, organization, roles, etc.).; how to do what? Evaluation - Training and empowerment: in parallel since the previous step - Detailed design, Implementation and diffusion: engineering the project c) Within most of the mentioned steps the Qualit project wants to provide: - a tool to analyse context and scenarios of changing process; - a change management guide giving vision of the process itself; - a tool for goal setting and organizational diagnosis, focusing specific problems and objectives for change of the unit; it includes preliminary QWL diagnosis, focusing empowerment opportunities; - information o- :lifferent "target" solutions, including opportunities offered by different options; - the access to a library of some significant structured case studies; - a tool to communicate with one another among the actors of the change; - a tool for assessing the feasibility of different targeted solutions; - a tool for QWL assessment of the considered targeted solutions; - tools for final evaluation and improvement, oriented to a new goal setting.
1010
LARGE SCALE APPLICATION OF QUALIT. Today, the challenge is to design - for general purpose and in specific sociotechnical settings - better man-machine interfaces, better software, better compositions of different tasks in more integral work roles, more supportive organization, more appropriate staff rules, an adequate education, a developmental social system and work culture. The phase of technological design is very important. The joint engineering of information technology and work should be considered a promising area of collaboration among technologists, ergonomist and cognitive psychologists. But it should also be considered as a preparatory and complementary area to a wider approach: the joint design of technology, organization and empowerment, where collaboration is required among different actors. Strategies and action plans for improving the multiple parametres of QWL in the techno-economic revolution should concern societal, organizational and technological affecting factors. Programs and projects of experimental approach in a single setting oriented to organizational, technological and human change - which adopt a strategic, dynamic and systemic approach and benefit from participation of people concerned - are here suggested. It is what we call: "Joint design of technology,
organization and empowerment (people growth)" (RSO, 1988). REFERENCES 1 2
S. Bagnara, L'attenzione, Bologna, I1 Mulino, 1984 F.Butera and J. Thurman, eds Automation and work design, New York, North Holland, 1984 3 F. Butera "Options for the future of work" in European Foundation for Living and Working Conditions Options for the future, London Cogan Page, 1989 4 L.E Davis., A. Cherns, The quality of Working Life, New York, Free Press, 1973 5 Y. Delamotte, Quality of working life in international perspective, Geneva International Labour Office, 1984. 6 G. De Michelis, F. De Cindio, C. Simone, "Dimensioni dell'usabilitY, universi del discorso e software d'ufficio" in Butera F. Dalle occupazioni industriali alle nuove professioni, Milano F. Angeli, 1987 7 L. Hirshorn, The workplace within, Cambridge Mass, The MIT Press, 1988 8 J. Rasmussen, Information processing and huma-machine interaction,New York, North Holland, 1986 9 T. Sheridan, "45 years of man-machine interaction", RSO International Conference The joint design of technology, organization and people growth, cit 10 RSO Institute, Proceeding International Conference The joint design of technology, organization and people growth, Venezia, Ottobre 1988 11 E. Trist, H . Murray (eds) The social engagement of social sciences: a Tavistock anthology , Philadelfia, The University of Pennsilvanya Press, 1990 12 A. Wisner, "Organizational stress, cognitive load and mental suffering" in Salvendy G. and Smith J.M Machine pacing and occupational stress, London Taylor and Francis, 1981
Symbiosisof Human and Artifact Y. Anzai, K. Ogawaand H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
1011
The Quality of Working Life Concept S. Downing*, G. Ryan*, A. McNeive*, M. Mariani**, O. Parlangeli** *QUALIT (Ireland), Department of Computer Science, University College Dublin, Belfield, Dublin 4, Ireland. **Multimedia Communications Laboratory, University of Siena, via del Giglio 14, 53100 Siena, Italy. ***
1. I N T R O D U C T I O N Interest in the nature of work organisation and its possible effects on employee motivation has caused many organisations to take steps to ensure that job design incorporates the intrinsic needs of employees. This can be seen manifested in the emergence of the Quality of Working Life Movement which is aimed at eliminating many of the problems associated with traditional work systems, making work more meaningful for employees and ensuring positive benefits for employers.
2. THE E M E R G E N C E OF THE QUALITY OF WORKING LIFE MOVEMENT Quality of Working Life has been defined as "the degree to which members of the work organisation are able to satisfy important personal needs through their experiences in the organisation." (Lloyd Suttle, 1977, p 4). Traditions from American Work Psychology have made significant contributions to the emergence of the Quality of Working Life movement. Taylor's principles of scientific management have had profound effects on work design with its emphasis on the division of labour into the smallest possible components which are then allocated to workers. However, Davis and Taylor (1972) conclude that we have been unable to design jobs to relate man with his needs and unique capabilities, to technology. This has led to the view that Taylorism and its approach to work design should be replaced by the 'humanization' of work. This has become a significant topic of concern for the Quality of Working Life movement. The concept of alienation, particularly among 'blue-collar' workers is another significant theme in the emergence of Quality of Working Life. The debate on this topic in the 1960's prompted those interested in Quality of Working Life to **"This paperwas prepared as part of the activitiesof the ESPRITProject 8162: QUALIT,QualityAssessment of Livingwith InformationTechnology.The supportof the EuropeanUnion's ESPRITProgrammeis gratefully acknowledged.
1012 seek means to overcome alienation primarily through the development and application of alternative strategies for work design and worker empowerment. The issue of deskilling has been a recurring theme within the Quality of Working Life movement and continues to be so with the increasing sophistication of Information Technologies. According to Braverman (1974) science and technology have deepened this trend towards deskilling and contributed to the Taylorist fragmentation of work. Finally, it has become increasingly evident that physical and mental health is adversely affected by a poor Quality of Working Life. (Gardell, 1971; Judge and Watanabe, 1993)
3. QUALITY OF WORKING LIFE, INFORMATION TECHNOLOGY AND THE QUALIT P R O J E C T The early optimism which saw Information Technology as providing solutions to many of the Quality of Working Life problems identified in previous years has not been justified (Butera, 1989). Many problems have remained. Advanced Information Technology has brought with it new Quality of Working Life problems such as concerns about long term employment opportunities, potential health hazards, and stress issues. Concern on the part of workers in the increasing use of these advanced technologies has led to demands for equal treatment, greater job satisfaction, and democracy in recent years (ILO, 1985). This has provided a new impetus to the Quality of Working Life movement to find solutions to many of these problems. The need for solutions has been recognised by the European Union with Quality of Working Life issues reflected in the Workprogrammes of ESPRIT (European Programme for Research and Development in Information Technology) and The European Foundation for the Improvement of Living and Working Conditions. Additionally, new technologies are also seen as essential in achieving both improved competitiveness and quality of life (CEC, 1989). It is in this context that ESPRIT Project 8162: QUALIT recognises, firstly, that new technology and new organisations require motivated and empowered people and secondly, improvement of Quality of Working Life is a key issue for the competitive advantage of European organisations. Therefore the current QUALIT Project aims to" • Support the empowerment of human actors and the management of organisational change associated with Information Technology introduction or use • Increase productivity or quality of products of an organisation by orienting the use and/or introduction of Information Technology to Quality of Working Life implications, people empowerment and organisational factors • Disseminate and make easily available case studies, models and usable knowledge • To start an autonomous action research program
1013 • Provide user friendly tools which help to support these objectives and which give their users a common language aiding dialogue
4. QUALIT AND T H E SOCIO-TECHNICAL SYSTEMS A P P R O A C H The QUALIT Project is based on two interrelated sets of concepts: the Sociotechnical Systems Approach and the Quality of Working Life Dimensions. The Sociotechnical Systems approach to organisation design integrates the technical and social subsystems into a single management system. Trist (1948) illustrated that behaviour was influenced by the context in which it was observed. Technology was one of the strongest elements affecting behaviours at work. Understanding of the effects of different socio-technical systems arrangements resulted from studies at the Glacier Metal Company and in the British coal-mining industry (Trist 1948). Differences in productivity, safety, absenteeism and morale resulted from new organisational design and the fact that multi-skilled miners were now working closely in teams. Later Emery's (1948) research looked at the level of the individual job in a socio-technical system. He suggests that for every individual there should be an optimal level of variety, learning opportunities, scope for decision setting that affect the outcome of the work, organisational support, a job worthy of societal recognition, and the potential for a desired future. He argues that because organisations employ whole persons, it is important to pay attention to human needs beyond those required for the regular performance of tasks dictated by the technology. He emphasises that self-management will become encapsulated in few organisations unless the education system itself is transformed to provide learners with greater control over their own experience of learning.
5. P R O B L E M S IN WORKING LIFE
THE
CONCEPTUALISATION
OF
QUALITY
OF
Quality of Working Life refers to a wide range of concerns and projects. Quality of Working Life has been variously defined as a variable, an approach, a set of methods, a movement (Nadler and Lawlor, 1983). For this reason there is no well-accepted or well developed definition of the term. A broad view of Quality of Working Life sees it as encompassing all aspects of work-related life which could possibly be relevant to worker satisfaction and motivation (Delamotte and Takezawa, 1984), while a narrow view of Quality of Working Life highlights workers need for meaningful and satisfying work and for participation in decision-making. Quality of Working Life lacks a clearly articulated and singular theoretical foundation. Recent research has acknowledged that four main issues must remain the focus of attention. They include; firstly, management and employees co-operating in the design and implementation of the programme, secondly, action plans developed must follow through to completion, thirdly,
1014 middle management must be supported from pressure from both top management and line workers to implement plans, and finally, the focus must be on joint objectives of increasing the Quality of Working Life and maintaining organisation efficiency; there must be no incompatibility of interests (Nadler, 1989). The European Commissions Esprit project QUALIT has adopted an innovative approach to issues relating to Quality of Working Life and Information Technology. The scope of QUALIT is extensive integrating both a management and trade union perspective. QUALIT has integrated within a Change Management Process Framework, Information Technology, Quality of Working Life and the socio-technical models. Recent experiences in organisations with New Information Technology have suggested the need for a change in focus towards; attention to human computer interactions, thinking of Information Technologies as patterns of social relations, to understanding the importance of organisational politics and to a concern for added value and organisational benefits (Blackler 1988). The remainder of this paper will focus on the Quality of Working Life Dimensions and the efforts of the QUALIT consortium to develop a model of Quality of Working Life. This model is based on earlier work carried out in ESPRIT Project 5374: QLIS. The basic concept underlying IRSO's model of the relationship between Information Technology and the Quality of Working Life is that there is no deterministic relationship between an increased level of automation and the enhancement or degradation of the Quality of Working Life of individuals involved. The model indicates that technology is just one of a set of several inter-related variables, all influencing the Quality of Working Life of the individual. The actual Quality of Working Life associated with a particular professional role or work process is best understood as the result of the complex interactions among its different dimensions. These dimensions are themselves influenced by a variety of independent influencing factors Butera (1990). The objective is to promote the utilisation of all their potentialities, thereby promoting personality development and contributing to organisational effectiveness and competitiveness. This approach incorporates 'Action Regulation Theory' its key concept is that challenging, societally useful, goal oriented activity plays an irreplaceable role in the development and preservation of mental processes and, thus of physical and mental health. The dimensions of the Quality of Working Life Descriptive Model have six key elements Butera (1990). They include; physical life which is a set of standards studied by the classic ergonomic discipline, cognitive life includes the structures, mechanisms and processes involved in the intake, elaboration and control of information; the integrity of 'cognitive life' is measured in terms of 'boarding conditions of dimensions like mental workload, mental fatigue, level of cognitive control. Emotional Life is considered as the meeting point between body and psyche. The integrity of 'emotional life' is measured in terms of 'border conditions' of dimensions like anxiety/insomnia, depression, psychosomatic symptoms, social malfunctions. Professional life is defined as the way to perform
1015 a role and is evaluated in terms of quality, harmony between the role and the social, organisational and technical context, broader conditions and opportunities for growth. Social life can be defined as the set of formal and informal relationships linking people belonging to the social community. Finally, Reflective life represents the relationships between our organism and our history. The integrity of 'identity' is measured in terms of social interaction, self esteem, internal/external locus of control. (Ryan and Bjorn-Andersen, 1992). The QUALIT project integrates the concept of empowerment into its framework. This involves the active participation of each individual in the organisation. Empowerment, Quality of Working Life, and stress are closely associated themes. Stress is the consequence of poor Quality of Working Life conditions and empowerment acts as a set of guidelines to improve Quality of Working Life and reduce the risk of stress occurrence. Creating the correct organisational environment to foster participation has become an important concern for organisations. Research indicates that the correct environment is one which involves the entire organisation. The issue for senior management is how to create this corporate culture within which employees are highly committed to working towards the achievement of business goals.
6. CONCLUSION: The Quality of Working Life Movement today is challenged by problems associated with the advances in Information Technology. Both employers and employees face these issues which include demands for equality, job satisfaction and increased participation. The current European Union QUALIT project aims to improve the Quality of Working Life by providing work environments in which employees can satisfy individual needs.
REFERENCES 1. Lloyd Suttle, 'Improving Life at Work Problems and Perspectives' in Richard Hachmans and J. Lloyd Suttles Improving Life at Work: Behavioural Science Approaches to Organisational Change, Santa Monica, Calif, Goodyear (1977) 2. Davis and J. C. Taylor, Design of Jobs, Cox & Wyman Ltd., London (1972). 3. Braverman, Labor and Monopoly Capital, Monthly Review Press (1974). 4. Gardell, Alienation and Mental Health in the Modern Industrial Environment, In: L. Levi (ed.) Society, Stress and Disease, Vol 1, The Psychosocial Environment and Psychosomatic Diseases, Oxford University Press, Oxford, (1971), T.A. Judge and S. Watanabe, Another Look at the Job Satisfaction-Life Satisfaction Relationship, Journal of Applied Psychology, 78 (1993) 939-948.
1015 5. Butera, Information Technology in the Office and the Quality of Working Life, (1989). 6. International Labour Office (ILO), World Labour Report 2, Geneva, (1985). 7. Commission of the European Communities, A Framework for Community R&D Actions in the '90's, Communication from the CEC, Sec (89) 675 final, Brussels, 13 June 1989. 8. Trist & H. Murray, Work organisation at the Coal Face: a Comparative Study of Mining Systems, London, Tavistock Institute of Human Relations, Document 506 (1948) 9. E Emery, E. L Trust, Socio-Technical Systems, in C. W Churchman, M. Verlhust, Management Science: Models and Techniques, London, Pergamon Press (1948) 10. Nadler and Lawlor, Quality of Work Life: Perspectives and Directions in Organisational Dynamics, Winter, (1983) 11. Delamotte and S.I. Takezawa, Quality of Working Life in International Perspective, International Labour Office, Geneva, (1984). 12. Nadler & M. Tushman, Organisational frame bending: Principles for managing reorientation. Academy of Management Executive, 3 (3), 194-204 (1989). 13. Blackler, 'Information technologies and organisations: Lessons from the 1980s and issues for the 1990s', Journal of Occupational Psychology, Vol 61 (1988). 14. Butera, Options for the future of work in: F. Butera, V. Di Martino and E. Kohler (eds) Technological Development and the Improvement of Living and Working Conditions: Options for the Future, Luxembourg: Office for Official Publications of the European Communities and Kogan Page, London (1990). 15. Ryan and N. Bjorn-Andersen, ESPRIT Project 5374: QLIS Final Report, Copenhagen Business School, (1992).
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
1017
User Requirements for Tools to Support H u m a n Oriented M a n a g e m e n t of Change Irene Odgaard General Workers' Union in Denmark Nyropsgade 30 DK- 1790 Copenhagen, Denmark Fax: (+45) 3397 1460 Phone: (+45) 3314 2140 Email: iodgaard@ inet.uni-c.dk The General Workers' Union in Denmark (SiD) is actively involved in the QUALIT project under the European Commission' s ESPRIT research and development program (project number 8162). The active involvement of the General Workers' Union in the QUALIT project amounts to the first instance where the particular perspectives and experiences of a trade union is directly represented in an ESPRIT project. QUALIT addresses central issues in the emerging business environment of modem manufacturing which is characterized by a new regime of market demands where factors such as the ability to adapt to changes, handle shorter through-put times, etc. become increasingly dominant. Manufacturing enterprises attempt to handle this challenge by reducing the number of levels in the organizational hierarchy, by changing the management style towards 'management by objectives', and by enhancing the responsibility and competence of operators in the executive functions of the organization. This general trend makes the motivation of employees a crucial factor. Accordingly, the dialogue between management and the involved employees concerning job design and the technical and other support provided for coping with the new tasks and responsibilities becomes imperative. In this context, QUALIT aims to support decision makers faced with issues of quality of working life, managers and trade unionists alike, in optimizing the design and implementation of IT systems in situations where the quality of working life could become severely compromised or where an improvement of the quality of working life is perceived as a precondition for a improvement in the competitive position of the company. The objectives of the QUALIT project are, inter alia, to develop software scenarios and other tools for supporting the necessary dialogue concerning the enhancement of the quality of working life and, hence, the crucial motivational factors in the transformation process. The General Workers' Union in Denmark (SiD) represents approximately 300.000 unskilled and semiskilled workers in different branches of the industrial sector, in building and construc-
1018 tion, transportation, and agriculture and forestry. It is the second largest union in Denmark, where the rate of unionization is very high - - 86%. The users of the QUALIT tools in SiD will be shop stewards in companies engaged in change processes. To be engaged in efforts of restructuring jobs and work organization in the process of technological development is a new experience for unions - - and for many companies as well. We have experienced many failures, and the most important ones perhaps because of the tendency to attach rigid models for the improvement of the quality of working life to reality inside the very different production systems. When the rigid models failed, the emphasis shifted to the process: if one could get all actors to articulate their needs, the 'correct' solution could be found... Again, failures arose due to lack of knowledge - - the participants' knowledge of economic, technical, and social parameters and the relation of these parameters to Quality of Working Life issues. At the threshold of the Nineties, we had 'grand visions' and a very positive attitude towards collaboration on organizational change in the companies but we very much needed operational guidelines for participation in practice, a 'common language' in the companies. In determining the requirements of the SiD users of the QUALIT Environment we are faced with the following specific problems: 1. The potential users may not, at present, be involved in QWL assessment to such a degree and extent that it is possible to analyze their practices in this regard. 2. The conceptual framework and methods of their QWL assessment activities cannot be assumed to be adequately developed. To the contrary, we have in setting-up the QUALIT project assumed that one of the major contributions of the project would be to give actors involved in radical socio-technical change a sophisticated conceptual foundation for their future involvement in improving their quality of working life. Accordingly, in determining the requirements of shop stewards we cannot proceed in the manner of conventional requirements engineering (developing conceptualizations of pertinent aspects of routine activities in the organizational settings). In stead, we had to try to identify the categories of quality of working life issues, the shop stewards are faced with. Thus, the task before us were: • to identify a set of prototypical QWL issues that covers the experience horizon of the potential users; • to relate the prototypical QWL issues to the categories of the underlying QWL model so that the question generation of the decision support system in QUALIT can relate to the experienced problems of the actors; • to relate these issues to the collection of cases so that cases (or aspects of cases) that are relevant to visualize certain prototypical QWL issues can be identified and, preferably, customized in the form of scenarios for the training package for the particular combination of QWL issues at hand. At the first workshop between shop stewards and the design team of the QUALIT project the following problems were presented: • Shop stewards are unable to visualize and present for management certain consequences of technological (technical) development, such as social isolation, which are unwanted for
1019 both employees and management. This kind of knowledge will often be easier accessible for workers' representatives than for managers. • Management (and employees) lack tools to give 'hints': recommendations and warnings regarding QWL issues in connection with changes in the organization. This is a serious problem, when QWL deteriorates in connection with "continuous improvement" projects, where motivation of the employees is crucial. ° Shop stewards find it difficult to be able to explain objectives and goals for projects in terms of concrete consequences for ordinary employees - - this makes it difficult to involve them. At a later workshop, presenting the framework for change management in the QUALIT conceptual model, shop stewards expressed the need for tools enabling them to: • select different procedures according to whether guarding of interests is involved or not; ° know at which points in time the different types of employees should be directly involved, creating a hierarchy of user specifications.
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
1021
N e w Forms of E m p o w e r m e n t using simulation games and leaming form cases K. Mertins, B. Schallock and P. Heisig Department of Systems Planning, Fraunhofer Institute for Production Systems and Design Technology, Pascalstrasse 8-9, D - 10587 Berlin, Germany
Abstract The employees should actively participate in the process of planning and optimization of sociotechnical systems and bring in their tacit knowledge in order to increase the efficiency of the sociotechnical systems. They should be able to take into consideration the quality of their working and living conditions and to contribute in their improvement. These abilities to plan and the knowledge of quality of working life circumstances cannot be presupposed but have to be mediated to the employees. Therefore, there is a need of training methods which are adapted to adults and support an active learning process. On the other side, these methods have to focus on the process of organizational change, introduction of new technologies and quality of working life standards. The PadeS ©tutorial for production design simulation represents an appropriate procedure for that change process. This tutorial links elements of a role game, a planning game and a case study at the example of an organizational restructuring process in a medium-sized metal manufacturing company. The ESPRIT project 8162 QUALIT ("Quality Assessment of Living with Information Technology"), which is partly funded by the European Union, offers the opportunity to enlarge the concept through a case study library and special knowledge of the design of quality of working life.
1.
INTRODUCTION
The permanent change of the market environment and the shortening of product life cycles requires not only new innovative products but also a permanent organizational adjustment of the companies [ 1]. These time constraints and the evolving environment need an integrated planning of organizational structures, technology and human resources. This integrated planning increases the complexity of organizational change processes. To shorten the planning times and efforts, the involvement of skilled employees from different hierarchical levels with their practical experience is more and more required. Their involvement could reduce the complexity of the planning, because specialized planning staff could concentrate on high level design while the involved workers could take over some detailed planning tasks [2]. This involvement increases at the same time the complexity of the management of the planning process. Therefore methodologies for the management of the change are required. With the PadeS ©methodology for Participative Design of decentralized Structures companies are offered various methods and tools to manage organizational change processes mainly associated with the introduction of new technologies [3].
1022 The competitive challenge of European industry could not only be overcome by the introduction of the most advanced technologies but also has to consider the human resources throughout the companies. The Quality of working Life of these human actors within the companies has to be enhanced with the increase of organizational effectiveness. Furthermore, new production technology which is strongly linked with information technology will only reach high effectiveness when people will actively make use of them instead of passively follow technical instructions [4]. The enhancement of productivity and Quality of working Life can be combined, when empowered employees will be actively involved in organizational change processes and the conduction of continuous improvement of the implemented sociotechnical system.
2.
EMPOWERMENT OF HUMAN ACTORS AND THE CHANGE PROCESS
The empowerment of human actors is understood as "the ability to act individually and in co-operation with others in order to control work processes, to positively influence the structures and to improve the performances of a socio-technical N Organization People system" [5] while considering Continuous the well-being of the individImprovement Processes ual. A key concept to empowTechnology ,~ Training and erment is an active participa_ / Implementat!on tion in a change process of organization, technology, work I selectionanddesignI processes and qualification [6]. I Socio-technical The PadeS©-methodology for specifications I ' the management of organizaI Representation tional change intends to inteandassessment I ~ grate and combine various Development J of alternatives I methods from different disciplines like engineering, I Currentstate analysis I psychology and business I Problem and goal administration to enhance determination I effective and participatory planning processes (Figure 1). training I To enhance this participatory Project-group planning methodology, IPK is establishment I ~_~"-" developing a new-form organizational design simulation game to train employees and managers in process competence and to empower them to be actively involved in organ- Figure 1. The PadeS©-methodology for management of izational change and continuous change with the method-toolbox. improvement taking into account their Quality of working Life [7].
1023 A traditional teaching that covers a presentation of facts can not serve the aim of empowerment. Learning by doing is the most appropriate form to acquire new skills. The simulation game was made to provide such an opportunity.
THE P a d e S ©- SIMULATION GAME FOR ORGANIZATIONAL CHANGE PROCESSES The simulation game helps to reach awareness of the complexity of the change process, enables to understand and use different tools for integrated planning. The objective of the PadeS©-simulation game is to mediate methodological, social and technical competence. PadeS ©is conducted by groups of planers, middle managers, workers' representatives and skilled workers from the affected areas. Normally, it is carried out in a five-day block seminar or in three weekend-seminars. It is based on a model of a representative medium-sized metal manufacturing company with 250 employees. After every step that was performed by the players, one or several sample results are available for the next step if the result was not fully satisfying. Within the first phase of orientation, the participants get familiar with the basic data of the model enterprise and with information specifically related to their function. The role game in this phase is intended to analyze the problems of the factory in a brainstorming session and a following discussion in the group. The use of this step will be an improvement of a global approach of the participants that takes into consideration all existing departments and functions. Furthermore it develops a result-oriented understanding of productional sequences and technical interrelations, adequate communication and information behaviors and the ability to fulfill other roles. The results of the first phase will be completed with the mediation of basic knowledge about communication (Von Thun's model of the four aspects of a message [8]) and applied in the next phase of goal definition. The working out of a system of goals requires, besides the abilities of communication and cooperation, a balancing of the various interests. A great problem is the influence of. the partial goals. The goal-system-editor supports and shortens the decision process with computer aid and thus provides a better data base for decision making. The second phase, the planning phase, will start by working out the goal definition and specification within the overall company strategy by the involved persons. A consensus about the project goals should be achieved among the involved persons. This consensus shall be reached through work groups which analyze and present the work organization, the material and information flow with special methods. Material flows are measured and analyzed by using the simulation system MOSYS [9]. The investigation of information flows is supported by the MOOGO tool using the method of integrated enterprise modeling (IEM) [10]. The following plenary session will join the partial results and result in an overall presentation of the model company. Based on these outcomes and on a presentation of selected case studies, alternative organizational configurations have to be specified and presented for participative assessment by the project team. The use of simulation games makes it possible to work out the effects of these alternative configurations very vividly. This phase ends with a presenta-
1024 tion of the results to the whole planning group. Participants have thus used various analytical methods and presentation techniques. Within the following phase, the specified alternatives have to be evaluated in small work groups. A preliminary step is the presentation of advantages/disadvantages of traditional evaluation methods. Afterwards, a new method is presented that balances objectives of the company (e.g. cost reduction) with the interests of employees (e.g. autonomy). Participants get to know the procedure of decision making in a team and the difficult evaluation of alternative solutions. The summary of all results provides the base for the definitive selection of the solution to implement. The chosen concept offers several implementation possibilities and ways of participation of employees of the involved departments. To exploit the tacit knowledge of employees, a sociotechnical specification is worked out in small groups after the presentation of a case study from another company (see Chapter 3), in order to identify organizational, technical and qualification requirements.
A-
centralcontrol
B:
or
or MRP ~k'~/~'~" t~
store
listings ~ ~ i foreman push ~~i)
C" decentralizedstructure
areacontrol
MRP ~/~k ~t ,~/~istore
MRP
assembly
•
i-
assembly worker
foreman ~ ~
push store
pull
~ ---~ m'--~
veryof parts
assembly
worker
~unctual deli-
Arc,tectura,°es,0n [ Feasibilityanalysis i Methods Toolbox
QUALIT Change Management Process Framework
QUALIT Case Study Library cow, i ~ r h a - PK e,,r,, 1 ~
Figure 2. Work out of design alternatives and inspiration from the case study library
1025 In the last phase, the participants set up an implementation plan and a qualification program for employees. This again is done in small work groups. Based on the goal system (second phase), criteria for the evaluation of the implemented solution and of the planning process itself have to be determined. A crucial step in the re-engineering or design of companies as well as in the design game, is to work out of organizational alternatives. This step, specially in real life situation, can much be eased by looking at other real examples. This is possible by looking at a case study library (Figure 2).
3.
THE CASE STUDY LIBRARY
The establishment of a Case Study Library is one of the features of the above mentioned ESPRIT project 8162 QUALIT. Three aspects are essential for a user of the Case Study Library: • sufficient, structured and detailed information about the case • advice for a sequence of actions • easy access to the information needed. The Case Study Library incorporates for every single case information about the goals of the described change process, the organization of the project, the options that had been available, the procedures and criteria for selection of different options, the technology being selected and introduced. The results of the restructuring in order to economic objectives and the Quality of working life are also shown. Where appropriate, the used methods and tools are indicated, too. Graphs and value tables will enhance the rapid understanding of the case. The case study presentations are arranged following the design steps of the Change Management Process Framework (CMPF) developed by the QUALIT consortium. When the user has been guided through the standardized sequence of steps in several cases, he can use this advice to follow these steps in his own reengineering project. He can also switch to the Change Management Process Framework, which is another tool of the QUALIT Environment, and learn the change steps in detail. The easy access to the specifically required detailed information is an important feature of the CSL. The user can indicate his problem and/or area of restructuring. He can start his inquiry for case selection or detail his inquiry by indicating branch, size of the company and other characteristics. It is intended to extend the Case Study Library and offer it as a commercial service Europe-wide.
INVOLVEMENT OF EMPLOYEES AND ENHANCED KNOWLEDGE ABOUT QUALITY OF WORKING LIFE The active involvement of employees in change projects and continuous improvement processes requires also an enhanced knowledge about their implications on quality of working life. This knowledge will be mediated to the participants using the Quality of Working Life Model within the Change Management Process Framework of the QUALIT project. The relations between Quality of working life dimensions and the change process will be shown for each step. Methods for diagnosis and analysis of the current quality of working life
1026 situation will be indicated and used in the simulation game. The consequences of each design option on the Quality of working life are presented. Thus, these implications could be taken into account within the evaluation and selection of the design solution. 5.
CONCLUSIONS
The QUALIT Environment consists of different tools and methods for re-engineering and for improving the Quality of working life, like the QUALIT Change Management Process Framework and diagnosis tools. The simulation game and the QUAL1T Case Study Library, sketched in this article, are only two elements of the QUALIT Environment. They foster the empowerment of people for increased ability of self directed working life, being essential for work satisfaction, effectiveness and innovation. REFERENCES
1.
2. 3.
4.
5.
6.
7.
8. 9.
10.
K. Mertins, R. Albrecht, H. Edeler: Manufacturing Philosophy for the New European Factory. In: I.A. Pappas, I.P. Tatsiopoulos (Ed.), Advances in Production Management Systems (B-13) Amsterdam, etc., Elsevier, 1993, pp. 31-38. Liedtke, U. Roessiger, G. Spur, R. Albrecht and P. Heisig: Gestaltung ganzheitlicher Arbeitsabl~iufe. ZwF 90 (1995) 3, (forthcoming). B. Schallock, Skill enhancing shop floor structures, in: P. Brfdner and W. Karwowski (ed.), Ergonomics of Hybrid Automated Systems III, Amsterdam, etc., Elsevier, 1992, pp. 169-176. T.J. Smith, R.A. Henning, K.U. Smith: Performance of Hybrid Automated Systems - A Social Cybernetic Analysis. In: The International Journal of Human Factors in Manufacturing, Vol. 5 (1) pp. 29-51 (1995). F. Butera: Quality of working life criteria and empowerment in re-engineering and continuous improvement of network organizations supported by information technology. Esprit 8162 QUALIT, Technical working paper (public), Milano 1995, 22 p. R. Greifenstein, P. Jansen, L. KiBler: Partizipationskompetenz und technisch-organisatorische Innovation - Ergebnisse dreier Fallstudien. In: L. KiBler (Hg.)" Partizipation und Kompetenz. Beitr~ige aus der empirischen Forschung. Opladen: Westdeutscher Verlag 1990, pp. 15-54. P. Heisig: PadeS©-Gestaltungsplanspiel zum interdisziplinaren Training f'tir betriebliche Planungsprozesse. In: Proceedings of the 10th European Forum on System Simulation and Management Gaming, Bad Neuenahr 14.-16.11.1994. F. Schulz von Thun: Miteinander reden. Strrungen und Kl~xungen. Reinbek bei Hamburg 1981,269 p. K. Mertins, M. Rabe, R. Jochem: Factory Planning Using Integrated Information and Material-flow Simulation. In: Proceedings of the European Simulation Symposium ESS'94, 09.-12.10.1994, Vol. II, pp. 92-96. K. Mertins, H. Edeler, R. Jochem: Integrated Enterprise Modeling - First Step Towards an Enterprise-Wide Optimization of Business Processes. In: Proceedings of IFAC/IFORS - Workshop on Intelligent Manufacturing Systems. Kopacek, P. (Ed.). Vienna, 13.-15.06.1994, pp. 199- 204.
VI.4 The I CHING and Modern Science
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
1029
The I C h i n g O n t o - / A x i o - G e n e s i s a n d the A n a l y t i c H i e r a r c h y Process: Decisions, N e g o t i a t i o n s a n d Conflict R e s o l u t i o n s Chung-ying Cheng Department of Philosophy, University of Hawaii at Manoa Honolulu, Hawaii, USA 96822 The Xici's statement that "Yi has the taiji, taiji generates two norms, two norms generates four forms and four forms generates eight trigrams" represents a process of ontogenesis based on the following principles: the principle of primordial totality, the principle of contrary polarity, and the principle of continuous division, What makes this process possible is a matter of ontocosmological observation and comprehension which we may identify with the process and goal of g u a n (comprehensive-contemplative observation). This ontogenetic process not only generates the 8 triagrams and 64 triagrams but also generates a relationship network of polaristic positions with different histories of derivation. This can be regarded as the world of forms of nature as actually given. This is also the world of representation and world of description. Furthermore this is a world of "feeling and response" (gan-ying) which makes our description of forms in basic y i n - y a n g paradigms possible The ontogenetic process as described above constitutes an open-end lattice structure rooted in one origin which not only explains the possibility of formation of Fuxi b a g u a / l i u s h i s i g u a diagrams (diagrams of original and ideal balance and harmony), but displays how our actual world is open-ended, many-leveled, in the making, capable of both vertical and horizontal developments, with different combinations of forms of different historical lineages, capable of both conflicting and harmonizing with each other. On the level of actuality, one can see how things form and integrate and how life grows and proliferates. It is a process of coordination, co-determination and co-differentiation of the ontogenesis which we may prospectively understand and also refer to as progenesis. One can also see how things disintegrate and dissipate and how life matures and recycles. It is a process of completion and fulfillment of inherent and intrinsic values of cosmo-ontogenesis which we may retrospectively understand and refer to as axiogenesis or retrogenesis. This axiogenetic or retrogenetic process also forms a lattice structure which can be considered completed and positioned or placed in a structure of relationships of things as values. It is the counter-image of the progenetic process and structure. (One may regard the totality of the ontogenesis and axiogenesis with actuality as interfacing as the structure of the dao.)
1030 Recognizing actuality in this light, we can also see how actuality is an open field of creative possibilities: it has a supply of infinite energy and momenta, but it allows diverse formations and realizations of diverse forms. Thus, while we may maintain an ontogenetic overview, on the other hand we can also appeal to our own norms and requirements and project our desires and passions on the world so that we m a y conceive changing the world by our participatory efforts and projections of our plans and goals. What we may in fact engage in can be described on two levels: the level of motivation and formation of p u r p o s e along the o n t o g e n e t i c process and structure; the level of formation of goal and implementation along the process and structure of axiogenesis. The former is the level of origination and the latter is the level of completion. Each has a lattice structure of triangularity, the former with triangular origination and the latter with triangular targeting. Together they form a process of origination, divergence inflection (by way of creative valuation), convergence and completion. This process also forms a structure of rectangular balance and symmetry which again can be regarded a superstructure return and regeneration. It is with this understanding that we may construe h u m a n decisions as ontogenetic and axiogenetic initiations or originations (or in Nelson Goodman's terminology, "world-making" initiations and originations). Decisions are not only matters of assuming responsibility or causingan effect, but creating or opening up or resetting new worlds, worlds which can be traced to their immediate causes or sources to the decisions which again must be understood in terms of some hidden ontogenetic/axiogenetic processes. We may divide decisions into two kinds: 1> decisions which are based on our u n d e r s t a n d i n g of the world as given in knowledge and historical information which lead to the formation of plans and designs; 2> decisions which are based on our understanding of the world as given by our on-hand experience and encounter with the actual flow of the world which leads to adjustments and adaptations in practice (this is a form of Greek "phronesis"). The former can be called "rationalanalytical decisions", the latter "intuitive-creative decisions", decisions which are directed to the open world or actual world and must take account of emergent factors. What needs to be stressed is that even in the "rational-analytical decisions" one needs to have ontogenetic considerations in order to arrive at a relevant and effective conceptual scheme as representation of the world as already given with a historical background. Thomas L. Saaty's "Analytic Hierarchy Process (AHP)" (in his book Multicriteria Decision Making: The Analytic Hierarchy Process, Pittsburgh, 1988) represents an expansion of rational consciousness basically using logical and scientific principles as guiding principles for categorization and subcategorization of the world. It does not take account of the ontogenetic process and structure of the I Ching as its model and foundation as it should. For otherwise we do not have any reason to hold to this process as a decision-making process or to even come to an
1031 u n d e r s t a n d i n g of the process. This process must be ontogenetically understood so that we can see w h y it naturally and necessarily comes to be so. Besides, h o w this process can function as a decision-making process requires the understanding of the axiogenetic process as a counter-image of the ontogenetic process. Finally, we need to understand the nature of decisions and decision-making as a process which has an objective reference in the world. It is not a relativistic nor a subjectivistic projection of whimsical desires and perceptions. Hence we need go to the foundational or transcendental problem of categorization and subcategorization not based on a prior rationality, but based on a comprehensive observation of the world across epochs of time. The I Ching ontogenesis of yin-yang differentiation and integration provides such a foundation and justification rooted on comprehensive observation (guan) of the world across epochs of time. The weighing of values also requires the axiogenetic model for making sense of evaluation and for justification. Finally, one must not forget that decisions involve applying and satisfying multiple criteria. H o w multiple criteria are to be identified, formulated and then integrated or ordered to achieve the optimum completion of m a x i m u m value is the essence of decision-making. The basis for such multi-criterial fulfillment must be found in an ontogenetic theory of integration of differences and that of integration of reasons for such integration. It is in light of the I Ching ontogenesis we can see the beginning of an answer to this question. One m a y think that one can identify anything's position or place in the scientific scheme or map of the world. But a scientific scheme or map of the world is given only in science. Our c o m m o n s e n s e w o r l d v i e w on the other h a n d is a manifest image which presents the world as we sense it but which nevertheless also allows scientific elaboration and transformation, such as the p h e n o m e n a of sunrise and sunset w o u l d allow us to u n d e r s t a n d that "The earth rotates from West to East", This means that our common-sense w o r l d v i e w (or for that matter our commonsense language or language games) is a metaworldview or metalanguage from which science derives its meaning under pre-designated scientific conditions. In this sense science needs not contradict our commonsense experience but on the contrary m u s t have to thrive on the openness of the c o m m o n s e n s e world and commonsense language. It must also need to depend on the specificity of purposes and conditions under which science would be adequate to serve and observe. N o w just as we may conceive science as an elaboration of commonsense one may also conceive commonsense as an elaboration of our perception of a world of change to be spelt out in the language of Yin and Yang. Why the language of yin and yang? It is because the commonsense worldview and language must be based on the ontogenetic process and structure as described above. Once decisions are properly understood in the context of the ontogenetic and axiogenetic processes and structures, we can see how negotiations as decisions based
1032 on two or more persons' efforts to make decisions are based on or capable of being based on (or referred to) the I Ching processes and structures of ontogenesis and axiogenesis. What is available and useful as resources and insights in the I Ching ontocosmology of ontogenesis and axiogenesis can be applied in negotiations. By the same token, conflict resolutions can be regarded as decisions to be made for and on behalf of two or more disagreeing and conflicting parties, the process and structure of which can again be represented in the ontogenetic and axiogenetic processes and structures like individual decision making, or personal or group negotiations. Both negotiation and conflict resolution can be thus seen as decision-making processes which involve ontogenetic or axiogenetic resettings or reinitializations of the world, which are ontogenetically or axiogenetically parts of the world as the taiji and the dao.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawaand H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
1033
P h i l o s o p h y of U n i t y in D i v e r s i t y -The D a n c e of Q u a n t u m a n d the I - C h i n g ' s S y m b o l Thomas In-sing Leung C.R.R.S. 200-8140 Leslie Road Richmond, B.C., Canada V6X 3W1
In 1952, David Bohm published a paper in Physical Review, 1 in which he suggests the use of the concept of hidden variable for interpreting the puzzles of Quantum Mechanics. He maintains that the particle and its movements are basically produced by an underlying wave motion. The universe is somewhat like a body of water, the wave packets became the phenomena of particles. They constantly unfold out of the body of wave and enfold in to it again. He introduces the term quantum potential to explain the essential features of quantum mechanics through a subquantummechanical level. There are enfolding and unfolding processes through which information comes in and out of the particles. This is the early causal interpretation of David Bohm. He then develops new ideas later. In his 1975 paper 2 and 1980 book Wholeness and the Implicate Order3 David Bohm contends that an implicate order lies behind all phenomena and forms the wholeness of the universe. He says "that in separable quantum interconnectedness of the whole universe is the fundamental reality and that relatively independently behaving parts are merely particular and contingent forms within this whole. ''4 He also introduces the term "implicate order" to explain the enfolding and unfolding processes of the quantum potential. He summarizes his thought in his 1985 paper Hidden Variables and the Implicate Order that "the causal interpretation of quantum mechanics and the implicate order are two aspects of one comprehensive notion. This can be described as an overall implicate order, which may extend to an infinite number of levels, and which objectively and self-actively differentiates and organizes itself into independent subwholes, while determining how these are interrelated to make up the whole. ''5 In 1993, he proposes an ontological interpretation of quantum mechanics in his book The Undivided Universe, 6 in which an ontology of quantum mechanics is developed. It becomes a philosophy that can compare to other philosophical systems.
1034 Some basic concepts in the ontology of David Bohm are quite close to the philosophy of I Chuan, which is the appendix that provides the orthodox interpretation of I Ching. The I Chuan proposes a process ontology with the concepts of I and Taichi. I in Chinese originally means change. The universe is perceived as always changing and transforming. The changing process is determined by a fundamental creativity (sheng). It says, "The greatest virtue of heaven and earth is creativity", "creative creativity" (sheng-sheng) is called I. Then the concept Tai-chi (the Great Ultimate) is introduced. "In the I, there is the Tai-chi. It creatively manifests the two modes, the two modes creatively manifest the form hsiangs (symbols or images), and the four hsiangs manifest the eight trigrams." The Tai-chi is the unique principle of I, it is the ultimate creativity that makes the manifestation of different symbols - images and trigrams, that is, the forms and beings of the world. From the I Chuan's ontology, the world is an inseparable whole. The many are manifested from this holistic Great Ultimate. On the other hand, the Great Ultimate also manifests the symbols. They are the trigrams and hexagrams in I Ching that form the way for understanding the many in the one. The symbols are the information that are manifested from the one and provide a ground for understanding. I Chuan also introduces the term enfolding and unfolding and uses these terms to describe the category change. It says, "The alternation between enfolding and unfolding is called change, the going forward and backward without ceasing is called penetration." All phenomena are manifested from the Great Ultimate and through the process of change and penetration, that is enfolding / unfolding and going forward / backward. One can understand these processes through the symbols, that is, the information which comes in and out of the Great Ultimate. The I Chuan ontology is a philosophy of wholeness. The ontology of David Bohm and I Chuan is so similar that it can open dialogue between contemporary scientific philosophy and Asian ancient philosophy of wholeness. This paper is a new attempt to bridge the old Chinese philosophy and new interpretation of scientific phenomena. REFERENCES David Bohm, "A Suggested Interpretation of the Quantum Theory in Terms of Hidden Variables", Physical Review, pp. 85, 166, 190 (1952).
1035 David Bohm and B.J. Hiley, "On the Intuitive Understanding of Nonlocality as Implied by Quantum Theory", Foundation of Physics, Vol. 5, No. 1, 1975. °
,
David Bohm, Wholeness and the Implicate Order, (London: Routledge & Kegan Paul, 1980). Same as note (2), p.102. This paper is first published in Zygon, Vol 20, 111, (1985). Later published in Quantum Implication, ed. by B.J. Hiley and D. Peat, (London: Routledge, 1987), p.44. David Bohm and B.J. Hiley, The Undivided Universe, (London: Routledge, 1933).
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
The I Ching and Non-Linear M a p p i n g : A M e t a - B i n a r y A p p r o a c h to Reflective Choice, DecisionM a k i n g , and Hierarchical I n f o r m a t i o n S y s t e m s
M. Secter Department of Communications, Simon Fraser University, Burnaby, British Columbia, Canada V5A 1S6 Institute of Asian Research, Center for Chinese Research, University of British Columbia, Vancouver, B.C. Canada V6T 1Z2
A Special Session" The I Ching (Eki Kyo ~ ~;~ v~L ) and Modern Science
1. 1.1
DISCLOSING A UNIFIED BINARY COMMUNICATIONS SYSTEM Introduction
This paper proposes a meta-binary, non-linear architecture for mapping and managing multi-logical systems and interactive virtual learning environments. It is an original redaction of a binary generated, six-dimension hypercube, inspired by two identical but apparently unrelated models: one from modern biochemistry and the other from ancient Chinese philosophy. The first is the DNA whose sixty-four amino acids embody the binary coding for life itself. The second is the I Ching, a 3000 year old Chinese text comprised of sixtyfour gua or hexagrams, aphorisms accompanied by binary symbols which objectify the coding for the dynamics of change and decision theory. Both DNA and I Ching have sixty-four identical sets, each set identified by a binary codon whose numerical value ranges from 0 to 63. And both models are apparently lacking the context of a formal coherent structure to legitimize or establish the binary numbers as sets in an authentic system. This hypercube construction not only provides a matrix for multi-dimensional operations but serves as a device for quantifying qualitative information and conditions. As such it supports cognitive ergonomics and human-computer interaction.
1037
1038
1.2
A Radical Alternative for Rethinking Information Systems Such a meta-binary machine as this proposed six-dimension hypercube
matrix can function as a server for- multi-logical information processing and object-oriented software architecture 1; interactive learning environments and computerized landscape metaphors 2; and multi-criteria, ontogeneticaxiogenetic, analytic decision hierarchies 3, in human-computer interaction. Reexamining the similarity between DNA and the I Chin.q, my self-appointed task was 'to identify the underlying system that would represent 64 binary sets in a cohesive model while complying with or satisfying the conditions and constraints inherent and explicit in the traditional I Ching literature.' The establishment of such a structure suggested a method for the rational, non-linear integration of binary numbers. This in turn implied a new approach to thinking about communication theory, information systems, and amino acids formulation. It also suggested a meta-binary information processing schema; that is, a system that informs rules which allow binary data to proceed simultaneously along more than one path direction from a particular binary position, set, mode, or field. The inherent implication is" a meta-binary machine based on conventional binary numbers, rules, and logic that transcends the constraints and limitations established by decision trees and conventional hierarchy models. In the same way that four nucleotides (00, 01, 10, 11) recombine to make up sixty-our (64) amino acids from which all life is formed, eight subsets or
gua (000, 001, 010, 011, 100, 101, 110, 111 ) recombine to make up the sixty-four six-digit binary sets (Gua) from which all else can be theoretically derived. In both the I Ching and DNA the sixty-four sets are considered to be the primary states, sets or fields. Higher levels of definition, refinement, or 'possible state' options are achievable by means of the hierarchical nesting of
additional systems within both the initial and the ending primary states.
1039
1.3
The Basic Issues or questions to be Addressed a)
how to establish the rules for constructing a system.
b)
how to establish the rules for applying or utilizing the system.
The first part of the paper outlines the development of the mathematical model. Specifically the algorithms that disclose it are described. The model emerges as one in which binary numbers become established as three-digit subsets which recombine to stabilize as six-digit sets. The sixty-four sets cohere into six subsystems that consolidate into a single system. This could be characterized as a neural network that embodies certain properties that might be expected of a unified theory for a six-dimension system or universe. The neural network configuration raises a further implication. Employing a relatively low-level or simple non-linear hierarchical matrix it is possible to generate multi-dimensional, non-parallel, linear interconnections. With a relatively small number of total sets, a vast number of decision paths or information processes is possible. From 64 amino acids all life emerges. The second part of the paper applies the methodology to both six-dimension information architecture, and the establishment of qualitative contextualizing for problems-solving and decision-making. The latter consists primarily of reframing operations in which cognitive processes and qualitative conditions are converted into subsets.. Because each subset is accorded an array of qualitative characteristics it becomes feasible to define conditions accordingly. Subsets are recombined (into pairs) and converted into six-digit binary values. These become the 64 categorized sets in the system. With the help of object-oriented menus the process of converting qualitative conditions into t w o primary subsets becomes relatively manageable. This indirect approach establishes a more precise correspondence between a condition and one of the sixty-four sets. At the same time the process reduces partiality, expectation, bias, and misinterpretation.
1040
1.4
An Original Six-Dimension Hypercube Information Matrix
The system is based on the rational integration of sixty-four interconnected binary sets into a non-linear matrix that supports simultaneous information processing at all six binary positions or levels. Each level denotes one metadimension. In addition there are three default transformation states each specifying a class of uncertainty" progressive inversion; revision; and binary progression. What emerges is a binary continuum between the system, user, and the interface. It more closely approximates real thought processing. This paper does not suggest that the original authors of the I Chin_q created, understood, or were even aware of the theory or system that I am proposing. However it is possible that its inherent cohesiveness, its internal character was intuitively grasped or acquired after years of study and working with it. In the same way that the gua sets or hexagrams suggest or imply an underlying system, so do the sixty-four sets of amino acids (defined as binary codons) suggest or imply an underlying system to which they belong, and whose rules inform and govern their existence and interrelationships. It appears evident that the sixty-four sets of the hypercube binary system establishes itself into six subsystems or energy continuums. It may be worth asking if: a) all six-dimension systems such as DNA share this subsystem organizational feature; and b) if we are perhaps living in a six-dimension universe, in which event there might be a correspondence between the subsystem structures and the six primary energy forms in physics. At a minimum, this model demonstrates that communication theory and genetics share identical binary systems. The importance of this cannot be underscored. We believe this model will have applications in interactive architecture and object-oriented, interactive software. Greg Tropea (HCI95 this session) David Smith (HCI95 this session) Chung-ying Cheng (HCI95 this session)
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
1041
E x p l o r i n g S e l f - D e v e l o p i n g M o d e l s In C o m p u t e r i z e d , I n t e r a c t i v e L e a r n i n g Environments D. A. Smith Hebrew University, 28 Bezalel Street. Jerusalem, 94501 Israel
1. THE STATE OF VIRTUAL REALITY SOFTWARE AS IT APPLIES TO LEARNING VR technology permits users to move objects and simulate physical actions in a cyberspace and promises radical changes in applications relying on object visualization and manipulation-for example: medical technology, Computer-AidedDesign (CAD), real-time simulation, shoot 'em up arcade games, and so on. Admittedly, this kind of enhanced interaction may exercise previously unfamiliar "cognitive-motor" capabilities and associations, but what can guarantee that users are going to function at higher levels of motivation and proficiency primarily because they happens to find themselves in cartoon-like virtual environments? Once the novelty wears off, why should being in a simulated world motivate anyone more than being in the real one? Ultimately, users are still left to their own habitual devices when it comes to refining behavioral patterns of creative learning, planning, organizing and elaboration in conventional software environments, let alone virtual ones. This situation has not evolved in a vacuum, but comes about, in part, because the trend in contemporary HCI design continues to be a mechanistic one in which the task is defined according to the perception of the tool, instead of the other way around. Accordingly, user participation is task-dedicated. Resulting software operations are, for the most part, devoted to forming or changing inanimate, or symbolic subject matter. Effects upon the user, while interesting, are usually relegated to chance and limited to a small set of mechanical skills formatting operations. 2. THE USER AS QUALITATIVE "PRODUCT" IN THE TASK ENVIRONMENT In contrast, we consider the user-and not necessarily the product (or, in this case, document)--to be the unique entity in the task environment. As suck a user's interactions might have direct, qualitative significance if only he or she were invited and guided to integrate them into responsive structures for awareness and learning. Efficacy of production is not threatened where heightened motivation and efficiency contribute to the accomplishment of the job at hand. The interface suggested here, then, is user-dedicated; one that treats the person as the qualitative, strategic variable in the performance of any task. The model acknowledges and
1042 quantifies individual differences in motivational and intellectual predilection, and offers tools for providing feedback. In taking the initial empirical steps to further evolve this paradigm for collaborative "being" and "learning" in a virtual reality (VR) environment, we propose to create and test a minimum sized "proof-of-concept" model--including its graphical user interface (GUI). We elaborate upon a form of virtual reality investigated in "An Interactive Media Environment for Creative Self-education-A Theoretical Definition," (Smith, 1988, 93). Named EDUGATES, this work defined a spatial landscape metaphor for graphical visualization of data. It discussed an expanded, alternate role for hypermedia and applied metaphor in learning and understanding. Theoretical guidelines and psychosocial methodologies are adapted from M.D. Caspi's '"Transformational Approaches to Creative Self Remaking," a comprehensive theory of "Self-Remaking (Caspi, 1985, 92, 94). The work offers an innovative theoretical approach and implementation that "unifies" neural network stimulus patterns responsible for: a) the semantic and spatial location of properties within a "fractal" database (the simulated spatial environment); together with, b) the criteria responsible for administering user and system generated events within that domain. Intersecting vectors describe a geomorphic lattice of "hard" data and "soft" user/system events that roughly describe 1:1 correlations on the surface of a hypersphere. These correspondences are mapped and continuously reinvested into the spatial and event horizons of the virtual environment. This mapping strategy essentially means that any given location in the virtual environment will contain arguments that at once promulgate and constrain the events that may occur therein. Visualization cues manifest relationships between the significance of data and the "shape" it assumes in a database. Data is a step closer to looking like the information it represents as the user invokes a virtual, micro-environment and dwells at the juncture -- or "place of moment"-- located between the real and virtual extents of the manifest "interface continuum."
3. CONCLUSION The aim is to provide the scaffolding for dialectic processes by which "learners" transform themselves into "doers" and "knowers" as they create the boundaries and rules that govern the changes in their virtual environment. The resulting virtual representation thus approaches a unique synthesis of significance, syntax and intention. In contradistinction to prevalent conventions that treat information as finite, discrete "products" for consumption, the way information is presented in this model encourages users to discover and benefit from how they, themselves, are changing in light of their interactions.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
1043
Business Rules, Revolutionary Discourse, and Multilogical I n f o r m a t i o n Systems G. Tropea Dept. of Philosophy, California State University, Chico, California, USA 95929-0730
1. THE HUMAN REALITY OF INFORMATION SYSTEMS The principles of systems theory apply both to the devices we use to process information and to the contexts in which they are used. Part One of this paper begins with an application of these principles to the task of understanding the human reality of information systems. It then builds on these ideas to present the basic theory of a new concept, the multilogical information system, including an account of its current context. To achieve an adequate account of the information system in context, the paper challenges the view of the information system as a set of devices external to human beings. In place of the traditional notion, the information system is sketched as a composite entity whose operation spans the division between the natural and the artificial. The goal of the opening pages of the paper is to radicalize the sense of object in the information system to include human beings. The conceptualization of human beings as system objects is not pursued as a stereotypical technological reduction, but rather as a way of developing a consistent object orientation at the theoretical level. It also shows how thorough the integration of information systems has actually become.
1.1 MODES OF DISCOURSE Part One continues with an analysis of modes of discourse. The theses of this discussion belong not only to the field of information science, but to philosophy and cultural anthropology as well. The main purpose of this part of the exposition is to establish a working sense of the concept of the multilogical information system as a kind of discourse that has hitherto been effectively prohibited by the rules of civilization, but now is required. Basically, the shift is from a preference for linear, sequential conversation, which we know as the dialog, to a new form of conversation in which multiple relevant signals are exchanged simultaneously, which we call a multilog. It is both culturally and technologically significant that this complex type of communication has become both possible and necessary at this point in history. The fact of this new requirement becomes the point of departure for Part Two of the discussion, which takes a more practical turn.
1044 2. ORDERLY PROCESSING IN A DISTRIBUTED COMPUTING ENVIRONMENT Part Two seeks to develop a basic architecture of a software solution to the perpetual information system problems of orderly processing (namely, integrity and contention) in a distributed computing environment. The proposed solution is grounded in the I Ching in two ways. First, it takes a cue from the metaphysics of the text in that it assumes change, not rest, as the default state of the system. Second, at a more practical level, the paper proposes inclusion of a run-time repository of information about objects and processes whose organization is based on the hexagram. This design specification is not an attempt to show that the originators of the I Ching were computer engineers or even anticipated the modern computer, but rather that their attention to the dynamics of change in a complex world may be of use to us, who must attend to the dynamics of change in complex information systems. Each part of the paper does its own work, but implications of the theoretical section become clearer in the application section, and motivation for specific features of the application will be found in the theory. Information system design for the foreseeable future will be driven by the fundamental ideal of modern quality assurance: everyone must know everything that is relevant all at once. Business rules guide the encoding of knowledge in the information system, but do not directly determine data structures or internal logic of a system. While there is an unhappy history of naive attempts to construct systems whose internals precisely match knowers' intuitions, the attempt to stay close to uncritical intuitions is not itself naive if independent evidence exists to motivate the design. Part Two's specification of the run-time repository as rooted in basic categories of perception and the presumption of dynamic equilibrium in the information system suggest that the foundational points explained in Part One can indeed be usefully mirrored in the system's internal logic.
2.1 PRIMARY DIMENSIONS AND METADIMENSIONS The main work of Part Two is specification and sequencing of the key elements of program logic in a multilogical information system. These elements are patterned closely on the essential characteristics of I Ching hexagrams. Some correspondences are more obvious than others, but the full proposal does not come into view until one grasps that a multilogical information system operates in ten dimensions, including four primary dimensions and six metadimensions. The importance of the six metadimensions is that they are available for constant reference from any point and provide a way for one object to connect abstractly with any other object. In other words, through the metadimensional repository, objects monitor other objects themselves and do not necessarily interact with the primary information or the internal control information of those objects. In Leibnizian terms, objects in the system function as monads, but not windowless monads. Each
1045 metadimension may have one of four values indicating something about the relationship of the object to other objects in the system. The four primary dimensions in an information system refer to the three space dimensions and one time dimension of ordinary experience. The six metadimensions correspond to psychological realities of past, X, present, Y, future, and Z dimensions. Each of these is assigned specific meaning in the context of the information system. After defining its terms, the paper goes on to show how this addition to the computing environment would work to maintain system integrity and help resolve contentions. Since there is an introduction of additional theoretical and procedural complexity to the system, even if implementation of this design may reduce actual processing, there is a legitimate concern that the cure might be worse than the disease. To address this issue, the paper includes a discussion of how its proposed additions can be maintained under program control.
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
1047
T h e I Ching as a P a r a d i g m for U n d e r s t a n d i n g C o r r e s p o n d i n g States in Fundamentally Different Systems J. W. Walls Simon Fraser University at Harbour Centre, 515 West Hastings, Vancouver, Canada V6B 5K3
1. Y I N - Y A N G A S B I N A R Y S C H E M A The word for 'science' in Chinese (~_~t~ kexue), Japanese (kagaku) and Korean (kwahak) languages means literally 'the study of classification.' Our awareness of the world around us begins with the most primitive form of classification: This, and That, perhaps first in the form of "I" and "Not-I". Early on, based on this primal distinction, we become aware of the difference between "awake" and "asleep," which soon become associated with "activity" and "rest" and "day" and "night". In ancient China such polarities came to be schematized as yang (11~) and yin (~), and the movement back and forth between any two such polarities came to be symbolized in the famous taijitu (~*~1~!), in which activity, heat, sun and daytime, for instance, are represented by the lighter element, while rest, cold, moon and nighttime are symbolized by the darker, inverted element. The endless cycle of movement back and forth between such polarities may be seen as symbolizing the essence of a system dynamic (1). Systems -- such as mechanical, biological, meteorological, astrophysical, economic and social systems -- may be observed and described as moving back and forth between cyclical periods of activity and rest, expansion and contraction, growth and decay, etc. These polarized descriptors may all be symbolized by yang and yin, and their dynamic interaction by the taijitu, which is the circular chart in the center of Figure 1 below. The yang principle may be represented by a single, unbroken line; the yin principle by a line with an empty space in the middle. Incidentally, yin is seen as representing the essence of zero and all even numbers, while yang represents one and all odd numbers -- a relationship that is essentially binary and digital (2):
active growing positiw hot
YANG
bright
p~sive declining neg~ive cold d~rk
sun
ll'lOOh m
m
YIN Figure 1. Yin-Yang and Taiji Chart But the yin-yang taijitu schematic system is not simply digital, as it clearly emphasizes "degrees of yang-ness" and "degrees of yin-ness", a mode of awareness that is clearly analogical.
1048
2. T A I J I T U
AS D I G I T A L - A N A L O G
SYSTEM
SCHEMA
Gradually, as people mature, they become aware that reality may be perceived not only as a multiplicity of "either-or's" (yin-yang's), but also as sequences of "more-or-less's". For example: morning and noon are sequences of magnitude, culminating at the meridian and followed by afternoon and night, which would be represented in the taijitu as the narrow-but-growing tail of the lighter element (morning), the head of the lighter dement (noon), the narrow-but-growing tail of the darker element (afternoon), and the head of the darker element (night). Through the process of analogy, we can easily see the same graph as illustrating the four seasons of the year: spring is the morning of the new year; summer is noon; autumn is the afternoon; and winter is the night. From here, it is quite obvious that the morning-noon-afternoon-night and spring-summer-autumn-winter sequence are analogous to the childhood-adulthood-middle age-old age stages of a person's life. On the level of the social system analysis, the evolution-devolution sequence is a perfect representation of what Chinese historians have called "the dynastic cycle": following the dissolution and overthow of a corrupt regime, a new dynastic order is seen as emerging in its energetic, idealistic youth -- its "young yang "stage, during which time the wrongs of the preceding dynasty are righted. The next stage is that of dynastic adulthood, the "flourishing" period when civilization reaches a high-performance peak. This is followed eventually by a transition period that enjoys some of the momentum inherited from the flourishing stage but also starts showing symptoms of decay. The final period is characterized by imperial indulgence, unreasonable taxation of the populace, corruption among local officials -- even natural disasters, usually regarded as signs of Heaven's displeasure with an immoral regime, seen in China as the father's displeasure with the son, since the Emperor was regarded as the "Son of Heaven" (3). S old
yang ~lulthood
young yang
young yln
childhood
middle age
W
old age
old yJn N
Figure 2. Young and Old Yang and Yin In the northern hemisphere, the four quarters of the compass, too, seem almost naturally analogous to the four seasons and four stages of life: The east and its rising sun are analogous to youth and springtime" the south with its scorching heat is analogous to summer, mid-day and
1049 adulthood; the west and its setting sun are analogous to evening and middle age; and the cold north is analogous to winter and old age. The quaternary cycle is symbolized by four pairs of lines (digrams) in permutations of "young yang" (solid line below broken line), "old yang" (two solid lines), "young yin" (broken line below solid line) and "old yin" (two broken lines). These four stages in the life cycle of systems are also the core symbols on the national flag of the Republic of Korea, where we find the I-Ching trigrams forqian (~, representing the yang principle of activity) and kun ( ~ , representing the yin principle of rest), positioned on opposite sides of the taegukki (taiji flag), with qian at the pinnacle of yang development, and kun at the pinnacle of yin development. The trigram for fire (~)is positioned halfway along the route from kun to qian, and water (~.~)is located halfway down the path from qian to kun. If we begin with a state of rest, then fire may be seen as leading to the pinnacle of yang activity, followed by water which douses the fire, leading back to a yin state of rest.
///' Figure 3.
Taegukki Korean Flag The yang element in the Korean flag logo is coloured red, which represents fire and heat, associated with spring and summer; the yin element is blue, representing the cool of autumn and winter. Even though the taijitu in the Korean flag has been "tilted", we still find the kun trigram placed by the pinnacle of yin, and the qian trigram by the pinnacle of yang.
3.
THE 'EIGHT TRIGRAMS' ELABORATION
The next larger cycle is symbolized by eight trigrams, the so-called "Eight gua" (3~,~1'). The eight trigrams are often placed in a sequence that seems to be an elaboration of the quaternary cycle: movement, brightness, pleasure and fulfillment on the yang side; penetration, danger, standstill and emptiness on the yin side. "Movement" (~-), also equated with "thunder", is the early harbinger of activity and developments to come. "Brightness" ( ~ ) , also equated with "fire" and "clinging", is typical of the undisciplined vigour found in young systems. "Pleasure" (.~), also associated with large expanses of still waters, is the stage of life wherein one can enjoy the benefits of youth without worrying about decline, because the pinnacle has not yet been reached. "Fulfillment" ( ~ ) , also seen as representing "Heaven", is the peak of development, the stage of maturity, after which systems tend to begin to devolve.
1050
fulfillment pleasure i
qian
i
penetration ii i
i
sun
dui brightness
danger i
i
i
i i i
i
li
kon
movement i
I
i
i
standstill empti ness
zhen
i
i
i
i
i
i
i
i
I
i
gen
kun Figure 3. The Eight Trigrams The first trigram on the yin side of the taijitu is "Penetration" (~.), associated with the wind and the trees, another harbinger of the fall that is coming. "Danger" (~.~) is equated with flowing water that sweeps away the fallen leaves in autumn, when cyclic decline is no longer deniable. "Standstill" (R:) is symbolized by the mountain, where sages meditate on the ultimate futility of all organized effort that is not in tune with the natural flow of Nature. "Emptiness" ( ~ ) is associated with earth, the ground to which all spent systems return after their fall and before their reincarnation for another cyclic tour of "rise-and-fall". This is the way most "non-linear systems" have been seen by Asian analysts to evolve and devolve over time. The schema is also seen as a paradigm for understanding the history of institutions, a traditional metaphor for which is ",~,~" ("rise-fall'). The gua sequencing in the I Ching is usually seen as an elaboration of this basic cycle.
4.
THE TWELVE HEXAGRAMS
The next degree of elaboration is called "The Twelve Inspiration-Expiration Hexagrams", with its implication that "Inspiration" embraces the yang evolutionary phases, while "Expiration" stands for the yin devolutionary phases. These twelve gua represent a linear progression of changes from yin lines to yang lines, but the linear progression describes a circle/cycle that is parallel and analogous to the yin-yang evolution-devolution cycle illustrated by the taijitu. It is interesting to note that the twelve "Inspiration-Expiration" hexagrams, when seen in the full Fu Xi circular arrangement of all 64 hexagrams, occupy positions removed from qian and kun by the numbers 1, 3, 7, 15, and 30 -- each double the distance of its predecessor. Space limitations do not allow for detailed discussion here, but the evolution-devolution syndrome is immediately apparent upon contemplation of the "Inspiration-Expiration" chart.
1051
breakthrough I
strength i i
fulfillment m
I
encounter
| i
qian
quai
i
i
gou
retreat |
i I
ii
da zhuang
dun
retrogression
progress I I I
•
I I I
i i n
n i .
pi
toi advancino I I I I
contemplation
I I I I
/
i I
lin
returni no
~_ ~_~ fu
collapse
em.....ptines.._.s E
E
I
I
E
guan
E bo
kun Figure 4. The Twelve Inspiration-Expiration Hexagrams
5.
F U R T H E R E L A B O R A T I O N S OF THE 'RISE-FALL' S C H E M A
Another level of elaboration of the same allegorical life cycle is is that of the "Twenty-four Seasonal Subdivisions", which demarcate the year's rise-and-fall with changes that occur each two or three weeks in a typical year. Notice how the seasonal sequence fits the growth phases of yang and yin when placed so that they circumambulate the taijitu, indicating analogical relationship with general system cycles. In the chart that follows, the duodecimal cycle of the zodiacal animals is placed between the taijitu and the 24 seasonal subdivisions, to remind us that the sequence of twelve 2-hour watch periods that mark the progress of each 24-hour day are also analogous to the seasonal subdivisions: Starting with "-~" (the Rat) at the "Winter Solstice", the first watch is 1 l:00pm to l:00am, the midnight watch; the second watch (the Boar) is l:00am to 3:00am, and so on through the morning, midday, afternoon and nighttime, in parallel sequence with the progress of the seasons through the year. Thus the yin-yang binary cycle and the four digrams may be seen as a rough, "macro-perspective" on systemic evolution; the eight hexagrams, 12 zodiacal figures and 24 seasonal subdivisions are a "meso-perspective" offering a mid-level of detail in isolating the significant stages of systemic evolution; and the 64 hexagrams of the full-blown I Ching offer more of a micro-perspective aid to understanding the many possible configurations of any complex system. Perhaps an inordinate amount of attention has been paid, over the years, to the "divination manual" aspect of the I Ching, and not enough to its "mnemonic" and "check-list" functions. It is a mnemonic device for remembering the characteristic elements, relationships and evolutiondevolution patterns of any system, from the simplest to the most elaborate. It is also a check-list for itemizing and analyzing the system states at various points in a life cycle.
summer Ilndn s o Is t ic e
in e~'
budsg~r' ~
'
slight
/
he~
summer ~ ~
t
begins
/ e /
gt'~Jh
ndn
tie.bright~ ~ "
vernal equinox
,
J~,
o~
/
~ ~,u autumn
/
begins
--J-'. - ~
,~
finish
E~
~
white autumn equinox
~
insects __--.---"-"
"---------__ cold
,.O'°'n° /// begins
"'°"r... begins
gr~=
cold slight
cold
winter solstice
he~w
light snow
she,
Figure 5. The 24 Seasonal Subdivisions The I Ching, therefore, may perhaps best be seen, understood, and used as a schematic paradigm, an elaboration of the yin-yang taijitu abstraction of basic systems dynamics, which requires the observer to take into account all the other typical stages of a system when considering any given stage in a non-linear cycle of "evolution-devolution". This requirement to consider any gwen state together with a variety of other potential states is, in fact, consistent with the spirit of contemporary "context theory". The 64 hexagrams of the I Ching are a goldmine of opportunity to hypertextualize a system of interconnected symbols, linking the related states of each hexagram in such a way that complexity may be recognized, its elements systematized, and connected into a network of relationships that deal with dynamic complexity in a "user-friendly" fashion.
REFERENCES 1. Cheng Yi, I Ching: The Tao of Organization (Thomas Cleary, trans.), Kuala Lumpur, Malaysia, 1991. 2. K. Walter, Tao of Chaos: Merging East and West, Austin, 1994, 115. 3. D. Bodde, Essays on Chinese Civilization (C. Le Blanc and D. Borei, eds.), Princeton, New Jersey, 1981,246.
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
1053
Nonlinear Computation in the I Ching K. Walter Kairos Center, 4608 Finley Drive, Austin, Texas, USA, 78731 1.
INTRODUCTION AND HYPOTHESIS
This work presents an innovative model for computation. Current computers are mostly binary and do not take nonlinear aspects into account. They certainly cannot do both binary and analog computation in the same process. But such things are possible. Our model will be modern DNA and the ancient / Ching. Each system combines within itself both /inear binary processing and ana/og ratio processing into a peculiar hybrid mathematics that has made DNA and the / Ching into the sturdy system-survival packages that they are. Each model manifests the principles of the new science of patterned chaos. The core of chaos theory is found in the Period 3 window of the periodic tree of bifurcating data. Yorke and Li showed mathematically that when the Period 3 window appears, abruptly you have not random chaos but patterned chaos, i.e., orderly but nonlinear structure, able to sustain and replicate itself with variation. This hypothesis says that mathematically the mRNA codon and the I Ching trigram each presents a Period 3 window of chaos patterning. More important, each system is nonlinear, combining analog and linear functions to synthesize a transcendent third operation whereby the system escapes to a higher order of organization. 2. DNA SWATCH, I CHING HEXAGRAM, AND ENTRAINMENT
A DNA swatch or an I Ching hexagram can be demonstrated to be two Period 3 windows counterposed against each other to form a fail-safe package of complementary chaos, or co-chaos, for short. In each system, its pairs of triplets may be arranged into 64 possible configurations...which naturally give us the 64 codons of the genetic code or the 64 hexagrams of the I Ching. Probably DNA took the form it did along the double helix because this paradigm of counterposing two Period 3 windows against each other across the double spiral gives the sturdiest possible mathematical structure. This counterbalance gives co-chaos, whereby one chaos system balances itself against another in a fail-safe supersystem that provides both stability and also the possibility for evolutionary change within that stability. Most contemporary gene sequencing programs are set up to handle only binary aspects, avoiding nonlinear aspects altogether. But to quote "Hacking the Genome," an article in the April 1992 Scientific American:" The clarity of the answers will depend on asking the right questions." it may be that asking the right questions for computation now is to look at DNA and the I Ching and ask how and why is each essentially a hybrid analinear system that utilizes the principles of co-chaos? Probing for these answers may reveal much about the basic patterns in life's physical and mental systems. It may offer a new way to build computers so that they can imitate the basic number framework hidden in
1054
life itself, and perhaps even the universal coding that no doubt forms the root of nature. Number itself forms the root. To find this deeply-embedded root, we do not discard traditional scientific linearity, but instead we add something new--analogs. Analog plus linear gives analinear number. It is not just linear. It combines the chunky lumps of binary sums with the flowing proportions of analog ratios to birth a transcendent third form. Some might call it nonlinear, but Stanislaw Ulam said this is a rather silly term, since most of life's problems are nonlinear. He said saying "non-linear" is akin to calling most of the animals in the zoo "non-elephants." Therefore, I prefer to use the term analinear, showing it combines both modes to create a synergistic third state. Binary number seeks a goal, a solution, the answer to a problem as a quantity of units. It is a discrete, end-stopped sum giving the goalmthe quantity. But analog number does not emphasize a solution, a goal, a final lump sum. Instead, it discusses the quafity of relationships along the way. This brings up all kinds of resonant associations that open the doors to ongoing process rather than closing them down into the sum of a final answer. That's the trouble with analogs, from a traditional computational point of view. Analogs are networking rather than end-stopped. They engender resonances that linearity doesn't want to encourage because it prefers to stay tidy and neat and hurry to a quick solution--not trigger a network of related resonances. Analog numbers resonate in networks that reinforce entrainment. Entrainment is the main signature of analog number. It does not care about the summary quantity but, rather, about the relative qualities along the way. Its comparisons shift in changing ratios, not striving for an end but rather for the consummate trip, so that finally it never gets there, because there becomes irrelevant. The end becomes no goal at all as instead, it just keeps on traveling. When analog and linear are combined into analinear number, it can do both m find straight-line solutions and keep traveling in cycles. The result is the spiral of change. 3.
I CHING, BINARY SEQUENCING AND ANALOG FLOW
The ancient Chinese I Ching provides an astoundingly complete computer model using binary sequencing plus analog flow. Its structure first of all shows binary number, a fact long evident to the West since the days of the German scientist Gottfried Wilhelm Leibniz. At the turn of the 1700s, Leibniz saw that the I Ching hexagrams may be read as binary number, counting from 0 through 63. Other scholars have since concurred in this observation. More recently, though, Western scientists have begun to recognize that the / Chin~s yang and yin can even be cross-coded in a binary way with the genetic code. Gunther Stent discusses this procedure in The Coming of the Golden Age, published in 1969, and Martin Schoenberger in The I Ching and the Genetic Code in 1973. Scientific American's January 1974 article by Martin Gardner explores the binary math of the I Ching. Then came Eleanor B. Morris's Functions and Models of Modem Biochemistry in the I Ching in 1978. In 1991 came Johnson Yan's book called DNA and the I Ching. Each author considers various binary aspects of this genetic code//Ching interface.
10:55
We may ask ourselves: how could the ancient East and the modern West, so far apart in space and time, come upon the same mathematical model, with the East seeing the I Ching as an oracle that codes for the flow of psyche, while the West sees the same structure as DNA that codes for the building of flesh? Obviously, its underlying root is where mind and body come together. To explain this, let us consider the I Chin~s fractal aspects. Tao of Chaos by Katya Walter presents an innovative view of the I Ching as a computer combining fractal analog and linear binary functions. This scientific explanation is based on the fractals of modern chaos theory, which can predict a trend without specifying its exact details. Chaos patterning is determined because it can predict an overall pattern, but it is also chaotic because it cannot specify any exact point of its next manifestation. The mathematician can determine its general form but not its exact contents. This same dynamic can also be seen in the ancient I Ching hexagrams, which describe 64 basic nonlinear patternings. Patterned chaos has its own special signature: • • • •
Order in the midst of apparent disorder. Cycling that repeats with continual slight variation. Scaling that fits one level into another like nesting boxes. Universal applicability.
Chaos theory has enabled us to find pattern within apparently random events. With it, we rise to a new level of vision and discover that there is simplicity within complex flux. Long ago in China, it was called the Tao. This strange nonlinear realm first began to be explored mathematically in the West during the 1960s, often on makeshift analog computers that charted a peculiar cyclic patterning. Its odd vocabulary of fractals, Julia and Mandelbrot sets, butterfly effects and strange attractors suddenly opened up a new nonlinear reality. This transcendent use of number is seen in the I Ching, developed perhaps 5,000 years ago. It is also seen in DNA, whose structure was discovered in the 1950s. Briefly, here is a synopsis of the parallel structures in the genetic code and I Ching: 0
Binary Tree 0
1 is~. Analog Tree
0 1 A A 0 10 1
0
1
A A 0 10
0
0~
1
1
m m mu mm
m
m
m
"--"
m
m
m
m
V m m
m
V ~/
m m
Figure 1. Model of Binary Tree and Analog Tree
m m
m '
V mmmm
,.1
binary
m m
analog
-
"
-
v 'x/
m m
-
1056
Each of the eight I Ching trigrams can be seen as a Period 3 window. This window can be read horizontally across the branches of a bifurcation tree in the typical linear way, or the branching can be read vertically in a fractal analog way. In other words, the system can read by both methods simultaneously, giving an analinear reading that balances one function against the other. It is quite remarkable! Furthermore, one trigram is then balanced against another in the I Ching to create the 64 hexagrams that may nowadays be seen as 64 sets of counterposed Period 3 windows utilizing both binary and fractal components. Each hexagram describes a unique dynamic process. Likewise, DNA may be seen in this same way. Its pyrimidines and purines use the same organization plan as the I Chin~s bigrams. Within a hexagram, these bigrams may be read across its two trigrams--as a cross-bigram--to encode the message of an amino acid, providing in all, the 64 codons of RNA. Furthermore, it can be shown that the I Ching and the genetic code not only use the same analinear mathematical structure, but they also cross-correlate into the same dynamic meaning for each of the 64 units--with the result that, for example, the Opal codon of the gene's full-stop signal actually equates to Hexagram 12 of Standstill. I Code Tree I Ching Tree .
.
.
.
... ..'."
.
.
-.
...
.-"'--'"
"-O"
Genetic Code Tree T/U
........
C
G
.......-
Pyrimidines
'O"
A
Purines
-_"_ T/U "..
..
-" C .."
"" G "..
Pydm"~dines
A
.~ .."
Purines
" O"
Figure 2. hing Tree, Genetic Code Tree, and "1 Code Tree" Thus, it can be seen that each system--genetic code or I Ching--gives a microcosmic rendition of the larger principle of chaos theory using analinear number. Fortunately, these models, ancient and modern, provide us a means to apprehend the two modes of number working together in a mathematical paradigm that is perhaps inherent in the fabric of the cosmos itself. Numbers hook up to create the patterns of the universe. Analogs form the networks of qualitative resonance in the timing and spacing of matter and energy, while linears develop discrete sums that quantify the units of whatever is being spaced or timed. T o g e t h e r - - a s analinear number-- they give flowing, connective quality to the universe's discrete quantities. To merge the analog with the linear offers a way into a truly universal computation method. When we apply chaos theory, we see that each hexagram and each DNA swatch becomes a nonlinear equation. Chaos theory provides the pivotal explanation for how East and West could find the same mathematical structure from such very different paths. Each version is rooted in chaos theory--more particularly, in analinear number. This number paradigm builds our bodies and our thoughts. It is bone-deep in the species, archetypally deep in the mind. But notice--concentrating only on the binary/digital aspects of number in these two systems would deprive us of the major key--those two counterposed Period 3 windows of complementary chaos that create 64 different dynamic patterns. Going only binary, we would miss out on the nonlinear equations that
1057
form the hexagrams. Since binary merely indicates the 0-1 shunt of a discrete chain of logic, it discounts the integrating fractal properties that are inherent in analog number, and thereby it misses the complex sophistication of cycling proportions in ratio. If we do not see this, we completely overlook this amazing combination of binary structure plus analog relationship which reveals the master code. The i Ching and the genetic code offer microcosmic renditions of this mesh of analog and linear number. To balance and harmonize the analog and linear is the special province of analinear computation. It is seen in the ancient I Ching and the modern discovery of DNA. By combining binary counting with fractal proportion, this paradigm creates analinear equations that may one day provide the means for a new kind of computation.
This Page Intentionally Left Blank
Symbiosis of Human and Artifact Y. Anzai, K. Ogawa and H. Mori (Editors) © 1995 Elsevier Science B.V. All rights reserved.
B i o m a t h e m a t i c s D e r i v e d f r o m the I C h i n g J. F. Yan Yan Research, P.O. Box 4115, Federal Way, WA 98063, U.S.A. The I Ching has inspired many scientists and philosophers in this century. Perhaps the most spectacular is its association with molecular biology and the genetic code. The two systems share a basic principle of using a quaternary system of numbers: four bases in DNA and RNA, and four digrams of the I Ching. Both systems have a total of 64 combinations: triple-base genetic codons and hexagrams for the I Ching. The mathematical association is not limited to these two "coincidental" number sets. The working of the I Ching divination process involves repeat applications of the Chinese Remainder Theorem, well known in modern number theory in its practice. These applications lead to a set of nucleotide numbers (quaternary units 0, 1, 2, 3) and a set of amino acid numbers (0 for "stop" codons, 1 for Trp, 2 for Ile, 3 for Met, and mostly prime numbers for other amino acids.) What is so marvelous is the I Ching can be applied not only to the broadest meaning of universal laws, but also to the finest details of life science: In the broad sense, hydrophobic protein segments are "yang", the hydrophilic ones are "yin"; both are supplemented and complemented with the time or sequential order of "old" and "young". In details, on using this unique set of amino acid numbers, protein sequences are a computable language from which sequence patterns and collective properties can be inferred. The gammatical rules of this language are expressed in the form of two number theorems. The language equivalents range from simple repeating phrases (like in baby talks) to elegant and sophisticated seven-word poems (like those of Li Bai). Sequence examples can be found in frequently occurring repeats in silk fibroin, collagen repeats and mutational hotspots, ancient ferredoxin, coiled coil and leucine zipper patterns, runs of amino acids in protein "huntingtin", implicit heptad repeats in amyloid and prion proteins, etc. Sequence patterns can be displayed with computer graphics, which can then be correlated with mutational data and compared results from molecular geometry. The I Ching is "the book of change", and chemistry is the science of change. In this sense, the I Ching may well be translated as "The Book of Biomolecular Chemistry."
1059
This Page Intentionally Left Blank
1061
Author Index Aar'~, A. Abed, M. Akamatsu, S. Allender, L. Alty, J. Angu6, J. Ankrum, D.R. Anola, K. Anzai, Y. Aoki, M. Arai, K. Asahi, N. Asahi, T. Asakura, T. Asano, Y. Ashibe, K. Athousaki, M. Averbukh, E.A. Bagolini, B. Balbo, S. Barrelle, K. Bastide, R. Bastien, J.M.C. Bevan, N. Black, A. Bodart, F. Brezillon, P. Bugajska, A. Burmester, M. Butera, F. Buur, J. Cai, Y. Candy, L. Carayon, P. Cesta, A. Chatty, S. Chen, Q. Chen, Q. Cheng, C. Chemomorets, V.A. Chignell, M.H. Chislov, V.V. Cohen, B.G.F. Coyle, G. Co~ff6, C. Dainoff, M.J. Danderkar, K. Danuta, R. Daonoff, M.H. Deane, F.P. Dehlholm, F.
575, 745 267 559 393 5 267 611 681 237, 243 725 541 33 381 687,699 423 535 459, 463 21,399 607 337 581 489 343 349, 885 187 367 5 745 899 1003 187 173 103 693 225 27 553 471 1029 969 323 459 745 529 91 745 109 745 745 581 891
Dennison, P. Downing, S. Drury, C.G. D'Aloisi, D. Ebina, T. Edmonds, E. Edmondson, W. Endoh, A. Endres, L.S. Esteban, O. Farenc, C. Felger, W. Fostervold, K.I. Fujigaki, Y. Fukuda, C. Fukuda, N. Fukumoto, T. Fukuzumi, S. Funada, M.F. Giannini, V. Goff, J.M.L. Gogoulou, R. Gohda, Y. Goonetilleke, R.S. Gosain, A. Granda, R.E. Grant, D. Greeson Jr., J. Grote, G. Gulliksen, J. Guo, Q. Haito, S. Hamada, H. Han, J. Hansen, E.E. Hara, F. Haratani, T. Haratani, T. Hashimoto, M. Hayashi, Y. Heggie, S. Heisig, P. Helander, M.G. Henderson, D.R. Higuchi, T. Hiji, M. Hiramatsu, K. Hisamori, Y. Hix, D. Hoang, K. Hollnagel, E.
587 1011 311 225 65 103 5 411 477 27 329 593 575 675,699 963,983 155 553 833 839,963,983 225 291 39 667 311 109 655 877 655 989 951 129 617 423 217 611 565 705 699 803 777,785 103 1021 375 581 713 521 237 405 109,355 629 5
1062 Hopff, H. Horgen, G. Horgen, G. Horie, Y. Hosogi, S. Huuntanen, P. Huuskonen, P. Huuskonen, P. Ichikawa, H. Ide, M. Idogawa, K. Iga, S. Iitaka, K. Ikeda, K. Ikeda, M. Imamiya, A. Inagaki, Y. Inoue, M. Inui, N. Iseki, O. Iskanderova, Z.I. Ito, A. Ito, A. Itoh, K. Itoh, K. Iwai, Y. Jo, K.-H. Johansson, A. Kaarela, K. Kahler, H. Kaminuma, Y. Kandolin, I. Kanenishi, K. Kano, H. Kanou, N. Kaplan, R.M. Karagiannidis, C. Kasamatsu, K. Kataoka, R. Kato, N. Kato, T. Katoh, M. Katz, I.R. Kawabata, T. Kawakami, N. Kawakami, T. Kawarada, H. Kayis, B. Kelley, A.J. Kelley, T. King, R.J. Kirpich, S.V Kirsch, C. Kishi, N.
895 745 575 817 135 681 291 417 509 737 963,983 231 917 45 725 731 803 845, 911 285 381 443 65 635 803 917 809 193 749 417 995 935 681 97 135 911 83 497 983 661 155 559 777 53,83 503 705 845 173 629 713 393 211 969 957 161
Kishino, S. Kita, K. Kitajima, M. Kiv, A.E. Kobayashi, H. Kobayashi, M. Kobayashi, Y. Kogi, K. Koike, T. Komatsubara, A. Konarska, M. Kotani, Y. Koumpis, A. Kubota, S. Kumamoto, T. Kuno, Y. Kurosu, M. Larsen, S. Lee, E.,S. Leino, T. Lepore, D. Leung, C.K.H. Leung, T.I. Leung, Y.K. Lie, I Lif, M. Lin, R. Lin, T. Lind, M. Littlefair, P.J. Macredie, R.D. Maeda, Y. Maenaka, A. Mahar, D. Mahfoudhi, A. Maloryan, V.L. Mariani, M Marin, I. Maruyama, M. Matsubayashi, K. Matsuda, R. Matsumoto, T. Matsuoka, S. McHale, S.T. McNeive, A. Meech, J.F. Meech, J.F. Melo, V. Meri, M. Mertins, K. Miller, L.A. Mimura, I. Minato, K. Minoh, M.
777 535 515 431,443 565 299 845,911 635 547 299 745 285 497 643 65 193 167 575 33,59 681 607 859 1033 211 575 951 199 217 951 623 945 135 33 581 267 443,459 1011 871 141 71 381 147 827 431 1011 291 5,945 323 291 1021 123 167 411 45
1063 Misue, K. Mitsuopoulos, Y. Miyao, M. Miyazaki, M. Mogaji, A.A. Mollaghasemi, M. Molle, F. Molyako, V.A. Morris, A. Motoyama, T. Moyoshi, M. Mukahi, T. Murasugi, K. Murata, A. Murray, B. Mutoh, K. Nagai, Y. Nakagawa, M. Nakagawa, S. Nakaishi, H. Nakashima, K. Nakatani, T. Nemeth, K.J. Nimomija, S.P. Ninomija, P. Nishida, S. Nishimura, T. Nisimura, H. Noma, T. Norcio, A.F. Ntuen, C.A. Nunokawa, H. Oda, M. Odgaard, I. Ogata, H. Ogata, M. Ogawa, K. Ohkubo, T. Ohshima, J. Ohsuga, M. Okada, H. Okada, K. Okada, N. Okuno, H.G. Oppermann, R. Orishchenko, V.G. Oswick, C. Palanque, P.A. Park, K.S. Parlangeli, O. Perry, M.J. Polozovskaya, I.A. Poison, P.G. Rankin, J.R.
135 923 599 521 975 387 607 431,443 587 323 765 273 273 719 103 317 853 155 547 599 845 503 611 839,963,983 983 77,141 45 285 205 471 455 521 559 865,1017 535 823 5,423 725 737 279,771 381 33 205 503 361 431 877 27,329,489 725 1011 623 431,443,459 515 179
Rauterberg, M. Reiterer, H. Rekimoto, J. Roast, C.R. Roberts, C.R. Rousseau, N. Ryan, G. Sagayama, S. Saito, S. Sakai, K. Sakamoto, T. Salcudean, S.E. Saliba, A. Sandblad, B. Sato, M. Sato, S. Savidis, A. Sawa, M. Scapin, D.L. Schallock, B. Scullica, L. Secter, M. Seki, Y. Senges, V. Sepp~il~i, P. Shafrir, U. Shahnavaz, H. Sharit, J. Shih, H.M. Shimoda, H. Shimojo, M. Shimono, F. Shirai, Y. Shiratori, N. Shtakser, G.V. Siddiqi, J.I. Sidhu, C.K. Siio, I. Smith, D.A. Sotoyama, M. Stanney, K.M. Stary, C. Stephanidis, C. Stephanidis, C. Sugamura, N. Sugiyama, K. Suthers, D. Suzuki, K. Suzuki, S. Sweitzer, G. Takahashi, A. Takahashi, M. Takata, K. Takayama, K.
449 361 255 483 705 103 1011 541 617 635 731 713 581 951 173 791 929 725 343 1021 607 1037 791 489 759 437 749 311 859 853 791 279,771 193 59 459 483 529 261 1041 617 123,387 115 39,497,923 929 541 135 5 785,797 963,983 649 791 635 765 135
1064 Takeda, M. Tamada, T. Tamura, H. Tamura, T. Tanaka, T. Tanimura, T. Taptagapom, S. Tawamura, I. Teguchi, K. Templeman, J.N. Teraoka, T. Terashita, H. Thomas, P.J. Thoresen, M. Toda, M. Tokuda, Y. Tokura, N. Tropea, G. Troxler, P. Tseng, M.M. Tsujino, Y. Turbati, M. Ueda, Y. Ui, T. Ulich, E. Umemura, M. Uyemov. A.I. Vanderdonckt, J. Vora, P.R. V/igland, A. Wagner, E. Wakamori, O. Walls, J.W. Walter, K. Wang, S.J. Watanabe, Y. Weik, S. Widerszal-Bazyl, M. Wolska, A. Wong, R.K. Wu, H. Wu, J. Wfifler, T. Yachida, M. Yagi, A. Yagi, Y. Yagyu, T. Yamadera, H. Yamaguchi, M. Yamaguchi, M.K. Yamamoto, M. Yamamoto, S. Yamamoto, T. Yamaoka, T.
785 141 305 667 963, 983 205 617 765 509 109 141 279, 771 945 745 279, 771 59 71 1043 957 859 71 607 45 273 957 509 459 329, 367 375 575 291 853 1047 1053 173 765 989 745 745 249 553 305 989 405, 553, 809 823 405, 809 405 167 285 559 547 827 737 77
Yamasaki, N. Yamashina, T. Yan, J.F. Yano, S. Yano, Y. Yasumura, M. Yazu, Y. Yokoyama, K. Yonemura, S. Yonezawa, Y. Yoshikawa, H. Yoshioka, O. Yoshioka, T. Yoshitake, R. Yoshizawa, Y. Zakharchenko, I.G. Ziegler, J. ZSlch, M.
243 737 1059 827 97,535 231 963,983 765 423 803 853 541 765 661 317 459 899 989
1065
Keyword Index 3D 33 3D graphics 141 3D object 173 active camera 553 active interface 225, 243 address input 541 air quality 635 allocation 989 ALS patients 911 analysis 273, 291, 431 analysis and design 267 analysis support 317 anthropomorphic media approach 565 anxiety 581 assesment 83 attitudes 581 auditory scene analysis 503 automated 989 biomathematics 1059 blind 929 brain activity 839 brain potential 823, 845 calligraphy 129 cardiovascular 771 case-based method 77 CFF 785 cocktail-party effect 503 collaboration 929 color 405 command 45 communication aids 845 computational model 521 context 5 control system 291 creative thinking 431,443, 459 criteria and standards 343 cryptography 147 cube-based 173 curve design 179 data 115 data structure 141 decision making 449, 1037 decisions 1029 desktop interface 123, 261 diagnosis 423 diagram 217 dialog system 547 dialogue 65, 71 dial devices 731 disabilities 923 discount analysis 323
display 623, 895 draw 155 dynamic changes 963 ecological interface design 423 EEG 827, 833 emotional workload 871 empowerment 1021 error 45 erromeous performance 437 evaluation 83, 285, 299, 329 expert system 599 face 553, 559 face robot 565, 877 font 135 front end 103 fuzzy reasoning 199 fuzzy set theory 279 graphical constraint 205 graphics 39, 161 GUI 187 hand-grasp 791 handicapped 917, 935 handwriting 803 head-coupled display system 593 heart rate 777 hospital 411 human communicatio 521 human identification 147 human operatior 455 human operator model 463 human-centered 193, 969 human-machine systems 21 human-robot interface 237 I Ching 1029, 1033, 1037, 1041 1043, 1047, 1053, 1059 icon 199 impression 559 information technology 951 input methods 725 intensity 797 intention recognition 77 interaction 5, 27, 65, 109, 255, 923 interaction cost 285 interface design 59, 91, 187, 387, 411 interface improvement 459 interface specification 483 ISO 661, 891 job 687 job content 681 job satisfaction 975 keystrokes 797
1066 language training learning life lighting linguistics management manipulation market means-ends media quality mental task mental variation mental workload menu metaphor model-based analysis modification motion motor-disabled person mouse multinationa musculoskeletal neural net non-visual object interface object-oriented object-oriented GUI ocular surface area office operators optometric intervention organization personal personality personality engineering petri net plant design pointing device problem solving process approarch prototyping quality control quality of use quantification theory quantitative evaluation R-R intervals real object real world reflective thinking rehearsal rendering requirements robot scenario
917 935,1041 1011 623,643 367 859,1003,1017 211 21 417 305 455 827 765,771,853 547 497 515 205 809 737 713,719 877 745,759 471 39 231 267 249 617 635,687,693,759 607 745 865,951,957,969 945 477 477 449 417 261,737 97 393 53 399 885 273 305 765 231 255 437 509 129 1017 249 497
seated posture 629 shifting time 983 SHIVA 899 sick building syndrome 587 skill aquisition 161 skin temperatures 817 slips 515 software 27,225,337, 355 software-ergonomics 361 space-vehicle control 311 specification 71 speech interface 529 speech recognition 535, 541 standard 361 standards 655, 885, 891,895 stationary metaphor 155 stress 675,681,705, 833 stressors 693, 699 subjective measurement 279 supporting system 405 symptom 575 system development 103 tailorability 995 task model-system model 489 taylorism 995 template model 483 TFF/LCD 661,667 theorem proving 217 think-aloud 375 tool 381 tool-based interface design 463 training 311 transformation 865 trouble 317 typeface 135 understanding 97 uric 785 usability 299, 323, 337, 343, 349, 355, 399 usability measure 387 usability testing 375 usability testing 381 usability testing 393 user 115, 123 user identification 237 user modeling 471 validation 899 VDT 575, 599, 983 VDT workplaces 749 vertical horopter 611 video conference 243 viewing distance 611 virtual auditory screen 803 vision-based 193 visual communication 167
1067 visual memory wavelet winking work load workflow workspace
509 777 911 817,823 859 33
This Page Intentionally Left Blank
