Communicating with Smart Objects
This page intentionally left blank
INNOUATIUE TECHNOLOGY SERIES INFORMATION SYSTEMS AND NETWORKS
Communicating with Smart Objects
edited by
Claude Kintzig, Gerard Poulain, Gilles Priuat & Pierre-Noel Fauennec
London and Sterling, VA
First published in France in 2002 by Hermes Science entitled 'Objets communicants' First published in Great Britain and the United States in 2003 by Kogan Page Science, an imprint of Kogan Page Limited Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms and licences issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the undermentioned addresses: 120 Pentonville Road London N1 9JN UK www.koganpagescience.comM
22883 Quicksilver Drive Sterling VA 20166-2012 USA
© France Telecom R&D and Lavoisier, 2002 © Kogan Page Limited, 2003 The right of Claude Kintzig, Gerard Poulain, Gilles Privat and Pierre-Noel Favennec to be identified as the editors of this work has been asserted by them in accordance with the Copyright, Designs and Patents Act 1988. ISBN 1 9039 9636 8
British Library Cataloguing-in-Publication Data A CIP record for this book is available from the British Library.
Library of Congress Cataloging-in-Publication Data Objets communicants. English Communicating with smart objects: developing technology for usable pervasive computing systems / edited by Claude Kintzig ... [et al.]. p. cm. — (Innovative technology series. Information systems and networks) Includes bibliographical references and index. ISBN 1-903996-36-8 1. Electronic apparatus and appliances—Automatic control. 2. Telecommunication systems—Technological inovations. 3. Wireless LANs. 4. Remote control. I. Kintzig, Claude, 1948- II. Title. III. Series. TK7881.2.O25132003 621.3815'4-dc21 2003013875
Typeset by Kogan Page Printed and bound in Great Britain by Biddies Ltd, Guildford and King's Lynn www. biddies.co.uk
Contents Introduction: The Role of Smart Devices in Communication Bruno Choquet
vii
Part 1. Interaction
1
1. New Distributed and Active Tools and Narrative Activities Franchise Decortis, Claudio Moderini, Antonio Rizzo and Job Rutgers
3
2. Smart Houses and Dependent People: Acceptability, Solvency and International Tendencies Chantal Ammi
9
3. Towards Multimodal Human-computer Dialogue by Intelligent Agents Patrice Clemente
17
4. Multimodal Interaction on Mobile Artefacts Laurence Pasqualetti, Laurence Nigay, Moustapha Zouinar, Pierre Salembier, Guillaume Calvet, Mien Kahn and Gaetan Rey
39
5. The Voice as a Means of Humanising Man-machine Interfaces Noel Chateau
47
Part 2. Software Infrastructure for Smart Devices/Ambient Intelligence Gilles Privat
57
6. Introduction to a Middleware Framework Vincent Olive and A. Vareille
61
7. A Model and Software Architecture for Location-management in Smart Devices/Ambient Communication Environments Thibaud Flury, Gilles Privat and Naoufel Chraiet
71
8. A Software Infrastructure for Distributed Applications on Mobile Physical Objects Mohammed Ada-Hanifi, Serge Martin and Vincent Olive
91
9. Integrating a Multimedia Player in a Network of Communicating Objects Jacques Lemordant
103
10. Reverse Localisation Joaquin Keller
111
Part 3. Networking Technologies for Smart Objects Pierre-Noel Favennec
117
11. Wireless Techniques and Smart Devices Jean-Claude Bic
121
vi
Communicating with Smart Objects
12. Wireless Local Area Networks Philippe Bertin
135
13. Radio Links in the Millimeter Wave Band Nadine Malhouroux-Gaffet, Olivier Veyrunes, Valery Guillet, Lionel Chaigneaud and Isabelle Siaud
159
14. Propagation of Radio Waves Inside and Outside Buildings Herve Sizun
169
15. Ad-Hoc Networks Patrick Tortelier
201
16. INDEED: High Rate Infrared Communications in the "Indoor" Context Jean-Christophe Prunnot, Adrian Mihaescu, Christian Boisrobert, Pascal Besnard, Pierre Pellat-Finet, Philippe Guignard, Frederique De Fornel and Fabrice Bourgart
209
17. Artificial Materials for Protected Communications Frederique de Fornel, Rabia Moussa, Laurent Salomon, Christian Boisrobert, Herve Sizun and Philippe Guignard
221
18. Free-space Optical Communication Links Olivier Bouchet and Herve Sizun
229
Part 4. Evolution of Smart Devices Claude Kintzig
243
19. Mobile and Collaborative Augmented Reality Laurence Nigay, Philippe Renevier, Laurence Pasqualetti, Pierre Salembier and Tony Marchand
247
20. Towards a Description of Information-seeking Tasks Contributing to the Design of Communications Objects and Services Andre Tricot and Caroline Golanski
257
21. Making Context Explicit in Communicating Objects Patrick Brezillon
273
22. Dynamic Links for Change-sensitive Interaction Philip Gray and Meurig Sage
285
23. Communicating Devices, Multimode Interfaces and Artistic Creation Guillaume Hutzler, Bernard Gortais and Gerard Poulain
293
24. Powering Communicating Objects Didier Marquet
311
Conclusion: From "Things That Connect" to "Ambient Communication" Gilles Privat
325
Index
339
Introduction
The Role of Smart Devices in Communication Bruno Choquet France Telecom R&D, France
Will communicating objects be the invaders of tomorrow? One speaks of it, one hears of it, but does one know what they will be, what they will represent, what they will do? What are "smart devices"? The concept of a device is commonly understood as that of a physical object composed of mechanisms, hydraulics, electronics, data processing. Its capacities depend on the whole or part of these components that will bring certain degrees of life to it and will produce an operational tool. Communication, experienced by such objects, is characterised by transfers of data which will take on a conversational aspect if the data involves action by the object which receives it, it having capacities of emission which will make react with another object. And since one speaks about communication and interactivity, why not introduce into this system such concepts as communicating entity? This intervention causes an opening up concepts like multitude, synchronisation or time sharing, human specificities and characteristics (emotion, mood, capacity of analysis and synthesis, intelligence, memory, adaptation, etc.) and therefore a large range of parameters in which the interest will be to be used as models with the physical objects.
CyberMonde CyberMonde is a research programme of France Telecom R&D, sponsored by the Scientific Direction. It is intended to take possession of new technologies to able to ensure the advances necessary for France Telecom and its business units and to be an engine of the innovation in: •
co-ordinating research around some great changes of technology and usage;
viii
• •
Communicating with Smart Objects
suggesting vision for their impacts for the future of the services and the networks; accelerating the transfer towards the market, preparing and testing in time unsuspected innovations from these changes.
The CyberMonde program addresses the general set of themes of the 'virtual environment' characterised by these guidelines: • • • •
to always be able to be everywhere and in a capacity of communication; to develop environments (and the associated interfaces) the communicating of which is based on smart physical devices (sensors, actuators); to be able to immerse itself in real, augmented or virtual spaces; to be able to project itself remotely in adapted forms (tele-presence, clones and avatars).
Two major objectives emerge: •
•
not to restrict CyberMonde to virtual environments only, but to consider as natural and real that contents that it is image, audio, interaction, etc., intervene and must intervene in any communication system; to keep in the centre of our concerns the communication dimension which constitutes the main goal of France Telecom and to have in line of sight the enduser, i.e. the human being (but not only) which will place the functioning of these environments at his disposal.
The two major objectives of the CyberMonde program take on again the traditional comprehension of the cyberspace which combines virtual and telematic reality and adds ubiquity, quasi-permanent, real or simulated presence (teleportation): *
To implement virtual environments within the traditional context, i.e.: to divide and make live joint virtual and real elements, to envisage the access and diffusion modes of shared information, to put in place the methods and tools adapted to the preceding tasks, to develop suitable supporting technologies, to adapt to the material constraints (networks, terminals, etc.).
•
To invent new fields of communication: to support all the modes of interactions, to offer forms of presentation, with the help of the available material, to free itself from the place and time.
Introduction
ix
Smart devices Smart devices will support interaction between CyberMonde and the environment: it is their main task; but also they will facilitate the internal evolution of CyberMonde and why not the environment itself. Indeed, CyberMonde, if it lives periodically in a closed area, needs to be built or to be reconstituted according to its own rules; also, the environment is subjected to conditions (forced or enrichments), free or not, which modify its own parameters. In reference to the proposals (incomplete at the date of writing), the role of smart devices appears diversified, made up however of more or less unifying poles of interest: •
•
•
•
•
•
•
management of its primary needs: that is a very basic role, but necessary to be maintained in life and to be able to fulfil the external requests for which the object is intended. The survival of the device should not depend on its capacities to feed (to increase its energy) or to carry out its own maintenance, but this must be decided on if it would become useless. letter box; the situation of the passive device or the device which fills, without fail, always the same functionality. Increased use can possibly occur, but it is known in advance. In this category, one will find probably sensors of information, displays, switches. at a higher level, the tool for specialised storage carries out classification of the data and information which will require greater software power and the setting in place of protocols, dialogues and indexing procedures, with human supervision. decentralisation of the processing capacities of information answers other problems and even corresponds to a certain vision of the organisation of the system, but this track becomes possible thanks to the performances related to the memories, the software developments of architecture and management, etc. This field develops, depending of technological progress and scientific projections, and will allow better adaptation to the specifications. a higher level step: attention carried to context. One finds the functions presented there previously, but also of the capacities of fusion of data and interactivity. The standard example is the physical localisation of a user. But one will easily imagine the need to obtain from it the identification and thus the identity. The access to local information also forms part of the context. more subtle will be the device which will succeed in perceiving the environment, but not only the physical environment or the data-processing links. To perceive the environment requires to be alert to an unspecified event, but also to analyse data which predict relations between communicating entities and especially the way in which each one perceives the other. This introduction of the relational fields is not without reference to the first steps of work on emotion. an additional element of the smart device will be its potentiality to carry out a behavioural analysis of the devices and environment which surround it. This
x
Communicating with Smart Objects
analysis will bring an obvious advantage since it is a question of communicating, to co-operate, co-ordinate, connect, and therefore to connect and to interact. These topics come up more and more frequently, but undoubtedly not yet enough. There is surely a gain in taking as a starting point the mono and multi-agents models. •
•
the device, companion of the communicating entity which is the individual, has the characteristics of its master and puts in place capacities of selection of information (at entry) and recopy (at exit) to assist and accompany the individual; to finish, but undoubtedly is not yet one of prime importance, one will evoke intelligence. But what intelligence for a device? Undoubtedly, software agents are an answer.
Elements, in particular chapters dealing with semantic and emotional aspects, will appear elsewhere. Other fields deserve to be studied, for example, the autonomy and the decision-making power of an object, the takeover of an action, proactivity, etc. Is it false to think that current efforts are not centred on these last subjects? Or is this an ignorance of the work of laboratories? Is a smart device only physical?
Parti Interaction
This page intentionally left blank
Chapter 1
New Distributed and Active Tools and Narrative Activities I
Fransoise Decortis1 , Claudio Moderini2 , Antonio Rizzo3 and Job Rutgers4 1FNRS Universite de Liege, 2Domus Academy, 3Universita di Siena and 4Philips Design
1. Introduction Situated among the technologies of the future, the development of one invisible, distributed and ubiquitious technology was the aim of a research project on which we currently work within the framework of project POGO (European Program 13 Intelligent Information Interfaces, Exploring New Learning Future for Children). This technological orientation rides on the concept of affordance as perceived by objects (Gibson, 1977) and of apparatuses of information (Norman, 1998), and considers the way in which the instrument supports the task so that it becomes an integral part of it, as if it were a natural extension of the human and his work. That implies a specialisation of the function so as to be in perfect agreement with real needs of the users, and to offer great simplicity and transparency. According to this philosophy, instruments will not be recognisable as such any more, so they will disappear from the sight and the consciousness of the humans. The movement towards an invisible technology (Norman, 1998), is manifested in instruments designed and thought out within the framework of project POGO - active instruments, new semiotics tools which should fit in a harmonious way into traditional instruments where any centralised unit of data processing disappears from the sight and consciousness of the user.
1.1. New active tools Six tools forming the POGO system were designed and evaluated. Beamer POGO is one tool which makes it possible to visualise and capture the purposes or appearance of the physical world and to import them into the virtual world by, in particular, projecting them directly on the screen. The beamer includes/understands a screen
4
Communicating with Smart Objects
sensitive to the touch and allows the children to draw on it, to write as if their finger were a pencil and also to capture various type images and to record them on a card. The cards' pastiches are memories of elements and of the made up images. The children can record elements there by acting on the beamer. Other cards contain predefined and preregistered elements (sky, sea, landscape). There are also cards which contain sounds. The flexible screen is used to visualise the basic images contained in the cards. The screen is provided with three small pockets which contain card readers which correspond to three different positions on the screen. The screen flexible device can be fixed to the wall or used on the ground. There is also an integrated device which allows the children to change the colour of the projection. The sound carpet is a carpet which allows them to play with sounds which are contained in cards. By inserting a card in the card reader, the children activate a basic sound of the environment which is played in loop plus a series of specific sounds which are activated on pressing on the various zones of the carpet. The tool voice enables the microphone to create distortions of the voice. At this moment it is possible to render the voice sharper or more serious in tone. The mumbo makes it possible to read an image contained in one card and to project its contents onto the flexible screen. It includes functionalities of zoom and rotation which makes it possible to make the elements move on the screen.
1.2. Active tools prototypes The philosophy of design of POGO instruments foresees simple tools affording the stimulation of sensory effects and a great reactivity with environment. They aim at a flexible and harmonious integration between the physical world and virtual world and, in the socio-cutural theoretical orientation, support with development of the narration as a vehicle of the direction and with the interaction between development of the narration as a vehicle of the meaning and with the interaction between world of the child and the images, models and significances existing in the culture. Their central question is to know with which future instruments could one provide advantages compared to what is done already in the school environment where training goes through an appropriation of meaning, with work on the emotion, imagination, exploration and social externalisation. We are interested in the effects of introduction of active and distributed new instruments on the narrative activities in a school environment. How do POGO instruments transform the activity of the children when they create stories?
New Distributed and Active Tools and Narrative Activities
5
The above shows the new tools of POGO: Beamer, the mumbo, the assembler, the microcomputer, the mobile camera, the sound carpet, and the sequences recorder.
6
Communicating with Smart Objects
2. New potentialities What are the new potentialities offered by the POGO system? The system allows the child to enter physically into the story thanks to the use of the table camera and of the camera. This camera also makes it possible for the children to be seen in activity. Their reflexivity to activity of narrative construction thus becomes possible and encouraged. The beamer is also used by the children to photograph part of their body and to modify, thanks to colour features or the morphing, and to add the effects of distortion of the images, and being of real play interest for the children. The beamer thus allows one, following the example of the camera, to project the body of child in the image, and a fortiori in history. To write with the fingers seems particularly appreciated by children. The Pogo system enables integration of the traditional tools for narrative construction (paper-pencil, drawings, account), by opening the field of new possibilities: disguises, sounds, vocal effects can be created, projected, combined. The introduction of several instruments being able to be simultaneously used to capture, handle and to combine images and sounds enables the increased participation of all the children in the construction of the contents history. The simultaneous use of the tools also introduces a more individual dimension into the narration: each child could at the same time take part in collective creation and give its contribution personal, if it wishes. The construction of history becomes multimode: video, virtual and real images, sounds, voice, can be combined, dissociated, be worked separately and simultaneously. The introduction of sound as a narrative element increases the expressive possibilities of Pogo: children can give their voice to the characters, improvise dialogues, recreate sound atmospheres, etc.
The system is user-friendly. There are analogies between the form of tools and their effects and average use. The functionalities of the system are distributed in a clear
New Distributed and Active Tools and Narrative Activities
7
and simple way among each tool. The system is appropriate for children from 6 years of age , who quickly apprehend the operation of the tools. Construction of history is not a work for the children, but take on an play aspect. School becomes a space of plays and discovery. The system encourages communication and co-operation among the children, necessary elements to make a real co-ordination of all the elements of history present at the same time. The system encourages the inspirational phases and production, opening up an enormous field of possibilities in the style of the stories, the manner of building, and in the various media backing the activities. In addition, we observe that the use of the instruments increases the collective dimension of the creative process and in particular the diversification of the roles and their participation. Finally the instruments support the children in the structuring of narrative to produce richer stories.
3. Perspectives What are the development prospects for the tools? The results indicate some inadequacies concerning the use of the instruments by children in class which seem to us to be of interest. It seems to us that the role of the teacher could better be supported by the Pogo system in particular with regard to the phases of exploration and production. Mobile, portable and wire-less tools allowing the capture of moving images and sound outside the classroom would enrich the quantity and quality of the experiments which the children can record and re-use in creation of narrative. With regard to the narrative structuring, the system seems to support and even improve the organisation of the history according to the Labov (1972) model. We estimate however that an evolution of the tools with more open methods of use could enrich the expressive potential of the children -a functionality of zoom and mobile camera. Compared with the prototypal precedents the possibility of zooming to obtain large plans of the elements of scene and a mobile camera allowing one to photograph the purposes of various points of sight offer to the children the possibility of developing richer narration, varied at the visual level. They can highlight, for example, the face of one character through a large plan or to change the catchment of sight into agreement with the point of sight of the character, etc. Our results also state that the use of the instruments does not seem to create interference with the activity they being integrated into the existing instruments; the beamer for example becomes a scheme of work, ideas collected outside or produced by children can be integrated in the system and to be thus developed. Moreover, the instruments are simple to use. Each action generates an immediately visible effect (for example creating purposes on the beamer is directly visible on the screen). The interactions are connected by the physical purposes. Those make for simple actions (by avoiding screen menus for example). These results return us to the concepts
8
Communicating with Smart Objects
suggested by Norman (1998) concerning information apparatuses, and to the fact that the tool is considered in the context of its way of supporting the task so that it becomes an integral part of it, as if it were an extension natural of the person and his/her work. That implies a specialisation of the function instrument as being in perfect agreement with the real needs of the users, and the offer of a great simplicity and transparency. Each tool is simple, request its proper method of operation. Each one must be learned, and to make it possible to carry out a specialised and appropriate task. We approach the idea according to which, in the long term, the instruments will not be recognisable any more in so much as they will form part of the task, so much that they will disappear from the sight and consciousness of the people. The distribution of the instruments in space seems to us also interesting. The use of the tablet directs us towards a possible incorporation of the units mnemonic in the physical purposes and of their handling in space (ie the possibility of transporting them, of re-using them in another space-time). Their handling of information is extended to space and is not thus any more confined to one centralised unit. The instruments also seem to us to go in the direction of one production located, the space of design and recording being integrated into the context of handling and of construction of natural purposes of the physical world suitable for children. These points testify thus to a movement towards an invisible technology.
4. References Eco, U. (1996) Six promenades dans le bois du roman et d'ailleurs. Grasset, Paris, France. Gibson, J.G. (1977) The theory of affordances. In R.E. Shaw & J. Bransford (Eds.), Perceiving, acting and knowing. Hillsdale, New Jersey, USA. Labov, W. (1972) Language in the inner city. University of Pennsylvania Press, Philadelphia, USA. Norman, D.A. (1998) The invisible computer, MIT Press, Cambridge University Press, Massachussets, USA.
Chapter 2
Smart Houses and Dependent People: Acceptability, Solvency and International Tendencies Chantal Ammi Department of Management, National Institute of Telecommunications, Evry, France
1. Introduction The appearance of new technologies has modified normal life. Home is becoming an intelligent open space adapted to people who live inside and are able to accept new systems. Integration of new technologies can help dependent people to stay at home as long as they want, and to help reduce their feelings of dependence.
2. Context During the last twenty years, the number of disabled has increased: in 2000 more than 23 millions in Europe (all types of handicaps), for two main reasons: • •
the number of elderly has grown; some of them, specially those 75 and over have more or less the same needs as the disabled; the progress of medical and associated tools have enabled people, especially young people with severe injuries or neuro-muscular dystrophy, to be saved.
These motor disabled persons have, very often, maintained their intellectual capacity. They could work, learn and live full lives if they could have the chance to live in an adapted environment. The adaption to an active and socially full life is very important from a number of points of view: •
Psychologically, to avoid the tendency to stay alone, to retire from the real world and to permit the maintenance of intellectual and physical capacities;
10
Communicating with Smart Objects
Economically, to help decrease dependence costs: hospitalisation, rehabilitation etc and to obtain a regular income; Socially, to include disabled people in a normal way of life. However, deficiencies exist and generate incapacity and disadvantages at home, at work and in the street. Rehabilitation technical aids contribute to compensate for these deficiencies. The use of new technological tools and means (smart house, telecommunication, computers etc) and their applications in normal life can reduce or eliminate these disadvantages. Used by normal people, these products allow dependent people (disabled and elderly) to decrease their dependence and to be able to: • • • • •
open and close doors, windows, shutters etc.; switch lights on and off; use appliances: TV, recorder, microwave, fridge etc.; use means of communication: phones, fax, computers; use various aids.
We can distinguish five types of rehabilitation technical aids: Mobility
—> wheelchairs, adapted cars
Environment control
—> smart house
Remote manipulator
—> robotics
Communication
—> devices, interfaces
Access to computers
—> adapted computers
To be useful and therefore used, rehabilitation technical aids must be adapted to the user needs.
3. Method of analysis The purpose is to evaluate, to modify and to develop existing or new products. These products must have the following characteristics: be useful. be adapted to needs;
be accept financially.
Smart Houses and Dependent People
11
To verify these different characteristics, it is necessary to list the functions of the products in order to analyse them: • • •
analysis of needs; analysis of costs; analysis of market.
3.1. Analysis of user needs Technical assistive aids can be resolved into 3 types according to special needs: • • •
need to control systems for greater comfort and increased security; need to manage equipment in environment; need to communicate with home, surroundings and office, and to have access to outside services for greater comfort and fun.
The satisfaction of the users depends of three variables: needs, adaptation and the accessability of the products. The consideration of needs, before the development of the products, needs a good knowledge of the disabled world with the different types of handicaps, the restrictions of possibilities and the capacities of adaptation and acceptance. Only a collaboration between all the actors (therapists, user associations, rehabilitation centres, engineers) can gain knowledge of all the variables to enable the best solution to be evolved. Depending on the situation, a study of the needs will be made before any further development can take place and will be completed by frequent iterative steps which will permit integration into the prototypes the results of the collected data. In spite of this integration of the special needs, we notice very often a maladjustment between the results and the needs and some difficulties in acceptance of the developed products and systems. In the case of motor disability, the end-users are obliged to use several type of assistive aids to compensate lost functions such as mobility, object telemanipulation and communication. Because of non standardisation between the different types of assistive aids which exist on the market, the users, who are severely disabled, find themselves confronted by many user interfaces. To decrease or to avoid these problems and to permit a better fit with the needs of disabled people, an evaluation phase is necessary. Conducted by therapists and ergotherapists, the evaluation facilitates integration of some valuable modifications (technical, ergonomic etc) to the final product and the ability to find adequate access interfaces, such as computer devices, voice recognition, sensors etc.
12
Communicating with Smart Objects
3.2. Analysis of costs But, in spite of these new applications, which allow reduced dependence for handicapped persons in their way of life, it is necessary to consider the problem of cost and economic opportunities to permit real access of these products for the users. To avoid extra and costly functions, we have to verify with qualitative and quantitative criteria: • • •
the duration of utilisation; the frequency of utilisation; the learning capacity.
For each function, we have to calculate the direct and indirect costs with the following aspects: • • • • • • • •
development; ergonomic; evaluation; communication; dissemination; technological research; market aspects; fabrication.
The costs are compared to the needs and the utility of the function. Too high a cost can have two origins: A higher cost: then, it is necessary to reduce direct and indirect charges and all the actors must be implicated in the new specification of the functions. Unsuited utility of the function: we can decide either to cut functions or to have the opportunity to put options on them.
3.3. Analysis of the market The relationships between user and technical aspects are based on the choice of two different strategies: Standardisation: the dependent user is considered to be a new opportunity for an existing market. The same technology and main functions are maintained and only the types of access are modified. The size of the market is large, but the utility and the use are low;
Smart Houses and Dependent People
13
Specialisation', the creation of new products where the dependent user is the only and final consumer. The utility and the use are respected, but the size of the market is too small to make profitable the development.
4. Remaining problems But in spite of real market opportunities, two main problems still remain and can explain actual difficulties for a real emergence of technical assistive aids: • •
the acceptability of the users; the affordability and economic rationale.
Leaving to specific details, these types of products may be conceived, developed and proposed in a productive sequence as shown following: Specification of Needs Determination of Target Conception and Production of Products and Services Potential Demand Problem of Acceptability Revealed Demand Problem of Solvency Real Demand Purchase
4.1. A cceptability To satisfy the demand of dependent people (disabled and the elderly), products and services are more and more often conceived according expressed needs, after long and expensive market study. But, in spite of this analysis, a part of the demand is still not satisfied or not concerned by all the products. The phenomenum of acceptability can explain the differences between declared needs and their acceptance by the users. Many criteria can be integrated: • • • • • • • •
cultural, sociological an religious environment; level of social life; technical acceptance; age of the users; profession of the users; medical and dependence situation; the weight of user association; the level of development;
14
Communicating with Smart Objects
• •
the rule of the public institutions; the size of the market etc.
The three following examples can explain the situation: •
•
•
quadriplegic, living alone at home, need assistive aids to prepare meals, to switch on TV or recorder etc. Some products are adapted to their handicap and can help them, but an individual's way of life, their culture, their personal history can lead them to reject the product. older people, especially dependent ones with problems of mobility and memory, prefer to stay at home instead of going to special institutions. Special equipment such as alarm and surveillance services exist, but they can be rejected because they symbolise age and dependency. religious aspects can explain a better acceptance of technologies. Protestants have a different conception of society from Catholics and in general they use technological products more readily.
4.2. Affordability After the transformation of needs into real demand a last step must be solved before the purchase of products, ie the financial resources of users. Despite public and official promises, most countries have not integrated into their social assurance system the financial contribution to finance for technical assistive aids. These products and services have a cost, and a limited market does not permit a decrease in the selling ie "ladder economies". In Europe the analysis of two different countries reveal the importance of the public financial contribution. In The Netherlands, technical assistive aids such as environment control and robots are supported by the public social assurance system as are medical expenses or pharmaceutical products. Special sites1 are implanted to fix these technical assistive aids on the wheelchair for instance. The consequences on the market are extremely positive2 and there is a real emergence of technical assistive aids for the benefit of disabled and older people. But in France, public social assurance is oriented to medical and pharmaceutical expenses. Technical assistive aids do not benefit from public financial support. Users who cannot finance these products and services themselves must find private
2
Hetdorp in The Netherlands. Sales of Manus are very significant in The Netherlands, more than hundreds.
Smart Houses and Dependent People
15
support eg users associations3. Users are left alone to manage integration of all these products in order to counter discomfort, disruptions, compatibility problems etc.
4.3. The international market In spite of a rising number of dependent people all other the world, and the actual tendency of the industrial sectors to resolve problems and to propose products and services not for a local market but for in international one, we can ask the following questions: • • • •
Can we speak of an international or local demand? Is the demand similar everywhere? How to measure the phenomenum of acceptability? How to solve the problem of solvency?
5. Conclusion In spite of a rising number of dependent people, too many potential users cannot benefit from adapted tools or a special environment which could permit them to decrease their dependency or to ameliorate their actual way of life. The main reasons are the disjunction between offer and demand and the lack of financial structure. Analysis of the international market can permit avoidance of errors in some countries and to provide orientation for the researchers.
6. References C. Ammi, The market of technical assitive aids, tendencies, problems, necessary adjustments, the French case, AAATE Llubjana, Slovenia, September 2001. C. Ammi, Problems in technical assistive aids, RAATE, Birmingham, Great Britain, November 2001. C. Ammi, Telecommunication and dependence, Hermes Sciences and FTR&D, Paris, April 2002.
3
AFM (French muscular dystrophy users association) and APF (French association for motor disabled), for instance.
This page intentionally left blank
Chapter 3
Towards Multimodal Human-computer Dialogue by Intelligent Agents Patrice Clemente France Telecom R&D, Lannion, France
1. Introduction Mobile telephones, PDA, GPS, communicating clothes, infra-red connections, Bluetooth technology, domestic networks, domestic robots, software agents, etc. The list is long. It is necessary to face the evidence: communicating objects have already started to invade us, and that will continue. This increasing volume foreshadows many problems of interaction for the future, between man and these objects, and between the objects themselves. Moreover, the absence of standards of communication between objects will lead to a multitude of protocols, and to interoperating problems between communicating objects. The difficulties of coherence and cohesion between these objects, due to their number and their autonomy, will generate unexpected and undesirable behaviour from the systems or networks of objects. The respect of the free-referee and the integrity of the individual will be complex to guarantee. Communications are inevitably increasing in number, and information of all kinds will submerge users, if they do not have intelligent and suitable mediators. If no precautions are taken, systems will thus become useless, or unusable. To avoid these pitfalls, one has to keep control of objects and systems. This requires from the latter to precisely "understand" the desires and needs of the user, an indispensable condition to satisfy them, and this, whatever the media used or wished for by the user. Objects and systems have to answer to the requests of the user and adapt their behaviour according to his/her personal profile, to the context (situation, history, etc), and to the type of task.
18
Communicating with Smart Objects
Intelligent agents, autonomous software entities, are able to reason, act, and bring interesting solutions to those problems. An intelligent agent is able to perceive and act on its environment. Thus, it can control "unintelligent" communicative objects such as actuators. Moreover, an intelligent agent can communicate, when provided with dialogue capacities. It can thus interact with other dialoguing agents, which for example deliver information or are integrated in communicating objects. The means of communication used is therefore an inter-agents' language of communication. Finally, an intelligent agent can converse in a natural way with humans. Throughout the dialogue, it can help people achieve their goals, deliver relevant information to them, carry out a certain number of actions (possibly for them), supervise their resources and all this in a dynamic way and upon request. In this case, the means of interaction traditionally used is a natural language, such as English. A system in which an agent is introduced benefits at the same time from the agent's intelligence. The agent constitutes a comprehensible and co-operative interlocutor. It can represent, for example, an assistant or a personal secretary and then will learn from its owner's specificities, and adapt to them. The agent can play the role of mediator and preserve the user from all kinds of intrusions from his/her environment, like undesired or non prior information. For example, when entering a store, a user does not systematically want the items compatible with the shopping list to appear on the PDA. When approaching objects in this same store, he may not want his/her PDA to indicate prices either, although this function is always available. We will develop into more detail a particular application of intelligent agents: human-computer dialogue (HCD). More particularly, we will treat multimodal HCD. Initially, we will point out traditional approaches of HCD and current multimodal HCD. We will then come to the gist of this chapter: the phenomenon of multimodal referring to objects. After having recalled the main problems of linguistic and multimodal referring, we will introduce our formalism for multimodal referring, 1
This language can be ACL (Agent Communication Language) proposed by the FIPA consortium. ACL is founded on the formal definition of communicating acts between agents, making it possible to carry out unambiguous interactions. " i.e. using several communication modalities. The communication modalities are defined by the structure of information which they convey (linguistic, graphic, haptic, etc) and their intrinsic properties. As they are linked with communication modes (acquisitive and productive modes), it is possible to classify them into inputs and output modalities (see [BER 97] for a survey on representation modalities).
Towards Multimodal Human-computer Dialogue
19
made possible by an original representation of objects. We will show a theoretical model of a multimodal referring act, illustrated by a short example. We will conclude with technical remarks on our model and its implementation and general ones on systems which it will allow to develop.
2. Human-computer dialogue 2.1. Various approaches There are various points of view concerning modelling and implementation of HCD sy systems3. Structural approaches assert the existence of an interaction structure, built on the regularity of exchanges appearing in a dialogue. According to these approaches, this structure is determinable a priori, and would be established in a finished way (see [SAD 99] for a recall of these approaches). Differential approaches consider a dialogue as the realisation of one or more communicative acts. Based on the principle that to communicate, is to act [AUSTIN, 1962], they start from the idea that communicative actions, following the example of classical actions, are justified by goals and are planned in this way. In particular, these goals relate to the change of the mental states (of the interlocutors), represented in terms of mental attitudes, such as knowledge, intention, uncertainty. These approaches consider the dialogue from more general models of the action and the mental attitudes (see [SAD 99] for a range of these approaches). Rational communicating agents fall under this approach, while insisting on the vision of natural and user-friendly communication as an intrinsically emergent aspect of intelligent behaviour (see [COM 90], [SAD 91], [COH 94] for the primary works and [SAD 99] for a more exhaustive overview). This approach makes the problems of flexible and co-operative interface design and intelligent artificial agents overlap. Flexibility appears by an unconstrained dialogue which makes it possible to evolve in the interaction freely, to deviate from the consensual behaviour of conversation, in order to, for example, signal a problem to the system to possibly remedy it. Cooperation appears in many forms: reaction to requests, adoption of user's intentions, sincerity of the system, relevance of the answers, etc. The rational communicating agent constitutes the core of the human-computer dialogue system. To establish the link with the user, the agent, which is nothing else 3
Conscious of the heaviness of 'a computer system whose human-computer interface is dialoguing', we will use, from now even if it is an abusive language 'dialogue system' or 'human-computer dialogue system'.
20
Communicating with Smart Objects
than a program, uses a certain number of interfaces made up of more or less physical and tangible communicating objects. Figure 3.1 illustrates this matter with several layers of communicating objects. The "high" layer, the agent one, communicates with the transition layer made up of recognition and synthesis systems. This layer is in relation with a lower layer which represents physical media of interaction (such as peripherals and physical communicating objects). The last layer is embodied by the user, since he/she can also be seen as a communicating object.
2.2. Multimodal human-computer dialogue The first systems of Multimodal HCD made it possible to dialogue in written natural language, for three major reasons. When those systems appeared, interaction devices were limited, typically to a keyboard and an alphanumeric screen. One thought that natural language was the prevalent means of dialogue, the most effective to exchange information and to understand one another. Since Turing ([TUR 50]), natural language was seen as the ultimate demonstration of intelligence. It is well known today that this is not true. Many other dimensions come into play during interactions and dialogue. Natural language in fact constitutes only one component of dialogue. Gestures, postures, gaze, facial gestures, prosodic cues, effects, cultural dimension, micro-social proxemic and some others are quite as important [COS 97].
Towards Multimodal Human-computer Dialogue
21
Figure 3.1 Layered model of communicative objects involved in a multimodal human-computer dialogue
For example, gestures are involved in thinking as much as language [MAC NEIL, 1992]. They can illustrate some mental images in the scene of speech, which the language cannot always do in a satisfying manner4. They make it possible, in other cases, to replace word groups and/or word traits5, thus dispersing the message on two communication modalities (which are natural language and gestures). New technologies bring new media of communication. They open a potential new way towards other modalities of interaction with the computer. Thus, to the old (but always useful) keyboards, mice and screens are added voice recognition, gesture recognition, gaze tracking, haptic devices (tactile screens, data gloves etc), natural language generation, graphic and image synthesis, voice synthesis, talking faces, virtual clones (see [BEN 98] and [LEM 01] for a broad range of these systems); it is already possible to convey fragrances on the WWW!
4
These gestures are called illustrative gestures. These gestures can be either called illustrative gestures or emblematic gestures (see [COS 97] for a survey on communicative gestures). 3
22
Communicating with Smart Objects
In order to build HCD more user-friendly and natural to use systems, much work over the past ten years has tried to benefit from these new technological potentials in trying to conceive multimodal HCD. The stakes are numerous. The user should be able to converse naturally, using the modalities he wishes6, to switch from one modality to the other (transmodality). The emotions expressed by the prosody of his voice or his facial gestures should be understood and taken into account. Symmetrically, the system is expected to answer using the best modalities: the selected modalities should be the most effective for the type of information to convey, those preferred by the user and the emotional dimension should also be present (see Chapter 5), etc.
3. Referring during dialogue 3.1. Problematics 3.1.1. Linguistic reference Natural language HCD systems make it possible for the user to question a server, or carry out research. For example, the user can manage his diary, his share portfolio, consult the weather forecast, etc. For all of these tasks, the dialogue relates to objects of the world7. Speaking about an object (e.g. using nominal groups) constitutes a linguistic reference to the object. In fact, in any dialogue, interlocutors talk about something, thus carrying out references. Searle, in his philosophical theory of language acts ([SEA 69]), goes further. He states that: When an agent performs an illocutionary act8, he performs by the fact referring and predicating acts. This means that each sentence consists of references (to objects in particular) and predications (for example to specify properties of these objects). Many works deal with the comprehension and the generation of linguistic references in dialogue systems. For the comprehension of referential expressions, one is confronted with the problems involved in voice recognition in vocal systems. For mono-speaker systems, a user voice training by the recognition tool is needed. For multi-speaker systems, 6
See [BEL 95] for an exhaustive definition of the various types of multimodalities. Object in the broad sense, i.e. physical, conceptual or virtual entity. For example, an Email address is an object of the world, as well as the car of the neighbor opposite. 8 An illocutionary act is the act achieved by the production of a succession of signs in a social relation context, which expresses an intention ("to inform", "to request" are illocutionary acts). 7
Towards Multimodal Human-computer Dialogue
23
vocabulary size is limited, and the performances of recognition are therefore restricted. This implies the development of strategies of semantic completion of the recognised propositions [CAD 95], a technique which has its own limits. Beyond the eventuality of speech recognition (which disappears with written natural language), the system tries to understand the reference, i.e. to identify the referent. To do so, it traditionally proceeds by satisfaction constraints (see [HEE 95]). The object descriptor refers to a whole of potential candidates. Each component of the expression brings a constraint, reducing the unit. Ideally, this process converges towards a single candidate but it is not always the case (none or several acceptable referents can appear). It is then necessary to make a clarification dialogue. Recent works [SAL 01] adopt a wider approach in modelling the mental representations of situation and the domain of reference. For the generation of referential expressions, other problems are encountered: the choice of the descriptors to be used for the reference [APP 85], [DAL 87], [REI 90], [REI 92], the calculation of unambiguous description [CLA 86], the referential collaboration which underlines the fact that a reference is often understood only after the succession of several references during a repair dialogue [HEE 91], [EDM 94], the co-presence of agents and objects [COH 81], [HEE 95], and the management of focus in dialogue [GRO 86], [REI 92]. Moreover, some works show the problems which dialogue systems have to solve to become a little more user-friendly. The majority of the current dialogues systems do not take into account space-time evolution of the world and are based on rather fixed representations of the world. The evolutionary referents (in time, space or their own nature) cause hard to model difficulties that current technologies cannot manage [PIE 97]. 3.1.2. Multimodal reference We think that the assertion of Searle (cf. §3.1.1.) applies to any type of communicative act, either linguistic, monomodal or multimodal. Thus, pointing at an object with the voice and gesture, constitutes a multimodal reference to that object. Multimodal referring should therefore bring a richness and additional expressiveness to HCD, in particular to exploit the properties of representation modalities.
24
Communicating with Smart Objects
3.1.3. Examples The two following examples underline all the potential of expressiveness, comfort and ease of use that multimodal references can bring. The first example presents a sample dialogue in which the user is talking to a computer. The second example illustrates several turns of natural language HCD. The last statement, produced by the computer, is a multimodal one. Comprehension of a multimodal reference by the communicating agent Let us take the example of a multimodal statement composed of a natural language (NL) sentence and of a gestural (G) designation on the screen. The HCD system allows the online purchase of vehicles at a car dealer. A user consults on a tactile screen the list of available automobiles, presented under the form of small photographs. He wants to know the price of a car (Figure 3.2). This multimodal utterance partly consists of references to an object, of automobile type. This one is referred, on the one hand, with a referential linguistic expression, by the demonstrative nominal group 'this car' which refers to instance_auto_21, a particular instance of the object category of automobile and, on the other hand, with a gestural deictic reference (gestural designation) which refers to the same object. There is therefore for this object a bimodal reference (or co-reference).
Figure 3.2 Multimodal utterance example. The modalities used are natural spoken language (NL) and designation gesture (G)
Generation of a multimodal reference by the communicating agent If a human being is intrinsically limited by his/her means of expression: voice and gesture, he/she can at best extend the modalities to writing, drawing, using charts, etc. However, the amount of time taken to exploit them is increasingly longer than for natural modalities. The intelligent agent does not suffer from this kind of problem and benefits from very large processing and storage computer capabilities. In a situation where a person would try to make a gesture to illustrate the content of his speech, the agent can replace this illustrative gesture, difficult to build and sometimes to understand, by a graphic visual representation strongly analogue9 to 9
See [BER 97] for an overview of analogue modalities.
Towards Multimodal Human-computer Dialogue
25
the original object or concept in question (a photograph for an object, and a diagram for a concept, for example). Figure 3.3 illustrates this phenomenon in a fictitious example of dialogue between a user (U) and his/her intelligent personal electronic assistant (S).
Figure 3.3 Fictitious example of a system multimodal utterance (white part of the HCD) 3.1.4. Difficulties Multimodal dialogue systems, as well as natural language dialogue systems, are confronted with problems directly or indirectly involved in the referring phenomena in multimodal HCD. First of all, things are not simpler for multimodal references than for linguistic ones. If intuition leads one to think that the possible system redundancies in input will allow it to make a more robust interpretation and avoid ambiguities, the facts are very different. This is partly due to recognition systems, which do not ensure optimal recognition. Thus, when two parallel messages coming from two different modalities are contradictory, it is difficult to know which one is erroneous. Fortunately, task and context can be helpful in this process. Then, certain problems relating to linguistic references remain within the framework of multimodal ones. It is obviously the case of the calculation of unambiguous object descriptions, identification of referent (and referential collaboration), co-presence of the agents and management of focus (all the more difficult as modalities are numerous) [CSI 94]. Other problems appear at the comprehension level of reference, like confusion between command gestures of communicative gestures, or between unintentional gestures and deictic ones [STR 97]; metaphor of display of the real world (confusion between the object and its representation) [HE 97]; temporal synchronism and the scheduling of the events.
26
Communicating with Smart Objects
The problems occurring in generation of multimodal references are the choice of descriptors on the selected modalities (this selection is related to the choice of modalities10 [REI 97]); the internal inference of attributes of a modality, starting from other attributes already known on other modalities or known of the categories of objects [AND 94], the metaphor of display, temporal synchronism, etc. 3.2. Multimodal mental representation 3.2.1. Psychological model The phenomenon of referring should be approached in a general way and the model of reference used by the agent has to be sufficiently close to that of the human one in order to enable him/her to plan the reactions of the computer just as he/she would do with a human interlocutor so as to continue to refer as he used to. In the same way, the software agent has to be able to envisage the interpretation of its reference by the user so as to build it accordingly. The interpretation of a reference is partly related to the knowledge of the recipient agent. This is why the object representation model (and its principle of dynamic construction) has to meet this need. Damasio [DAM 94] indicates that the mental representations of objects that humans build consist of perceptive elements acquired during sensitive experiments on these same objects. Let us take the example of a person z "known by sight" by a person j This person j keeps in memory a certain amount of perceptive information about z, like visual information (e.g. his face, his size, his style of clothes), like auditory information (e.g. the sound of his voice), etc. Using this information intelligently, j can build references to z in order to bring his/her interlocutor to identify him/her (i.e. z). The mental representations are also partly made up of linguistic object descriptors [PAI 69] that describe on the one hand the semantic category of the object, and on the other, its particular properties. Most of the sensitive and linguistic elements are redundant, because they are encoded in a double way by the phenomenon of dual coding [PAI 69], [PAI 86]. During perception, dual coding converts sensitive (resp. linguistic) elements into linguistic (resp. sensitive) equivalents and stores everything in memory.
10 It is known for example that a piece of geographical information is better conveyed by graphic than a piece of abstract information for which the text will be more appropriate.
Towards Multimodal Human-computer Dialogue
27
3.2.2. Computing model In order to formalise this organisation of object memory, we introduce, for an agent, the concept of multimodal mental representations (MMR). A MMR is a formal entity corresponding to the intuitive idea of multimodal mental representation of an object that an agent has. A MMR consists of a set of acquisitive object representations (OR), which appeared during linguistic and sensitive perceptions by the agent and of a set of productive OR produced for linguistic and sensitive references. All sensitive OR (acquisitive and productive ones) constitute the entire sensitive mental image of the object and all linguistic OR (also acquisitive and productive ones) constitute the entire linguistic mental image of the object. These two mental images constitute the entire MMR. OR make it possible to refer to the generic and particular properties of objects. Generic properties are in fact categorical descriptors of objects. Here is an example of linguistic categorical descriptors: "animal" —> "mammalian" —> "dog". Specific properties make it possible to code the particular attributes of the object. For example: "dog" —> "brown "-^"Droopy", etc. While using these OR, the agent will be able to build references. Our model of mental representation is to be brought closer to that of Appelt and Kronfeld [APP 87], [KRO 90] who used the term individuating set (Is) for mental representation. An Is is composed of intensional object representation^) (IOR), which represent11 the referred object, if this one exists. Appelt and Kronfeld defined two types of IOR: speech IOR which results from linguistic acts of referring in the discourse and perceptive IOR which result from perceptive acts of referring in the speech. We do not consider this model sufficiently precise for multimodal reference. The definition of IOR remains too vague, does not detail enough various possible natures of perceptive IOR (in terms of various modalities of interaction). As we just saw, we defined two types of OR, acquisitive OR (input of the agent) and productive OR (produced by the agent). Acquisitive OR occurs after an act of perception. This act of perception can be of two types. The first one corresponds to the sensitive perception of real objects. The vision of an object for example makes people perceive its form, its size, its colour, its aspect, etc. These sensory descriptors of the object will constitute acquisitive sensitive OR. The second one corresponds to the perception of linguistic references 11 We take the term of representation used by Maida [MAI 92], which we prefer at the original term of denotation employed by Appelt and Kronfeld.
28
Communicating with Smart Objects
in the speech. These linguistic descriptors of the object will constitute acquisitive linguistic OR. Productive OR appear when the agent tries to build a reference. In this case, there are also two possibilities. The first possibility is the computerised method of dual coding (cf. § 3.2.1.). However, there is a small difference which is that this method will only be triggered by need, i.e. whether for reasoning or referring, not during perception. In order to do that, we use a set of categorical and semantic associations between linguistic and sensitive descriptors. The second possibility makes it possible to generate new traits or properties of the object on the same modality, starting from generic knowledge either on the category of objects or on the domain, in a deductive way. Both methods can generate linguistic and sensitive OR.
3.3. Multimodal object representation Our approach makes it possible to combine OR related to potentially different communication modalities in a new multimodal OR (MOR). That thus gives the agent the capacity to use several OR in one MOR to refer to an object in a multimodal way. Formally, this combination carries out the semantic sum of OR components. Some rules specify certain characteristics of MOR (e.g. temporal layout of OR components). In order to model this capacity of combination, we propose the formal predicates Mor_combine (Figure 3.4). Mor_combine(mor, orj,..., orj is true if and only if the MOR mor is the combination of every OR from or; to orn which are all related to different modalities. Figure 3.4 Multimodal OR combination predicate 3.4. Act of referring The preceding prerequisite enables us to introduce a formal model of act of referring which can be integrated into a theory of rational interaction [SAD 91], on which dialoguing agents developed in FT R&D are based. This theory of rational interaction is founded on an integrated formal model of the mental attitudes and rational action, which makes it possible to take into account various components and capacities implied in communication. It is the case of
Towards Multimodal Human-computer Dialogue
29
rational balance, a relation established on the one hand, between the various mental attitudes of an agent (belief, uncertainty, intention) and, on the other hand, between its mental attitudes and actions. The communicative actions fit within this framework. They can be recognised and planned like traditional actions by the primitive principles of rational behaviour. Sadek proposed some models of communicative acts in his theory of rational interaction. They characterise, on the one hand, the reasons for which the act was selected (called rational effects) and, on the other, the preconditions of feasibility having to be satisfied so that the act can be planned (we will reconsider these concepts a little further). In this theory, the various models of communicative acts (like the inform act for example), are made operational using logical principles of rationality. These principles for example will make it possible for the agent to select the actions which lead to its goals. The model of act of referring we propose integrates the theory of rational interaction. This act can be planned and carried out by the same principles of rational behaviour. To define this model of act, we partly take the one proposed in [BRE 95] adapted from definition of Appelt and Kronfeld [APP 87], and of Maida [MAI 92]: An agent refers to an object whenever the agent has a mental representation which represents what the agent believes to be a particular object, and when the agent intends the hearer to reach a mental representation which represents the same object. We define an act of referring for each set of relevant multimodal reference12. The act presented below (Figure 3.5) enables one to produce a reference of the same type as the reference to the car in Figure 3.2, i.e. to refer to a MMR using a conclusive nominal group and a deictic gesture. The heading of the act13 expresses that an agent i performs a referring act to an agent j using the MOR mori to refer to the MMR mmrj. The preconditions of relevance to the context (ConP) of the act express the conditions depending on the context which must be true so that the act is accomplished. If they are false, the act, unrealisable, will not be selected by the agent. For example, if the system needs a physical connection to WWW it does not have, it will not seek to have one. The preconditions of relevance to the context presented in Figure 3.5 mean that to perform a linguistic (natjang) and gestural (gesture) reference, these two modalities must be available (Available(m)}. 12 These types of multimodality are so numerous that we eliminated those considered to be irrelevant or useless. They were built from a taxonomy of unimodal modalities, based on [BER 97]. 13 For reasons of clarity, the model presented here is simplified. In particular, the distinction between productive and acquisitive OR does not appear.
30
Communicating with Smart Objects
ConP: Modality_isa(m1, gesture) A Modality_isa(m2, not Jang) A Available(m1) A Availablefm^ A CapP: Orm(mori) A Mmr(mmrj) A Has_mmr(i, mmri) A Belong(mor}, mmri) A Or(orj) A Ofor^ A Belong(ori, mmrj) A Belong(or2, mmrj A Modality_cat(ori, mj) A SemioticFnct"_isa(or•/, deictic) A Location(loci, mmri) A Visiblefloci) A Destination(or•], locj A Modality_cat(or2, m^ A Txt_isa(or2, nom_dem_gr) A Mor_combine(mori, or/, or^) A Desc_ident(mori, rmmj) A ^for^ RE: Has_rmm(j, rmmj') A Represent_same_individual(rmmi, rmm}')
(1) (1) (2) (2) (3) (3) (4) (5)
Figure 3.5 Logical model of the act of referring. Example taken with a multimodal OR made of a linguistic OR and a gestural OR The preconditions of capacity (CapP) relate to the capacity of the agent to perform the act. If the conditions are false, the agent can plan the actions which will make them true. The preconditions of capacity presented here mean the following things. The conditions (1) are present in every act of referring: the MOR used to refer to a MMR must belong to this MMR (Belong), an agent i can refer only to one of its MMR. The rational effect (ER) is also common to all acts of referring (we will return to this point further). The conditions (2) specify that or/ and or2 are OR and must belong to the MMR mmri. The conditions (3) express that the category of modality (Modality_Caf) of or/ is gesture, that its semiotic function (SemioticFnct_isa) is deictic, that loc1 is the site (Location) of the object represented by the MMR mmrb that loci is visible and that the destination of the deictic gesture is loc} (location of the object to be referred). The conditions (4) mean that the category of modality of or2 is natural language (nat_lang) and that its textual category14 (Txtjsd) is a conclusive nominal group (nom_dem_gr). The conditions (5) express that ormj is the combination of or/ and or2. The Descjdent predicate expresses that MOR ormj is an identifying description of the MMR mmrj. §(or$ specifies additional properties on or2 which we do not detail here (see [PAN 96]). To illustrate the fact that the preconditions of capacity can be planned, let us take the example of the visibility of the location locj. If the location /oc/ is not visible, the agent can plan a succession of actions to make it visible (e.g. according to the context: to physically move the object, to move its representation, to move (in) the scene of discourse). The rational effect (RE) (the expected effect) of this act is that its addressee, the agent j, will have a MMR mmrj' representing the same object as the one represented by mmri.
See [PAN 96] for the definition of textual category.
Towards Multimodal Human-computer Dialogue
31
Let us illustrate our act with an example, while partly taking the one of Figure 3.2. The user asks: 'What is the price of this car' but this time, the user points at a spot on the screen where there is no vehicle. This spot is actually located between the photographs of two cars. Thanks to the model of act of reference shown in Figure 3.5, the agent will note that the reference is a bimodal one with a natural language OR and a gestural OR (white part of Figure 3.2). Natural language OR is of conclusive nominal group type and gestural OR is of deictic type. The agent will think that there is a MOR combining the preceding two OR. Thus, these two OR are the realisation of only one co-reference to a single object. Conclusive nominal group OR will make it possible for the agent to identify the category of the object as being an instance of the automobile object category. Unfortunately, the gesture deictic pointing on no vehicle, the agent will not be able to identify the referent. A possible reaction of the agent (directed by the primitive principles of the rational behaviour) will be for example to undertake a clarification dialogue in order to obtain an identifying description of the object. The agent can also, thanks to its knowledge manager, elect several possible candidates for this referent. Then he will be able to answer the user by using the act of referring: "This vehicle costs 12500 US Dollars...' (by indicating the vehicle located above the spot indicated by the user) '...and this vehicle costs 15000 US Dollars' (by indicating the vehicle located below the spot indicated by the user).
4. Concluding remarks The model of multimodal act of referring we propose is based on an original representation of objects. It makes it possible to integrate information related to different modalities. Thanks to this model, the agent can understand a multimodal reference produced by another agent, and identify the referent. It can also produce a multimodal reference. One of the characteristics of this model is that it can easily be extended beyond the referring to objects. Indeed, it makes also possible to refer in a multimodal way to relations existing between objects, as well as with other properties of multimodal utterances, like facial gesture reference to illocutionary force. Our model is in the course of implementation within a system of dialogue, already operational15 in natural language. This model, once implemented, should partly allow richer and user-friendly dialogues. The number of media and modes of communication which could be used 15 This system integrates a dialoguing rational agent called ARTIMIS [SAD 97], founded on a theory of rational interaction [SAD 91].
32
Communicating with Smart Objects
will diversify interactions. The users will be able to define their preferred modalities, to switch from one to another. User and system will be able to use several modalities at the same time, and adapt the presentation to the content. The multimedia and multimodal dimensions of information processing systems will finally be exploited to its right measure, i.e. in an intelligent and adapted way.
5. References [AND 94] E. Andre, T. Rist, Referring to World Objects with Text and Pictures, In Proceedings of COLING-94, 530-534, 1994. [APP 85] D.E. Appelt, Planning English Referring Expressions. Articifial Intelligence, 26(1), pages 1-33, 1985. [APP 87] D.E. Appelt et A. Kronfeld, A Computational Model of Referring, In Proceedings of the 10th IJCAI, pages 640-647, Milan, Italy, 1987. [AUS 62] J.A. Austin, How to Do Things with Words, Harward University Press, 1962. [BEL 95] Y. Bellik, Interface Multimodales: Concepts, Modeles et Architectures, Thesis Dissertation, University of Paris-XI, France, 1995. [BEN 98] C. Benoit, J-C. Martin, C. Pelachaud, L. Schomaker and B. Suhm, AudioVisual and Multimodal Speech Systems, In D. Gibbon (Ed.), Handbook of Standards and Resources for Spoken Language Systems - Supplement Volume, 1998. [BER 97] N. O. Bernsen, A Toolbox of Output Modalities, Representing Output Information in Multimodal Interfaces, The Maersk Mc-Kinney Moller Institute for Production Technology, Odense University, Denmark, 1997. http://www.nis.sdu.dk/ publications/papers/toolbox_paper/index.html [BRE 95] P. Bretier, F. Panaget, D. Sadek, Integrating Linguistic Capabilities into the Formal Model of a Rational Agent: Application to Cooperative Spoken Dialogue, In Proceedings of the AAAI Fall Symposium on Rational Agency, Cambridge, 1995. [CAD 95] V. Cadoret, Determination d'actes de dialogue: une approche combinant representations explicites des connaissances et apprentissage connexionniste, Thesis Dissertation, University of Rennes I, 1995.
Towards Multimodal Human-computer Dialogue
33
[CLA 86] H.H. Clark et D. Wilkes-Gibbs, Referring as a collaborative process. Cognition, 22:1-39. Reprinted in (Cohen, Morgan and Pollack, 1990, pages 463493), 1986. [COH 81] P.R. Cohen, The need for referent identification as a planned action. In Proceedings of the Seventh International Joint Conference on Articifial Intelligence (IJCAI-81), pages 31-36, 1981. [COH 90] P.R. Cohen, J. Morgan, M.E. Pollack (Ed.), Intentions in communication, MIT Press, 1990. [COH 94] P.R. Cohen, H.J. Levesque, Preliminaries to a collaborative model of dialogue, In SPECOM'94, pages 265-274, 1994. [COS 97] J. Cosnier, J. Vaysse, Semiotique des gestes communicatifs. Nouveaux actes semiotiques, 54, pages 7-28, 1997. [CSI 94] A. Csinger, Cross-Modal and the Attention Problem, Technical Report from the Department of Computer Science at the University of British Columbia, 1994. [DAL 87] R. Dale, Cooking up referring expressions. In Proceedings of the 27th Annual Meeting of the Association for Computational Linguistics, pages 68-75, 1987. [DAM 94] A.R. Damasio, Descarte's Error: Emotion, Reason and the Human Brain, New York: Grosset/Putman Press, 1994. [EDM 94] P. G. Edmonds, Collaboration on reference to objects that are not mutually known, Proceedings of the 15th International Conference on Computational Linguistics (COLING-94), Kyoto, pages 1118-1122, 1994. [HE 97] D. He, G. Ritchie, J. Lee, Referring to Displays in Multimodal Interfaces, In Workshop on "Referring Phenomena in a Multimedia Discourse and their Computational Treatment", ACL - SIGMEDIA, 1997. [HEE 91] P. Heeman, A computational model of collaboration on referring expressions, Master's thesis, Department of Computer Science, University of Toronto, 1991. [HEE 95] P. Heeman, and G. Hirst, Collaborating on referring expressions. Computational linguistics, 21(3), pages 351-382, 1995. [GRO 86] B.J. Grosz, and C.L. Sidner, Attention, Intentions, and the Structure of Discourse, Computational Linguistics, 12(3), 1986.
34
Communicating with Smart Objects
[GRO 95] BJ. Grosz, A.K. Joshi, S. Weinstein, Centering: A framework for modeling the local coherence of discourse. Computational Linguistics, 12(2), 203-225, 1995. [KRO 90] A. Kronfeld, Reference and Computation, An Essay in Applied Philosophy Of Language, Studies In Natural Processing, Cambridge University Press, 1990. [LEM 01] P. Le Mer, Modele de communication homme-clone pour les environnements virtuels collaboratifs non-immersifs, Thesis Dissertation, University of Lille, 2001. [MAI 92] A.S. Maida, Knowledge representation requirements for description-based communication, In Proceedings of Knowledge Representation'92, pages 232-243, 1992. [MCN 92] D. Me Neill, Hand and Mind: What Gestures Reveal about Thought. University of Chicago Press, 1992. [PAI 69] A. Paivio, Mental Imagery in Associative Learning and Memory. Psychological Review, 76, pages 241-263, 1969. [PAI 86] A. Paivio, Mental representation: A dual coding approach, New York, Oxford University Press, 1986. [PAN 96] F. Panaget, D'un systeme generique de generation d'enonces en contexte de dialogue oral a la formalisation logique des capacites linguistiques d'un agent rationnel dialoguant, Thesis Dissertation, University of Rennes I, 1996. [PIE 97] J-M. Pierrel, L. Romary, G. Sabah, J. Vivier, A. Vilnat, A. Nicolle, Machine, Langage et Dialogue, Collection Figures de 1'interaction, Paris, L'harmattan, 1997. [REI 90] E. Reiter. The Computational Complexity of Avoiding Conversational Implicatures. In Proc of 28th Meeting of the Association of Computational Linguistics (ACL-1990), pages 97-104. MIT Press, 1990. [REI 92] E. Reiter, A fast algorithm for the generation of referring expressions. In Proceedings of the 14th International Conference on Computational linguistics (COLING-92), pages 232-238, 1992. [REI 97] E. Reiter, Discussion Paper: Choosing a Media for Presenting Information, Discussion paper for Electronic Transactions on Artificial Intelligence, 1997. http://www.dfki.de/etai/statements/reiter-nov-97.html
Towards Multimodal Human-computer Dialogue
35
[SAD 91] D. Sadek, Attitudes mentales et interaction rationnelle: vers une theorie formelle de la communication, Thesis Dissertation, University of Rennes I, 1991. [SAD 97] D. Sadek, P. Bretier, F. Panaget, ARTMIS: Natural Dialogue Meets Rational Agency, In Proceedings of 15th IJCAI, pages 1030-1035, Nagoya, Japan, 1997. [SAD 99] D. Sadek, Design Considerations on Dialogue Systems: From Theory to Technology - The Case of ARTMIS, In Proceedings of the ESCA TR Workshop on Interactive Dialogue for Multimodal Systems (IDS'99), Allemagne, 1999. [SAL 01] S. Salmon-Alt, Reference et dialogue finalise: de la linguistique a un modele operationnel, Thesis Dissertation, University of Nancy I, 2001. [SEA 69] J.R. Searle, Speech Acts, Cambridge University Press, New York, 1969. [STR 97] M. Streit, Active and Passive Gestures - Problems with the Resolution of Deictic and Elliptic Expressions in a Multimodal System. In Workshop on "Referring Phenomena in a Multimedia Discourse and their Computational Treatment"., ACL SIGMEDIA, 1997. [TUR 50] A. Turing, Computing machinery and intelligence, Mind, LIX, 1950.
6. Glossary ACL (Agent Communication Language): Inter agents communication language proposed by consortium FIPA. ACL is founded on the formal definition of communicating acts between agents, making it possible to carry out unambiguous interactions. Actuator: Apparatus, body of an apparatus operating a computer so as to modify its state or its behaviour. Adaptability: Capacity to adapt. For a system of man-computer interaction, that indicates its capacity to modify its behaviour according to the profile of that which uses it, of the context, of the task. Agent: An agent is a data-processing, autonomous and active process permanently. It is able to perceive and act on its environment, to communicate and to even move (to migrate). Belief: The belief characterises what the agent believes true, it thus constitutes the model of the world of the agent.
36
Communicating with Smart Objects
Bluetooth: It is a short waves radio technology which makes it possible to remove wiring between electronic, computers and telephones units by a short radio link (ten meters) with low need of energy. Communication mode: splitted into two types of modes: perceptive mode (related to the human senses, i.e., visual mode, auditive mode, ...) or productive modes (related to the action capabilities of human being, i.e., oral mode, gestural mode). Communicative act: It is a behaviour intended to be observed by (and thus to change the mental state of at least) another agent that its author. An agent adopts a behaviour of this type to communicate an intention. Deictic: Designation by pointing. Dialoguing agent: It is an intelligent agent equipped with capacities of dialogue. In fact a really intelligent agent sees its capacities dialogue to emanate naturally from its intelligence. Dual coding: Process which, during perception, linguistics or sensitive, carries out the encoding and the at the same time linguistic and sensitive storage of the object of perception. Gaze tracking: Processes which make it possible to determine the precise position where the gaze of the user is. Gestures recognition: Very recent processes which make it possible to recognise certain categories of gestures and their mean. GPS (Global Positioning System): System of localisation per satellite. Haptic (interfaces): Emergent interfaces of which the tactile screens form parts, based on the tactile or space interactions. Illocutionary act: Act achieved by producing a succession of signs in a context of social relation, and which consists in expressing an intention (to inform, to request, are examples of illocutionary acts). Image synthesis: Process able to generate the image of one object using its semantic or visual description (form, size, colour, etc). Intelligence: See Intelligent agent. Intelligent agent: The intelligence of an agent results in a behaviour known as intelligent or rational, like the aptitude for the reasoning, the aptitude for the comprehension, the capacity of reaction, autonomy.
Towards Multimodal Human-computer Dialogue
37
Intention: An agent has the intention of a proposal when he wants that this one to be true in the future. Intention maintain thus a strong link with action. Interworking: Capacities of initially independent and different systems to being able to function in interaction. Knowledge (of the agent): See Belief. Locutionary act: Act consisting in producing a succession of signs (phonems, graphems, etc), according to a given language grammar. Media: Physical device of interaction. There are various types of media: monodirectional in input of the system (keyboard, mouse, tactile screen) ...or at output (screen, loudspeaker) ...the and bi-directional ones which are most of the time only the combination of mono-directional medias (telephone, PDA, etc). Mediator: Intelligent intermediary able to moderate the actions of the interlocutors, to establish the link between them, etc. Mental attitude: Various mental attitudes are classically modelled within the intelligent agents such knowledge, the desire, the intention, uncertainty ... Modality: Structure of information used at the time of a communication (linguistic, textual, graphic, tactile, ...). The modalities are more or less directly related to the modes of communication. Multimedia: Use of several media. Multimodal: Use of several modes of communication or several modalities. Multimodal referring: Reference carried out by using several modalities during its production or its perception. PDA (Personal Digital Assistant): electronic organiser, portable electronic diary, etc. Perlocutionary act: Act achieved by the fact of stating a succession of signs. "To convince", "to encourage", "to alarm", "to frighten" are examples of perlocutionary acts. The perlocutionary acts are especially characterised in terms of perlocutionary effects as they appear as effects of illocutionnary acts and are not necessarily accomplished intentionally. Predicating act: Act allowing to describe properties on the world (such of the relations between the objects).
38
Communicating with Smart Objects
Prepositional act: Act made up of referring and predicating acts. Prosody: Set of parameters involved in the force and the duration of the phonic units (phonemes) in a vocal signal. Protocol: Preset sequence of events allowing to guarantee the good course of a communication (communications protocol) of actions (operational protocol). Rationality: The principles of rationality constitute one of the important aspect of what one regards as characteristic of the intelligence. Rationality is in particular what will push an agent to be acted by selecting in an optimal way the actions which will lead it to its goals. Referring act: Act allowing to indicate objects, while referring to its mental representation or with the object itself. Service: Set of functionalities gathered around a total offer intended for the user to facilitate the achievement of a task to him or to return it to him more pleasant: information services, purchase/sale services... Software agent: See Agent. Tactile screen: Screen equipped with a tactile matrix on which one can indicate an area on the screen using his finger or a pen. The size of this area depends on the number of points of the matrix. However, the allowed precision remains most of the time rather weak. Talking heads: Computer graphic designed faces, in 3 dimensions whose facial muscles and mouth can be articulated and whose movements can be synchronised with the uttering of a vocal message. Trait (of word): The trait of a word corresponds to the one of the attributive or characteristic aspect of its mean. Example: the verb "to walk" has several traits: the fact of advancing, with feet, slowly, etc. Transmodality: That represents the physical switch of a modality to another. Uncertainty: Characterise the uncertainty of agent on the veracity of a proposal. Virtual clone: like-human entity represented in 2 or 3 dimensions. Voice recognition: Signal processing techniques which return a sequence of recognised words. Voice synthesis: Reproduction of a speech signal which follows a written text.
Chapter 4
Multimodal Interaction on Mobile Artefacts Laurence Pasqualetti, Laurence Nigay, Moustapha Zouinar, Pierre Salembier, Guillaume Calvet, Julien Kahn and Gae'tan Rey France Telecom R&D, University of Grenoble and University of Toulouse, France
1. Introduction Recent progress achieved with the miniaturisation of microprocessors and wireless networks make it possible to consider that the "grey box" of the personal computer is condemned to disappear or at least not to be the only place of interaction between the user and the numerical world. This comes about as a result of a double movement resulting from technological work on the concept of ubiquitous computing and the disappearing computer and from the evolution of ideas in the field of models of interaction. Indeed, research is now gradually being directed towards models of interaction in which data-processing resources are distributed in a multitude of usual objects with which the user will interact in an explicit or implicit way. The idea here is to use the environment as an interface, as a system of manipulation of technical resources functionally limited but contextually relevant (concept of "tangible" interface). The device can be physically handled in a meaningful way: action with the device returning to a preset function (that requires the definition of a semantics and possibly of a syntax of the interaction with the device). The concept of communicating objects covers very diverse technological and conceptual realities; among the specific properties generally attached to these objects one will retain in particular: •
•
numerical increase rather than substitution: the direct idea is to set out again usual objects in which one endeavours to preserve the intrinsic advantages inherent in their material constitutivity (in particular properties of affordability and support of awareness) at the same time associating additional functionalities (eg: paper and the numerical pen, the communicating refrigerator etc). transportable character of devices: it makes it possible to be given "embarked" resources of processing and communication and thus support mobility and
40
•
Communicating with Smart Objects
intellectual nomadism. These devices can take several forms (PDA, portable telephone, communicating clothing etc). capacity to communicate in an autonomous or controlled way: in addition to their traditional function of support to the communication between users, portable devices can detect the presence of a device of the same type (or dataprocessing resources distributed in the environment) and exchange information according to possibly pre-established rules but independently of any command given intentionally by the user.
In the HOURIA project we study the way in which certain physical achievements of this concept of communicating object could be used easily and in an effective way by mobile individuals, and to be integrated without causing rupture to their daily activities and the physical and social environments in which they are located at every moment. We justify our choice of the multimodality by the following: •
•
These artefacts rely on physical devices with restricted capacities, outside the traditional framework (a large screen, a keyboard and a mouse). It is thus advisable to conceive methods of interaction relying on paradigms other than direct handling "screen-mouse" like the tangible interfaces or the "embodied user interface". The context of use of these artefacts is by definition very variable. Indeed, the physical (noise etc) and social characteristics (intimacy, intrusiveness etc) of the environment determine a whole set of contextual constraints which will require interactional adaptations; from this point of view, multimodality constitutes a type of answer adapted to this requirement of adaptability and adaptivity. The development of mobile computing thus represents a field highly suited to the application of multimodal interaction techniques.
2. Problems and aims of the study Multimodality has given rise to much theoretical and empirical work. The theoretical work is mainly concerned with the definition of the concepts of modality and multimodality, and with the development of "design spaces". These spaces are conceptual frameworks which provide a whole set of concepts making it possible to describe the modality and the possibilities of combination of these in comparison with the interaction user-system. For example, the models TYCOON (Martin & Beroule, 1993) and CARE (Coutaz & Nigay, 1996) propose a whole set of concepts which describe various types of theoretical relationships of composition or "cooperation" between modalities: assignment, complementarity, equivalence, redundancy, competition, etc. Empirical work has explored the effective (real) use of the modalities and the real contribution (i.e. effectiveness) of multimodality in situations of interaction with more or less simulated multimodal systems. From the point of view of the user, these studies made it possible to show how the users combine various modalities to
Multimodal Interaction on Mobile Artefacts
41
interact with the systems and in which cases they use multimodality. Certain studies, such that of Oviatt, DeAngeli, & Kuhn (1997) for example, thus showed that multimodality is not always used (approximately 20% of the time of a session of interaction); the cases of use of this appear when the users describe in their commands some spacial information (for example: localisation, size, orientation or shape of an object). In addition, several types of combination were observed (Guyomard, Le Meur, Poignonnec & Siroux 1995; Mignot & Carbonel, 1996; Oviatt et al.. 1997, etc.): combinations of complementarity (for example, the user supplements a verbal statement by pointing by tactile designation an object aimed on the screen) and of the combinations of redundancy (for example, the user indicates orally and explicitly an object and, at the same time, tactily indicates it on the interface). From the point of view of effectiveness, few studies have systematically approached this point and the evaluation criteria used are in the majority concerned with the saving of time which brings multimodal interfaces to be compared with monomodal interfaces Although it produced many interesting results on the use of multimodality, this work approached only one limited whole of input modalities (mainly speech, pointing or writing). Moreover, few of these studies analysed the phenomena of appropriation of the multimodality through many sessions of interaction with the system. Lastly, the tasks suggested to the subjects were generally "new" for the users. A limit to this choice is that it leaves aside the study of the consequences of multimodality in the realisation of familiar tasks with more "classical" systems. The general objective of the study presented here aims at contributing to design multimodal systems and the analysis of their use by approaching these various points, i.e.: • • •
To apprehend multimodality in situations of interaction "meaning" for the users, i.e. contexts of tasks/activities which are familiar to them. To study the processes of appropriation of the multimodality through several sessions of interaction. To explore the use of "new" / "original" modalities in order to see empirically which place they take in man-machine interaction, the problems which they pose individually and in their relationship to other modalities.
In addition, by this study we aim at answering a certain number of precise questions related to the use of multimodality: how is it carried out and what guides the choice of multimodality? How are the choices and the changes of modalities of the user carried out? What is the role of the modality properties in this choice? Which are the criteria which lead a user to choose or give up a modality of interaction to the benefit of another? How do users combine the modalities? From a practical point of view, our objective is to manage to release ergonomic recommendations as generic as possible for the design of multimodal systems.
42
Communicating with Smart Objects
We shall start from published work (Coutaz & Nigay, 1996) to consider a modality as a means of communication which implements a physical device and a language of interaction. We will consider multimodality from the user's point of view and it will be regarded as a means of production of one's intentions through various modalities (glance, gesture, speech, handling of the artefacts, etc). From this point of view, we regard multimodal systems as those which allow specialised uses (a modality exclusively dedicated to a command), equivalents (a modality can be used for all the commands), complementary (several modalities can be combined to carry out the same command).
3. Method In order to answer these questions, we chose to develop a partially simulated multimodal system (Wizard of Oz technique). Our objective was to force the interaction as little as possible while limiting the development cost of the system and to free us from the current technological limitations at exploratory ends centered on new modalities. 3.1. Subjects Ten subjects took part in this experimental phase; they all had an electronic mail and were regular users of Internet but none was a computer specialist.
3.2. Task and nature of the modalities tested In order to put the subjects in a situation of realisation of a familiar task, they were asked to use an application of consultation of electronic mail by means of a pocket computer (PDA Jornada 540 series, Hewlett Packard) which was connected, for the purposes of the experimentation to their personal electronic mail. Four modalities studied were implemented: • • •
•
the vocal modality is presented as a mode of interaction in natural language without particular constraints (no preset statements to be used by the subjects); tactile modality, which requires a stylet to point the command buttons; gestural modality, which relies on a preset code which forces one to hold the PDA in one hand and to carry out the gesture with the free hand in front of the apparatus; improperly qualified modality and temporarily "embodied", which consists of associating commands with certain preset handling of the artefact (changes of orientation, various movements, etc.).
Multimodal Interaction on Mobile Artefacts
43
Only tactile modality was really implemented, the other modalities were simulated by the accomplice thanks to an identical interface and a numerical video system.
Figure 4.1 Implementation of the tactile, gestural and "embodied" modalities
Each modality allowed a realisation of the same set of commands with the tactile modality as a reference. 3.3. Experimental procedure and platform The activity of the subjects during the sessions (15 minutes average duration) was recorded by means of various sensors (cameras, sensors of system events). The experimental sessions were preceded by a training and recall phase, and were spaced over several days. They were immediately followed by autoconfrontation which was video-recorded. 4. Data processing and analyses Different data were collected during the study. During the interaction, the commands and the windows used by the subjects were indexed temporarily and filed automatically. The actions of the subjects, the modalities used, the duration and the contents of the commands were supplemented a posteriori starting from video film of the interaction. Lastly, the recording of the autoconfrontations was carried out in order to allow qualitative analyses starting from the identification of intentions and the strategies of the subjects as for the use of the modalities.
44
Communicating with Smart Objects
5. Results 5.1. Global use One of the first results obtained about the use of the modalities indicates that all of the modalities were used by the subjects during the various sessions. In addition, one observes an evolution in the use of multimodality during sessions: •
• •
During the first sessions one observes a homogeneous distribution in the use of the modalities which corresponds to a phase of an informal test of the system. This fact is corroborated by the data resulting from the autoconfrontation. But some preferences in the use of the modalities quickly appear in the following sessions. These preferences vary according to the subjects.
On a very global level (all sessions and all subjects mixed), one does not note specialisation of one modality to the realisation of one or several commands. Therefore, if there is specialisation, it can be located only at individual level, bringing the need of carrying out a specific analysis of the data for each subject.
5.2. Intra-individual specialisation As mentioned above, one observes in many subjects a tendency to preferential uses of modalities. These tendencies are specific (they appear only during only one session of interaction) or recurring (they appear in other sessions), are individualised (the same modality is used to carry out only one and same command), plural (the same modality is used to carry out several commands). 5.3. Influencing factors for the choices of modality The activity graphs and the verbalisations of the subjects made it possible to identify several factors which are likely to direct the choice and the change of use of the modalities. We present only the four principal ones here: Context of the activity in progress (recurring operational procedure) We could identify that the tendencies to preferential use generally result in sequences of recurring actions carried out by the subjects. These sequences are strongly related to the context of the activity in progress. Change of goal in the activity The changes of modality are often associated with phases of local reorientation of the activity.
Multimodal Interaction on Mobile Artefacts
45
Properties of the modalities (implementation) During the evaluation of a multimodal device, it is advisable to distinguish well the inherent characteristics of a given modality from the implementation mode of this modality. The fact that a modality is little used does not mean that this modality is unsuited but that its technical realisation can be inadequate or constraining. Thus, during the experimentation the tactile modality sometimes was under employed by the subjects because of the constraint induced by holding the stylet. This last is perceived as an external appendix of the PDA, which one possibly ends up putting down to not having to hold what will contribute to limiting its use or to restrict it with specific cases (problem with the other modalities to achieve particular actions such as closing error messages). Error correction Many changes of modalities appeared in dysfunctioning situations in order to correct errors coming either due to the subject, the system or of the accomplice. 6. Conclusions and prospects In this article we presented an exploratory study of the use of a portable multimodal system. The results obtained tie in with those obtained in preceding work: effective use of multimodality (all of the modalities was used), interindividual differences, appearance of preferential tendencies), change of modalities in dysfunctioning situations. Beyond this convergence, an original result of this study is that the subjects are easily appropriate to unusual modalities (such as the "embodied" modality for example). From their nature the communicating objects are likely to be used in very variable contexts. Thus the difficulty of their use in situations arises where the modalities of interaction will not be equally adapted. The multimodality takes on crucial importance then, insofar as it offers to the users a means of regulation of the contextual variations (environmental constraints, social constraints etc) by the adoption of the most suitable modality. It is this that study illustrates with regard to for example the recovery of the dysfunction. Moreover for each usual object considered, it will be necessary to identify the modalities of interaction to be integrated in input as at output. Thus, for an environment including multiple communicating objects, it will be necessary to define mechanisms making it possible to specify the recipient object of the command. From this point of view multimodality could bring solutions; it remains nevertheless to study the conceptual and practical viability of strategies of assignment by construction of a particular modality to an object or a type of object in a whole range of communicating objects.
46
Communicating with Smart Objects
7. Bibliography Calvet, G., Kahn, J., Nigay, L., Rey, G., Pasqualetti, L., Salembier, P. & Zouinar, M. (2001) HOURIA- Nouvelles interactions multimodales, Rapport final contrat de recherche FT-R&D, CLIPS-IMAG, GRIC-IRIT. FT-R&D DIH-UCE, Issy-lesMoulineaux. Calvet, G., Kahn, J., Salembier, P., Zouinar, M., Pasqualetti, L., Nigay, L., Rey, G. & Briois, J.C. (2001). Etude empirique de 1'usage de la multimodalite avec un ordinateur de poche. Actes de la conference "IHM-HCI 2001", 10-14 September, Lille, France. DeAngeli, A., Wolff, F., Lopez, P. & Romary, R., Relevance and perceptual constraints in multimodal refering actions (1999) In Proceedings of the Workshop on Deixis, Demonstration and Deictic Belief, Eleventh European Summer School in Logic, Language and Information (ESSLLI-99), 9-20 August, Utrecht, The Netherlands. Guyomard, M., Le Meur, D., Poignonnec, S. & Siroux, J. (1995) Experimental work dor the dual usage of voice and touch-screen for a cartographic application. Proceedings of the ESCA tutorial and research workshop on Spoken dialog systems, Vigso, Denmark, 30 May-2 June, pp. 153-156. Martin, J.C. & Beroule, D. (1993) Types et buts de cooperations entre modalites dans les interfaces multimodales. Actes des Semes journees sur l 'ingenierie de l 'interaction Hammer Machine, 19-20 October, Lyon, France. Mignot, C. & Carbonell, N. (1996) Commande orale et gestuelle: etude empirique. Technique et Science In-formatiques, 15, pp. 1399-1428. Nigay, L. & Coutaz J. (1996) Espaces conceptuels pour 1'interaction multimedia et multimodale, Technique et Science Informatiques, special Multimedia et Collecticiel, AFCET & Hermes Publ., Vol 15(9), pp. 1195-1225. Oviatt, S.L. (1999) Ten myths of multimodal interactions, Communications of the ACM, Vol. 42, N°l 1, November, pp. 74-81. Oviatt, S.L., DeAngeli, A. & Kuhn, K. (1997) Integration and synchronization of input modes during multimodal human-computer interaction. In Proceedings of Conference on Human Factors in Computing Systems CHI '97 (22-27 March, Atlanta, GA). ACM Press, New York, pp. 415^22.
Chapter 5
The Voice as a Means of Humanising Man-machine Interfaces Noel Chateau France Telecom R&D, France
1. Introduction Recent and future technological progress in the fields of signal processing, algorithms and artificial intelligence foresee communication between men and machines becoming increasingly close to that between two human beings, thus revealing a strong tendency towards anthropomorphism. In the field of telecommunications, the principal medium of communication is still the voice. Today, machines can speak (through vocal synthesis) and hear (using word recognition). They even "reason" and are capable of holding a discussion by means of software agents for intelligent dialogue. However, although they carry out the principal functions of vocal communication, it is tempting to say that, pragmatically, machines are still deaf and dumb. We should remember that pragmatism is to a certain extent the "function of form", that is to say, all of the information borne by the form of the signal (vocal, in this instance) which may modify the initial sense of the semantic content. Irony is a good example: a certain tone of the voice will lead to an altogether different understanding of a message like "The weather is still fine today", compared to a neutral tone, especially if the weather is rainy when this sentence is uttered. This chapter presents the bases for studies concerned with the emotions carried by vocal interaction and attempts to illustrate to what extent these studies lead to manmachine interfaces, especially with regard to communicating objects that are simpler and more pleasant to use by users.
48
Communicating with Smart Objects
2. Emotions 2.1. Affect, emotion and sentiment According to Aboulafia et al. [ABO 01], three levels of emotions may be distinguished, which differ through their intensity and duration. Firstly, the affect relates to an intense emotional state of relative short duration (a few seconds) which calls upon short-term memory. Secondly, emotion characterises emotional states which go beyond the situation which triggered them. Consequently, they call upon episodic memory since they exceed the duration of storage of sensorial memory. Finally, the sentiments are associated with emotional states which relate to the history of the individual, his beliefs, social or religious choices, etc. and form part of his personality. For their part they call upon the long term memory (which is commonly known as memory). Thus, when we speak of emotion in MMIs, through the accurate definitions provided by Aboulafia et al., we note that we are misusing language and that it would be better to speak of affect in the MMIs, since studies in this field are still just in their infancy. However, as this term conveys less of a common meaning and is less of a trigger of dreams, it is extremely little used and discarded in favour of emotions.
2.2. The nature of emotions From ancient Greek times where nomos (rules or reason) and physis (nature, impulsiveness, emotions) were in conflict, and up to the middle of the 20th century, the emotions were studied very little, since they constituted an uncontrollable variable in experimentation which was likely to falsify subjective and behavioural data [RUSI 01]. Moreover, in popular language a significant number of proverbs or sayings invoke the impulsive and uncontrollable nature of the emotions [DESP 01], one of the most famous of which is doubtlessly that of Pascal: "The heart has its reasons which reason does not know". However, this is viewed totally differently today, and psychologists such as Izrad [IZR 77] even argue that the emotions form the physiological base on which the perceptive processes (one does not see and hear the same thing, depending on one's emotional state), the cognitive processes (one does not remember the same elements of an image depending on its emotional charge) and action (fear, through the release of adrenalin which, for example, will trigger the action) are organised. The trend is totally reversed today. Emotions are a subject that has been frequently studied by psychologists since the middle of the 20th century. The "marketeers" also wish to know the emotional impact of advertising on consumers. Studies on emotions have seen, as is often the case in psychology, two opposing chapels of thought which we will be presenting briefly below, using two neologisms, the "categorialists" and the "dimensionalists".
Humanising Man-machine Interfaces
49
2.3. Categorialists and dimensionalists For categorialists, emotions may be grouped in a finite number of categories, with each category itself able to include more subtle sub-categories. Ekman [EKM 84] may be considered to be the principal representative of the categorialists. Through work done on facial emotions, they have demonstrated that it is possible to identify a limited number of mimic arts which express six basic emotions and which can be found in the majority of cultures that exist around the world. These six emotions are represented in a caricatural fashion (see Figure 5.1). They are (from left to right and from bottom to top) anger, fear, surprise, disgust, joy and sadness. Ekman has produced a model of facial animation (the FACS) which provides for seventy units of action, each composed of a group of facial muscles and which enables a synthetic face to express these six basic emotions.
Figure 5.1 The six basic emotions of Ekman's model
For their part, the dimensionalists consider that all the emotions may be organised in accordance with a limited number of continuous dimensions. Russel and Mehrabian [RUSS 77] thus suggests three basic dimensions; the hedonistic valence of emotion (positive or negative), its intensity and the degree to which the person experiencing it controls it. Figure 5.2 illustrates these three dimensions and places the six basic emotions of Ekman's model in Russel and Mehrabian's three dimensional space.
50
Communicating with Smart Objects
Figure 5.2 The three dimensions ofRussel and Mehrabian 's model
It cannot be considered that one approach is better than another. In fact, the approach to select for one's own work depends on the objectives and methodologies of measurement that one wishes to use. For example, a post facto evaluation of the emotion felt by the designation of pictograms (as in the PreEmo [DESM 00]), in the field of automobiles) would be appropriate for a categorialistic approach, whereas an evaluation in real time using electro-physiologic measurement tools (electro dermal conductance, blood pressure, pulse rate, respiration or muscular activity) would be more appropriate to a dimensionalistic approach [DET 01]. 3. Emotion in the voice 3.1. Why have emotion in the voice? In the field of human communication, emotions in general and in the voice in particular have two fundamental roles. The first is to regulate communication, in the same way as a large number of non vocal signs emitted by the interlocutors (in particular, mimicry and gestures) do. For example, if one of the interlocutors shows himself to be astonished or annoyed, he will express this through the intonation of his voice, which will immediately trigger a responding reaction in his interlocutor. Moreover, a particular intonation will signal the end of the intervention of one of the two interlocutors and his desire to give way to someone else.
Humanising Man-machine Interfaces
51
The second role of emotion in the voice is to enable the meaning of the verbal content to be conveyed precisely by the pragmatic content which allows access to connotation. As has been shown in the introduction, one can, for example, distinguish several ways of interpreting the same sentence through the intonations in the voice, which will enable listeners to understand whether the sentence uttered is a question or an affirmation and if this affirmation is meant seriously or ironically. Thus, emotion in the voice in this case enables a great deal of ambiguity in the interpretation of the message to be resolved. What we have just said with regard to communication between human beings may just as relevantly be applied to man-machine communication, as long as the machines are endowed with interpretation and the generation of emotion in the voice. In fact, a number of studies including those bearing on the CASA (computers are social actors) paradigm, Nass et al. [NAS 94] have in the last few years stressed the fact that computer users in the wider sense frequently behave with these machines as they do with humans; they use emotional signs to express content or discontent, stress or fatigue without having any return reaction. This is a form of release, gratuitous affect with no consequence but this may address the human context if there are people around the person. Based on the fact we have ascertained, that man-machine communication has a strong tendency to anthropomorphism, that the technologies which enable these same machines to speak (voice synthesis), hear (word recognition) and reason (agents that dialogue) are reaching maturity and may be transferred to a wide range of software platforms, and finally that the CASA paradigm is a reality, it appears that future developments in the field of interfaces and voice technologies will have a major bearing on the production and interpretation of emotional content. Thus, one might envisage, in five or ten years' time, interfaces, material or not, which will be sensitive to emotions in vocal commands and which might be able to react in accordance with the command itself, of course, and also in accordance with the way it has been uttered. With the prospect of the increasingly strong resemblance between man-man and man-machine communication we have just evoked, a development like this would logically improve the usability of systems (in particular by reducing the number of errors of interpretation) while improving their user friendliness (by making communication more natural) through drawing on the two fundamental roles played by emotions in the voice which we have referred to above. It may be considered that there are two major fields of study covering emotions in the voice. The first concerns the production, perception and analysis of emotion in the natural voice and the second the generation through voice synthesis of expressions of emotions which can be found in the natural voice. We will be presenting these two fields briefly later.
52
Communicating with Smart Objects
3.2. Emotions in the natural voice Studies on emotions in the natural voice have been mainly centred on the identification of physical correlations with the various expressions of emotions in words (DAY 64] [SCH 86] and [FRA 00]. With this approach, the vocal signal is analysed with a view to explaining the emotional state of the speaker as it is perceived by his audience. In his detailed review of literature, Scherer [SCH 86] put forward twelve basic emotions which may be distinguished in words (happiness/pleasure, joy/gladness, displeasure/disgust, scorn/disdain, sadness/despondency, grief/despair, uneasiness/ anxiety, fear/terror, irritation/icy anger, rage/temper, boredom/indifference and timidity/culpability) and reports for each of these the principal acoustic parameters identified as being strongly correlated with them. These studies have enabled the relationship between the vocal signal emitted by a speaker and the emotion he expressed through his voice to be modelled. In a recent study, Maffiolo and Chateau [MAP 01] have shown that the emotions perceived in a vocal signal depend closely on the semantic content of the sentence uttered. Thus, it may be imagined that by coupling such models with a word recognition system and a system of artificial intelligence (used by dialogue agents), it would be possible to identify automatically the various emotions expressed by a speaker in a given semantic field.
3.3. Emotions in the synthetic voice Once the emotional content of the voice of a user has been correctly identified, it is a matter of responding to it with ad hoc emotional content. This raises the principal and prior problem of fact: does the person desire symmetry, dissymmetry, wellmeaning neutrality, etc? As part of using a synthetic voice, it is necessary to dispose of algorithms which will make it possible to "breathe into" the acoustic signal being constructed the characteristics of the intonations of the human voice [PIN 89], [CAH 90], [MUR 93]. That might be done a priori by using the bases of specific acoustic data (for example, those recorded with styles of elocution which call upon a variety of pragmatic content) and also by applying particular patterns of prosody post facto to an "emotionally neutral" signal. In this case, re-exploiting the models for analysing emotions in the natural voice might be envisaged "in order for them to supply the target values of the acoustic parameters which the synthetic voice has to achieve if it is to imitate the natural voice". MIT has proposed the Kismet robot (http://www.ai.mit.edu/projects/humanoidrobotics-group/kismet/kismet.html) which uses a synthetic voice in order to express the six basic emotions of the Ekman model. However these emotions are still prototypes and a more subtle approach is required in order to obtain a synthetic voice that is more natural, with more realistic and less caricatural intonations.
Humanising Man-machine Interfaces
53
4. Communicating objects, emotions and the voice Most objects are created by man to fulfil a function, itself generated by a need. It may be considered that only artistic creations have no need to supply men with functional objects or tools, although works of art themselves meet a need, which, however, cannot be described as functional but rather a need for expression and communication. All objects, however functional they are, incorporate aesthetic dimensions. A simple stroll around the hammer counter of a DIY shop will confirm this. Shapes, colours, materials and packaging are some of the aspects of shape and design on which creators and marketing work to promote the object (the product) and set it apart from its previous version or those of its competitors. The balance between function and aesthetics will depend on the object as well as on fashion and trends. Even though the dichotomy between function and form appears to be facile or even trivial to achieve (the form is only considered by some to be a common, ultimate touch of paint, whereas everything in the object is expressed by its function), several creators and designers have asked questions on the relationship between function and form. The objects and buildings produced by the Bauhaus School are among other things an attempt to reply to these questions. They show clearly that function and form are inextricably linked from the start of the design procedure and that there is no reason to separate them. At present, users have developed expectations and even requirements, which are expressed by the taking into account of a new dimension in creation, with regard to products. Products should no longer merely meet a need by being useful and simple to use; they must blend into the way of life of the users by providing them with a certain pleasure. Designers must concentrate on what Pat Jordan [JOR 01] describes as affective design or emotional design. Given that large numbers of products have similar prices, functions and ease of use, Jordan claims that the difference between them in the market is based on the affective and emotional dimension they have to satisfy. The success of certain models of mobile phone or the Imac appears to confirm Jordan's theses. All of the views we have set out on objects in general obviously apply to communicating objects in particular. For these, equipped with functions that are ever more subtle as a result of the new technologies they incorporate, are now capable of hearing (word recognition) and speaking (voice synthesis). However, as we stated in the introduction, we may consider that, emotionally speaking machines and objects are currently still deaf and dumb. In view of the preponderant role played by emotions in vocal communication (see paragraph 3.1.) and the increasing importance of the aspects of pleasure in using products, as evoked by Jordan, it appears inevitable that changes in the development of communicating objects will include taking emotions in the voice into account, whether these relate to the input (voice command by users) or output (restitution by synthetic voice) of the system.
54
Communicating with Smart Objects
5. Conclusion At present, after having deliberately ignored for a considerable period, the production, perception and interpretation of emotions have become huge fields for psychological investigation. An increase in the intensity of research in this field has even been noted in the last ten years as a result in particular of its association with various technologies, whether vocal and also video (the automatic recognition of the facial expression of emotion) or electro-physiological (the capture and analysis of electro-physiological signals correlating with emotions). However, it may be considered that knowledge in this field, especially in the field of emotion in the voice, is still in its infancy. A decisive initial step would be to be able to understand and correctly convey in a non-caricatural fashion, the six basic emotions of Ekman's model, that is to say, joy, anger, fear sadness, surprise and disgust. Subsequently, coupling with the algorithms of word recognition (access to the initial semantic content) and intelligent dialogue (access to the context) will be necessary in order to be able to progress further and incorporate systems that'are sensitive to emotions in material or immaterial interfaces. 6. Bibliography Aboulafia, A., Bannon, L. & Fernstrom, M. (2001). "Shifting perspective from effect to affect: Some framing questions," Proceedings of the International Conference on Affective Human Factors Design, Asean Academic Press, London, 508-514. Cahn, I.E. (1990). Generating expression in synthesized speech. Technical report, MIT. Davitz, J.R. (1964). The communication of emotional meaning. New York: McGraw-Hill. Desmet, P.M.A., Hekkert, P. & Jacobs, JJ. (2000). "When a car makes you smile: Development and application of an instrument to measure product emotions," in S.J. Hoch and R.J. Meyer (Eds.), Advances in Consumer Research, 27, 111-117. Despret, V. (2001). "Le pouvoir des desirs: les emotions entre science et politique," actes du colloque Pouvoir, Desir et Emotion, UTC, Compiegne. Detember, B.H. (2001). "Measuring emotional responses in human factors research: Some theoretical and practical considerations," Proceedings of the International Conference on Affective Human Factors Design, Asean Academic Press, London, 124-130.
Humanising Man-machine Interfaces
55
Ekman, P. (1984). "Expression and nature of emotion," in K. Scherer & P. Ekman (Eds.), Approaches to Emotion, Hillsdale, NJ: Lawrence Erlbaum Associates, 319343. France, D.J., Shiavi, R.G., Silverman, S, Silverman, M. and Wilkes, D.M. (2000). Acoustical properties of speech as indicators of depression and suicidal risk. IEEE Transaction on Biomedical Engineering, 47, 829-837. Izrad, C.E. (1977). Human Emotions. New York: Appleton-Century-Crofts. Jordan, P. (2001). Proceedings of the International Conference on Affective Human Factors Design, Asean Academic Press, London, 342-348. Maffiolo, V. & Chateau, N. (2001). "Speech's emotional quality in vocal services," Proceedings of the International Conference on Affective Human Factors Design, Asean Academic Press, London, 342-348. Murray, I.R. and Arnott, J.L. (1993). Toward the simulation of emotion in synthetic speech: a review of the literature on human vocal emotion. Journal of the Acoustical Society of America, 93, 1097-1108. Nass, C., Steuer, N. & Tauber, E.R. (1995). "Computers Are Social Actors," ACM CHI'94, 72-78. Pinto, N.B., Childers, D.G. and Lalwani, A.L. (1989). Formant speech synthesis: Improving production quality. IEEE Transaction on Acoustics, Speech and Signal Processing, 37, 1870-1887. Rusinek, S. (2001). "A la recherche d'une definition de 1'emotion: ce que 1'emotion fait faire par-devers soi," actes du colloque Pouvoir, Desir et Emotion, UTC, Compiegne. Russel, J. & Mehrabian, A. (1977). "Evidence for a three-factor theory of emotions," Journal of Research in Personality, 11, 273-294. Scherer, K.R. (1986). Vocal affect expression: A review and a model for future research. Psychological Bulletin, 99(2), 143-165.
This page intentionally left blank
Part 2 Software Infrastructure for Smart Devices/Ambient Intelligence Gilles Privat France Telecom R&D, France
1. Introduction Software for smart networked devices could, not long ago, have been envisioned as a special case of embedded software, and would as such have adhered to the dominant solutions previously adopted in this domain. These solutions placed a strong emphasis on the hardware and power constraints of embedded platforms, for which software used to be streamlined and carefully optimised. Full advantage may now be taken of improved processor + memory capabilities to adopt standards-based, open solutions in lieu of vertically integrated software, dedicated to hardware. The benefits of time-proven and widely adopted approaches inherited from the general-purpose computing field far outweigh the overhead of the corresponding additional software layers, for all but the most cost-constrained devices. When embedded devices with such high-level of processing capability become networked devices, a new domain opens up which is the ideal application field for the vast body of research performed over the years in the general fields of distributed software and distributed systems theory. Distribution is not a mere theoretical nicety added as an afterthought for obtaining performance improvements: it is inherent in the very idea of networked devices, and will be applied here on a much larger scale than anything previously done with classical distributed systems made up of coarse-grain computing nodes (e.g. servers as network hosts). Generic network and middleware infrastructures still have to be adapted to specific constraints of smart networked devices, even it is not in the way it was done previously: rather than hardware-constrained optimisation, what is required is optimisation for robustness of a large-scale distributed system of connected devices, in an environment where devices may appear or disappear dynamically, be
58
Communicating with Smart Objects
disconnected abruptly, and have to "discover" each over and interoperate without previous knowledge. Specific temporal constraints may also have to be taken into account for communication types which may be as far from bulk data transmission as from isochronous multimedia flows: hard real-time and strictly deterministic latencies may be required for the distribution of asynchronous, possibly bursty events. 2. Generic middleware The idea of layering smart-device applications on top of generic middleware corresponds to moving up from such low-level connectivity interfaces as IP to HTTP, CORBA, RMI or IIOP. The dial-tone metaphor inherited from legacy POTS networks captures best this idea of universal connectivity interface: the dial-tone for smart devices definitely needs to correspond at least to the level to one of these generic interfaces. The universal availability of web-based clients and corresponding connectivity has fostered the use of HTTP as the protocol of choice for a new generation of networked embedded devices. A complete HTTP stack is already offered as a standard addition by most embedded OS vendors. Actually the benefits of HTTP come from its universal availability as a least common denominator applicationlevel interface, not from its adaptation to the network-based control of smart device services. HTTP was originally conceived as a protocol for the retrieval of static documents, and it is still limited by these origins. Numerous proprietary or de facto standard extensions have been grafted onto HTTP to enable server-side interaction with scripts or programs. This interfacing with native server-side programs is essential for all smart device applications, where static document retrieval is in itself useless, HTTP acting merely as an entry point to direct interaction. This confusing plethora of kludgy and mutually incompatible solutions (CGI, ISAPI, NSAPI, ASP, JSP, to name a few) have unfortunately compromised the very simplicity and universality that made HTTP useful and successful in the first place. It should by now be clear that higher-level, more general solutions have to be adopted to replace HTTP if smart networked devices are to take full advantage of the evolution of distributed software. For one, dissymmetric client-server solutions are by themselves less general than peer to peer solutions such as proposed in the framework of generic distributed software infrastructures. In this evolution, devices are not restricted to be software clients, or servers, for that matter: they can be both providers and requesters of software services (i.e., at this granularity, methods called from software objects) to and from one another. The relevant common-denominator abstraction is no longer a text-based document, but a full-fledged software object that can be invoked transparently throughout the network. CORBA, GIOP, IIOP, Java RMI are some of the relevant standards and corresponding software infrastructures, providing this level of object-based connectivity. They are, however,
Software Infrastructure for Smart Devices
59
still a long way from spreading from general-purpose distributed computing, where they have not yet even reached mainstream status, to the domain of networked embedded computing.
3. Infrastructures for spontaneous networking Beyond using such generic middleware layers, ubiquitous computing/ambient networking needs to address the higher-level problem of interoperating vast numbers of spontaneously networked, on-off, possibly mobile devices forming ad hoc, temporary federations as they get in touch with one another using whatever wireless network support is available, without any previous knowledge of the environment. For this, new generic services are needed on top of general-purpose middleware, to address the following needs specific to ambient networking environments: •
• •
A universal bootstrap mechanism making it possible for a new device to discover a new, unknown network environment and its general services (specifically what kind). A (centralised or distributed) directory/lookup service making it possible for devices to advertise their services and for other devices to query them. A connectivity mechanism making it possible for devices to use the services of other devices once they have located them.
These three kinds of services do already exist, replicated at several lower levels, in regular network environments (e.g. ARP, DHCP or BOOTP for bootstrap, DMS or RMID for directories, sockets, RPC or RMI for connectivity). These solutions are usually too low-level and need to be federated at a higher level to work in such highly heterogeneous environments as addressed here. The directory/lookup service especially needs to address the description of extremely varied types of services/devices, with support for sophisticated queries such as, e.g. where is the physically closest network printer with colour A3 capabilities. Efforts currently undertaken by industry leaders in various fora and consortia indicate the importance of such service discovery protocols and APIs on one hand, and service description languages on the other. A slew of solutions, that may in some cases be viewed as an uppermost middleware layer, have been proposed to handle theses specific problems of dynamic distributed network configuration. Their pivotal interfaces go all the way from least-common-denominator, text-based data, to the highest level APIs. Basically all of these technologies use IP-multicasting on a shared medium (e.g. Ethernet). Some of these solutions do take lower-level network connectivity for granted, others propose solutions to provide it if needed. To date, the most publicised among such technologies are: Universal Plug and Play (UpnP, www.upnp.org), initiated by Microsoft and currently championed by the UpnP Forum), Service Location Protocol (SLP,www.srvloc.org) which, as an IETF work item has been jointly developed by researchers both from academia and
60
Communicating with Smart Objects
industry as a widely accepted and usable Internet service for service discovery; Jini (www.sun.com/jini, www.jini.org), initiated and still controlled by Sun Microsystems, which provides an architecture for service discovery entirely based on Java RMI as the middleware platform, and Salutation, again a solution pioneered by an industrial forum (www.salutation.org). Less publicised open-source alternatives such as the customisable Jonathan ORB core (www.objectweb.org) could also be streamlined and fitted as infrastructures for embedded devices. 4. The digital divide for smart devices Moving up to such a high-level connectivity interface as Jini, or the like, does, however, raise the bar by several orders of magnitude for the minimal amount of software a device has to integrate before being able to engage in a minimal dialogue on the same level of protocol as its fellow network citizen devices. Moore law notwithstanding, it is not obvious that the finest granularity smart devices, those with strict per-unit-cost constraints, will ever reach this threshold. The model that can be envisioned for the time being is that of a two-tier network, in which first-class network citizens are able to talk to one another at the higher level of protocols, enabling transparent, dynamic configuration and all benefits that come with this more abstract level of discourse. Second-class devices will not be seen directly from the network at large. They will have to be represented by a proxy, which could be viewed as a software agent residing somewhere on a network server, or on another (first-class) device. The lowly device will be allowed to talk with its proxy device using some closed proprietary protocol, provided this low-level communication is entirely hidden behind the proxy for other devices attached to the network at large.
Chapter 6
Introduction to a Middleware Framework Vincent Olive and A. Vareille France Telecom R&D, France
1. Introduction This paper is an introduction to middleware frameworks for distributed systems. The aim is not to cover all the scope because a complete book would not be sufficient for that purpose, but to explain some of the recent architectures still considered as new. These are currently not often deployed but they are already promising many success stories in the near future.
2. Need for middleware frameworks As operating systems manage the collaboration of tasks on a given computer, a set of applications distributed over interconnected machines needs equivalent services but with coherency guarantees: this is the role of middleware. Furthermore, the actual trend is to put most of the middleware services on standalone machines. For instance, the discovery services and the event services proposed by the OSGI framework can efficiently be used on standalone computers. So many actual component-based approaches for the conception of operating systems are proposing middleware services as main foundations. The common method for building any software application is decompose into components that application (distributed or not). The components export interfaces the only way to use them (interfaces for controlling, emitting or receiving data, configuring, testing, securing, billing, debugging, etc). The distribution should be considered as a process for optimizing hardware resources, so the implementations should be automatically generated.
62
Communicating with Smart Objects
3. Some middleware frameworks 3.1. The Jini framework Jini middleware provides the main services of any framework for distribution applications. The same services are also provided in .Net (UpnP/Soap) from Microsoft, and also in Corba from OMG, the only difference being in the way it is implemented. In order to facilitate the distribution of Java applications, Sun has developed a framework that extends the remote method invocation (RMI). The service provider registers his interface (according to the language definition) for a given time period at a registry service with attributes specifying more applicative and non-functional properties (name, keywords, version, author etc). Querying of registry service are performed by multicast messages, the code and attributes transportation are performed by the mean of http servers (Figure 6.1).
Figure 6.1 The Jini framework
64
Communicating with Smart Objects
When querying a service, according to attributes and an interface used as query parameters, the interface is downloaded, in order to build a surrogate that manage Remote Method Invocation via RMI for the implemented service. In the Jini technology, three different types of discovery mechanisms are used: •
•
•
The multicast query protocol is used by the entity needing the nearest available registry service, also called lookup service. This protocol is activated at the initialisation of all Jini services. The multicast announcement protocol is used by the registry service in order to notify its availability after initialisation. It is also used when restarting all the services after a break in the network. The unicast discovery protocol is used by the entities for communicating with a registry service. It is used when services are on a different sub-network and the multicast does not work.
3.2. The UpnP framework First proposed by Microsoft and now supported by the consortium Universal Plug and Play Forum, the UpnP framework specifies descriptions of services and devices. With UpnP, devices can dynamically connect to a network, obtain the IP address, provide its properties and look up the availability and properties of other devices, all these steps automatically. In other words, devices can communicate directly with other devices and establish a P2P architecture. UpnP relies on the standard TCP/IP protocol and on other internet protocols. The basic building blocks of UpnP framework are: devices, services and control points. A control point send actions, devices receive actions and a Node is either a device or a control point. 3.2.1. Service discovery When connecting to an UpnP framework, a device tries first to get an IP address with either DHCP protocol or AUTO-IP. Having an IP address, it can notify the control points the availability of his services. Control points will have similar behaviour when introducing in a network. In the both cases and also for withdrawing, multicast messages are generated (multicast httpmu queries and unicast httpu for answers). Control points use http protocol for exploring the URLs obtained during the registry phase. 3.2.2. Device control It performed through SOAP protocol, which transmits using HTTP the XML encoded remote procedure calls. Each control point uses SOAP for sending control
Introduction to a Middleware Framework
65
messages and receiving error reports. Relying on http, the SOAP messages can overcome firewalls and also use secure sockets. 3.2.3. Events General event notification architecture (GENA) has been specified in order to provide sending and receiving of notifications using HTTP over TCP/IP and multicast UDP. GENA specifies the notions of subscribers, notifying resources and subscription arbiters for routing the notifications, all handling the events. The models and headers of GENA are used in UPnP for notifying the availability or state change of services throughout the SSDP protocol simple service discovery protocol) also relying on Httpu and Httpmu. A control point interested by events will send a request containing the name or characteristics of the service, the return address for the events and the duration of the subscription. In comparison with Jini, UpnP is not language oriented, using mainly http and its variants httpu and httpmu as transport protocol which has the advantage of passing through firewalls and be lighter than using UDP/IP. 3.3. P2P framework Peer to peer (P2P) frameworks are parts of distributed architectures, the services described in the last chapters are encompassed in this new paradigm. Opposed to the hype surrounding emerging services such as Napster, Scour, Gnutella and others, peer to peer technology has existed for 80 years. It is the convergence of two trends: 1. A political one, aiming to decentralise fully the applications in order to optimise availability and scaling. 2. A technical one, following the increasing power of computers and high throughput of networks. Usenet and FidoNet are old but still efficient examples, illustrating the robustness of that technology. The first, as distributed application, provides transparent sharing of files, although it was conceived at the time that data synchronisation was performed at night using point to point modems. Here is the first main property ! Indeed, that was a constraint and is now a main characteristic: in P2P technology exchanges are peer to peer, each of the two partners having a priori the same resources (storage, communication, computing) and so on along all the nodes, each performing part of the tasks: a general function is performed (e.g. Napster proposes a musical resource sharing, Gnutella is rather a general file sharing service and Seti@home a computing
66
Communicating with Smart Objects
resource sharing; there are many messages sharing services such as FidoNet, AIM etc). Sun, very aware, already proposes a Java toolbox Jxta for P2P applications. Jxta is a collection of six protocols not all compulsory for building P2P applications. The heart of Jxta is constituted of managing and searching mechanisms for peer groups, data transmission, control and load balancing of channels, peer activity and networks. In addition, indexation, searching and sharing services are provided for the developers. 3.3.1. Discovery service in the P2P The discovery service allows peers to find themselves so they can interact; this service is, in fact, fundamental and essential. Here are some ways to realise such service: 3.3.2. The explicit way This is a method that avoid the implementation of discovery service, all the nodes know a priori the peers they are communicating with, but not all. This solution does not scale easily, because of the difficulty of maintenance and the lack of dynamic connection of new nodes. Nevertheless, this way simplifies security by fixing the names of the peers (i.e. authentication is implicit, access rights are simplified). 3.3.3. The dynamic way The first method relies on the existence of nodes with the directory where other nodes can register and search others. These directories can also be peers (the DNS is an excellent example of hierarchical directories). This model facilitates directory management and provides a better quality of service, which is nearly centralised. However, as for Napster, it is also easy to stop that. 3.3.4. The networking way As the name suggests, all the peers are dynamic and none knows the whole network. Each peer has a list of possible neighbors and asks them to accept it as neighbor. Once all connections established (each peer is normally accepted by many neighbors) the distributed algorithms can be deployed. This list looks like the first, except that it is renegotiated along all the communication of the peers, one of the background task being to maintain the most efficient neighbors. This is the way for maintaining the list of the main active peers in the network, and so has grown Gnutella.
Introduction to a Middleware Framework
67
3.3.5. The diffusion When the multicast is available, without knowing how many and who are available, a broadcast message allows one to find the interesting and interested peers. This protocol is available in Jxta suite of Sun but is not suitable for the whole network, the multicast being usually stopped by the routers.
3.3.6. The security of P2P applications Security is mandatory when two entities share resources, when they do not know each other and are without any previous contact. The peer node authentication and the shared resource integrity and identity ask security to be guaranteed. This is a weakness of the P2P application. The level of security is the same as in all distributed systems: • • •
authentication:appears under both levels, authentication between peers and authentication of users; authorisation: in order to work on a resource after authentication, authorisation gives rights on this resource, rights on reading, modifying or running; encription: it plays several roles: - it renders unintelligible data flow between peers in the Internet network which does not offer any security; - it guarantees data integrity and authentication through signature of both clients.
4. Conclusion The need for software infrastructure to build distributed applications is essential to controlling complexity and following the more and more dynamic aspect of applications. Nevertheless none of infrastructures described above is predominant. For instance, it exists at the moment very few hardware components compatible with UPnP or Jini, and other competitor standards arrive such as Havi for audiovideo appliances. The architecture job consists in understanding common features as well as underlining interesting properties of every infrastructure, in order to use the best of each.
5. References The work group Peer-to-Peer is a consortium for the establishment of a standard in the domain: http://www.peer-to-peerwg.org O'Reilly's site, http://www.openpep.com/, is dedicated to this technology
68
Communicating with Smart Objects
The practice of security: http://www-106.ibm.com/developerworks/security/ The consortium OSGI: http://www.osgi.org/ Gong, Li Sun Microsystems: Project JXTA: A Technology Overview. 2001 http://www.jxta.org/project/www/docs/TechOverview.pdf Jini™ Technology Architectural whitepapers/architecture.html
Overview,
1999
http://www.sun.com/jini/
Understanding Universal Plug and Play White Paper 2000 http://www.upnp.org/ download/UPNP_UnderstandingUPNPTomFRaviRaoEditsFinal.doc JXTA vl.0 Protocols Specification 2001 http://www.jxta.org/project/www/docs/ ProtocolSpec.pdf UPnP Device Architecture June 2000, UPnP Forum Technical Committee http://www.upnp.org/download/UPnP_Device_Architecture_Generic_20000710.ppt Jini Technology Core Platform Specification October 2000, http://www.sun.com/ jini/specs/core 1_1 .pdf Jini Architecture Specification October 2000, http://www.sun.com/jini/specs/ jinil_l.pdf Jini Device Architecture Specification October 2000, http://www.sun.com/jini/ specs/devicearchl_l .pdf Universal Plug and Play: Background 2001 http://www.upnp.org/forum/default.htm Universal Plug and Play Device Architecture June 2001 www.upnp.org/ download/UPnPDAl0_20000613 .htm Rekesh, John UPnP, Jini and Salutation - A look at some popular coordination frameworks for future networked devices June 1999 http://www.cswl.com/ whiteppr/tech/upnp.html SOAP http://www.w3.org/TR/SOAP SOAP FAQ http://www.develop.com/soap/soapfaq.htm SSDP http://www.upnp.org/draft-goland-http-udp-04.txt SSDP http://www.upnp.org/download/draft_cai_ssdp_v1_03 .txt GENA http://www.upnp.org/draft-cohen-gena-client-01 .txt
Introduction to a Middleware Framework
69
XML http://www.w3.org/XML/ Chakraborty, Dipanjan Chen, Harry Discovery in the future for mobile commerce Jan 2001 http://www.acm.org/crossroads/xrds7-2/service.html AutoIP 03.html
http://www.alternic.org/drafts/drafts-i-j/draft-ietf-dhc-ipv4-autoconfig-
Project JXTA:An Open, Innovative Collaboration 2001 http://www.jxta.org/project/ www/docs/Openlnnovative.pdf
This page intentionally left blank
Chapter 7
A Model and Software Architecture for Location-management in Smart Devices/Ambient Communication Environments1 Thibaud Flury, Gilles Privat and Naoufel Chraiet France Telecom R&D, France
1. Introduction Location is the most obvious and easily exploitable piece of information upon which context-aware applications can rely. Based upon the narrowly-defined requirements of specialised location-based services, a number of programmatic interfaces, textbased or binary formats and protocols have been proposed to handle it in a more or less ad hoc and piecemeal fashion [2], most of them failing to capture the more general theoretical framework in which the location problem could be set. The grand idea behind using physical location to address information is to bridge the gap between the informational and physical worlds, which prevailing desktopcentric interface metaphors have contributed to widen. This goes much beyond the cellular-network-location-based services already on offer, and even beyond the more general context-awareness idea. Such a far-reaching vision of "information in places"[3],"worldboard"[4], "situated information spaces"[5], has been eloquently articulated, and partially illustrated in such projects as MapPlanet [6] and Confluence[7]. In this view, the physical world is the most compelling interface metaphor to cyberspace, and geo-location is used, not only as the user's own position, but as a unifying navigation anchor and an intuitive representational tool, to make sense of the overwhelming multidimensionality of the information space. As such, location information may be used in either an abstract or concrete sense, from the smallest to the largest possible scale. What is located may be either the user himself, the information he retrieves, the physical objects or the other people with whom he interacts.
1 Work partially funded under the ITEA Ambience Project.
72
Communicating with Smart Objects
This paper attempts to give a broad outline of a location architecture that could encompass the diverse requirements of all these potential applications. We begin by specifying the requirements for a location infrastructure with different models of physical space corresponding to increasing levels of abstraction. From this we try to describe a general template for a location infrastructure that draws upon an analogy to the layered models of network protocols and examine one first example of a minimal implementation based on Jini . 2. Requirements for a location infrastructure A location infrastructure should become a basic service of ubicomp environments, just as basic in fact as naming or directory services may be in regular network environments. It should establish a correspondence between location entities (e.g. regions of physical space) which we will, for the sake of this description, call locus/loci and locatable entities which we will call locants. In a typical ubicomp environment, a locant may be either a passive object fitted with a tag, a physical networked device with information/communication services attached, or a human user (with or without a device). In a broader view of "situated information space", it might be any abstract entity, such as a purely informational service, or a piece of static content data (e.g. an entry in a bulletin board or yellow pages service), not necessarily attached to a physical device but linked to some more or less abstract instance of a location concept. Physical location may as such be viewed as a unifying addressing mechanism, whether used directly or indirectly. The proposed solution should be independent of location-sensing technologies [8]. There are two ways in which location information may be used. The first is for the location infrastructure to generate, when locants move through loci, location events forwarded to the interested subscribers. These events may have to be filtered by an intermediate agent to retain only the relevant moves, depending on the space model and the application concerned. The other is to respond to either direct or inverse location queries, as detailed below. In both cases, information has to be provided in real-time by the infrastructure. We will not try, for the time being, to fully integrate the temporal dimension in the infrastructure, which would amount to take into account the complete trajectory of a locant trough various loci.
2.1. Direct location queries The primary type of query a location infrastructure is required to support is: "where can I find something?". In our model, this amounts to providing the identification (this might be incomplete, using properties or attributes) of a locant as input to the query, and expecting the identification of one (or a list of matching) locus as an output. Both input and output are relevant to a particular model of space understood by the application that spawned the query, as explained in the following section.
Location-management in Smart Devices
73
2.2. Inverse location queries The other kind of query a location infrastructure is required to support is: "what is there around here?" This amounts to providing the identification of a locus as input to the query, and expecting the identification of one or several locants as output(s). Again this query may be formulated in a semantic, human-understandable way like, "what can I find in this building", or in a lower-level metric way like "what can I find within aim radius from these WGS84 coordinates".
2.3. Composite queries Many applications scenarios will correspond to direct location queries followed by inverse location queries, especially in the case of a mobile user requesting first to be located by way of their mobile device, then to know what service of a given type are available in this neighbourhood. In a general case, a composite query is composed of a set of elementary ones (in some sort of composition).
3. Location models Various models of space do always implicitly underlie location infrastructures, [2], [9], yet they are rarely set in a proper theoretical framework by going back to the basics of what a space can be in pure mathematics. Though these models are purely abstract, they will be used in association with a particular location-sensing or ranging technology, from which they retain only relevant characteristics that can be mapped to a corresponding notion of space. They will serve to characterise both loci and locants.
3.1. Set-theoretic In this strictly minimal model, no metric information whatever about the shape of a locus or the precise position of a locant is assumed to be available. All that can be known is the presence/absence of a locant in a locus, modelled as an element belonging or not to a subset of space.
3.2. Topological In these models, location properties are defined on a point-based abstraction of a locant, a locus corresponding (loosely) to the mathematical concept of neighbourhood in a topological space. Location is defined relative to a point, with special properties attributed to a given neighbourhood of this point that may not be
74
Communicating with Smart Objects
fully characterisable in metric terms. The spatial continuity property inherent in this topological notion of neighbourhood may itself be useful in some cases, yet places strong constraints on the underlying physical location technology if taken absolutely. Other purely topological notions such as simple connectedness (absence of "holes" in an open set) may also be relevant. Quite different in mathematical terms, yet related for being an alternative potential abstraction of RF location technologies, would be a location model based on fuzzy set theory, on which we will not elaborate.
3.3. Metric These models assume the minimal possibility to quantify a distance between points. The physical size of a locant itself may also be metrically bounded, making it possible to go beyond the point-based abstraction of a locant. A locus may also be a similarly bounded region.
3.4. Affine and affine-euclidean Affine and affine-euclidean spaces are a richer and practically more significant case of metric space, where it is assumed that absolute location information of a locant may be defined with respect to a suitable coordinate system. Geodetic coordinates are practically the most important example, with three main classes of coordinate systems may be used: • • •
Geocentric cartesian coordinates. Polar geographic coordinates (latitude, longitude, elevation). Planar projection coordinates.
A locus may be an arbitrary region of space defined with respect to such a coordinate reference system, while richer properties may also be attached to a locant, such as its precise geometrical shape or its orientation.
3.5. Locant graphs A completely different mathematical species, where the vertices (nodes) of the graph are locants, and loci may correspond to various subsets of this graph (e.g. paths, walks, cycles, or arbitrary subgraphs). This may be used in conjunction with a metric model (yielding a valuated graph) with a location technology that is purely relative and bilateral between objects themselves, rather than related to more or less fixed loci. Such a model is also compatible with a purely distributed management of location, eschewing any fixed infrastructure for both the location devices and the software location infrastructure: each locant may manage its own neighbourhood of
Location-management in Smart Devices
75
objects, actually those objects for which it has a bilateral relative location information. General graphs may be used to model all kinds of bilateral or multilateral relationships between their nodes, besides relative location as put forward here. Of course, all purely network-based models of communication already use such models, and this is not what we are attempting to reinvent...Other relationships may be described for which location may still be used as a metaphor, by extracting topological properties from the graph itself. Semantic relationships may, for example, be described in a structural way, enabling inverse location queries similar to those that may be made in a physical location model.
3.6. Loci graphs By contrast to the previous case, vertices of these graphs are the loci and not the locants. These models may be seen as enrichments of a set-based topological model, modelling not only loci as subsets or neighbourhoods of space but also their structural relationships. This is implicitly the kind of model underlying the cell pavings used in cellular networks, where adjacency relationships between cells are used for the handover of a locatable entity from one cell to another. Adjacency is but one particular case of relationship, and many other kinds of region structurations could be modelled: a hierarchical model of space (loosely underlying most of the semantic models used in directories) is another obvious case.
3.7. Semantic/symbolic models In these models, spatial location may be defined implicitly rather than explicitly, by reference to more or less abstract concepts relevant to a given universe of discourse, i.e. a semantic frame of reference. Loci may correspond to such divisions as streets, precincts, municipalities, regions, states, as used in regular directories. At a lower level and a smaller scale, buildings, floors, rooms, or even shelves in a cupboard, cells on a given shelf, etc could be used as loci providing a spatial reference for all kinds or locants, which will themselves be defined by some supposedly well-known characterisation rather than their physical properties. These loci may themselves be mapped to one of the lower-level models described before, i.e. either a hierarchical graph model, a topological model or a metric model. These characterisations may be compounded with other non-univocal high-level properties associated with a particular locus. These may correspond to a typing or profiling of a particular locus (e.g. authorisation, security constraints, electromagnetic compatibility). Though these models are purely abstract, they will be used in association with a particular location-sensing or ranging technology [8], from which they retain only relevant characteristics that are mapped to their model of space. They may be used to characterise both loci and locants, as detailed in the following table.
76
Communicating with Smart Objects Table 7.1 Location models
Model Set theory Fuzzy set theory Topological space^ Metric space
Affine
Affineeuclidean
Information provided Presence/ absence of a locant in a locus
Locus concept A subset of space
A [0-1] degree of presence of a locant in a locus Presence/ absence in a neighbourhood
A fuzzy subset of space
Relative distance to locant Position of locant w.r.t. absolute coordinate reference system
An open ball
Position+orientation ~ >mapping (translation+rotation) from absolute to relative coordinates Path to locus
Point or region of space, defined from absolute coordinates
Locus graph Path to locant Locant graph Semantic/ symbolic models
Symbolic/semantic mapping to underlying models
An open set/ neighbourhood
Point or region of space, defined from absolute coordinates
Node of the graph & locus previously defined A sub-graph (e.g. a path, a tree, a cycle)) Semantics of a locus defined in the previous models
Locant concept Usually abstracted to a point Usually abstracted to a point A point, or more generally a closed set A point or closed ball Point or region of space, defined from relative coordinates Point or region of space, defined from relative coordinates A locant for the underlying model A node (vertex) of the graph Semantics of a locant defined in the previous models
These models are to be used in combination, each providing a different kind of information based on a different conceptual view of location.
r\z
The spatial continuity property inherent in the topological notion of neighbourhood may itself be useful in some cases, yet places strong constraints on the underlying physical location technology. Other purely topological notions such as simple connectedness (absence of "holes" in an open set) may also be relevant.
Location-management in Smart Devices
77
4. A generic location architecture Fulfilling the above requirements calls for an architecture that jointly articulates these different models of location.
4.1. A layered architecture model for location We propose to define a layered template for this architecture that draws inspiration and is conceptually similar to those used in generic network services and protocols: the bottom layers are closest to the physical properties of space, and as we go higher they get more and more abstracted away and closer to concepts understandable by human users, much as we move up from physical connection, to MAC addressing, to IP, then to DNS and possibly UDDI addressing, in network-based identification protocols. This model juxtaposes two vertical categories of information, orthogonal to layers, corresponding respectively to loci and locants.
Figure 7.1 General location architecture model
78
Communicating with Smart Objects
4.1.1. Physical layer By analogy to the physical layer of network protocols, this is the lowest level of our architecture, directly related to the location-sensing and identification technologies used: from the variety of technologies available, we can distinguish two (overlapping) categories: technologies that identify locants, and technologies that locate them in space. It is only by combining the two kinds of sensor that you can track a specific locant through space. The identification sensor itself may provide some minimal kind of location information if it has a limited range, asserting the presence of the identified locant within this range.
Figure 7.2 Physical layer
The relation between entities and sensor can be seen in the two directions, because with some technologies, the sensor activately searches for objects, or the entity can itself announce its presence, both for identifying or locating. The final aim of this layer is to provide a relative location for an identified entity (the sensor may not be unique but it could result from the combination several of them) it could be vague (near some identifying sensor) or more accurate. But at this state it is only relative to the sensor(s); it has no real sense for the physical space.
4.1.2. Geometrical/topological layer At this level, relevant loci are geometrically defined as sets or neighbourhoods that aggregate information coming from the physical layer, depending on the actual distribution of sensors through space. Position information for a sensor allows mapping of the relative position information it provides to a more global coordinate reference system.
Location-management in Smart Devices
79
Figure 7.3 (detail) Exchanges between physical and topological layer, loci part
Figure 7.4 (detail) Exchanges between physical and topological layer 4.1.3. Structural layer In this layer, multiple locants and loci will be associated in complex ways to model the structural relationships between them. This corresponds to a graph-based model for which vertices (nodes) may be either loci or locants. In the first case, these models may be seen as enrichments of a set-based/topological model, modelling not only loci as subsets or neighbourhoods of space but also their structural relationships. This is implicitly the kind of model underlying the cell pavings used in cellular networks, where adjacency relationships between cells are used for the handover of a locatable entity from one cell to another. These models are also used in navigation systems where it is necessary not only to locate the user,
80
Communicating with Smart Objects
but also to find a route for him to some destination. Adjacency is but one particular case of relationship. A complementary hierarchical model loosely underlies most of the semantic models used in directories, but is also an implicit model for the space within a building, as decomposed in floors, rooms, cabinets, etc. A much more abstract view of location corresponds to the case where the vertices of the graph are locants, and loci may correspond to various subsets of this graph (e.g. paths, walks, cycles, or arbitrary subgraphs). This may be used in conjunction with a metric model (yielding a valuated graph) with a location technology that is purely relative and bilateral between objects themselves, rather than related to more or less fixed loci. General graphs may be used to model all kinds of bilateral or multilateral relationships between their nodes, besides relative location as put forward here. Of course, all classical network models are based on graphs modelling their connectivity relationships, and this is not what we are attempting to reinvent. Other relationships may be described for which location may still be used as a metaphor, by extracting topological properties from the graph itself. Some semantic relationships may, for example, be described in a structural way, enabling inverse location queries similar to those that may be made in a physical location model. 4.1.4. Semantic layer In these models, spatial location may be defined implicitly rather than explicitly, by reference to more or less abstract concepts relevant to a given universe of discourse, i.e. a semantic frame of reference. Loci may correspond to such divisions as streets, precincts, municipalities, regions, states, as used in regular directories. At a lower level and a smaller scale, buildings, floors, rooms, or even shelves in a cupboard, cells on a given shelf, etc could be used as loci providing a spatial reference for all kinds or locants, which will themselves be defined by some supposedly well-known characterisation rather than their physical properties. These symbolic locus descriptions will themselves be mapped to one of the lowerlevel models described before, i.e. either a hierarchical graph model, a topological model or a metric model, and this correspondence has to be accounted for by the architecture. These characterisations may be compounded with other non-univocal high-level properties associated with a particular locus. These may correspond to a typing or profiling of a particular locus (e.g. authorisations, security constraints, etc).
Location-management in Smart Devices
81
4.2. Direct location search queries One of the main goals of the infrastructure shown before is to respond to a query for direct location. The query is of the type "Where can I find something?" something being a communicating entity and the response to where being a locus. So the general path to respond to the query is from the entity to locus.
4.2.1. Targets of the query In a human understandable way, the something is likely to be expressed with a semantic definition (like where is the coffee-machine?). You can also research an entity upon the relation it may have with other entities that build an higher level device (for example a streaming server which can be linked to a video client such as a giant TV flat screen, the query would be, where are the TV located?). Finally you can search for an entity with its low level identification (like with an Ethernet address or an unique URI), but that sort of request should be transparent for users and only accessible for performance purpose (because it skips in some way the infrastructure model).
4.2.2. Path followed by the query On the entity column, the request follows an up to down path, being transcoded layer to layer. It may descend to the physical identification before being translated to the locus column, and then it follow an upward path, stopping at the level required by the asker.
82
Communicating with Smart Objects
Figure 7.5 Direct location query
4.2.3. Type of answer The response could takes many forms too (depending on the need of the request), it could be an absolute positioning in a specific reference frame (for example GPS coordinates if you are outdoor). The response could be the neighbourhood in which the object is located (for example, the coffee-machine is inside room F107) or the structural path to access to it (the coffee machine is on the first floor of the building, section F, between the room F105 and F109, in front of the corridor etc). And finally, it may be a semantic response, especially if the asker is human (the coffee machine is in the cafeteria of section F). Of course these answers could also be combined to fulfill the requirements of an application (especially if you're using a navigating software to guide you to the coffee machine).
Location-management in Smart Devices
83
4.3. Inverse location search queries As explained in the requirements, there is another type of query which is useful, about inverse location search, responding to question like "what entities can I find there" there being a locus. It may be seen on the opposite side, from locus to entities.
4.3.1. Targets of the query The query is likely to be addressed nearly anywhere on the locus column. From the semantic viewpoint, very understandable for human beings with questions like "who is in the conference room?" (people considered as communicating entities through the use of pda or mobile phone) for example. In this case, the place is semantically known to be a conference room and semantic properties are attached to the entity searched (in fact humans, not coffee-machine). But the query can target the structural relation between loci (answering to query of the type "what device can I use in the room next door" or "what can I find inside this entire building"). For proximate selection, a lot of queries are of the type "what services can I use near me". "Near me" can be understood as in the same set of space, or in my neighbourhood, or near my absolute position. Neighbourhoods are also targeted for not proximate selections (asking about entities in neighbourhoods or sets in whose you are not). Absolute positioning can also being used if you have no idea of the semantic identity or the set of space concerned with the position you are asking for (particularly useful if you are outdoor). Finally, but this should not be used, is the direct targeting of the sensors (you ask about the relative positioning or identification of located entities).
4.3.2. Path followed by the query The way of the request is nearly symmetric to direct location query, going down on the Locus column, and up in the entity one.
4.3.3. Type of answer As in direct location queries, the response can be of different types, fulfilling the specific needs of the application asking. It may be some basic identification of the
84
Communicating with Smart Objects
entities responding to the criteria formulated in the request found () or directly some high level semantic definition of it, depending on the possibility of the subjacent.
Figure 7.6 Inverse location query 5. A Jini-based discrete location architecture As a simple illustration of these ideas, we present previous work on the first Jinibased implementation of a location architecture. This project was intended to demonstrate small-scale location-based services within a room. As a user enters or leaves the room, or gets close to devices within the room, the environment reacts to is actions. The smart devices spontaneously propose their services to the user according to his/her position.
5.1. Goals of the project The major ideas was to use Jini to show the concept of inverse location search and to break apart the link between the physical world of detection technologies (very low level information) and location data used at an higher level by the infrastructure. So data used by applications were independent from the hardware used. In this first
Location-management in Smart Devices
85
exploratory work, the location model used was a set-theoretic discrete location scheme using neighbourhoods (the world being divided into sets and neighbourhoods within which you are located or not). Locatable entities were determined as belonging to these regions of space and the mobile entity in the world was the user with a PDA. When the PDA (and so the user) enters a set of space, all services (linked to smart devices) available by proximity are shown to him via some sort of graphical user interface (that also indicates the direct positioning of the pda in the world).
5.2. Location technologies used The detection technology used is mainly RFID because it can output an accurate identification and if adequately placed in space offers reliable information on discrete location (this locant enters in this room for example). To illustrate this, the PDA carried by the user is bound to an RFID tag, and there are RFID tags on the phycons (such as the books in our library). Another discrete location technology has been used, a sensitive pad that detects the presence of locant by pressure.
5.3. Software infrastructure This project has been implemented on top of the Java platform for all its portability and cross-platform advantages and RMI for remote procedure call through the network. For the service discovery infrastructure, we have chosen Jini because it was the more open and used middleware available at the beginning of the project. Staying in a Java world, we have used JMF for multimedia streaming. And accordingly, to the use of Jini, we used Javaspace for a kind of concurrent-access database as a normalised service within the middleware.
5.4. Interaction demonstrations The demonstration is for now constrained to a single room, and based on a single user equipped with a pda and its wireless connection. Services are dynamically offered to him depending on its location. When he enters the room, a window automatically appear on the pda screen, showing the direct location of the user (the room, the floor and the building), and a list of the services available in the room. For example, there is a robotic arm in our room that can be remotely controlled by the user with its pda. Another example of using location as interface is when the user gets close to the bookshelf, an inventory of it automatically appears on the pda screen. Or when the user takes a book (in fact it is a locant, a phycon whose role is to present the path of a multimedia content, for example a movie trailer or a song) from the bookshelf, relevant services are spontaneously offered. If this is a movie, so there is video content, the TV flat screen exports a control interface to the pda screen and gets ready to receive video streaming. The dolby prologic speakers system does
86
Communicating with Smart Objects
the same for audio. And, if possible, a multimedia server that can access to the content is also offered. We can therefore consider that a higher entity of distributed multimedia interface is spontaneously created upon the relations between some of the locant. But the problematic of the distributed interfaces through space, time and modality is another topic. Another extension not yet implemented of our system is to permit follow-me applications (for example with the sound that follows the user with no interruptions if the rooms have some kind of smart audio devices) but it could not be possible while we do not have established the structural layer that links the loci. 5.5. Implementation of the location architecture Our architecture is a simplified adaptation of the general model discussed before, mainly for study purposes. On this first work, the layers and blocs communicate with three main mechanisms. It functions using the possibility of discovering and the advertising of Jini and remote or direct methods calls (we have developed a shared panel of basic interfaces, in the Java understanding to have a little a priori about how to communicate with the main blocs). It also uses the mechanism of events to trigger events quickly (the Java events have been overloaded for this specific purpose). The last mechanism used is a shared access to some values and properties through a common space (in some sort of basic database, specifically useable in Jini known as Javaspace). Of courses these three ways to communicate are combined depending on the case (for example, the Javaspace might throw an event if some specific value is changed, and that can trigger a remote call of a method).
Location-management in Smart Devices
87
Figure 7.7 An example of implementation of the general architecture 5.6. Description of this particular architecture
At the lower physical level, sensors detect locants in space. They relay their data to the concentration service that centralises the data, filters them, and relays them to the identification service and the concerned neighbourhoods. The data relayed is in this case a basic relative location information with the presence of entity regrouped into a particular neighbourhood. The Identification Service links the low levels identifications brought by the detection services to higher levels (it identifies the locant, themselves offering a service recognised by the Jini infrastructure). It is also designed to manage the profiles for users, knowing if a client is allowed to use a particular service. Each Neighbourhood Service manage a physical zone and register clients within it (clients of the neighbourhoods are usually the services associated with the locants). These services also relay (if asked) the information about the entering or the leaving of the clients in the controlled area.
88
Communicating with Smart Objects
Location Registration Service is the bridge between clients and the location architecture, because a client a priori does not know in which neighbourhood he will be registered. Kiosk service shows the different services available through the use of the neighbourhood's registered clients. 5.7. Limitation of the current implementation This preliminary work, if working quite correctly, lacks some very important concepts about location. In the first place, it is relevant for a single and very specific understanding of space. At the lower level, we do not integrate technology for continuous localisation, or a more accurate relative positioning sensor. So with only punctual and discrete information, we can only use theoretical sets and topological models. The structural layer have not been taken into account up to now and as a consequence, relationship between loci cannot be used, and data are not persistent when the user move and changes neighbourhood. The qualifying of the semantic layer of the top level interfaces is for now an abuse of the language because it does really make sense only in a programmatic way in a Jini world; we cannot base any request on a semantic definition of loci or locant (neither on semantic properties). And finally, the really big hole in our implementation is that it cannot really interoperate with heterogeneous applications or services because it does not support any declarative interface for queries.
6. Conclusion: integration of location-management with service discovery infrastructures Service discovery infrastructures such as SLP[15], Jini™[16], UPnP™[17], Salutation[18] and UDDI[19] attempt to move up from purely network-based addressing, to account for higher-level descriptions of networkable entities. They provide a bootstrap mechanism that makes possible the dynamic, spontaneous hookup of services and devices in ubicomp environments. They use, in a centralised or distributed fashion, a generalised lookup service that may build upon and subsume the more specialised naming, trading or directory services provided by underlying middle wares and protocols. As such, both direct, inverse and combined location queries should be directly supported by these infrastructures. We are still far from this objective, if only for the lack of widely adopted representation standards for location information and associated data, at all the different levels described before. Programmatic interfaces for services and their attributes, such as used by Jini™, are both highly expressive and close to implementation (that is why they were chosen in the above-described implementation), but place a probably too high bar on interoperability. Declarative XML-based interfaces are a more pragmatic solution for minimal interoperability
Location-management in Smart Devices
89
and could make it possible to interoperate at all levels of the architecture described above, provided standard DTDs, (preferably schemas), become widely adopted for each of these. There is already a substantial amount of work on this, which unfortunately has led to the definition of different and incompatible DTDs (or, worse, binary formats) by all the different consortia and standardisation bodies interested in this topic, from geographers to mobile telecom operators, to specialists of 3D graphics[2]. For the lower layers where relevant ontologies are fairly straightforward and could be agreed on by all parties, this multiplication of overlapping and competing would-be standards could have been avoided. As usual, a shakeout will let one of these emerge, but this may take some time. For the structural and semantic layers, the nature of the concepts to be manipulated is much more open and an a priori definition more difficult. Here, as in the general issue of semantic service discovery [20], a meta-level specification of location ontologies should be possible: we could have, for example, different models for intra-building location based on different kinds of architectural entities, or culturally-dependent street/precinct hierarchy models for describing location in an urban area.
7. References [ 1 ] www.extra.research.philips.com/euprojects/ambience/ [2] Mari Korkea-aho, Haito Tang: "Experiences of Expressing Location Information for Applications in the Internet", Proceedings of the Workshop on Location Modeling for Ubiquitous Computing, Atlanta, Georgia, September 2001. [3] J.C. Spohrer, "Information in Places", IBM Systems Journal, vol 38 n° 4. [4] www.worldboard.org [5] G. W. Fitzmaurice, "Situated information spaces and spatially aware palmtop computers", CACM, vol 36 n° 7. [6] www.mapplanet.com [7] www.confluence.org [8] J. Hightower, G. Bordello, "Location Systems for Ubiquitous Computing", Computer, August 2001. [9] Svetlana Domnitcheva: "Location Modeling: State of the Art and Challenges", Proceedings of the Workshop on Location Modeling for Ubiquitous Computing, Atlanta, Georgia, September 2001. [10] Martin Bauer, Christian Becker, Kurt Rothermel: "Location Models from the perspective of Context-Aware Applications and Mobile Ad Hoc Networks",
90
Communicating with Smart Objects
Proceedings of the Workshop on Location Modeling for Ubiquitous Computing, Atlanta, Georgia, September 2001. [11] Thomas O'Connell, Peter Jensen, Anind Dey, Gregory Abowd: "Location in the Aware Home", Proceedings of the Workshop on Location Modeling for Ubiquitous Computing, Atlanta, September 2001. [12] Barry Brumitt, Steven Shafer: "Topological World Modeling Using Semantic Spaces", Proceedings of the Workshop on Location Modeling for Ubiquitous Computing, Atlanta, September 2001. [13] Natalia Marmasse, Chris Schmandt: "Location Modeling", Proceedings of the Workshop on Location Modeling for Ubiquitous Computing, September 30, Atlanta, Georgia, 2001. [14] www.autoidcenter.org [15] www.srvloc.org [16] www.sun.com/jini [17] www.upnp.org [18] www.salutation.org [19] www.uddi.org [20] R. McGrath, M. Mickunas, R. H. Campbell, "Semantic discovery for Ubiquitous Computing".
Chapter 8
A Software Infrastructure for Distributed Applications on Mobile Physical Objects Mohammed Ada-Hanifi, Serge Martin and Vincent Olive France Telecom R&D, France
1. Introduction In this paper we present some ideas related to work started in the POPCORN project (MOA: DS) and, pursued in the Cybermonde/Pervasif project (sub-project 4). The earlier results were exploited within the cocooning DIN project, in order to develop a generic remote command. We deal in this context with distributed services in a house. The technologies presented are equally able to cover a range from small things (person or sensor) to large things (national services). This paper is divided into three parts. In the first part, we explain the technology devoted to installation and updating software services involved in the home automation field, through a framework, specified by the OSGi consortium. Moreover, the intended and unintended advantages and limitations of this technology are pointed out. In the second part of this paper, this technology is extended to allow adapted distributed applications to be used by implementing Jonathan middleware is described. The properties of this extension are underlined. Finally, in the third part, we present via an example the first practical results obtained for the generic remote command. 2. The OSGi infrastructure Over the last decade, the concept of the global network has increased in significance with the widespread deployment of the Internet. At present, it is not the monopoly of big companies, but it is deployed, too, at the residential level. At the same time, major evolutions have occurred within equipment and terminals, using more improved processors and electronic chips both in terms of execution speed and memory size. Since these devices have become more smart, and are able to
92
Communicating with Smart Objects
communicate and interact together, new prospects have arisen concerning interoperability between remote software/hardware environments. Thanks to these network-connected devices, it has become possible to read e-mail on the TV or on a cellular phone, or again to control the state of home equipment.
Figure 8.1 Interconnection of a home network to the internet network through OSGi gateway
2.1. The OSGi framework An industrial consortium of telecom operators, telecom equipment and home equipment manufacturers was created in 1999 in order to define and promote an open standard (framework) allowing the development and deployment of e-services in the home, accessible and remotely controlled on the Internet. This standard, written in the Java language, is named OSGi (Open Source Gateway initiative) gateway. Presently, there are more than 5 gateway products on the market, and some of them are less than 300Kbytes in size and can be concurrent candidates for embedded systems. 2.2. History and evolution In the 1.0 OSGi specification release, dated May 2000, the emphasis was put on the definition of basic APIs, intended for implementation in the framework and available to service developers. These APIs were written in Java. This language was chosen for portability reasons, its downloading capacity and its security properties. In the 2.0 release of the OSGi APIs, dated October 2000, security was strengthened and a special management bundle was added in order to fully define and control the security aspects of a bundle in real time. The minimal infrastructure of the gateway deals with: • •
The Java environment: the set of packages and classes required for the framework work. The Framework which defines APIs that load, create and launch the services.
Distributed Applications on Mobile Physical Objects • •
93
The Log service that collects information concerning the framework during execution. The Http service which defines the APIs that allow the launch and use of an http server.
This minimal infrastructure can be grown (enhanced) by more services; both by infrastructure services like (Jini, Corba, uPnP, Havi etc) and by application services (Figure 8.2).
Figure 8.2 OSGi framework architecture 2.3. Functional description and properties The OSGI framework architecture relies on some concepts that we describe below. For a given application, installed in the framework, it is assumed that it offers one or more services. These are encapsulated in a "bundle" that is a .jar file in which we put together the service and other files such as pictures, sounds, movies and all the resources necessary for the working service(s). It also needs to contain a manifest file, which describes the name and the contents of the bundle and an activator which manages the "starting" and "stopping" states of a bundle and /or a service.
94
Communicating with Smart Objects
Once a bundle is in the "installed" state, the framework is able to manage the services in the bundle. These are registered with their properties within the framework, and bundle states are dynamically managed (lifecycle management). The framework is informed about service dependencies and sends an event notification after each state change. Service launching is performed by a class that encapsulates the service and that manages the starting and the stopping states by two methods Start() and Stop(), defined and implemented in the service activator.
Figure 8.3 The lifecycle of a bundle 3. A middleware solution based on Jonathan The OSGi infrastructure architecture presented in the previous section offers infrastructure services accessible only in the execution space of the Java virtual machine. For example, a "trader service" that helps us to retrieve a service searches for it only in a local space of the framework. Thus it is impossible to retrieve a service that has not already been loaded and registered locally within the framework. This working hypothesis is justified when all the locally installed services are considered as an autonomous collection; it means that a set of the imported services is self-consistent. Bearing in mind this last hypothesis, what happens if we try to make communication between several OSGi frameworks, or simply if the framework needs to deal with other communication functions different from the loading function? If we formulate
Distributed Applications on Mobile Physical Objects
95
this in terms of object language concepts, we need to express importation and exportation of the services (represented by their interfaces) between different OSGi frameworks (therefore between different JVMs). Thus, when we propose a middleware based on an ORB such as Corba or RMI, this technology will allow us to extend the local OSGi service notion to a distributed OSGi service notion. In fact we want to extend this approach and provide to each OSGi framework an ORB (Object Request Broker) which allows it to communicate with another framework. We define two steps for this approach: 1. 2.
implementation of the infrastructure services needed to connect distributed applications; definition of a typical architecture for distributed applications.
3.1. Implementation of an ORB service If we decompose ORB services into elementary services, we have two sets: •
•
A set of offered services: - Synchronous or asynchronous remote method calls with parameters passed by value and/or reference. - Management of object references. - Management of naming spaces which provide a translation between a name and its object reference. - Services related to trading, events, and memorisation are useful, but are considered to be additional services and are not strictly necessary. A set of internal services, which are not exported since they are not usually part of an ORB but are rather part of an operating system (memory management, session management, multithreading management, protocol stack, etc). (http://www.objectweb.org) is an open source and provides the possibility to share services.
The first experiment considered the Corba personality of Jonathan naming service as OSGi services. As a result two new OSGi services are offered by the framework. They enable an application service to call another application service operating in a remote OSGi framework.
96
Communicating with Smart Objects
Figure 8.4 Service A and service B call an ORB to manage linking objects (proxy, relay, stub, skeleton)
Some remarks can be formulated concerning the introduction of an additional framework for managing the distribution of services: •
• •
Trading services are duplicated: one for the external services, and another for local services. Moreover, they provide identical functions; searching for services by their names and/or their properties. Their programmable interfaces are thus different. Do we need a naming service by framework for all non-local services, or do we need only a simple relay (proxy) pointing to an external naming service? Could we link the naming service and its relays in order to propagate the requests from one server to another so as to obtain the requested services?
In fact, all these combinations are suitable and correspond to several optimisations, each one taking into account physical constraints such as computing power, transmission speed and throughput, and memory size. •
•
•
A relay pointing to an external naming service can be sampled, for example, by a mobile architecture with enough memory size, obtaining the address of the remote naming service during its connection. This service is then connected either by linking or directly to other services available in the network. A local naming service can be illustrated by a PDA that register its self exported services. During a connection it allows accessibility to its internal services by exporting its own naming service. On the other hand the linking in the importation mode of naming services to the PDA corresponds then to all the available external services.
3.2. Definition of an architecture for a distributed application By exploration of different possibility to organise our naming service, according the same concern of homogeneity and to maintain the same approach, the other applications services will follow the same exploration. The built-in distributed
Distributed Applications on Mobile Physical Objects
97
application is organised on the basis of several services running in different frameworks. The criteria of the services distribution are not the aim of this paper, they take certainly into account an available resource optimisation function and notions like safe and security. In this paper only technical solutions are involved to ensure the distribution. When we put a distributed application at disposal we start by the search of its compound services: this search can be extremely complicated depending on the number and the function of the services involved. It is anyway a complete research field to try to express the composition of services and to deal with request on compound services. Another service thus arise, the loader: all the services are not necessarily reached locally or remotely. This property leads to a dynamic configuration of the services: the configuration can be adapted according to criteria such as physical localisation, availability of the services, shared carrying, autonomy, breakdown tolerance, etc. Finally the distributed application is in place. Some services are locally installed and other remotely installed, thus the application can be activated. This architecture briefly described will be applied and integrated for the home gateway achievement on a PDA within the DIN project ie Cocooning. The principles of architecture design developed here will be applied to the communication between an OSGi home gateway, installed on a PC, and a PDA on which all the home gateway services can be reached dynamically through their interfaces. The PDA can then drive the different multimedia services of the "familibrary project" like video recorder, picture viewer, photolibrary, hifi device, TV etc. From this last described context we present in the next section a simplified example of application for pedagogical reasons.
4. Example of application: the generic remote command When we use a remote command service we apply on one hand the principles described previously concerning deployment of services in an OSGi framework and on another hand exploitation of inherent services of the middleware Corba installed in an OSGi framework. The generic remote command is built as an application allowing command of remote equipment installed in home local network through a home gateway (OSGi framework). The remote command service was installed on a PDA and it was implementing the command interface of a service present on a remote gateway. As the application service is open, the remote command function remind itself open, so that is the significance of generic notion. The configuration ie the remote command (number and name of the command) can be programmed according to and by the remote controlled service.
98
Communicating with Smart Objects
The example presents the remote command application itself as a service of an OSGi framework, installed on a PDA. It controls through the middleware a video recorder present in a home network. This video recorder is connected to a HAVi network, where the management is assumed by another OSGi service, installed on a home desk computer (PC). We have here a typical example of communication between two OSGi gateways. We describe below the communications steps involved between the remote command and a home HAVi service (video recorder management).
4.1. Service presentation The HAVi service (video recorder management) is installed with two bundles on an OSGi home gateway: • •
the video recorder control service; the remote command relay service.
On the PDA, where we find the remote command, are installed two other bundles: • •
the remote command with a graphical interface; the video recorder control relay service.
4.2. Description of the communication mechanism We assume this application works in scenario "Plug and Play". It is the video recorder control that appears in the home network (either it is just powered on or it is just installed in the framework etc). It will activate the exchange necessary to find a command interface service. The sequence diagram (in Figure 8.5) presents the synchronous remote method calls between the different application services on the first part: remote command and video recorder, and the infrastructure services on the second part: trading and naming services. The four steps of the scenario are: •
• • •
Initialisation: each application service registers itself at the different infrastructure service, address of the naming service is assumed to be known beforehand. Search of the complementary service of the remote command by the video recorder. The active steps of the remote command. Escape of the remote command by the video recorder.
Distributed Applications on Mobile Physical Objects
99
This scenario assumes that the remote command is an available service. If it is not the case, the video recorder service can test steadily its presence or better it can register to the event service so as to be informed about apparition of the remote command service. Finally all the services are supposed present (but not necessarily running) on their respective frameworks. Now, for reasons of place, updating or flexibility reasons, these services could be loaded on demand. Thus when starting a service it follows: registration at a naming service, then at a trading service and finally at loading service.
100
Communicating with Smart Objects
Figure 8.5 Sequence diagram between video recorder (device) and the remote command. The diagram issue is an UML specification
5. Conclusion Starting from a framework, managing locally a set of services, we demonstrate the feasibility of extension by completing it with distributed software infrastructure services. This new distributed framework allows us to build distributed applications with higher static and dynamic architecture flexibility.
Distributed Applications on Mobile Physical Objects
101
Depending on the low use of our infrastructure, we think that this approach is an alternative to those proposed presently for mobiles, if we guarantee other properties like a high level of security on the framework, large scale integration and portability. The portability and large scale integration will be studied in Project Vision: Pervasif. The security aspect can match those studied in other projects (ex: SPDA, THINK etc).
This page intentionally left blank
Chapter 9
Integrating a Multimedia Player in a Network of Communicating Objects Jacques Lemordant Gravir-Inria, France
1. Introduction Heterogeneous groups of hardware and software components can be federated into a single network. We have developed a framework allowing MPEG-4 players to join a federation using Jini technology. MPEG-4 players are based on a classical clientserver architecture. By joining a federation, an MPEG-4 player is able to have access not only to predefined services associated with its application server, but also to dynamic services provided by the federation. Reciprocally the player is offering its display capabilities to devices in the federation and a connexion to its application server. This work has been done under the 1ST project SoNG (2000-2002).
2. Motivation and orientation 2.1. Sharing a world In his article What does it mean to share a World?, Bob Rockwell listed ten technical requirements that will have to be met by any interpersonal/interoperable VRML environment. These included the ability to: 1. 2. 3. 4.
Insert/Delete objects (e.g. avatars) in scenes at run-time (more generally: to modify the structure of the scene graph). Merge multiple sound streams from distributed sources into the shared scene's current ambient sound (e.g. voices over music). Track and communicate the state/behaviour of objects in real time (this implies a database of "who needs to know what how often?"). Allow (sets of) objects to be "driven" by users in real time (i.e. provide a UI as well as an API to runtime object control).
104
Communicating with Smart Objects
5.
Let imported objects become persistent (i.e. make them a permanent part of the scene). 6. Protect the scene from damage by imported objects (ultimately, this implies the whole range of data-integrity issues). 7. Assign objects to a series of different "owners" (to insure control over access to object behaviour). 8. Support persistent roles (for people) and rules (for scenes) (i.e. a use-model for scene/object access controls). 9. Link objects dynamically to external data/functions (in particular, to support authentication certificates). 10. Support the free exchange of information among objects (from chat and business cards to arbitrary data containers/streams). The concept of scene sharing includes not just multiple users but also multiple developers. It is the basis for all component-based applications, from the use of electronic cash to database access. One way to implement this concept is through sharing the scene graph by exporting it as a service in a federation and having downloaded code in the player be a client of the services provided by the federation. 2.2. Multimedia players in a federation of devices In dataflow systems used in scientific visualisation, multimedia players are considered as sinks or terminal elements of the system. Here, we consider them as devices. In our terminology, a device is a software or hardware component being part of a federation. An MPEG-4 device is a device capable among other things to play mp4 files. We can think of MPEG-4 devices using many kind of networking technology (IP, Bluetooth,...) and willing to speak to various kind of devices (HAVi devices, ...). The java2 world is in a good position to be the glue between them, but multimedia players are often embedded in an internet browser such as IE and consequently have access to a limited Java world specially with respect to networking. For example the SoNG 3D MPEG-4 player is a COM-ActiveX component. We have developed a bridge, specific to multimedia players, between the COM and Java worlds. This bridge could be reused for other players running outside of Internet explorer, on mobile devices without a JVM for example. To build this bridge, we have used the Jini surrogate architecture and the MPEG-J scene graph api. The original goal of MPEG-J was to have a parametric way to modify the multimedia content of the MPEG4 scene, but equally important is to be able to dynamically find and use external Java objects of all kind (directories, agents, servers, UIs,...). A Jini federation brings together many different types of devices with Java technology as the common base. With the use of Jini connection technology, devices such as cell phones, pagers, PDAs, and TV set-top boxes can speak a common language. There are three basic parts to the Jini Network technology, which makes it simple to use. They include:
Integrating a Multimedia Player in a Network
• • •
105
Lookup Service: where Jini technology enabled services announce their availability. Discovery Protocol: a process to find the required lookup service. Proxy Object: an interface to the Jini technology enabled service.
Figure 9.1 A federation of Jini services
These three simple parts are enough to understand how an entire Jini federation operates. The figure above gives an idea of such a federation. We have build in the SoNG project a theatre demo, described later, which shows how such a federation can work. The federation comprises Javacards, PocketPCs and MPEG-4 3D players as shown below. MPEG-4 players are client of the SmartCards and PDAs are client of specific display services offered by the player. For mobile users, a local Lookup Service can be found with the help of a general Lookup Service using geographical coordinates.
106
Communicating with Smart Objects
Figure 9.2 Jini services used in the theatre demo 3. Architecture specification
Our framework is made of two independent parts as can be seen on the next figure: •
•
A graphic streaming channel or interconnect made of: - SKELETONS inside the scene graph manager. - STUBS implementing a remote MPEG-J. A bi-directional application-level protocol has been defined such that the implementation of the SKELETONS and the STUBS can be done independently. Components to join a Jini network: -Surrogates. We have witten two surrogates. One is used to publish as a service the remote MPEG-J, the other one is specific to the theatrer demo of the SoNG project and is a client of the Jini services published by the smartCard. -Monitor. It's role is to tell the surrogate host that the MPEG-4 device is still on and to pass to the surrogate host the url of the surrogate(s) which will represent the MPEG-4 device inside the Jini federation.
Figure 9.3 Architecture of the MPEG-4 player showing the modules used for its incorporation in a federation of services
108
Communicating with Smart Objects
4. An example: the theatre demo This example shows a Jini smartCard services being used by the SoNG MPEG-4 player.We thank Laurent Lagosanto from Gemplus Research for helping us in integrating this research product from Gemplus in the SoNG theatre Demo. We also thank FT R&D for helping us when writing the XML messaging system between the javascript User Interface and the java MPEG-J layer above the scene graph api.
Figure 9.4 An e-commerce application: the theatre demo
An electronic ticket is stored in the smartcard if a valid pin code is entered using the synthetic layer2D keyboard. Its XML description is shown below: 03435 42678 ONCE
Integrating a Multimedia Player in a Network
109
<promise> Theatre de 1'atelier <sName>La directrice et le financier <seat>192 <seat>189 <seat>398
5. References K. Arnold, O'Sullivan, Sheiffer, Waldo, and Wollrath. "The Jini Specification". Addison Wesley, 1999. Ken Arnold: "The Jini architecture". Proceedings of the 36th ACM/IEEE Conference on Design Automation, June 1999. Jim Waldo: "The Jini architecture for network-centric computing". Communications of the ACM, Volume 42 Issue 7, July 1999. The Community Resource for Jini Technology, http://www.jini.org Jini network technology, http://www.sun.com/jini/index.html The Jini Surrogate Project http://developer.jini.org/exchange/projects/surrogate/ MPEG-4: Coding of audio-visual objects - Part 1: Systems, ISO/IEC 14496-1. Information technology. Multimedia Systems, Standards, and Networks. Edite par Atul Puri et Tsuhan Chen. Signal Processing and Comunications Series, ISBN: 0-8247-9303-X, 2000. Java™ 2 Platform, Micro Edition (J2ME™ Platform): java.sun.com/products/midp/ JAVA CARD™ TECHNOLOGY: http://java.sun.com/products/javacard/
This page intentionally left blank
Chapter 10
Reverse Localisation Joaquin Keller France Telecom R&D, France
1. Localisation and reverse localisation 1.1. Localisation The term 'localisation', when applied to a physical communicating object, refers to the function that represents a given object and specifies its physical location, i.e. the place where the object is situated. This is the function that describes the geolocation services, i.e. the services that depend on the object's physical location. Localisation of a physical object is, indeed, useful for the services that rely on the material or immaterial resources located near to the object1. But localisation is in itself not all-important; usually knowledge of the resources near to the object is enough to fulfill the service. Moreover, knowing the coordinates of an object does not guarantee ability to identify the resources located near to the object and/or the resources related to its location. So, to implement geolocation related services, localisation is neither necessary nor sufficient.
1.2. Reverse location Reverse localisation is, as its name implies, the reverse of the localisation function: it is the function that takes a physical location as an input argument and returns the communicating objects located near or associated with this location.
1
Not all geolocation services fit in this scheme. Particularly, this is not the case of services that rely more on the path followed by the object than on its actual location: management of vehicle fleets, computation of an "optimal" path for a vehicle (car, plane, boat etc) or a pedestrian.
112
Communicating with Smart Objects
And among the "objects" that are associated with a given location, it is possible to include the databases that register the non-computational resources located or related with this location. Reverse localisation completes and extends the localisation of communicating objects, and if both are available, all localisation problems are probably covered. But, are all these functions needed for most geolocation services?
1.3. Telelocation and local localisation It is worthy to note that the problems of localizing an object from nearby and localizing it from afar are not the same. For example: A physical communicating object having at its disposal a GPS, or even better, a motionless physical communicating object, is able to know precisely where it is located. Nevertheless, it is not sure that this object could be remotely located. Indeed, localisation and reverse localisation cover four different functions:
From far Local
Localisation Reverse Localisation C) Telelocalisation A) Telelocalisation Find out the position of a remote Find out the objects that are located in a given place object D) Local reverse localisation B) Local localisation An object knows its own position An object knows the objects in its own surroundings
Some comments regarding these "localisation" functions: These functions are related one to each other and they partly overlap. Particularly and trivially, if function (A) is available, a fortiori, function (B) will be so. And similarly, if functions (C) and (B) are available, (D) will be also available. Provided that some conditions are fulfilled (see further on), the availability of local reverse localisation (D) is enough to compute functions (B) and (Q. So, depending on the kind of geolocation service, some functions may not be needed. Even more, for privacy reasons, some functions, like telelocalisation, might be unwanted or forbidden. Also, reverse telelocalisation could be also restricted to public spaces.
Reverse Localisation
113
2. Implementing reverse localisation 2.1. Cellular reverse localisation The naive approach to the implementation of a reverse localisation function is to cover the space with cells2. For each cell there is a server which is kept informed of the communicating objects located within (or associated with) the cell area. To know the objects situated in a given location, one just needs to request them from the server of the corresponding cell (for local reverse localisation one's own cell server). This approach has many problems.
2.1.1. Cell operating cycle When an communicating object enter a cell, it must inform the corresponding server: How does the object know in which cell it is located? Does it know its position (then it need local localisation) and the geometry of all the cells? Moreover, a network allowing communication with one's cell server is needed, and only the communicating objects that are enabled for this telecommunication network will be taken in account. A solution could be that given as a fact that an object is probably in communication with some of its neighbors (that means that local reverse localisation is partially available) will be able, by this means, to know in which cell it is located and tell the corresponding neighbour about its presence.
2.1.2. Deployment and coverage Prior to the adoption by most of the communicating objects, the coverage (or the promise of coverage) should have an extension wide enough, and that without knowing exactly how to dimension the system: • •
The coverage level depends on the system's success: - Cells must be smaller if the number of communicating objects is large And the system success depends on the coverage level The system will be adopted by a high number of communicating objects only if the coverage is wide enough and the cells small enough to handle the objects' density.
Cell like in cell phone, eg CDMA or GSM.
114
Communicating with Smart Objects
In addition, complex financial mechanisms (billing, peering agreements, who pays what etc) should be implemented to put up the money for the cell servers and all that without knowing how much income the reverse localisation will generate. 2.1.3. Scalability Since cells can only handle a limited number of communicating objects, a cell can become overloaded at any moment (meanwhile other cells might be nearly empty). Therefore, the system that determines the cell areas and boundaries should be highly dynamic (particularly in respect of adding new cell servers) and oversized. 2.1.4. Conclusion Although it is technically feasible, the cell-based approach is probably unrealistic unless the cells have other applications. For example, it might be interesting to add reverse localisation functions to the GSM and/or the UMTS equipments and networks. In this manner it will be possible to gather the GSM/UMTS devices that are present in a given cell.
2.2. Reverse localisation using a network of peers Another approach, inspired by peer-to-peer, gnutella-like systems or ad hoc networks, is conceivable. In this approach, the communicating objects participate and collaborate to provide the reverse localisation function. The communicating objects organise themselves into a peer network in which each participates depending on their capabilities (energy, bandwidth, CPU etc). In this system there is no need for dedicated resources and the resources of the system are provided by the end users. 2.2.1. How it works Let us start with the hypothesis that each communicating object is aware of and able to establish a communication with its nearby neighbors and hence that it implements local reverse localisation. At first glance, this hypothesis may seem paradoxical since it states that prior to implementing local reverse localisation we need local reverse localisation. But the paradox is only apparent and means indeed that the property (knowledge of and ability to communicate with surroundings neighbors) is maintained (at least partially) most of the time, by most of the communicating objects and that the communicating objects should work constantly to maintain the property.
Reverse Localisation
115
This property (and also local reverse localisation) can be extended to software-only communicating objects related to a location (eg a database or any software object related to a given location) that can be considered in this case to be virtually present at the location and participating in the peer network like any object physically present at the location. 2.2.2. Maintaining the network of peers Keeping knowledge of the vicinity is performed by neighborhood collaboration: communicating objects at an given place inform one another of the approach and moving away (the appearance and removal) of the communicating objects they know. Knowledge and communication with the neighbors can be achieve using either a short-medium range network (eg bluetooth or and ad hoc network), or a global network (eg UMTS), or both. It is worth noting that the more dynamic the network (mobility communicating objects), the greater the number of exchanged messages (up to the extreme limit of a system changing too fast to function). If some peers are permanently associated with a location (kind of "servers") they help stabilizing the peer network around this location. Any non moving physical object or any logical object, permanently associated to a location, may play this "server" stabilizing role. 2.2.3. Reverse telelocalisation The problem is the following: How to "interrogate" from far the local reverse localisation "service"? How to know which communicating objects are located in a given place? Since every communicating object (element of the peer network) knows its neighboring objects, to know the whole communicating objects located at a place it is enough to interrogate one of these objects. Also, as soon as a given object knows the communicating objects in a particular area (and if it is able to communication with them), it is possible to assume that the object is virtually located in that area and integrate it into the peer network.In this way it will participate in maintaining the peer network and help in providing the reverse telelocalisation function. The association with this area or place is not arbitrary, since the object not only knows the (objects located in the) place, but by calling the reverse telelocalisation function it has shown a clear interest in the place. The reverse telelocalisation system is a relational network (a graph) of either logical or physical communicating objects. Each node or object knows its own, either
116
Communicating with Smart Objects
virtual or physical, position (a least approximately) and it is able to communicate with its neighbors (it may be able to communicate with further objects). To identify the communicating objects that are located in an given place (reverse localisation), one just finds, to begin with, one communicating object (physical or logical, mobile or motionless) that is located in that place and then enquires of this object about its neighbors (these objects may be also requested if necessary); so step by step, all the objects in that area will be eventually known. The first object or starting point is found using distributed search techniques (a la gnutella) within the peer network. 2.2.4. Conclusion Reverse localisation of communicating objects is not provided by the deployment of servers or equipment but by the organisation of all communicating objects concerned in a peer network (each object has, a priori, the same role) in which all the participants collaborate in providing, one to another, the reverse localisation service. However, this approach is not incompatible with the notion of a cell and it is possible, if there a background business logic, to deploy reverse localisation "servers", i.e. motionless communicating objects that are (totally or partly) dedicated to the reverse localisation of a area (around their position). The advantage of using a peer network is that the deployment costs are minimal and the available resources are utilised to their maximum, all with very low cost of administration and maintenance. The main disadvantage is that no quality of service (in terms of response delay, spacio-temporal availability, accuracy, exhaustiveness etc) can be completely guaranteed. The user have to conform to the available service (more than often nothing but sometimes satisfactory).
3. Perspectives Either direct or reverse, local or distant, localisation functions are rich in potential applications. Geolocation services to be implemented in the near future will rely on localisation functions. But today, except for outdoor local localisation (GPS and in the near future Galileo), few technical solutions (maybe none) could implement localisation functions on great portions of the globe surface and for a great number of objects. And in the times of nanotechnologies and pervasive computing, most manufactured objects will probably be communicating objects, that may be subject to reverse or local localisation.
Parts Networking Technologies for Smart Objects Pierre-Noel Favennec France Telecom R&D, France
The concept of smart objects rests on the connection which exists between two objects thus enabling them to communicate. This connection can be associated with a real physical link between the objects or with a virtual link created during the transfer of information, and only during this exchange, of a transmitting object towards one (or from) receiving objects and reciprocally. Concerning the connections with real link, the connection can be wired. One can have an electric connection: copper wire allows indeed the transport of information by the displacement of electric charges going from an object towards one (or from) another object; the intensity and the format of modulation then defines the order between the objects. One can also have a guided optical link. The link is made by optical fibres or more generally by optical guides, information being then transported by guided photons characterised by their energy (their wavelength), their intensity and their format of modulation. Modern smart objects do not have to have wired links between them, the only physical link can be electromagnetic waves in open space connecting the objects. In this book, we present the technology of connection establishing communication between objects with the only reservation that it is unwired. We have not considered the transmitters and receivers which them form an integral part of the object itself. The use of radio as a physical link and to control smart objects is most common. But this radio operator link, invisible and diffusing, can have, according to its characteristics, completely different properties with respect to its use - desired mode communication: outdistance, flow rates, quality of service etc. The frequency of the radio wave is one of the first characteristics defining it. Broadly, this frequency can range from a few hundred kilohertz up to frequencies of a hundred gigahertz. One goes from the normally used frequencies to
118
Communicating with Smart Objects
telecommunications between mobiles (standard GSM, GPRS, UMTS) ...to other higher frequencies specific to smart objects (Bluetooth) or worms of higher flow rates towards 5GHz (Hiperlan, IEEE 802.11). Then, at even higher frequencies, not yet used for smart objects, applications are studied, in particular around the frequency of 60 GHz (known as millimetre-length wave for its wavelength is about the millimetre). Generally, the frequencies of communication are strongly related to the potential flow rates. The higher the frequencies, the greater the flow rates in the communications can be. Working Bluetooth with 2.4 GHz allows exchange flow rates of 700 Kbits/sec. 60 GHz allows flows of more than 100 Mbits/sec. Another useful parameter for these radio waves is the emission power. Indeed, the greater the power of emission, the better the wave will carry far information, or also the power can be adapted to the distances desired for the communications. But the emission powers used must be reasonable if one wants to minimise other disturbing effects (electromagnetic compatibility, biochemical effects). The transport of the information bits between the objects can also be done at frequencies of the much higher then placing us in the field of optics. Classically, one will work then with wavelengths for which the technologies of the transmitters and receivers are mature, i.e. in the optical ranges corresponding to visible and the infra red. These optical beams, non-guided, being propagated in open space, allow very high flow rates. The objects could communicate between them with flow rates of the order of Gbits/sec and even more ... The optical beams are propagated in a straight line in the atmosphere and are weakened throughout their course. They do not allow propagation at very long distances. One hopes however to be able to communicate outside for several kilometres and with flows of some gigabits/sec. In a confined habitat, i.e. in a part of a few tens of m2, the communications between objects if they are in direct sight will allow probably connections with very high flows that are so necessary; if it is in nondirect sight, the photons are reflected on any obstacles which there will appear on its course (walls, ceilings, floors, windows and pieces of furniture) making the flows less usable lower but remaining high compared with those that one would obtain by traditional radio technologies. But also, technologies of optical sources and transmitters enable one to foresee the possibility of beams of a few photons and in particular one photon. The singular character of the photon, which is not any more the same one if it were detected, allows prediction of communications with 1 photon of which it is absolutely impossible to copy and to know the contents. There is a technology for smart objects here absolutely singular and expensive, but can require certain quite specific situations. Both technologies, radio and optical, each have advantages and disadvantages. Apart from the performances themselves (flow rates, distance, treatment of the signal), an aspect important to consider is the relation wave-user, or wave-man in an
Networking Technologies for Smart Objects
119
electromagnetic sea. Even if it is not shown scientifically that, engulfed in an atmosphere in which multiple radio waves are propagated, it is suffers physical deterioration, one can provide that an environmental requirement of the whole (or a part) of the population will be to reduce these radio waves in rooms, places of work and the atmosphere. The communication between objects by optical beams fills this requirement; moreover, associated with strong potentialities in flow rates one can predict an attractive future for this technology. The future of communication by optical beams will have to also take into account the public health aspect. It will be essential to quantify with the assistance of the ophthalmologists what are the limits (wavelength, outdistances and power) for which the optical beams do not present any danger to the eyes. For the security of the contents of information between the smart objects,, solid state physics proposes original solutions which will make it possible to ensure communication between objects in a confined space). First attempts to bar, in optical flows, only one wavelength while leaving total transparency for the others are in hand. They could be generalised for all frequencies and wavelengths and in particular for smart objects communicating by radio waves. Except in very particular cases, the communicating objects are not directly visible. The quality of the communications is strongly dependent on the properties of the environment of the electromagnetic wave propagation. Indeed, of pieces of furniture, people, buildings, trees or any other obstacle can attenuate, reflect, diffract or diffuse the wave being propagated in open space. Thus, the wave received by a receiving object is not only the wave emitted by the transmitting attenuated and delayed object but the sum of a multitude of counterparts of the emitted wave. These counterparts follow ways different according to topography from the places at the moment of the exchange of information, and they are characterised by a delay and an attenuation which are clean for them. This phenomenon of multitrajets can cause interference and degrade the quality of the communications. For more certain and the most faithful possible exchanges, and whatever the desired situation of the communications between objects (in a crowd, at the seaside, in a room, a workshop), many of the studies of propagation, signal processing, physics are still necessary. Lastly, before finishing this brief presentation of radio and optical technologies for unwired smart objects, one can dream of objects which communicate at long distances by using technologies of the radio and/or optical ad hoc networks.
120
Communicating with Smart Objects
Frequency
IHz
1KHz
1MHz
1 GHz
ITHz
1 PHz
Wavelength
300000 km
300km
300m
0,3m
300 \im
0,3 (im
Field of the millimetre-length and centimetre waves
These diagrams illustrate frequencies \) and the corresponding wavelengths L (1 = C/D, C being the speed of the light in the air) of the electromagnetic waves usable or foreseeable uses in 2002 for the communications between objects. Some frequencies (wavelengths) are more specifically used and of the more precise details are indicated in the following chapters: * chapter 11 ** chapter 12 *** chapter 13 **** chapter 16 ***** chapter 18
Chapter 11
Wireless Techniques and Smart Devices Jean-Claude Bic Department of Communications and Electronics, GET/ENST, France
1. Introduction Second generation mobile systems, GSM and DECT in Europe, IS-95 in the US, were originally designed essentially for voice and short messages services dedicated to person to person communications. 2.5 G evolutions, without modification of the air interface, have introduced data services with bit rates of tens of kbit/s paving the way to communications more appropriate to smart devices. Simultaneously, the first wireless LANs have been devoted to connections between machines, either wireless Ethernet as Hiperlan or IEEE 802.11, or wired substitution as Bluetooth. The number of foreseeable smart devices have been at least an order of magnitude greater than the number of human beings; the necessary capacity in terms of information rates for the links between Smart devices is a major challenge for the near future. The 3G UMTS, based on a new radio interface, will provide a bit rate of hundreds of kbit/s enabling new multimedia services connecting more sophisticated smart devices. The future WLAN will also offer higher bit rates. The 4G, which generates now a lot of prospective works, can be seen as the convergence of these two kinds of systems toward a global system optimising the radio resources depending on the environment. The air interface will play a major role to take into account an important growth of traffic as the radio spectrum is a scarce natural resource which unlike wired systems is not extensible. As in any radio problem, an air interface has to be designed according to the radio channel characteristics spectrum efficiency, capacity, available frequency bands and quality of service will be at the core of the considerations.
2. Air interface The purpose of the air interface is to adapt the signal to the transmission channel and to optimise the quantity of information transmitted from the source to the receiver.
122
Communicating with Smart Objects
At the transmit side it consists of wave shaping functions with forward error correction often with interleaving, modulation, radiofrequency devices (filter, amplifier, antenna, frequency converter etc) and multiple access techniques for the simultaneous transmission of several signals in the same transmission channel. (Figure 11.1). At the receive side after antenna and amplification and frequency conversion, are implemented demodulation, and specific techniques, e.g. diversity, for correction of the channel distortions (Figure 11.2).
Figure 11.1 Air interface, transmission
Figure 11.2 Air interface, reception
Wireless Techniques and Smart Devices
125
2.1. Transmission channel Adapting the signal to the channel assumes a good knowledge of the propagation mechanisms. They are defined by the laws of electromagnetism and physical characteristics of the propagation media.
2.1.1. Characteristics of smart device links i) Distances and environment Links between smart devices may exist in different environments: • • •
cellular networks, with distances ranging from several hectometres to several kilometres in urban and suburban environments with many obstacles, Radio LAN for indoor coverage and also outdoor picocells (campus) with distances of a few tens of meters, domestic or personal networks with distance of a few meters to connect terminal often but not always in line-of-sight conditions.
ii) Frequency bands A very wide spectrum, depending on radio regulations, is possible for smart device links from some hundreds of MHz to a few tens of GHz and beyond in the infrared and optical bands. The low limit is due to the size of antennas and the high limit to coverage problem considering the possible transmitted power. At present, the frequency bands most used are 806-960 MHz (GSM, AMPS), 1700-1900 MHz (GSM, DECT, PCS), 1920-1980/2110-2170 and 1900-1920/2010-2025 MHz (UMTS FDD and TDD), 2,4-2,5 GHz and 5 GHz for RLAN, 3,5 and 26 and 40 GHz (40.5-43.5 GHz) for wideband fixed access. The 54.2-66 GHz band is considered for mobile broadband indoor access (MBS). Infrared networks also exist. Hi) Bandwidth occupancy The trend is an increase of bit rates for most of the multimedia services. Bit rates of tens of Mbit/s are envisaged for video access. Generally, the larger the mobility is, the lower are the bit rates (see Figure 11.3). A macrocellular network will not provide Mbit/s to high velocity mobiles. The bandwidth occupancy is directly related to the bit rate and coding and modulation parameters.
126
Communicating with Smart Objects
Figure 113 Mobility and bit rate
2.1.2. Propagation effects Path loss is the first effect of propagation. Free space loss expressed in decibel increases as 20 log d where d is the distance and as 20 log f where f is the frequency. Using higher frequency bands is therefore costly because the possibility to trade off the propagation loss by directive antennas, easier to implement at higher frequency, is not really possible in a mobile context. In a mobile cellular environment, propagation is rarely in a line-of-sight (LOS) situation but more often in non-line-of-sight (NLOS) depending on reflection and diffraction. Roughly, the loss is 35 log d. The NLOS propagation is limited to lower frequencies up to a few GHz because the diffraction losses, increasing as 10 log F, become too large. So it is sensible to use the lower frequencies for cellular networks and higher frequencies for short range communications. On the contrary, reflections still exist at high frequencies. Globally, in most of the cases multipath propagation occurs, created by the addition of several replicas of the transmitted signal with different amplitudes and delays. The multipaths can be classified into two categories; small scale multipaths or rapid fadings due to obstacles very close to the receiver (somme wavelengths) which are modelled by Rayleigh (NLOS) and Rice (LOS) processes; and large scale multipaths corresponding to reflection and diffraction on distant obstacles, expressed by the impulse response or the power delay profile of the channel. The delay spread Dr second order moment of the power delay profile or coherence bandwidth Bc (satisfying DrBc > constant) are frequently used to characterise these selective fadings. The range of Dr is large depending on the frequency band and the environment, from a few ns in a room at 60 GHz to several tens of us in mountainous terrain. Channel impulse response models are
Wireless Techniques and Smart Devices
127
recommended especially for the comparison of 3G air interfaces and it is important that realistic models are available to estimate the performances. A rule of thumb is to compare the symbol rate Ds to Dr (or the signal bandwidth to the coherence bandwidth). If Dr > Ds (or Bs > Be) then the channel will distort the signal. 2.1.3. Other channel distortions The transmission channel in a wide acceptance including transmit and receive radio frequency equipment (antennas, amplifiers, frequency converters) is characterised by different kinds of noise, sky noise, thermal noise or low noise amplifier, interferences caused by other signals (co-channel interference in the same frequency band from neighbouring cells, adjacent channel interferences from adjacent frequency bands). In the simplest approach, the noise is modelled as a white additive Gaussian of constant density No/2. 2.2. Performance of radio interface The main criteria to evaluate the performances of the air interface are: i) spectrum efficiency
or more generally for
cellular systems with frequency reuse in various cells the cellular spectral efficiency
ii) link quality: essentially error probability (bit error rate or frame error rate), and others variables like delay conditions for decoding: some services such as voice services being strongly delay-constrained contrary to data services which accommodate several hundreds of ms. iii) signal to noise ratio of the signal power S divided by the noise power N (usually in digital communications a ratio Es/No or Eb/No symbol energy or bit energy to noise density). Note that these criteria could be inconsistent, i.e. a better spectral efficiency could be obtained with a higher interference level at the expense of the BER. Optimisation choices will depend on the system itself.
128
Communicating with Smart Objects
2.2.1. Modulation and coding The maximum bit rate (or capacity per time unit) for which the error probability can be reduced arbitrarily small in a white additive Gaussian noise channel of bandwidth B is given by the famous Shannon formula: DC(bit/s) = B log2 (1 + S/N) . It imposes a theoretical limit and recent results in coding and modulation have shown that it could be closely approached. Forward error correction is obtained by introducing N-K redundancy bits computed from K information bits in order to make the codewords 'more distant' in terms of a given distance (often the Hamming distance, number of positions where two codewords differed). The coding rate R is the ratio K/N (R < 1). The code performances depend on the minimum distance and roughly they are improved when for given N and R, the distance increases.
Figure 11.5 BER vs. Eb/No
Modulation is defined by the number M and the position of states or points in a constellation, the waveform which determines the power spectral density (important for interferences considerations) of the transmitted signal. For M < 8, the classical modulations are PSK (phase-shift-keying) modulations BPSK, QPSK, 8-PSK and for M> 8 QAM (quadrature amplitude modulation). 16-QAM constellation is given in Figure 11.4. In-phase (cos (2rcfoT+(t>o)) and in-quadrature carrier (sin (2rcfoT+4>o)) are the two axes.
Wireless Techniques and Smart Devices
129
Like FEC, performances are related to a distance (often the Euclidean distance). For a given average power, the further the points in the constellation, the lower is the error probability. This implies that the error probability increases as the number of points, i.e. for M-PSK modulation where all the points are equally distributed on a circle with the same radius, the distance between two adjacent points is smaller when the number of points is larger. BER is generally plotted versus the Eb/No ratio. The shape of these plots is given in Figure 11.5. Practically, BER objectives after FEC decoding on a radio link depend on the type of services in the range of 10-3 to 10-10. Revisiting the formula giving the maximum bit rate R < C to obtain an arbitrarily small error probability, it can be derived that the theoretical limit for Eb/No is -1.9 dB. At present, the best coding schemes provide BER of 10-5 for about 1 dB. For a given bit rate Db, the symbol rate Ds is expressed by Ds = Db / (R Iog2 M). The signal bandwidth B varies as Ds. The spectrum efficiency nE is therefore improved with modulations having a large number of points M and FEC of high coding rate R. For instance, EDGE owing to 8-PSK has a spectrum efficiency roughly 3 times better than the GMSK modulation of GSM, which behaves like a binary modulation. Combination of modulations with an adjustable number of states and variable rate FEC is one of the key issues of link adaptation which provides the best compromise between link quality and spectrum efficiency in given propagation conditions. One of the most significant example is HIPERLAN2 air interface which combines different codes and modulations enabling, in the same 20 MHz bandwidth, a low bit rate of 6 Mbit/s with R = 1/2 and M = 2 and a high bit rate of 54 Mbit/s with R = and M = 64 according to the propagation conditions. (Bertin paper).
2.2.2. Multiple access Multiple Access Techniques are designed to transmit simultaneously several signals in the same transmission channel. The basic principle is signal orthogonality. Frequency Division Multiple Access (FDMA) has been used in the first generation of mobile systems. It has been drastically improved with OFDM (orthogonal frequency division multiplexing) which gives a better spectrum efficiency by sparing guard bandwidths between carriers. Time Division Multiple Access (TDMA) is implemented in GSM on a single carrier. Several carriers are frequency multiplexed in the allocated bandwidth. Frequency reuse in a cellular scheme is the key factor for cellular spectrum efficiency.
3/4
130
Communicating with Smart Objects
Code Division Multiple Access (CDMA) is used for IS-95. Every information bit of a signal for a given user is 'multiplied' by a digital sequence (the spreading code) made of 'chips'. This technique, also known as direct sequence spread spectrum, is a spin-off of Shannon's results on the channel capacity. The UMTS air interface is based on an improved version, wideband CDMA (WCDMA). Another spread spectrum technique is frequency hopping. It is specified in the Bluetooth standard. For RLAN, the 802.11b standards are based on CDMA, while 802.11a is an OFDM access. Spectrum efficiency comparisons between all these techniques are usually tricky, depending on a lot of parameters. Multiple access techniques have also specific properties relevant to the channel distortions. For example, OFDM by splitting the signal on several carriers, is robust to selective fadings but sensitive to non linearities. Rake receivers exploit the CDMA spreading properties to combine multipaths and improve BER. Spatial Division Multiple Access (SDMA) is another type of multiple access based on antenna arrays. Beamforming allows the array to focus the antenna pattern simultaneously towards different mobiles located in distinct spatial directions. 2.2.3. Diversity techniques Fadings are predominant features of radio channels. Diversity techniques are designed to correct the fadings effects. i) Time diversity: FEC with interleaving spread the information on several bits in different time slots in order to correct the error bursts due to rapid fadings. Bit interleaved coded modulations (BICM) provide a robust solution with a good spectrum efficiency for channels with and without fading. ii) Frequency diversity: OFDM and CDMA spread the information on several frequencies or frequency bands and offer a better resistance to selective fadings than a single carrier access. iii) Spatial diversity: Using several receive antennas to decorrelate rapid fadings is a well known technique. Antennas arrays could be considered as reconfigurable directive antennas which minimised the effects of multipaths and interference from other users. 3. Perspectives Enhancing air interface characteristics is a prerequisite to offering higher capacities and allow the growth of smart devices links. In this respect, important evolutions are presently investigated.
Wireless Techniques and Smart Devices
131
i) Multiple access combinations of CDMA and OFDM in order to increase the spectrum efficiency: multi-carrier-direct-sequence CDMA (MC-DS-CDMA) where data are spread with the same code on several sub-carriers, multi-carrier CDMA (MC-CDMA ) where each sub-carrier is associated to a 'chip' of the spreading code, Multi-Tone CDMA (MT-CDMA ) where the spreading code is the same for each sub-carrier. ii) Ultra Wide Band Transmission (UWB) using very short pulses positionmodulated, multiple access being is obtained by time-hopping in different time slots (converse of frequency-hopping) not to improve the spectrum efficiency itself but to simplify the access to the channel and the deployment (no licenses?) iii) MEMO diversity (multiple input, multiple output) or spatio-temporal diversity: it is based on multiple antennas at the transmit and receive sides in a propagation medium favourable to multipath, creating several distinct propagation channels and giving a new degree of freedom. If the signals transmitted on each transmit antenna are independent (BLAST) they can be recovered at the receive side by appropriate channel estimation and digital processing. Another solution introduces a time dimension by using FEC codes before spreading the signal on several antennas. Information theory has shown that the Shannon capacity could grow linearly with the number of antennas. iv) Multi-users detection: Multiple-access signal are never fully orthogonal because of distortions introduced by the channel. Resulting interferences added to noise will reduce the performance and the spectrum efficiency. The principle is to consider these interferences not as random noise as in an usual receiver but as other signals to be detected and subtracted from the useful signal. Processing is rather complicate but affordable in a base station. v) Software Defined Radio: its rationale is found in the context of radio networks interoperability and air interfaces for seamless links. Experience has shown that it was not sensible to expect a universal standard which would be technically intractable due to the variety of situations. From a radio point of view, the terminal will be adaptatively reconfigured in different frequency bands and for different combinations of access/modulation/coding, an improvement of multi-modes terminal with pre-programmed processing. One of the key technical issues is the analog to digital converter which must be as close as possible of the receive antenna. High resolution 16-bi,t 500 Msample/s, good dynamic range and sufficient bandwidth, 100 MHz, are the important characteristics to be satisfied. 4. Bibliography Correia L.M. "Wireless Flexible Personalised Communications", John Wiley & Sons, 2001.
132
Communicating with Smart Objects
Frodigh M. et al. "Future-Generation Wireless Networks", IEEE Personal Communications, October 2001, pp. 10—17. Lehne P.H. "Wireless Future", Numero special Telektronikk 1, 2001. Morinaga N. "Wireless Communications Technologies, New Multimedia Systems", Kluwer, 2000. Rappaport T.S. "Wireless Communications", Prentice-Hall, 1996. Verdu S. "Wireless Bandwidth in the Making", IEEE Com Magazine, July 2001, pp. 53-58.
5. Glossary BER: Bit Error Rate Bluetooth: Standard for Wireless around 2,45 GHz CDMA: Code Division Multiple Access DECT: Digital European Cordless Telephony Eb/No: Energy per bit to Noise density ratio: used for evaluating link performances EDGE: Enhanced Data rates for the GSM Evolution FDD: Frequency Division Duplex FDMA: Frequency Division Multiple Access FEC: Forward Error Correction GMSK: Gaussian Filtered Minimum Shift Keying (GSM Modulation) GSM: Global System for Mobile IEEE 802.11: Standards for WLAN LMDS: Local Multipoint Distribution System MC-CDMA: Multicarrier CDMA MC-DS-CDMA: Multicarrier Direct Sequence CDMA
Wireless Techniques and Smart Devices
MIMO: Multiple Input Multiple Output MT-CDMA: Multitone CDMA OFDM: Orthogonal Frequency Division Multiplexing PCS: Personal Communication System PSK: Phase Shift Keying QAM: Quadrature Amplitude Modulation RLAN: Radio Local Area Network SDMA: Space Division Multiple Access TDD: Time Division Duplex TDMA: Time Division Multiple Access UMTS: Universal Mobile Telecommunication System: 3G standard UWB: Ultra Wide Band WCDMA: Wideband CDMA (UMTS Access technique) WLAN: Wireless Local Area Network
133
This page intentionally left blank
Chapter 12
Wireless Local Area Networks Philippe Bertin France Telecom R&D, France
1. Introduction Wireless Local Area Networks (WLAN) technologies have recently take off technologically and commercially. From the first corporate WLAN standards 802.11 to the personal area networks built with Bluetooth technology and using the very high bit rate HiperLAN, radio technologies are emerging in various user domains (commercial and public enterprise, personal networks etc). Initially designed for computer science and telecom applications, these technologies are being integrated into mass market terminals (mobile phones, audio-visual terminals) and make possible naturally communicating support adapted to the needs of communicating objects: high rates, multi-terminal connectivity, data and voice support, with range relatively restricted, manageable power consumption and easy integration. Use can then be developed in this kind of domain, probably in surprising and unexpected applications, and as such the potential in terms of communicating services is huge. In this contribution, we concentrate on the description of WLAN standards in order to position them with respect to others in terms of technology and maturity. Then, after a positioning, we introduce generalities to be considered for technology design and describe more precisely mechanisms of 802.11 and HiperLAN/2 standards. In order to complete this state of the art review, technological options of Bluetooth standard are finally considered in the last section.
2. Positioning the WLAN standards Recent and rapid grow of wireless technologies as well as standards and interest group multiplicity make it difficult to position the various solutions. Therefore in this section we are trying to position the main initiatives. 802.11 standards are defined in IEEE. They target the specification of WLAN standards. Their initial use domain is the enterprise; however they are able to easily being used or adapted in other contexts (domestic, campus, public "hot spot").
136
Communicating with Smart Objects
Targeted terminals are PC (most are laptops) and PDA based. 802.11 specifies a core standard which is complemented by a number of extensions, each one being identified by an extension letter (802.11, 802.11 a, 802.11b). Most current WLAN products conform to the 802.11b extension and support the "Wi-Fi" stamp, supposed to guarantee their interoperability. HomeRF is a "voice/data" standard dedicated to domestic applications. It is based on a scheme derived from 802.11 for data support and a scheme derived from DECT for voice support. First products, only data oriented, have appeared on the market. However, it appears that most HomeRP promoters are turning to other technologies. HiperLAN/2 is a European standard specified by ETSI in order to deploy very high bit rate WLANs. Initially, targeting the support of Wireless ATM networks, the standard has been opened to support other types of networks (Ethernet, IEEE 1394), which make it an adaptable standard for different environments eventually restrictive in terms of quality of service (enterprise, audio-visual etc}. Bluetooth had an initial scope of cabling replacement in the vicinity of mobile terminals, for example of the GSM type (PC connectivity, personal digital assistant, auricle, printer etc). This type of use should bring the technology to large developments, in particular for mobile Internet services support. It enables one to foresee the dynamic set-up of true "personal" networks connecting the various terminals of a given person. These initial aims made it short range (from one to several meters) and a low consumption technology. Since the first announcements, industrial developments have evolved the technology by using higher emitting power. Those evolutions target directly the introduction of Bluetooth in WLAN products, used at a housing or enterprise scale with bit rates largely lower than those supported by 802.11 and HiperLAN/2 products. 3. Wireless networks generalities Before describing existing WLAN standards, it is necessary to introduce some generalities applicable to the different systems.
3.1. Functions defined in WLAN standards As for Local Area Networks (e.g. Ethernet), Wireless Local Area Networks standards specify layers 1 and 2 of the OSI model: Layer 1, the physical layer, support the radio transmission service. It defines the transmitted signal (frequency band, channel bandwidth, modulation, filter, framing) as well as the necessary channel coding to ensure radio transmission robustness. Layer 2, the data link layer, is sub-divided into two sub-layers:
Wireless Local Area Networks
•
•
137
The MAC sub-layer support the media access service for the frame transmission. Depending on standards, this type of access can being supported with contention based or contention free schemes. The link control sub-layer is responsible for handling logical connections and interface with upper layers. Depending on the standards, the link control sublayer may support the error detection and retransmission scheme using ARQ (Automatic Repeat Request) algorithm; admission control functions; connections setup and handling functions; radio resources control functions etc.
Hence, layer 2 support a transport service for data units delivered by the higher layer, i.e. layer 3 ("network layer" of the ISO model). Then, WLAN technologies are commonly used to deliver IP datagrams over the radio link. However, in order to simplify implementation, current products offer the radio transport of Ethernet frames. This allows the delivery of a full service equivalent for higher layers, whether it is done over a classical wired LAN or a WLAN; the terminal protocol stack (e.g. the TCP/IP stack in a PC) will use the same internal interfaces (drivers) whatever the media.
3.2. WLAN architectures Two types of architecture are supported for WLAN (depicted in Figure 12.1): •
•
In "centralised" (or infrastructure) architectures, wireless access is provided through an access point which manages the radio resources in a given cell. It permits access to the rest of the local networks through a "bridge" function implemented between the wireless and the wired LAN. In "ad-hoc"architectures, the WLAN is built over a set of wireless terminals in radio visibility range with each other which form a completely distributed system. This type of architecture permits the setting up of a network in a dynamic way depending on terminals which are in the vicinity of each other. It does not preclude the connectivity to a wired network as this service can be provided by a terminal supporting the two types of interfaces coupled with a bridge or a router function between the two networks.
Generally, WLAN standards are designed to operate alternatively in the two types of architectures.
138
Communicating with Smart Objects
Figure 12.1 WLAN architectures
3.3. Wireless terminals It is foreseen a large number of wireless types of terminals: electronic pen, auricle, cellular phone, personal digital assistant, laptop, printer, web pad, digital camera and recorder etc. First are WLAN applications being developped in enterprises networks; WLAN products are currently oriented in this market segment and support PC interfaces essentially based on PCMCIA and PCI formats. With the coming of lower power consumption such as Bluetooth, some mass market product integration might appear.
3.4. Frequency bands Globally, two frequency bands are identified for WLAN use: the 2,45GHz and 5GHz bands which, depending on continent and country, have different regulatory constraints.
3.4.1. The 2,45GHz band It is the frequency band used for most current WLAN products. The total bandwidth is 80MHz (2400 to 2483,5 MHz). It is a "ISM" (Industrial, Scientific and Medical) band that can be used by any material conforming to electromagnetic compatibility standards. It is then not exclusively reserved for network operations, which implies that the system has to face important interference generated by objects of different types (such as microwave ovens for example). This band is available worldwide with some local restrictions in terms of emitted power or uses as summed up in the table below.
Wireless Local Area Networks
139
Table 12.1 The 2,45GHz band; power of emission and uses
Indoor EIRP lOOmW
Outdoor EIRP 500mW
Other restrictions —
100mW
100mW
France 2001
100mW (2446,52483,5 MHz only) l0mW (full band)
100mW (2446,52483,5 MHz only) 2,5mW (full band)
France 2004
100mW (full band)
100mW (2446,52483,5 MHz) 10mW (full band)
Limitations for public access in some countries Outdoor use at 100mWis authorised only in private areas with a preliminary authorisation from the Defence Ministry. Outdoor use restrictions at 100mWtobe clarified.
North America Europe
Concerning system channeling, 2 types of wireless techniques are foreseen: •
•
the DSSS (Direct Sequence Spread Spectrum) technology uses 14 channels of 22MHz with 5MHz spacing (ie there is some overlap between adjacent channels); the FHSS (Frequency Hopping Spread Spectrum) technology uses 79 channels of 1 MHz each.
Bluetooth technology makes use also of a frequency-hopping transmission technique.
3.4.2. The 5GHz band The following sub-bands are identified for being used by future WLAN systems operating at 5GHz: 5150-5350MHz (worldwide use), 5470-5725MHz (only open in Europe), 5.725-5.825 (only open in North America). Globally, this permit use of up to 455MHz in Europe. However, the regulation allows only a sub-part of the band to be open, which should be of at least 330 MHz. Several WLAN systems (Hiperlan/2, 802.11a) are targeting the use of this band. However, they are based on very similar physical layers in order to permit economy of scale in chipset production. The effective opening of these frequency bands is subject to local regulation in each country. At the European level, CEPT recommendations are foreseen:
140
• • •
Communicating with Smart Objects
Full band allocation to Hiperlan/2 systems operating in indoor (max EIRP of 200mW). The possible use in outdoor exclusively in the higher sub-band (max EIRP of 1W). Sharing with radar and satellite systems using the 5GHz band is supported with the implementation of DFS (Dynamic Frequency Selection) and TPC (Transmit Power Control) which guarantee that the Hiperlan system will generate a limited interference level for the other systems.
Currently, in France, only the lower sub-band is open which allows only indoor use. The following table sums up the worldwide situation: Table 12.2 5Ghz band: power and use
North America
Indoor EIRP 200mW (full band)
Europe
200mW (full band)
France
200mW(51505350MHz)
Outdoor EIRP 1W (5250-5350 MHz) 4W (57255825 MHz) 1W (5470-5725 MHz)
Others restrictions
Dynamic frequency selection and Transmit power control.
No outdoor use
Compared with the 2.45GHz band, the 5GHz band is providing the following advantages: • •
Higher bandwidth availability, permitting larger channelling (20MHz) and the coexistence of several networks with limited interference level. Spectrum sharing between a limited number of standardised systems and indoor use specifically dedicated to WLAN types of systems, which limit considerably inter-systems interferences.
This give the 5GHz band most attractive for applications needing high bit rates and guarantee Quality of Service. However, the current competition between the different standards as well as the European regulatory constraints may delay the worldwide market stabilisation for WLAN operating at 5GHz.
3.5. Range and capacity Typical range for WLAN systems are of about 20 to 40m in a typical office environment and of up to 100 or 200m in Line of Sight environment. They are then relatively short, which is due to two main reasons:
Wireless Local Area Networks
•
•
141
The emitted power is restricted both for practical reasons (battery consumption) and regulatory ones (power restrictions as seen in section 3.4); moreover, local area networks types of services are requiring high peak data rates, only possible with a good link budget, which limits the acceptable transmission attenuation and then the range. Those technologies were defined in priority for indoor private types of applications, ie not to cover outdoor extended areas.
In terms of capacity, current WLAN products support bit rates of 11 Mbit/s over the radio link, which permits really a useful bit rate of about 5Mbit/s at the IP layer. Emerging standards in the 5GHz band are targeting visent support of max bit rates of 54 Mbit/s at the physical layer. Lastly, WLAN systems are using time division schemes for sharing the radio resource, when a terminal is emitting it uses the complete channel bandwidth and then the associated peak rate.
3.6. Mobility Terminals mobility between WLAN Access Points (in a centralised architecture) is managed by the terminals themselves that depending transmission conditions select the Access Point on which to associate. The handoff from one Access Point to another one is much closer to a cell "re-selection" scheme than a cellular handover controlled by the network as it is done in cellular networks. During this handoff, layer 2 connectivity is re-established. Then, when both Access Points are connected to the same local infrastructure (typically the same IP sub-net in a TCP/IP network), the network layer connectivity is maintained. However, when Access Points are parts of different sub-networks, the network connectivity can't be maintained as the terminal need to change its IP address and then start again its ongoing applications. In this case, in order to support a mobility service, it is necessary to use specific networks schemes, e.g. such as the implementation of the Mobile IP protocol. This mobility problem is not critic in enterprise networks which generally are based on switched Ethernet architecture and use routers only in splitting with the external Internet network. However, the problem may become more crucial for a campus size deployment where we may face to routed network architectures".
3.7. Security Even if it is possible to implement security schemes in higher layers (e.g. by using the IPsec protocol at the network layer or end to end security at the application layer), the wireless link should not introduce security weaknesses in the communication system. Then, data prevention against eavesdropping as well as network protection against misuses access restraint the development of WLAN systems, particularly considering that radio propagation does not restrict waves to the user private place. Security functions defined for WLAN systems are:
142
• •
Communicating with Smart Objects
authentication preventing from network access of non authorised terminals; encryption preventing radio eavesdropping.
One of the main issue is to support a secured system for key generation and exchanges between terminals and the security manager (the Access Point or a centralised network server). Indeed, this is necessary as ciphering keys maybe broken when they are not revoked regularly. Two approaches are then possible: •
•
Secret key use which is based o the fact that each terminal owns a secret key, only known from him and the network and used as the basis for authentication and ciphering. The issue is then to provide a secure scheme to distribute secret keys in terminals and all the network elements implied in the authentication/ciphering mechanisms. Use of a combination of both public and secret keys: this type of approach permits a terminal and network elements to "publish" a key usable for another party to encrypt data that only the terminal having published the key can decrypt with an associated secret key. Considering that it requires more CPU, this type of mechanism is generally used only for authentication protocols and secret key exchanges to be used for data encryption. This type of scheme support regular revocation for temporary secret keys.
4. Standards 4.1. IEEE 802.11 standards As already mentioned, WLAN IEEE standards are specified by the 802.11 technical group. The initial 802.11 standard was published in 1997 [IEEE 802.11, 1997] to operate at 1 or 2 Mbit/s in the 2,4 GHz ISM band. Since 1997, some extensions were published: the 802.lib specifies higher bit rates to allow transmissions at up to 11 Mbit/s whereas the 802.1 la specifies a new physical layer able to operate in the 5GHz band. Other sub-groups are also targeting extensions on specific issues such as the introduction of Quality of Service policies. 4.1.1. Architecture Both types of architecture introduced in section 3 are supported by 802.11 standards: •
•
In the "Infrastructure" mode, each cell is managed by an Access Point which controls the association/disassociation of the terminals belonging to the cell. Access points are also performing filtering and bridging between the wireless and the wired domain. In the "ad-hoc" mode, the terminals in a given radio environment are sharing the physical medium.
Wireless Local Area Networks
143
The 802.11 standard defines subsystems (see Figure 12.2): •
•
The Basic Service Set (BSS) is the basic building block of a 802.11 network. It could be seen as a cell in a cellular network. In the ad-hoc mode, the wireless network is built as an "Independent BSS". In the infrastructure mode, each BSS is connected to the infrastructure through an Access Point. The Distribution System is seen as the wired infrastructure which interconnects Basic Service Sets. The Distribution System allows the mobility of stations between the BSS. The Distribution System can rely on any wired LAN technology. 802.11 standards does not describe the protocols to be used between Access Points through the Distribution System.
When a station enters a Distribution System, it has to associates itself with an Access Point before issuing any data transfer. It is also recommended to use an Authentication scheme to authenticate stations before any association. Once associated, the station can moves though the different BSS of the given Distribution System using a re-association scheme with the relevant Access Points ("roaming" function described in 802.11 standards).
Figure 12.2 Architecture of the Distribution System IEEE 802.11
4.1.2. The IEEE 802.11 MAC layer 802.11 MAC layer provides an asynchronous data service compatible with classical LLC entities as in wired LAN. Security services are also provided by the WEP (Wired Equivalent Privacy) mechanism. Whatever the underlying physical layer (802.11, 802.1 la or 802.11b), the MAC layer remains the same.
144
Communicating with Smart Objects
Three types of MAC frames are defined: • • •
Control frames are used to control the transmission (e.g. Acknowledgements, short signaling preceding a frame transmission Polling). Management frames are used for signalling purposes (e.g. Beacon, Authentication/Deauthentication, Association, Reassociation, Deassociation). Data frames are used to deliver data payloads.
The basic medium access scheme allows the stations to access the medium in a distributed way with contention resolution. It is standardised as the Distribution Coordination Function (DCF). Optionally, an Access Point can regularly use the Point Coordination Function (PCF) to temporally preempt the medium for provide access in a controlled and contention free manner. Medium access in the DCF relies on a specific scheme named CSMA/CA (Carrier Sense Multiple Access/Collision Avoidance). This scheme uses a "listen before talk" mechanism in order to wait for idle channel before any transmission start. It is derived from CSMA/CD used in Ethernet and 802.3. The main difference is due to the fact that unless wired Ethernet station, the wireless stations can't detect collisions in real time as they can only send or receive data at one time (half duplex scheme). Then, CSMA/CA has been designed to avoid as much as possible collisions to occur. The scheme can be divided in different phases (see Figure 12.3): •
•
The other stations which have pending frames to be transmitted defer access until the end of the current transmission and a given Inter-Frame time. Then, they can enter in the contention resolution mode in order to compete for accessing the physical medium. In the contention resolution phase, each station which have frames to be transmitted senses the medium for a given slotted Backoff Window. At the first time a station enters in the contention phase for a given MAC frame, the Backoff Window is calculated through a random function. Once it's Backoff Window becomes null, the station enters in the transmission phase. Hence, other stations for which the backoff window is not null notice that the medium is used and stop to decrement their Backoff to defer access until the next contention phase.
Data frames have to be acknowledged once they are correctly received. This is done through short ACK frames sent just after the reception of a valid frame. When a data frame is not acknowledged, the sender station considers that a collision occurred. In a system with hidden nodes (e.g. stations from the same BSS which are not able to listen to each others), it can appear that very frequent collisions occur. In order to reduce this problem, the standard supports optionally the exchange of short control frames before the transmission of a complete data frame. Hence, once the sender gets access to the medium, it first send a RTS (Ready To Send) frame and waits for receiving a CTS (Clear To Send) before starting the transmission of the data frame.
Wireless Local Area Networks
145
To enter in the centralised coordination PCF mode, the Access Point has first to preempt the medium, which is done by using a shortest waiting interval that the one used in DCF mode. Then, it polls the stations which are authorised to emit. Once the point coordination period ends, the systems comes back in the DCF mode. In order to support the above described mechanisms, three "InterFrame Spaces" are defined in the standard: • • •
The DIPS (Data InterFrame Space) is the interval to be waited at the end of a frame transmission before considering the medium "idle"; The PIFS (Polling InterFrame Space) is waited by the Access Point before preempting the media to go in PCF mode; The SIFS (Short InterFrame Space) is the interval waited between a data frame reception and the emission of the corresponding acknowledgement. SIFS is also waited between the reception of the RTS frame and the sending of the CTS answer.
It can be noted that SIFS < PIFS < DIPS.
Figure 12.3 CSMA/CA access scheme
The WEP (Wired Equivalent Privacy) algorithm is defined in the 802.11 standard to ensure data exchanges confidentiality. This algorithm can be complemented by a Shared Key Authentication scheme also standardised. The WEP algorithm is based on a secret key which, concatenated with an Initialisation Vector, gives a seed used as the input of a Pseudo Random Number Generator to generate a key sequence with the same length as the data to be transferred (use of the RC4 algorithm). This key is then used to generate the encrypted text to be transmitted concatenated with the initialisation vector. The way to distribute keys not standardised, most of existing products support manual configuration but enhanced scheme to generate and
146
Communicating with Smart Objects
distribute dynamically session keys have been proposed recently by different manufacturers. 4.1.3. IEEE 802.11 physical layers As already mentioned, IEEE 802.11 group specified several physical layers for 2,45 and 5 GHz frequency bands: • •
•
the FHSS (Frequency Hopping Spread Spectrum) physical layer is specified in the initial IEEE 802.11 standard to operate at 1 and 2 Mbit/s in the 2,45GHz band; the DSSS (Direct Sequence Spread Spectrum) physical layer is specified in the initial IEEE 802.11 standard to operate at 1 and 2 Mbit/s and completed in the IEEE 802.11b specification to operate also at higher bit rates of 5,5 and 11 Mbit/s. It uses also the 2,45GHz band; the multi-carrier COFDM (Coded Orthogonal Frequency Duplex Modulation) physical layer is defined in IEEE 802.1la standard in order to permit bit rates between 6 and 54 Mbit/s using the 5GHz frequency band.
The FHSS specification is based on frequency channels of 1 MHz. Up to 79 channels are available depending on the region regulation (e.g. only 35 channels are authorised to be used in France). Each wireless station and Access Point which belong to a given Basic Sub System uses a standardised frequency hopping sequence, the permutation between two frequency channels is done each 0,4 second. A 2GFSK and a 4GFSK modulations are used in the 1 Mbit/s and the 2 Mbit/s modes respectively. In the DSSS specification, each bit is encoded on a suite of 8 or 11 bits which is then spread over a frequency channel of 22 MHz. Up to 14 frequency channels are identified in the standard. The specified modulation schemes are: DBPSK for the 1 Mbit/s rate, DQPSK for the 2 Mbit/s rate, CCK for 5,5 and 11 Mbit/s. The COFDM specification introduced in 802.1la standard allows communications at rates of 6, 9, 12, 18, 24, 36, 48 and 54 Mbit/s (only support of 6, 12 and 24 Mbit/s rates is mandatory). An harmonised effort between IEEE 802.11 and ETSI BRAN led to the specification of very closed physical layers for both 802.1 la and HiperLAN/2 standards (see section 4.2.3). 4.1.4. 802.11 standards evolution The IEEE 802.11 working group carries on he standardisation effort through different sub-groups. The main issues concern: harmonisation effort permitting cooperation or even interworking between North-America and European standards; liaison with international spectrum regulation instances; MAC layer enhancements
Wireless Local Area Networks
147
permitting to introduce quality of service management schemes; specification of an inter-Access Point protocol; specification of a new high rate physical layer based on COFDM in the 2,45 GHz band (recently approved); the introduction of dynamic frequency selection and transmit power control algorithms required for being used in the 5GHz band in Europe; enhanced security schemes. 4.2. Standards ETSIHiperLAN The ETSI published the first WLAN standard operating in the 5GHz frequency bands in 1997. Named HiperLAN (High Performance Radio LAN) Type 1, it enables operation at 23,5 Mbit/s, which were largely higher than the max 2Mbit/s throughput supported by 802.11 standards at that time. In spite of technical assets, the standard did not receive the industrial support necessary for issuing commercial products and only Dassault Electronique did develop advanced prototypes which shown the its efficiency [BERTIN, 2001]. From 1997 to now, the ETSI BRAN (Broadband Radio Access Network) project focused on the standardisation of HiperLAN Type 2 systems [TR101031, 1999]. The first complete set of HiperLAN/2 standard was published in mid of 2000 [TS101475, 2000] [TS101761-1, 2000] [TS101761-2, 2000] [TS101493-1, 2000] [TS101493-2, 2000]. Further enhancements are still under discussion in ETSI BRAN, particularly through partnerships with IEEE to carry on standards harmonisation as well as with 3GPP targeting interworking between and UMTS 3rd generation cellular networks but also under European research programs [BRAIN, 2000].
4.2.1. HiperLAN/2 reference model and architecture The HiperLAN/2 reference model includes a PHY (Physical) layer, a DLC (Data Link Control) layer and a Convergence Layer (see Figure 12.4). The role of the Convergence Layer is to adapt the above protocol requirements to the DLC layer services. In this way, several Convergence Layer will permit to support several higher layer protocols whereas the DLC layer remains the same. In practice, three types of Convergence Layers are already specified: • • •
the Ethernet convergence layer which, through a packet based sub-layer, adapts Ethernet and 802.3 types of frames to the underlying DLC layer; the IEEE 1394 which uses the same packet based sub-layer to support IEEE 1394 types of frames; the ATM convergence layer which, through an internal Cell based sub-layer, adapts ATM cells.
Later, new Convergence Layers could be standardised in order to support other types of protocols over the radio.
148
Communicating with Smart Objects
Figure 12.4 HiperLAN/2 protocol stack
The DLC protocol provides a connection-oriented data transport service. It support the transmission of fixed size protocol units of 54 bytes (including DLC header) over the radio. A control plane is specified to manage connections, resource and association control. In the lower part of the DLC layer, control and data units are distributed among a MAC structure of fixed length of 2ms. MAC frames are transported by the PHY layer through a COFDM (Coded Orthogonal Frequency Division Multiplexing) scheme bit rate.me with link adaptation permitting the selection of the most adequate bit rate.
Figure 12.5 Comparison between centralised and direct modes transmission
Wireless Local Area Networks
149
The HiperLAN/2 standard supports centralised architectures where the Access Point controls the repartition of radio resource between active connections with Mobile Terminals operating in the cell. However, an option has been integrate to permit direct data transfer between terminals belonging to the same cell, which avoid data frames to be relayed by the Access Point when unnecessary. Even in the direct mode, Access Point keeps its role of central controller and signaling units remains exchanged in the centralised way. However, direct mode permits the support of adhoc types of networks.
4.2.2. HiperLAN/2 DLC layer As depicted in Figure 12.4, the DLC layer is sub-divided into: a control plane based on the RLC (Radio Link Control) Protocol; a user plane which support Error Control schemes, a MAC (Medium Access Control) lower part providing the transport of user and control planes information. The MAC sub-layer The MAC protocol is based on a TDD/TDMA centralised scheme. The Access Point controls Medium Access by allocating time frames to active stations in order to manage and serve connections. The MAC sub-layer relies on a fixed duration frame structure that permits to concatenate different types of information: •
• •
control information broadcast by the Access Point to the Mobile Terminals belonging to its cell (network and Access Point Identifiers, MAC frame structure description, radio resources allocation, etc); downlink, i.e. information sent by the Access Point to one or several stations (data and control); uplink, i.e. information sent by stations to the Access Point (resources requests, data, etc).
150
Communicating with Smart Objects
Figure 12.6 HiperLAN/2 MAC frame structure
The turnaround time between Downlink and Uplink phases is fixed dynamically by the Access Point which allows very flexible resource repartition depending on the needs for asymmetric traffics. The different types of information are grouped in "transport" channels which corresponds to a standardised physical structure. Above the MAC layer, Logical channels are introduced to differentiate information types coming from the different sublayers. Logical channels are mapped onto transport channels in a specified way. The RLC sub-layer The RLC (Radio Link Control) sub-layer provides 3 functions: Association Control Functions (ACF), Radio Resource Control (RRC) and DLC User Connection Control (DCC). The association control is supported up to an association procedure initiated by the mobile when it "enters" in the network. During this procedure, the Access Point allocates to the mobile a temporary MAC Identifier to be used only as long as the mobile remains under the control of the given Access Point. A mutual authentication scheme between the terminal and the Access Point is provided as well as the use of encryption functions. The mutual authentication permits to valid not only the mobile identity but also the Access Point one in order to prevent against traffic eavesdropping from false Access Points. The disassociation procedure could be initiated either by the terminal or the Access Point. Radio resource control function provides the specification for the following procedures:
Wireless Local Area Networks
•
•
• •
151
Quality measurement registered at the terminal side and eventually reported to the Access Point. Three types of handover controlled by the le terminal: the Sector Handover procedure is used when the terminal moves from one sector to another one in case of a multi-sector Access Point; the Radio Handover is specified in the case of the mobile terminal moves from one Access Point transmitter to another one belonging to the same Access Point; the Network Handover supports the movement between two Access Points. The Dynamic Frequency Selection is performed by the Access Point in order to select a free frequency channel to be used depending on radio quality measurements carried out by itself as well as those reported by the mobiles associated to it. The Power Control permits to adapt the transmit power control to the transmission situation between each terminal and Access Point (e.g. depending on the distance). The Power Saving is specified to save power consumption when the terminal has no data to send or receive.
The DLC user connection control supports setting up, maintaining, renegotiating and closing DLC connections between a wireless station and an Access Point. Those procedures may be initiated by the station as well as the Access Point. To each connection is given an identifier unique in the cell. Multicast and Broadcast types of connections can also be established. The error control protocol For error control, three modes of operation are provided: the acknowledged mode is based on a selective repeat scheme for un-acknowledged protocol units; the repetition mode allows repetition of protocol units (usable for multicast transmissions); the unacknowledged mode provides no repetition of loss or corrupted data. In the acknowledge mode, the error control protocol relies on a SR-ARQ (Selective Repeat Automatic Repeat Request) scheme: the sender buffers unacknowledged data packets; when it receives negative acknowledgement, it retransmits only the corrupted packets. The acknowledgment scheme, named "partial bitmap acknowledgement", is based on the sending of bitmaps from each bit corresponds to a given protocol unit and indicates whether it is positively (1) or negatively (0) acknowledged. 4.2.3. The HiperLAN/2 physical layer As introduced in section 4.1.3, during the standardisation process, IEEE 802.1 la and HiperLAN/2 physical layers have been harmonised and are then similar. Each layer specificity is relying of its adaptation to the different MAC approaches, in particular
152
Communicating with Smart Objects
the centralised channel access scheme for HiperLAN/2 and the distributed one for 802.11. Table 12.3 HiperLAN/2 PHY layer parameters Parameter Useful symbol part duration TU Cyclic prefix duration TCP Symbol interval Ts Number of data sub-carriers NSD Number of pilot sub-carriers NSP Total number of sub-carriers NST Sub-carrier spacing Af Spacing between the two outmost sub-carriers
Value 64*T 3.2 us 16*T 0.8 us (mandatory) 80*T 4.0 us (TU+TCP) 48 4 52(N SD +N SP ) 0.3125 MHz (1/TU) 16.25 MHz (NST*Af)
8*T 0.4 us (optional) 72 *T 3.6 us (TU+TCP)
The HiperLAN/2 physical layer provides a "protocol data units train" transport service. Those "data units trains" are formed through the concatenation of transport channels broadcast to the whole terminals (generic control data for the cell and the frame, broadcast user data) or personalised (data and signaling destined to each mobile). In the uplink, the protocol units trains are provided to transport data and signaling for each mobile as well as the random channel access requests. The PHY layer is based on a multi-carrier COFDM modulation. Each OFDM symbol contains data and pilot carriers. Then, 64 sub-carriers are used, from which 48 are reserved for data and 4 for pilots. The remaining 11 subcarriers are used as guards. Numerical values for OFDM parameters are given in Table 12.3. The emission of a complete MAC frame (2 ms) corresponds to 500 OFDM symbols, excluding guard intervals and turnaround time between uplink and downlink transmission. Table 12.4 PHY layer transmission modes Modulation
Coding rate R
Nominal bit rate [Mbit/s]
BPSK BPSK QPSK QPSK 16QAM 16QAM 64QAM
1/2 3/4 1/2 3/4 9/16 3/4 3/4
6 9 12 18 27 36 54
Coded bits per subcarrier NBPSC 1 1 2 2 4 4 6
Coded bits per OFDM symbol NCBPS 48 48 96 96 192 192 288
Data bits per OFDM symbol NDBPS 24 36 48 72 108 144 216
Wireless Local Area Networks
153
As already introduced, seven different modes are supported in order to dynamically adapt to the radio situation of each mobile terminal (interference level, distance to the Access Point...). Those modes and the correspondent parameters are given in Table 12.4.
4.3. Bluetooth The Bluetooth technology was developed in the Bluetooth SIG (Special Interest Group). As introduced in section 2, Bluetooth was initially specified for network applications limited in terms of capacity (1Mbit/s), range (10m) and power consumption. This result I a standard mostly destined to WPAN (Wireless Personal Area Network) types of applications than WLAN. However, in order to better position this technology in regards of Wireless Local Area Networks, we introduce hereafter the designed technologic options. Bluetooth uses the 2,45 GHz frequency band. In order to prevent from interferences from other systems operating in the same band, a "fast" frequency hopping physical layer technology is adopted (up to 1600 hops per second). The GFSK type of modulation supports a bit rate of 1 Mbit/s. The protocol uses a temporal duplex and a centralised architecture where a "master" station shares the radio ressource use in time between "slaves" stations. The, a slave station can't take the medium as long as it receives a "polling" packet from the master. The master station and associated slaves constitute a Bluetooth network, called "piconet". Several packets formats are defined: ACL (Asynchronous Connection Less) packets are adapted to data transfer whereas SCO (Synchronous Connection Oriented) packets support synchronous streams such as voice. Each packet can use 1, 3 or 5 time slots of 625 us each, the system changing of frequency between the emission of two packets. Figure 12.7 illustrates the resources sharing principle in "pointmultipoint" between SCO and ACL packets.
154
Communicating with Smart Objects
Figure 12.7 Resource sharing in a Bluetooth piconet
Lastly, two transmission power are supported: 100mW allowing a range of about 15 meters and 1mW limiting the range to a maximum of 5 meters. Within a piconet, up to 8 nodes can operate and maximum three voice and seven data communications can be supported. 5. Conclusion Appeared on the market last years, WLAN products have a technical and commercial increasing development. It is however needed to start to face emerging of several standards which could be positioned as concurrent or complementary. For personal networks types of applications built around a cellular terminal, Bluetooth technology is well positioned to be developed quickly and a large number of mass market terminals are already announced (phones, digital recorders, digital cameras, PCs, printers...). For local area networks types of applications, 802.11b products are already widespread but new generation such as 802.1la and HiperLAN/2 are being implemented by industrials and should allow the proposal of the first very high bit rate products early in 2002. Currently, those technologies are confined to the professional sphere but could be developed quickly in domestic networks and above all as Internet high rate wireless access points in public places (already widespread used in united states). They are also a natural support for communicating objects needs, either in personal ad-hoc networks, or in interface with telecommunication networks. Hence, with the
Wireless Local Area Networks
155
integration of new types of interfaces and software solutions for service discovery and selection; objects and users will be able to get access in total transparency to communication services dynamically adapted to the environment (connectivity, exchanges and automatic synchronisation with terminals present in the vicinity). We should then assist to the apparition in the next years of a large number of uses on those technologies, the most surprising being probably still to be invented. 6. References [BERTIN, 2001] "A trial of Home Applications over Hiperlan type 1"; Philippe Bertin et Regis Cady; PIMRC 2001, London. [BRAIN, 2000] IST project BRAIN Deliverable 3.1 - Technical requirements and identification of necessary enhancements for HIPERLAN Type 2. Sept. 2000. [IEEE 802.11, 1997] IEEE 802.11 "Standard for Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specification", 1997. [TR101031, 1999] ETSI TR 101 031 V2.2.1 (1999-01), "Broadband Radio Access Networks (BRAN), High PErformance Radio Local Area Network (HIPERLAN) Type 2, Requirements and architectures for wireless broadband access". [TS101475, 2000] ETSI TS 101 475 V1.1.1 (2000-04), "Broadband Radio Access Networks (BRAN); HIPERLAN Type 2; Physical (PHY) layer. [TS101761-1, 2000] ETSI TS 101 761-1 VI.1.1 (2000-04), "Broadband Radio Access Networks (BRAN); HIPERLAN Type 2; Data Link Control (DLC) Layer; Part 1: Basic Data Transport Functions. [TS101761-2, 2000] ETSI TS 101 761-2 V1.1.1 (2000-04), "Broadband Radio Access Networks (BRAN); HIPERLAN Type 2; Data Link Control (DLC) Layer; Part 2: Radio Link Control (RLC) sublayer. [TS101493-1, 2000] ETSI TS 101 493-1 V1.1.1 (2000-04), "Broadband Radio Access Networks (BRAN); HIPERLAN Type 2; Packet Based Convergence Layer; Part 1: Common Part. [TS101493-2, 2000] ETSI TS 101 493-2 V1.1.1 (2000-04), "Broadband Radio Access Networks (BRAN); HIPERLAN Type 2; Packet Based Convergence Layer; Part 2: Ethernet Service Specific Convergence Sublayer.
156
Communicating with Smart Objects
7. Glossary ACL: Asynchronous Connection Oriented. Asynchronous packets transmitted in non-connected mode in Bluetooth systems (support). AP: Access Point. Wireless access point, implements generally a bridge function between a WLAN radio cell and a classic wired local network. ARQ: Automatic Repeat Request. Retransmission schemes for loss protocol units. ATM: Asynchronous Transfer Mode. BRAN: Broadband Radio Access Networks. ETSI group specify broadband radio systems. BSS: Basic Service Set. Radio cell in the EEE802.11 standard. CSMA/CA: Carrier Sense Multiple Access/Collision Avoidance. Channel access method based on a listen before talk mechanism as well as a collision avoidance method. Used in IEEE 802.11 WLAN. CSMA/CD: Carrier Sense Multiple Access/Collision Detection. Channel access method based on a listen before talk mechanism with a collision resolution method. Used in Ethernet Local Area Networks. CTS: Clear To Send. Short signalling scheme sent in answer to the reception of a RTS frame in a IEEE 802.11 system. DCC: DLC Connexion Control. Connection control function in the DLC layer of an Hiperlan/2 system. DCF: Distribution Coordination Function. IEEE 802.11 transmission mode permitting that each station gets the same probability in medium access. DECT: Digital European Cordless Telecommunications. DIPS: Data Interframe Space. Minimum time interval between the emission of 2 data frames in a IEEE 802.11 system. DLC: Data Link Control. Couche de controle de lien. Domestic network: communication network used at the scale of an home. EIRP: Effective Isotropic Radiated Power. Ethernet: communication protocol used in Local Area Networks.
Wireless Local Area Networks
157
ETSI: European Telecommunication Standard Institute. Hiperlan: High PERformance Radio Local Area Network. European WLAN standards specified by ETSI. IEEE: Institute of Electrical and Electronics Engineers. IEEE 802.11: IEEE WLAN standardisation group. ISM: Industrial, Scientific and Medical. LAN: Local Area Network. MAC: Medium Access Control. OFDM: Orthogonal Frequency Division Multiplex. technique.
Multi-carrier modulation
OSI: Open Systems Interconnection. OSI layer: layer defined in the ISO model. PCF: Point Coordination Function. WLAN IEEE 802.11 transmission mode where an access point controls the communication media sharing. PIFS: Polling Interframe Space. Time interval specified in IEEE 802.11 system between the transmission of a frame and the radio media preemption by an access point working in PCF mode. RLAN: Radio Local Area Network. Other denomination for WLAN. RLC: Radio Link Control. Radio control function. RRC: Radio Ressource Control. Radio resources control functions in Hiperlan2 systems. RTS: Ready To Send. Short signaling frame that can be sent before the transmission of a dataframe in IEEE 802.11 systems in order to ensure from radio connectivity. SCO: Synchronous Connection Oriented. Synchronous packets transmission in Bluetooth systems (support of voice). SIFS: Short Interframe Space. Time interval specified in IEEE 802.11 system between the transmission of a data frame and the corresponding acknowledgement. TDD: Time Division Duplex.
158
Communicating with Smart Objects
TDMA: Time Division Multiple Access. UMTS: Universal Mobile Telecommunication System. 3rd generation cellular networks. WEP: Wired Equivalent Privacy. IEEE 802.11 security functions. WLAN: Wireless Local Area Network. WPAN: Wireless Personal Area Network.
Chapter 13
Radio Links in the Millimeter Wave Band Nadine Malhouroux-Gaffet, Olivier Veyrunes, Valery Guillet, Lionel Chaigneaud and Isabelle Siaud France Telecom R&D/DMR, Belfort, France and France Telecom R&D/DMR, Rennes, France
1. Introduction Communicating objects represent a subject of much interest in the domain of service creativity or in the field of new implemented techniques. The latter are based on wireless communication systems that combine simplicity set-up with a great use facility. Wireless local area networks (WLAN) are indispensable for communicating objects and their expansion induces an increasing need for research and development on these systems; in particular concerning the increase of data rate. This increasing need of data exchange associated with pressure on scarce spectrum resources led the manufacturers of WLAN systems to investigate the millimeterfrequency bands (30 to 300 GHz). With such frequencies, the available bandwidth high data rates of about 120 Mb/s to be reached. However, the "outdoor links" above 10 GHz are affected by atmospheric and weather conditions that are of importance to the link budget. In this context, FTR&D initiated a study to simulate the loss due to hydrometeors (rain, snow etc) above 30 GHz, which is the validity limit of the UIT-R models. In the field of the local area networks, the band around 60 GHz shows interesting characteristics to optimise the deployment of indoor systems. Indeed, strong attenuation in open space, thus reducing interference between cells, as well as a large available bandwidth, will be a major asset for the realisation of future high data rate indoor systems.
160
Communicating with Smart Objects
2. Outdoor radio links in the millimeter wave band At frequencies above 10 GHz, the transmitting rate is appreciably better but some natural factors, such as atmospheric gaseous and precipitation, strongly affect propagation. In particular, interactions between electromagnetic waves and hydrometeors, such as rain, hail and snow, produce attenuation by energy absorption in particles and energy scattering in all directions. Thus, the designers of high frequency communication systems require from the propagation specialists, the ability to be able to foresee these attenuation effects to determine the adequate fading margins and to ensure reliable predetermined signal levels for different weather conditions. An overestimation of the propagation effects can result, on one hand, in a very expensive conception of system because of the high prices of the implemented devices to limit the disturbing effects of propagation and, on the other hand, on interference with other services. Besides, an underestimation of these propagation effects can lead to unreliable systems. So, the studies of propagation should be as accurate as possible to satisfy in the best conditions, in a statistical way, the quality criteria of the operational radio links. For some years, important efforts have been initiated by the international community to update the radio climatic databases from which global predictions of propagation are established for terrestrial radio links in the millimeter wavelength band (30-300 GHz). These works were the subject, among others, of CLIMPARA conferences, organised by the committee F of the URSI and the European project (COST) and Canadian project. Many countries combined their efforts to establish recommendations about the propagation aspects of the radioelectric waves in nonionised environments, under the aegis of the International Telecommunications Union for Radio communications (ITU-R). However, these techniques of global predictions are known to suffer from an important lack of observations of propagation in various parts of the world. So, this explains why FTR&D decided to establish an experimental setup (Figure 13.1), near Belfort in France. This experiment should contribute to a detailed knowledge of the physical mechanisms of atmospheric and meteorological phenomena and also of their interactions with the electromagnetic waves. The purpose of the setup is to supply propagation data over a short range for which the weather conditions are almost constant.
Radio Links in the Millimeter Wave Band
161
KEY: Pluie = rain; neige = snow; grele = hail; spectropluviometre = spectra rain gauge; capteur ... = identification hydrometeor, identification and temperature sensors; station meteorologique — meteorological station; pluviometre a auget = bucket rain gauge Figure 13.1 Belfort experimental station set up to supply propagation data over short range
The experimental device consists of multifrequency transmission links over an 800 meters terrestrial path. The propagation path is horizontal, with direct visibility about 5 m above the ground. Four dynamic narrow band links operate at 30, 50, 60 and 94 GHz, with vertical polarisation. Meteorological equipment, set out all along the path, includes three tipping bucket rain gauges to measure rainfall rates and their equivalent for other hydrometeors, two disdrometers to measure raindrop size distributions, two identification sensors to characterise the kind of precipitation and a meteorological station to measure temperature, atmospheric pressure, hygrometry, wind speed and direction. Investigation of the important radioelectrical and meteorological database led to development of attenuation propagation models due to the hydrometeors and thus to satisfy the new needs of the systems designers.
162
Communicating with Smart Objects
3. Rain attenuation One of the most fundamental aspects of knowledge of the rain attenuation characteristics is the relationship between the linear attenuation A (dB/km) and the rainfall rate R (mm/hour). The ITU-R model for rain attenuation intended for the forecast methods "Rec. ITU-R p. 838, 1999", recommends the following relationship: where the coefficients k and a are functions of the frequency. Such a relationship is interesting for its simplicity and its accuracy for the most common attenuation ranges in practical applications. The experiment showed that the values of k and a, recommended by the ITU-R, are respectively underestimated and overestimated above 30 GHz for the climate in the north east region of France. Other authors have already observed these results in other locations of the world. The ITU-R model tends to overestimate the rain attenuation above 50 GHz for a rainfall rate exceeding approximately 20-25 mm/hour. Figure 13.2 shows a comparison of the linear attenuation (dB/km) at 30, 50, 60 and 94 GHz, modelled according to the measures, versus the rainfall rate (mm/hour). It clearly shows that the rain attenuation increases with the rainfall rate. Furthermore, for a given rainfall rate, rain attenuation will be more significant with the highest frequencies, at least in the 30-100 GHz band.
Figure 13.2 Comparison of modelled rain attenuation (dB/km) at 30, 50, 60 and 94 GHz
Radio Links in the Millimeter Wave Band
163
These studies represent only a sample of a number which were conducted in this domain. Thus, this vast program allowed apprehension of the complexity of the propagation phenomena in natural environments and to establish statistical distributions about the occurrence of parameters involved in propagation, as needed by the designers. These statistical studies provide reliable transmission support for high data rate systems over short radio links. 4. Millimeter wave indoor LAN 4.1. Introduction In recent years, the increasing use of processing applications with high data speed exchanges have involved an important development of the local area networks (LAN). At the same time, the progress of wireless communication systems lead to development of WLAN which are very attractive because of their layout flexibility and the portability of their terminals. Currently, the two most advanced standards (Hiperlan 2 and IEEE 802.11) working at frequencies of about 2, 4 or 5 GHz offer data rates of about 20 and 50 Mbit/s that remain lower than those for cable networks. In order to exceed these limits and to reach data rates higher than 100 Mb/s, the 60 GHz band is currently studied. Indeed, this frequency band is particularly adapted to local area networks. The strong attenuation of the waves at this frequency ensures relative safety between rooms and the outside. That makes possible to employ limited cells by partition of the rooms and facilitate the engineering deployment. The low dimension of the antennas working at 60 GHz is also an advantage compared with lower frequencies. Moreover, the available bandwidth at 60 GHz is a major asset compared to the other lower frequencies and makes possible to reach rates higher than 100 Mb/s. 4.2. Study of the propagation channel Because of the multipaths (multiple reflections from the ground, the walls, furniture etc), the received signal is composed of several differently attenuated and delayed components. This behaviour is translated in the frequency field by a different selectivity from the various spectral components of the propagation channel that behaves like a selective filter with regard to frequency. It is thus necessary to optimise the system according to the propagation conditions. The deployment of a wireless indoor system must thus obey two major constraints: • •
To ensure a controlled radio coverage of the building in order to optimise the use of the frequency band and the cost of the infrastructure. To know the selectivity of the channel, which allows the optimisation the system parameters (modulation, coding etc) in order to reach the target rate with
164
Communicating with Smart Objects
a minimal error rate. The preliminary estimation of the characteristics of data rates and error rates attached to the services envisaged is an essential stage.
4.3. Statistical study of the channel of propagation The first stage consists in knowing how to establish a link budget of typical connection according to the distance transmitter-receiver and the environment. The statistical study of the channel was carried out from measurements taken in various kinds of environments: offices for the R-LAN and residential, for home applications (multimedia). Line of sight (LOS) measurements carried out (Figure 13.3) present a decrease that is proportional to the logarithm of the distance. On the other hand, in the none line of sight (NLOS) conditions this loss law in function of the distance is not respected (Figure 13.4).
Figure 13.3 Propagation loss in LOS condition [GUILET, 2001]
Figure 13.4 Propagation loss in NLOS condition [GUILET, 2001]
On the other hand, taking into account the number of crossed partitions gives the significant results. Figure 13.5 presents the additional loss measured compared to that of free space as a function of the number of crossed partitions. The values are, for a partition (plasterboard), about 19 dB and of 27 dB for two partitions. These high values show that a propagation model integrating the number of crossed partitions will be better.
Radio Links in the Millimeter Wave Band
165
Figure 13.5 Influence of the number of partitions crossed by the direct path on the propagation loss [1]
The estimated link budget cannot thus be only based on the direct path but needs to take into account additional elements such as the interior architectures and the propagation phenomena (reflection, transmission, diffraction) that a ray model, based on geometrical optics, can simulate.
4.4. The propagation model Modelling the propagation at 60 GHz in an indoor environment has two distinct interests: • •
To help to deploy and to give a cost estimate of the coverage for such systems implemented in different environments. To generate a channel model that can be used in the transmission-simulated system and allows an optimisation of the parameter setting of the system.
A 3D ray-tracing model has been developed and adjusted at 60 GHz for an indoor environment, coverage examples for different heights of reception (antennas on PC, in height) are described in Figure 13.6.
166
Communicating with Smart Objects
Figure 13.6 Radio coverage with 2m (a), 1.5m (c) and 0.5m (d) above the ground for the configuration (b)
A study of the model parameters allows optimisation of the number of reflections to be calculated to simulate the indoor propagation at 60 GHz [CHAIGNEAUD, 2001]. From this tool, a propagation channel model can be generated in order to test various configurations of link without having to realise the corresponding measurements. 4.5. System study The simulated system study was realised in synergy with the characterisation of the channel in order to adapt the system to the propagation conditions in this band. The objective is to define the physical layer most adapted to reach data rates higher than 120Mb/s. 4.5.1. Multi-carriers techniques The techniques of multi-carrier transmissions like COFDM were developed for different wireless systems such as DAB, HIPERLAN... different multi-carrier techniques were used to evaluate the performances of the physical layer for the 60
Radio Links in the Millimeter Wave Band
167
GHz RLANs systems. MC-CDMA techniques initiated in 1993, were confronted with COFDM modulations. MC-CDMA, in particular, allows one to combine the advantages of COFDM modulations (multipaths robustness) with the spread spectrum techniques (high capacity). On the other hand, it increases the complexity of the receiver. 4.5.2. Results The first results in a multi-carries configuration in "offices" and "residential" environments are promising in spite of the un-optimised parameter setting. The results in Figure 13.7 represent the error rate obtained according to the useful bit energy proportional to the system's signal to noise ratio. These curves were simulated for two configurations: • •
LOS: Transmitter and receiver are of visibility. NLOS: Transmitter and receiver are masked by one or more elements of interior architecture (partition, movable, carries..).
For each of these configurations (LOS and NLOS), the tests were made with "typical" files corresponding to the propagation channel having some average wideband characteristics. These first results show that for an error rate of 10-2, a signal-to-noise ratio needs to be twice more powerful in none-visibility (NLOS) than in visibility (LOS). Optimisation of parameter setting of the COFDM chain in progress will allow improvement of these results. Moreover, it appears starting from the first curves, that the use of multi-input/output techniques in none-visibility is to be considered. The degradation induced by RF stages is particularly penalizing in the COFDM modulation, and the need to evaluate a broad panel of system solutions resulted in also studying mono-carrier solutions integrating the algorithms of turbo-equalisation in reception [HELARD, 2001].
Figure 13.7 Evaluation of the error rate according to various favourable and unfavourable positions in residential environment [SIAUD, 2001]
168
Communicating with Smart Objects
5. Conclusions These studies allowed apprehension of the complexity of the phenomena of propagation in the millimeter bands. Thanks to the outdoor studies, prediction models for the losses due to the hydrometeors were developed in order to offer reliable support for transmission in the millimeter band for high data rates proximity links. The characterisation and the modelling of the indoor propagation channel at 60 GHz allow use of simulating systems working at data rates higher than 100 Mb/s. Thanks to the systems working in the millimeter bands, the expectation of doubling or even tripling the data rates of the current systems offer new possibilities of services in many fields (multimedia, professional, medical, video) for communicating objects.
6. References [GUILET, 2001]: V.GUILLET. Narrowband and wideband characteristics of 60 GHz radio operator propagation in residential environment, Electronics letters, 2001, Vol37,N° 21. [CHAIGNEAUD, 2001]: LIONEL CHAIGNEAUD, VALERY GUILLET AND RODOLPHE VAUZELLE, A 3D Ray tracing tool for broadband wireless systems, VTC 2001, Atlantic City, October 2001. [SIAUD, 2001]: I SIAUD, R. LEGOUABLE, Mr. HELARD, 'One Multicarrier Transmission Techniques over Recorded Indoor Future Propagation Channel Models for Broadband RLANs at 60 GHz, PIMRC', 2001, 30 September-October 2001, San Diego, California. [HELARD, 2001]: Mr. HELARD, I SIAUD, C LANGLAIS, 'Principles of the turbo equalisation: Application to the transmissions radio operator Indoor with 60 GHz', to appear in the fourth days of study 'Electromagnetic propagation in the atmosphere of decametric in the angstrom', March 2002, Rennes.
7. Glossary COFDM: Orthogonal Coded Frequency Multiplexing Division. Hydrometeors: Rain, snow, fog. MC-CDMA: Multi-Carrier-Codes Division Multiple Access. Millimeter-length: Frequency band from 30 to 300 GHz.
Chapter 14
Propagation of Radio Waves Inside and Outside Buildings Herve Sizun France Telecom R&D, Belfort, France
1. Introduction Wireless communications between radioelectric systems and communicating objects require a good knowledge of the propagation channel. First it is necessary to evaluate the behaviour of the waves in the environment considered (rural, suburban, urban, dense urban, indoor) to be able to parameterise the emission power, the polarisation and to choose the antennas, the modulation, the transmission protocol, etc. A first part of this chapter will consist in briefly defining what is a radio wave and then presenting the various phenomena being able to affect it (reflection, transmission, diffraction, diffusion, guidance). In a second part the difficulties of transmission will be detailed resulting from the interactions of the waves, between them and with the environment, outside and inside buildings (shadowing, interferences, fading, etc). The last part will approach the available techniques for the study of propagation between communicating objects, mainly with models of propagation in various environments (rural, mountainous, suburban, urban, inside buildings, etc.) macro cell, small cell, micro cell, pico cell, launching and ray tracing models. These two last are more particularly adapted to understand and illustrate the various phenomena, inside and outside the buildings. 2. Electromagnetic waves The parameters characterizing the electromagnetic wave propagation are the electric field E, the magnetic field H, the electric flux density D and the magnetic
170
Communicating with Smart Objects
induction B. Only E and D vectors produce actions by which it is possible to measure the electromagnetic field. The vectors D and B are connected to E and H vectors by the following linear relations:
The coefficients 8 and u are respectively the permittivity and the magnetic permeability of the medium. In the absence of charged particles, they are linked by Maxwell's equations:
Their resolution brings us to the Helmholz equation for each vector E, B , D and H. The vector V (where V belongs to the unit E, B , D or H) is then given by the following equation:
where: Ais the Laplacian operator: grad(div) - rot (rot) In sinusoidal mode, the oscillations of the vectors E and H are propagated in space perpendicularly one to the other in the form of a wave (Figure 14.1) at velocity The time interval between two equal successive elongations in direction and size is called the period. The space crossed by the wave during this time is the wavelength. It is given by the relation:
where: f is the wave frequency, co is the pulsation, T is the period.
Propagation of Radio Waves Inside and Outside Buildings
171
Figure 14.1 Illustration of the propagation of an electromagnetic wave The Pointing vector describes amplitude and direction of transported power flux. Electromagnetic waves are generated by oscillating electric circuits, electric vibrations of dipoles or by electronic tubes. Their spectrum is very broad: we have ELF (fc>100 km), VLF (10<X<100 km), LF (1<X<10 km), MF (100<X<1000 m), HF (10<X<100 m), VHF (1<X<10 m), UHF (10
172
Communicating with Smart Objects
irregularity d, met during propagation, compared to the wavelength (connected to the frequency) plays an important role. The three following cases are considered: • • •
d/X < 1: irregularities are very small compared to the wavelength; statistical methods are then applied. The physical phenomena concerned will be primarily attenuation resulting from absorption and scattering. d/A, = 1: irregularities are of the same order as the wavelength; approximations are then not possible (resonance zone). dfk > 1: irregularities are very large compared to the wavelength; asymptotic methods based on optics and rays theory are applicable. The physical phenomena concerned will be primarily reflection, transmission, diffraction, etc.
3. Mechanisms of propagation Propagation paths (Figure 14.2) are of a different nature: direct, transmitted, reflected, scattered, diffracted or guided.
Figure 14.2 Various mechanisms of propagation
3.1. Direct paths A path is known as direct when the transmitter and the receiver are in radioelectrical visibility (in line of sight): the first Fresnel ellipsoid is not obstructed. This ellipse delimits space area through which nearly all the energy passes. It is the place of the
Propagation of Radio Waves Inside and Outside Buildings
173
M points which fulfills the following equation (E indicates the site of the transmitter, R that of the receiver, A, is the wavelength):
3.2. Transmitted paths Most of the time the direct path does not exist: it is blocked by obstacles (masks). Transmission is a phenomenon which allows waves to cross an obstacle (wall, building, vegetation, etc). 3.3. Reflected paths Reflection occurs when the wave meets a surface whose dimensions are large in comparison with the wavelength (ground, wall, fa9ade of building etc). The reflection characteristics of an unspecified surface depend on several factors: the surface of the material (smooth or rough), the wavelength and the incidence angle. The roughness of a surface relative to the wavelength constitutes an important parameter for the shape of the reflection diagram. A smooth surface reflects the incident radiation in only one direction as a mirror (specular reflection). On the contrary, a rough surface will reflect the incident radiation in all the directions. A surface is regarded as rough, according to the Rayleigh criterion, if the following relation is satisfied:
where: £ is the maximal height of the surface irregularities, A, is the incident radiation wavelength, 0i is the incidence angle. For an infrared radiation characterised by a wavelength of 1550 nm, under normal incidence, a surface is known as rough if the maximum height of the irregularities £ is greater than 0,19 jam. This result shows that the majority of surfaces met inside buildings are regarded as rough with infrared radiation; the diagram of reflection then presents a diffuse component (diffuse reflection).
174
Communicating with Smart Objects
3.3.1. Specular reflection Specular reflection, a phenomenon common to all the frequencies, is due to a perfectly plane homogeneous surface. The loss path induced by such reflections arises from the Fresnel relations and depends on the dielectric characteristics of the reflective surface (conductivity a, permittivity e). For example, the majority of surfaces inside the buildings have a reflection coefficient, in the infrared band, ranging between 0.4 and 0.9 [GFELLER, 1979]. Various reflection coefficients have been mentioned in the literature for different materials [YANG, 2000]. 3.3.2. Diffuse reflection Diffuse reflection is due to the reflections by surfaces which are not plane but rough; such surfaces present irregularities. The, an incident wave is not reflected in only one direction but diffused in multiple directions. Two models are usually used to represent reflection of the infrared band: Lambert and PHONG model. Lambert model Some surfaces are very irregular and reflect the infrared radiation in all the directions independently of the incidental radiation. Such surfaces are known as diffuse and can be represented by the Lambert model. The model is very simple and very easy to implement inside software. It is described by the following equation:
where: p is the surface reflection coefficient, Ri represents the incident optical power, 00 is the observation angle. Phong model The reflection diagram of several rough surfaces is well represented by the Lambert model except around specular reflection where the experimental diagram presents an important component. The Phong model considers the reflection diagram as the sum of two components: diffuse and specular components. The percentage of each component, depending mainly on the surface characteristics is a parameter of the model. The diffuse component is given by the Lambert model. The specular component is given by a function which depends on the angle of incidence 0i and the angle of observation (angle of reflection) 90. The Phong model is described by the following relation:
Propagation of Radio Waves Inside and Outside Buildings
175
where: p is the surface reflection coefficient, Ri represents the incident optical power, rd is the percentage of the ray which is considered in diffuse form (it is a value ranging between 0 and 1), m is a parameter which controls the directivity of the specular component of the reflection, 9i is the incidence angle, 90 is the observation angle. It will be noted that the Lambert model is equal to Phong's by taking rd equal to one. The Phong model, dependent on the observation and incidence angle, is more complex than Lambert's. The computing times are lengthened. The reflection pattern presents a principal lobe centered around the direction of the specular reflection. 3.4. Diffracted paths Diffractions occur when a wave meets an edge (hill, building, vegetation, roofs, corners of buildings, road structures etc.) whose dimensions are large compared to the wavelength. It constitutes one of the most important factors intervening in the propagation of radio waves [BOITHIAS, 1983]. The use of the geometrical theory of diffraction (TGD) makes possible to represent this phenomenon in the form of rays [McNAMARA, 1990]. Approximate expressions were proposed for attenuation evaluation by diffraction [BOITHIAS, 1983]:
for v > —0.7 and is usable in the vicinity of 0.
valid for v > 1,5
176
Communicating with Smart Objects
where:
d1 and d2 are respectively the distances from the edge to the transmitter and the receiver. The reader will find hereafter, as an example, some typical values of attenuation at 4 frequencies (20, 40, 60 and 100 GHz) for edges (of height 1 and 10 m) located at 1 km of the transmitter and receiver. The results are summarised below. Table 14.1 Typical values of attenuation by diffraction (in dB)
Diffraction attenuation (dB) Frequency = 20 GHz fREQUENCY =40gHZ
Frequency = 60 GHz Frequency =100 GHz
d=l km, h=1m 10 14 16 18
d=lkm, h=10m 30.3 33.3 35 37
3.5. Guided paths Certain environments (deep streets (street canyons), corridors, tunnels, etc.) behave like true waveguides with respect to the propagation of the radiowaves consecutively to multiple successive reflections on the walls (application of propagation modes and ray theory more particularly when the wavelength is very small compared to the transverse section of the tunnel, for example). 4. Propagation channel properties These various paths of different amplitude and phase interfere while arriving at the receiver. The interferences are constructive (the different paths arrive in phase) leading to signal reinforcement or destruction; one then has signal fade. Moreover, it should be noted that the mobile moves in this figure of interferences. It sees successively luminous and dark patches (interference fringes) causing fading of the signal. Because of the presence of multiple paths and displacement of the receiver, the propagation channel has three fundamental properties: attenuation, variability and selectivity in frequency. In analogical communication, the attenuation concept was sufficient to study the propagation channel. In the context of numerical communication, fading due to
Propagation of Radio Waves Inside and Outside Buildings
177
variability and the selectivity induce deterioration of the quality of the communication independently of attenuation.
4.1. Attenuation We generally distinguish the free-space attenuation and the attenuation relative to free-space (excess attenuation) Attenuation in free-space (loss transmission) is due the dispersion of energy in the vacuum as it moves away from the transmitter. It is given by the following relation:
where: d is the distance between the transmitter and the receiver, A, is the wavelength, f is the frequency. Example: d=lkm f =900 MHz, A0 = 91.4 dB f=1800MHz,A 0 = 97.4dB This supposes that one has only one radioelectric path whose 1st Fresnel ellipsoid is not obstructed. It attenuation is governed by the 1/d2 law (energy conservation). That means that we lose 6 dB each time the distance is doubled. Attenuation relative to free-space attenuation is the difference between the basic transmission loss and the free-space basic transmission loss expressed in decibels; this due to absorption by gases, the hydrometeors, the walls, the vegetation, attenuation by diffraction, etc.
4.2. Variability The propagation environment fluctuates: the passage of vehicles and people, the wind in the trees, the opening of doors cause fluctuations of the radioelectric paths and generate fast variations the observed signal. Combined with the movement of vehicles, these phenomena create a variability of propagation in space and time. The variations of the signal are of random nature. Statistical analysis will make it possible to evaluate the impact of the multiple paths on the transmission of radiomobiles systems. The law of the field received and the fading average time for example are essential data to dimension the transmission equipment and their ability to fight against fading (interlacing, diversity).
178
Communicating with Smart Objects
Modelling of the fast variations was proposed [CLARKE, 1968]. By supposing that the mobile moves in an interference figure generated by the superposition of a great number of plane waves of amplitudes, independent phases and random directions, the field received (the complex envelop) is, by the application of the theorem of the central limit, a Gaussian variable. The narrow band signal envelope, the power received, then follows a Rayleigh distribution. One speaks about Rayleigh fading. In the case of a predominant path (line of sight, open environments such as suburbs, railway stations, rural areas, etc), the narrow band signal envelope, the received power, then follows a Rice distribution. This is observed when one of the paths is predominant. The Rice distribution corresponds to a Rayleigh distribution when r = 0 (absence of direct path) and makes it possible to identify a direct path and its preponderance. It is also characterised by the K coefficient (Rice parameter) defined by the relation:
The K parameter represents the relation between the power of the direct path and the contribution to power in the secondary paths following a Rayleigh distribution. The larger the parameter K, the more important the direct path is in comparison with the multiple paths and the clearer the connection is. Conversely, if the predominant path is weak, (K < -5 dB) one considers that the Rice distribution is identified with a Rayleigh distribution [CCIR, 1990]. The K value depends on the environment (dense urban, suburban, rural, etc). Others distributions, such as those of Weibull and Nakagami [BRAUN, 1991] also make it possible to characterise the radiomobile signal envelope. In order to determine the variation law followed by the signal one applies statistical tests for example the Kolmogorov-Smirnov test [BARBOT, 1992] to its fast variations resulting from the interference of the waves received. Knowing the statistical characteristics of the signal (density of probability, function of distribution), makes it possible to determine the relevant parameters of a radioelectric system operation among which: •
•
probability of going below a certain level. When the signal power is lower than the noise threshold tolerated by the receiver (thermal noise, jamming, industrial parasites, etc), the signal is mothered by the noise. The receiver is then unable to correctly interpret the transmitted information. statistical fading duration. During fading, information packages are lost. The slower the mobile moves, the longer the cut-off times.
Propagation of Radio Waves Inside and Outside Buildings
179
4.3. Selectivity When the differences of the multiple paths delays are important, the transfer function is not constant over all the width of the spectrum: the path loss depends on the frequency used. The channel is known as selective in frequency. Wideband modelling of the channel is then essential to evaluate the performance of a complete transmission chain, to conceive the new systems and to ensure the transmission quality of transmission of numerical signals. The radioelectric channel is represented by its variable impulse response in time, h(t-t), i being the delay and t pointing out the dependence on time (and thus in space since the vehicle moves). A function of two variables, it translates the three characteristics of the channel: the attenuation, the variability (t) and the selectivity (I). The dual variables, by Fourier transform of T and t are respectively the frequency and Doppler speed [PARSONS, 1992], [KATTERBACH, 1995]. The output signal y(t), as a function of the input signal x(t), is represented in the form of an equation of convolution whose convolution core is variable in time:
where h(t-x) is the impulse response at the moment t to a radioelectric impulse x(t) which one would have emitted at the moment t-T. It completely defines the propagation channel. It makes possible to distinguish the various echoes according to their delays from propagation. It is useful to qualify the selectivity of the channel by parameters deduced from the average profile of power of the impulse response. The most used are the mean delay, the rms delay spread, the delay interval, the delay window and the coherence bandwidth of the channel [FAILLY, 1989].
180
Communicating with Smart Objects
Figure 14.3 Schematic representation of the temporal evolution of the impulse response of the propagation channel
4.3.1. Average delay profile One defines the average delay profile P(x) of the impulse response from h(t-i) by the relation:
It corresponds to an average over a certain duration T (Figure 14.3) selected so that the measured impulse responses can be presented as a stationary and an ergodic random process [LAVERGNAT, 1997]. 4.3.2. Average delay The average delay is the average of the delays weighted by their power. It is given by the first moment of the impulse response:
where: TLQS is the delay of the line of sight path, 13 is the instant when P(t) exceeds the cut-off level for the last time, Pm is the total energy of the impulse response, defined by the following relation:
Propagation of Radio Waves Inside and Outside Buildings
181
where: P(t, i)is the power density of the impulse response, T is the excess delay, TO is the instant when P(t) exceeds the cut-off level for the first time.
4.3.3. The rms delay spread The rms delay spread is the power weighted standard deviation of the excess delays. It characterises the variability of the rms mean delay. It is given by the second moment of the impulse response:
The rms delay spread illustrates the risk of appearance of intersymbol interferences and the disturbing effects that the remote and powerful echoes are likely to generate.
4.3.4. The delay interval The delay interval at X dB is defined as the time interval between the time T1 when the amplitude of the impulse response exceeds for the first time a given threshold and the time 12 when this amplitude becomes for the last time lower than this threshold (Figure 14.4).
182
Communicating with Smart Objects
Figure 14.4 Example of power delay profile: highlighting of the delay interval at X dB
Figure 14.5 Example of power delay profile; highlighting of the delay window containing y% of the total energy found in the impulse response
4.3.5. The delay window The delay window is the duration of the central portion (T2-T1) containing y % of the total energy found in the impulse response. Times T1 and T2 are defined by the relation (Figure 14.5):
Propagation of Radio Waves Inside and Outside Buildings
183
4.3.6. The correlation bandwidth The correlation bandwidth (Bc) of the channel is defined in the following way. Let C(t, f) be the autocorrelation of the transfer function (Fourier Transformation of the power of the impulse response).
The correlation bandwidth is defined as the frequency for which |C(t, f)| is equal to X % of C(t, f=0) [PARSONS, 1992]. It indicates the amplitude of selective attenuation depending on separation in frequency. The correlation band is thus the frequency from which the autocorrelation function of the transfer function falls below a given threshold. For the analysis of the experimental data, the UIT-R recommends use of delay intervals for thresholds of 9, 12 and 15 dB below the peak value, the delay windows for 50%, 75% and 90% of energy and a correlation bandwidth for 50% and 90% correlation. The correlation bandwidth is linked to the delay spread by the following relation [LEE, 1993]:
The propagation channel is more or less selective according to the environment. The form of the impulse response is also different according to the environment. In pico cell environments (inside buildings), the decrease of the power with respect to delay on the impulse response is exponential whereas in small and micro cells important delays are very clearly distinguished. One will note, in Figure 14.6, that the time scales are not the same for a pico cell and for small and micro cell environments.
184
Communicating with Smart Objects
-40:
From left to right: inside building, small and micro cell environments [Retard relatif= relative delay] Figure 14.6 Examples of impulse responses measured in various environments 5. Propagation modelling In the face of unceasingly increasing demands, the operators were made to improve their networks: they increased the basic stations number and reduced cell size. The cell is the area zone covered by a base station. One generally distinguishes four kinds of cells (Figure 14.7). They sum up particular physical attributes of the base station antenna and its given geographical area: their characteristics are related to the power used, to the position and the height of the base station antenna and to the geographical environment.
Figure 14.7 Various types of cells The largest cell is the macro cell. The surrounding medium is in general rural or mountainous and the base station antenna is positioned at a very high point. The distance between the base station and the mobile can be greater than ten kilometers. This geographical area, little urbanised, involves a certain number of paths in great
Propagation of Radio Waves Inside and Outside Buildings
185
delays (up to 30 [is). Moreover, the diffusers, being moved away and in a restricted number, do not generate very marked fast fading. Facing the increasing number of mainly downtown users, it was necessary to reduce the cells size in order to cut down the re-use of the allocated frequencies distance. The most current of the established downtown cells is the small cell. It covers a ray of less amplitude than a few kilometers and the base station antenna is located above roof level. The maximum duration of the impulse response is 1010uS. In a very dense urban zone, the small cell is replaced by a micro cell which has an activity ray of a few hundreds of meters. For antennas under roof level, the wave propagation is guided by the streets. The maximum duration of the impulse response is 2 us. There is finally the pico cell, of a few ten meters ray, which corresponds to a communication inside the buildings in which the antennas of the base stations are placed. The maximum duration of the impulse response is 1 us. To each environment corresponds a certain number of models (theoretical, empirical or statistical, or semi empirical). The theoretical models are based on the fundamental laws of physics with adequate approximations. The empirical models are based on the statistical analysis of a great number of experimental measurements carried out according to various parameters such as the frequency, the distance, the effective height of the emission antenna, the reception antenna, etc. The semi-empirical models combine an analytical formulation of physical phenomena (reflection, transmission, diffusion, diffraction) and an adjustment of the variables using experimental measurements. Propagation models are used when designing radio interface to optimise the performances and also during the systems deployment to determine the radioelectric cover. We will only detail here the most-often adapted models to the deployment of communicating objects: the microcell model, ray launching model, penetration model and ray tracing model inside as well outside buildings.
5.1. Microcell model Microcell modelling is built on a duality related to the short distances considered (a few hundreds of meters at most): a calculation of line of sight (LOS) and a calculation in non line of sight (NLOS). When the transmitter is located in the street below roof level, the propagation is guided along the streets: either in the street where is a transmitter (LOS), or in the adjacent streets after the passage of at least one corner (NLOS). We find these two kinds of calculation in the majority of the
186
Communicating with Smart Objects
analytical models 2D (profile) suggested in the literature, in particular the double slope model [XIA, 1993] which considers an attenuation in d2 (20log10d) for line of sight (LOS) at short distances and an attenuation in d4 (401og10d) for non line of sight (NLOS) or for line of sight at long distances (combination of the direct path and the path reflected on the ground). The breakpoint is generally given by the following relation:
where hb and hm represent respectively the height of the emission station and the mobile one's. In practice, it is very difficult to estimate the slope value in NLOS because it strongly depends on the angle a between the streets. In fact, it works on the uniform theory of diffraction (TUD) [KELLER, 1962], [KOUYOUMJIAN, 1974] leading to a precise analytical formulation of the propagation phenomena in urban zones [BERTONI, 1994], [BERG, 1995], [JAKOBY, 1995]. An analysis of the propagation at 900 MHz and 1800 MHz on passing a street corner, by simulations carried out using electromagnetic methods (TUD) led to simplified analytical expressions for the propagation mechanisms in line of sight, reflection and diffraction depending on the origin and destination of widths streets of width respectively W1 and W2, of distance from the transmitter to the corner (D), distance from the corner to the receiver (X), angle a formed by the two streets and with emission frequency (Figure 14.8). Attenuation (Aff) on the level of the receiver after passing of the corner breaks up in the following way:
where: AffVis is the line of sight attenuation at distance D
AffRef is attenuation due to the reflection [WIART, 1993].
with
Propagation of Radio Waves Inside and Outside Buildings
187
f is the frequency used, D is the distance between the transmitter and the street corner, S represents the slope of the power decrease with passing of the street, W1 and W2 are the street widths in line or in non line of sight, X is the distance from the receiver to the street corner, f(a) is a function of the angle a of the street. AffDif is the attenuation due to diffraction [WIART, 1993].
where:
Figure 14.8 Standard case of the passage at a crossroads in microcell model
To determine attenuation between a transmitter and a receiver, the model seeks all possible paths between the emission point and the reception point. The contributions delivered by the various paths are totalled in Watts, and the resultant is converted into decibels.
188
Communicating with Smart Objects
5.2. Ray launching model The technique of ray launching is a particularly promising deterministic technique. Being based on very precise geographical data bases and on physical theory, it is particularly well adapted for urban environments and makes it possible to obtain computation results with very rich information like the impulse responses. It is for these reasons that, recently, many models using this technique have appeared, [KURNER, 1993], [LAWTON, 1994], [LIANG, 1998]. The RAY model developed by FTR&D is a model based on a technique of ray launching and is designed to carry out a systematic research of the paths connecting the transmitter to the receiver using a combination of the physical phenomena reflections, horizontal and vertical diffractions, penetration and crossing of vegetation. All these phenomena are deduced only from their theoretical values given by the uniform theory of diffraction (TUD) for diffractions and by the Fresnel and Beckmann formulae for the reflections. The traditional method is to launch, in all directions, a significant number of the guidelines starting from the transmitter. Each ray represents the part of the electromagnetic wave included in a cone around this guideline. For reasons of saving calculation, some assumptions make it possible to simplify the geometry of the problem. Figure 14.9 describes the actual steps followed [ROSSI, 1991], [ROSSI, 1992]. Initially, only the horizontal plane is considered, ie the base station E as all the obstacles are represented by their vertical projection on this plane. Then the rays are sent, starting from the transmitter E, at regular angular intervals in the plane. At this stage, in 3D space, the ray does not represent any longer a cone around the ray but a vertical section of space. When a ray comes up against a building face (point 1 on Figure 14.9), it has the possibility either to pass over, or to be reflected. The two cases are explored by creating, starting from this point, two branches: one continues its way in a straight line, the other is directed in the specular reflection direction. The same duplication is produced each time the ray meets an obstacle until the ray i too much attenuated or comes out of the framework of the ground data. When the ray meets a receiver R, the third dimension is then considered by unfolding the ray according to its curvilinear co-ordinates on the viewgraph (Figure 14.9); the reflective buildings are represented there by a simple vertical bar. The way followed by the ray corresponds to the shortest way between the base and the mobile which passes above the diffracting buildings. Once this cut is obtained, the rigorous attenuation calculation on the connection is possible. The average power received by the receiver is the resultant of the powers of each ray arriving at this point. Figure 14.10 gives an example of all the rays connecting a transmitter and a receiver whose power is in a dynamics of 20 dB.
Propagation of Radio Waves Inside and Outside Buildings
Figure 14.9 Principle of a ray launching model
189
190
Communicating with Smart Objects
Figure 14.10 Diffracted and reflected rays between a transmitter (e) and a receiver (r) (ray model)
5.3. Penetration model
The penetration attenuation in a building is defined as being the power loss undergone by the electromagnetic field between the outside of the building and one or many positions inside the building. It is calculated by comparing the external field and the field in the parts of the building where the receiving mobile is situated. The attenuation models, integrated into the field prediction tools, must take into account the environment close to the building studied. The parameters affecting the values of penetration attenuation are multiple and their effects intermingle most of the time. Among all these parameters, one generally distinguishes the following traditional parameters: neighbouring environment. One will distinguish districts with more or less large open viewing towers from each other, and the more traditional districts comprising average height buildings. depth of reception in the buildings. The field decreases when the mobile moves from the front towards the bottom of a room inside the building. The effect of the face inhomogeneities decreases when one approaches the bottom of the room. The waves penetrate much easier into the building through the glazed parts than the brick walls and consequently certain paths are more or less attenuated or even shut out. The glazed faces in general present a 6 dB penetration attenuation power compared to the unglazed faces [RAPPAPORT,
Propagation of Radio Waves Inside and Outside Buildings
• •
• • • •
191
1994]. At the bottom of the building attenuation is more important but also much more homogeneous. angle of incidence angle. It acts on the reflection and transmission coefficients through a surface. reception height more commonly called "floor effect". In a general way this floor effect is expressed in the form of a reduction in the penetration attenuation or of a relative power gain compared to the lower floor in a small cell context (emission antenna located above the roof level). The calculation base is thus the penetration attenuation at ground floor determined comparative with the external field. In a small cell, the power gain observed is classically about 2 to 3 dB per floor at 900 and 1800 MHz. However, the strong diversity of the situations causes a dispersion of the gain by floor; values of about 4 to 7 dB were already measured in practice [GAHLEITNER, 1994]. The floors of the lower part are illuminated by rays undergoing a number of reflections and diffraction on the roofs and in the street whereas the floors of the higher part have the advantage of a much more significant illumination, sometimes even direct. The effects on the penetration are thus varied [RAPAPPORT, 1994] and [WALKER, 1983]. distance between the transmitter and the receiver, when the building sheltering the mobile is visible from the transmitting antenna. Then the penetration attenuation depends on the distance by the free space propagation law. height of the emission antenna. frequency. type of crossed materials. The crossing of materials causes an electromagnetic wave attenuation, from about 4 dB (wood) to 10 dB (concrete) [COST231, 1999].
Various measurement techniques were developed to characterise the penetration attenuation due to building materials. One can in particular quote the method with two reverberating rooms [FOULONNEAU, 1996]. Most traditional models draw from the Motley Keenan model [MOTLEY, 1988] used for propagation inside the buildings. They take into account the penetration attenuation according to parameters such as: the distance between the transmitter and the building external wall where the receiver is located, the distance between the external wall and the receiver, the number of internal walls cut by the profile, the floor effect, the attenuation of the building's external wall, the attenuation of the internal walls. The path loss (L) is expressed by a sum of the losses in free-space (L0), losses due to the obstacles crossed by the direct ray (flagstones, walls, doors, windows), of a constant (Lc) [MOTLEY, 1988]. The data base can differentiate the various
192
Communicating with Smart Objects
obstacles with which a particular attenuation value is associated. It is the most used model.
where: NJ is the number of crossed walls of type J, LJ are the losses due to walls of type J, N is the number of the types of different walls, Nf is the number of crossed flagstones, Lf are the losses by flagstone. 5.4. Indoor modelling Propagation inside buildings depends on the kind of environment: dense (buildings of office type), open (buildings of office type, large offices being able to accommodate several people), broad (buildings having very large rooms such as warehouses, airports, stations) and corridor (transmitter and receiver being located in the same corridor). It is also multi path: the predominant mechanisms are reflection, transmission, diffraction and scattering [HASHEMI, 1993b], [VALENZUELA, 1997].
5.4.1. Empirical models One distinguishes the profile models by distance, the Motley-Keenan profile type [KEENAN, 1990] and the multi-rays models called corridor model. In the case of the profile models at a distance, the parameters taken into account are the frequency and the distance between the transmitter and the receiver. Two models are proposed, the first [COST231, 1999] supposes a logarithmic dependence of attenuation of function of the distance, the second supposes a linear dependence [COST231, 1999]. The two models are usable for situations of non line of sight. The models of the Motley-Keenan type apply to the cases of non line of sight in a dense environment (office). The loss path (L), as in penetration, is expressed by a sum of the losses in free-space (L0), losses due to the obstacles crossed by the direct ray (flagstones, walls, doors, windows), and of a constant (Lc) [MOTLEY, 1988]. The data base can differentiate the various obstacles with which a particular value of attenuation is associated. It is the most used model.
Propagation of Radio Waves Inside and Outside Buildings
193
where: NJ is the number of crossed walls of type J, LJ are the losses due to the walls of type J, N is the wall kinds number, Nf is the number of crossed flagstones, Lf are the losses by flagstone. 5.4.2. Deterministic models In the ray launching technique, the models suggested are generally in 3 dimensions because the propagation medium is dense. Developed by many authors [RAPPAPORT, 1994], [SEIDEL, 1994], [CICHON, 1994], they lead to encouraging results [RAPPAPORT, 1994]. They constitute good references for the settling of the multipath model but remain of restricted use in engineering because of substantial computing time. The ray tracing technique consists of building rays by using the image theory. It makes it possible to easily take into account the mechanisms of propagation (multiple reflections, diffractions, etc). This technique was used by many authors [MC KOWN, 1991], [COST231, 1999], [VALENZUELA, 1994], [JENVEY, 1994], [LAURENSON, 1993]. The two other approaches, the finite differences method [MURCH, 1994], [LAUER, 1994], and the Tayleig-Gans approximation [LU, 1993], use relations derived from Maxwell's equations. The propagation medium is meshed. The radioelectric field propagates, iteration after iteration, in space surrounding the transmitter. The computing time is very important because the mesh step must be lower than wavelength (about A/ 8). However, one currently does not have representative results of their performances, the methods being very recent. 5.5. Ray models Ray models are deterministic models. They rest on a precise knowledge of the reality and require to have geographical data bases of "contour" type or interior of buildings. They make it possible to predict the various propagation paths in a given configuration. After adjustment with the frequency band considered, these models also make it possible by simulation to carry out parametric studies to analyse for example the influence of the antenna's diagram pattern or material characteristics, which is much more economic than the realisation of multiple series of measurements. We give below, in illustration, two examples of ray tracing: one outside the buildings (Figure 14.11) and the other inside the buildings [CHAIGNEAUD, 200la, 2001b, 2002] (Figure 14.12).
194
Communicating with Smart Objects
Figure 14.9 Example of ray tracing outside buildings
Figure 14.10 Example of ray tracing inside buildings
Propagation of Radio Waves Inside and Outside Buildings
195
6. Conclusion After having briefly described electromagnetic waves we presented the various mechanisms of propagation (reflection, transmission, diffraction, scattering etc) as well as the properties of the channel (attenuation, variability, selectivity). The selectivity of the channel is defined starting from parameters deduced from the average delay profile (average delay, delay spread, delay interval and delay window, correlation bandwidth). These various parameters make it possible to characterise the propagation medium necessary for the definition and deployment of the communicating objects. In the last part are presented the various models for the deployment of the mobile systems and communicating objects in various environments (rural, mountainous, suburban, urban, interior of buildings) and various types of cell (macrocell, small cell, microcell and pico-cell) by primarily limiting us to the models with narrow band. Moreover, more details concerning these various models can be obtained in the literature [LAGRANGE, 2000]. The broadband models such as path models (COST 207, ATDMA [RACE, 1994], IUT-R, CSELT), representation models (model of channel recorded in deterministic propagation) and geometrical models were not approached. 7. Bibliography [BARBOT, 1992] BARBOT J.P., LEVY A.J., BIG J.C.; "Estimation of fast fading distribution functions", Com. URSI Commission F Open Symposium, 1992. [BERG, 1995] BERG J-E.; "A recursive method for street microcell path loss calculations", PIMRC'95, Toronto, Canada, pp. 140-143, 1995. [BERTONI, 1994] BERTONI H.L., HONCHARENKO W., MACIEL L.R., XIA H.H.; "UHF propagation prediction for wireless personal communications", Proceedings of the IEEE, Vol. 82, n° 9, pp. 1333-1359, 1994. [BOITHIAS, 1893] BOITHIAS L.; "Propagation des ondes radioelectriques dans 1'environnement terrestre", Dunod, 1983. [BRAUN, 1991] BRAUN W.R., DERSCH U.; "A physical mobile radio channel", IEEE Transactions on Vehicular Technology, Vol. 40, n° 2, pp. 472-482. [CHAIGNEAUD, 2001a] CHAIGNEAUD L., GUILLET V., VAUZELLE R.; "3D ray tracing method for indoor propagation modelling at 60 GHz", European Conference on Wireless technology, London, September 2001.
196
Communicating with Smart Objects
[CHAIGNEAUD, 2001b] CHAIGNEAUD L., GUILLET V., VAUZELLE R.; "A 3D ray tool broadband wireless system", Vehicular Technology Conference, Atlantic City, October 2001. [CHAIGNEAUD, 2002] CHAIGNEAUD L., GUILLET V., VAUZELLE R.; "Methode de trace de rayon 3D pour la modelisation de la propagation en interieur a 60 GHz", Propagation electromagnetique dans 1'atmosphere du decametrique a 1'angstrom, Rennes, March 2002. [CICHON, 1994] Cichon D.J., Wiesbeck W., "Indoor and outdoor propagation modelling in pico cells", PIMRC'94, Personal Indoor Mobile Radio Communications, September 1994. [CLARKE, 1968] Clarke R.H., "A statistical theory of mobile-radio reception", B.S.T.J. pp. 957-1000, August 1968. [CNET/CSELT Cooperation, 1998] CNET/CSELT Cooperation, Data transmission on DECT standard, "Definition of common propagation models, regeneration scheme and performance evaluation criteria for the aligment of the two radio link simulators", March 1998. [COST 259] COST 259, Web informations: http://www.lx.it.pt/cost259. [COST 231, 1999] COST 231, "Evolution of land mobile radio (including personal) communications", Final report, Information, Technologies and Sciences, European Commission, 1999. [CROCHIERE, 1981] Crochiere R.E, Rabiner L.R., "Interpolation and decimation of digital signals - a tutoral review", Proceedings of the IEEE, Vol. 69, n° 3, pp. 300-331, March 1981. [FAILLY, 1989] Failly M., "Final Report of COST 207, Digital Land Mobile Radio Communications", CEE Luxembourg, 1989. [FOULONNEAU, 1996] Foulonneau B., Gaudaire F., Gabillet Y., "Measurement method of electromagnetic transmission loss of building components using two reverberation chambers", Elect. Letters, 7, Vol. 32, N° 23, pp. 2130-2131, 1996. [GAHLEITNER, 1994] Gahleitner R., Bonek E., "Radio waves penetration into urban buildings in small cell and microcells", Technische Universitat Wien, Vienna, Austria, Proceedings Vehicular Technology Conference, Stockholm, pp. 887-891, June 1994. [GFELLER, 1979] GFELLER F.R., BAPST U.R.S., "Wireless in-house data communication via diffuse infrared radiation" Proceedings of the IEEE, Vol. 67, n° 11, 1979.
Propagation of Radio Waves Inside and Outside Buildings
197
[HASHEMI, 1993b] Hashemi H., "The Indoor Radio Propagation Channel", Proceedings of the IEEE, Vol. 81, N° 7, pp. 943-968, 1993. [HATA, 1980] Hata M., "Empirical formula for propagation loss in land mobile radio service", IEEE Transactions on Vehicular Technology, Vol. 29, pp. 317-325, 1980. [IUT-R, 1996] International Telecommunication Union Study Groups "Guidelines for evaluation of radio transmission technologies for IMT-2000/FPLMTS", FPLMTS.REVAL, Question ITU-R, document 8/29-E, June 1996. [JAKOBY, 1995] Jakoby R., Liebenow U., "Modelling of radiowave propagation in microcells", Proc. Intern. Conference on Antennas and Propagation, ICAP, Eindhoven, The Netherlands, pp. 377-380, 1995. [JENVEY, 1994] Jenvey S., "Ray optics modelling for indoor propagation at 1.8 GHz", Proceedings of the IEEE 44th Vehicular Technology Conference, Stockholm Sweden, June 1994. [KATTENBACH, 1995] Kattenbach R., Fruchting H., "Calculation of system and correlation functions for WSSUS channels from wideband measurements", Frequenz 49, 3^, pp. 42-47, 1995. [KEENAN, 1990] Keenan J.M., Motley A.J., "Radio coverage in Buildings", British Telecom Technol. J., Vol 8, N° 1, January 1990. [KELLER, 1962] Keller J.B., "Geometrical theory of diffraction", JOSA Vol. 52, pp. 116-130, 1962. [KOUYOUMJIAN, 1974] Kouyoumjan R.G., Pathak P.H., "A uniform geometrical theory of diffraction for an edge in a perfectly conducting surface", Proceedings of the IEEE, Vol. 62, N° 11, pp. 1448-1461, November 1974. [KURNER, 1993] Kurner T., Cichon D.J., Wiesbeck W., "Concepts and Results for 3D Digital Terrain Based Wave propagation Models: an overview", IEEE Trans. Selected Areas in Com., Vol. SAC 11, N° 7, September 93, pp. 1002-1012. [LAGRANGE, 2000] LAGRANGE X., "Les reseaux mobiles"; Chapitre 2: Propagation radioelectrique (SIZUN H., BIC J.C.), Reseaux et Telecoms, Information-Commande-Communication, HERMES, 2000. [LASPOUGEAS, 2000] Laspougeas P., Pajusco P., Bic J.C., "Radio propagation in urban small cells environment at 2 GHz: Experimental spatio-temporal characterisation and spatial wideband channel model", Proceedings of the IEEE Vehicular Technology Conference (VTC'2000), Boston, 2000.
198
Communicating with Smart Objects
[LAUER, 1994] Lauer A., Bahr A., Wolff I., "FDTD simulations of indoor propagation", Proceedings of the 44th Vehicular Technology conference, Stockholm, Sweden, June 1994. [LAURENSON, 1993] Laurenson D.I., McLaughlin S., Sheikh A.U.H., "The application of ray tracing and the GTD to indoor channel modelling", IEEE Conf. GLOBECOM'93, Houston, USA, December 1993. [LAVERGNAT, 1997] Lavergnat J., Sylvain M., "Propagation des ondes radioelectriques", Collection Pedagogique de Telecommunication, MASSON, 1997. [LAWTON, 1994] Lawton M.C., Macgeehan J.P., "The application of a deterministic ray launching for the prediction of radiochannel characteristics in small cell environment", IEEE Transactions on Vehicular Technology, Vol 43, N° 4, pp. 955-969, November 1994. [LIANG, 1998] Liang G., Bertoni H.L., "A new approach to 3D ray tracing for propagation prediction in cities", IEEE Transactions on Antennas and Propagation, Vol. 46, N° 6, June 1998. [LU, 1993] Lu Y.E., "Site precise radio wave propagation simulations by time domain finite difference methods", Proceedings of the 43th Vehicular Technology Conference, Meadowlands, USA, May 1993. [Me KOWN, 1991] Mc Kown J.W., Hamilton R.L., "Ray tracing as a design tool for radio networks", IEEE Network Magazine, November 1991. [McNAMARA, 1990] McNAMARA D.A., PISTORIUS C.W.I., MALHERBE J.A.G.; The Uniform Geometrical Theory of diffraction, Artech, House, London, 1990. [METAMORP, 2000] Metamorp Project, "Description of the modeling method", Deliverable C2/1, 2000 (http://www.nt.tuwien.ac.at/mobile/projects/METAMORP/ en/. [MOTLEY, 1988] Motley A.J., Keenan J.M., "Personnal communication radio coverage in building at 900 MHz and 1700 MHz", Electronics Letters, Vol. 24, N° 12, June 1988. [MURCH, 1994] Murch R.D., Cheung K.W., Fong M.S., Sau J.H.M., Chuang J. CL. "A new approach to indoor propagation prediction", Proceedings of the 44th Vehicular Technology conference, Stockholm, Sweden, June 1994. [PARSONS, 1992] Parsons J.D., "The mobile radio propagation channel", Pentech Press Publishers, 1992.
Propagation of Radio Waves Inside and Outside Buildings
199
[RACE, 1994] RACE ATDMA Project, "Channel models Issue 2", R084/ESG/ CC3/DS/029/bl, Ed. R. GOLLREITER, May 1994. [RAPPAPORT, 1994] Rappaport T.S., Sandhu S. "Radio Wave Propagation for Emerging Wireless Personal Communication Systems", IEEE Antennas and Propagation Magazine Vol. 36, No. 5, pp. 14-23, October 1994. [ROSSI, 1991] Rossi J.P., Bic, J-C., Levy A.J., Gabillet Y., Rosen M., "A ray launching method for radio-mobile propagation in urban area", IEEE Antennas and Propagation Symposium, London (Ont.), Vol. 3, pp. 1540-1543, June 1991. [ROSSI, 1992] Rossi J.P., Levy A.J., "A ray model for decimetric radio-wave propagation in an urban area", Radio Science, Vol. 27 N° 6, pp 971-979, November-December 1992. [ROSSI, 1997] Rossi J-P., Barbot J-P., Levy A.J., "Theory and measurement of the angle of arrival and time delay of UHF radiowaves using a ring array", IEEE Transactions on Antennas and Propagation, Vol. 45, N° 5, pp. 876-884, May 1997. [SIAUD, 1996] Siaud I., "A digital signal processing approach for the mobile radio propagation channel simulation with time and frequency diversity applied to an indoor environment at 2.2 GHz", Personal Indoor Mobile Radio Communications Conference, PIMRC'96, Taiwan, 15-18 October 1996. [SIAUD, 1997] Siaud I., "A mobile propagation channel model with frequency hopping based on a digita signal processing and statistical analysis of wideband measurements applied in micro and small cells at 2.2 GHz", IEEE Vehicular Technology Conference, Phoenix, AZ, Vol. 2, pp. 1084-1088, 4-7 May 1997. [SIAUD, 1997] Siaud I., "Simulation du canal de propagation radiomobile en environnement urbain pour 1'etude des performances des systemes de communication de 3ieme generation avec diversite de frequence", 3 iemes journees d'etude, "Propagation electromagnetique dans 1'atmosphere du decametrique a 1'angstrom, pp. 277-282, 7-9 October 1997. [SEIDEL, 1994] Seidel S.Y., Rappaport T.S., "Site-specific propagation prediction for wireless in building personal communication system design", IEEE Transactions on Vehicular Technology, Vol 43, N° 4, November 1994. [VALENZUELA, 1994] Valenzuela R.A., "Ray tracing prediction of indoor radio propagation", PIMRC'94, Personal Indoor Mobile Radio Communications, September 1994. [VALENZUELA, 1997] Valenzuela R., Landron O., Jacobs, D.L., "Estimating Local Mean Signal Strength of Indoor Multipath Propagation", IEEE Transactions on Vehicular Technology, Vol. 46, N° 1, pp. 203-121, 1997.
200
Communicating with Smart Objects
[WALKER, 1983] Walker E.H., "Penetration of Radio Signals into Buildings in the Cellular Radio Environment", The Bell System Technical Journal, Vol. 62, N° 9, pp. 2719-2730, November 1993. [WIART, 1993] Wiart J., Marquis A., Juy M., "Analytical microcell path loss model at 2.2 GHz", PIMRC'93, Yokohama, September 8-11, 1993. [XIA, 1993] Xia H.H., Bertoni H.L., "Radio propagation characteristics for line-ofsight microcellular and personal communications", IEEE Antennas and Propagation, Vol. 41, N° 10, October 1993. [YANG, 2000] YANG, H., LU C.; "Infrared wireless LAN using multiple optical sources", IEEE Proc. OptoElectron., Vol. 147, N° 4, 2000. Acknowledgements My thanks to Mr. Jean Claude BIC for the assistance brought to the drafting of the "Radioelectric Propagation" chapter and to "Les reseaux radiomobiles", of the Reseaux and Telecoms HERMES collection, under the direction of Xavier LAGRANGE, a principal source of this paper.
Chapter 15
Ad-Hoc Networks Patrick Tortelier France Telecom R&D, France
1. Introduction Just as the Greeks had felt the need to encapsulate their notion of a human being as 'an animal which lives in spoils (zoon politikon)\ in the same way one could define current mankind as the only animal which communicates with its like at distances much greater than those that its senses can achieve - hearing, seeing, sense of smell (soon to be possible) etc. Thus, if one limits oneself only to technologies of wireless communication, one can note that if the first systems transmitted speech only, present day and more especially future systems will make it possible to see the interlocutor (transmission of animated images). There is also a scheme of communication where source and recipient are the same person at two different moments: for example when we note the hour and the place of an appointment, a telephone number to be recalled, a list of tasks to be carried out i.e. all kinds of personal data which we need to available for future use. Personal numerical assistants help us in this task. A last point is that, even if multimedia traffic is today rather asymmetrical (the flow towards the user being greater than that emitted by the user), multiplication of the means of creation of numerical content (images, sound, videos) or of their replication(hacking) and the development of the peer to peer networks change this situation. A consequence of that is that we are surrounded by an increasing number of increasingly sophisticated objects, which are supposed to make our personal life simpler, more lucid or more creative, so that technologies, which yesterday were reserved for laboratories, can be seen today in our living rooms. One finds a PC in every home, often a printer, personal assistants, CD engravers, DVD readers, and kits of satellite reception. All these objects require a number of cables, either to feed them on the sector, or to interconnect them or to the network, and one finds oneself quickly in the midst of a forest of not very aesthetic cables.
202
Communicating with Smart Objects
2. Communicating objects Naturally one would like to remove all these wires and to connect these devices by radio, just as one had been freed from the wire connection to the telephone network, in cities, with the cellular networks like GSM, or at home with for example DECT handsets. Of course there certain requisites for this kind of radio interface: 1. This interface must be standardised: objects coming from different manufacturers must be able to communicate; the number of potential candidates (IEEE 802.11b, HomeRF, Bluetooth) indicates that one is not yet at this stage. 2. It should not be expensive: Bluetooth would have a certain advantage on 802. lib in this respect. 3. It must allow a high data rate; in this respect 802.11b outperforms Bluetooth. 4. If designed for short ranges (a few tens of meters) it must consume little energy: if each object thus modified requires to be plugged into an outlet in order to work one will not have saved much cabling; they must thus work if possible using batteries, and must not be very greedy (batteries end up being costly!). 5. Finally, the management of all these objects and the routing of information in this network should not require the competence of a network engineer! It would be nice if the network reconfigures itself every time an object is introduced into the environment. This last aspect of the problem is the object of this contribution. 3. The Ad-Hoc networks Battlefield communications are an area of research which, at the very beginning of the Eighties, addressed a rather similar problem: how to deploy a radio communication network between "mobiles" (i.e.: soldiers, vehicles etc) disseminated on territory which, by its nature, does not have an infrastructure of communications (which it should not have, for it would be one of the first targets of the enemy). In such a context, the traditional scheme of the cellular network (where the mobiles of the same zone, the cell, are connected to a base-station) is impossible. In the solution that emerged each mobile uses a radio interface to transmit packet data and plays the role of a router (it transfers the packets from other users) within a meshed network that reconfigures itself automatically (it takes into account the mobility of the nodes, their traffic, their emergence or possible disappearance). More precisely it is a network whose infrastructure is made by the terminals themselves. The following figure illustrates the concept: the randomly distributed points are nodes, with for some of them the range of their radio interface (large circles in dotted lines). To go from A to B (which is out of reach of A) the packets pass by the intermediate nodes Cl C2 and C3.
Ad-Hoc Networks
203
Figure 15.1 The idea for Ad-Hoc networks
In spite of its military origin, the idea was quickly considered to be interesting for civil applications: •
•
•
The facility of deployment is an advantage in certain circumstances where one has no longer an infrastructure of communications: it is enough to think of the zones as victims of a catastrophe, a natural one (earthquake for example) or not: a good example was given after the attacks of last 11th September, where the rescuers could use as communication network the meshed Ricochet network system whose some relays (on pole tops) remained in the devastated zone; this network is an Ad-Hoc one. The fixed wireless access can benefit from this technique to relieve the constraint of direct visibility between base station and user; in a suburban habitat, each user having an antenna on the roof of his house can be used as a relay for other users, the link between a user and the access point to the fixed network being completed in several hops (multihop networks). The deployment is simplified. The capacity that have the nodes to be used as a router comes from the nature of the traffic generated at these nodes: this type of network is interesting in packet mode, for bursty traffic alternating active and idle periods where the radio interface can be used for routing the traffic from the other nodes.
204
Communicating with Smart Objects
burst n
burst
Figure 15.2 Model of traffic by bursts
There is another advantage which is of more physical origin and can be presented in the following simplified form. One can suppose all nodes to be of the same kind and p one notes the minimum received average power R that each receiver needs to work correctly. The power which must be emitted for a hop of length D is then of the form
PT = CstexPRda where a is path loss exponent; equal to 2 for a propagation in free free space and lying between 3 and 4 for more realistic environments. By supposing a distance L divided into say N hops of the same length d=L/n, the total power used for the TV transmissions overall is:
which shows a reduction by a factor n^(a-l) compared to a single hop transmission with a length L. The first idea which comes is to decrease the average distance between nodes of the network; this results in an increase of the density of users (which is interesting) and for each of them a decrease of the transmit power. But routing is a big difficulty.
4. The problem of routing When one joins by an edge the nodes which are reciprocally in the range of their radio interface, one draws a graph (a random one) which looks like the one depicted in Figure 15.3.
Ad-Hoc Networks
205
Given two arbitrary nodes, the problem of routing results in calculating 'the best' path which makes it possible to join them. It is a shorter path problem in a graph using several powerful algorithms (Ford-Bellman, Dijkstra, are best known) to solve it, but the true difficulty is elsewhere. Each node must have its own up to date routing table, and it is necessary to update these tables (the nodes are mobile, they can appear or disappear, and their activity varies with time). This constraint is the cause of a considerable overhead of data in the network, the mobiles having to emit updating information for these tables. This overhead increases with the size of the network and the mobility of the nodes. The various routing algorithms which are proposed (for example at IETF) aim to solve this problem.
Figure 15.3 Random grid
There is a difficulty in the choice of the criterion that states that a path is 'better' than another. It is a question of assigning a metric ('length') to each edge of the graph, so that the length of a path is the sum of the lengths of the branches that constitute it. Several metrics are possible, in particular: • •
The number of branches of the path. The quality of each branch, in order to favour the hops of good quality because they allow the transmission of a greater information rate. Moreover, if one mechanism of retransmission is envisaged on each hop, choosing a good hop minimises the number of retransmissions and thus the delay from end to end.
One can also mix several metrics.
206
Communicating with Smart Objects
5. A simplified solution It is clear that the mobility of the nodes causes a complexity increase at routing level. Thence arises the idea (at first sight contradictory with the initial idea) to conceive a mesh whose nodes are fixed! The difficulty is then to know what realistic usage corresponds to this scenario. Two commercial applications give us some indication: 1.
The first comes from Nokia-Rooftop, originally a start-up spun off from work financed by Darpa, which is now part of Nokia. It is an alternative solution to the wireless local loop, in which the constraint of visibility between the user and the access point to fix the network is slackened: the various users each have a modem/router on the roof of their home (where from the name of the product) and they constitute a mesh network where each node acts as a router for his neighbours towards the access points to the fixed network.
Figure 15.4 The Rooftop concept (ref. I)
2.
In the preceding example nodes do not have mobility any more; the second example presented introduced mobility of the users, but the users are no longer the nodes of the Ad-Hoc network. The Ricochet system (developed initially by the Metricom company, which has since gone bankrupt, but the network was repurchased) is a micro-cellular system in which the base stations form a mesh network (they are boxes fixed on pole tops in streets); a user is connected to one of these nodes with another radio interface that is used to connect the nodes, or the same interface in a different frequency band as shown in the following figure.
Ad-Hoc Networks
207
Figure 15.5 The Richotet system
6. Conclusion Contrary to the initial vision of a completely decentralised network, a less general and more pragmatic solution appears by introducing some constraints which simplify the routing problem: fixed terminals, or, if one really wishes to keep a certain mobility, a micro-cellular system in which the infrastructure consists of an Ad-Hoc mesh network, which should largely simplify its deployment, thus keeping this aspect. The Ad-Hoc network is often presented as a manner of connecting objects in an indoor environment, but a telecommunication operator (either fixed or with mobiles) can use them to deploy his network infrastructure. 7. References The bibliography on the Ad-Hoc networks is immense, so we will restrict it to some interesting papers or web links: The site of Nokia Rooftop: http://www.nwr.nokia.com/
208
Communicating with Smart Objects
T.J. Shepard: Decentralized Channel Management in Scalable Multihop SpreadSpectrum Packet Radio operator Networks, Report/ratio MIT/LCS/TR-670, July 1995. "Ad hoc mobile Networking (MANet)" http://www.ietf.org/html.charters/manet-charter.html
Chapter 16
INDEED: High Rate Infrared Communications in the "Indoor" Context Jean-Christophe Prunnot1,5, Adrian Mihaescu2, Christian Boisrobert1, Pascal Besnard2, Pierre Pellat-Finet3, Philippe Guignard5, Frederique De Fornel4 and Fabrice Bourgart5 1
LPIO Nantes, 2ENSSATLannion, 3LAUBS Lorient, 4LPUB Dijon, 5 France Telecom R&D, Lannion, France
1. Introduction The introduction of digital wireless links in local and private networks has been boosted by the development of portable PCs in business and leisure environment. The need for physical resources follow the constant increase of services in applications like videoconferencing and multimedia. Consequently, technology will lead to more and more intelligent functions in all kind of devices and equipment. They will be designed to communicate between themselves and the rest of the world. The connection through cables and wires is now considered inconvenient and unbearable in many situations and a major handicap. Furthermore, we must not forget the overlapping of wire connections caused by the multiplicity of links. The tremendous anticipated need for high quality communications justifies the development of radiowave or infrared solutions. Compared to what is considered to be their competitor, infrared optical waves can support heavy traffic, high bit-rates and more privacy/confidentiality in high density areas [Heatly, 1998]. Although Bluetooth and wide personal area network is commercially growing in 2002 as well as Hiperlan/2 (ETSI standards) in Europe and IEEE 802.1 la in the US, wireless local area networks, wireless infrared networks follow the IEEE 802.11 standards. IrDa [Williams, 2000] live a more harmonious existence with a universal standard IrDa which should allow for a full use of a large bandwidth opened to infrared waves (§ 3). Complete exploitation of infrared technology will justify the highest rates, far more than Hiperlan/4, which defines 155 Mbit/s on the lower level regarding physical connections, ie OSI model physical layer.
210
Communicating with Smart Objects
INDEED project (InDoor InfrarEd Eyesafe Dialogues) follows this context. The object of this study is to consider high rate infrared indoor communications problems, including one application which aims at creating a high rate safety bidirective link, in the 1550 nm optical area, between a top lightwave source and a mobile receiver in an office room. One part of this project will consist of evaluating a realised link by integrating it in ATM support TIC (Terminal Installation Client) of FT R&D [Bourgart, 2000].
2. Wireless infrared transmissions We firstly summarise the history of some of the applications and position the project in its context.
Figure 16.1 Different types of wireless transmissions [Kahn, 1997]
Several research teams have already studied indoor wireless transmission links (Figure 16.1) spectrally centered on 1550 nm. The two closest experiments concerning INDEED project are summarised in Table 16.1. Direct light detection and On Off Keying (OOK) modulation are used. The experimental results given in this table correspond to a bit error rate (BER) of 10-9.
INDEED: High Rate Infrared Communications
211
Table 16.1 Characteristics of wireless systems at 1550 nm
Reference Transmitter power Tx Receiver surface Photo-detector Sensitivity Rx Optical filter Attenuation Concentrator Gain Surface/FOV Cell area dimensions MAC Protocol Rate Propagation loss Margin
[Wisely, 1997] © + 8dBm 16 Tx Bootstrapped 1 mm2 PIN -32 dBm Yes -IdBm Yes +23 dB 710° 1m 2 4x4x2 m3 100 Mbit/s -60 dB +3dB
[Jungnickel, 1998] ® + 7 dBm ITx Unegalised 0.04 mm2 APD -39 dBm No Lens +6dB 25 cm2/ 20 m2 room H=2m CSMA/CD 140 Mbit/s -46 dB No
Wisely (British Telecom member) considers an area divided into cells and illuminated by 16 transmitters with a total power of 6.3 mW (8 dBm) and receivers with bootstrapped large area PIN photo detectors in a "line-of-sight" (LOS) scheme. In the non-LOS system proposed by Jungnickel (Institut Heinrich-Hertz, Berlin), the transmitter output power is 5 mW (7 dBm). The incident light shines on the avalanche photodiode through a concentrator. There is no power margin in the case of the Jungnickel's system. Experimental improvements have been obtained at higher bit rates using network access protocol, here CSMA/CD. 3. Eye safety The effects of the optical energy of a collimated beam laser are very different from the same optical energy transmitted by other light sources and this is due to its concentration, coherence, collimation and divergence properties. As for all illuminated surfaces, the biological tissues receive a high power density. The main phenomenon caused by the laser light is caused by high light density absorption. Absorption happens between atomic or molecular energy levels and is a process dependent on wavelength. Hence, it is wavelength that determines the tissue that laser can damage. Figure 16.2 gives the range and designation of different wavelength bands of the electromagnetic spectrum.
212
Communicating with Smart Objects
Wavelengths Figure 16.2 The electromagnetic spectrum, from the UVC to the IRC
Some biological tissues of the skin, the eye lens and more precisely the retina can show non-reversible modifications caused by long time exposure to moderate light levels. These changes are the result of photochemical reactions which occur after molecule activation by photon capture. They can be drastic if the time of exposure is too long or if the exposures are repeated over a long period. The eye, in Figure 16.3a, is an optical instrument which receives and focuses the light beams on the retina. The importance of the injuries and damages depend on the transmission and absorption characteristics of the tissues as well as the capacities to recover and regenerate from the lesions. We must emphasise that the fovea is the most important area of the retina since it senses the images for the clearest vision.
Figure 16.3 a) Transversal view of human eye and b) spectral absorption of different human media taken separately in human being [Sliney, 1980]
According to the International Electrotechnical Commission (IEC), 1550 nm wavelength is part of IRB. Pathologies associated with this spectral band are: •
On the ocular level: inflammation of aqueous humor, cataract and cornea burn.
INDEED: High Rate Infrared Communications
•
213
On the skin level: burns.
The grades relative to safety laser products are given by standards [CEI 825-1, 2000] and [CEI 825-1/All, 2000].
Figure 16.4 Characteristics of absorption and transmission of light in human eye: a) in visible and in the IRA, b) in the IR B and C, so that the UVB
For infrared beams in the B band, either the aqueous humor absorbs more than the cornea, or the cornea aborption is widely dominating (from 70% to 100%, in function of wavelength, as indicated by *• in Figure 16.3b). In Figure 16.4a, the light is focused on the retina whereas, in Figure 16.4b, light is absorbed by the cornea and by the lens. A retina burn can involve partial or total loss of vision, temporarily or in a definitive way. A lesion of the lens leads to cataract. Laser transmitters of UVC and far IRC radiation are dangerous for the cornea while visible and near infrared radiation are transmitted to the retina, the cornea protecting the lens against exposures superior to 4 W/cm2 [Pitts, 1980]. Lasers which can operate in free propagation without danger for users' eyes, must hence transmit in the spectral bands in which the absorption is very weak. The most appropriate spectral region lies around 1552 nm since it is covered by many commercially available optical products (fiber optics technology); the possibility of establishing direct link between a local network and cornea light absorption looks low enough in its region. At 1550 nm, the first ocular media crossed by light beam is the cornea, the absorption being not maximal (75%); it is spread over its entire thickness. We can note that if absorption was of 100% (that is, for a wavelength of 2 nm or upper than 2.5 nm), the aborption length would be weaker and therefore the dissipated energy per unit of volume would be more important, hence leading to more important damage. Power transmitted to aqueous humor is therefore weak and its absorption of 25 % has little influence. The cornea has a resistance to radiation that is equal to the skin resistance and can regenerate itself very quickly. In an extreme case, a cornea graft can still be possible.
214
Communicating with Smart Objects
The Maximum Permitted Exposure levels a 1550 nm on a subject taken from references [CEI 825-1, 2000] and [CEI 825-1 /All, ,, 2000] are: • •
1000 W.m"2 at cornea level. From 100 W.m-2 to 1000 W.m-2 at skin level, MPE vary inversely for the surface exposed, from 0.1 m2 to 0.01 m2 respectively. For a larger surface, greater than 0.1 m2, the limit 100 W.m"2 applies.
A transmitter can be considered to be accessible under maximum security conditions when its light output power does not exceed 10 mW. It is then considered a "class I" light source. Its radiometric characteristics and measurement conditions are: • • •
a 50 mm aperture diameter, a radiation pattern inside a 0.01 steradian solid angle, at a distance of 100 mm.
The calculated values given above are given for specific conditions: • •
time of exposure greater than 10s, observation aligned with the source axis.
We can conclude this normative overview by giving a definition of Nominal Ocular Hazard Distance (NOHD), in which illumination or energy exposure and MPE are equal on the cornea level. Beyond this distance, the risk is nil. In conclusion NF EN 60825-7, June 2000 (1st version), rewritten by European Committee of Electrotechnical Normalisation (CENELEC) about CEI 825-7 reference of IEC purposes sources of wavelengths greater than 780 nm (IR) used in free space data transmissions. This document specifies higher elements given by references [CEI 825-1, 2000; CEI 825-1 /All, 2000] and gives more details about essential limit values from which we establish one INDEED system specification.
4. The INDEED solution The INDEED solution is a cellular-type hybrid line-of-sight link (Figure 16.1, (D).
4.1. Downstream transmitter The downstream light sources are 1550 nm Fabry-Perot cavity semiconductor laser diodes (-FP-LDs). Each cell transmitter contains three diodes and one-diode optical output power is 8 mW. Their essential characteristics and specifications are:
INDEED: High Rate Infrared Communications
• • •
215
Several longitudinal modes around 1550 nm and a low degree of coherence to avoid speckle problems at the output of optical multimode fibre divergent adaptors. The resonance frequency must be higher than 1 GHz to avoid excess relative intensity noise and limits of the modulation frequency. The emitting area is approximately 1 to 2 um2 and its radiation pattern opens up to 40°.
4.2. Receiver The photo-detector is a semiconductor PIN-hetero-j unction photodiode. The semiconductor is a III-V compound: InGaAs/InP. The active area is 0.2 mm2 (Figure 16.5).
Figure 16.5 The InGaAs/InP photodiode
The OOK modulated signal, coded in NRZ (No Return to Zero), is then amplified and processed to obtain a minimum detectable power (or sensitivity) of -40 dBm (100 nW) at 155 Mbit/s at 10'9 BER. Typically, a concentrator [Kahn, 1997; Street, 1997] is used, with a 1550 nm selective filter. The presence of a lens (light energy concentrator) justifies itself especially in directive systems where the concentration bonus is inversely proportional to numerical aperture of receiver (FOV-Rx) [Street, 1997]. As diffuse systems, the gain in power can however reach 3 dB with sources approaching 900 nm [Kahn, 1997].
216
Communicating with Smart Objects
Compatibility of INDEED point-to-point system Power/Number Tx AN Type of source Number FOV Receiver surface AN Surface Rx/FOV Sensibility Rx, BER=10"9 Cell area Concentrator Gain Active area FOV Rate Propagation loss Penalty margin
ATM ATM 25.6 Mbit/s 140 Mbit/s + 15dBm/6 0.4 FPLD 1/Tx 40° 10 mm2 0.12 0.2 mm2/60° -44dBm -41dBm 3.5m 2 Yes + 17dB 10mm 2 13° 32 Mbit/s 155 Mbit/s -52dB +7dB +4dB
Figure 16.6 The transmitter-receiver LOS hybrid INDEED point to point link
The INDEED photo-receiver unit is very different from the devices tailored (custom-designed) for optical fiber applications: •
The photodiode is a "large sensitive area" component to collect the highest number of photons. Therefore, the parasitic "function+metallisation" capacitance is much higher. The structure of the low signal-low noise transimpedance amplifiers need to be revised.
INDEED: High Rate Infrared Communications
217
The photodiode is exposed to ambient radiation and must be protected and shielded. The noise sources are due to: - Shot noise, noise due to random nature of light emission and thus detection, associated with exit current at darkness (photons generation and recombination), - Scintillation noise due to random movements of junction mobile charges, - Detector thermal noise (Background Limited Infrared Photodetector BLIP, detector very sensitive to IR and the obscurity noise which is essentially due to photons issued from ambient radiation), - Surface effects noise, as the consequence of capture and release of mobile charges. 5. Perspectives and conclusion Adaptation, Figure 16.7, an holographic diffuser at reception level should allow an homogeneous diffusion spot on the receiver with bigger dimension and with the same shape as the detector. Besides, this diffuser, followed by a reflecting mirror, before the detector, will allow a certain mobility of receiver, within the retrieval of diffusion spot in the detector active area.
Figure 16.7 Adaptable optics of the receiver
This holographic technology has been explored for some years [Eardley, 1996], before relapsing into a certain anonymity. A system with a holographic diffuser and reflecting mirror is also proposed by Jivkova, from the University of Pennsylvania (United States) [Jivkova, 2001], but the source emits at 850 nm.
218
Communicating with Smart Objects
Integration of the receiver is to follow, the present system being not thought of as a unit so early in its conception. Many devices have been tested, but toward wavelengths close to 900 nm [Kahn, 1997; Street, 1997]. It is still possible to think again about the adaptability of some of these solutions, without neglecting research more orientated toward advanced technologies. 6. References [Bourgart, 2000] F. Bourgart & G. Ramel, ATMOSFEERIC-2 Extensions du domaine d'emploi d'ATMOSFEERIC aux installations collectives et aux services domotiques, Tech. Report., FT R&D, 2000. [CEI 825-1, 2000] NF EN 60825-1, Securite des appareils a laser: partie 1, classification des materiels, prescriptions et guide de l 'utilisateur, 2000. [CEI 825-1 /All, 2000] NF EN 60825-1/A11, Amendement a la norme NF EN 60825-1 sur la securite des appareils a laser, 2000. [Eardley, 1996] P.L. Eardley, D.R. Wisely, D Wood & P. McKee, Holograms for Optical Wireless LANs, IEE-Proc.-Optoelectron., volume 143, number 6, 1996, pp. 365-369. [Heatly, 1998] DJ.T. Heatly, D.R. Wisely, I. Neild & P. Cochrane, Optical Wireless: The Story So Far, IEEE Communications Magazine, 1998, pp. 72-82. [Jivkova, 2001] S. Jivkova & M. Kavehrad, Receiver Designs and Channel Characterization for Multi-Spot High-Bit-Rate Wireless Infrared Communications, IEEE Transactions On Communications, volume 49, number 12, 2001, pp. 21452153. [Jungnickel, 1998] V. Jungnickel, C.V. Helmot, & U. Kriiger, WireLan: A Broadband Wireless IR Lan Architecture Compatible With The Ethernet Protocol, ECOC'98 (Madrid, Spain), 1998, pp. 367-368. [Kahn, 1997] J.M. Kahn & J.R. Barry, Wireless Infrared Communications, Proceedings of the IEEE, volume 85, number 2, 1997, pp. 265-298. [Pitts, 1980] D.G. Pitts & al, Determination of ocular threshold levels for infrared radiation caractogenesis, US Dept. Health and Human Sciences, National Institute for Occupational Safety and Health Publications, 1980, pp. 80-121. [Sliney, 1980] D. Sliney & M. Wolbarsht, Safety with laser and other optical sources, Plenum press, New York, 1980, p. 145.
INDEED: High Rate Infrared Communications
219
[Street, 1997] A.M. Street, P.N. Stavrinou, D.C. O'Brien & DJ. Edwards, Indoor Optical Wireless Systems - a review, Optical and Quantum Electronics, volume 29, 1997, pp. 349-378. [Williams, 2000] S. Williams, IrDA: Past, Present and Future, IEEE Personnal Communications, volume 7, number 1, 2000, pp. 11-19. [Wisely, 1997] D. Wisely & I. Neild, A 100 Mbit/s Tracked Optical Wireless Telepoint, Proceedings IEEE: 8th Symposium Personal Indoor And Radio Communications (Helsinki, Finland), 1997, pp. 964-968.
7. Glossary ATM: Asynchronous Transfer Mode, switching technique between cells used to accommodate many different traffics in data transmission. Beam collimation: beams are collimated when they propagate parallel along the same axis toward the same 'target'. BER: Bit Error Rate for example 10-9 corresponds to one wrong bit among 109 bits. Cornea: transparent thin skin which covers the aqueous humor in front of the iris and the lens and sealed to the sclera. CSMA/CD: Carrier Sense Multiple Access with Collision Detection. Divergence (beam): Cross section increase of an electromagnetic beam vs the distance to the source. ENSSAT: Ecole Nationale Superieure de Sciences Appliquees et de Technologic. ETSI: European Telecommunication Standards Institute. Hiperlan: High PERformances Radio Local Area Network. IEEE (Institute of Electrical and Electronic Engineers): International industrial organisation in the field of electrical and electronics components and systems whose one of the main tasks is the definition and the publication of standards. IrDA (Infrared Data Association): created in 1993 to coordinate infrared optical transmissions of data. LAUBS: Association of research laboratories of Southern Brittany University.
220
Communicating with Smart Objects
Lens: transparent, biconvex, nearly spherical lens which focuses the images through the pupil on the retina. LPIO: Laboratory of Insulator Physics and Optronics. LPUB/OCP: Physics Laboratory of Burgundy University / Near Field Optics. MAC: the Media Access Control is a protocol which manages the stations connected to and sharing the same network/resources and gives them the right to transmit. Mode (electromagnetism): one solution of Maxwell equations which represents an electromagnetic field in a given space and belonging to a set of independent solutions defined by given boundary conditions. Multimode source: source of electromagnetic waves propagating in several modes. NRZ: Non-Return to Zero, simplest coding/modulation digital scheme one can design from pure binary concept. Numerical Aperture (N.A.) of an instrument: N.A. = n0.sin©0 where n0 and 00 are respectively the refractive index of the incident beam medium and the instrument acceptance angle in the same medium. OSI model (Open Systems Interconnect): the seven layer model of an open system. Photo-detector: any device which responds electrically to an optical illumination. Photodiodes: usually semiconductor p/n-junction devices used under a reverse bias. PIN photodiode: a thick intrinsic semiconductor region has been inserted between the p- and the n-regions of a junction to improve the photon absorption and therefore its quantum efficiency. Retina: sensitive area of the internal ocular sphere which receives the images, set of sensitive cells which convert the image local light energy into chemical energy transmitted along the optical nerves to the brain. WLAN: Wireless Local Area Network. WPAN: Wireless Personal Area Network.
Chapter 17
Artificial Materials for Protected Communications Frederique de Fornel1, Rabia Moussa1, Laurent Salomon1, Christian Boisrobert2, Herve Sizun3 and Philippe Guignard4 'OCP/LPUB Dijon, 2LPIO Nantes, 3FTR&D Belfort, 4France Telecom R&D, Lannion, France
1. Introduction Infra-red technologies in open space constitute an interesting solution for communicating objects and in particular for indoor communications: the introduction of optics gives rise to the solution to the problem related to high data transmission. Its use in open space contributes, by mean its flexibility, to the quality of the service offered [Prunnot 02]. As for other types of information transfer, it is essential to ensure the transmission confidentiality of the exchanges, which is more difficult due to the "not guided" nature of these communications. The securisation has in fact two different goals: 1. Guarantee of the confidentiality of the transmitted information. 2. Minimisation of the disturbances brought to the external environment of the room within which the exchanges of information take place. Before giving the physical details of the problem, the question of the confidentiality level must be clarified. Thus, the different kinds of applications dictate the lower or higher level of confidentiality. A typical case where the confidentiality must be very high is for banking data transmission. Of course one will have to find a compromise between the desired level of confidentiality and the cost of the corresponding technical solutions. We have not yet completely defined the criteria of confidentiality; this will constitute one of the objectives of this study, namely: for an given establishment receiving-transmitter, which are the parameters to be respected for the materials constituting the walls and the glazing so that the intensity transmitted outside the room will be lower than a threshold level?. This threshold level will correspond to our criterion of confidentiality. So, let us consider a simple case to illustrate these remarks.
222
Communicating with Smart Objects
2. Description of a standard establishment If a standard room is considered, the infra-red radiation is stopped (reflected or absorbed) by the walls. On the other hand, a part of this radiation can escape through the openings: glazing, open doors. Numerical simulations make it possible to study the light propagation in the room [Sizun 2002]. We wish to indicate some orientations of search that allow us to find solutions to limit or prohibit the escape of the photonic component of the communication through the windows. In particular, our work concerns the choice of the materials which constitute the windows and the wall lining.
Figure 17.1 A diagram of a room showing the path of a incident wave which is reflected or transmitted through obstacles it meets
The securisation of the openings can be made in various ways, principally while cutting the communication at the time of the opening of the doors: it is an effective and radical means but which we will not retain for this study. The other method consists in determining the optical properties of the walls and the glazing which will minimise the stray reflections which would result in an escape of light through openings when the doors are open. The confidentiality level can be affected by the furniture arrangement in the room. In our simulation, we will suppose that confidentiality is assured by the materials consisting the walls. The securisation of the glazing is to be taken into account. Indeed, the glazed surface of the room can represent a surface higher than that of one of the walls. Under these conditions, it is necessary to find glazing which have the property to let pass all the light of the visible spectral, in order to ensure first visual comfort and second to reflect the signals of the wavelength chosen for the communications, here: 1,5 5 urn.
Artificial Materials for Protected Communications
223
3. Artificial materials In our study, we will call 'artificial materials' the structures which have particular optical properties (absorption, reflex ion, transmission) that a natural material could not have. These properties are due at the same time to the properties of the materials which constitute the structure and the geometrical parameters of these [Yablonovitch 87, John 87, Yeh 94, Joannopoulos 95]. Thus, while structuring the matter, one manages to create specific and particular optical properties, these properties not being able to be obtained for a homogeneous material. Namely, we will consider a new family of artificial materials: photonic crystals. The photonic crystals are periodic structures with a periodicity of one, two or three dimensions [Yablonovitch 87, John 87, Yeh 94, Joannopoulos 95].
ID
2D
3D
Figure 17.2 Examples of artificial structures of photonic crystal, one, two and three dimensions
Since their appearance, photonic crystals have seen a number of applications growing in an extremely strong way, in very varied directions. The field of application of photonic crystals opens new prospects, namely for example the inhibition of spontaneous emission, the control of electron-hole recombination in the materials [Yablonovitch 91, Ozbay 95, Russel 95, Gerard 02]. The applications of photonic crystals exist now in the field of microwaves or millimeter-wavelength as well as in the optical region. One can quote: perfectly reflective mirrors, filters, polarisers, modulators, substrates for millimeter wavelength antennas, switches, high efficiency light emitting diodes LED and finally lasers without threshold. These enthralling applications open a new prospect in all the branches of physics such as atomic physics, optoelectronics, telecommunications and optical interconnections. Moreover, the nonlinear effects in such photonic crystals contribute to the development of powerful machines such as the optical computer. Thus, electronic switches and transistors will be replaced in the more or less long term by their optical parents. Motivated by these significant applications, technologists for their part provide an enormous efforts in order to create other structures which meet future needs. For
224
Communicating with Smart Objects
example the Yablonovite [Yablonovitch 91] was fabricated to operate in the millimeter-wavelength, structure "layer by layer" [Ozbay 95], two-dimensional structures of parallel cylinders etc The manufacture of photonic crystal operating in the optical region proves to be a real challenge which currently puts into competition different laboratories at international level such as MIT, Corning, the University of Southampton, the University of Glasgow, and in France: Alcatel, the LETI, the LEOM, the IEF, etc [Russel 95, Gerard 02]. 4. Application of artificial materials to the confidentiality of the indoor transmission Based on photonic crystals, such structures can conceive perfect reflectors for certain wavelengths, whereas, for the other wavelengths, the structure is completely transparent. For these confidentiality reasons we have to choose a structure which will fit the problem for indoor communications. The ideal structure would be the three dimensional structure. However, the technological realisation of such a structure applied to the glazing seems to us premature. We therefore proceeded by steps, the first consisting of the study of one dimensional photonic crystals. In the next, we had to include more dimensionality (i.e. two or three dimensions) and its intrinsic and geometrical parameters to the required confidentiality criteria. Figure 17.3 shows the first structure studied.
Figure 17.3 Schema of the one-dimensional photonic crystal, consisting of alternating layers of index «/ and n2
The optical transmission curves through two identical structures but of different material geometries are shown on the following figures. In Figure 17.4, a normal incidence of the beam with respect to the periodic structure also is considered, the refractive indices of each system being respectively: (1,5/1) and (1,7/1,4). We notice that all the visible region is transmitted and reflect perfectly the signal at l,55um. The transmission can reach in this case -90 dB.
Artificial Materials for Protected Communications
Figure 17.4 Transmission curves versus the incident wavelength of the two different n,/n2: (1,5/1) and(1,7/1,4)
225
systems
Unfortunately, for this one dimensional structure the answer depends and is strongly affected by the change of the angle of incidence. Indeed, when the angle exceeds 30° our structure as illustrated in Figure 17.4 deviates from its principal goal which is reflecting the incident wave at 1.55um (see Figure 17.5). Thus, a shift of the photonic band gap to the higher wavelength and a decrease in its magnitude are simultaneously obtained.
Figure 17.5 Transmission curve versus the incident wavelength for the second system (n//n2 ) - (1,7/1,4) for different angles
226
Communicating with Smart Objects
For the application aimed in our case, if one limits oneself to an attenuation of the transmitted signal of -40dB, the solution consists in using at least two sets of multilayer systems, which cover higher numerical aperture. If one requires a rejection rate higher than -40dB, it is necessary either to optimise the multi-layer systems parameters (i.e. the refractive index, the ratio r/a and the lattice constant a) or to consider a photonic crystal of two or three dimensions.
5. Discussion and prospects To secure the indoor communications, a certain level of confidentiality has to be guaranteed. This will force us to minimise at the maximum the transfer of the information outside the room where the communications take place. We showed starting from a simple example that the use of structured materials could guarantee the confidentiality. The artificial materials structured with one dimension already make it possible to have very good transmission of the visible region and a very high reflection at l,55um. The transmission is therefore lower than -70dB for angles of incidence lower than 20°. Now, if the angle of incidence exceeds 30° the properties of the structure do not meet our aim any more. To have a rejector filter at l,55u.m whatever the angle of incidence, it is possible to use more complex structures, initially a combination of multi-layer systems of different optical and geometrical characteristics. Another alternative consists in using structured materials according to two or three dimensions. Photonic crystals with three dimensions make it possible to carry out perfect reflectors whatever the incident angle of the light. In parallel with the theoretical studies, it is necessary to consider the realisation criteria of such structures. Some of these criteria are the manufacturing cost of the definite structures which will be taken into account in the final choice of the solution. It is evident that one dimensional structures were cheaper and easier to fabricate than their counterpart in two dimensions. It is also necessary to take into account the integration of these structured materials in buildings that already exist or are to be built. Thus, we have to predict such structures which can be constructed during the manufacture of the glazing or to be added to existing glazing. Furthermore, for the choice of the materials used it will be necessary to take into account the reliability of the structure (reliability with respect to thermal variations, pollution, and cleaning product etc). Until now, and in the preceding discussions we have not taken into account the nature of the connection (the source, the detector, the direct or diffuse transmission link etc). Thus, to completely define the confidentiality level of the system, we should integrate in our simulations the radiation diagram of the source. For a given structure, the effective confidentiality rate will depend on the nature of the link: direct, or diffuse.
Artificial Materials for Protected Communications
227
6. References [Gerard 02] D. Gerard, L. Berguiga, F. de Fornel, L. Salomon, C. Seassal, X. Letartre, P. Rojo-Romeo; P. Viktorovitch: "Near-field probing of active photoniccrystal structures", Optics Letters, 27, 3, 173, 2002. [Joannopoulos 95] J. D. Joannopoulos, R. D. Meade, and J. N. Winn, Photonic crystals: Modelling the Flow of Light, Princeton University Press (1995). [John 87] S. John, "Strong localization of photons in certain disordered dielectric superlattices", Phys. Rev. Lett. 58, 2486 (1987). [Ozbay 95] E. Ozbay, G. Tuttle, J. S. Me Calmont, M. M. Sigalas, R. Biswas, C. M. Soukoulis, and K. M. Ho, "Laser micromachined millimeter wavesphotonic bandgap cavity structure", Appl. Phys. Lett, 67, 1969 (1995). [Prunnot 02] J.-C. Prunnot, G. Normand, C. Boirobert; A. Mihaescu, P. Besnard, P. Pellat-Finet; F. de Fornel; F. Bourgart, P. Guignard "Les Objets Communicants", Chapitre 16: INDEED Communication infrarouge haut debit dans le contexte 'indoor', Hermes (2002). [Russell 95] P. St. J. Russell, T. A. Birks, F. D. L. Lucas, "Confined Electrons and Photons: New Physics and Applications", E. Burstein and C. Weisbuch, Eds. Plenum, New York (1995). [Sizun 02] H. Sizun: "Les Objets Communicants", Chapitre 14: La propagation des ondes radioelectriques a 1'interieur et a 1'exterieur des bailments, Hermes (2002). [Yablonovitch 87] E. Yablonovitch, "Inhibited Spontaneous Emission in Solid-State Physics and Electronics", Phys. Rev. Lett. 58, 2059 (1987). [Yablonovitch 91] E. Yablonovitch, T. J. Gmitter, and K. M. Leung, "Photonic band structure: The face-centered-cubic case employing nonspherical atoms", Phys. Rev. Lett. 67, 2295 (1991). [Yeh 94] C. Yeh, "Applied Photonics", Academic Press, San Diego, CA, (1994). 7. Acknowledgements This work is undertaken within the framework of the contract France Telecom R&D "INDEED", reference 01 IB 397. One of the authors R.M. was given financing by the "la region de Bourgogne" within the framework of a post doctoral grant.
This page intentionally left blank
Chapter 18
Free-space Optical Communication Links Olivier Bouchet and Herve Sizun France Telecom R&D, Cesson-Sevigne and France Telecom R&D, Belfort, France
1. Introduction Today the wireless numerical data transmission world market is based primarily on Hertzian type technologies. All the futurologies of large industrialised countries show some limit to this technology. They will not be able to absorb the growing development of the requirements in new transmission channels for cordless phone, networks data-processing, high definition television, communicating objects etc. Several techniques are available: microwaves, links at frequencies higher than 100 GHz and free-space optical communication links. This last technique uses in free-space the modulation of an infrared laser beam to exchange binary data full duplex by the intermediary of a Transmitter/Receiver couple. Several factors determine the revival of this technique: absence of regulation, free licencing, easy, fast and inexpensive deployment, high data rates. In this contribution we shall treat systems and propagation aspects, including restrictive effects. The aspect system relates to apparatuses laser safety, results of a survey of manufacturers as well as an analysis of their uses. The performance evaluation of such links passes by a knowledge of the atmospheric effects on the propagation in the used frequency spectrum. Fog, rain, and snow in particular constitute very restrictive elements in their use.
2. System aspects 2.1. Apparatuses laser safety Any laser can present danger to man, at ocular level and at cutaneous level; the human eye being very sensitive to infrared radiation. The most important factors to take into account to evaluate the risks are: the signal wavelength, power and beam
230
Communicating with Smart Objects
form. Eye safety is governed by International Electrotechnical Commission (IEC) standards. Those in force in France are: NF EN 60825-1 and NF EN 60825-2. The lasers are divided in various classes according to the known risks which they represent, allowable exposure limit (AEL), allowable maximal exposition (AME) and nominal distances of ocular risk (NDOR).
2.2. Manufacturers and products A review manufacturers was carried out, based on various sources of information (conferences, reviews, magazines, research engines, web sites etc) in order to obtain the most complete possible panorama of products and their capabilities. The principal parameters considered are transmitted data type, range, data rate and the type of suggested application or other parameters such as the wavelength, type and number of optical transmitters (laser and possibly LED), emitted power, control, deployment facilities, maintenance, cost etc. Table 18.1 gives an overall picture of most of the products listed starting with information obtained by manufacturers or by consulting their web site. Figures 18.1 and 18.2 (at the end of the chapter) give the data rate suggested (Mbits/s) and range in function of the wavelength for various equipment.
Free-space Optical Communication Links
231
Table 18.1 Manufacturers of free-space optical products Manufacturers
City
Country
Internet site
ACT1POLE AirFiber Inc Airlinx
Le Bouscat
France
www.laser-com.com
San Diego (CA)
USA
www.airfiber.com www.laseroptronics.com
Astroterra
San Diego (CA)
Canada USA UK
www.cablefree.co.uk
Amsterdam Munster
Holland Germany
www. canon-europa. com
Eagle Optoelectronics Firlan
Boulder
USA
Ottawa
Canada
www.firlan.com
FSONA
Ricmond B.C. Dreieich
Canada Germany
www.fsona.com
GoC
Infrared Com. Syst. JOLT
Edmonds Jerusalem
LASERBIT
CABLEFREE GUN CBL
www. astroterra.com
www.cbl.de
USA
www.goc.de www.laserinfraredwireless.com
Budapest
Israel Hungary
www.jolt.co.il www.laserbitcommunications.com
USA
www.powertechnology.com
Laser Syst Diode.
Hawthone (Ca)
USA
www.ldsc.net
LIGHTPOINTE
Boulder
USA
www.lightpointecom.com
LSA
Exton
USA
Lucent Technologies MDS OPTEL
Plessis Robinson
France
www.lsainc.com www.lucent.com
Taluyers Hamburg
France Germany France
www.mds.fr www.optel.de www.opticalaccess.com
Israel England
www.oraccess.com
Canada
www.plaintree.com
Canada USA
www.silcomtech.com
Laser Coin. Syst.
OPTICAL ACCESS
GIF/Yvette
OrAccess
New Indus. Zone
PAY Dated Systems
Cumbria Nepean
Plaintree Systems SILCOM TeraBeam Networks
Ontario Seattle
www.pavdata.com
www.terabeam.com
2.3. Uses In this paragraph the various uses suggested by the manufacturers are mentioned. In an artificial way, they were classified in three types of approaches: geographical, application and technical.
232
Communicating with Smart Objects
2.3.1. Geographical approach Initially, an approach related to geographical specificities, is presented below, including: • • • • •
Isolated sites (mountainous sites, places of interest) or risks (lightning, IEM), Charged electromagnetic environments (airports, factories, etc), Classified sites, private domains, Not easily passable obstacles (motorways, rivers, etc), Dense urban zones.
2.3.2. Applications approach A second approach relates to the applications aspect of a free-space optical communication link. •
• • • •
temporary link, - event action link (sport, concert, etc), - intervention link (work, modification or network repair, etc), - help link (accident, fast commercial action, etc), "last mile" connection, inter-site connection, independent, rented or private networks, closing an optical loop, inter-cell GSM or UMTS link (micro or picocell), etc.
2.3.3. Interface approach This approach relates to the various types of interfaces available with a laser link. Products with interfaces of any types in the various potential fields of application are proposed by the majority of manufacturers: • • • •
LAN Networks (Ethernet, Fast Ethernet, GigaEthernet, FFFDI, Token Ring, etc), Telecom (El, ...E3,T1 ...T3, ...ATM, SONET, SDH, PDH, etc), Video (SDI, SMPTE, etc), Mixed LAN-Telecom (Ethernet lOBaseT and 2xEl).
3. Propagation aspects The various types of attenuation to be considered when deploying a free-space optical communication network are geometrical attenuation, atmospheric attenuation, scintillation and optical mispointing.
Free-space Optical Communication Links
233
3.1. Geometrical attenuation The beam emitted by the transmitter being diverging (1-3 mrad), the receiving cell will collect only some of the emitted energy. The following relation gives the geometrical attenuation:
where: Scapnin.-' Receiver capture surface (0,005 m2, 0,025 m2 for example), 0: Beam divergence, d: Transmitter-receiver distance.
3.2. Atmospheric attenuation Atmospheric attenuation results from an additive effect of absorption and dispersion of the infrared light by aerosols and gas molecules present in the atmosphere. Transmittance in function of the distance is given by BEER relation:
where: T(d): Transmittance at distance d of the transmitter, P(d): Power of the signal at a distance d of the transmitter, P(0): Emitted power, a: Specific attenuation or extinction coefficient per unit of length. Attenuation is connected to transmittance by the following expression:
The extinction coefficient a is the sum of four terms:
where: am is the molecular absorption coefficient (N2, O2, H2, HO, CO2, O3,..),
T
an is the absorption coefficient by the aerosols (small solid or liquid particles present in the atmosphere (ice, dust, smoke etc) (3m is the Rayleigh scattering coefficient resulting from the interaction of the wave with particles of size smaller than the wavelength, (3N is the Mie scattering coefficient. It appears when particles are of the same order of magnitude as the transmitted wavelength. Absorption dominates in the infrared while scattering dominates in the visible and ultraviolet band. Being given the low values of molecular and aerosol absorption coefficients as well as Rayleigh scattering coefficient, the extinction coefficient can be written by the following relation:
where: V is the visibility in km X is the wavelength (nm) Q is the size distribution of the diffusing particles = 1,6 for strong visibility (V>50 km), = 1,3 for average visibility (6
3.3. Cloud attenuation When the air cools beyond its saturation point, the water vapor condenses to form water droplets or ice crystal if the temperature is very low. Generally, the water particles thus formed have small sizes (<100 urn), but their concentration can be important (a few hundreds per cm3). The existence of the clouds is strongly related to the climate of the area considered. Cold moderate areas generally have a minimal cloud cover in summer whereas continuous rainfall is maximum. In the Mediterranean, the reverse is observed. Presence of clouds is characterised by a nebulosity index which indicates the fraction of the sky covered and it is expressed in l/10ths of percentage. The effects of multiple diffusion are important when a light beam crosses the cloud. The diffusion, in addition, involves a time and frequency scattering as well as a depolarisation.
Free-space Optical Communication Links
235
3.4. Fog and haze attenuation Fog and haze consist of very small water droplets (<100 urn) suspended in the air. It is formed by a process of condensation close to the ground following radiative cooling of the ground (especially at night) or to air movement on cold ground (advection fog or haze). To distinguish haze from fog, it is generally admitted that the visibility is higher than 1000 m in the presence of haze while it is lower than 1000 m in the presence of fog. An excellent correlation was found between attenuation due to fog and liquid water concentration. The following relations for the attenuation coefficient were obtained at 0,63 urn and 10,6 urn [VASSEUR, 1997]:
where W is the liquid water concentration (g/m3). The liquid water concentration in the fog is typically equal to approximately 0,05 g/m3 for a moderate fog (visibility of about 300 m) and to 0,5 g/m3 for a thick fog (visibility of about 50 m) [UIT-R P. 840-3]. Weather measurements taken in Belgium in 16 stations during about twenty years provide indications on its maximum (worse case) and median (not exceeded in 50% of the cases) annual frequency. Table 18.2 gives the fog annual frequency [BODEUX, 1977]. Table 18.2 Annual frequency of fog
Light, moderate or thick fog (V<1000 m) Moderate or thick fog (V<500 m) Thick fog (V<200 m)
Median frequency 0,055% 0,035% 0,020%
Maximum frequency 0,14% 0,11 % 0,08 %
3.5. Attenuation due to precipitation The rain is formed from the water vapor contained in the atmosphere. It consists of water drops whose form and number are variable in time and space. Attenuation due to rain is due primarily to the scattering phenomenon as in the case of aerosols. In infrared the wavelength is very much smaller than the raindrops' diameters. The value of the standardised cross section, Qd remains equal to 2 whatever the wavelength.
236
Communicating with Smart Objects
The expression of the rain scattering coefficient is:
where R and dN(r) are respectively given in cm and in number/cm 4 . When the size of the irregularities due to precipitations becomes important compared to the wavelength, the wave is attenuated by reflection and refraction. Attenuation, independent of the wavelength, is a function of the precipitation intensity R (in mm/h) according to the following relation: Attenuation (dB/km) = 0,365R0,63 where R is the rain intensity measured in mm/hour. Rain intensity is the fundamental parameter being used to locally describe the rain. Its measurement is carried out either directly on the ground by means of pluviometers or comparable apparatuses or in an indirect way by means of weather radar. The latter is particularly well adapted to the analysis of the rain structure. The relation given by Carbonneau et al.. [CARBONNEAU, 1998] is slightly different, attenuation values being much higher: Attenuation = 1.076 R0.67 dB/km
3.6. Other hydrometeors (snow, hail, etc) Many other hydrometeors are present in nature. They are mixtures of ice, air and/or liquid water. Among these mixtures, snow is one of most complex. Snow generally falls in the form of flakes which are crystal aggregates of ice. The flakes can reach diameters of 15 mm. Although they do not have a defined form, one compares them to spheres and one classifies them by their water content after fusion. Attenuation due to snow is strongly related to its humidity or density. Analysis of the experimental data [VASSEUR, 1997] makes it possible to release the following results: • • •
For low humidity snow fall (dry snow), attenuation can reach 20 dB/km in the visible and 40 dB/km in infrared. For wet snow falls, attenuation varies from 4 to 8 dB/km as well in the visible as in infrared. Hail can be regarded as ice containing air bubbles. They are the largest observable hydrometeors (up to 8 cm in diameter).
Free-space Optical Communication Links
237
3.7. Refraction and scintillation Under the influence of thermal turbulence within the propagation medium the formation of random, variable size (10 cm-1 km) and of different temperature cells occur. These various cells have different refraction indexes thus causing scattering, multiple paths, arrival angle variations: the signal received quickly fluctuates at frequencies ranging between 0,01 and 200 Hz. The wave front varies in a similar way causing focusing and defocusing of the beam. Such signal fluctuations are called scintillations. Scintillation amplitude and frequency depend on the cell size compared to beam diameter. When heterogeneities are large compared with the beam cross section, it is deviated, when they are small, the beam is widened. The tropospheric flicker effect is generally studied starting from the logarithm of the amplitude xt^B] of the observed signal ("log-amplitude"), defined as the ratio in decibels of its instantaneous amplitude and its average value. Intensity and speed of the fluctuations (frequency of scintillations) increase with the wave frequency. For a plane wave, a weak turbulence and a specific receiver, the scintillation variance a.2 [dB2] can be expressed by the following relation:
where: k [m-1] is the number of waves (2rc/X), L [m] is the link length, Cn2[m-2/J] is the structural parameter of the refraction index representing the turbulence intensity. Scintillation peak-to-peak amplitude is equal to 4crand attenuation related to scintillation is equal to 2cr . For strong turbulence, one observes a saturation of the variance given by the relation above [BATAILLE, 1992]. One will note that Cn2 parameter does not have the same value at millimeter and optical wavelengths [VASSEUR, 1997]. Millimeter waves are especially sensitive to humidity fluctuations while in optics, refraction index is a primarily function of the temperature (the water vapor contribution is negligible). One obtains in millimeter waves a value of Cn2 of about 10-13 m-2/3 which is an average turbulence (in general in millimeter range we have 10-14< Cn2 <10-12) and in optical a value of Cn2 about 2xlO- 15 m-2/3 which is a light turbulence (in general in optics we have 10-16< Cn2 <10~ 13 ), [BATAILLE, 1992].
3.8. Optical misalignment Being given the equipment characteristic (low divergence of the laser beam), very precise alignment is necessary. The alignment of the transmitter and the receiver
238
Communicating with Smart Objects
characterises optical link coupling. This one can be disturbed following mechanical vibrations.
4. Conclusion The advantages and the disadvantages of use of a free-space optical link are summarised. Among the advantages we have quoted: high data rates (from 2 Mbps to 2500 Mbps and more), transparent protocol, interfaces diversity, absence of licence and regulation, simple, fast and reusable deployment, relatively protected transmission etc. Among the disadvantages we have quoted: no data guarantees by the manufacturers in term of availability and QoS, limited distances, line of sight technology, new technology, thus retreat and lack of information, parameters of ocular safety related to the laser class of the equipment. In the search for equipment, one will choose the weakest laser class in order to limit the number of precautions and restrictions at the moment of deployment and maintenance. It appears that the optimal establishment configuration of an equipment seems to be a short link, a high data rate and a point-to-point service. We have also presented the various aspects of the propagation to be considered at the moment of the deployment of a free-space optical links (geometrical attenuation, atmospheric effects, scintillations, beam misalignment). We saw that the weather phenomena which bring the most important attenuation are in order, fog, dry snow, rainstorms (generally short) and light rains. Fog constitutes the most important phenomenon. It is frequent in certain years and slow to be dissipated. The installation of experimental links should make it possible to show that freespace optical links can constitute a reliable alternative broadband to the optical fiber installation and lead to a better acceptance of this technology in industry than the high data rate telecommunications networks necessary, between others, for the future deployment of many communicating objects.
5. Bibliography [BATAILLE, 1992] Bataille P.; "Analyse du comportement d'un systeme de telecommunications optique fonctionnant a 0,83 micron dans la basse atmosphere", These de doctoral, Universite de Rennes 1, 1992. [BODEUX, 1997] Bodeux P.; "La frequence du brouillard en Belgique", Bruxelles: Institut Royal Meteorologique de Belgique, 1977.
Free-space Optical Communication Links
239
[CARBONNEAU,1998] Carbonneau H.T., Wiseley D.R.; "Opportunities and challenges for optical wireless; the competitive advantage of free space telecommunications links in today's crowded marketplace", SPIE conference on optical wireless communications, Boston, Massuchussetts, Vol. 3232, 1998. [KIM, 1998] Kim 1.1., Kootz J, Harkakha H., Adhikari P. Stieger R., Moursund C, Barclay Mr., Stanford A., Ruigrok R., Shuster J, Korevaar E; "Measurement of scintillation and link margin for the TerraLink communication system", SPIE vol. 3266, 1998. [LAVERGNAT, 1997] Lavergnat J., Sylvain M.; "Propagation des ondes radioelectriques, Introduction", Collection Pedagogique de Telecommunication, Masson, 1997. [UIT-R P.840] "Attenuation due to clouds and fog", Rec. UIT-R P. 839, 1999. [VASSEUR, 1997] Vasseur H., Oestges C Vander Vorst A.; "Influence de la troposphere sur les liaisons sans fil aux ondes millimetriques et optiques", Propagation electromagnetique du decametrique a Tangstrom, 3iemes journees d'etude, Rennes, 7, 8, 9 October 1997.
Page 240 shows Figure 18.1 Data rate (in Mbps) in function of the wavelength for various equipments Page 241 shows Figure 18.2 Range in function equipments
of wavelength for
various
This page intentionally left blank
Part 4 Evolution of Smart Devices Claude Kintzig France Telecom R&D, France
Before considering progress in the field of smart devices, and considering their possible characteristics, it is necessary to get a look backwards to find tracks that have led to their emergence and current development.
1. Areas of development Several areas contribute to the development of devices with processing and communication capacity. 1. In the industrial world (leaving computers to be treated later) two types of devices can be identified. First of all, relatively passive devices which return after solicitation, information of 'yes or no' type have been bestowed with more highly evolved processing and communication capacities. They can receive an order, apply a more or less sophisticated, processed reply. They unite devices that have been constructed with strong processing capacities (microprocessors), and communication capacities that make them qualified to offer high levels of functionality. "Electronic tags" are devices of the first type, and "electronic cards" without contacts correspond to devices of the second type. These are physical entities of relatively small dimension. But be careful not to attribute all smart devices to objects of small size. Indeed, billboards, that can be of great size and updated by mini-messages via a SMS function are strong smart devices. It is the same, for example, with washing machines, that detect a breakdown and call the after-sale service via the GSM terminal. In our analysis, it would be necessary not to neglect objects that can, because of their size or functionality, not be included in seminaries devoted to communicating devices, and to be thereby excluded from research. By extension, an object that has a minimum of processing and communication capacity has access to the status of small devices.
244
Communicating with Smart Objects
2. In the computer world, the miniaturisation of components on the one hand, and progress of communication protocols on the other hand, are at the root of significant progress in research and creativity. In general, the object, the slave of an object master in a first phase, detaches itself by the integration of more sophisticated communication tools. By claiming its autonomy in a second phase, it passes on a status of extension of what? To that autonomous tool. On the way, it wins some functionality that slips from the older master to the new. It is the path followed by objects such as "Palm" and other Pocket PC, pens and electronic paper (that are in a phase of development). Conversely, the world of telecommunications is going to endow mobile phones by function increasingly rich to reply to the needs of the office. It is necessary to notice in recent years a strong evolution of the fact that these objects acquire increasingly "intelligence" to take into account communication by their own resources. Protocols such as Jini or Bluetooth enter this category. This tendency is confirmed with protocols of higher level such as P2P. They get symmetrical communication between devices without passing by servers one of whose roles is to ensure this symmetry by coupling two asymmetries. 3. Without restraining considering of smart devices only as physical entities, software agents (which execute well with physical machines) constitute an area of very fertile research. These are software entities which have processing and communication capacity. In view of the suppleness of implementation, studies are mainly conducted mainly on architectural aspects and dialogue with humans or other software agents. That does not mean that they include the totality of data-processing especially classic, generic, and centralised but it concerns an available resource and is deployed according to need.
2. The future So what evolutionary trends that become apparent? What can we draw from the areas discussed above? If we accept the postulate that there can be no dialogue without reciprocal context knowledge (reference and intention) we can suppose that integration of the context (or at least of a formalisable part of the context) to the environment of smart devices will be an important research area in years to come. This tendency is already perceptible with the software entities that characterise their dialogue according to an evaluated and re-evaluated context permanently. This dialogue must going to simplify (by being careful to note that the simplification becomes more complex) both between communicating devices (progress made in quoted protocols above show it) and between them and users. Current work on t multi-modality comes within the framework of this step.
Evolution of Smart Devices
245
For the moment all of these small devices need a source of energy. The area of utilisation of these objects is going to widen according to progress in this area (pace combustible batteries). The miniaturisation of smart devices, that affects the physical connection of the different functionalities (access, interface, use, etc.), would have, through the constraints of each area, to bring elements of responses and integration at the same time. This will help with clarification of risks. For example, the dialogue interface will have to integrate the various modalities in the place function. Through the totality of these progresses, smart devices will be better and better integrated with the arts (music, cinema, dance, photography, etc.), and used in the health domain, entertainment, among others. A question remains opened: what will be the autonomy of smart devices? Will they stay under the user's control, or will these objects, through their autonomy, propose to the users non-solicited actions? Might we see some users suppress or destroy objects or devices becoming too intrusive? Might we see objects or devices at war? We manufacture smart devices, experiment with them, sell them and chuck them out all the time. They are becoming more and more ubiquitous, and more varied, so they may strike us. Give us a chance to domesticate them to improve their usefulness instead of to leaving them in the wild state, under the pretext of innovation.
This page intentionally left blank
Chapter 19
Mobile and Collaborative Augmented Reality Laurence NigayI, Philippe RenevierL , Laurence Pasqualetti3 , Pierre Salembier4 and Tony Marchand4 1
Department of Computer Science, University of Glasgow, UK, 2CLIPS-IMAG, University of Grenoble, France, 3France Telecom R&D, Issy-les-Moulineaux, France and 4GRIC-IRIT, University Paul Sabatier, Toulouse, France
1. Introduction One of the recent design goals in human computer interaction (HCI) has been to extend the sensory-motor capabilities of computer systems to combine the real and the virtual in order to assist the user in his environment. Such systems are called augmented reality (AR). This is also the objective of other innovative interaction paradigms such as ubiquitous computing, tangible bits, pervasive computing and traversable interfaces. Augmented reality (AR) has been the subject of growing interest. In [Dubois 99], we emphasised the diversity of AR systems and presented one important classification characteristic: • •
Systems that enhance interaction between the user and her/his real environment by providing additional capabilities and/or information. We call such systems augmented reality (AR) ones. Systems that make use of real objects to enhance the interaction between a user and a computer. We call such systems, augmented virtuality (AV) ones.
There are many application domains of augmented reality (AR) and augmented virtuality (AV), including construction, architecture and surgery [Dubois 99]. Examples of AR systems are the computer assisted medical intervention (CAMI) systems, also called augmented surgery. AR plays a central role in the medical domain because the key point of CAMI systems is to "augment" the physical world of the surgeon (the operating theatre, the patient, the tools etc.), by providing preoperative information including the pre-planned strategy. Information is transmitted between the real world and the computer world using different means: computer
248
Communicating with Smart Objects
screens, mouse, pedals, tracking mechanisms, robots, etc. Examples of AV systems in human-computer interaction involve input modalities based on real objects (cubes), such as the phicons (physical icons) [Ullmer 98]. Ishii has described this interaction paradigm as the tangible user interface. All these systems are all based on the manipulation of objects in the physical environment. AR and AV systems are numerous in many different application domains but the most attention has been paid to the technical issues related to image processing and data fusion. Very little effort has been applied to modelling the interaction between the user and the system. The design approach so far has been technology-driven. In our project, we adopt a complementary user-centred approach providing a usercentred design method based on scenarios. Our application domain is archaeological prospecting. In this chapter, we first present the main characteristics of archaeological prospecting. Indeed we base our study on a specific mobile fieldwork: archaeological prospecting We then explain our design approach for mobile AR systems based on field studies and on the design of scenarios of actual and expected activities. We then describe the conceived and developed interaction techniques via the MAGIC (mobile augmented reality group in context) platform. 2. Archaeological prospecting Archaeological prospecting influences whether excavation at a site will take place. It provides a global overview of the environment, including a systematic census of the archaeological clues. The goal is to check the state of the archaeological archives and the potential of the sites [Dabas 99]. The archaeological evaluation must fulfil the following requirements [Nigay 2002]: (1) establish the location of the deposit, (2) find the boundaries of the site, (3) define their nature (habitat, necropolis, etc.), (4) evaluate the density of the structures, and (5) date the site [Blouet 94]. Prospecting is done by a group of archaeologists and consists initially of a ground analysis based on a systematic division of the zone. If necessary, the archaeologists consult a specialist whose opinion will determine whether prospecting will continue. Currently, this consultation with the expert is inefficient, because it requires repeated trips to and from the site by the archaeologists. The prospecting requires long journeys between sites, where the topographic characteristics are often poorly known. The long distances of the journeys cause problems since they result in asynchronous interaction with distant specialists who possess specialised knowledge whose nature cannot be anticipated a priori. As explained in [Nigay 2002], the characteristics of the archaeological prospecting activities (e.g., the co-operative process, the nature of shared information, the type of interaction), appear to be representative of the co-operative activities found in mobile situations.
Mobile and Collaborative Augmented Reality
249
3. Design approach In the context of archaeological prospecting activities, our work focuses the harmonious fusion of the real world or physical environment with the digital world or data processing in mobile and collaborative situations. The objective is to understand the use of the mobile supports and the services awaited in collaborative situation for a user task in the real world, justifying the fusion of the two worlds, the physical and digital worlds. Two properties are the bases of our study: transparency of the interaction and ubiquity. •
•
The transparency sought in the interaction makes it possible to the user to pay her/his attention on the task to be realised and not on the use of the computer. It is therefore necessary not to separate the user from his physical environment during the use of the computer. The aim is to reduce the contrasts between the two work contexts of the user: there is on the one hand the real world and on the other hand computer equipment and digital information. Ubiquity arises from the use of the mobile supports. The user wishes to reach services and to collaborate with her/his colleagues at any time and from anywhere. The objective is thus to conceive group ware on mobile supports: the user is then no more prisoner of the workstation on her/his desk to be able to collaborate and communicate.
Figure 19.1 Design steps
250
Communicating with Smart Objects
These two properties fall under the current approach of the human computer interaction (HCI) which preaches the universality of the interfaces. The dataprocessing tool is integrated into the physical environment, accessible from everywhere and by all. To satisfy these two properties, we adopted a design approach based on scenarios [Carroll 001 presented in Figure 19.1. In addition to the recognised interests of a scenario based design approach [Jacobson 951, the fact that the scenarios are centred at the same time on the users and the various tasks makes it possible for us to determine their needs and their relationships to the physical environment. As shown in Figure 19.1, from activity scenarios, we carry out a projection of what could be the activity of the users after the introduction of new tools. These hture activities are described in projected scenarios. The formalism of the scenarios is described in [Salembier 971. Based on an empirical analysis of the real task and on an analysis of the activity, we elaborate real task and activity scenarios. They constitute the basic material for the requirement elicitation. Based on the requirements, specifications are then elaborated. Of these specifications, we define projected scenarios. Initially, the specifications are carried out without any limit, by taking account of no constraint. Then in order to be able to develop a concrete platform, we adapt the specifications to the technological limitations. The projected scenarios initially based on these new specifications (11) are used to adjust the specifications (first loop of iteration). After a phase of software design and development, we carry out experimental pre-tests with the end-users. The results may lead to modification of the initial specifications (second loop of iteration). In [Nigay 20021, we fully describe the design steps and illustrate them. In particular examples of real task and projected scenarios are provided. Having presented the design steps, we now present the outcomes, the designed and developed MAGIC platform. 4. Magic platform
MAGIC (Mobile, Augmented reality, Group Interaction, in Context) is a hardware and software platform dedicated to archaeological prospecting activities. It enables the archaeologists to perform ground analysis of the site and to communicate with other mobile archaeologists working in the site as well as with distant archaeologists. We first describe the hardware and then the software responsible for the fusion of the two worlds, the physical and digital worlds. The complete software and its architecture are detailed in [Renevier 011.
Mobile and Collaborative Augmented Reality
251
Figure 19.2 A MAGIC user
Figure 19.3 User interface of the MAGIC pen computer
The hardware platform is an assembly of commercial pieces of hardware. We use a Fujitsu Stylistic 3400 pen computer. This pen computer is a PC (processor Pentium [II 450 MHz, 196 Mb of RAM), with a tactile screen having the size of a A4 sheet of paper. Its weight is 1,5 kg. Moreover, it has a video exit allowing the dual display, to which we connect a semi-transparent Head-Mounted Display (HMD), SONY LDI D100 BE. A camera is fixed between the two screens of the HMD (between the two eyes). The hardware platform also contains a magnetometer (HMR 5000 of Honywell) which determines the orientation of the camera as well as a GPS
252
Communicating with Smart Objects
which locates the mobile user. For sharing data amongst users and communication between users, a WaveLan network by Lucent (11 Mb/s) was added (PCMCIA). Figure 19.2 shows a MAGIC user, fully equipped. Based on the above described hardware platform, we designed and developed interaction techniques that enable the users to perform the functions associated with the scenarios. Figure 19.3 presents the graphical user interface displayed on the tactile screen of the pen computer. The developed software offers several functions for communication (electronic forum and messages), for coordination (archaeologists location on the map of the archaeological site) and for production (editing tools, database of found objects). In order to smoothly combine the digital and the real, we create a gateway between the two worlds. This gateway has a representation both in the digital world on the screen of the pen computer (bottom right part window in Figure 19.3) and in the real environment, displayed on the HMD.
4.1. From the physical world to the digital world Information from the real environment is transferred to the digital world thanks to the camera carried by the user. The camera is positioned so that it corresponds to what the user is seeing, through the HMD. The real environment captured by the camera can be displayed in the gateway window on the pen computer screen as a background. Based on the gateway window, we allow the user to select or click on the real environment. The interaction technique is called "Clickable Reality". Before taking a picture, the camera must be calibrated according to the user's visual field. Using the stylus on screen, the user then specifies a rectangular zone thanks to a magic lens [Bier 93]. The specified rectangular zone corresponds to a part of the real environment. By selecting the button "take" (to take a picture), the user carries out a capture of the part of real world contained within the framework of the lens. The real world becomes clickable like digital objects. All the images thus captured are recorded in a database with their localisation thanks to the GPS.
4.2. From the digital world to the real world Information from the digital world is transferred to the real environment, via the gateway window, thanks to the HMD. For example the archaeologist can drag a drawing or a picture stored in the database to the gateway window. The picture will automatically be displayed on the HMD on top of the real environment. Moving the picture using the stylus on the screen will move the picture on top of the real environment. This is for example used by archaeologists in order to compare objects, one from the database and one just discovered in the real environment. In addition, when an archaeologist walks in the site, s/he can see discovered objects removed from the site and specified in the database by colleagues: we called this
Mobile and Collaborative Augmented Reality
253
interaction technique "Augmented Stroll". The technique consists in superimposing an image of an object in its original real context (in the real world), thanks to the semi-transparent HMD. Because a picture is stored along with the location of the object, we can restore the picture in its original real context (2D location). "Augmented Stroll" is an example of mobile and collaborative augmented reality (AR) technique. On the one hand, it is a mobile AR technique because the augmentation of the real world is based on the current position (GPS) and orientation of a user (magnetometer). On the other hand, it is an asynchronous collaborative technique: a user initially captures an object in its physical context before removing it from the site; another user can later on perceive the object in its original real context. While walking in the archaeological site, the user can observe green spots superimposed in the real world to indicate that objects are available. By selecting a green spot, the user can see the digital object in its original real context. The reuse of digital objects initially from the real world but no more physically present supplements the interaction cycle between the user, her/his environment and the computer. This mode makes it possible to follow the evolution of the archaeological site through space (the movements of the user within the site) and time (the various recorded information collected by several users). Our approach consists of augmenting the user: a MAGIC user is wearing and holding the MAGIC platform. Nevertheless, from the point of view of the user, physical objects or places are augmented. Indeed the approach adopted is to assist the user by providing extra information about the physical field via a device carried by the user. Another approach [Mackay 98] would be to augment the physical environment itself. Each time the user removes a physical object, s/he would have to place an input/output devices at the position of the object. The links between the physical and digital worlds would then have been dynamic but explicit: the user would have to perform a specific action to define the link. Our design solution enables links between the physical and digital worlds that are dynamic but implicit, because the localisation of an object is automatically computed by the system.
5. Conclusion In this paper we have focused on the fusion between the physical and digital worlds. We have explained our design method based on scenarios: real task and activity scenarios as well as projected scenarios that describe the future activity of the user with the tools to be developed. Projected scenarios are therefore used as a support for predictive evaluation during the design. We applied our design method for the design and implementation of a mobile collaborative augmented reality (AR) system dedicated to a group of archaeologists prospecting fields: the hardware and software MAGIC platform. In the MAGIC platform, the technological solutions for the fusion of the two worlds include the gateway window as well as the "clickable reality" and "augmented stroll" techniques. Although our solutions are designed and developed for a given application domain, archaeological prospecting activities, our interaction techniques are generic. One of our research avenues is to prove this generic aspect
254
Communicating with Smart Objects
of the designed techniques by developing a mobile collaborative game based on the MAGIC platform. 6. References [Dabas 99] Dabas, M., Deletang, H., Ferdiere, A., Jung, C., Haio Zimmermann, W., 1999, La Prospection. Paris: Errance. [Dubois 99] Dubois, E., Nigay, L., Troccaz, J., Chavanon, O., Carrat, L., 1999, Design Space for Augmented Surgery, an Augmented Reality Case Study, Proceedings of INTERACT'99, Sasse A. & Johnson C. Eds, IFIP IOS Press Publ., p. 353-359. [Bier 93] Bier, E. et al., 1993, Toolglass and Magic Lenses: The See-Through Interface. Anaheim, Proceedings of SIGGRAPH'93, ACM Press: New York Publ., p. 73-80. [Blouet 94] Blouet, V., 1994, Essais de comparaison de differentes methodes d'etude archeologique prealable, Les nouvelles de Parcheologie, 58, p. 17-19. [Carroll 00] Carroll, J.M, 2000, Making use. Scenario-based design of computer interactions, MIT Press, Cambridge, Massachusetts. [Jacobson 95] Jacobson, I., 1995, The Use Case Construct in Object-Oriented Software Engineering, In John M. Carroll, editor, Scenario-Based Design: Envisioning Work and Technology in System Development, John Wiley and Sons, p. 309-336. [Mackay 98] Mackay, W., Fayard, A.-L., Frobert, L., Medini, L., 1998, Reinventing the Familiar: an Augmented Reality Design Space for Air Traffic Control. Proceedings of CHI'98, ACM Press: New York Publ., p. 558-565. [Nigay 2002] Nigay, L., Salembier, P., Marchand, T., Renevier, P., Pasqualetti, L. Mobile and Collaborative Augmented Reality: A Scenario based design approach, Proceedings of Mobile HCI 02, Springer-Verlag, 2002, a paraitre. [Renevier 01] Renevier, P., Nigay L., 2001, Mobile Collaborative Augmented Reality, the Augmente Stroll, Proceedings of EHCI'2001, LNCS 2254, SpingerVerlag, p. 315-334. [Salembier 97] Salembier, P., Kahn,,J., Zorola, R. & Zouinar, M., 1997, Cognitive modeling, Deliverable WP6, RHEA Project, DG VII, EC. [Ullmer 98] Ullmer, B., Ishii, H., Glas, D., 1998, MediaBlocks: Physical containers, transports, and controls for online media, Proceedings of SIGGRAPH'98, ACM Press: New York Publ., p. 379-386.
Mobile and Collaborative Augmented Reality
255
7. Acknowledgements We wish to thank our partners from the CEA (Centre d'Etude d'Alexandrie in Egypt) for welcoming us.
This page intentionally left blank
Chapter 20
Towards a Description of Information seeking Tasks Contributing to the Design of Communications Objects and Services Andre Tricot and Caroline Golanski CNRS and Universite de Toulouse 2, France
1. Introduction useit.com, the site of one of the leading specialists in Web ergonomics, Jacob Nielsen, contains many interesting considerations concerning the Web, Wap and the development of communications objects. In brief, the success of the Internet and the Web from the mid 1990s onwards led observers to believe that it was possible to design almost anything in the field of communications objects, in particular in terms of information and document access. Fairly soon, however, it became clear that more than half of the information searches performed on the Web resulted in failure. Furthermore, Web and Wap access via communications objects such as mobile telephones or PDAs, which were initially considered to be a development or extension of the Web, have not proved themselves to be particularly usable and are employed only very infrequently. A number of hypotheses have been considered on the basis of this observation. It was thought that technical advance, in particular in terms of data rates would solve some or all of the problems relating to the development of any given tool. Other analysts suggested that users' skills would develop, as they always do, and thus solve the utilisation problem. In brief, the development of usability was expected to boost utilisation. In this chapter we intend to defend a different point of view. We may consider that, considering among other things its usability characteristics, any given communications object is specifically useful for a particular information-seeking task but not for others. A description and a categorisation of the information-seeking tasks and the establishment of a relation between this categorisation and the communications objects which do or do not permit the implementation of each of these tasks would allow designers to choose the most appropriate tools in the light of
258
Communicating with Smart Objects
the services that they want to develop. According to Tricot and Nanard [TRICOT, 1998] the description of an information task should take account of: • • •
The user's representation: the representation that users construct of the task and their level of expertise in the field in question together with their skill in using the tool or the information service; The implementation of the goal: the address and the number of targets in the system, the procedures to be used in order to access these targets, the general structure of the system and the interface; The context of the activity: the reason why a subject chooses to use a particular system in order to search for certain information (learning, document design, problem solving etc.).
We have defined four objective variables which are independent of the user or the topic in question, and which make it possible to characterise the implementation of the goal in an information-seeking task. These variables are the repetitiveness of the task, the level of explicitness of the targets, the location of the targets and the quantity of targets. We have started to analyse the hierarchical relations which may exist between these variables. This has enabled us to describe 12 informationseeking tasks and classify these tasks on the basis of the mean performances they result in during information searches on the Web. 2. Wap ergonomics Nielsen [NIELSEN, 2000] reports a study into Wap usability that was conducted in late 2000. Since Nielsen is acknowledged as the leading specialist in the field of Wap ergonomics and given that the study in question was extremely thorough, this report is our exclusive reference point. This study reveals that, in its current state, Wap utilisation is not satisfactory. During their information-seeking tasks, users are regularly confronted with a range of problems that occur at various levels and relate, for example, to connection, navigation or information retrieval. These problems are due to a large extent to the fact that Wap is still a very new technology and it is to be expected that the connection and download problems will be solved within a few years. However, the problems relating to information searches and retrieval will not be entirely resolved by technological advances. These are design problems which will have to be solved by designers and engineers. The constraints due to the size of the screen and the limitations in the way these tools can be handled (small keypad) would seem to make it clear that the data available via Wap and the navigation options that can be used to obtain this data should be specific to this type of tool. However, it would appear that this is not always the case.
T
2.1. Searching for information Wap navigation is based on the same principles as Internet navigation. The idea is that users choose options from a series of menus until they find what they are looking for. Each of the Wap networks attempts to provide all the information that is likely to be of interest to its customers. To do this, it offers all the services that users might need in the form of lists and makes these available directly via the gateway. The limited screen size makes it impossible to display all the options clearly and precisely in a single screen. These are therefore grouped into simple categories in a portal (news, entertainment, etc.). Once the required option has been found, a sublist with a new set of options is displayed. Given the large number of available options, users have to pass through many menus and submenus before they access the desired information. However, they often do not get that far after being prematurely disoriented in their search.