Self-organising Software: From Natural to Artificial Adaptation

Natural Computing Series Series Editors: G. Rozenberg Th. Bäck A.E. Eiben J.N. Kok H.P. Spaink Leiden Center for Natura...

Author: Giovanna Di Marzo Serugendo | Marie-Pierre Gleizes | Anthony Karageorgos

40 downloads 1079 Views 7MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

Natural Computing Series Series Editors: G. Rozenberg Th. Bäck A.E. Eiben J.N. Kok H.P. Spaink Leiden Center for Natural Computing

Advisory Board: S. Amari G. Brassard K.A. De Jong C.C.A.M. Gielen T. Head L. Kari L. Landweber T. Martinetz Z. Michalewicz M.C. Mozer E. Oja G. P˘aun J. Reif H. Rubin A. Salomaa M. Schoenauer H.-P. Schwefel C. Torras D. Whitley E. Winfree J.M. Zurada

For further volumes: www.springer.com/series/4190

Giovanna Di Marzo Serugendo Marie-Pierre Gleizes Anthony Karageorgos Editors

Self-organising Software From Natural to Artificial Adaptation

Editors Dr. Giovanna Di Marzo Serugendo Centre Universitaire d’Informatique Université de Genève Battelle, Bâtiment A, route de Drize 7 1227 Carouge Geneve Switzerland [email protected] Prof. Marie-Pierre Gleizes Institut de Recherche en Informatique de Toulouse (IRIT) Université Paul Sabatier route de Narbonne 118 31062 Toulouse Cedex 9 France [email protected] Series Editors: G. Rozenberg (Managing Editor) [email protected] Th. Bäck, J.N. Kok, H.P. Spaink Leiden Center for Natural Computing Leiden University Niels Bohrweg 1 2333 CA Leiden The Netherlands

Dr. Anthony Karageorgos Department of Forestry and Natural Environment Management Technological Educational Institute of Larissa Karditsa Branch, Terma Mavromihali 431 00 Karditsa Greece [email protected]

A.E. Eiben Vrije Universiteit Amsterdam Amsterdam The Netherlands

ISSN 1619-7127 Natural Computing Series ISBN 978-3-642-17347-9 e-ISBN 978-3-642-17348-6 DOI 10.1007/978-3-642-17348-6 Springer Heidelberg Dordrecht London New York Library of Congress Control Number: 2011938285 ACM Computing Classification (1998): I.2, F.1, D.1 © Springer-Verlag Berlin Heidelberg 2011 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Cover design: KünkelLopka GmbH Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

Preface

Stable and dependable IT services and infrastructures are nowadays of paramount importance not only for modern enterprises but also for home users. However, as distributed information infrastructures continue to spread and grow, resulting in Internet-based, wireless and mobile systems, traditional solutions for managing and controlling the software that sustains them appear to have reached their limits. As a result, new challenges in software engineering have arisen demanding reliable, robust and scalable software systems operating in extremely dynamic and unstable environments, able to take care of themselves with a minimum of user intervention. The main issue is that engineers of contemporary software systems and services can now only seldom rely on centralised control or management, high reliability of devices, or secure execution environments. For example, high-speed Internet connections, ad hoc sensor networks and ubiquitous computing devices have made possible to embed millions of sophisticated software components into interconnected and dynamically changing local environments. In such cases, centralised and deterministic control is practically impossible or at best prohibitively expensive. As a result, a natural solution to the problem can be building software systems capable of efficiently adapting to failures, component replacements and changes in the environment, without human intervention or centralised management. In other words, such systems should be able to autonomously change their organisation, or self-organise, as and when needed until they achieve, or emerge to, a satisfactory or selected state. Self-organisation and emergence phenomena have long been observed in numerous natural systems, both living and non-living. Examples are the social order observed in human and animal social systems and the ordered orientation of magnetic spins appearing with lowering temperature in magnetic materials. As of recently, the idea that self-organisation and emergence can be harnessed for the purpose of solving tricky engineering problems inherent in modern IT systems has become increasingly popular. Researchers working in many diverse IT fields, such as computer networks, distributed software systems, operating systems and software agents, have begun to apply these ideas in a variety of problems with quite promising results. These efforts have given rise to the term Self-organising Software. Selforganising software systems are able to dynamically change their structure and v

vi

Preface

functionality without direct user intervention in response to changes occurring in user requirements, their environmental context and their internal state. The overall functionality delivered by self-organising software typically changes progressively, mostly in a nonlinear fashion, until it reaches (emerges to) a state where it satisfies the current system requirements, and therefore it is commonly referred to as selforganising behaviour. In the majority of cases, the overall self-organising behaviour is the result of execution of a number of interrelated individual components, which locally interact with each other aiming to achieve their local goals. Typical examples are systems based on software agents or distributed objects. The main characteristic of such systems is their ability to achieve complex collective results with relatively simple individual behaviours, applied without central or hierarchical control. Self-organising software engineers often take inspiration from the real world, for example from biology, chemistry, sociology and the physical world and apply the observed principles to implement self-organising functionality in software. Typical such examples are software systems that reproduce socially based insect behaviour, such as ant-based systems, artificial life systems and robot swarms. Furthermore, detailed methodologies specifically targeting the engineering and control of selforganising behaviour in software have started being increasingly used. However, despite that advances made so far have started maturing, the majority of the work done is still scattered throughout research publications and technical reports, and there is no clear starting point for those wanting to get acquainted with the field, for example students and junior researchers. The idea of this book germed during the meetings of the Technical Forum Group on Self-Organisation in Multi-Agent Systems,1 supported by the EU-funded AgentLink2 Network of Excellence. Some concepts and topics covered by this book have been the subject of debate, discussions and presentations during the group meetings. The decision to write a book then derived from the need to provide a unified view of self-organisation and its applicability to software in a neat way so that to be able to be used by instructors and readers in relevant courses, as well as by young researchers seeking an introductory, and at the same time a comprehensive, discussion of the issues involved. As a result, this book provides an introductory yet comprehensive review of recent work done in the field of self-organising software. The first chapters elaborate extensively on self-organisation concepts, mechanisms and engineering techniques. They are supported by examples which aim to facilitate the reader in gaining a better understanding of the self-organisation approach and its applicability. In the subsequent chapters, the book pays attention to providing instructive descriptions of application areas where self-organisation has successfully been used in software to provide the solution. Such areas include manufacturing control, computer network management and security, P2P protocols, and optimisation problem solving. All chapters are supplemented with puzzle questions, unsolved exercises and mini-projects aiming to be useful for teaching purposes. The solutions together with 1 http://www.irit.fr/TFGSO 2 http://www.agentlink.org

Preface

vii

additional teaching materials are contained in an instructor’s manual accompanying the book and available through the Technical Forum Group on Self-Organisation in Multi-Agent Systems web page. London, UK

Giovanna Di Marzo Serugendo Marie-Pierre Gleizes Anthony Karageorgos

Acknowledgements

This book is the result of the collective effort of the community working on Selforganisation and Multi-Agent Systems. Its realisation would not have been possible without the contributions of a number of persons, to which we are greatly indebted. First of all, we thank all the authors who accepted to participate in this book and to put notable efforts in contributing original pedagogical chapters and questions/exercises related to their chapters. We are particularly grateful to all researchers that have since 2003 participated in the Technical Forum Group on Self-organisation in Multi-Agent Systems which we have been organising. Without them, we would have never reached that broad view of the research in the self-organising systems area that enabled us to produce this book. A detailed list of the participants, their activities and the meeting outcomes can be found on the Technical Forum Group on Self-organisation in Multi-Agent Systems web page.1 We thank the AgentLink EU Network of Excellence and particularly the Technical Fora organisers for supporting the meetings of the Technical Forum Group on Self-organisation in Multi-Agent Systems from 2003 to 2005. We also thank the European Workshop on Multi-Agent Systems (EUMAS) for hosting the group meetings as a separate event since 2006.

1 http://www.irit.fr/TFGSO

ix

Contents

1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Giovanna Di Marzo Serugendo, Marie-Pierre Gleizes, and Anthony Karageorgos

Part I

1

Main Concepts and Background

2

Self-organising Systems . . . . . . . . . . . . . . . . . . . . . . . . . Giovanna Di Marzo Serugendo, Marie-Pierre Gleizes, and Anthony Karageorgos

7

3

History and Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . Giovanna Di Marzo Serugendo, Marie-Pierre Gleizes, and Anthony Karageorgos

33

4

Self-organisation in Natural Systems Inspiring Self-organising Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Paul Marrow and Jean-Pierre Mano

5

75

Agents and Multi-Agent Systems . . . . . . . . . . . . . . . . . . . . 105 Marie-Pierre Gleizes, Valérie Camps, Anthony Karageorgos, and Giovanna Di Marzo Serugendo

Part II

Self-organisation Mechanisms

6

Stigmergy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Christine Bourjot, Didier Desor, and Vincent Chevrier

7

Gossip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Márk Jelasity

8

Trust and Reputation for Successful Software Self-organisation . . . 163 Jean-Marc Seigneur and Pierpaolo Dondio

9

Cooperation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Jean-Pierre Georgé, Marie-Pierre Gleizes, and Valérie Camps xi

xii

Contents

10 Immune Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 Vincent Hilaire, Abderrafiâa Koukam, and Sebastian Rodriguez 11 Holonic Multi-Agent Systems . . . . . . . . . . . . . . . . . . . . . . 251 Sebastian Rodriguez, Vincent Hilaire, Nicolas Gaud, Stephane Galland, and Abderrafiâa Koukam Part III Engineering Artificial Self-organising Systems 12 Engineering Self-organising Systems . . . . . . . . . . . . . . . . . . 283 Carole Bernon, Marie-Pierre Gleizes, Frédéric Migeon, and Giovanna Di Marzo Serugendo 13 Middleware Infrastructures for Self-organising Pervasive Computing Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 Matteo Casadei, Marco Mamei, Cynthia Villalba, Mirko Viroli, and Franco Zambonelli Part IV Applications of Self-organising Software 14 Self-organisation in Constraint Problem Solving . . . . . . . . . . . 347 Pierre Glize and Gauthier Picard 15 Adaptive Trust Management . . . . . . . . . . . . . . . . . . . . . . 379 Jean-Marc Seigneur, Gabriele Lenzini, and Bob Hulsebosch 16 Security in Artificial Systems . . . . . . . . . . . . . . . . . . . . . . 405 Noria Foukia and Melanie Middlemiss 17 Region Detection in Images . . . . . . . . . . . . . . . . . . . . . . . 425 Vincent Chevrier, Christine Bourjot, and Vincent Thomas 18 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447 Giovanna Di Marzo Serugendo, Marie-Pierre Gleizes, and Anthony Karageorgos Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459

Contributors

Carole Bernon IRIT, Université Paul Sabatier, Toulouse, France, [email protected] Christine Bourjot LORIA, Université Nancy, Nancy, France, [email protected] Valérie Camps IRIT, Université Paul Sabatier, Toulouse, France, [email protected] Matteo Casadei Università di Bologna, 47023 Cesena, Italy, [email protected] Vincent Chevrier LORIA, Université Nancy, Nancy, France, [email protected] Didier Desor URAFPA, Université Nancy-UHP, Nancy, France, [email protected] Giovanna Di Marzo Serugendo Birkbeck College, University of London, London, UK, [email protected] Pierpaolo Dondio Trinity College Dublin, Dublin, Ireland, [email protected] Noria Foukia Information Science Department, University of Otago, P.O. Box 56, Dunedin, New Zealand, [email protected] Stephane Galland UTBM, Belfort, France, [email protected] Nicolas Gaud UTBM, Belfort, France, [email protected] Jean-Pierre Georgé IRIT, Université Paul Sabatier, Toulouse, France, [email protected] Marie-Pierre Gleizes IRIT, Université Paul Sabatier, Toulouse, France, [email protected] Pierre Glize CNRS/IRIT, Université de Toulouse III, Toulouse, France, [email protected] Vincent Hilaire UTBM, Belfort, France, [email protected] xiii

xiv

Contributors

Bob Hulsebosch Telematica Instituut, Enschede, The Netherlands, [email protected] Márk Jelasity University of Szeged and Hungarian Academy of Sciences, P.O. Box 652, 6701 Szeged, Hungary, [email protected] Anthony Karageorgos Technological Educational Institute of Larissa, Larissa, Greece, [email protected] Abderrafiâa Koukam UTBM, Belfort, France, [email protected] Gabriele Lenzini University of Luxembourg, Luxembourg-Kirchberg, Luxembourg, [email protected] Marco Mamei Università di Modena e Reggio Emilia, 42100 Reggio Emilia, Italy, [email protected] Jean-Pierre Mano UPETEC, Ramonville Saint-Agne, France, [email protected] Paul Marrow BT Group plc, Ipswich, UK, [email protected] Melanie Middlemiss Information Science Department, University of Otago, P.O. Box 56, Dunedin, New Zealand, [email protected] Frédéric Migeon IRIT, Université Paul Sabatier, Toulouse, France, [email protected] Gauthier Picard École Nationale Supérieure des Mines de Saint-Etienne, SaintEtienne, France, [email protected] Sebastian Rodriguez CITAT, Universidad Tecnológica Nacional—Facultad Regional Tucumán, San Miguel de Tucumán, Argentina, [email protected] Jean-Marc Seigneur University of Geneva, Geneva, Switzerland, [email protected] Vincent Thomas LORIA, Université Nancy, Nancy, France, [email protected] Cynthia Villalba Università di Modena e Reggio Emilia, 42100 Reggio Emilia, Italy, [email protected] Mirko Viroli Università di Bologna, 47023 Cesena, Italy, [email protected] Franco Zambonelli Università di Modena e Reggio Emilia, 42100 Reggio Emilia, Italy, [email protected]

Acronyms

ABT ACL ACO ADELFE

Asynchronous BackTracking Act Communication Language Ant Colony Optimisation Atelier de Développement de Logiciels à Fonctionnalité Emergente—Toolkit to develop software with emergent functionality ADOPT Algorithm for Distributed Constraint Optimisation AGR Agent Group Role AI Artificial Intelligence AIS Artificial Immune System AL Artificial Life AMAS Adaptive Multi-Agent Systems AMAS-ML AMAS Modeling Language AMP Adenosine Monophosphate ATL Atlas Transformation Language AOSE Agent-Oriented Software Engineering APC Antigen Presenting Cell APER A Peer Entity Recognition API Application Programming Interface APO Asynchronous Partial Overlay AUML Agent Unified Modeling Language AWCS Asynchronous Weak-Commitment Search B2C Business to Consumer B- and T-Cells B lymphocytes and T lymphocytes BT Bluetooth BTS Base Transceiver Stations cAMP Cyclic Adenosine Monophosphate CAS Complex Adaptive Systems CCD camera Charge-Coupled Device Camera COP Constraint Optimisation Problems CPU Central Processing Unit xv

xvi

CSP DAI DANTE

DB DBA DC DCOP DIET DisCSP DNA DSR DoS EDOS EMF EPE ER ERA FIFO FIPA FIRA FOAF GA GPD GPS HIS HMAS ICT ID IDA IDRS IDReAM IDS IR IRA IRS IT Ja-Net KQML LS LTTM

Acronyms

Constraint Satisfaction Problems Distributed Artificial Intelligence Domain ANalysis and Trust Extraction. A trust model able to identify its input among application elements and exploit them for a trust computation Data Base Distributed Breakout Algorithm Dendritic Cell Distributed Constraint Optimisation Problem Decentralised Information Ecosystem Technologies Distributed Constraint Satisfaction Problems Deoxyribonucleic Acid Danger Signal Receptors Denial of Service Environment for the development and Distribution of Open Source software Eclipse Modelling Framework Emergent Programming Environment Entity Recognition Environment Reactive rules and Agents First In First Out Foundation for Intelligent and Physical Agents Federation of International Robot-soccer Association Friend-Of-A-Friend Genetic Algorithm Generalised Prisoner Dilemma Global Positioning System Human Immune System Holonic Multi-Agent Systems Information and Communication Technologies Intrusion Detection Intrusion Detection Agent Intrusion Detection and Response System Intrusion Detection and Response executed with Agent Mobility Intrusion Detection System Intrusion Response Intrusion Response Agent Intrusion Response System Information Technology Jack-in-the-Net Knowledge Query and Manipulation Language Local Search Longo Temporal Trust Factors. A trust model introduced by Luca Longo that bases its computations entirely on the time distribution of entities’ activity

Acronyms

MA MAS MAY MHC μADL MOCA NCS OEM OGSI OMG OSS oAW P2P PAMP PC PD PDA PIN PMI PROSA PRR PSO QA RA REGRET ReSpecT RFID RIO RT RUP SECURE SI SI model SIR model SLAC SPEM SQL STAFF STM TCP TCR TFGSO

xvii

Mobile Agent Multi-Agent Systems Make Agents Yourself Major Histocompatibility Complex μ Architecture Description Language Modèle Organisationnel et Componentiel pour les systèmes multi-Agents Non-Cooperative Situation Original Equipment Manufacturer Open Grid Services Infrastructure Object Management Group Open Source Software open-Architecture Ware Peer to Peer Pathogen Associated Molecular Pattern Personal Computer Prisoner Dilemma Personal Digital Assistant Postal Index Number Project Management Interface Product-Resource-Order-Staff Architecture Pattern Recognition Receptors Particle SwarmOptimisation Quality Assessment Representative Agent REGRET: A Reputation Model for Gregarious Societies Reaction Specification Tuples Radio Frequency Identification Role Interaction Organisation Recommending Trustworthiness Rational Unified Process Secure Environments for Collaboration among Ubiquitous Roaming Entities Suspicion Index Susceptible Infected model Susceptible Infected Removed model Selfish Link-based Adaption for Cooperation Software Process Engineering Metamodel Structured Query Language Software Tool for Adaptive Flood Forecast Short Term Memory Transmission Control Protocol T-Cell Receptor Technical Forum Group on Self-Organisation in multi-agent systems

xviii

TOTA Tropos4AS TuCSoN ULPC UML VER VO WD WfMS WiFi WLAN WWW XML

Acronyms

Tuples On The Air Tropos for Adaptive Systems Tuple Centres over the Network User Local Probability Component Unified Modeling Language Vision Entity Recognition Virtual Organisation WorkDefinition Workflow Management Systems Wireless Fidelity Wireless Local Area Network World Wide Web eXtensible Markup Language

Chapter 1

Introduction Giovanna Di Marzo Serugendo, Marie-Pierre Gleizes, and Anthony Karageorgos

Objectives After reading this chapter the reader will: • Understand the need and motivation for self-organising software; • Have an understanding of the goal of the book; • Have a clear view of the book structure.

1.1 Motivation The spread of the Internet, mobile communications and the proliferation of new market models, such as e-commerce, have resulted in the whole information infrastructure operating as a global dynamic system. Therefore, contemporary software needs to adapt to ever changing requirements, by providing highly customised services to a huge user population. Furthermore, new software maintenance restrictions have to be observed; for example certain types of software need to run continuously, and hence such software must evolve to meet changing requirements. In a large interconnected system this is a task beyond centralised management techniques. At

G. Di Marzo Serugendo Birkbeck College, University of London, London, UK e-mail: [email protected] M.-P. Gleizes IRIT, Universitè Paul Sabatier, Toulouse, France e-mail: [email protected] A. Karageorgos () Technological Educational Institute of Larissa, Larissa, Greece e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_1, © Springer-Verlag Berlin Heidelberg 2011

1

2

G. Di Marzo Serugendo et al.

the other end of the scale, latest technological advances, such as ad hoc sensor networks and MEMs devices, are making it possible to embed millions of smart computing agents into the execution environment. In such cases the system needs to be able to recover from constant failures, replacement of its components and changes in the environment, without human intervention. As a result, it has become clear that dynamic adaptation to changes coupled with autonomous, and in most cases decentralised, reorganisation will play an important role in the future of IT systems. A way to meet these requirements is implementing and utilising self-organising and emergent functionality (also referred to as behaviour) in software, namely making use of concepts such as self-organisation, self-regulation, self-repair and selfmaintenance. The overall functionality delivered by self-organising software typically changes progressively, mainly in a nonlinear fashion, until it reaches (emerges to) a state where it satisfies the system requirements at the time. Self-organisation is in most cases the result of execution of a number of individual application components that locally interact with each other aiming to achieve their individual goals. Examples of such components are software agents and distributed objects. The main characteristic of such systems is their ability to achieve complex collective tasks with relatively simple individual behaviours, and without central or hierarchical control. However, in artificial systems, environmental pressures and local interactions and control may lead to unpredicted or undesirable behaviour. Understanding how to engineer the correct self-organisation behaviour is thus an issue of major concern. Among major open issues are how to correctly engineer desirable emergent behaviour in self-organising software and how to avoid undesirable one given the requirements and the application environment. Furthermore, additional issues of great concern include validation, verification, exception handling and finally pilot deployment, testing and maintenance of self-organising software in real-world settings. To engineer artificial self-organising behaviour, approaches originating from diverse areas such as nonlinear optimisation, knowledge-based programming and constraint problem solving have so far been explored. Furthermore, engineers of selforganising software often take inspiration from self-organising systems found in many areas including biology, chemistry, sociology and the physical world. Examples of such systems are those reproducing socially-based insect behaviour, for example the behaviour shown in ant colonies and bird flocks. These efforts have resulted in advances made both in terms of the definition and study of the relevant concepts and processes and in applying the respective principles and mechanisms in various problem domains. For example, multi-agent simulations and analytic modelling have been used to study self-organising and emergent behaviour in real systems. On the other hand, approaches based on complexity theory have been extensively applied in the engineering of self-organising agent-based and autonomic software systems suitable for real-world applications. The results achieved so far are quite promising, and the work is continuing with strong motivation. However, most of the findings are scattered at research publications making it difficult for the interested reader to get the complete picture.

1 Introduction

3

1.2 The Book Goal Our decision to produce this book initiated from the fact that at the time there was no book providing a unified view of self-organisation in a neat way so that to be able to be used by instructors and readers in relevant courses as well as by young researchers seeking an introductory and at the same time a comprehensive discussion of the issues involved. To fill this gap, the decision to produce a textbook was taken. The book intends to provide an integrated view of the most recent advances in self-organising software with particular emphasis on distributed, agent-based software systems. In this mission, our efforts were directed along four dimensions: 1. The self-organisation concepts and principles dimension concerning definitions, background and theoretical deliberation on relevant concepts. 2. The self-organisation mechanisms dimension concerning the principles governing the interactions among individual entities of a self-organising system. 3. The engineering techniques dimension concerning how self-organising behaviour can be explicitly modelled, analysed and systematically engineered in software component behaviour. 4. The application dimension focusing on the types of problems that can be solved via self-organising software and on providing examples of such applications that are currently available. The book comprises chapters covering all four dimensions by synthesising upto-date research and latest technologies and applications. The chapters are accompanied with solved examples, unsolved exercises and mini-project descriptions, followed by directions for further work. Therefore, the book is suitable both for advanced undergraduate and postgraduate teaching and for research background reading. Since the book aims to be the first of its kind in this field, its contents were selected to provide both an introductory and at the same time a broad view of the topics covered. Therefore, emphasis was paid to covering the relevant background in adequate depth and to describing in detail the methodologies and tools currently available for developing self-organising applications. Furthermore, to provide insight for the applicability of the techniques discussed, the decision to include detailed descriptions of real-world case studies was made.

1.3 The Book Structure The structure of the book is based around the fundamental self-organisation concepts and mechanisms. It is organised into four parts with several chapters in each part. • Part I: Main Concepts and Background introduces the concepts of self-organisation and emergent phenomena, presents the notion of self-organising systems

4

G. Di Marzo Serugendo et al.

and provides an overview of software systems utilising self-organisation and emergence. Furthermore, it reviews the main natural self-organising systems and mechanisms, which currently serve as a source of inspiration when engineering artificial self-organising systems. Finally, it introduces the notion of autonomous software agents, which constitute the basic building blocks towards the engineering of artificial self-organising systems. • Part II: Self-organisation Mechanisms extensively explains how mechanisms allowing natural systems to achieve self-organisation have been implemented into artificial systems. This part details the following mechanisms: stigmergy, gossip, trust and reputation, cooperation, immune systems and holonic systems. • Part III: Engineering Artificial Self-organising Systems highlights the need for systematic engineering techniques when developing self-organising software. It describes various approaches which use agent technology and are currently applied for developing self-organising systems. Additionally, this part provides an insight into the use of specific middleware integrating built-in features for selforganising mechanisms, which are useful for easing the engineers’ task when programming self-organising software. • Part IV: Applications of Self-organising Software describes several self-organising systems pertaining to different application domains: from constraint optimisation problems to trust management, to security and to region detection.

Key Points • Self-organising software is able to change its structure and functionality due to changes in the requirements and the environment; • Self-organisation and emergence are properties inherently found in many natural systems since they offer particular advantages, such as robustness, scalability and cost; • Self-organising systems in most cases consist of autonomous units acting without any central control to achieve their individual goals. The unit individual behaviours and interactions result in a collective, global system behaviour; • There is currently a growing interest in self-organising systems, not only from a research point of view but also from a commercial one; • Most self-organising software systems currently mimic natural systems to implement self-organising behaviour.

Part I

Main Concepts and Background

Chapter 2

Self-organising Systems Giovanna Di Marzo Serugendo, Marie-Pierre Gleizes, and Anthony Karageorgos

Self-organisation and emergence in software systems.

Objectives The objective of this chapter is to introduce the subject of selforganising software systems. When you will have read this chapter, you will: • Know the answers to key introductory questions to self-organising systems; • Understand what self-organising software is and why it is important; • Obtain an understanding of different self-organising system types that can be found in nature; • Obtain an overview of various self-organising applications.

2.1 Self-organising Systems: An Overview The study of self-organising systems is a field that has been explored at least since 1953 with the work done by Grassé [43], who studied the behaviour of insect so-

G. Di Marzo Serugendo Birkbeck College, University of London, London, UK e-mail: [email protected] M.-P. Gleizes IRIT, Université Paul Sabatier, Toulouse, France e-mail: [email protected] A. Karageorgos () Technological Educational Institute of Larissa, Larissa, Greece e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_2, © Springer-Verlag Berlin Heidelberg 2011

7

8

G. Di Marzo Serugendo et al.

cieties. Grassé found that these societies show changing forms of order occurring without any central point of control. In other works it has been found that many existing systems demonstrate selforganisation, such as planetary systems, organic cells, living organisms and animal societies. All these systems exhibit recurrent properties inherent to self-organisation, and are therefore termed self-organising systems. Self-organising systems are encountered in many scientific areas including biology, chemistry, geology, sociology and information technology, and considerable research has so far been undertaken to study them. A large number of artificial self-organising systems are designed based on selforganisation mechanisms inspired by nature. Furthermore, recent research has been oriented towards introducing self-organisation mechanisms specifically for software applications, as well as entire software development techniques supporting selforganisation [33]. This trend originates from the fact that current software applications need to cope with requirements and constraints stemming from the increased dynamism, sophisticated resource control, autonomy and decentralisation inherent in contemporary business and social environments. The majority of these characteristics and constraints are the same as those which can be observed in natural systems exhibiting self-organisation. The variety of systems where the notion of self-organisation is found makes it difficult to find a precise definition of what self-organisation is. In its simplest form, self-organisation can be considered as the autonomous arrangement of parts of a system in such a way as to be non-random. A more detailed discussion on this topic is provided in Chap. 3. In the following we provide a brief description of the notions of self-organisation and emergence, and we outline the main types of self-organising systems. In all systems, an important issue is their capacity to deploy the effective global behaviour that permits the realisation of their intentional or non-intentional goals.

2.1.1 Self-organisation and Emergence Intuitively, self-organisation refers to the fact that a system’s structure or organisation appears without any explicit control or constraints imposed from outside the system. In other words, the organisation is intrinsic to the self-organising system, and it results from internal constraints and mechanisms, which are based on local interactions between its components [24]. These interactions are often indirect and are carried out through the environment [43]. The non-deterministic and dynamic nature of interactions causes emergent system properties to appear, which transcend the properties of all individual sub-units of the system [49]. Furthermore, the dynamic operations affect and modify the environment the system is situated in, and in turn alterations to the environment influence the system again in a feedback loop [68]. In most cases environmental influences and perturbations do not affect the internal mechanisms which cause dynamic re-organisation. As a result, the system

2 Self-organising Systems

9

evolves dynamically [17] either in time or space, and it can either aim to maintain a stable form or show transient phenomena. Generally, the emergent phenomena is an externally identifiable outcome, for example a particular pattern or structure, property, behaviour or system state, which, although not explicitly represented at a lower level, appears at a higher level. That complex collective behaviour usually occurs without any central control, and it is derived from the simple local individual behaviours and interactions. One wellknown example of emergent system behaviour is the collective behaviour shown in a colony of ants which sort their eggs without having any particular ant knowing and centrally applying some sorting algorithm. Self-organisation and emergence are not perfect; units in a self-organising system are prone to opposing actions, their behaviour may induce needless redundancies, and decentralised control limits the ability of the system to find the globally optimal solution [84]. However, for systems that are complex and operate in a dynamic environment, the use of self-organisation offers significant advantages such as increased scalability and robustness and reduced communication and unit processing costs [66]. The body of work in the area of self-organising systems and emergent phenomena follows different streams. There is a large body of work dedicated to the study of natural systems to understand their underlying mechanisms (e.g. understanding natural swarms, human organisations, earthquakes, etc.). Self-organisation and emergent phenomena are also studied from a theoretical and generic perspective in the field of Complex Adaptive Systems [11]. Alternatively, the mechanisms or theoretical concepts identified during the study of self-organising systems provide nature-inspired techniques for engineering artificial systems in many scientific areas, such as Computational Biology [25], Synthetic Biology [36], Artificial Life [19], Robotics [78], Computer Science [34]. This book concentrates on the latter, exploiting techniques inspired from natural self-organising systems in ICT-related applications, such as optimisation problem solving, P2P protocols, sensor networks or access control.

2.1.2 Natural Systems Self-organising and emergent phenomena can be observed in many natural systems. For example, insects that live in colonies, such as ants, bees, wasps and termites, have been shown to seamlessly integrate their individual activities, while every single insect seems to operate individually and without any central supervision. Interaction among members in such colonies is carried out indirectly by asynchronously modifying and subsequently perceiving the state of their environment. Other examples of natural self-organising systems include flocks of birds and schools of fish. In these cases, collective self-organising behaviour is achieved by synchronous perception of the environment and by following simple rules. For example, by aiming to remain in close proximity while avoiding collision with similar group members

10

G. Di Marzo Serugendo et al.

Fig. 2.1 Path emerging from foraging ants

and at the same time by attempting to keep a distance from dissimilar entities, birds or fish significantly reduce the probability of being attacked by a predator [30]. Such systems exhibit many interesting complex behaviours, and they have emergent properties resulting from local interactions between elementary behaviours exercised individually. The emergent collective behaviour is the outcome of selforganisation processes in which members are engaged through their repeated actions and interactions with their evolving environment [49]. A typical example of self-organisation and emergence in natural systems concerns foraging ant colonies [31]. Foraging ants explore their environment seeking food. When they succeed in their task, they return to the nest after placing in the environment chemical substances which are subsequently used by other ants as indicators of proximity to food source. Not long after the food source is discovered, a path consisting of dropped chemical substances is created between the food source and the nest; actually, it is the shortest possible path given the environmental constraints, and it emerges as a result of the collective activity of ants (see Fig. 2.1 for a simulation view). That path can be identified only by external observers; for example, the individual ants cannot see it as a whole. The field of research that investigates models and computational techniques inspired by nature is termed Natural Computing, and it attempts to understand systems found in nature in terms of information processing. It is a highly interdisciplinary field that connects the natural sciences with computing science, both at the level of information technology and at the level of fundamental research. Natural computing covers many areas including pure theoretical research, algorithms and software

2 Self-organising Systems

11

applications, as well as biology, chemistry and physics experimental laboratory research [55]. Natural systems can be broadly classified as Physical Systems, Biological Systems and Social Systems. A brief description of each category is provided in the next sections, and more details are provided in Chap. 4.

2.1.2.1 Physical Systems Theories of self-organisation were originally developed in the context of Physics and Chemistry to describe the emergence of macroscopic patterns out of processes and interactions defined at the microscopic level [45, 76]. A common characteristic of physical self-organising systems is the existence of some critical threshold, which causes an immediate change to system state when reached. That critical threshold can be a combination of values of certain system variables, for example temperature, pressure and speed. The self-organisation effect is observed globally when the system transits from a chaotic disordered state to a stable one. For example, in a thermodynamic system, such as one consisting of a gas quantity, the system properties temperature and pressure are emergent since they are not determined from any particular gas particle. Instead, they are defined by the positions and velocities of all particles in the system. Similarly, chemical reactions create new molecules that have properties that none of the atoms exhibit before the reaction takes place [11]. Moreover, the magnetisation of a multitude of spins in magnetic materials1 is a clear case of self-organisation because, under a certain temperature, the magnetic spins spontaneously rearrange themselves pointing all in the same direction and creating thus a strong emerging magnetic field [48]. Additional physical and chemical systems are described in Chaps. 3 and 4.

2.1.2.2 Biological Systems Self-organisation is a common phenomenon in subsystems of living organisms. As a result, an important field in biological research is the determination of invariants in the evolution of living organisms and in particular the spontaneous appearance of order in living complex systems due to self-organisation. In the optic of biological research, the common meaning of self-organisation is defined by the global emergence of a particular behaviour or feature that cannot be reduced to the properties of individual system’s components such as molecules, agents and cells [24]. An example described in [75] is the self-organisation of the cytoskeleton due to collective processes of reaction and diffusion of the cytoskeletal filaments. The cytoskeleton is the basis of the internal architecture of the cytoplasm of eukaryotic cells. Eukaryotic cells are cells belonging to higher-level organisms which contain 1A

spin is a tiny magnet.

12

G. Di Marzo Serugendo et al.

a true nucleus bounded by a nuclear membrane. Such cells are found in plants and animals, and they are commonly organised into organs, such as the liver, or subsystems, such as the nervous system, which, by means of metabolic processes, despite the increase of system complexity, remarkably provide higher-level richer functionality. That resulting functionality is completely new, and it transcends all individual functionalities offered by the respective constitutive cells. For instance, human body subsystems transparently manage vital functions, such as blood pressure, digestion, or antibodies creation. Eukaryotic cells are the fundamental functioning units of life, and they evolve progressively according to external changes to their environment. Evolution is achieved with the use of internal metabolic processes such as mutation or natural selection, and it provides the basis for the evolution observed in living organisms and natural species. Additional examples of self-organising biological systems include the immune system of mammalians, the regeneration of cells and the human brain behaviour. A typical example of the latter is the apparition of conscience in humans. Conscience is viewed by Searle [83] as a property of the brain at the higher or global level. Biologically, the brain is a complex system composed of a set of neurons and interactions between them. Although conscience is a result of neuron operations done at a lower level, it is currently not possible to understand or explain human conscience by observing the brain neurons and their interactions.

2.1.2.3 Social Systems In natural system’s societies, entities commonly exhibit social behaviours leading to self-organisation, self-adaptation and self-maintenance of the society organisation, which often is observed in the form of some global societal behaviour. Individual social behaviours range from those observed in biological entities, for example bacteria, cells and insects, such as spiders, to those observed in larger animals and humans. One important characteristic of global societal behaviours is their formation as results developed from relatively simple interactions in a network of individuals. These resulting behaviours are considered to be driven by dynamic processes that are governed by simple but generic laws [12, 37]. For example, an important reason that has historically triggered collective behaviour in natural societies is the ultimate goal of species survival. This goal is not explicitly expressed at the individual level, but it is reflected in the collective behaviour of society members towards the emergence of social functions and group dynamics allowing the maintenance of the system’s organisation. For instance, social insects organise themselves to perform activities such as food foraging or nest building. Communication among insects is realised by an indirect communication mechanism, termed stigmergy, which is implemented through the society environment. Insects, such as ants, termites or bees, implement stigmergy by marking

2 Self-organising Systems

13

their environment using a chemical volatile substance, termed pheromone,2 and subsequently arrange the direction of their movements based on the location of pheromones in their environment, for example, as ants do to mark a food trail. The pheromonal information constitutes an indirect communication means through the insects’ environment. Insects generally have a simple behaviour, at least as far as the collective outcome is concerned [84], and none of them alone can provide the complete solution to the problem; for example, no ant is aware of the exact location of the food source found. However, the interactions between individual behaviours give rise to an organised society collective behaviour, for example ants are able to explore their environment, find food and efficiently inform the rest of the colony. Apart from other living organisms, humans commonly organise into advanced societies and organisations which can serve many purposes. For example, humans can use direct communication, engage in negotiations, build whole international economies and organise stock markets. Social behaviour of humans is typically selforganising, and it normally gives rise to emergent complex global behaviours. In many cases, individual human behaviour is based on small-range local information, and communication is carried out on local direct or indirect interactions which produce complex societal behaviours. A representative example of an emergent phenomenon in human societies is that of common beliefs, for example work ethics, developed and fostered through local communications such as gossip.

2.1.3 Business and Economics Systems We distinguish natural systems from business and economic systems, since generic laws guiding self-organisation in the former type of systems are dictated by nature, whereas in the latter, self-organisation is governed by business and market laws. In business and economic systems, individual behaviours are goal-oriented, and their primary goal is to increase their profit. In this case, the system’s dynamics is handled by the activity developed to face business and economic constraints to reach a global equilibrium through which the system can survive. In this context, various models governing business operations, such as management, marketing and economic models, which are based on self-organisation have been developed. For example, one such model from the marketing domain is the one-to-one variable pricing model [46] which refers to providing an individual offer to each customer using Internet technologies. The model involves self-organisation of the marketing policies by changing customers targeted and the prices quoted based on market dynamics, customer characteristics and the business goals. An example of a company using this type of marketing is FedEx [101]. The company allows customers to access computer systems, via the Web site, to monitor the status of their packages. For corporate customers, FedEx provides software tools that enable the 2 Pheromones are chemical substances deposited by insects in their environment at regular intervals

so that other insects sense and follow them.

14

G. Di Marzo Serugendo et al.

organisation to automate shipping and track packages using their own computing resource. Each customer is offered different prices depending on a variety of parameters. Many websites, such as eBay, also apply variable pricing for their offers. Another example of a self-organising business model from the area of management is the theory of activity described in [100]. According to that model, a business is viewed as consisting of networks of working groups that can change their structure, links and behaviour in response to business requirements. The aim of that model is to balance the self-organisation decisions that are both taken by managers and shaped from interactions between employees during business operations. The emphasis of the theory of activity is on solving potential conflicts of interests both of the inner and outside co-operative activity of the company.

2.1.4 Artificial Systems Artificial self-organising systems are those built from the beginning with embedded self-organisation capabilities. Mechanisms can be borrowed from existing natural systems, or they can be created explicitly for that purpose. Among artificial self-organising systems, we observe different trends ranging from application of naturally-inspired self-organising models to the establishment of new mechanisms and whole infrastructures supporting self-organisation of artificial systems. Swarms provide a great source of inspiration, especially for fixed and mobile networks systems management [20], such as routing, load balancing [74], or security [41]. Holarchies as well have inspired researchers dealing with e-Government and e-Society issues [93]. At the level of whole infrastructures (middleware) supporting artificial self-organising systems, some works take their inspiration from magnetic fields [65] or ants [10]. Furthermore, self-organising models aimed specifically for particular applications have been developed. For example, an artificially engineered self-organising system concerning coordination and organisation of a group of robots is reported in [80]. In that context, robots transport objects between two rooms connected with two corridors which cannot be used by more than one robots at a time. Robots have local perception and apply a cooperative attitude in selecting which corridor to use. The result is system performance improvement arising from corridor specialisation where movement in each corridor is done in a specific direction.

2.2 Self-organising Applications Self-organisation is increasingly used in software applications to provide the solution to problems of various types. This is leveraged by the distributed nature of contemporary software, the highly changing environments, the large numbers of heterogeneous components software systems commonly consist of, and the difficulty in imposing central control in distributed software.

2 Self-organising Systems

15

In the following we present an overview of the most common applications of self-organisation in software systems.

2.2.1 Multi-Agent Systems An agent is a physical (robot) or a virtual (software) entity situated in an environment that changes over time: the physical world or an operating system respectively. Through its sensors, the agent is capable of perceiving its environment, and through its effectors, it is capable of performing actions that affect the environment. For instance, a robot may take notice of obstacles with an embedded camera and remove them from its way with an articulated arm. A software agent may understand a user’s request through a user’s interface and send an e-mail to the user once the request has been satisfied [54, 81]. Every single agent has one or more limitations, which can be categorised into cognitive limitations, physical limitations, temporal limitations and institutional limitations. Cognitive limitations resemble the fact that individuals are rationally bounded. It means that the data, information, and knowledge an individual can process and the detail of control an individual can handle is limited. As tasks grow larger and more complex, techniques must be applied to limit the increase of information and the complexity of control. Individuals can be limited physically, because of their physiology or because of the resources available to them. Temporal limitations exist where the achievement of individual goals exceeds the lifetime of an individual, or the time over which resources are available for achieving a goal. Finally, individuals can be legally or politically limited. To overcome their limitations, agents group together and form multi-agent systems, or societies of agents, where they work together to solve problems that are beyond their individual capabilities [77]. A robot has to bring a cup from the office to the kitchen. It may ask the need of another agent to bring the cup in the kitchen if it is itself unable to reach the kitchen whose door is closed. On the basis of its knowledge about the user’s behaviour, an assistant agent may decide to regularly inform that user about new computer science books, without the user having explicitly notified the agent to do that. In order to obtain this information, the agent may need to contact other agents aware of computer science books. Agents interact (communicate, coordinate, negotiate) with each other and with their environment. Interactions among agents are normally governed by some coordination model. An agent can communicate directly or indirectly with other agents for cooperation or competition purposes. Since the agent perceives its environment and interacts with other agents, it is able to build a partial representation of its environment, which constitutes its knowledge. Usually, in a multi-agent system, interactions are not pre-defined, and there is no global system goal. The interaction dynamics between an agent and its environment lead to emergent structures or emergent functionality, even though no component is responsible for producing a global goal. More details about the concepts of agent and multi-agent systems are provided in Chap. 5.

16

G. Di Marzo Serugendo et al.

2.2.2 Computational Grids and P2P Systems Computational Grids provide the software and networking infrastructure required to integrate computational engines, scientific instruments, data repositories and human expertise to solve a single large problem generally in science and engineering domains. Grid computing essentially refers to any distributed cluster of compute resources that provides an environment for the sharing and managing of the resource and for the distribution of tasks based on configurable service-level policies. Current resources being provided within Computational Grids are owned by national or regional centres, or by research institutions, and therefore access rights and usage need to be pre-defined and approved. However, as resources from less trusted users are provided, the need to organise them into dynamic communities, based on a number of different criteria, such as performance, trust or cost of ownership and usage, becomes significant. Self-organisation therefore plays an important role in identifying how such communities can be formed and subsequently disbanded. For example, a utility-based approach for forming such communities is explored in [64]. Other issues relate to resource selection and management, such as the distribution of tasks to the best available node and the distribution and management of data between compute nodes, while optimising global performances. Solutions vary from the use of mobile agents for resource and task distribution [51] to human market inspired techniques for adaptively changing the application placement and workload assignment to satisfy the dynamic applications workloads [62], to self-adaptive techniques based on monitoring of processors and computation of efficiency to automatically adapt the number of processors or migrate computation away from overloaded resources [103], to techniques inspired by bee foraging behaviour [59] allowing grid clients to dynamically select the most appropriate algorithms for executing chunks of data. In parallel to computational grids, there has been an emerging interest in trying to integrate resources across organisational boundaries through file or CPU sharing software (such as KaZaA [3] and Gnutella [2] for file sharing and Entropia [1] and UD [4] for CPU sharing). These individual resources are often geographically distributed and may be owned by different administrators (or exist within different independently administered domains). Self-organisation in P2P and MANET encompass gossip-based overlay topology management [53] or decentralised techniques for routing, updates and identity maintenance [8].

2.2.3 Service Emergence In many cases, the service provided by some applications emerges from joint provision and interaction between constituent subservices. For example, Itao et al. [52] propose Jack-in-the-Net (Ja-Net), a biologically-inspired approach to design emergent network applications and services in large-scale networks. In Ja-Net, network

2 Self-organising Systems

17

applications and services are dynamically created from local interactions and collaboration of self-organising entities, called cyber-entities. Each cyber-entity implements a simple functional component of the overall service or application. Furthermore, it follows simple behavioural rules, similar to the behaviours of biological entities, such as energy exchange with the environment, migration or replication, and relationship establishment. Services emerge from strong relationships among cyber-entities. Indeed, cyber-entities record information about peer cyber-entities during a relationship. Once relationships among a collection of cyber-entities are strong enough, they form a group and create a service. Relationship strengths are evaluated on the basis of the utility degree of each cyber-entity participating in the service. The utility degree is estimated using user feedback on the delivered service. In addition to service emergence, the system exhibits a natural selection mechanism based on the notions of energy stored or spent by cyber-entities and the diversity of services created. Due to the migration, replication and possible deaths of entities, the system is also able to adapt to networks changes. Along the same lines, with self-adaptability and evolvability in mind, Viroli et al. [98] propose to engineer ecosystems of services getting inspiration from natural ecosystems. Services and data sources are seen as autonomous individuals, and rules governing the ecosystem are inspired by those found in natural ecosystems, e.g. biochemical mechanisms.

2.2.4 Dynamic Web Page Communities Flake et al. [39] show that Web pages self-organise into communities identified by pages connectivity. A Web page is a member of a community if it has more hyperlinks within the community than outside it. Flake et al. have defined a specific algorithm which highlights communities of related Web pages, on the basis of hyperlinks contained in the pages. From a self-organisation point of view, authors of Web pages independently put them on the Web with hyperlinks on other pages. Inserting a page on the Web modifies the environment, for example the world space of the Web pages, and this in turn modifies the behaviour of other authors of pages. It becomes then possible to reach existing pages from a new location as it becomes possible to reference and go through these new pages. By analogy with the ants metaphor, authors place Web pages (pheromone) on the Web (food trail). These Web pages contain specific information for other authors, who will reinforce (or not) the strengths among Web pages by referencing them. Authors then collectively but independently organise Web pages into communities.

2.2.5 Network Coordination Since the early days of swarm intelligence research it has been argued that insectlike behaviour of simple entities working in groups could provide a powerful

18

G. Di Marzo Serugendo et al.

metaphor enabling the development of completely decentralised systems. A notable advantage of such systems was emphasised to be the collaboration of individual entities to produce collective and more complex global emergent behaviours [18]. These advantages have inspired computer and telecommunications network scientists to mimic insect foraging behaviour on the coordination and control of data network traffic [50]. A representative example is the ant-like approach suggested by Di Caro and Dorigo in [32], which uses swarm-based models and algorithms to solve network coordination problems. The hypothesis is that ant-based coordination would be able to better cope with network dynamics than direct human supervision. The proposed approach involves introducing an artificial ant network where each node periodically launches artificial ants assigned with the task of finding the route to a given destination. By simply sensing the intensity of the artificial pheromones located along the neighbour paths of the node, the artificial ants generate a map that provides the shortest route to any destination. In case of congestions, it was shown that this approach outperformed all other popular, at the time, network routing algorithms in terms of time needed to identify routes that would avoid traffic jams. Additional examples of applications of nature-inspired algorithms to network coordination include applying ant-like behaviours to optimise network traffic [9, 16] and mimicking the division of labour and task allocation met in insect societies to optimise load balancing in network systems [82]. More details on bio-inspired swarm algorithms are provided in Chap. 4.

2.2.6 Network Security The use of Mobile Agents in sophisticated applications offers advantages for constructing flexible and adaptable wide-area distributed systems. Notably, applications such as Intrusion Detection Systems (IDSs) and Intrusion Response Systems (IRSs) have become even more relevant in the context of large-scale network infrastructures, where traditional security mechanisms demonstrate severe weaknesses [67]. Indeed, as they can be retracted, dispatched, cloned or put in stand-by, mobile agents have the ability to sense network conditions and to load dynamically new functionality into a remote network node (such as a router). The network awareness of mobile agents can also significantly contribute to the detection of intrusions and enables providing appropriate response. Deriving from this trend, mobile agents have been proposed as support for Intrusion Detection (ID) and Intrusion Response (IR) in computer networks. The originality of this approach lies on the design of the intrusion detection and response system (IDRS). Indeed, the organisation of mobile agents follows the behaviour of natural systems to detect an intrusion as well as to answer an intrusion [41]. Schematically, there are two natural paradigms that have been referred to. Firstly, the human immune system, because the IDS is based upon principles derived from the immune system model, where Intrusion Detection Agents (ID Agents) map the functionality of the natural immune system to distinguish between normal and abnormal events (respectively “self” and “non-self”

2 Self-organising Systems

19

in the immune system) as explained in [40]. Secondly, the social insect stigmergy paradigm, because the IRS is based upon principles derived from this paradigm. In fact, Intrusion Response Agents (IR Agents) map the collective behaviour of an ant population by following a synthesised electronic pheromone specific to the detected intrusion until the source of the attack—in order to perform its response task. This pheromone has been previously diffused throughout the network by an ID Agent when it detected the attack. This kind of collective paradigm is very interesting because it consists in having each ant execute a rather light task (mobile agents play the role of ants in the IR System) to induce collectively a more complex behaviour. This approach is also very powerful because the ID System, as well as the IR System, are completely distributed in the network without any centralised control: both systems are essentially constituted by mobile agents which travel across the network, dynamically adjusting their routes according to collected events, without any simple way to trace them. Furthermore, mobile agents are quite polyvalent because they can detect and respond to intrusion. This enhances the difficulty for an attacker to distinguish between ID Agents and IR Agents. The use of AIS for addressing security has also been considered in MANET routing [69, 99]. More generally, de Castro and Timmis [26] propose a framework for engineering artificial immune systems. They provide a detailed review of the biological immune system and several artificial ones from different domains. More details on mobile agents are given in Chap. 5. Network security and the human immune system are further discussed in Chap. 16.

2.2.7 Bio-inspired Robot Teams Researchers have also been inspired by living systems to build robots. Lots of recent researches in robotics use insect-based technology where robots self-organise to accomplish a task (gathering a set of objects at a precise location for instance). As it is the case for ants’ populations, the populations of robots own a local view of their environment, they can share individual information with other robots and co-operate with them. One direct application of the self-organisation with robots is the building of a global cartography of the environment where they are immersed without having each the knowledge of the global topology of the environment. In the approach described in [97] the robots’ perception of their environment is tightly connected to their action, similarly to many successful biological systems. Robots perceive their environment locally by sending a simple visual information to their control system. The overall behaviour of the system emerges from the coordination, integration and competition between these various visual behaviours. Other examples include robot-based exploration [88], swarms of robots assembling together to create an organism similar to a multi-cell robots organism [7, 61, 63], swarms of homogeneous self-assembling robots to overcome obstacles or to transport object [5, 44], and heterogeneous cooperating robots [6].

20

G. Di Marzo Serugendo et al.

2.2.8 Manufacturing Control The self-organising behaviour exhibited in insect colonies has often been utilised in agent-based software aimed for manufacturing control [22, 23, 29, 56, 71, 89, 90]. A representative example of this approach is the PROSA3 system described in [95, 96]. PROSA is an agent-based reference software architecture, where agents are modelled as artificial ants, and the whole system emulates an ant colony. There are three basic types of agents, namely product agents, order agents and resource agents. Each agent type is responsible for one aspect of manufacturing control, respectively: (i) recipes or process plans, (ii) internal logistics and (iii) resource handling. These basic agents are structured using object-oriented concepts like aggregation and specialisation. Staff agents can be added to assist the basic agents with expert knowledge. Product agents own a ‘product model’ of a product type, and they act as information servers to other agents, delivering the right recipes to the right place. Order agents represent tasks. They are responsible for performing the corresponding work correctly and on time. They manage the physical products being produced, the product state model, and all logistic information processing related to the manufacturing operation. Resource agents provide a reflection of the underlying production system in the world of agents. They offer spaces for other agents to navigate through, to percept their neighbours and to store information that can observed by other agents and decay after a specified time, in a similar manner as the pheromones are deposited, sensed by ants and evaporate in the physical world. Virtual agents (for example order agents or other task agents created for a particular purpose) acting as artificial ants move through this reflection of the physical world and collect information, which they make available elsewhere. Firstly, these ants collect information about the available processes, travel upstream and place the collected information at routing points. Secondly, exploring ants explore possible routings for the products being made, make a selection and propagate the resulting intentions through the ‘reflection’ (see Fig. 2.2). Resource agents receive this information about the intentions of their users and compile short-term forecasts for themselves. These forecasts allow up-to-date predictions of processing times used by the ants exploring routes and propagating intentions. All these activities are subjected to an evaporation (time-out) and refresh process that enables the system to keep functioning in a highly dynamic environments (frequent changes and disturbances). An important aspect of the PROSA model is that agents can be aggregates consisting of multiple agents (see Chap. 11 for more details on holonic systems). In addition, agents can create new agents that virtually travel through the manufacturing system to create and maintain the dissipative fields that coordinate the behavior of the individual agents. Another example where self-organisation is used for manufacturing control is described in [13]. The approach followed is termed Bucket Brigades, and it aims 3 PROSA

stands for Product-Resource-Order-Staff Architecture.

2 Self-organising Systems

21

Fig. 2.2 Order agents and exploring ants in the PROSA approach

to optimise the work allocation in manufacturing systems. To this purpose, allocation of work is constantly updated by self-organising acts, each time improving load balancing and system performance. The important advantage of the achieved self-balancing work allocation is that there is no need for explicit supervision for measuring and reallocating work when needed. Furthermore, bucket brigades reduce the need for planning and management since they require no work-content model to share work effectively. Bucket brigades have been applied in various sectors of manufacturing including assembly lines [14] and distribution processes [15]. An intuitive example discussed in [13] uses bucket brigades to mitigate time and motion expenses in an assembly line by enabling it to re-balance itself when needed. The application of the bucket brigades concept on assembly lines is simple: each worker on the assembly line carries a product towards completion. However, when the last worker on the line finishes the product, he sends it off and then walks upstream to take over the work of his predecessor, who in turn walks back to take over the work of his predecessor and so on. (No overtaking is allowed.) Finally, after relinquishing his product, the first worker walks back to its starting position to begin a new product. When workers are sequenced from slowest to fastest (along direction of product flow), then it can be shown empirically, and proven mathematically, that workers spontaneously gravitate to the optimal division of work so that throughput is maximised.

2.2.9 Self-organising Sensor Networks Self-organising wireless sensor networks are used for civil and military applications, such as volcanoes, earthquakes monitoring and chemical pollution checking. Sensor networks consist of self-organised nodes, which dynamically need to set up an ad hoc P2P network, once they are deployed in a given area. They need as well to calibrate themselves in order to adapt to their environment [102]. Sensor networks benefit also from recent technology enabling integration of a complete sensor

22

G. Di Marzo Serugendo et al.

system into small-size packages, as for instance the millimetre-scaled motes provided by the SmartDust project [86]. Another representative example is the STAFF real-time simulator [42] which uses an adaptive model for flood forecast composed of two levels of self-organising multi-agent systems (representing sensor nodes). The goal of each upper level agent is to compute the water level variation during a unitary period (typically an hour) and uses for that a weighted sum of agents in the lower level. The model’s adaptive nature is obtained by adjustment of these weights, decided from the co-operation between the agents. It makes the model generic and improves its performances. Recent works on self-organising sensor networks focus on routing, synchronisation, power conservation [73] or decentralised collaborative detection of events [38].

2.2.10 Workflow Management Workflow Management Systems (WfMSs) are software applications supporting the specification, design and enactment of business processes. In the global economy, businesses are constantly changing their structure and processes according to market dynamics. Therefore WfMSs need to be able to self-organise, namely adapt their functionality to support changes needed in business operations [21]. The majority of self-organising WfMSs are based on the agent metaphor. Agents provide flexibility and adaptability, and therefore they are particularly suitable for realising self-organising infrastructures. Self-organising WfMSs implement self-organisation in three broad ways: by viewing business as Complex Adaptive Systems and applying relevant theories and models, by viewing business as holonic systems and implementing holonic system algorithms and by viewing business as self-organising organisational structures realising self-organising business models like the ones mentioned in Sect. 2.1.3. Examples of each category are given below.

2.2.10.1 Complex Adaptive WfMSs Complex adaptive WfMSs are increasingly used to support adaptive business processes with typical examples being dynamic workflow management [27] and intelligent manufacturing scheduling [85]. They are based on distributed software components executing without central top-down control and possibly in an asynchronous manner. Each component may be designed with different goals and structure such that the resulting behaviour is not practically deducible from a specification of its parts in a formal way. Such systems can exhibit emergent behaviour with unpredictable results. For example, an emergent phenomenon in complex adaptive WfMSs supporting supply chain management is that the variation in the demand experienced by a low-tier supplier is much wider than that experienced by the OEM, sometimes increasing by as much as two times from one tier to the next [79].

2 Self-organising Systems

23

In a different approach, complex adaptive WfMSs can be developed in terms of autonomous agents interacting in groups according to local rules. The global emergent behaviour results from local interactions between agents. In such systems, nonlinearity is considered as the main cause of emergent behaviour [57, 87]. There are various sources for nonlinearity in business systems. For example, in manufacturing systems three main sources of nonlinearity are capacity limits, feedback loops and temporal delays [79].

2.2.10.2 Holonic WfMSs The holonic WfMSs aim to support holonic enterprises [70]. This concept has emerged from the need for flexible open, reconfigurable models, able to emulate the market dynamics in the networked economy. Due to market dynamics, it is required that business strategies and relationships evolve over time and are modified according to the dynamic business environment. The main idea in the holonic enterprise model stems from the work of Arthur Koestler [60]. Koestler postulated a set of underlying principles to explain the selforganising tendencies of social and biological systems. Koestler has identified structural patterns of self-replicating structures in natural systems, which he named holarchies, and he proposed the term holon to describe the elements of these systems. Holarchies have been envisioned as models of the Universe’s self-organising structure, which consist of holons, at several levels of resolution in the nested hierarchy [28]. Holons behave as autonomous wholes, and yet, as cooperative parts for achieving the goal of the holarchy, and therefore holons can be regarded as nested agents. In such a nested hierarchy, each holon is a sub-system retaining the characteristic attributes of the whole system. A holonic enterprise is a holarchy of collaborative enterprise units or even individual enterprises, where each enterprise unit is regarded as a holon. Holons represent both physical and logical entities such as production departments and machines. The holonic enterprises paradigm provides a framework for information and resource management in global virtual organisations by modelling enterprise entities as software agents linked through the Internet [104]. In this parallel universe of information, enterprises enabled with the above mechanism can evolve towards better and better structures while at the same time self-organising their resources to optimally accomplish the desired objectives. Holonic WfMSs implement the holonic enterprise model considering that holonic structures are dynamically changing through self-organisation. Self-organisation is achieved using appropriate holon configuration meta-models, such as centrally optimised or supervisory configuration and dynamic reconfiguration based on experimental data [94]. To determine the optimum holonic structure, various evolutionary and genetic approaches can be used. A representative example is the fuzzy-evolutionary approach proposed by Ulieru [91]. That approach clusters the entities of a holonic enterprise into a dynamically configured optimal holonic structure and applies selforganisation and evolution models inspired from natural systems. Self-organisation

24

G. Di Marzo Serugendo et al.

is induced by minimising the entropy. Entropy minimisation is achieved by measuring the information spread across the system and acting towards reaching an equilibrium involving optimal interaction between the system’s parts. The evolution of the current system into a better one is achieved by interaction with external systems which are located via genetic search strategies. These search strategies mimick mating with most fit partners in natural evolution, and they inject new information in the system. The new optimal organisational structure of the system, reached by minimising the entropy in self-organisation, is then better than the one before evolution. Latest applications of the above approach have been reported in the design of Internet-enabled soft computing holarchies for telemedicine, for example e-Health, telehealth and telematics [92]. More details about self-organisation of holonic structures are provided in Chap. 11.

2.2.10.3 Self-organising Workflow Models Self-organising business models are frequently used to implement self-organisation in WfMSs. Such models are implemented inherently in the software infrastructure. For example, Helal and Wang [47] propose an agent-based infrastructure and appropriate interaction protocols that would be useful in negotiating service bundles for the different services offered by agents. In that approach, agents are organised in e-business communities (Malls). Within a community, highly relevant agents group together offering special e-Services for a more effective, mutually beneficial and more opportune e-Business. Self-organisation is based on a hierarchy of brokers which control membership and operations of the different communities. The aim of the broker hierarchy is to achieve scalability of the system and interoperability of the various heterogeneous e-business components. Furthermore, in many cases pre-existing business and social models are modified to use the benefits of self-organisation. For example, interest by IBM, as part of their Autonomic Computing [58] program, and by Microsoft, as part of the Dynamic Systems Initiative [72], indicates the importance of self-organisation for managing distributed resources and shaping the way business is carried out in dynamic environments. In addition, applications based on invisible intelligent technology have started being embedded in clothes, walls and cars, enabling users to carry out completely new operations such as personalised virtual shopping, e-purse-based micropayments and context-based traffic guidance [35]. The pervasive, large-scale and embedded nature of these applications leads naturally to design them based on selforganisation so that to handle the vital issues of robustness and adaptability among others.

2.3 A FAQ About Self-organisation and Emergence This section attempts to summarise answers to fundamental questions about selforganising systems and to provide our view on the subject. The question–answer

2 Self-organising Systems

25

Table 2.1 A FAQ about self-organising systems Question

Answer

What is Self-organisation?

Self-organisation is the process with which a system changes its structure without any external control to respond to changes in its operating conditions and its environment.

What is Emergence?

Emergence is the phenomenon where a nonpredetermined outcome, such as a structure or a state, is reached progressively following multiple self-organisation acts of the system.

When is a system Self-organising?

A system is self-organising when it is able to apply selforganisation and emergence in order to function as needed in a dynamic environment.

What are the Self-* properties?

The Self-* properties are system capabilities which can be exercised without any external control. They are all covered by the more generic term ‘self-organisation’. Examples of self-* properties are self-management, self-healing and self-adaptation.

What are examples of natural systems showing self-organising behaviour?

Typical examples of self-organising systems are the human immune system, ant and bee swarms, schools of fish and flocks of birds.

What are examples of applications of self-organising software?

Typical examples of self-organising software applications include adaptive manufacturing control, P2P systems, network coordination and security, collective robotics and workflow management.

What are the main benefits of selforganisation?

The main benefits of self-organisation are increased robustness, scalability and individual unit communication and processing costs. Furthermore, self-organisation enables the engineering of underspecified software.

What are the main drawbacks of selforganisation?

The main drawbacks of self-organisation include the possibility of occurrences of opposing actions among system units, the appearance of needless redundancies in unit behaviours and the lack of guarantees for finding the globally optimum solution.

format used is a very effective way to provide a succinct introduction to the subject of self-organising software. See Table 2.1 above.

2.4 Conclusion Self-organising systems have initially been observed in nature, one of the most cited example being the termites behaviour following a stigmergy mechanism reported by Grassé. Their appealing characteristics is the relative simple behaviour of the indi-

26

G. Di Marzo Serugendo et al.

vidual system’ components when compared to the complex collective behaviour they produce. As artificial software systems become more difficult to manage and maintain, researchers and industrials have started to take inspiration from those systems and to translate their underlying mechanisms into artificial systems. Many applications domains have been considered ranging from static and dynamic optimisation problems (resource allocation, manufacturing control) based on ant foraging behaviour, to network security following basic immune systems principles, to workflow management for the holonic enterprise. This chapter provided a short introduction to several natural and artificial systems highlighting their specificities.

2.5 Problems–Exercises 2.1 Given the initial understanding of the self-organisation and emergence concepts obtained after reading this chapter: (a) Provide three examples of natural systems where self-organisation and emergence are inherently applied. (b) For the provided examples, describe how self-organisation is carried out. For example, in the case of ant foraging self-organisation is carried out using stigmergy. (c) For the provided examples, describe what are the emergent outcomes or states where applicable. For example, in the case of foraging ants an emergent outcome is the path between nest and food source which is formed by foraging ants. 2.2 Consider a system consisting of an automatic door and a sensor. The door opens automatically when the sensor identifies some movement within a specific distance and radius. If there is no movement, the door remains in the closed position or returns to it if it was open. (a) Do you consider this system as self-organising? Justify your answer. (b) Identify the emergent outcomes that can possibly result from system operation, if any. 2.3 Provide three examples of application areas where self-organising software is used. For each one: (a) Discuss the benefits gained from using self-organisation as compared to using a traditional system. (b) Identify conditions where the use of a deterministic, centralised system would be more appropriate.

2 Self-organising Systems

27

Key Points • Introduction to well-known natural self-organising systems, such as ant foraging, chemical reactions, immune systems or business systems; • Introduction to self-organising application domains, such as, among others, Computational Grids, network security, swarm robotics or sensor networks; • An FAQ table for a quick review of introductory notions.

2.6 Further Reading The Origins of Order: Self-organization and Selection in Evolution. A foundational introduction to self-organisation and emergence in complex systems. (S. Kaufman, 1993, Oxford University Press.) Swarm Intelligence: From Natural to Artificial Systems. A comprehensive book on self-organising algorithms drawn from natural systems. (E. Bonabeau, M. Dorigo and G. Theraulaz, 1999, Oxford University Press.) Self-organization in Biological Systems. A detailed presentation of self-organisation mechanisms in biological systems. (S. Camazine, J.-L. Deneubourg, N.R. Franks, J. Sneyd, G. Theraulaz and E. Bonabeau, 2001, Princeton University Press.) Fundamentals of Natural Computing: Basic Concepts, Algorithms, and Applications. A comprehensive introduction, including appropriate algorithms, to evolutionary computing, swarm intelligence and immune systems. (Leandro Nunes de Castro, 2006, Chapman and Hall.) The many Facets of Natural Computing. An up-to-date review article on recent advances in natural computing. (L. Kari and G. Rosenberg, 2008, Communications of the ACM, 51(10):72–83.) Self-managed Systems and Services. An overview of the uses of self-organisation in implemented systems and services. (J.P. Martin-Flatin, J. Sventek and K. Geihs, 2006, Guest Editorial of special issue, Communications of the ACM, 49(3): 36–39.) That special issue includes additional relevant papers as well. Emergent computation: self-organizing, collective, and cooperative phenomena in natural and artificial computing network. An overview of emergence in computing systems. (Stephanie Forrest, 1991, Special issue of Physica D, MIT Press.) More foundational readings are provided at the end of Chap. 3.

References 1. http://www.entropia.com (2003)

28 2. 3. 4. 5. 6. 7. 8.

9. 10.

11. 12. 13. 14. 15. 16.

17. 18. 19. 20.

21. 22. 23.

24.

25. 26. 27.

28.

G. Di Marzo Serugendo et al. http://www.gnutella.com (2003) http://www.kazaa.com (2003) http://www.ud.com (2003) http://www.swarm-bots.org/ (2005) http://www.swarmanoid.org/ (2010) http://www.symbrion.eu/ (2013) Aberer, K., Datta, A., Hauswirth, M.: P-Grid: dynamics of self-organizing processes in structured peer-to-peer systems. In: Peer-to-Peer Systems and Applications. LNCS, vol. 3485, pp. 137–153. Springer, Berlin (2005) Appleby, S., Steward, S.: Mobile software agents for control in telecommunications networks. BT Technol. J. 12(2), 104–113 (1994) Babaoglu, O., Meling, H., Montresor, A.: Anthill: a framework for the development of agentbased peer-to-peer systems. In: 22th International Conference on Distributed Computing Systems (ICDCS’02), pp. 15–22. IEEE Computer Society, Los Alamitos (2002) Bar-Yam, Y.: Dynamics of Complex Systems. Perseus Books, Cambridge (1997) Barabási, A., Albert, R.: Emergence of scaling in random networks. Science 286(5439), 509– 512 (1999) Bartholdi, J.J., Eisenstein, D.D.: A production line that balances itself. Oper. Res. 44(1), 21–34 (1996) Bartholdi, J.J., Eisenstein, D.D.: Using bucket brigades to migrate from craft manufacturing to assembly lines. Manuf. Serv. Oper. Manag. 7(2), 121–129 (2005) Bartholdi, J.J., Eisenstein, D.D., Foley, R.D.: Performance of bucket brigades when work is stochastic. Oper. Res. 49(5), 710–719 (2001) Bonabeau, E., Henaux, F., Guerin, S., Snyers, D., Kuntz, P.: Routing in telecommunications networks with “smart” ant-like agents. In: Intelligent Agents for Telecommunications Applications ’98 (IATA’98) (1998) Bonabeau, E., Dorigo, M., Théraulaz, G.: Swarm Intelligence: From Natural to Artificial Systems. Oxford University Press, London (1999) Bonabeau, E., Dorigo, M., Theraulaz, G.: Inspiration for optimisation from social insect behaviour. Nature 406, 39–42 (2000) Brooks, R.: Artificial life: from robot dreams to reality. Nature 406, 945–947 (2000) Brueckner, S., Parunak, H.V.D.: Self-organising manet management. In: Serugendo, G., Karageorgos, A., Rana, O.F., Zambonelli, F. (eds.) Engineering Self-Organising Systems, Nature-Inspired Approaches to Software Engineering. Lecture Notes in Artificial Intelligence, vol. 2977, pp. 20–35. Springer, Berlin (2004) Buhler, P.A., Vidal, J.M.: Towards adaptive workflow enactment using multiagent systems. Inf. Technol. Manag. 6(1), 61–87 (2005). doi:10.1007/s10799-004-7775-2 Bussmann, S., Jennings, N.R., Wooldridge, M.: Multiagent Systems for Manufacturing Control. Springer, Berlin (2004) Buyurgan, N., Meyyappan, L., Saygin, C., Dagli, C.H.: Real-time routing selection for automated guided vehicles in a flexible manufacturing system. Int. J. Manuf. Technol. Manag. 18(2), 169–181 (2007) Camazine, S., Deneubourg, J.L., Franks, N.R., Sneyd, J., Théraulaz, G., Bonabeau, E.: SelfOrganization in Biological Systems. Princeton University Press, Princeton (2001). 2nd edn. (2003) Cardelli, L.: Machines of systems biology. In: Bulletin of the EATCS 93, pp. 176–204 (2007) Castro, L.D., Timmis, J.: Artificial Immune Systems: A New Computational Intelligence Approach. Springer, Berlin (2002) Chen, Q., Hsu, M., Dayal, U., Griss, M.: Multi-agent cooperation, dynamic workflow and XML for e-commerce automation. In: Sierra, C., Gini, M., Rosenschein, J.S. (eds.) Proceedings of the Fourth International Conference on Autonomous Agents, Barcelona, Catalonia, Spain, pp. 255–263. ACM, New York (2000) Christensen, J.H.: Holonic manufacturing systems: initial architecture and standards directions. In: Proceedings of the First European Conference on Holonic Manufacturing Systems (1994)

2 Self-organising Systems

29

29. Clair, G., Kaddoum, E., Gleizes, M.P., Picard, G.: Self-regulation in self-organising multiagent systems for adaptive and intelligent manufacturing control. In: Second IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO’08, 20–24 October 2008, pp. 107–116. IEEE Computer Society, Los Alamitos (2008) 30. Craig, W.R.: Flocks, herds and schools: a distributed behavioral model. SIGGRAPH Comput. Graph. 21(4), 25–34 (1987) 31. Deneubourg, J.L., Goss, S., Franks, N., Sendova-Franks, A., Detrain, C., Chretien, L.: The dynamics of collective sorting robot-like ants and ant-like robots—simulation of animal behaviour. In: Meyer, J., Wilson, S. (eds.) First International Conference on Simulation of Adaptive Behaviour, pp. 356–363. MIT Press, Cambridge (1991) 32. Di Caro, G., Dorigo, M.: Ant colonies for adaptive routing in packet switched communication networks. In: Proceedings of PPSN V—Fifth International Conference on Parallel Problem Solving from Nature. Lecture Notes in Computer Science. Springer, Berlin (1998) 33. Di Marzo Serugendo, G., Karageorgos, A., Rana, O.F., Zambonelli, F. (eds.): Engineering Self-Organising Systems: Nature-Inspired Approaches to Software Engineering. Lecture Notes in Computer Science, vol. 2977. Springer, Berlin (2004) 34. Di Marzo Serugendo, G., Gleizes, M.P., Karageorgos, A.: Self-organisation in multi-agent systems. Knowl. Eng. Rev. 20(2), 165–189 (2005) 35. Ducatel, K., Bogdanowicz, M., Scapolo, F., Leitjen, J., Burgelman, J.C.: Scenarios for ambient intelligence in 2010. Technical Report, Institute for Prospective Technological Studies (2001) 36. Endy, D.: Foundations for engineering biology. Nature 438, 449–453 (2005) 37. Faloutsos, M., Faloutsos, P., Faloutsos, C.: On power-law relationships of the Internet topology. In: Proceedings of the Special Interest Group on Data Communications (ACM SIGCOMM’99), pp. 251–262. ACM, New York (1999) 38. Fernandez-Marquez, J.L., Arcos, J.L., Di Marzo Serugendo, G.: A decentralized approach for detecting dynamically changing diffuse event sources in noisy WSN environments. In: International Conference on Self-Adaptive and Self-Organising Systems (SASO’10). IEEE Computer Society, Los Alamitos (2010) 39. Flake, G.W., Lawrence, S., Giles, C.L., Coetzee, F.M.: Self-organisation and identification of web communities. Computer 35(3), 66–71 (2002) 40. Forrest, S., Hofmeyer, S.A., Somayaji, A., Longstaff, T.A.: A sense of self for Unix processes. In: Proceedings of the IEEE Symposium on Research in Security and Privacy, pp. 120–128. IEEE Computer Society, Los Alamitos (1996) 41. Foukia, N.: IDReAM: intrusion detection and response executed with agent mobility. In: International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS’05), Utrecht, The Netherlands, pp. 264–270. IEEE Press, New York (2005) 42. Georgé, J.P., Peyruqueou, S., Régis, C., Glize, P.: Experiencing self-adaptive MAS for realtime decision support systems. In: Demazeau, Y., Pavón, J., Corchado, J., Bajo, J. (eds.) 7th International Conference on Practical Applications of Agents and Multi-Agent Systems (PAAMS 2009). Advances in Intelligent and Soft Computing, vol. 55, pp. 302–309. Springer, Berlin (2009) 43. Grassé, P.P.: La reconstruction du nid et les coordinations interindividuelles chezbellicositermes natalensis etcubitermes sp la théorie de la stigmergie: Essai d’interprétation du comportement des termites constructeurs. Insectes Soc. 6(1), 41–80 (1959). doi:10.1007/ BF02223791 44. Gross, R., Dorigo, M.: Towards group transport by swarms of robots. Int. J. Bio-Inspired Comput. 1(1/2), 1–13 (2009) 45. Haken, H.: Synergetics: an introduction. Springer, Berlin (1978), reedited (1983) 46. Hardaker, G., Graham, G.: Energizing your e-commerce through self-organising collaborative marketing networks. Tech. Rep. (2002) 47. Helal, S., Wang, M.: Service centric brokering in dynamic e-business agent communities. J. Electron. Commer. Res. 2(1), 32–47 (2001). Special Issue on Intelligent Agents in Ecommerce

30

G. Di Marzo Serugendo et al.

48. Heylighen, F.: The science of self-organization and adaptivity. In: Kiel, L.D. (ed.) Knowledge Management, Organizational Intelligence and Learning, and Complexity, The Encyclopedia of Life Support Systems, EOLSS, pp. 253–280. Eolss Publishers, Oxford (2001) 49. Holland, O., Melhuis, C.: Stigmergy, self-organisation and sorting in collective robotics. Artif. Life 5(2), 173–202 (1999) 50. Horst, F.W., Muddassar, F.: A comprehensive review of nature inspired routing algorithms for fixed telecommunication networks. J. Syst. Archit. 52(8), 461–484 (2006) 51. Hulaas, J., Binder, W., Di Marzo Serugendo, G.: Enhancing Java grid computing security with resource control. In: International Conference on Grid Services Engineering and Management (GSEM’04) (2004) 52. Itao, T., Suda, T., Aoyama, T.: Jack-in-the-net: adaptive networking architecture for service emergence. In: Asian-Pacific Conference on Communications (2001) 53. Jelasity, M., Babaoglu, O.: T-man: Gossip-based overlay topology management. In: Engineering Self-Organizing Systems. LNCS, vol. 3910, pp. 1–15. Springer, Berlin (2006) 54. Jennings, N.R., Wooldridge, M.J. (eds.): Agent Technology: Foundations, Applications and Markets. Springer, Berlin (1998). This is a cross-referenced BOOK (collection) entry 55. Kari, L., Rosenberg, G.: The many facets of natural computing. Commun. ACM 51(10), 72–83 (2008) 56. Karuna, H., Valckenaers, P., Saint-Germain, B., Verstraete, P., Zamfirescu, C.B., Van Brussek, H.: Emergent forecasting using stigmergy approach in manufacturing coordination and control. In: Engineering Self-Organizing Applications (ESOA’04), pp. 210–226. Springer, Berlin (2004) 57. Kaufman, S.: At Home in the Universe: The Search for the Laws of Self-Organization and Complexity. Oxford University Press, London (1995) 58. Kephart, J.O., Chess, D.M.: The vision of autonomic computing. Computer 36(1), 41–50 (2003). doi:10.1109/MC.2003.1160055 59. Ko, S.Y., Gupta, I., Jo, Y.: Novel mathematics-inspired algorithms for self-adaptive peer-topeer computing. In: International Conference on Self-Adaptive and Self-Organising Systems (SASO’07) (2007) 60. Koestler, A.: The Ghost in the Machine. Hutchison and Co, London (1967), the Danube edn. 61. Levi, P., Kernbach, S. (eds.): Symbiotic Multi-Robot Organisms. Cognitive Systems Monographs, vol. 7. Springer, Berlin (2010) 62. Li, Y., Chen, F., Sun, X., et al.: Self-adaptive resource management for large-scale shared clusters. J. Comput. Sci. Technol. 25(5), 945–957 (2010) 63. Luhmann, N.: Familiarity, confidence, trust: problems and alternatives. In: Gambetta, D. (ed.) Trust: Making and Breaking Cooperative Relations, pp. 213–237 (2000). Chap. 13. Published Online http://www.sociology.ox.ac.uk/papers/gambetta213-237.pdf 64. Lynden, S., Rana, O.F.: Coordinated learning to support resource management in computational grids. In: Second IEEE Conference on Peer-to-Peer Computing (P2P 2002), pp. 81–89. IEEE Computer Society, Los Alamitos (2002) 65. Mamei, M., Vasirani, M., Zambonelli, F.: Self-organizing spatial shapes in mobile particles: the TOTA approach. In: Brueckner, S., Di Marzo Serugendo, G., Karageorgos, A., Nagpal, R. (eds.) Engineering Self-Organising Systems. Lecture Notes in Computer Science, vol. 3464, pp. 138–153. Springer, Berlin (2005) 66. Martin-Flatin, J., Sventek, J., Geihs, K.: Self-managed systems and services. Commun. ACM 49(3), 36–39 (2006) 67. Martino, S.: A mobile agent approach to intrusion detection. Tech. Rep., Joint Research Center—Institute for Systems, Informatics and Safety (1999) 68. Maturana, H.R., Varela, F.J.: Autopoiesis and Cognition: The Realization of the Living. Boston Studies in the Philosophy of Science. Springer, Berlin (1991). 2nd edn. (1980), 1st edn. (1973) 69. Mazhar, N., Farooq, M.: BeeAIS: artificial immune system security for nature inspired, MANET routing protocol, BeeAdHoc. In: Artificial Immune Systems. LNCS, vol. 4628, pp. 370–381. Springer, Berlin (2007)

2 Self-organising Systems

31

70. McHugh, P., Merli, G., Wheeler, W.A.: Beyond Business Process Reengineering: Towards the Holonic Enterprise. Wiley, New York (1995) 71. Meyyappan, L., Soylemezoglu, A., Saygin, C., Dagli, C.H.: A wasp-based control model for real-time routing of parts in a flexible manufacturing system. Int. J. Comput. Integr. Manuf. 21(3), 259–268 (2008) 72. Microsoft: Dynamic systems 2007: Get started with dynamic systems technology today. Tech. Rep. (2007) 73. Mills, K.: A brief survey of self-organization in wireless sensor networks. Wirel. Commun. Mob. Comput. 7, 823–834 (2007) 74. Montresor, A., Meling, H., Babaoglu, O.: Messor: load-balancing through a swarm of autonomous agents. In: Moro, G., Koubarakis, M. (eds.) Agents and Peer-to-Peer Computing. Lecture Notes in Artificial Intelligence, vol. 2530, pp. 125–137. Springer, Berlin (2003) 75. Nedelec, F., Surrey, T., Karsenti, E.: Self-organisation and forces in the microtubule cytoskeleton. Curr. Opin. Cell Biol. 15(2), 118–124 (2003) 76. Nicolis, G., Prigogine, I.: Self-Organization in Non-Equilibrium Systems. Wiley, New York (1977) 77. O’Hare, G.M.P., Jennings, N.R. (eds.): Foundations of Distributed Artificial Intelligence. Software Agents. Wiley, London (1996) 78. Parunak, H.V.D.: Synthetic pheromone mechanisms for coordination of unmanned vehicles. In: AAMAS, Bologna, Italy (2002) 79. Parunak, H.V.D., VanderBok, R.S.: Managing emergent behavior in distributed control systems. In: ISA-Tech’97, Anaheim (1997) 80. Picard, G., Gleizes, M.P.: The adelfe methodology: designing adaptive cooperative multiagent systems. In: Bergenti, F., Gleizes, M.P., Zambonelli, F. (eds.) Methodologies and Software Engineering for Agent Systems. The Agent-Oriented Software Engineering Handbook, vol. 11. Kluwer Academic, Amsterdam (2004) 81. Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach. Prentice Hall Series. Prentice Hall, New York (1995) 82. Schoonderwoerd, R., Holland, O., Bruten, J.: Ant-like agents for load balancing in telecommunications networks. In: 1st International Conference on Autonomous Agents, pp. 209– 216. IEEE Computer Society, Los Alamitos (1997) 83. Searle, J.R.: The Rediscovery of the Mind. MIT Press, Cambridge (1992) 84. Seeley, T.D.: When is self-organization used in biological systems? Biol. Bull. 202, 314–318 (2002) 85. Shen, W., Norrie, D.H.: Agent-based systems for intelligent manufacturing: a state-of-the-art survey. Knowl. Inf. Syst. 1(2), 129–156 (1999) 86. Smartdust project. http://robotics.eecs.berkeley.edu/~pister (2003) 87. Stewart, M.: The Coevolving Organization. Decomplexity Associates LtD, Rutland (2001) 88. Thakoor, S., Morookian, J.M., Chahl, J., Hine, B., Zornetzer, S.: Bees: exploring mars with bioinspired technologies. Computer 37(9), 36–47 (2004) 89. Ueda, K.: Emergent synthesis approaches to biological manufacturing systems. In: Cunha, P.F., Maropoulos, P.G. (eds.) Digital Enterprise Technology: Perspectives and Future Challenges, pp. 25–34. Springer, New York (2007) 90. Ueda, K., Markus, A., Monostori, L., Kals, H.J.J., Arai, T.: Emergent synthesis methodologies for manufacturing. CIRP Ann. Manuf. Technol. 50(2), 535–551 (2001). doi:10.1016/ S0007-8506(07)62994-1 91. Ulieru, M.: Emergence of holonic enterprises from multi-agent systems: a fuzzy evolutionary approach. In: Soft Computing Agents, vol. 13, pp. 187–215. IOS Press, Amsterdam (2002) 92. Ulieru, M.: Internet-enabled soft computing holarchies for e-health applications (soft computing enhancing the Internet and the Internet enhancing soft computing). In: Zadeh, L.A., Nikravesh, M. (eds.) New Directions in Enhancing the Power of the Internet. Springer, Berlin (2003) 93. Ulieru, M.: Adaptive information infrastructures for the e-society. In: Brueckner, S., Serugendo, G.D.M., Karageorgos, A., Nagpal, R. (eds.) Engineering Self-Organising Systems:

32

94. 95.

96.

97. 98. 99.

100.

101. 102. 103. 104.

G. Di Marzo Serugendo et al. Methodologies and Applications. Lecture Notes in Artificial Intelligence, vol. 3464, pp. 32– 51. Springer, Berlin (2005) Valckenaers, P., Brussel, H.V.: Holonic manufacturing execution systems. CIRP Ann. Manuf. Technol. 54(1), 427–432 (2005) Valckenaers, P., Brussel, H.V., Kollingbaum, M., Bochmann, O.: Multi-agent coordination and control using stigmergy applied in manufacturing control. In: Luck, M., Marik, V., Stepankova, O., Trappl, R. (eds.) Multi-Agent Systems and Applications, 9th ECCAI Advance Cource ACAI 2001 and AgentLink’s 3rd European Agent Systems Summer School, EASS 2001, Selected Tutorial Papers, vol. 2086, pp. 317–334. Springer, Berlin (2001) Van Brussel, H., Wyns, J., Valckenaers, P., Bongaerts, L., Peeters, P.: Reference architecture for holonic manufacturing systems: Prosa. Comput. Ind. 37(3), 255–274 (1998). doi:10.1016/ S0166-3615(98)00102-X Visual behaviours for mobile robots project. http://www.isr.ist.utl.pt/vislab/projects.html (2003) Viroli, M., Zambonelli, F.: A biochemical approach to adaptive service ecosystems. Inf. Sci. 180(10), 1876–1892 (2010) Visconti, A., Tahayori, H.: Detecting misbehaving nodes in MANET with an artificial immune system based on type-2 fuzzy sets. In: IEEE Proceedings of the 4th International Conference for Internet Technology and Secured Transactions (ICITST09). IEEE Computer Society, Los Alamitos (2009) Vittikh, V.A., Skobelev, P.O.: Multi-agent systems for modelling of self-organization and cooperation processes. In: XIII International Conference on the Application of Artificial Intelligence in Engineering, Galway, Ireland, pp. 91–96 (2002) Werbach, K.: Syndication—the emerging model for business in the Internet era. Harvard Bus. Rev. 78, 85–93 (2000) Wokoma, I., Sacks, L., Marshall, I.: Biologically inspired models for sensor network design. In: London Communications Symposium (2002) Wrzesinska, G., Maassen, J., Bal, H.: Self-adaptive applications on the Grid. In: 12th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (2007) Zhang, X., Norrie, D.H.: Holonic control at the production and controller levels. In: Proceedings of the 2nd International Workshop on Intelligent Manufacturing Systems, pp. 215–224 (1999)

Chapter 3

History and Definitions Giovanna Di Marzo Serugendo, Marie-Pierre Gleizes, and Anthony Karageorgos

Defining and implementing Self-organisation and Emergence.

Objectives The objective of this chapter is to discuss the origins and definitions of self-organisation and emergence. When you will have read this chapter, you will: • Obtain a clear understanding of the terms self-organisation and emergence and the pros and cons of the various definitions. • Have been introduced to most common self-organising system properties. • Understand technical and theoretical issues which are important for effectively using self-organisation and emergence in software applications.

3.1 Introduction The issue of developing large-scale systems consisting of several (simple) components which, without explicit central control from some external authority, would G. Di Marzo Serugendo Birkbeck College, University of London, London, UK e-mail: [email protected] M.-P. Gleizes IRIT, Université Paul Sabatier, Toulouse, France e-mail: [email protected] A. Karageorgos () Technological Educational Institute of Larissa, Larissa, Greece e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_3, © Springer-Verlag Berlin Heidelberg 2011

33

34

G. Di Marzo Serugendo et al.

collectively produce some complex global result has long been the subject of many research efforts. That global result, albeit complex, would not be chaotic or disordered, but instead it would be liable to some type of control laws, and it would demonstrate some observable properties. In other words, that result would be a desired identifiable organisation, for example a pattern, a process producing that pattern, or a global system function ensuring that the system would fulfil some specific requirements. Such a process and the resulting outcome are commonly referred to as self-organisation and emergent phenomenon respectively. The occurrence of an emergent phenomenon is commonly referred to as emergence. There has been recently a great interest in the study, analysis and design of systems capable of producing a collective outcome via self-organisation and emergence [14, 15, 29, 32, 72, 77]. In this context, of particular interest is the ability of self-organising systems to have a complex collective response arising from interactions among relatively simple individual components with limited abilities [54, 78]. The terms involved, for example organisation, self-organisation and emergence, have been used in various disciplines, such as mathematics, physics, biology and philosophy. These terms are increasingly used in computer science, and in particular they are widely met in the MAS research community. We need to establish a common, consensual and operational meaning of them to enable researchers to know if their artificial systems are self-organising or not and if there is emergent phenomenon or not. To this purpose, Sects. 3.2 and 3.3 define self-organisation and emergence from the view of self-organising software, and in addition they provide an overview of the most common other definitions available in the literature.

3.2 Self-organisation 3.2.1 History The concept of self-organisation has been first discussed in ancient Greek philosophy (see Haken in [41] referring to Paslack in [66]). In early modern times (18th and 19th centuries), self-organisation was discussed by the German philosophers I. Kant and F.W.J. Schelling [66]. Kant related self-organisation particularly with the formation of the planetary system, while Schelling’s treatment of the subject was, as Haken states in [41], rather vague. In more recent times the notion of “self-organising system” is traced back to the cybernetician W. Ross Ashby, who first used it in 1947 (see [41, 68] referring to [3]). By early 1950s, the term self-organisation had been well established in modern sciences by Nobel Prize winner Ilya Prigogine and his colleagues through studies in thermodynamics [35]. The concept of self-organisation introduced by Prigogine was essentially based on the assumption that open systems aim to decrease their entropy (order results from disorder) when some external energy is applied to them. The additional energy imposed often causes matter within the system to alter its organisational structure to facilitate the system reaching a new state with lower entropy.

3 History and Definitions

35

The export of entropy does not directly explain how or why self-organisation does take place. Prigogine noted that self-organisation typically takes place in nonlinear systems which are far from their thermodynamic equilibrium state. That statement was supported by concrete observations of self-organisation phenomena which occur in physical systems, and it was accompanied with classical examples such as the Benard phenomenon1 example. The observations of self-organisation cases in physical systems were complemented with more abstract and high-level analyses of self-organising behaviour in complex autonomous systems which were carried out by cyberneticians. These efforts resulted in the first conference on self-organising systems which was held in 1959 in Chicago and, as stated by Heylighen [43], was organised by the same multidisciplinary group of visionary scientists who had originally founded the discipline of cybernetics. Among the conference outcomes was the introduction of three fundamental principles concerning self-organising systems: • According to the “principle of self-organisation”, introduced by W. Ross Ashby, a dynamical system, independently of its type or composition, always tends to evolve towards a state of equilibrium. That evolution towards an equilibrium reduces the uncertainty regarding the state of the system, and therefore it decreases the system’s statistical entropy. The resulting equilibrium can be interpreted as a state where the different parts of the system are mutually adapted. Such an equilibrium state has since been commonly known as an attractor [45, 52]. • Another cybernetician, Heinz von Foerster, formulated the principle of “order from noise”. Foerster noted that, paradoxically, the larger the random perturbations (“noise”) that affect a system, the more quickly it will self-organise (produce “order”). This can be intuitively explained as follows: the more widely a system is made to move through its state space, the more quickly it will end up in an attractor. If the system was not forced to move between states, no attractor state would ever be possible to be reached, and hence no self-organisation could take place. • In addition to the above, Prigogine proposed the related principle of “order through fluctuations”. According to that principle, nonlinear systems generally have several attractor states. When a system resides in between attractors, it will be in general a chance variation, commonly known as a “fluctuation” in thermodynamics [35], that will force it to move and stabilise in some particular attractor state. At the same time further work on self-organisation was done in social systems. A representative example is the theory of stigmergy which was introduced by Grassé in 1959 after studying the social behaviour of termites [37]. Grassé summarised the theory of stigmergy in the phrase “the work excites the workers” meaning that the philosophy of stigmergy is to have members of a society attracted to places in the system where most activity has recently taken place. As a consequence, the activity at these points will be increased even further until the causes of the activity, 1 The

Benard phenomenon refers to the appearance of hexagonal cells or pattern rolls in liquids when heated from below. See Heylighen [43] for more detail.

36

G. Di Marzo Serugendo et al.

for example the food availability when the goal of the societal activity is to collect food, at these points no longer exist. In this case, self-organisation refers to changes to the points where society members carry out their activities, which are dictated by attractions due to stigmergy. A main aspect of the theory of stigmergy is that interaction between society members can be effectively carried out through the environment and therefore direct communication is not mandatory. As a consequence, no direct interactions are necessary to coordinate a social group, and coordination and regulation tasks in a society can be realised based on information deposited into the environment without central control. In the case of ant colonies, stigmergy is realised by depositing chemical substances in the environment, which, as already mentioned in Sect. 2.1.2.3, are termed pheromones. There is a fundamental difference between the concept of self-organisation introduced by Prigogine and the one suggested by Grassé. In the first case, selforganisation is the result of some external energy imposed to the system from its environment. In the second one, self-organisation is the result of internal actions of elements from within the system itself (for example in the case of an ant colony self-organisation actions initiate from the ants themselves which implement the stigmergy mechanism by depositing and following pheromones). In further developments, Koestler established in late 1960s the definition of holons and holarchies [55]. As mentioned in Sect. 2.2.10.2 and detailed in Chap. 4, holons are both whole systems and parts of larger systems at the same time. Holarchies are hierarchies of such holons. Koestler presents a hierarchical view of self-organisation, which is applicable in a variety of cases ranging from enterprise organisations to the universe as a whole. The idea here is that order can result from disorder with progressive reorganisation of relations between complex structural elements at higher levels of abstraction. That high-level reorganisation results from statistically evolving simple relations between system elements at lower levels. More specifically, Koestler defined holons as autonomous, self-reliant units that possess a degree of independence and handle contingencies without asking higher authorities for instructions. These holons are also simultaneously subject to control from one or more of these higher authorities. The first property ensures that holons are stable forms that are able to withstand disturbances, while the latter property signifies that they are intermediate forms, providing a context for the proper functionality for the larger whole. Holarchies consist of self-regulating holons that function first as autonomous wholes in supra-ordination to their parts, secondly as dependent parts in sub-ordination to controls from higher levels, and thirdly in coordination with their local environment. In the 1970s Maturana and Varela, aiming to characterise the nature of living systems, introduced the notion of autopoiesis (literally meaning “auto (self)creation”) as the process with which an organisation is able to reproduce itself [61]. Autopoiesis applies to closed systems, such as living organisms, consisting of autonomous components that interact and collectively (re-)produce higher-level components within the system. For example, the eukaryotic cell is made of various biochemical components such as nucleic acids and proteins, and is organised into bounded structures such as the cell nucleus, various organelles, a cell membrane and

3 History and Definitions

37

cytoskeleton. These structures, based on an external flow of molecules and energy, produce the components which, in turn, continue to maintain the organised bounded structure that gives rise to these components. An autopoietic system is to be contrasted with an allopoietic system, such as a car factory, which uses raw materials (components) to generate a car (an organised structure) which is something other than itself (the factory). The notion of autopoiesis is often associated with that of self-organisation. The reason is that the dynamics of autopoietic systems resemble those of nonequilibrium systems; that is they include states, often called dissipative structures [43], which remain stable for long periods of time despite matter and energy continually flowing through them. However, an autopoietic system is autonomous and operationally closed, in the sense that every process within it directly contributes to maintaining the whole. Furthermore, autopoietic system elements are structurally coupled with their medium in a dynamic relationship that can be parallelised to sensory-motor coupling. In addition, autopoiesis presupposes an organised “self” [62], which, as Maturana argued, “arises with the system” and therefore “cannot organise itself ” (see [21] referring to Maturana). Therefore, self-organisation is not autopoietic in its strict sense. Furthermore, since the mechanisms for selforganisation are inherent in the system itself and not modulated by an external organisation, it is commonly accepted that it is the system that organises itself [22]. In the 1980s the first attempts to apply self-organisation concepts to various domains were made. One such attempt is the development of “neural networks”, which are simplified computer models (networks of artificial “neurons” linked with each other either directly or indirectly) mimicking the way brain neurons are linked and interact. There is no centralised control in neural networks, but they are capable of processing complex patterns of input. Another example is the production of laser light. Laser light beams result from the emission of photons of the same type, at the same time and at the same direction from atoms or molecules that are excited by an input of energy, and are synchronised under particular circumstances [43]. Based on observations of such collective results resulting from cooperation (or synergy) between system components, the German physicist H. Haken proposed the field of Synergetics to study such phenomena [41]. In the mid-1980s the proliferation of inexpensive and powerful computers enabled the use of computer simulation to explore self-organising system models of various degrees of complexity. Such systems typically included large numbers of interacting components and were difficult to be mathematically modelled. Based on this approach, a number of researchers associated with the Santa Fe Institute in New Mexico pioneered the domain of Complex Adaptive Systems (CAS) [45, 52]. CAS consist of many interacting components, which undergo constant change, both autonomously and in interaction with their environment. The behaviour of such complex systems is typically unpredictable, yet exhibits various forms of adaptation and self-organisation [47]. The components of CAS are typically considered to be agents [44, 47]. Examples of CAS include natural ecosystems comprising organisms which belong to many different species and interact with each other (compete or cooperate) and their shared physical environment. Additional CAS examples are markets,

38

G. Di Marzo Serugendo et al.

where different producers compete and exchange money and goods with consumers in a distributed and autonomous fashion. Despite markets being highly chaotic and nonlinear systems, they tend to converge towards approximate equilibria where conflicting goals of producers and consumers are mutually adapted and on average satisfied. This balancing mechanism, achieved by market dynamics, which Adam Smith, the father of economics, called “the invisible hand”, can therefore be characterised as self-organisation [43]. The methods and tools used for the study of CAS enabled research in various related areas. The biologist S. Kauffman studied the development of organisms and ecosystems. Through simulations, he showed that sufficiently complex networks of chemical reactions will necessarily self-organise into autocatalytic cycles, the precursors of life [52, 53]. Another complexity theorist associated with the Santa Fe Institute, J. Holland, studied adaptation through natural selection of self-organising systems to a variable environment. By generalizing from the mechanisms through which biological organisms adapt, he provided the foundations of Genetic Algorithms2 and developed models of cognitive, ecological and economic systems [45, 46]. Both Holland’s and Kauffman’s works have provided essential inspiration for the new discipline of Artificial Life (AL), which was initiated by C. Langton [56]. AL involves developing and executing software models mimicking lifelike properties, such as reproduction, sexuality, swarming and co-evolution, and subsequently concluding about real systems exhibiting these properties. During the last 20 years, research in artificial systems has been oriented towards introducing self-organisation mechanisms specifically for software applications. These different works originate from different starting points including stigmergy [39, 59], autopoiesis [12] and the holon concept [11, 76]. Recently, in addition to reproducing natural system behaviour into artificial systems, latest research efforts have been oriented towards introducing self-organisation mechanisms specifically for software applications [29, 34]. Part II describes such mechanisms in more detail.

3.2.2 Definitions of Self-organisation The concept of self-organisation has been defined in many disciplines, and it has been viewed from different angles. In the following the most common definitions found in the literature are presented, and a view of self-organisation from a software engineering perspective is provided.

2 Genetic

Algorithms is a general approach to computer problem solving which is based on mutation and recombination of partial solutions, and the subsequent selective reproduction of the most “fit” new partial solution combinations.

3 History and Definitions

39

3.2.2.1 Overview of Self-organisation Definitions Since the first appearance of the term, self-organisation essentially referred to a spontaneous, dynamically produced organisation in a system without external control. For example, as mentioned in the previous section, Kant used the term to refer to the autonomous formation of the planetary system without some central coordination point. The formation of structures in physical systems was the basis of the view of self-organisation adopted later in thermodynamics as well. According to Prigogine, self-organisation resulted to alterations to structure of materials while energy was emitted to the environment in order to lower the entropy of the system [35]. A common example is crystallisation, the appearance of a symmetric pattern of dense matter in a system of randomly moving molecules [43], such as in the case of snow crystals [41]. Prigogine and his colleagues proposed four necessary conditions that must hold for self-organising behaviour to occur [65]: 1. The Mutual Casuality condition. A circular relationship must exist between at least two of the system elements resulting in mutual influence of each other. 2. The Autocatalysis condition: There exists at least one system component which is causally affected by other components, in a way that increases the quantity of the first component in a nonlinear fashion.3 3. The Far-From Equilibrium condition: the system imports a large amount of energy from outside the system, uses the energy to help renew its own structures (autopoietic behaviour) and dissipates, rather than accumulates, the accruing disorder (entropy) back into the environment. 4. The Morphogenetic Changes condition: The system must exhibit morphogenetic changes, namely the system components should be able to change their characteristics and behaviour [23], given appropriate conditions and stimulus.4 To receive external stimulus, at least one of the system components must be open to external random variations originating in the system’s environment. Bonabeau et al. [10] provide a definition drawn from the behaviour of dynamically evolving natural systems. They consider self-organisation as a set of dynamical mechanisms whereby structures appear at the global level of a system as a result of interactions among its lower-level components. The rules specifying these interactions are executed on the basis of purely local information, without reference to a global pattern. Therefore, the resulting of structures is an emergent property of 3 In

Chemistry autocatalytic reactions are reactions in which at least one of the products is also a reactant. Such reactions are fundamentally nonlinear, and this nonlinearity can lead to the spontaneous generation of order. A dramatic example of this order is the one found in living systems. This spontaneous order creation initially seems to contradict the Second Law of Thermodynamics. However, this contradiction is resolved when the disorder of both the system and its surroundings are taken into account and it is recognised that energy is dissipated into the environment to lower the entropy of the system.

4 The term Morphogenesis (from the Greek morphe = shape and genesis = creation) was originally introduced in biology to refer to a process that would cause an organism to develop its shape.

40

G. Di Marzo Serugendo et al.

the system, rather than a property imposed upon the system by an external ordering influence. For example, in the case of foraging ants the emergent structures are the spatiotemporally organised networks of pheromone trails. Bonabeau and colleagues identified four basic requirements for self-organisation: 1. Positive feedback (amplification) which involves simple behavioural “rules of thumb” that promote the creation of structures. Examples of positive feedback include recruitment to a food source and reinforcement to adopt a particular role observed in natural systems. 2. Negative feedback which counterbalances positive feedback and contributed to stabilising the overall collective pattern. Examples of negative feedback include satiation, exhaustion or completion. In the example of foraging ants in particular, negative feedback could result from the limited number of available foragers, the exhaustion of food source, and the crowding at the food source, or competition between food sources, respectively. 3. Amplification of fluctuations such as random walks, errors and random-task switching. In other words, small perturbations in system behaviour may increase and affect the collective system behaviour significantly. For example, when a new food source is found, the concentration of pheromones in the pheromone path formed increases significantly. 4. Multiple interactions among individuals which are the basis for the collective behaviour. More specifically, self-organisation generally requires a minimal density of mutually tolerant individual behaviours capable of using the results of both their own activities and the activities of others. For example, trail networks can self-organise and are formed by pheromones deposited by more than one ant. Similar definitions drawn from natural systems are adopted by many authors, for example in [16]. In other domains self-organisation is typically defined as the evolution of a system into an organised form in the absence of external pressures [69]. This is explicitly stated by Haken [40], who considers a system as self-organising if “it acquires a spatial, temporal or functional structure without specific interference from the outside”. As another example, Polani [68] considers self-organisation as a phenomenon under which “a dynamical system exhibits the tendency to create organisation ‘out of itself ’ without being driven by an external system”. Along the same line Heylingen [43] defines self-organisation as the “spontaneous creation of a globally coherent pattern out of local interactions”. Because of its distributed character, this organisation tends to be robust, resisting perturbations. The dynamics of a self-organising system is typically nonlinear, because of circular or feedback relations between the components. Positive feedback leads to an explosive growth, which ends when all components have been absorbed into the new configuration, leaving the system in a stable, negative feedback state. Heylighen stresses that the basic mechanism underlying self-organisation is the (often noise-driven) variation which explores different regions until it enters a stable state (an attractor state). In a similar manner Collier [22] defines self-organisation as “a process by which larger scale (macro) order is formed in a system through the promotion of fluctu-

3 History and Definitions

41

ations at a smaller (micro) scale via processes inherent solely in the system dynamics, and modulated by interactions between the system and its environment”. Self-organisation occurs when the properties of a system allow it to take on a more ordered state through the dissipation of energy (production of entropy) some of which goes into the newly formed structure. However, not all definitions of self-organisation involve spontaneous creation of organisational structures. Maturana and Varela introduced autopoietic systems [61] where some form of organised system must pre-exist for subsequent changes to the organisation to take place, in a manner similar to re-organising systems discussed by Collier in [21]. Maturana and Varela view autopoietic systems as networks of interacting processes that are capable of producing (generating, transforming and destructing) other, possibly similar, processes and that satisfy the following requirements: 1. They are continuously regenerated and transformed through interaction and reproduction of its members processes. 2. They constitute a concrete unity in the space where the member processes exist and the topological domain of their associations is specified [75]. In other words, in autopoietic systems self-organisation refers to dynamic system regeneration and transformation to some known form. Finally, in many computer applications self-organisation has been used to the process of changing the system’s organisation based on some internal central point of planning or control within the system, as is the case for example in [1, 60]. This approach is followed in many cases in natural systems5 and has certain advantages, such as better coordination efficiency and control. However, it can suffer from the known centralised point weaknesses such as robustness and performance. In other cases, changes to the organisation of the system are the collective result of distributed autonomous points of control within the system. Examples include self-organised coordination [49], multi-agent system cooperation [71] and multiagent system group formation [58].

3.2.2.2 Software Definition of Self-organisation Self-organisation is an attractive approach to handle the dynamic requirements in software. Considering the definitions given in other domains, the term selforganisation can be used in software engineering to refer to a process where a software system changes its internal organisation to adapt to changes in its goals and the environment without any explicit external directing (command and control) mechanism. Therefore: 5 For

example, in societies of termites the queen selects a location and deposits pheromones symmetrically at equal distances resulting in nest arches being equally distanced from the queen location [10].

42

G. Di Marzo Serugendo et al. self-organisation is the process enabling a software system to dynamically alter its internal organisation (structure and functionality) during its execution time without any explicit external directing mechanism.

Furthermore, due to the wide range of self-organising software systems, it is necessary to further distinguish them into those involving no explicit centralised control (either external or internal) and those based on some internal centralised coordination and planning point. By consequence the following definitions are given: Strong self-organising systems are systems where self-organisation process decisions are distributed locally among the system components without involving any centralised point of control (either internal or external). Weak self-organising systems are those systems where, from an internal point of view, selforganisation is internally administered by a centralised point of planning and control.

It is important to emphasise that the central point of control in a weak selforganising system is not visible from an external observer located outside the system itself. For example, the users of a cluster-based e-commerce web site that selforganises to balance the incoming user load will not notice any difference regardless whether load is balanced in a centralised or a distributed manner within the system. Typical examples of strong self-organising systems are those implementing ant algorithms. In such systems there is no internal point of central control since they are built with the aim to mimic the behaviour of ant colonies when foraging. Representative examples of weak self-organising systems are those based on architectures involving centralised control such as general mediator systems, client-based systems and systems based on star and hierarchical architecture topologies. Self-organisation can often result in emergent software behavioural patterns that can be either desirable or undesirable. Due to the dynamism and openness of contemporary software environments and the ever increasing distribution, complexity and dynamic changes in application requirements, understanding the mechanisms that can be used to model, assess and engineer self-organising behaviour in software is an issue of major interest.

3.2.3 Properties of Self-organising Systems Self-organising systems are distinguished from traditional mechanical systems studied in Physics and Engineering by a number of mandatory and optional properties, which can be considered as part of what defines them.

3.2.3.1 Mandatory Properties of Self-organising Systems Self-organising systems are characterised by the following fundamental properties: • Global Organisation. The process of self-organisation brings the system into some ordered and relatively stable state, in which it can fulfil its function and

3 History and Definitions

43

purpose given the constraints imposed from its environment. For example, if we consider a system consisting of a light bulb and having the purpose of providing light to a warehouse, then satisfactory organised states for the system would be one where the light bulb would be switched off during the day and one where it would be switched on during in dark hours. The resulting system organisation can be static, where the organisational positions of the system components are fixed, as is the case in the light bulb example, or stationary. In the latter case, system components continuously change their organisational positions, but this is done in some ordered fashion and according to some stable organisational pattern, such as the Benard cells pattern (see Sect. 3.3.2.1). The Benard cells pattern refers to the formation of hexagonal cells which appear when a liquid is evenly heated from below and cooled on its surface [43]. In Benard cells there is an upward flow of liquid on one side and a downward flow on the other. Despite the continuous movement, the Benard cell pattern remains stable as long as the heating and cooling of the liquid are not altered. • Dynamic Adaptation. Self-organising systems are capable of changing their organisation dynamically to adapt to changes in their intended function and the conditions of their environment. For example, insect populations behave differently when mating than when collecting food [10], and their behaviour further varies according to environmental conditions, such as ground and weather conditions, and time of day. A common question then is what behaviour can be considered as part of the dynamic reorganisation process that realises system’s adaptation to the current environmental conditions, and what is considered as normal, possibly dynamic, application behaviour that realises the system’s intended function. This question becomes hard to answer since reorganisation and application behaviours are largely interweaved, and the former commonly affects the latter, namely application behaviour is often modified as a result of reorganisation. In simple systems, dynamic reorganisation can be perceived and modelled as normal application behaviour, although this is not generally a good practice since it increases model complexity. For example, let us consider a sensor attached to a light bulb capable of emitting light linearly and in inverse proportion to the amount of natural light sensed. The emitted light will gradually increase when the amount of light in the surrounding space decreases, for instance if natural light is reduced due to cloudy weather. One could intuitively view the increase in light emittance level as system reorganisation, considering the level of emitted light as an organised system state and the actual light emittance as the functionality delivered by the system. One other could argue that the proportionally adjusted light emittance is simply predesigned application behaviour and that changes in natural light availability are simply external input and not changes in the system’s environment. In such simple cases, where the possible reorganisation results and the stimuli that triggers reorganisation can be perceived or calculated in advance, reorganisation and application behaviour can indeed be a matter of definition and modelling perspective. For instance, in the above example the light emittance can be exactly calculated as a linear function of the absence of natural light, and hence it can be easily perceived as application behaviour.

44

G. Di Marzo Serugendo et al.

In general, however, it is not always possible to exactly determine the system configuration that would result from the reorganisation process in advance. For example, it is difficult to predetermine the exact path that will be formed by foraging ants since the number of possibilities is too high. In these cases, dynamic reorganisation is considerably harder to be modelled as application functionality, since the adapted organisation that will realise application functionality is not known in advance. Therefore, it is a good practice to follow intuition and model reorganisation separately whenever possible because that increases modularity, ease of understanding and separation of concerns. For example, in an ant-based system it is easier to model the behaviour of searching for food and the behaviours of carrying and processing food separately. • Lack of External Control. The dynamic reorganisation taking place in selforganising systems is executed without any external control, and the order produced is endogenous. For example, if we have some external operator switching the light bulb on and off, then the light bulb system would not be self-organising. On the other hand, if the system comprises both the light bulb and a sensor device capable of switching it on and off according to the light conditions, then that is a clear self-organising system example. Consequently, it is very important to define the system boundaries before characterising a system as self-organising or not. To clarify this with an additional example, if we consider a central light sensor connected to a computer that remotely operates light bulbs in different rooms, then each individual light bulb alone is not a self-organising system. However, if we considered a larger system comprising all light bulbs and the operating computer, then we would have a clear case of a weakly self-organising system.

3.2.3.2 Optional Characteristics of Self-organising Systems In addition to the above mandatory properties that allow us to determine if a system is actually self-organising, there are several characteristics that can appear in selforganising systems, either natural or artificial, in various combinations. • Nonlinearity and Complex Interactions. Systems showing instability are characterised mainly by nonlinear dynamics (small fluctuations when near some critical point can provide significant modifications of the system), by sensibility to initial conditions and parameter sensibility (small changes on a parameter produce different patterns). Thus the overall properties cannot be understood simply by examining separately the components. Self-organising systems commonly exhibit complex, nonlinear behaviour which enables them to adapt to a larger range of environmental conditions. Therefore, the more complex the overall system behaviour, the higher the adaptation capability of the system. Nonlinear system behaviour can trivially be the additive result of nonlinear behaviours of individual system components. For example, let us assume a number of pipes flowing water into an irrigation basin, and that the flow of each pipe is

3 History and Definitions

45

controlled by an intelligent sensor device. The device regularly senses a number of environmental parameters and controls the water flow according to some nonlinear irrigation model by operating a valve attached on the pipe. Depending on the environment parameter values, the flow of water through each pipe will vary, and the overall water flow will be the sum of flows of all water pipes. More complex collective system behaviours can be achieved when the number of system components increases. For example, let us assume that we have dozens of light bulbs distributed in some area. Each bulb is associated with a sensor and it is able to emit different amounts of light according to the level of natural light perceived by its sensor. Assuming that the levels of natural light are not the same in all locations within the designated area, that system will be emitting only the necessary light in each location, saving thus in energy as compared to a system having only one central strong light bulb. It is intuitively obvious that by increasing the number of light bulbs in the area we can achieve higher light emittance granularity and detail, and hence better energy savings. This is generally the case: the more individual components are included in the self-organising system, the more sophisticated overall collective behaviour can be achieved. The complexity of the overall system behaviour increases even further when there are interactions between system components which affect their individual behaviours. For example, instead of having each pipe controlled by a sensor executing some independent irrigation model, sensors could interact with each other; for instance, they could be connected nodes in a neural network that would be exchanging data and influencing each other. That would result in more complex overall behaviour, such as implementation of a more complex irrigation model, since there would be interrelations between individual behaviours. • Decentralised Control. Organised systems are commonly associated with some hierarchy of internal or external controllers which guide and direct them. The controller hierarchy is a physically distinct subsystem that exerts its influence over the rest of the system. In these cases the control is characterised as centralised. For example, most business organisations have leaders, boards of directors and unit managers that develop policies and coordinate business operations. An additional example is the human body whose actions are largely planned and controlled by the brain. In weak self-organising systems control is centralised. In strong self-organising systems however, control is distributed over the whole system. All system components contribute evenly to the resulting arrangement. For example, in spin magnetisation all spins influence each other to maintain the overall spin direction which has been reached and no spin or group of spins can deviate from it. Another example is the human brain which is organised over a network of interacting neurons. Different brain regions are specialised for different tasks, but no neuron or group of neurons has overall control. This is demonstrated in the cases where some brain parts get damaged, for example due to some accident, and the whole brain functioning continues unaffected. Centralised control offers advantages, such as more autonomy and stronger specialisation for the controller [43]. Furthermore, in decentralised control, units

46

G. Di Marzo Serugendo et al.

are prone to opposing actions, their behaviour may induce needless redundancies, and it is not guaranteed that the globally optimal solution will be found [25]. However, for systems that are complex and operate dynamic environments, the use of decentralised control offers significant advantages such as increased scalability and robustness and reduced communication and unit processing costs [13]. • Simple Behaviours and Local Interactions. In addition to acting autonomously in a decentralised manner, system components commonly have simple behaviours with limited perception abilities, and hence they do not have a global view of the system. For example, ants in an ant foraging colony can perceive the existence of pheromone or food only at a short radius. Spins in a piece of magnetised material exert non-negligible influence on their near neighbours. In the Benard example, liquid molecules influence only a few molecules that they collide with. Yet, most ants eventually move along an emerging path, the piece of material becomes magnetic as a whole, with a single North pole and South pole, and the liquid as a whole starts cycling through a sequence of rolls. In other words, despite the locality of interactions, the system reaches a global organised state where all components are significantly correlated. The locality of interactions implies that neighbouring configurations are strongly correlated, and this correlation diminishes as the distance between configurations increases. Furthermore, for an external influence to transfer from one region of the system to another, it must pass through all intermediate regions, and it will be affected by all fluctuations and activity that is taking place in them. For example, if we randomly place an obstacle on the foraging ants path, ants will progressively form a different path to avoid the obstacle, and this will eventually affect the majority of foraging ants. Ants closer to the point where the obstacle was placed will be affected more significantly, and the effects will be less as we move away from the obstacle. Simple interacting behaviours can still lead to quite complex patterns. Furthermore, they have the advantage that the information stored in the description of each behaviour is less than the information needed to describe the pattern: it is only needed to store information about how to produce the pattern, not the pattern itself. • Robustness, Resilience. Self-organising systems consisting of large number of interacting components can be particularly robust, namely they can be relatively insensitive to errors or perturbations from the environment. For example, if we remove some bees from a bee swarm or an ant colony, the harvesting result will be the same. In another example, if some nodes or links of a neural network are removed, the task the network was trained to perform will be largely still carried out. A reason for robustness is the redundancy inherent in such distributed organisation systems, and the remaining components can cover for the removed ones. Furthermore, such systems are commonly resilient, namely they have a strong capacity to restore themselves. For example, an ecosystem that has been damaged from a fire will in general recover in a short time. In a magnetic material, if part of the spins are diverted from their alignment, the magnetic field produced by the

3 History and Definitions

47

rest of the spins will put them back. Apart from redundancy, another reason for resilience is the random fluctuations, or “noise”. For example, foraging ants can move to some random direction with some small probability. This randomness allow them to initially discover the food source and repair the emergent path in case it gets damaged. Finally, a third reason for resilience is the stabilising effect of feedback loops which will be described below. • Feedback Loops. In complex, nonlinear systems the relation between cause and effect is not straightforward, and small causes can have large effects and vice versa. For example, let us assume that a magnetised piece of iron is subjected to an external magnetic field with a direction different from its own field. In the beginning, a large increase in the external field has practically no effect until a threshold is crossed. After that, any small further increase suddenly reverses the polarisation of the whole system. This is due to the feedback relation which commonly holds between components of self-organising systems. Each component, for example a spin in a magnet, affects the other components, but these components in turn affect the first component. This cause and effect is circular. Any change in the first components is fed back via its effects to the other components to the first component itself. Feedback can be either positive or negative. In positive feedback the recurrent influence reinforces or amplifies the initial change. In other words, if a change takes place in a particular direction, the reaction being fed back takes place in that same direction. In negative feedback the reaction is opposite to the initial action, namely the original fluctuation is suppressed or counteracted, rather than reinforced. Positive feedback makes deviations grow in a runaway, explosive manner. It leads to accelerating development resulting in a radically different organisation. Negative feedback, on the other hand, stabilises the system by bringing deviations back to their original state. • Emergent Properties. The resulting organisation can give rise to particular organisational constructs, such as structures, patterns or other system properties that cannot be reduced to the properties of its elements. The emergent outcomes exist and can be perceived only after self-organisation has taken place. For example, Benard cells are created after self-organisation of the liquid, and they are characterised by the direction of the molecules rotation, which is an emergent property. For an independent molecule, such rotation is not defined. A higher-level emergent property typically constrains the behaviour of the lower-level components. For example, the overall rotation characterising a Benard roll will force the liquid molecules to move in particular directions instead of others. This is called downward causation. Downward causation refers to a higher organisation level exerting influence downwards to a lower organisation level, causing in the Benard rolls example the molecules to move in a particular way. Random perturbations are crucial for emergence, since they enable discovery of new solutions, and fluctuations can act as seeds from which structures nucleate and grow. For example, foraging ants may get lost by following trails with some

48

G. Di Marzo Serugendo et al.

level of error, and this phenomenon may lead them in finding new, inexplored food sources and recruit nestmates to these food sources. In the general case, selforganisation can be witnessed without emergence [27]. For example, in the light bulb system there is no property that occurs only after system self-organisation. Further discussion on the emergence concept is provided in Sect. 3.3. • Symmetry Breaking. Non-organised systems are originally in a disordered configuration meaning that possible states for the individual components have the same probability. Therefore, despite differences in component behaviours on the global, macroscopic level, the system is considered homogeneous and symmetric since every behaviour is represented at the same degree, and hence it will look “the same” from whatever direction it is observed. After self-organisation, however, the global organisation configuration dominates all others, and therefore, the symmetry in probability is lost. For example, the probability of spin direction in a magnet will be either one or zero. This is referred to as “symmetry breaking”. • Bifurcations. The evolution from disordered to ordered configuration is normally triggered by a change in the external situation, the boundary conditions of the system. That evolution can take place through different possible sequences of organisational states, the system may be settling in during self-organisation, which can be conceived as evolution paths. We can have system variables whose values are associated with the states the system would settle in. These variables are called order parameters. It is possible that when order parameters reach a particular value, any further increase in their value would result in system evolution along different paths that have been arbitrarily chosen when the particular value was reached. If we represent the possible system states vs the values of an order parameter in a diagram, we can notice one or more clear branchings or bifurcations. For example, the speed of molecules in a Benard cell can be increasing with increasing temperature to a direction arbitrarily chosen when temperature had reached a particular threshold (see also Fig. 3.1 taken from [51]). • Far-from-Equilibrium Dynamics. In the absence of external perturbation, the system is expected to stabilise in some states in which emergent properties can be observed. This implies a kind of dissipation of some “energy”, otherwise the system would be continuously changing. Self-organising systems can reach a static global organisation, or equilibrium, or stationary involving ongoing activity. Equilibrium is characterised by the absence or entropy production or, equivalently, by the fact that all excessive energy has been dissipated to the environment. When a constant input of energy is provided in the system, as for example in the Benard cell case, the system cannot reach equilibrium, and it constantly dissipates energy to its environment. Such systems are termed Far-from-equilibrium systems. A far-from-equilibrium system is in principle capable of producing a much greater variety of regulating actions leading to multiple stable configurations. However, due to the dependency on an external source of energy, such systems are more fragile and sensitive to changes in the environment and also more dynamic and capable to react.

3 History and Definitions

49

Fig. 3.1 Feigenbaum Cascade bifurcations

Complex self-organising systems tend to reside on the “edge of chaos”. The “edge-of-chaos” is the narrow domain between equilibrium and turbulent, chaotic activity. The mechanism by which complex self-organising systems tend to maintain on this critical edge has been called “self-organised criticality”. The system behaviour on the edge of chaos is typically governed by a “power law” [43] which states that large adjustments are possible but they are much less probable than small adjustments. • Adaptability. The problem of adaptability is to maintain a particular organisation in spite of environmental changes, and the question involved is which action to use in which circumstances. It can be modelled as a problem of regulation or control to achieve “self-regulation as to operate the same in a changing environment” [43]. Not all self-organising systems are adaptable. For example, magnets and Benard cells are adaptive but not adaptable since they do not maintain the same function for changing environments. In various disciplines such as organisational management and ecology, adaptability is described as the ability to cope with unexpected disturbances in the environment while maintaining a particular organisation. For example, the immune systems are adaptable systems since they maintain the organisation (health) of the living organism despite any perturbations organisms receive from their environment through contact with viruses. To find the right type of antibodies, immune systems simply produce a vast variety of different antibody shapes. However, only the ones that “fit” the invaders are selected and reproduced in large quantities. The aim is to minimise deviations from a goal configuration by counteracting perturbations before they become large enough to endanger the essential organisation. This means that: 1. The system must produce a sufficient variety of actions to cope with each of the possible perturbations.

50

G. Di Marzo Serugendo et al.

2. The system must select the most suitable counteraction for a given perturbation. Variety can be fostered by keeping the system sufficiently far from equilibrium so that it has plenty of stationary states to choose from. Selectivity requires that these configurations are in sufficiently small numbers and sufficiently stable to allow an appropriate one to be selected without risk of modifying the overall organisation. Another typical example of simple variation and selective reproduction of fit components underlies economic and ecological adaptation mechanisms. • Organisational Closure and Hierarchy. Self-organising systems at some point settle into a negative feedback regime or into an attractor state. Subsequently, they become relatively impervious to external disturbances and largely independent from their environment. Therefore, they can be considered that they are “closed” against relatively small influences from the outside. For the outside observer, closure determines a clear distinction between inside (components that participate in the closure) and outside (those that do not). In other words, closure defines a boundary separating the organisational structure from the environment, This boundary can encompass all components of the original system, for example in case of a magnet, or only part of them as in a Benard cell. A self-organising system may settle into a number of relatively autonomous, organisationally closed subsystems. These subsystems will continue to interact in a more indirect way, determining subsystems at a higher hierarchical level, which contain the original subsystems as components. For example, a cell is an organisationally closed system encompassing a complex network of interacting chemical cycles within a membrane that protects them from external disturbances. Cells are themselves organised in circuits and tissues that form a multicellular organism. Organisms are further connected in cyclical food webs, collectively forming ecosystems.

3.3 Emergence Our aim is to analyse the emergence concept from two perspectives: One concerning the properties that when observed are sufficient to identify emergence, and another focusing on those system characteristics that when all present the system has the capability to produce emergence. Subsequently, we attempt to provide an operational definition of emergence.

3.3.1 History Emergent properties were first studied in the Greek antiquity in the context of philosophical treatises [24, 36]. For example, while referring to the significance

3 History and Definitions

51

of “wholes” in the natural world, Aristotle wrote: “. . . the totality is not, as it were, a mere heap, but the whole is something besides the parts . . . ”,6 which is interpreted as “the whole is something before, over and above its parts, and not just the sum of them all.” [24] or simply “. . . the whole is more than the sum of its parts . . . ” [27, 30]. The expression “whole before its parts” refers to the ontological distinction between parts and wholes and in particular to the explanatory precedence of a whole entity over the parts of which the whole is made up. This is exemplified in the response given by Aristotle to a famous paradox introduced by Zeno. As [36] details, Zeno argued that a distance of any length could be divided into an infinite number of shorter segments. Hence, covering the distance would require traversing an infinite number of shorter segments, which, as a result, would take an infinite amount of time. That was obviously paradoxical since we do cross distances in finite lengths of time. Aristotle’s response was that a length was first and foremost a whole. That whole might indeed be divided into an infinite number of parts; however, the whole was fundamentally irreducible to those parts, and in fact it was only due to the preexistence of the whole that it could be traversed. The pre-existence of a coherent whole that cannot be described merely as a sum of its constituent parts was later endorsed in the concept of Gestalt, which originated from theories of Johann Wolfgang von Goethe, Immanuel Kant and Ernst Mach [9, 30, 36] and refers to whole forms or global configurations. Initially, the term ‘Gestalt’ referred to secondary qualities that emerged from the parts constituting a whole. For example, Goethe considered a Gestalt to be “a natural unity that was the endpoint of an entelechetic development out of primordial chaos” (see [36] referring to [42]). This had some similarities with current notions of emergence, for example with the one adopted in complex systems where it is considered that order emerges from chaos [46, 53]. After various ramifications, the term Gestalt was accepted as a perceptually primary concept, defining the parts of which the whole is composed. For example, the founder of modern Gestalt psychology Christian von Ehrenfels illustratively remarked that perception takes place through recognising whole patterns: “the whole is greater than the sum of the parts” (see [36] referring to [42]). In modern Psychology the Gestalt effect refers to the form-forming capability of our senses, particularly with respect to the visual recognition of figures and whole forms instead of just a collection of simple lines and curves. A typical example given to demonstrate the Gestalt effect is that of a picture of a dog, which can be perceived as a whole at once, and it is not recognised by identifying its parts first (such as feet, ears, and nose), and only then inferring the dog from those component parts [74]. Neither Gestalt nor the Aristotelian view of emergence explain how the emergent outcome is produced. Furthermore, they both assume the existence of a pre-given whole, and therefore they capture emergence only in a static sense. The dynamic aspect of emergence is considered in the technical definition of the term which was 6 Aristotle,

Metaphysics, Book H 8.6.1045a:8-10.

52

G. Di Marzo Serugendo et al.

Fig. 3.2 The Proto-emergentist view of emergence

given by the English philosopher G.H. Lewes in 1875. Lewes was based on an earlier philosophical work of J.S. Mill concerning differentiation of types of causation, and he classified chemical compounds resulting from chemical reactions as either resulting or emergent (see [36] referring to [57]): although each effect is the resultant of its components, we cannot always trace the steps of the process, so as to see in the product the mode of operation of each factor. In the latter case, I propose to call the effect an emergent. It arises out of the combined agencies, but in a form which does not display the agents in action . . . Every resultant is either a sum or a difference of the co-operant forces . . . [and] is clearly traceable in its components . . . the emergent is unlike its components insofar as these are incommensurable . . . and it cannot be reduced either to their sum or their difference.

The newly defined concept was widely adopted in late 19th century, and it was placed in the core of a loosely joint movement covering social sciences, philosophy and theology which was originally termed emergent evolutionism (see [9] for history and review) and later known as proto-emergentism [27, 30, 36]. Despite that several proto-emergentists employed the term ‘Gestalt’ for describing emergent phenomena, the proto-emergentist view is radically different. The emergent process is viewed as a black box (see Fig. 3.2). A number of intermediate system organisation levels is considered, but only the inputs and outputs at the lowest and highest levels respectively can be realised, and the emergent outcome is formed dynamically without any knowledge of how the entries are transformed to outputs. Many well-known researchers including G.H. Lewes, C.L. Morgan, J.S. Mill, S. Alexander, D. Broad, W. Wheeler and A.N. Whitehead participated in that movement trying to explicit the characteristics of emergent phenomena. However, the lack of adequate explanations regarding the causes of emergence resulted in the deterioration of the movement in early 20th century. After 1930 a different perspective started gaining ground, realised by a movement termed neo-emergentism [27, 30, 36]. Neo-emergentism had its roots in approaches followed to study phenomena and dynamics of systems in positive and computer sciences. In contrast to proto-emergentism, this movement aimed at understanding and reproducing the processes which lead to emergent phenomena. The view is that the unique features of emergence can be better apprehended by sketching out its scientific and mathematical sources, as is done in complexity theory for instance. Representative examples of neo-emergentist approaches include the works of H. Haken, J. Holland, S. Kauffman, C. Langton and I. Prigogine. Their efforts concentrated on developing theories, tools and methodologies which aimed at enabling the expression of emergent processes as less dense using more primitive models and consequently as less miraculous (see Fig. 3.3). The neo-emergentism movement is still

3 History and Definitions

53

Fig. 3.3 The Neo-emergentist view of emergence

actively developing having so far shown important advances in modelling, engineering and controlling emergence [29, 32, 54, 79]. In the early neo-emergentist approaches developed in cybernetics, information theory and general systems theory, emergent phenomena were not the explicit focus of research per se. The reason is that these earlier approaches investigated systems which were simple, linear and equilibrium seeking [36]. In time the interest gradually shifted to complex, nonlinear and nonequilibrium systems, for example such as those targeted by complexity theory [45]. The basic idea in neo-emergentism is that the dynamical characteristics of emergence can be better understood by considering its association with the arising of attractor states during system operation which are not pre-given in the sense of a Gestalt. Attractors technically pre-exist of course; however, they are not possible to be known or accurately predicted in advance due to system complexity,7 and they reveal themselves when a dynamical system bifurcates. The convergence to an attractor state signifies both a quantitative and a qualitative metamorphosis of the system. Upon their appearance, these “new” attractor states dominate the system and hence enable the emergence of something radically novel in respect to what existed before [46, 53, 64, 65]. An important strand in neo-emergentism is connectionism [70]. Connectionism is a set of approaches originated in the 1980s in the fields of artificial intelligence, cognitive psychology, cognitive science, neuroscience and philosophy of mind that models mental or behavioural phenomena as the emergent processes of interconnected networks of simple units. There are many forms of connectionism, but the most common forms use neural network models (see for example [48]). Although the basic idea of connectionism has received fierce criticism from the proponents of the symbolic AI school, connectionist models are believed to be a step in the direction toward capturing the intrinsic properties of the biological substrate of intelligence, since they have been inspired by biological neural networks and seem to be closer in form to biological processes. Furthermore, such models are capable of handling incomplete, approximate, and inconsistent information, as well as generalisation. Their main weakness however is the difficulty in development, which is still an open research issue [14, 27, 32, 79].

7 Complexity

can be simplistically perceived as the minimum amount of information necessary for system description. Various definitions of complexity exist; for example see [4].

54

G. Di Marzo Serugendo et al.

Fig. 3.4 Benard Cells appear at macroscopic level when temperature reaches a critical point

3.3.2 Definitions 3.3.2.1 Examples of Emergent Phenomena Phenomena that can be characterised as emergent include organisational structures and frameworks, behavioural processes, particular system states, and even outcomes corresponding to functions8 not explicitly implemented in the system. The foraging ant path and Benard cells are representative examples of emerging organisational structures. As already mentioned in Sect. 3.2.3.1, Benard cells are convection cells that appear spontaneously in a liquid layer when heat is applied from below (see Fig. 3.4 taken from [2]). The setup includes a layer of liquid, for example water, between two parallel planes. Initially, the temperature of the bottom plane is the same as the top plane. The liquid then tends towards an equilibrium, where its temperature is the same as that of the surrounding environment. Once there, the liquid is perfectly uniform, and it appears the same from every direction. When the temperature at the bottom increases up to a critical point, then hexagonal convection cells appear (see Fig. 3.5). The microscopic random movement of the liquid molecules spontaneously becomes ordered on a macroscopic level, with a characteristic correlation length. The rotation of the cells is stable and will alternate from clock-wise to counter-clockwise as we move horizontally along the liquid. If the warmth increases, this phenomenon disappears. Typical examples of emergent behavioural processes and system states can be found in the well-known Game of Life, which is a cellular automaton devised by the British mathematician J.H. Conway in 1970.9 The game consists of a collection of cells which are placed on a board with a connexity of 8. Cells can have the status of ‘dead’ or ‘alive’, and at each game round they can change their status and affect the status of their neighbouring cells according to a few simple mathematical rules. Alive cells are coloured, and depending on the initial conditions, they can form 8 The

term function here refers to a process or operation and not to some direct input–output mechanism such as a mathematical function. For example, an foraging ant-colony can find the shortest path between food source and nest without it being an explicit function of the system.

9 http://www.bitstorm.org/gameoflife/

3 History and Definitions

55

Fig. 3.5 Benard cells appearing in heated water

Fig. 3.6 Glider rotation in the Game of Life

various patterns throughout the course of the game. Experimentation with the game involves creating an initial cell configuration and observing how it evolves during game execution. Based on different initial configurations, several types of emergent states, patterns and behavioural processes can be observed [5, 7, 46]. A representative emergent pattern in Game of Life is a glider which appears to be moving across the board (see Fig. 3.6) diagonally at a speed of 1/4 of cell at each round. The glider pattern is often produced from randomly generated starting configurations [7]. Gliders are important to the Game of Life because they are easily produced, they can be collided with each other to form more complicated objects, and they can be used to transmit information over long distances. Other characteristic emergent patterns appearing in the game of life include “still lives”, oscillators and patterns that translate themselves across the board (“spaceships”) [5, 7]. A typical process that can emerge during the game of life is Glider Gun which produces gliders repeatedly (see Fig. 3.7). A glider gun is a pattern consisting of a main part that repeats periodically like an oscillator and emits gliders at regular time intervals. The discovery of the glider gun process eventually led to the proof that Conway’s Game of Life could function as a Turing machine. An example of emergent state in the Game of Life is the R-predomino pattern (see Fig. 3.8). R-predomino acts as a glider gun producing exactly six gliders in 1103 game rounds. The pattern subsequently settles down to a stable state, and any movements it makes on the board are restricted into a 51-by-109 cell region. This is a representative example of an emergent phenomenon that consists of an emergent state. There are numerous examples of function outcomes obtained indirectly through emergence, such as emergent construction of course timetables [67] and development of manufacturing schedules [64]. In these cases the system is modelled as several components which interact locally, for example along the lines of an ant colony. The outcome of interest, such as timetable, manufacturing plan, assembly movement sequence and

56

G. Di Marzo Serugendo et al.

Fig. 3.7 Snapshot of a Glider Gun emitting gliders

Fig. 3.8 The R-predomino pattern

machine configuration, corresponds to a global property which obtains the desired value as a result of the emergence process that takes place based on component interactions.

3.3.2.2 Overview of Emergence Definitions Definitions of emergence can be broadly classified in two categories based on their view of emergent properties [50]. In the first category any system properties are considered as emergent when they cannot be analysed (or “reduced”) through functional decomposition. In these cases it is said that, “. . . the system is more than the sum of its component parts . . . ” [5, 36, 44]. On the other hand, definitions belonging to the second category accept that “. . . any unexpected properties are emergent . . . ” [18, 19, 50, 73]. A representative example of the first category is the view of emergence introduced by proto-emergentists (also termed British emergentists). The protoemergentist view was inspired by properties of living systems where “. . . no mere summing up of the separate actions of system ingredients will ever amount to the action of the living body itself . . . ” (see discussion about proto-emergentist view of emergence in [50]). According to proto-emergentists, emergent properties are produced as heteropathic system effects. Such effects appear in complex biological and chemical systems where the conjoint actions of system components cannot be characterised as the sum of any individual causes. For example, the addition of sodium hydroxide to hydrochloric acid produces sodium chloride and water. However, it is unclear how such a reaction could be characterised as the sum of individual chemical compounds. In contrast, any properties occurring as homeopathic effects are not considered as emergent. Homeopathic effects arise when system causes acting

3 History and Definitions

57

together are identical to the sum of the effects of those causes acting in isolation. For example, forces acting on an object can have the same effect when applied in combination or separately. Proto-emergentists proposed a layered view of complexity in which the world is divided into different strata. At the bottom there are the fundamental physical laws. On this foundation we can observe chemical, biological, psychological and social interactions at ever increasing levels of organisational complexity. In upper levels, we can identify properties that cannot be understood in terms of the individual observations of underlying physical phenomena. These emergent properties can be influenced by behaviours at lower levels in this layered approach. Research in physical sciences, therefore, investigates fundamental properties and laws that are broadly applicable. The remaining “special sciences” focus on properties that emerge from complex systems. Despite having irreducibility as the common basis, proto-emergentist definitions of emergence have considerable differences as to how irreducibility can be modelled. For example, Alexander argues that emergent properties are novel qualities and associated, high-level, causal patterns which cannot be directly expressed in terms of more fundamental system entities and principles (see Johnson in [50] for a discussion). A common example of such a property is consciousness which is intrinsically a system level property and quite distinct from the underlying physiology of lower-level system components [26]. Along this line, it makes little sense to talk of human cognition in terms of individual neurons. However, this clear separation of emergent properties and their causal processes contrasts other proto-emergentist definitions. For example, Mill accepts that strong emergent properties, although irreducible to physical laws of causal composition, can still be described in terms of other laws or patterns of behaviour. Therefore, we can still talk about causes of patterns of cognitive behaviour despite that we cannot explain in detail how cognitive behaviours relate to underlying electrochemical changes in the brain. In this view, some relation between the emergent phenomena and the underlying system layers can be accrued by experimental analysis, for instance using simulations and/or statistics. For example, this approach has been extensively followed by 19th century bridge builders who used to develop experimental models of bridge behaviour when they did not have a sufficient theory to describe why some bridges failed while others did not. Proto-emergentists do not consider interactions of the system with its environment, and they do not provide any suggestions about design methods or development techniques [50]. However, the latter view of emergence provides some hope for ‘controlling’ or at least anticipating emergent properties by aiming to understand the source of any non-determinism and the resulting emergent phenomena, for example by studying the underlying properties of lower-level components within a system. In this respect, functional decomposition can still be considered as a possible tool for engineering emergence. For example, one could attempt to develop multivariate statistical models linking the electrochemical brain changes with different of cognitive behaviours and then aim to achieve particular behaviours by applying specific electrochemical brain variations.

58

G. Di Marzo Serugendo et al.

The definition of emergence has been largely a philosophical issue not only in early and proto-emergentist approaches but also in recent times as well. For example Bedau proposes three types of emergence, nominal, weak and strong [5, 6]. Nominal emergence refers to systemic properties that the system parts cannot individually have. In this approach, a system comprising points that are equidistant from a fixed point of origin is considered to have the nominally emergent property of “circle”. The reason is that being a circle is a property which none of the individual points can have. Furthermore, strong emergence refers to nominally emergent properties which are supervenient to the properties of the individual system components. In addition, strong emergent properties exhibit irreducible causal powers affecting component behaviour, something which Bedau refers to by the term downward causation. Typical examples of strongly emergent properties are phenomenal qualities, such as qualia and consciousness. Bedau further claims that the irreducibility of strongly emergent properties to properties of system components renders strong emergence scientifically irrelevant. Finally, a nominally emergent property is considered as weakly emergent if it can be derived but only by simulation “. . . a nominally emergent property P possessed by some locally reducible system S is weakly emergent if and only if P is derivable from all of S’s micro-facts but only by simulation . . . ”. In other words, weak emergence is generally reducible, although only with considerable difficulty. Therefore, Bedau argues that weak emergence can be quite useful for scientific study of numerous real-world phenomena. In the majority of neo-emergentist approaches, emergence refers to the process that gives rise to phenomena which are novel and unpredictable. For example, Goldstein in [36] defines emergence as the “. . . arising of novel and coherent structures, patterns and properties during the process of self-organisation in complex systems . . . ”. Goldstein considers emergent phenomena to be conceptualised at a systemic, macro-level in contrast to the micro-level components and processes out of which they arise. Along the same line, Johnson in [50] defines emergent properties as “. . . unexpected behaviours that stem from interaction between the components of an application and the environment . . . ”. Johnson emphasises that emergent properties can be beneficial, for example, if users adapt products to support tasks that designers never intended. However, they can also be harmful, for instance if they undermine important safety requirements. According to this view, unpredictability of emergent phenomena is caused by system non-determinism, which can largely result from system underspecification, for example when designers fail to correctly specify the behaviour of non-deterministic application processes. However, nondeterminism can also be introduced by other factors, such as environmental conditions. In a layered system view, for example along the lines of stratified system view suggested by proto-emergentists, non-determinism can also stem from interactions between adjacent layers. Some definitions cover both irreducibility and unpredictability of emergent phenomena by considering different emergence types. For example, Chalmers argues for the existence of three types of emergence, termed strong, weak and intermediate [18, 19]. In this view, high-level phenomena are characterised as strongly emergent with respect to a low-level domain if they are not deducible, even in principle,

3 History and Definitions

59

from truths in that low-level domain. However, although strongly emergent phenomena are not deducible from truths from lower-level domain, they are still correlated to them. Regarding irreducibility, this notion of strong emergence is similar to that introduced by proto-emergentists; however, Chalmers further argues that the only emergent phenomenon that is truly strong is consciousness. To provide an example of consciousness as a strongly emergent phenomenon, Chalmers assumes a colour-blind scientist that is given complete physical knowledge about brains. He then emphasises that the scientist would nevertheless not be able to deduce and obtain a conscious experience of the red colour. Weakly emergent phenomena are also defined similarly to the neo-emergentist approaches, since they are considered as unexpected given the principles governing the low-level domain. Chalmers describes formations in cellular automata as representative examples of weakly emergent phenomena. Finally, intermediately emergent phenomena are not deducible from lowlevel laws and initial conditions, but they would be only from low-level facts. This type of emergence involves downward causation, meaning that higher-level phenomena are not only irreducible, but they also exert a causal efficacy to the low-level components. Typical examples of such phenomena are the otherwise inexplicable complex chemical compounds, which, once created, further constrain the behaviour of the participating chemical elements. The majority of emergence definitions pays emphasis to axiomatically defining the physical nature of emergent phenomena. For example, Stefan in [73] argues that weakly emergent phenomena must have three fundamental properties: (a) they must be instantiated by systems consisting exclusively of physical entities (physical monism), (b) they must be global, namely they should not occur in individual system parts (systemic properties), and (c) they should depend nomologically on the microstructure of the system (synchronic determination), namely changes in the status of emergent phenomena should necessarily result from changes in the properties of the system parts or their environment. Stephan further defines diachronic emergence as having the additional properties of genuine novelty in system evolution and structure unpredictability, and synchronic emergence which further has the property of irreducibility. These three types of emergence are analogous to the previously described nominal, weak and strong emergence types introduced by Bedau. There are also cases where the emergent phenomenon is not necessarily global. For example, Haan [38] argues that certain local properties can emerge in parallel with global systemic ones, a phenomenon which he terms conjugate. More specifically, Haan considers three types of emergence: • Discovery Emergence. In this case the emergent phenomenon is observed only by an external observer and no conjugate occurs. • Mechanistic Emergence. Here the global emergent phenomenon is again identified by an external observer, but there is also local emergence (conjugate). Furthermore, the dynamics producing the local emergence actually depends on the global emergent pattern, due to downward causal powers (downward causation). • Reflective Emergence. In this case no external observer is required. The system objects have some reflective capacity, which enables them to observe the emer-

60

G. Di Marzo Serugendo et al.

gent phenomena, both global and local, they produce. This case also involves both conjugate and downward causation. Furthermore, there are definitions of emergent phenomena which have been given specifically for artificial systems. For example, Forrest [33] defines an emergent computation phenomenon as having three fundamental properties: (a) System behaviour should be determined from a micro-level dynamic process resulting from interactions of a collection of individual agents, (b) an epiphenomenon, such as a particular system state, should be produced by the aforementioned process at the macro level, and (c) the phenomenon should be a natural interpretation of the produced epiphenomenon either as computation process or computation results. Similarly, Muller [64] considers a phenomenon as emergent iff: 1. It occurs in a system of entities in interaction whose expression of the states and dynamics is made in an ontology or theory D. 2. The system dynamics produce a epiphenomenon, such as a process, a stable state, or an invariant, which is necessarily global regarding the system of entities. 3. The emergent phenomenon itself is identified by interpretation of the aforementioned global epiphenomenon, either by an external observer or by the entities themselves, via an inscription mechanism in another ontology or theory D . Muller claims that the nonlinearity of component interactions guarantees the irreducibility of D to D, and he considers two types of emergence: (a) strong Emergence that is observed by an internal observer, for example a social structure in social systems and (b) weak Emergence that is observed only by an external observer, for example the path formed by foraging ants. Finally, computational emergent phenomena have been operationally defined in IRIT [17] as having two fundamental properties: (a) they must be realisations of specific system goals, such as solving specific problems or providing a particular, possibly evolving over time, functionality that must be confirmed by relevant system users, and (b) the details of the phenomenon, such as the solution given or the functionality produced, should not have been exactly pre-engineered in advance by system designers, but they should have been adaptively produced as a result of system dynamics.

3.3.2.3 The Essence of Emergent Phenomena To be able to engineer emergence to achieve intended results, several issues need to be clearly understood. One such important issue is the scope of emergent phenomena. The consensus is that emergent phenomena are global and they concern the system as whole despite being dependent on the individual system components [5, 18, 36, 44, 64]. This is aligned with the common view that systems producing emergent phenomena can be perceived as involving two levels (see [44, 73] for instance). The system component (micro) level, where the mechanisms producing the emergent phenomenon are executed, and the systemic (macro) level, where the emergent phenomenon is perceived. For example, in a foraging ant colony, the

3 History and Definitions

61

individual ant movements belong to the micro level, while the path that is formed emergently between nest and food source is considered to reside at the macro level. The question then is who can perceive the emergent phenomenon. Considering their global scope, emergent phenomena can generally be identified by some observer located outside the system that produces them [36, 38]. For example, the ant path can only be identified by those examining the foraging ant colony from some distance. Ants cannot perceive the whole path since they can only perceive their surrounding environment. However there are cases where emergent phenomena are identifiable from system components as well [18, 36, 64]. Examples of such cases are phenomena that emerge in social systems such as emergent leadership and emergent work ethics. In both cases the emergent phenomenon is identified by all society members. Upon arising the emergent, phenomena subsequently affect the behaviour of system components. In most definitions the view is that component behaviour is affected by downward causal powers appearing due to the emergent phenomenon. This downward influence is commonly referred to as downward causation [6, 36, 44]. In other cases it is suggested that emergence can also take place at the micro level giving rise to emergent component properties which evolve in parallel with the global emergent phenomenon. An example of such dual emergence is the conjugate proposed by Haan [38]. Depending on whether the emergent phenomenon is visible by the system components or not, its downwards effects are direct or indirect respectively. For example, once an ant path or a flock of birds has been shaped, then movements of individual ants or birds are aligned to follow the path or to not collide with each other in the flock respectively. This is done indirectly since neither the birds not the ants are aware of the existence of the whole flock or path respectively. In contrast, once a leader has emerged in a society, as is the case of a leader emerging in music quartets for instance (see Goldstein [36] for more details), all society members become aware of the leader and receive direct influence. The predictability of emergent phenomena is another issue of concern. In fact, in many definitions of emergence any unpredictable/unexpected phenomena are considered to be emergent [18, 36, 44, 50, 73]. In a first glance this seems to contradict the view that emergence could possibly be harnessed and used in artificial systems for specific purposes. The obvious question arising would be that if some outcome is indeed unpredictable, then how can we be certain that it will happen at all. Following this line of thought, we would conclude that we cannot hope to engineer emergence as the solution to a given problem. However, this problem is mitigated by the clarification that unpredictability of emergent phenomena actually means unpredictability in practice. In other words, emergent phenomena can be considered as completely unpredictable only the first time ever they are perceived since they could not be predicted in advance. After initial discovery of emergent phenomena however, models of cause and effect (both statistical and simulation ones) can be established between component behaviour and emergent outcomes, and hence emergent phenomena can become predictable in principle. In fact many emergent phenomena are discovered by simulations, and that approach is also widely applied to confirm that certain initial conditions, environmental dynamics and selected component behaviours will

62

G. Di Marzo Serugendo et al.

lead to specific emergent outcomes [5, 8, 28]. However, only probabilistic estimations of the type and time of occurrence of emergent phenomena can generally be made, and in cases where the environment has stochastic behaviour an emergent phenomenon cannot be exactly calculated in advance not even by simulation. In particular, some authors consider emergent phenomena as radically novel (see for example [36, 73]); however, this cannot generally be the case. Emergent phenomena are in most cases only practically novel since, given the system dynamics, it is not easy to accurately predict them in advance. For example, it is not easy to exactly predict a hurricane and its characteristics before it is shaped. However, the occurrence of a hurricane is not something radically novel, it is simply practically impossible to exactly determine in advance the detailed characteristics of the particular hurricane that finally occurred. As Chalmers in [18] argues, a Laplace machine fed with all necessary data would in principle be able to accurately predict any phenomenon except consciousness.10 Another issue of concern is the in principle derivability or reducibility of emergent phenomena. In many definitions, irreducibility characterises strong forms of emergence, while unpredictability is sufficient to justify weak ones [5, 18, 50]. However, irreducibility in principle has been strongly criticised. For example, in many cases irreducibility is considered as lack of knowledge, which an appropriate theory explaining how the currently irreducible emergent phenomenon is produced would cover [36, 50, 73]. In particular, apart from offering a means to conceptually link cause and effects in emergent phenomena that otherwise would have no concrete explanation, irreducibility does not assist in our understanding of emergence. For example, Bedau considers strong emergence as being scientifically irrelevant for the study of natural phenomena [5, 6], while, as mentioned above, Chalmers accepts only one case of strong emergent phenomenon, that of consciousness [19]. Another important issue in understanding emergence is how emergent phenomena occur. In many cases emergent phenomena are considered to pre-exist, for example as in the Aristotelian view of emergence. In other cases there is no explicit reference to the emergence process, for example as is done in proto-emergentist definitions where the production process is seen as a black box (for example, see Johnson in [50] referencing Alexander). Similarly, there is no reference to the mechanisms producing the emergent phenomena in definitions where emergent phenomena are considered as in principle irreducible to the properties and behaviour of the system components (for example, see [19, 73]). In the majority of cases, however, the emergent phenomenon is considered to be produced by some process which is executed at the component level. That process is necessarily dynamic [5, 36, 38, 44, 46]. As Goldstein emphasises, “. . . emergent phenomena are not pre-given wholes but arise as a complex system evolves over time. As a dynamical construct, emergence is associated with the arising of new attractors in dynamical systems . . . ”. The dynamism of the micro-level processes is 10 Chalmers

argues that the only irreducible emergent phenomenon is consciousness and any other phenomenon can be in principle derived given initial conditions and computing power. See [18, 19] for more details.

3 History and Definitions

63

mandatory for two main reasons: Firstly, our reasoning is done on the basis of physical monism,11 according to which the scientifically relevant emergent phenomena under examination are created only by physical powers and lows. Secondly, we require that emergent phenomena be practically (for example analytically) unpredictable in advance. The mechanisms that produce the emergent phenomenon are often considered to be based on interactions of individual components. The reason is that autonomy of interactions and a large number of components result in nonlinear system behaviour, which cannot be accurately predicted in advance. Therefore, several authors agree that component interactions in complex systems operating at the edge of chaos (namely exhibiting far from equilibrium dynamics) can lead to emergent outcomes [36, 44, 53]. In fact, this nonlinearity requirement has led many authors to propose that decentralised control, large number of components, and autonomous interactions are mandatory system properties in order to exhibit emergent phenomena [27, 31, 36, 44]. Nonlinearity is not caused only by local interactions among distributed components however. For example, in dynamical systems we can have behaviours based on iterated functions,12 which can lead to nonlinear behaviour and in particular to interesting emerging results [63]. The behaviour of such systems is largely unpredictable since it is governed by deterministic chaos. Among other characteristics, such systems can exhibit the butterfly effect [43, 45], namely their trajectory through state space is sensitively dependent on the initial conditions, and hence unobservably small causes can produce large effects. This fact also relaxes the requirement for the system to necessarily have a large number of components, and it actually can allow having emergent phenomena in systems with just one component. Stephan [73] provides a good relevant example of nonlinear behaviour produced by logistic functions. The example involves an iterated function defined on the logistic function y = mx(1 − x). Stephan demonstrates that for different values of m, there is different behaviour of the logistic function, for example, for m = 1, 5, the function converges to a constant value depending on initial value of x. Some authors have linked the appearance of emergent phenomena with selforganisation [31, 36, 43]. In particular, in many cases emergence is thought to occur only in self-organising complex systems operating in far-from-equilibrium dynamics and being in the edge of chaos [52]. In complex systems, the research very often centers on the emergent global dynamics of a whole system. It is usual in this approach to view the global properties of the system as emerging from the actions of its parts, rather than seeing the actions of the parts as being imposed from a dominant central source [45, 46]. Properly defined, however, there may be instances of selforganisation without emergence and emergence without self-organisation, and it is clear from the literature that the phenomena are not the same (see [27] for instance). For example, crystallisation or the creation of sand dunes do not require any form 11 See 12 The

Stephan in [73] for more details on physical monism.

values of iterated functions are calculated sequentially in a manner such that the output of each calculation is the input to the next one.

64

G. Di Marzo Serugendo et al.

of self-organisation. Both phenomena result from a dynamic process which eventually stabilises giving rise to the emergent outcome. Furthermore, dynamic behaviour does not occur only in complex systems. For example, the Game of Life cannot be characterised as a complex system, and yet it exhibits emergent phenomena. Therefore, the link between emergence and self-organisation generally remains an active research question [27, 31]. Further to the characterisation of emergent phenomena, there is a common view that they need to exhibit a level of adaptability and presistence, which implies that the system will have reached an organisational regime having some degree of stability and resistance to perturbations [20]. In fact, it is this very resistance to perturbations that for some authors differentiates true emergent phenomena from epistemological artefacts appearing due to pure chance. For example, Goldstein [36] discusses the case of some hill in USA which, when viewed from a certain angle and at a certain time of day, resembles the profile of the American president J.F. Kennedy. Obviously that is an epistemological artefact which does not really exist. Such epistemological artefacts, for instance shapes formed from play of light on leaves in a breeze, are considered to include serendipitous novelty and are clearly separated from authentically emergent phenomena (see Holland [46] for a discussion). The stable organisational regime linked with occurrence of emergent phenomena can be an equilibrium or an organisationally closed set of states, which, as mentioned in Sect. 3.3.1, are commonly referred to with the term attractor [46, 53]. In this respect, self-organisation is quite relevant since it can cause a system to reach an attractor regime and hence create the conditions for emergent phenomena to be created.

3.3.3 Operational Definition of Emergence in Computer Science Considering the definitions discussed above, the term emergence can be used in software engineering to refer to a dynamic process producing a global phenomenon that is practically unpredictable in advance. Such an emergent phenomenon will be dependent on system component functionality, and its unpredictability will be due to nonlinear system behaviour. This view of emergence includes both reducible and irreducible types of emergence described in the existing definitions, since operationally all cases of emergence in software systems are reducible in this respect. Therefore, we provide the following definition: Emergence is the process that causes a software system to produce an emergent phenomenon.

The emergence process can most commonly, but not necessarily, be a process of self-organisation. The emergent phenomenon can be anything perceived from the system’s stakeholders based on the system organisational status, which was not exactly predicted in advance. For example, an emergent phenomenon can be a particular system response, a piece of information that can consist of a solution to a given problem, a particular system state, or a particular process that the system may become capable of executing. An emergent phenomenon is generated and becomes

3 History and Definitions

65

identifiable when the system reaches some organisational regime, for example an attractor, namely a particular state or a set of states the system convergently settles to or iterates within.13 Therefore, an emergent result can be understood as an interpretation of a particular attractor the system has converged into. Hence, we can define an emergent result as follows: An emergent phenomenon produced by a software system is an interpretation of an attractor the system has converged into, which is practically unpredictable given the functionality of system components.

An attractor represents for a process a model of causal closure built with system states [44, 46]. When reaching the attractor, the process “closes in” and can not reach out except when significant perturbations are applied to the system. Attractors can have many different shapes, sizes and dimensions. The simplest one is a zero-dimensional point attractor which consists of a single state. This describes the situation where a system reaches an equilibrium. Furthermore, a quite common attractor is an one-dimensional limit cycle, where all states of the attractor are revisited at regular intervals. This describes certain far-from-equilibrium configurations where the system exhibits periodical behaviour, such as the Benard rolls. Other types of attractors, which are termed “strange”, are characterised by a non-integer, fractal dimension. This is a representative characteristic of certain types of chaotic processes [44]. An emergent result is generally perceived and identified as such by some observer who is external to the system producing it. There are cases, however, where the system components can also perceive the emergent result as a whole once it occurs. As mentioned previously, the emergent result always affects component behaviour and hence depends on whether system components can also perceive the emergent result that influence, termed downward causation (see Sect. 3.3.3), can be characterised as direct and indirect respectively. In the first case, system components will be directly informed of and affected by the emergent outcome, for example as is the case of a multi-agent system where an agent emerges as coordinator at some point and subsequently all other agents start contacting it to receive coordination commands. In the second case the behaviour of the system components will be affected without them being aware of the global emergent outcome, similarly to foraging ants moving along an emergent path without being aware of its existence. Hence we can define two types of emergent phenomena: Weak emergent phenomena are those that are identified only by an external observer and have an indirect influence on system component behaviour. Strong emergent phenomena are those that are identified both by an external observer and system components and have a direct influence on system components behaviour.

Although not exactly predictable in advance, emergent phenomena are in general not completely unexpected; if they were, they would not be particularly useful in software engineering. Totally unexpected emergent results are usually linked 13 Heylighen

[43] uses the term organisational closure to refer to the convergence of a system to a set of particular states as a result of a dynamic self-organisation process.

66

G. Di Marzo Serugendo et al.

with undesired system behaviour, one that was not intended when the system was designed but occurred indirectly during dynamic system operation. Therefore, in artificial systems emergent results should generally be expected with some probability. In fact, in the extreme case that the system components do not function stochastically, emergent results should be able to be completely regenerated given the same initial conditions and system operation rules. For example, in a software system simulating a foraging ant colony where ants have deterministic behaviour, the same ant path will re-emerge if we re-execute the system with the same initial conditions. Therefore, the challenge for software engineers is to build appropriate functionality into system components, so that, given appropriate initial conditions and environmental input, desired emergent results will be produced.

3.3.4 Properties of Emergent Phenomena In the various definitions of emergence, several properties have been attributed to emergent phenomena. Considering the definition given in the previous section, an emergent phenomenon is characterised by the following properties: • Global coherence. The phenomenon must have global scope, and it be must coherent and dependent on the functionality of the system components while at the same time being clearly separated from the properties of the constituent parts of the system. Furthermore, it should be understood as being perceived at a macro level while the underlying processes will be executed at a micro level. Therefore, there is a strong dependency between the dynamics observed at both macro and micro levels. • Novelty. Novelty refers to the fact that although the resulting phenomenon is derived by interpretation of some particular system organisation, reached by a process involving micro-level system parts, it is radically different from the individual properties of these parts, and it cannot be directly estimated from them. In particular, to identify the emergent phenomenon, different concepts and theories from those used to describe the micro-level activities are generally required. For example, in a foraging ant system the ant movements are described in terms of pheromones, while the emergent ant path is described in terms of concentration of ants to particular locations.14 Therefore, emergent phenomena are viewed as interpretations of particular system organisational arrangements. • Unpredictability. The emergent phenomenon is not practically predictable before its occurrence. Although emergent results appearing in software systems are derivable in principle, for example if resource and time consuming procedures such as simulations are applied, in practice it is not obvious when they will occur. Therefore in practice emergent phenomena can be anticipated only probabilistically. In fact, given that the exact phenomenon is not known in advance, the 14 See

[33, 64] for a discussion on how emergent phenomena are interpretations of epiphenomena produced from dynamic system operations.

3 History and Definitions

67

user can actually anticipate a class of emergent phenomena, a particular instantiation of which will occur with a given probability based on initial conditions, micro-level processes and environmental factors. Therefore from a requirements fulfilment perspective, such as problem solving, the system designer should attempt to engineer a generic class of phenomena that will potentially provide the problem solution or fulfil the particular requirements and then probabilistically expect, given past experience, that a member of this class will instantiate and provide the desired result. However, the exact occurrence of this particular instance is not known in advance because it depends on many factors ranging from the inherent system dynamism to stochastic environmental dynamics. For example, a system designer modelling a problem solving process as ant colony optimisation can anticipate that a solution will eventually emerge given the initial parameters and environmental dynamics based on the known behaviour of ant colony optimisation algorithms. However, she will not be able to predict the exact solution in advance; if that was the case, then the problem would be have been solved in the first place. It is similar to anticipating, based on previous knowledge, that some ant path will be formed between a food source and the ant nest. However, we will not be able to directly estimate the exact path in advance. • Dynamic (nonlinear) system behaviour. The emergent phenomenon is an interpretation of an attractor the system has converged to. Hence, the system where an emergent phenomenon is identified needs to exhibit dynamic nonlinear behaviour. A chain of linear activities enables explainability and predictability of a collective phenomenon. On the opposite, an emergent phenomenon originates from nonlinear activities at the micro-level, and is realised and becomes identifiable when the system reaches an attractor organisational regime.

3.3.5 Engineering Nonlinearity To be able to engineer software systems capable of producing emergent phenomena that fulfil particular requirements, we need to identify what necessary characteristics such systems must have. The source of emergent phenomena is nonlinearity, and hence it needs to be inherent in the behaviour of such systems. Subsequently, system designers need to engineer appropriate behaviours into system’s components, whose interaction will eventually produce some instance of a class of desired emergent phenomena with a given probability. Such component behaviours would be typically known from existing cause and effect models to generate particular overall system behaviours and produce emergent phenomena. There are different ways to engineer nonlinearity in artificial systems. As mentioned in Sect. 3.3.2.3, nonlinearity can be mathematically modelled in the behaviour of certain system components, for instance by using iterated functions or centralised rule-bases that result in deterministic chaos. However, this approach is not particularly strong in addressing the requirements of contemporary software. For example, centralised solutions suffer from the well-known bottleneck problem.

68

G. Di Marzo Serugendo et al.

Furthermore, they are more cumbersome as far as it concerns adapting to changing environmental conditions. Distributed nonlinear systems, on the other hand, are flexible and can adapt to the changing environment while approaching the solution to the problem. For example, an ant colony will converge to a path linking food source and nest despite any environmental perturbations, such as stones thrown on the forming ant path during foraging. Therefore the most beneficial sources of nonlinearity in this respect are complex interactions in distributed systems, such as the complex adaptive systems. Such systems are based on distributed components acting autonomously in a decentralised manner aiming to achieve own goals. Interactions are local and normally simplistic, but overall they result to global nonlinear system behaviour which can give rise to emergent phenomena. The resulting dynamic behaviour of such systems is typically self-organising. Furthermore, the mechanisms dealing with component interactions can be classified in two types: external and internal. The external mechanisms enable modification of the system’s behaviour to be initiated by its environment, for example by imposing constraints, enforcing rules and delegating artifacts. The internal mechanisms are ways to change the interaction dimensions that are unfolded by processes within the system. Interaction is essential to this framework because the events of novelty and innovation within a system arise from the interactions of these agents with each other and with the environment. The challenge for system designers therefore is how to specify local component behaviours so that they will overall form a complex adaptive system and particularly that they will produce a desired emergent outcome. More details about methodologies for engineering self-organisation and emergence are provided in Chap. 12.

3.4 Summary The concept of emergence has been studied since the Ancient Greece times, and it appeared in various domains such as philosophy, mathematics, physics, thermodynamics, systemics and complex systems. Its description has often been resumed to the phrase “a whole that is more than its parts”, but a lot of different definitions of emergence currently exist. For artificial systems, a sufficient definition considers emergent phenomena as interpretations of a particular system organisational regime, which are not practically predictable in advance. Emergence refers to the occurrence of some outcome which, despite not explicitly represented at a lower organisation level, appears at a higher level and cannot be understood by simply observing individual component behaviours. Emergent properties are intimately linked with dynamic self-organising systems having decentralised control and local interactions. Their source is commonly, although not necessarily, the nonlinearity arising from the locality of component interactions. Hence we accept that self-organising systems have the capability to alter their organisation (which results to changing the functionality they deliver) to adapt to external influences. What is organisation and what is functionality can be a matter

3 History and Definitions

69

of definition (since in the end they both change to react to external stimuli). However, in complex self-organising systems, where the possible organisations are too many to be exactly calculated or observed, and often stochastically determined, selforganising systems are still capable of producing them and function as to fulfil their purpose. Instead of attempting to eliminate emergent phenomena, it could be interesting to explore how this might be deliberately achieved and harnessed. That is, elaborate on how to engineer artificial systems with desirable emergent properties.

Key Points • Most self-organising software systems currently mimic natural systems to implement self-organising behaviour.

3.5 Problems–Exercises 3.1 Provide examples of cases where we need strong and weak self-organisation: (a) in the physical world; (b) in software systems. 3.2 Discuss the definition and principles of self-organisation as these were introduced for dynamic systems. Are the effects of self-organisation permanent? 3.3 Describe the requirements and implement a proof of concept demonstrator of a software system that exhibits the mandatory self-organising system properties introduced in Sect. 4.3.2.1. 3.4 Justify whether it is possible to have self-organisation without emergence and provide examples. 3.5 Describe the main emergentist schools and discuss the advantages and disadvantages of each approach. 3.6 Provide a definition of the term emergence, suitable for software systems, together with examples. Is it always necessary to have a decentralised system consisting of interacting components for emergent phenomena to appear? 3.7 Provide a definition and examples of the term “downward causation”. 3.8 Provide a definition together with examples and discuss the main properties of emergent phenomena.

70

G. Di Marzo Serugendo et al.

3.9 Describe an example and implement a proof of concept demonstrator of a software system capable of exhibiting emergent behaviour.

3.6 Further Reading The Vision of Autonomic Computing. A foundational introduction to Autonomic Computing. (J.O. Kephart and D.M. Chess, 2003, IEEE Computer, 36(1):41–50.) Advances in Applied Self-organizing Systems. A collection of papers describing software applications using self-organisation and emergence. (M. Prokopenko, (ed.), Advanced Information and Knowledge Processing series, 2008, Springer, London.) Self-organization. An instructive review article on history and present status of self-organisation mainly in physical and biological systems. (H. Haken, Scholarpedia, 2008, 8(1), http://www.scholarpedia.org/article/Self-organization.) Special issue on Self-organization in Distributed Systems Engineering. A special issue comprising papers on self-organising mechanisms and applications. (F. Zambonelli and O.F. Rana (eds.), 2005, IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, 35(3).) The Science of Self-organization and Adaptivity. A foundational review of the self-organisation and emergence concepts and models as they appear in various scientific areas. (F. Heylighen, 2001, Knowledge Management, Organizational Intelligence and Learning, and Complexity, in: The Encyclopedia of Life Support Systems, EOLSS, L.D. Kiel (ed.) 2001, Eolss Publishers, Oxford, pp. 253–280.) Swarm Intelligence: From Natural to Artificial Systems. A comprehensive book on self-organising algorithms drawn from natural systems. (E. Bonabeau, M. Dorigo and G. Theraulaz, 1999, Oxford University Press.) Self-organization in Biological Systems. A detailed presentation of self-organisation mechanisms in biological systems. (S. Camazine, J.-L. Deneubourg, N.R. Franks, J. Sneyd, G. Theraulaz and E. Bonabeau 2001, Princeton University Press.) The many Facets of Natural Computing. An up-to-date review article on recent advances in natural computing. (L. Kari and G. Rosenberg, 2008, Communications of the ACM, 51(10):72–83.) Self-managed Systems and Services. An overview of the uses of self-organisation in implemented systems and services. (J.P. Martin-Flatin, J. Sventek and K. Geihs, 2006, Guest Editorial of special issue, Communications of the ACM, 49(3):36– 39.) That special issue includes additional relevant papers as well.

References 1. Abdallah, S., Lesser, V.R.: Modeling task allocation using a decision theoretic model. In: Proceedings of Fourth International Joint Conference on Autonomous Agents and Multiagent Systems, Utrecht, Netherlands (2005)

3 History and Definitions

71

2. Administrator of English Wikipedia: Diagram of convection cells. Creative Commons Attribution, vol. 2011 (2007). http://en.wikipedia.org/wiki/File:ConvectionCells.svg 3. Ashby, W.R.: Principles of the self-organizing dynamic system. J. Gen. Psychol. 37, 125–128 (1947) 4. Bar-Yam, Y.: Dynamics of Complex Systems. Perseus Books, Cambridge (1997) 5. Bedau, M.A.: Weak emergence. In: Tomberlin, J. (ed.) Philosophical Perspectives: Mind, Causation and World, vol. 11, pp. 375–399. Blackwell, Malden (1997) 6. Bedau, M.A.: Downward causation and the autonomy of weak emergence. Principia 6(1), 5–50 (2002) 7. Berlekamp, E.R., Conway, J.H., Guy, R.K.: Winning Ways for Your Mathematical Plays, vol. 2, 2nd edn. AK Peters, Wellesley (2001) 8. Bernon, C., Gleizes, M.P., Picard, G.: Enhancing self-organising emergent systems design with simulation. In: O’Hare, G.M., Ricci, A., O’Grady, M.J., Dikenelli, O. (eds.) Engineering Self-Organising Systems. Third International Workshop, ESOA 2005, Utrecht, The Netherlands, 25 July 2005. Lecture Notes in Artificial Intelligence, vol. 4457, pp. 284–299. Springer, Berlin (2007). Revised Selected Papers 9. Blitz, D.: Emergent Evolution: Qualitative Novelty and the Levels of Reality. Episteme, vol. 19. Springer, New York (1992) 10. Bonabeau, E., Dorigo, M., Théraulaz, G.: Swarm Intelligence: From Natural to Artificial Systems. Oxford University Press, London (1999) 11. Brennan, R.W.: Holonic and multi-agent systems in industry. Knowl. Eng. Rev. 16(4), 375– 381 (2001) 12. Briscoe, G., Dini, P.: Towards autopoietic computing. In: Colugnati, F.A.B., Lopes, L.C.R., Barretto, S.F.A. (eds.) Digital Ecosystems. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol. 67, pp. 199–212. Springer, Berlin (2010) 13. Brueckner, S., Parunak, H.V.D.: Self-organising Manet management. In: Serugendo, G., Karageorgos, A., Rana, O.F., Zambonelli, F. (eds.) Engineering Self-Organising Systems, Nature-Inspired Approaches to Software Engineering. Lecture Notes in Artificial Intelligence, vol. 2977, pp. 20–35. Springer, Berlin (2004) 14. Brueckner, S., di Marzo Serugendo, G., Karageorgos, A., Nagpal, R. (eds.): Engineering Self-Organising Systems. Lecture Notes in Artificial Intelligence, vol. 3464. Springer, Berlin (2005) 15. Brueckner, S., Robertson, P., Bellur, U. (eds.): Second IEEE International Conference on SelfAdaptive and Self-Organizing Systems (SASO 2008), Venice, Italy. IEEE Computer Society, Los Alamitos (2008) 16. Camazine, S., Deneubourg, J.L., Franks, N.R., Sneyd, J., Théraulaz, G., Bonabeau, E.: SelfOrganization in Biological Systems. Princeton University Press, Princeton (2001). 2nd edn. (2003) 17. Capera, D., Georgé, J.P., Gleizes, M.P., Glize, P.: Emergence of organisations, emergence of functions. In: AISB’03 Symposium on Adaptive Agents and Multi Agent Systems (2003) 18. Chalmers, D.J.: Varieties of emergence. Tech. Rep., University of Arizona (2002) 19. Chalmers, D.J.: Strong and weak emergence. In: Davies, P., Clayton, P. (eds.) The Reemergence of Emergence, pp. 244–256. Oxford University Press, Oxford (2006) 20. Churchland, P.: Matter and Consciousness. MIT Press, Cambridge (1984) 21. Collier, J.: Fundamental properties of self-organisation. In: Arshinov, V., Fuchs, C. (eds.) Emergence, Causality, Self-Organisation, pp. 150–166. NIA-Priroda, Moscow (2003) 22. Collier, J.: Self-organization, individuation and identity. Rev. Int. Philos. 59, 151–172 (2004) 23. Contractor, N.S., Seibold, D.R.: Theoretical frameworks for the study of structuring processes in group decision support system—adaptive structuration theory and self-organising systems theory. Hum. Commun. Res. 19(4), 528–563 (1993) 24. Corning, P.A.: The re-emergence of “emergence”: a venerable concept in search of a theory. Complexity 7(6), 18–30 (2002) 25. Craig, W.R.: Flocks, herds and schools: a distributed behavioral model. SIGGRAPH Comput. Graph. 21(4), 25–34 (1987)

72

G. Di Marzo Serugendo et al.

26. Cunninham, B.: Capturing qualia: higher-order concepts and connectionism. Philos. Psychol. 14(1), 29–41 (2001) 27. De Wolf, T., Holvoet, T.: Emergence versus self-organisation: different concepts but promising when combined. In: Brueckner, S., Di Marzo Serugendo, G., Karageorgos, A., Nagpal, R. (eds.) Engineering Self-Organising Systems. Lecture Notes in Computer Science, vol. 3464, pp. 77–91. Springer, Berlin (2005) 28. De Wolf, T.D., Holvoet, T., Samaey, G.: Engineering self-organising emergent systems with simulation-based scientific analysis. In: Brueckner, S., Serugendo, G.D.M., Hales, D., Zambonelli, F. (eds.) Engineering Self-Organising Systems. Third International Workshop, ESOA 2005, Utrecht, The Netherlands, 25 July 2005. Lecture Notes in Artificial Intelligence, vol. 3910, pp. 138–152. Springer, Berlin (2006). Revised Selected Papers 29. Di Marzo Serugendo, G., Karageorgos, A., Rana, O.F., Zambonelli, F. (eds.): Engineering Self-Organising Systems: Nature-Inspired Approaches to Software Engineering. Lecture Notes in Computer Science, vol. 2977. Springer, Berlin (2004) 30. Di Marzo Serugendo, G., Gleizes, M.P., Karageorgos, A.: Self-organisation and emergence in MAS: an overview. Inform. Slov. 30(1), 45–54 (2006) 31. Di Marzo Serugendo, G., Gleizes, M.P., Karageorgos, A.: Self-organisation in multi-agent systems. Knowl. Eng. Rev. 20(2), 165–189 (2005) 32. Di Marzo Serugendo, G., Martin-Flatin, J.P., Jelasity, M., Zambonelli, F. (eds.): First International Conference on Self-Adaptive and Self-Organizing Systems (SASO 2007). IEEE Computer Society, Cambridge (2007) 33. Forrest, S.: Emergent computation: self-organizing, collective, and cooperative phenomena in natural and artificial computing network. In: Proceedings of the Ninth Annual CLNS Conference. MIT Press, Cambridge (1991) 34. Gershenson, C.: Self-organizing traffic lights. Complex Syst. 16(1), 29–53 (2005) 35. Glansdorff, P., Prigogine, I.: Thermodynamic theory of structure, stability and fluctuations. Am. J. Phys. 41, 147–148 (1973) 36. Goldstein, J.: Emergence as a construct: history and issues. Emergence 1(1), 49–72 (1999) 37. Grassé, P.P.: La reconstruction du nid et les coordinations interindividuelles chezbellicositermes natalensis etcubitermes sp la théorie de la stigmergie: Essai d’interprétation du comportement des termites constructeurs. Insectes Soc. 6(1), 41–80 (1959). doi:10.1007/BF02223791 38. Haan, J.D.: How emergence arises. Ecol. Complex. 3, 293–301 (2007) 39. Hadeli, K., Valckenaers, P., Kollingbaum, M., Van Brussel, H.: Multi-agent coordination and control using stigmergy. Comput. Ind. 53(1), 75–96 (2004) 40. Haken, H.: Information and Self-organization: A Macroscopic Approach to Complex Systems. Springer, Berlin (1988) 41. Haken, H.: Self-organization. Scholarpedia 3(8) (2008) 42. Harrington, A.: Reenchanted Science: Holism in German Culture from Wilhelm II to Hitler. Princeton University Press, New Jersey (1999) 43. Heylighen, F.: The science of self-organization and adaptivity. In: Kiel, L.D. (ed.) Knowledge Management, Organizational Intelligence and Learning, and Complexity, The Encyclopedia of Life Support Systems, EOLSS, pp. 253–280. Eolss Publishers, Oxford (2001) 44. Heylighen, F.: Complexity and self-organization. In: Bates, M.J., Maack, M.N. (eds.) Encyclopedia of Library and Information Sciences. Taylor & Francis, London (2008) 45. Holland, J.H.: Hidden Order: How Adaptation Builds Complexity. Addison-Wesley, Longman, Redwood City (1995) 46. Holland, J.H.: Emergence: From Chaos to Order. Oxford University Press, Oxford (1998) 47. Holland, J.: Studying complex adaptive systems. J. Syst. Sci. Complex. 19(1), 1–8 (2006). doi:10.1007/s11424-006-0001-z 48. Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. 79(8), 2554–2558 (1982) 49. Ishida, T., Gasser, L., Yokoo, M.: Organisation self-design in distributed production systems. IEEE Trans. Knowl. Data Eng. 4(2), 123–134 (1992) 50. Johnson, C.W.: What are emergent properties and how do they affect the engineering of complex systems? Reliab. Eng. Syst. Saf. 91(12), 1475–1481 (2006)

3 History and Definitions

73

51. Jones, J.D.: Bifurcations. Accessed on 24-07-2011, http://www.mcasco.com/Order/bifurcat.html (2011) 52. Kaufman, S.: The Origin of Order: Self-Organization and Selection in Evolution. Oxford University Press, New York (1993) 53. Kaufman, S.: At Home in the Universe: The Search for the Laws of Self-Organization and Complexity. Oxford University Press, London (1995) 54. Kephart, J.O., Chess, D.M.: The vision of autonomic computing. Computer 36(1), 41–50 (2003). doi:10.1109/MC.2003.1160055 55. Koestler, A.: The Ghost in the Machine. Hutchison and Co, London (1967), the Danube edn. 56. Langton, C.: Artificial Life: An Overview. MIT Press, Cambridge (1997) 57. Lewes, J.: Problems of Life and Mind, vol. 2. Kegan Paul, London (1875) 58. Malvile, E., Bourdon, F.: Task allocation: a group self design approach. In: International Conference on Multi-Agent Systems, pp. 166–173. IEEE Press, New York (1998) 59. Mamei, M., Zambonelli, F.: Programming stigmergic coordination with the TOTA middleware. In: Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS’05, The Netherlands, pp. 415–422. ACM, New York (2005) 60. Maturana, F., Norrie, D.H.: Multi-agent mediator architecture for distributed manufacturing. J. Intell. Manuf. 7, 257–270 (1996) 61. Maturana, H.R., Varela, F.J.: Autopoiesis and Cognition: The Realization of the Living. Boston Studies in the Philosophy of Science. Springer, Berlin (1991). 2nd edn. (1980), 1st edn. (1973) 62. Maturana, H.R., Varela, F.J.: The Tree of Knowledge: The Biological Roots of Human Understanding. Shambhala, Halifax (1992) 63. Meiss, J.: Differential Dynamical Systems. SIAM, Philadelphia (2007) 64. Muller, J.P.: Emergence of collective behaviour and problem solving. In: Omicini, A., Peta, P., Pitt, J. (eds.) Engineering Societies in the Agents World 4th International Workshop, ESAW 2003, London, UK, 29–31 October 2003. Lecture Notes in Artificial Intelligence, vol. 3071, pp. 1–20. Springer, Berlin (2004) 65. Nicolis, G., Prigogine, I.: Self-Organization in Non-Equilibrium Systems. Wiley, New York (1977) 66. Paslack, F.: Urgeschichte der Selbstorganisation. Vieweg, Braunschweig (1991) 67. Picard, G., Bernon, C., Gleizes, M.P.: ETTO: emergent timetabling by cooperative selforganization. In: Engineering Self-Organizing Applications—Third International Workshop (ESOA) at the Fourth International Joint Conference on Autonomous Agents and MultiAgents Systems (AAMAS’05), July 2005, Utrecht, Netherlands. Lecture Notes in Artificial Intelligence (LNAI), vol. 3910, pp. 31–45. Springer, Berlin (2005) 68. Polani, D.: Foundations and formalizations of self-organization. In: Prokopenko, M. (ed.) Advances in Applied Self-Organizing Systems, pp. 19–37. Springer, London (2008) 69. Prokopenko, M.: Design vs. self-organization. In: Prokopenko, M. (ed.) Advances in Applied Self-organizing Systems, pp. 3–17. Springer, London (2008) 70. Rumelhart, D.E., McClelland, J.L. (eds.): Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Foundations, vol. 1. MIT Press, Cambridge (1986) 71. Shehory, O., Kraus, S., Yadgar, O.: Emergent cooperative goal-satisfaction in large scale automated-agent systems. Artif. Intell. 110(1), 1–55 (1999) 72. Stassner, J., Dobson, S., Fortes, J., Gowsami, K. (eds.): International Conference on Autonomic Computing (ICAC 2008). IEEE Press, Chicago (2008) 73. Stephan, A.: Emergentism, irreducibility, and downward causation. Grazer Philos. Stud. 65(1), 77–93 (2002) 74. Sternberg, R.J., Mio, J.S.: Cognitive Psychology. Wadsworth, Belmont (2005) 75. Varela, F.J.: Principles of Biological Autonomy. Elsevier, New York (1979) 76. Vrba, P., Tichy, P., Ma˘rik, V., Hall, K.H., Staron, R., Maturana, F.P., Kadera, P.: Rockwell automation’s holonic and multiagent control systems compendium. IEEE Trans. Syst. Man Cybern. Part C, Appl. Rev. 41(1), 14–30 (2011) 77. Yousif, M., Rana, O.F., Fortes, J., Goswami, K. (eds.): International Conference on Autonomic Computing (ICAC 2007), Jacksonville, Florida, USA. IEEE Press, New York (2007)

74

G. Di Marzo Serugendo et al.

78. Zambonelli, F., Parunak, H.V.D.: Signs of a revolution in computer science and software engineering. In: Petta, P., Tolksdorf, R., Zambonelli, F. (eds.) Engineering Societies in the Agents World 3rd International Workshop, ESAW 2002, Madrid, Spain, September 2002. Lecture Notes in Computer Science, vol. 2577, pp. 120–125. Springer, Berlin (2003) 79. Zambonelli, F., Gleizes, M.P., Mamei, M., Tolksdorf, R.: Spray computers: frontiers of selforganization for pervasive computing. In: Second International Workshop on Theory and Practice of Open Computational Systems (TAPOCS 2004) in 13th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE’04), Modena, Italy, pp. 397–402. IEEE Press, New York (2004)

Chapter 4

Self-organisation in Natural Systems Inspiring Self-organising Software Paul Marrow and Jean-Pierre Mano

Self-organisation in software offers advantages in dealing with dynamic environments and extensive communication loads. Nature is the teacher we humbly have to follow, and learn from its examples.

Objectives This chapter will provide the reader with an understanding of: • The diversity of self-organising systems in the natural world which can provide inspiration to software development; • Examples of models of natural self-organising systems which can be developed or simulated in software; • Examples of where inspiration from natural self-organising systems has led to applications in self-organising software.

4.1 Introduction To effectively construct self-organising software, it is worthwhile investigating existing systems which are already successful in utilising self-organisation. Thus, unravelling natural systems is of interest, in the widest sense, meaning not just living P. Marrow BT Group plc, Ipswich, UK e-mail: [email protected] J.-P. Mano () UPETEC, Ramonville Saint-Agne, France e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_4, © Springer-Verlag Berlin Heidelberg 2011

75

76

P. Marrow and J.-P. Mano

systems but also the mathematical, physical, chemical and biochemical systems that underlie them. Biological systems can give numerous examples of self-organisation in many different taxonomic groups and at many scales. It should not be forgotten that human beings are themselves living organisms and human behaviour and social interactions can give insights into self-organisation as well. This chapter is concerned with examples of self-organisation in natural systems paying particular emphasis on how they can inspire the development of selforganising software. A number of examples are discussed, and where appropriate references are made to software applications that have drawn upon these natural examples. In some cases this provides an introduction to chapters later in the book where particular examples will be considered in more detail.

4.2 Theoretical Notions Emergence and self-organisation in natural systems are based on a number of key underlying concepts, which will be summarised in this section before we proceed to discuss more specific examples. As mentioned previously in Chaps. 2 and 3, Self-organisation is the change of a group of entities from a disordered to an ordered state, without the need of an external driving force. This is such a general term which is difficult to define more precisely. Camazine et al. [10] provide many biological examples of self-organisation. Emergence is often described as the process driven by Self-organisation, or indeed the two terms are confounded. There is a lack of agreement about what falls under the heading of emergence, except that it is generally agreed to include processes that lead to system self-organisation, and thus is relevant to this chapter. Morowitz [49] attempts to review many different examples of convergence; Johnson [37] provides a popular introduction. Natural systems, with respect to self-organisation, are often restricted to biological systems (for instance see Camazine et al. [10] for many outstanding examples). Although these biological systems depend on a complicated set of chemical and biochemical interactions, all natural self-organising systems are based on the same mathematical and physical concepts. Some of these areas have already been mentioned in previous chapters, but in this chapter natural systems will be further elaborated on to include those mathematical, physical, chemical and biochemical systems that are required to understand the inspiring role of living systems on self-organising software, without attempting to describe them in detail. Evolution through natural selection underpins the majority of living systems. The mechanism for its operation was identified almost in parallel by Charles Darwin and Alfred Russell Wallace, but it is the former who most successfully publicised this landmark scientific achievement [17]. Evolution depends on the existence of populations made up of individuals with variation in characteristics that are heritable over generations. Individuals must be able to reproduce, and the degree to which the parent’s characteristics are represented in the offspring depends on interaction between the population and its environment (selection). After Darwin’s classic work

4 Self-organisation in Natural Systems Inspiring Self-organising Software

77

[17] there have been many more recent textbooks that are more accessible for the present-day reader (for example see Futuyma [24]). Ecology is the study of the interaction of living organisms with each other and with their environment. Living individuals or populations do not exist in isolation. They interact, with each other, especially to obtain food (for instance, as predators or prey), and with the resources and conditions of the environment in which they exist. Ecology is relevant to self-organising systems, and it provides inspiration for the development of self-organising software because many features of interactions in living systems self-organise and persist without external effects, despite that external effects, and particularly those originating from humans, are becoming increasingly of attention. Begon et al. [8] provide a comprehensive introduction to this topic. Behaviour is the daily activity of animals, and although this can depend on the decisions of individuals in isolation, these decisions often lead to the convergence of many animals in groups, under many different names, such as swarms, flocks and shoals. Such groups may consist of many creatures whose individual intelligence and ability are limited, as is the case in many social insects, but which can achieve great results when combined in large swarms (see Bonabeau et al. [9]). Alternatively, more complex creatures can achieve greater self-organisation through combined individual behaviours than they could alone, which is why this is an important topic for this chapter (see Camazine et al. [10]).

4.3 Self-organisation in Non-living Systems Chapters 2 and 3 have already introduced some of the principles involved with the concept of self-organisation. This chapter is concerned with the inspiration that selforganisation in natural systems can provide to the construction of self-organising software. When we think of natural systems, we tend to think of living systems. However, natural systems can be also viewed from other perspectives, such as mathematical, physical, chemical and biochemical, which underlie and can model and explain phenomena observed in living systems. Due to space limitations, we do not examine non-living systems in detail; we only highlight certain self-organisation aspects that have attracted researchers’ attention in recent years.

4.3.1 Chaotic Dynamics Mathematicians have studied differential equations, both in discrete and in continuous time, in considerable detail. For example, Poincaré studied the properties of equations used to describe the motion of a pendulum and showed that they are sensitive to initial conditions and that the error in oscillations grows exponentially as parameters are changed [3, 50]. Poincaré suggested that prediction was in that case impossible, but later work has shown that it is generally possible to visualise the transition from predictable oscillations to unpredictable behaviour through phase

78

P. Marrow and J.-P. Mano

portraits or bifurcation diagrams. This unpredictable behaviour has become known as chaos [3]. ω dω = − − sin θ + g cos φ, (4.1) dt q dθ = ω, (4.2) dt dφ (4.3) = ωD . dt Although bifurcation diagrams appear to show behaviour of extreme complexity in the chaotic region, in fact a lot of information can be gained from this analysis, such as within what range of parameters systems can be feasibly expected to selforganise into regular behaviour. The equations above represent the behaviour of a pendulum forced away from its regular oscillation (see [3]), and plotting g versus ω at a fixed phase in the cycle will produce a bifurcation diagram. It is also of interest that existing natural complex systems are in fact generated by chaotic dynamics, and it has been suggested that many natural phenomena arise in this way [27]. Understanding when chaotic dynamics is likely to occur could be an important starting point in self-organising software development. Therefore it is necessary to identify the types of software where self-organisation can play a role, for example in cases where some form of periodicity is likely to occur.

4.3.2 Complexity Scientists have often approached the problem of self-organisation as part of the wide subject of complexity science. Examples of questions posed include why there is a high number of complex natural phenomena and how complexity can be comprehensively defined. In particular, it is an open issue whether it is possible to define a notion of complexity that would enable quantitative complexity comparisons between different systems. Gell-Mann [26] discusses several definitions of complexity and stresses that complexity measures may be theoretically or practically uncomputable in contexts where they could be useful. Shalizi and Crutchfield [57] use an approach based on statistical mechanics, and many other alternatives exist as well. Many researchers in complexity science refer to complex adaptive systems as a general term which covers both living and non-living systems that exhibit properties of complexity, both natural and artificial, including the areas covered in this book. If some agreement could be established on the nature of complexity, this could contribute to the design process of self-organising software. Software components could efficiently self-organise to achieve complex systems management. Alternatively, this understanding of complexity could be useful for identifying whether the self-organisation process itself is actually useful for reducing or maintaining software complexity at a useful level. However, there are limited examples of such approaches at present time.

4 Self-organisation in Natural Systems Inspiring Self-organising Software

79

4.3.3 Chemical Systems Chemical and biochemical systems also exhibit self-organisation. A famous chemical example is the oscillator discovered by Belousov, and published by Zhaboniskii, now more often known as the Belousov–Zhaboniskii or B–Z reaction (for example see [50]). Starting from a mixture of chemicals, this can lead to organised oscillatory states, which have been shown to be the basis of oscillatory phenomena in living organisms [51] as well as in non-living chemical reactions. Chemistry was also the origin of the concept of quasispecies (see [52] cited in Chap. 3), which refers to a collection of gene sequences generated by a mutation– selection process. This can be confusing because the term species intuitively leads to some type of living organism, while quasispecies refers to groups of identical molecules. However, in reality not all molecules are exactly identical because mutation and selection actually modify them in an evolutionary manner thus transforming them to quasispecies. n dxi xj fj qj i − φxi . (4.4) = dt j =0

Equation (4.4) shows how fast sequence i can be obtained, depending upon the availability of sequence j and the rate at which sequence j can generate sequence i through mutation and selection [52]. The φ term refers to removal of sequences so that the total population size is constant. Based on this equation, we obtain a means for describing the development of self-organisation through molecular evolution, and thus acquire insight on non-living chemical systems which can be useful for representing change in living systems. This information can assist in the development of self-organising software, as it would be easy to implement the above equation in an algorithm.

4.4 Self-organisation in Biological Systems Fundamental to the emergence of self-organisation in biological systems is evolution, which is why this is considered first. Evolution depends on the interaction between organisms—that is ecology, which we consider next. A third general phenomenon of self-organisation found in many biological systems is that of pattern formation, which is also described next. Subsequently, we proceed to a range of different biological systems that show particularly interesting properties relevant to self-organisation in software: cellular systems, and behaviour in vertebrates and social insects.

4.4.1 Evolution Evolution through natural selection has been responsible for the great diversity of living organisms, and the interaction of the outcome of biological evolution with

80

P. Marrow and J.-P. Mano

development, ecological interactions and behaviour has resulted in a great deal of self-organising systems, from which developers of self-organising software have learned a great deal. The most immediate insight that computer scientists have from biological evolution has been the creation of evolutionary algorithms, which implement evolutionary processes in computers that are often difficult or impossible to solve by alternative algorithms.

4.4.1.1 Biological Evolution In biological evolution it is not easy to say when a species starts evolving: new species arise by the process of speciation from existing species, but there are no simple generalisations about what this involves, and since we are often dealing with current species where the only evidence for speciation is from fossils, it may be very difficult to get details. However there is much current data indicating that biological evolution through natural selection is going on today (for example see [20]). Fitness is a term used in evolutionary biology to describe the potential or actual capability of an individual to pass on heritable characteristics to future generations. There are different ways to measure fitness depending on the species, environmental or experimental context, and purpose of measurement. However, there is a common feature with which most measures of fitness correlate with reproductive success, which is contribution to offspring in future generations. Therefore, individuals that have inheritable characteristics that produce high fitness are more likely to pass them on to offspring. Biological offspring may be almost identical to their parents if they are produced asexually. In that case the main factor that can cause changes is mutation, near-random changes in the genetic material of an organism stimulated by the environment. However many living organisms reproduce sexually, and in this case two sources of genetic material are combined to create a new genome in the process of recombination. Thus offspring can have very different characteristics from either one of their parents, and mutation can introduce some additional change. The probability of offspring surviving and contributing to future generations depends, as in the case of their parents, on their fitness. Offspring fitness in turn derives from the characteristics that offspring have inherited from their parents, and their interaction with their environment through the processes of development, ecology and behaviour. This is selection. Since populations of living organisms last for multiple generations fitness development can be viewed as an iterative and evolving process similar to those modelled by loops in software, something that has been a key inspiration for the origination of evolutionary algorithms. The fitness development loops can continue as long as a species exists, in principle until a new species arises as a result of evolution from an old one (speciation), or until a species becomes extinct, which are both not particularly common cases.

4 Self-organisation in Natural Systems Inspiring Self-organising Software

81

Fig. 4.1 Pseudocode of an evolutionary algorithm. In actual evolutionary algorithms, code will be significantly more complex. For explanation of terms, see text

4.4.1.2 Evolutionary Algorithms The pseudocode of Fig. 4.1 describes a very basic form of an evolutionary algorithm. The idea of an evolutionary algorithm was inspired by thinking about biological evolution in an algorithmic manner, while motivated by the need to find novel algorithms to tackle problems which many existing algorithms were not very successful at finding solutions—often ones where there were multiple solutions which were sub-optimal and distracted from the optimal solution. All evolutionary algorithms are based on software that describes initially a population of individuals each of which is defined by a genotype. A genotype may be as simple as a list of integers that is used to distinguish individuals, or it may be more complicated, depending on the nature and aims of the algorithm. A fitness function created in relation to the problem that the evolutionary algorithm is created to address is used to evaluate the fitness of each individual. The algorithm in its simplest form then enters a loop, where the results of the fitness evaluation of the individuals in the population are used to identify which individuals generate new offspring. This is usually a sub-set of the existing population, depending on the threshold of fitness which must be passed for reproduction. The genotypes of the individuals that have been selected for reproduction then go through processes of crossover, where input from the genotype of more than one individual is used to generate a new genotype for an offspring individual, analogous to recombination between sexually reproducing organisms in nature, and mutation, where stochastic changes are made to the genotype of the offspring in a manner inspired by the process of mutation in nature. The precise details and relative importance of crossover and mutation can vary substantially between different implementations of evolutionary algorithms. The fitness function is then used to evaluate the fitness of the offspring produced, which is compared with the fitness of the parent individuals. Low-fitness parents are removed and replaced with high-fitness offspring. At this point the individual and mean fitness of the population can then be compared with the objective that the algorithm has been created to solve. Ideally such a process should lead to an increase, or at least stabilisation in mean fitness over time, and the element of stochasticity introduced by mutation and by choice of individuals for recombination should avoid all individuals converging on an identical genotype that is sub-optimal. When this process is repeated over many iterations (equivalent to biological generations), it can lead to populations that have quite different characteristics from those started with.

82

P. Marrow and J.-P. Mano

Evolution has been tremendously useful in living systems in generating organisms that self-organise, as the following sections will indicate. Nothing that takes place in ecology, biological development or behaviour takes place in isolation from the evolutionary basis of living organisms. This is even true of human behaviour, although many would argue that we are a very unusual example. It is thus not surprising that evolutionary algorithms have also been useful in many contexts, often relating to self-organisation. The concept of an evolutionary algorithm has diverged into a number of different strands, including: • Genetic Algorithms, where the problem is represented as a string of numbers. This type of evolutionary algorithm has often been applied to mathematical optimisation problems and was originally introduced by Holland [34]. • Genetic Programming, where the genotype itself is a program, and so the evolutionary algorithm process seeks to improve a program applied to a particular function. • Evolutionary Programming, where the parameters of a fixed program are evolved, thus improving a program in a different way from Genetic Programming. • Evolutionary Strategies, where problem is represented as a vector of real numbers, and mutation rates can adapt. Evolutionary algorithms have been applied in many different contexts such as optimisation, network design [29], resource management [44], multi-agent systems [46] and ecology [67]. From the perspective of self-organisation perhaps their most important feature is that they can be used to optimise the control of distributed systems which would otherwise persist in a disordered state due to their complexity. This is inevitably true in many networked applications or services where it is no-longer possible to gain a global view of the network and applications running on it, because of its scale and the rapid pace of change of operations.

4.4.2 Ecology Ecology is the study of interactions between organisms, and between organisms and their environment. It is inevitably linked into evolution, because the environment, part of which is composed of other organisms, imposes a selective pressure on populations to evolve in new directions. It is linked to biological development, because multicellular creatures go through this period of change on the way to adulthood during which they are subject to pressures from their environment in ways which are different from those that they will face as adults. Ecology is also linked to behaviour, because individual behavioural decisions may place an organism in the unfortunate position of facing superior predators as a prey item, or in the opposite case of identifying a tasty item of food. Ecology is key to self-organisation in natural systems because interactions between individuals and populations lead to networks of great complexity and, despite

4 Self-organisation in Natural Systems Inspiring Self-organising Software

83

dynamic change, to considerable stability and robustness. Plants and other photosynthetic organisms draw upon the Sun and chemicals available from the environment to generate the nutrients they require to persist, and thus provide the basis of food chains. Herbivorous organisms live on plants but are themselves subject to predation by predators. There may be top predators (who only consume other predators) in some food chains. When all cycles of energy and resources in addition to those in between organisms are taken into account, we have an ecosystem. Since a biological ecosystem depends so fundamentally on climate and organisms that can persist in appropriate environmental conditions to support herbivores, this would seem extraordinarily unstable. What is surprising is that in a lot of cases ecosystems and food webs which have been observed to change so much over lengthy and detailed studies retain much similarity and appear very robust [42, 64]. A simple and classic set of equations gives us clues as to why ecological systems can often self-organise and persist. The Lotka–Volterra equations (described in this form in [50] for instance) dN = N(a − bP ), (4.5) dt dP = P (cN − d), (4.6) dt were originally developed to explain predation in the Adriatic. Here N is the number of prey, P the number of predators, and t is time. Although much simpler than any realistic predator–prey interaction, they show a relevant characteristic: the interdependence of predator and prey—the number of one always depends on that of the other. So although the predator can through increased activity reduce prey numbers, this will result in predator numbers falling because there will not be enough to support the predator, and consequently prey numbers will increase again. This oscillation can allow both species to persist. Although more complicated in real ecosystems, this interdependence is one of the reasons for the persistence of interacting species, and the complexity of biological ecosystems has inspired artificial ones. Multi-agent systems are a good example in software, where many software components interact. The DIET system was specifically inspired by ecosystems, in developing lightweight agents that could be implemented into large scalable and evolvable populations [33, 46]. Since that project, the capabilities of networked computing technologies have advanced, and there is more scope for drawing upon the properties of robustness despite complexity of natural ecological systems in developing self-organising software.

4.4.3 Pattern Formation Self-organisation emerges in many living things through pattern formation: complicated patterns emerge through the life-history of the creature rather than any external decisions. While the majority of the diversity of living organisms on our planet

84

P. Marrow and J.-P. Mano

(something like 90%) are made up of simpler procaryotic cells and are typically microscopic and unicellular, it is the more sophisticated eucaryotic minority in which we can see many examples of more complicated macroscopic organisms that show examples of emerging pattern formation [62]. The slime moulds and slime nets are perhaps one of the most extraordinary examples. They form four kingdoms within the eucaryotes, with the common feature that they spend part of their life as singlecelled organisms, grouping into multicellular structures when harsh environmental conditions require it. What is more extraordinary is that they can disperse from this multicellular structure if food becomes plentiful again. The slime mould Dictyostelium discoideum has been studied a great detail to give insights into biological development, summarised in [51]. The widely studied nonlinear Belousov–Zhabotinskii reaction has been observed in Dictyostelium development and has provided a basis for the understanding of the aggregation process of individual amoebae. There is also a question of how separate cells synchronise their movement so as to carry out this aggregation. The secretion of the biochemical cyclic AMP (cAMP) into the external environment has been suggested as a trigger, but it cannot be very specific if many cells are secreting it at once. Research into this issue, reviewed in [51], suggests that modified receptors lead to variation in feedback to cAMP, leading to spirals in biochemical density in the local environment around an amoeba, causing convergence from many cells secreting at different levels to secretion at uniform levels, and resulting movement towards each other. This has been supported by modelling and by experimental studies. It has been argued that models of this form, although not depending upon the same biological basis, can explain other types of biological synchronisation. For applications of self-organisation in distributed computer systems, because this is an extremely well-understood system, it may be good in situations where some form of broad spread of messaging in local environments is allowed. But the increase in messaging costs as messages spread further afield, and the lack of gain from convergence to multicellularity or the equivalent in a distributed system, makes it less likely to be useful in a wider context. The animal kingdom also shows some amazing examples of pattern formation. Perhaps the best comes during the development of the fruitfly Drosophila melanogaster. This insect develops rapidly from an egg to an embryo and then an adult fly, and its embryonic development is significantly affected by mutations in relatively few genes, making it an ideal subject for experimental study. A number of mutant genes affect segment formation in the embryo, for example hunchback, knirps, Kruppel and giant (reviewed in [39], Chap. 13). The way that these genes operate has been studied in detail, and the biochemical interactions between developing segments are well understood. Many feedback pathways result in stable or unstable development of segments. What relevance is this to self-organised computing? This natural pattern formation through feedback inspired an algorithm for bandwidth allocation for adjacent cells in mobile telephone networks [60].

4 Self-organisation in Natural Systems Inspiring Self-organising Software

85

4.4.4 Cellular Systems Although many living organisms are unicellular, the most interesting spend their mature lifespan as complex systems of cells. Even then they still pass through a unicellular “egg phase” during the process of development or ontogeny. 4.4.4.1 Development: From One Cell to n Cells Developmental biology studies the processes that lead from one cell to a complete organism. Considering vertebrates and many other taxa, the first step is embryogenesis. That means broadly the differentiation of the egg cell into stream cells that will proliferate and following their genetic and local environmental pressures further differentiate to build a blueprint of the final organism. The next steps are growth, adaptation/learning about the environment, maturation, senescence and death. Except for the last event, cells communicate between themselves and with their environment and generally cooperate in order to keep the organism alive. That is homeostasis. An advantage arising from cells working collectively is particular cells can specialise. Fundamentally, cells are autonomous; they carry the DNA that potentially can produce a rather identical organism. Although true for stream cells, this assertion becomes as false as cells are differentiated. Working together, cells whose capabilities are limited can distribute their tasks and each specialise on the most appropriate task. Specialisation allows some cells to store energy (adipocytes, hepatocytes), others to transform energy into movement (myocytes), other to carry information (neurons), and other will produce molecular defences against external elements (lymphocytes). Humans are composed of more than 220 cell types that communicate. When in touch, cells can identify if they are of the same type or of the same lineage— they can link up or flee, aggregate or segregate, send diffusing messages to attract or repulse other cells, help them to proliferate, or trigger apoptosis (cell suicide). Based mainly on chemical diffusion, those mechanisms generate pattern formation [4, 61, 63]. Finally, differentiated cells are combined in functional tissues in organs and systems. During development, cells coordinate via chemical communication that enables induction: a tissue produces a signal that will trigger transformation of some cells in another tissue. This new tissue will in turn produce new information and so on; the organism is further developed through a combination of genetic information and developmental interactions. Self-organisation is still occurring in mature organisms and increases their adaptive potential. Beyond basic homeostasis, the nervous system and the immune system are the best examples of collective cellular adaptation. 4.4.4.2 Neural Systems Vertebrates possess different nervous systems that manipulate and propagate information. Some of them are rather simple and autonomous like that along the gut,

86

P. Marrow and J.-P. Mano

while others are very complex and structured such as the central nervous system. The central nervous system is composed of many tissues that protect, feed and regulate the neural tissue. Neurons alone are not able to deal with information, they interact with numerous cells, trivially, other neurons, but also myelinizing cells that stabilise them and permit a quicker conduction of nerve impulse, and astrocytes that help neurons in their metabolic activity, cleaning wastes and producing neurotrophines used for neuron growth and life. Collectively those cells constitute a system for treatment and transmission of information. Neurons use a large part of their energy to maintain an electric potential on their membranes, and under some conditions, variation of potential can diffuse and propagate along membranes [31]. Once the electrical signal passes a threshold, that signal can become an action potential, self-maintained and propagated without loss down a cell’s extremities and reaching synapses which connect with other neurons. Synapses are connections between a neuron and another cell (most of the time another neuron) that transmit and modulate the incoming electric signal though a chemical or electrical form to the downstream cell [11]. Neurons are cells. Despite their neural function, they interact locally via their membrane and using a lot of chemicals, with many other cells that can lead selforganising processes. During the brain’s early development, stream cells will differentiate into precursors of neurons and of glial cells, which in turn will differentiate again to produce different kinds of neurons and glia. Embryonic brain tissue appears very heterogeneous, but its construction follows extremely deterministic processes. Every neuron is produced in the same region and then migrates to its final location, following a path generated by glial cells. Once its body is in place, the neuron begins growth of axon and dendrites up to their targets. This growth is often controlled by signals sent by targets, and neurons that do not receive enough signals have a great chance to disappear. That is plasticity, a means to adapt the brain to the organism it is embedded in. Even after birth different kinds of plasticity perpetuate. Short- and long-term potentiation and depression are known to be bases of learning and memory; they refer a Hebbian regulation of synaptic information transfer by addition or suppression of receptors and/or neuromediators [45]. Research has shown that this plasticity has impacts beyond memory: environmental stimulation can have repercussions on the cortex [56], and either a perception or an organ is over-used like hearing in blind people or hands in professional violinists [53]. In spite of the first part of the nervous system being deployed before any nervous activity [25], a second part requires spontaneous activity [58] to engender functional sub-units (column, barrels), and a third part relies on the environment to develop [28]. The neural learning system is modified by the learning itself: the more the environment is informative, the more the stimulated brain will be able to deal with incoming information by producing new neurons, and conversely a brain will not develop correctly in an under-stimulating environment [40]. The adaptive capability of the nervous system is affected by stress and ageing but is still present in the adult; in the case of trauma or pathology, cortical remodelling may be induced [38]. And the nervous system’s robustness resides more in adaptive potentiality than in the redundancy of its neural networks.

4 Self-organisation in Natural Systems Inspiring Self-organising Software

87

From the perspective of self-organising software systems, the strongest inspiration that the robustness and learning ability of neural systems have given is in the area of (artificial) neural networks, much simpler computer programs than the biological neural networks they are named after, but still with useful properties in learning about their context and solving problems. Beale and Jackson [7] provide an approachable introduction.

4.4.4.3 The Immune System The vertebrate immune system arises from multiple interactions involving multiple entities; the most known protagonists and mechanisms are presented in Fig. 4.2. This system protects against external agents (which maybe pathogens) by various collective mechanisms. The most relevant components of the immune system are cells called lymphocytes, these are white blood cells, divided into B- and T-cells. Before their maturation, T-lymphocytes are produced randomly and are tested in the thymus: cells that do not interact with the immune proteic complex are killed, and those cells that should trigger an immune response inside the thymus, that means those that react against the “self”, are also killed. This is the negative selection of T-cells in the thymus [13]. B-cells assist the process of antigen recognition by secreting antibodies corresponding to an antigen (external agent). Antibodies can link to antigens by possessing a complementary shape. Antigens are significant because they stimulate the immune response, without antigens a response will not occur. There are two types of immune response. The primary response comes from natural or innate immunity, this response is provoked when an antigen is encountered for the first time in the organism. The B-cell activated by the antigen generates many antibodies oriented to the antigen shape. With this first activation the antigens can be more easily destroyed. When an immune response is engaged, stimulated lymphocytes proliferate and produce their free antibodies corresponding to the antigen. During this proliferation, some DNA mutations appear in the new B-cells. These somatic hyper-mutations improve the capacity to link to the appropriate antigen. As the affinity of an antibody to its matching antigen improves further, the corresponding B-cell is cloned. This process is the maturation of the immune response, an important part of the immune learning mechanism. As maturation progresses further, it is possible to create memory cells that persist in the body for a long time in order to have the potential to generate a secondary immune response. The secondary response, also called acquired or specific immunity, occurs after a similar re-infection and allows the system to produce more quickly and more massively specific antibodies by memorising the original shape of a previously encountered antigen. Based on these observations, the immune network theory tries to explain the organisation of the immune system. This theory is based on lymphocytes ability to enter in interactions, so constituting a network that can develop unless antigens stimulate, so as to produce antibodies. This theory is based on the assumption of

88

P. Marrow and J.-P. Mano

Fig. 4.2 A representation of the maturation of the vertebrate immune system with B-cell and T-cell paths. In both cases antigenic activation induces proliferation of cells. B-cells produce free antibodies that link to corresponding antigenic shapes. T-cells are killer cells that produce toxins against unrecognised cells either infective or tumoral

the possibility of a complete immune repertory that is be able to recognise all various antigens [15]. This hypothesis was completed by Jerne [36]: if the immune repertory is complete, antibodies of the same body would have to react with their complementary binding sites. In doing so, some antibodies may induce an immune response against other antibodies. These two hypotheses converge in the idiotypic network development theory that shows how the system can include a sufficient diversity to recognise never met before antigens. In other words, such a stimulation network potentially maps the whole universe of possible shapes. Further discussion about the basis of the vertebrate immune system continues among researchers, but there is no doubt that it has attracted much interest among computer scientists, interested in the way that it can stimulate self-organisation in software systems, where there is no need to mimic all the details of the natural system [12, 13]. As the immune system plays a defensive role in living organisms, it has also attracted interest in the context of computer security, where networked machines are increasingly exposed to unpredictable risks [12, 23]. This topic is taken up further in Chap. 10.

4 Self-organisation in Natural Systems Inspiring Self-organising Software

89

4.4.5 Vertebrate Behaviour The behaviour of animals has attracted much interest from one particular species of animal, ourselves, but may also give insights into how to design self-organising software, since animal behaviour often leads to self-organisation.

4.4.5.1 Living in Groups Krause and Ruxton [41] and Couzin and Krause [16] consider the many benefits to animals of group living. This may give us useful insights for developing selforganising software because natural selection has produced behaviour in different species that is both robust and adaptable, very positive features in potential selforganising software applications. An important aspect of animal group living is the enhanced ability to defend against predators. Krause and Ruxton [41] identify several different forms which this could take. • The many eyes theory suggests that synchronisation is useful because many prey individuals can watch for predators at once, thus being able to cover more ground and being more likely to take avoidance behaviour at the right time. • The dilution of risk model assumes that even in a large group of prey it is impossible for all to get eaten because of the sheer numbers. If a large group is present all at once, the chances are that the one(s) that get eaten will not be the single individual that would be exposed to a high risk of predation if on its own. • Finally the predator confusion model which relates to both the above ideas assumes that even when presented with an abundance of potential prey, it is difficult for predators to decide which prey to consume. There is considerable evidence from the shoaling behaviour of fish subject to predation that their motion is both designed to, and does in fact, confuse predatory organisms such as dolphins. Networked software applications are under continual attack from hackers: while computer security products are at a very sophisticated stage, learning from what is already exploited in groups of animals may improve the defensive properties of self-organising applications. In contrast to predator-avoidance behaviour is foraging behaviour. Many foragers are herbivores and base their foraging around diurnal stimuli of the availability of their forage and predator-avoidance behaviour as above. But for predators foraging includes search for prey. When prey has been identified, synchronisation of predator behaviour into groups may be particularly relevant, given that other predators act as scavengers, typically not killing prey directly, but seeking to obtain it from other predators. For example, Fanshawe and Fitz [22] showed that African wild dogs could defend their prey against scavenging hyenas when acting in groups rather than as individuals. As in the case of predator-avoidance behaviour, the number of different stimuli involved in generating synchronisation among many individuals that are foraging, makes it difficult to generalise from foraging to particular self-organising software

90

P. Marrow and J.-P. Mano

applications, but we can compare foraging to the intake of resources which almost any self-organising software system will need to do, therefore it is a useful comparison to make.

4.4.5.2 Mating Behaviour It is obvious that mating involves an element of self-organisation and coordination between individuals. Self-organisation in the context of mating behaviour is required so that particular sexes are able to mate at particular times (in different species, the constraints on times when one or other sex can mate vary from extremely strict to non-existent), and so that individuals carry out particular behaviours at appropriate times both prior to mating itself and in mating. An example of a species that requires a very restricted time to reproduce is the reef-building coral Acropora millepora. Since it is sessile, multicellular individuals must spawn almost simultaneously in order to ensure reproduction. It turns that this is mediated by receptors that sense moonlight [43]. Since moonlight varies periodically according to the orbit of the Earth around the Sun and the gravitational locking of the Moon’s orbit around the Earth, this provides a periodic source which can be detected all over the regions where this species of coral occurs. This could inspire a broadcast-based control of synchronisation in a networked computer system, but such an application would arguably be very wasteful of resources in message passing. The behaviours prior to mating may include displaying to others of the same sex in order to overcome a challenge of access to mates of the opposite sex (for simplification ignoring the possibility of organisms such as some fungi where in effect very large numbers of sexes are possible), or displaying to others of the opposite sex in order to demonstrate the potential as a possible mate. For more information, see Maynard Smith and Harper [47]. Labelling elements of a self-organising software system akin to different sexes of living organisms and allowing them to interact in a way akin to sexual reproduction could be one way of controlling the organisation of multiple elements within a self-organising software system. Biologically inspired computing techniques such as evolutionary computation, for example [34] acknowledge this in effect, by generating new generations of individuals through multiplication between individuals in an existing population—but there is a lot more to draw upon in natural systems. Lekking is another example of self-organisation in mating behaviour. Leks [59] are locations where male and female birds gather to display to each other and to breed. A high degree of organisation by both sexes is required in order to be present at the right location at the right time, and to display at the right time in order to compete effectively with others of the same sex. Males display, and as a result of this display, females choose mates, and mating takes place. The advantages of this synchronised mating, other than the general advantages of living in groups discussed above, are difficult to establish, since the species involved do not have any physiological or anatomical limitations in mating as separate pairs.

4 Self-organisation in Natural Systems Inspiring Self-organising Software

91

A suggestion is that they allow females to better assess the range of male quality available, in effect browsing the male population. A lek may also reduce the level of harassment of females by males seeking mates, because of the large number of other individuals present of both sexes. Synchronisation in lekking or other types of mating could definitely be carried forward into self-organising software if software components had labels which enforced synchronisation of this kind. A related example is from the work of Babalogu et al. [2], drawing inspiration from the synchronised signalling of male fireflies during their mating season. They use a mathematical model of this firefly signalling to drive synchronisation in an overlay network, which could be used to self-organise network resources in a changing environment [2].

4.4.6 Social Insect Behaviour The behaviour of social insects, as opposed to that of other animals, deserves a section all to itself. This is because of the extraordinary self-organising properties that social insects have evolved [9, 10]. Although insects are typically much smaller than mammals and other vertebrates, and consequently have simpler nervous systems and comparatively limited behaviour, social insects combine simple behaviour of individuals into large groups which can carry out complex and efficient activities. This arises through two properties: the evolution of specialisation of social insects into castes and the ability of individual insects to direct each other’s behaviour through stigmergy. • Castes define the functional specialisation of individuals. In ants, bees and termites, the queen is typically the sole reproductive female, resident in the nest except when migration is taking place. Males are rarely present, except during reproduction, when drones are the reproductive males. Workers are nonreproductive females, but they are by far the most interesting of the castes, because they carry out the behaviour which keeps the nest or colony functioning, and they can collaborate in large numbers to produce outcomes of behaviour far beyond what one could do individually (such as building a termite mound). • Stigmergy is a process by which worker insects can learn from other workers about what behaviour to carry out. This takes place indirectly, rather than by direct interaction between workers (although the latter occurs also), and may be mediated by the environment of the nest or based on items other workers lay down. The association of pheromones with such items stimulates behaviour in other workers. This may be construction or foraging behaviour, for example. This is a highly effective process because very large numbers of workers can be deployed, and if some directions are not successful in generating pheromones, there are always other workers to follow alternative directions. Alternatively, if a particularly good direction is found, many workers can coalesce from different directions to exploit a resource rapidly. See [9, 10] for more details.

92

P. Marrow and J.-P. Mano

Camazine et al. [10] give many examples of why these properties are so effective in the life and behaviour of social insects. Because of caste specialisation, social insects are significantly organised, and because of response to stigmergy, worker insects have the capability to self-organise under the right environmental stimulus, even though an individual worker has very limited capability to do so. It is therefore not surprising that this has attracted much attention as a model for applying self-organisation from nature to computer systems. Researchers have started from the perspective of many simple entities akin to social insect workers, each with limited intelligence, and developed algorithms that can solve complex problems. This field has become known as swarm intelligence because it builds on the behaviour of a swarm. References [9, 19] give a variety of examples of how this can be applied: to combinatorial optimisation, network routing, transport planning, robot control and others. Further developments of this such as particle swarm optimisation use particles flying over a fitness landscape to solve problems [54]. This only begins to consider the potential of natural social insect systems to inspired self-organisation in software. Later in this book, Chap. 6 focuses in more detail on stigmergy, and Chap. 13 develops further swarm-based applications.

4.5 Self-organisation in Human Behaviour Humans cannot be considered apart from the natural context in which they exist, and in recent years the analysis of human social interactions has inspired much activity in the study of self-organising systems. Apart from vertebrate neural systems, which provide the basis for the complexity of human behaviour and have stimulated the highly active field of modelling via artificial neural networks described above, and the vertebrate immune system, which provides humans and other vertebrates which a higher effective defence against infection, but has also attracted interest in computational applications, human social behaviour has also attracted a great deal of interest when considering the emergence of self-organisation. This is not at all surprising, since, wherever we look, people are part of incredibly complicated societies embodying a great deal of self-organisation, even when this is denied by the participants. Consider your own life—even though the relationships you are engaged in may seem trivial or obvious to you, consider how difficult they would be to analyse or generalise about for a third party. It is thus not surprising that the social networks that we humans form have become the subject of much sociological and scientific interest. While sociology has been focusing specifically on human social behaviour for many decades, it is only recently that these studies have linked up with mathematical and computational studies of networks which were originally carried out independently.

4 Self-organisation in Natural Systems Inspiring Self-organising Software

93

4.5.1 Social Networks It might be expected that natural systems would have properties similar to the random networks known in mathematical graph theory (which deals with network properties as random graphs), given the lack of external control or order in their origin. But in many situations this is not the case: self-organising systems produce networks that, although not completely regular, are not completely random. Ironically, Paul Erd˝os, a pioneer of the theory of random graphs, also stimulated an example of such a self-organising network that lies between order and randomness. Erd˝os was one of the most prolific mathematicians of all time, and as well as publishing an astonishing number of papers, collaborated with very many other researchers [32]. After his death, mathematicians started considering his network of collaboration through the concept of the Erd˝os number: the number of collaborative links through scientific publication it takes an academic researcher to get to Paul Erd˝os. He, of course, has an Erd˝os number of 0, not needing to collaborate with himself, and his direct collaborators have an Erd˝os number of 1. More indirect collaborators have higher numbers. However it turns out that not very large numbers of links are needed to include very large numbers of mathematicians, and researchers in other disciplines, in a very large network. Specifically, the clustering coefficient of the Erd˝os number collaboration graph, which measures the extent to which nodes adjacent to a particular third node in a network are adjacent to each other, turns out to be dramatically smaller than would be expected for a random graph [6]. In the real-world context, this means that mathematicians and other scientific researchers do not pick their collaborators at random. This is not surprising, but it confirms that some of the networks that result from human social interaction, although largely self-organising, are not random networks. In fact sociologists had been addressing related problems since the 1960s. Early experiments suggested that only six links were needed to connect everyone [66]. Further more rigorous analysis and experiments in many different contexts showed that not everybody in the world was connected through six links—we all know that some people have more or less friends or acquaintances—and that some links were much more important, or stronger, than others. However it does turn out that many of the networks that can describe human social interactions, and other phenomena, have a particular class of mathematical properties that is distinct from random graphs or fully ordered networks, and have become known as small-world networks because the network required to describe the world turns out in some ways to be smaller than expected. The study of small-world networks has become a valid research area [65]. As well as human social interactions and scientific collaborations, the type of network that exhibits this property includes power distribution grids [66] and the Internet [5, 6]. Why is this relevant? In considering how to develop self-organising software systems that will lead to the emergence of effective solutions from simpler components, we are often going to consider applications distributed across networks. Thus an understanding of the properties of those networks is important—and if we are considering the Internet or applications distributed across human social groups,

94

P. Marrow and J.-P. Mano

small-world properties may impact. Alternatively, self-organising systems may be deployed as applications for many networked users. Here the individual requirements and preferences of the users may need to be taken into account—as well as the degree to which they interact with other users of the same or similar systems.

4.5.2 Epidemics Ever since human beings have engaged in social behaviour, they, like other living organisms, have found their interaction exploited by pathogens, the cause of epidemics. Epidemics continue to be a problem, causing much death and suffering to humans and other organisms despite advances in medical technology. Fortunately, understanding of how to deal with them has been much advanced by the development of mathematical models to represent their impact on populations. dX = − λ + μ(a) X(a), (4.7) da dY = λX − ν + μ(a) Y (a), (4.8) da dZ = νY − μ(a)Z(a). (4.9) da The equations above provide a model of the effect of an epidemic on a human population (see [1], Chap. 4). In these equations a is age, X is the proportion of the population that are susceptible to infection (so X(a) is that proportion at a particular age), Y is the proportion that are infected, and Z is the proportion that are immune as a result of having recovered from infection. λ is the force of infection, the rate of movement from susceptible to infected, ν is the recovery rate, and μ is the death rate. This is simpler than many real-world biological epidemics ([1]; see also Chap. 10 of [52]) but provides a good starting point for exploring many different directions for particular disease scenarios. It is also of relevance because being able to describe an apparently disorganised process through such an organised mathematical system gives clues as to how an epidemic may self-organise in nature, and has inspired computer scientists interested in developing novel ways of information transmission across networks. Eugster et al. [21] compare the processes of biological epidemics with the requirements of information dissemination in distributed systems and argue that this can be an effective means of robust information dissemination in peer-to-peer or ad hoc networks. Thus this is another example where apparently disorganised features of natural systems show some self-organisation and have inspired self-organisation in software.

4.5.3 Trust and Reputation Interactions between people depend upon agreements that allow valuable items (such as goods, services, information) to be transferred. This is because without

4 Self-organisation in Natural Systems Inspiring Self-organising Software

95

such agreements there is a risk that the supplier of the items will not benefit adequately from the transfer, and the receiver of the items will be able to take the benefit without compensating the supplier adequately. Trust is a level of risk accepted by an individual in order to allow such a transaction to take place. The reason that many such transactions can take place is because networks of trust have built up between people and between organisations which represent collections of people. These have partly arisen through social or familial links, but also through business profiles, or laws, which show people, organisations or even countries as trustworthy entities to do business with. As well applying to people, it can be applied to software entities, and because of the unpredictability of interactions of self-organising systems over the Internet is extremely relevant to the successful construction of such systems. How should trust be measured? In designing a self-organising system, anticipating the circumstances for the encounter of a new software entity is difficult: such an entity has no history, and should be treated with caution, perhaps behind a firewall until full evaluation can be completed. Entities (and in human social interactions, people) who have a history of interaction have something that can be used to calculate the degree of trust for a particular type of interaction. This is their history of interactions where they have behaved in a positive or negative fashion in relation to what was expected by other participants in the interaction, also known as their reputation. If an entity has a good reputation, then they can be reasonably trusted in current and future interactions without threatening the performance of the selforganising system. If an entity has a moderate or poor reputation by comparison, the level of trust must be lower. Ramchurn et al. [55] provide an example of how trust can be developed in multi-agent systems by building on reputation. Trust and reputation are very important topics when self-organising software systems may be constructed to support applications that interact with financial systems, or that may be exposed to unpredictable attack from hackers and hostile software released across the Internet. Ways of quantifying these concepts and implementing them in self-organising systems are dealt with in more detail in Chap. 8.

4.5.4 Gossip The spread of information in human communication has inspired another important area of research into information spread in computer networks, which by its very nature is self-organising. This is through gossip. Gossip amongst people is generally used to refer to the unpredictable spread of information by informal discussion between contacts. In fact the properties of networks of gossip in social networks can be mimicked in computational networks. Jelasity et al. [35] describe an algorithm that uses a gossip-like protocol to produce aggregation in large dynamically variable networks. A gossip-based model is focused on nodes in a network being able to detect and connect to their neighbours through some underlying network, but there being no need for a node to have global knowledge of the underlying network. Once having established a link with a neighbour, a node can exchange information

96

P. Marrow and J.-P. Mano

with it, and if this process is repeated many times across many links throughout the network, and over many iterations, it enables the spread of information and can also facilitate other useful properties like the aggregation of similar nodes [35]. Gossiping, although apparently disorganised, can assist self-organisation in networked computing. Computational applications have related properties to computational applications of epidemic information dissemination (Jelasity et al. [35] compare with Eugster et al. [21]). Gossip-based information spread in software is studied in more detail in Chap. 7.

4.6 Applications A number of different reasons for focusing on self-organisation in natural systems as inspiration for self-organising software have been discussed above, and in the process a number of different applications have been identified. This section focuses on just two application areas, that between them have drawn upon a variety of inspiration from natural systems, and have where such inspiration has had considerable impact.

4.6.1 Markets Humans have been trading in markets as part of their social interactions for many centuries. The development of software has allowed some of the effort of participation in the market to go from the human trader to software acting on behalf of humans, although this is still a very active area of research with many questions still open about how best to deploy software in this area. One form of human activity in markets which has been operating for a number of centuries is the (variable price) auction, where multiple traders bid for an item with either rising or falling prices, and which trader gains the item depends on the activity of the other traders and the terms of the auction. The speed and range of such auctions where only human traders participate is limited by the alertness of the traders and by the communications they have access to. By bringing software traders into auctions the nature of auctions can be changed, and indeed the mechanisms of auctions and markets can themselves be developed [18]. While such auctions could be useful for improving trading in real goods online, there is also interest in using artificial markets, to control applications. Trading of artificial goods could be used to assign value to limited resources. Clearwater [14] first introduced this idea, in the context of resource allocation for distributed applications, and this collection [14] covers a range of applications. Lewis et al. [44] combine the idea of an artificial market mediated by software agents with evolutionary algorithms that can optimise the agent’s trading strategies, and show that this can produce load-balancing across a dynamic network environment.

4 Self-organisation in Natural Systems Inspiring Self-organising Software

97

However it is likely that the most effective trading agents are deployed in financial markets, and their capabilities will be carefully concealed by the financial institutions that use them, in order to preserve those institutions’ profits.

4.6.2 Networks Networks can be viewed from different perspectives, by, e.g., electrical engineers, computer scientists or users sitting in front of their computer at home ordering something over the Internet. But nevertheless they have similar properties (as [65] shows), and often properties of self-organising natural systems have led to applications in the network area. One area of application is network design. Network design can be viewed as an optimisation problem if the requirements of that network are going to remain relatively fixed, and this is where techniques inspired by natural systems that can be applied to complex optimisation problems, such as evolutionary algorithms, can be useful [29]. But the demands placed on networks are typically changing. Having designed a network, there is the problem of network management. Historically, network management has been based on a global and hierarchical top-down control viewpoint, but this is no longer appropriate with the rapid changes in computing and communications technology, and the Internet itself has demonstrated the relevance of this in being developed in a manner that allows the addition and modification of domains and nodes without disrupting its overall function. Although a physical network may be tightly constrained by costs of deployment and maintenance, there is the possibility of placing a software overlay network on top in order to mediate management activities [2, 35, 48]. An overlay network can dynamically change to aggregate similar nodes [35, 48], can synchronise behaviour between network nodes [2] or can mediate information transmission between nodes [35], all through management algorithms inspired by self-organisation in natural systems. Once network design and management are dealt with, there is the problem that networks exist in the real world, and network nodes and links are subject to continual attack by hackers and by malware, software specifically designed to cause problems. Some of these issues can be dealt with by software implemented at the node level, such as firewalls and anti-virus software, but this has to continually become more sophisticated. The vertebrate immune system, which provides analogous self-organising defensive functions to living organisms (see Sect. 4.4.4.3 above) has not surprisingly attracted a lot of attention in computer applications [12, 23] and will be considered further in Chap. 10. Computer security is one aspect of the application of networks to the real world; unfortunately, there is another that is even more demanding. Networked computer and communications are required for modern warfare to take place, and it does occur, even though many people would like it not to. Links and nodes in networks

98

P. Marrow and J.-P. Mano

deployed on battlefields do not just risk attack by hackers and malware, although that may be a problem, they risk more proximate problems of being attacked and destroyed, as do the military serving in the field. This is a human-generated selforganising system, but it is extremely unpredictable, and robust software solutions that can change in real-time are needed. Healing et al. [30] use a multi-agent system that can adapt to the changing requirements of a battlefield environment to manage a highly robust network.

4.7 Conclusion Natural systems whatever level they are observed at, from the formal mathematical, to the physical or chemical or biochemical, to more accessible living systems, from cells to small or large multicellular organisms, including human beings, all show a considerable element of complexity, a term difficult (and often subject to disagreement [26]) to define but suggesting some degree of dis-organisation. At the same time they can be observed as possessing some capability for self-organisation at the mathematical or physical level [3, 65], at the cellular level [4], at the evolutionary level [39], at the ecological level [8], or at the behavioural level [9, 10], and even in the context of human behaviour [5, 6, 65, 66]. Not only is it possible to observe self-organisation at different levels of observation in both living and non-living systems, it is also possible to represent this selforganisation through mathematical or software models which have often inspired software applications. Various applications are described throughout the text, or references are made to sources of further information about applications. Because of the wide range of areas covered in this chapter, the applications section focuses on only two areas which are often addressed in different ways drawing upon inspiration from self-organisation in nature: markets and networks. After reading this chapter you should have an indication of the range of sources of such inspiration from natural systems and of its impact on applications. Because of the wide range of topics covered, it is not possible to include everything in substantial detail, but the reference list gives indications for further reading, and the exercises that follow provide the opportunity to explore some of the issues raised. The chapters that follow focus individually on some of the areas of self-organising software systems in more detail.

4.8 Problem–Exercises 4.1 Provide examples of interaction media that enable self-organisation processes to occur in natural systems. 4.2 Examination of night satellite pictures of United States commonly reveals a regular distribution of illuminated towns, indicating a positive relation between town

4 Self-organisation in Natural Systems Inspiring Self-organising Software

99

size and inter-town distances. Based on these indications, which self-organising process can be suspected?

Key Points • Natural systems, although apparently disorganised, show many aspects of self-organisation; • Study of natural systems, from non-living to living, including human beings, has shown means of representing and modelling their self-organising properties; • Based on this inspiration from natural systems, it is possible to develop a variety of self-organising software systems; • Self-organising software systems can generate useful properties for applications, especially in unpredictable environments.

4.9 Further Reading The Vision of Autonomic Computing. A foundational introduction to Autonomic Computing. (J.O. Kephart and D.M. Chess, 2003, IEEE Computer, 36(1):41–50.) Advances in Applied Self-organising Systems. A collection of papers describing software applications using self-organisation and emergence. (M. Prokopenko (ed.) Advanced Information and Knowledge Processing series, 2008, Springer, London.) Self-organisation. An instructive review article on history and present status of self-organisation mainly in physical and biological systems. (H. Haken, Scholarpedia, 2008, 8(1), http://www.scholarpedia.org/article/Self-organization.) Special issue on Self-organisation in Distributed Systems Engineering. A special issue comprising papers on self-organising mechanisms and applications (F. Zambonelli and O.F. Rana (eds.), 2005, IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, 35(3).) The Science of Self-organisation and Adaptivity. A foundational review of the self-organisation and emergence concepts and models as they appear in various scientific areas. (F. Heylighen, 2001, Knowledge Management, Organisational Intelligence and Learning, and Complexity, in: The Encyclopedia of Life Support Systems, EOLSS, L.D. Kiel (ed.) 2001, Eolss Publishers: Oxford, pp. 253–280.) Swarm Intelligence: From Natural to Artificial Systems. A comprehensive book on self-organising algorithms drawn from natural systems. (E. Bonabeau, M. Dorigo and G. Theraulaz, 1999, Oxford University Press.) Self-organisation in Biological Systems. A detailed presentation of self-organisation mechanisms in biological systems. (S. Camazine, J.-L. Deneubourg, N.R. Franks, J. Sneyd, G. Theraulaz and E. Bonabeau 2001, Princeton University Press.)

100

P. Marrow and J.-P. Mano

The many Facets of Natural Computing. An up-to-date review article on recent advances in natural computing. (L. Kari and G. Rosenberg, 2008, Communications of the ACM, 51(10):72–83.) Self-managed Systems and Services. An overview of the uses of self-organisation in implemented systems and services. (J.P. Martin-Flatin, J. Sventek and K. Geihs, 2006, Guest Editorial of special issue, Communications of the ACM, 49(3):36– 39.) That special issue includes additional relevant papers as well. The Self-made Tapestry: Pattern Formation in Nature. A very large covering book on self-organised pattern formation with beautiful illustrations (Philip Ball, 1999, Oxford University Press). Tree of Knowledge. A didactic and easy to read book presenting authors vision on a self-organising world (Humberto R. Maturana and Francisco Varela, 1992, Shambhala, Rev Sub edition). Acknowledgements Paul Marrow’s research is supported by BT’s Strategic Research Programme and Digital Home Research Programme. He also acknowledges the support of the EU project CASCADAS (IST-FP6-027807) in his research on self-organising systems.

References 1. Anderson, R., May, R.: Infectious Diseases of Humans: Dynamics and Control. Oxford University Press, Oxford (1992) 2. Babaoglu, O., Binci, T., Jelasity, M., Montresor, A.: Firefly-inspired heartbeat synchronization in overlay networks. In: Di Marzo Serugendo, G., Martin-Flatin, J.P., Jelsity, M., Zambonelli, F. (eds.) Proc. 1st Intl. Conf. on Self-Adaptive and Self-Organizing Systems (SASO 2007), pp. 77–86. IEEE Computer Society, Los Alamitos (2007) 3. Baker, G., Gollub, J.: Chaotic Dynamics: An Introduction. Cambridge University Press, Cambridge (1992) 4. Ball, P.: The Self-Made Tapestry: Pattern Formation in Nature. Oxford University Press, New York (1999) 5. Barabási, A.: Linked: How Everything is Connected to Everything Else and What it Means for Business, Science and Everyday Life. Penguin Books, London (2003) 6. Barabási, A.L., Joeng, H., Ravaz, R., Néda, Z., Schuberts, A., Vicsek, T.: Evolution of the social network of scientific collaboration. Physica A 311, 590–614 (2002) 7. Beale, R., Jackson, R.: Neural Computing: An Introduction. Institute of Physics Publishing, London (1990) 8. Begon, M., Townsend, C., Harper, J.: Ecology: From Individuals to Ecosystems. Blackwell, Oxford (2006) 9. Bonabeau, E., Dorigo, M., Théraulaz, G.: Swarm Intelligence: From Natural to Artificial Systems. Oxford University Press, London (1999) 10. Camazine, S., Deneubourg, J.L., Franks, N.R., Sneyd, J., Théraulaz, G., Bonabeau, E.: SelfOrganization in Biological Systems. Princeton University Press, Princeton (2001). 2nd edn. (2003) 11. Cariani, P.: As if time really mattered: temporal strategies for neural coding of sensory information. Commun. Cognit. Artif. Intell. 12, 161–229 (1995) 12. Castro, L.D., Timmis, J.: Artificial Immune Systems: A New Computational Intelligence Approach. Springer, Berlin (2002) 13. Castro, L.D., Timmis, J.: Artificial immune systems as a novel soft computing paradigm. Soft Comput. 7, 526–544 (2003)

4 Self-organisation in Natural Systems Inspiring Self-organising Software

101

14. Clearwater, S. (ed.): Market-Based Control: A Paradigm for Distributed Resource Allocation. World Scientific, Singapore (1996) 15. Coutinho, A., Möller, G.: B cell mitogenic properties of thymus-independent antigens. Nat., New Biol. 245, 12–14 (1973) 16. Couzin, I., Krause, J.: Self-organization and collective behaviour in vertebrates. Adv. Study Behav. 32, 1–76 (2003) 17. Darwin, C.: The Origin of Species by Means of Natural Selection or the Preservation of Favoured Races in the Struggle for Life. Penguin Books, Harmondsworth (1982), reprint edn. 18. Dash, R., Jennings, N., Parkes, D.: Computational mechanism design: a call to arms. IEEE Intell. Syst. 18(6), 40–47 (2003) 19. Dorigo, M., Gambardella, L., Birattari, M., Martinoli, A., Poli, R., Stützle, T. (eds.): Proc. 5th Intl. Workshop on Ant Colony Optimization and Swarm Intelligence (ANTS 2006). Springer, Berlin (2006) 20. Endler, J.: Natural Selection in the Wild. Princeton University Press, Princeton (1986) 21. Eugster, P., Guerraoui, R., Kermarrec, A.M., Massoulié, L.: Epidemic information distribution in distributed systems. Computer 37, 60–67 (2004) 22. Fanshawe, J., Fitz, C.: Factors affecting the hunting success of an African wild dog pack. Anim. Behav. 45, 479–490 (1993) 23. Forrest, S., Hofmeyer, S., Somayaji, A.: Computer immunology. Commun. ACM 40(10), 88– 96 (1997) 24. Futuyma, D.: Evolution. Sinauer, Sunderland (2005) 25. Gazzaniga, M.: Organization of the human brain. Science 245, 947–952 (1989) 26. Gell-Man, M.: What is complexity? Complexity 1, 1–9 (1995) 27. Gleick, J.: Chaos: The Making of a New Science. Penguin Books, London (1988) 28. Greenough, W.: What’s special about development? Thoughts on the bases of experiencesensitive syaptic plasticity. In: Greenough, W., Juraska, J. (eds.) Developmental Neuropsychobiology, pp. 387–407. Academic Press, New York (1986) 29. He, L., Mort, N.: Hybrid genetic algorithms for telecommunications network back-up routeing. BT Technol. J. 18(4), 42–50 (2000) 30. Healing, A., Ghanea-Hercock, R., Duman, H., Jakob, M.: Nexus: self-organising agent-based peer-to-peer middleware for battlespace support. In: Pechoucek, M., Thompson, S., Voos, H. (eds.) Defense Industry Applications of Autonomous Agents and Multi-Agent Systems, pp. 1– 14. Birkhäuser, Basel (2008) 31. Hodgkin, A., Huxley, A.: A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. 117, 500–544 (1952) 32. Hoffman, P.: The Man Who Loved only Numbers: The Story of Paul Erd˝os and the Search for Mathematical Truth. Fourth Estate, London (1999) 33. Hoile, C., Wang, F., Bonsma, E., Marrow, P.: Core specification and experiments in diet: a decentralised ecosystem-inspired mobile agent system. In: Proc. 1st Intl. Conf. on Autonomous Agents and Multi-Agent Systems (AAMAS’02) (2002) 34. Holland, J.H.: Adaptation in Natural and Artificial Systems. MIT Press, Cambridge (1975). 2nd edn. (1992) 35. Jelasity, M., Montresor, A., Babaoglu, O.: Gossip-based aggregation in large dynamic networks. ACM Trans. Comput. Syst. 23(3), 219–252 (2005) 36. Jerne, N.: The somatic generation of immune recognition. Eur. J. Immunol. 1, 1–9 (1971) 37. Johnson, S.: Emergence: The Connected Lives of Ants, Brains, Cities and Software. Penguin Books, London (2002) 38. Kaas, J.: The reorganization of sensory and motor maps after injury in adult mammals. In: Gazzaniga, M. (ed.) The New Cognitive Neurosciences, pp. 223–236. MIT Press, Cambridge (2000) 39. Kaufman, S.: The Origin of Order: Self-Organization and Selection in Evolution. Oxford University Press, New York (1993) 40. Kempermann, G., Gast, D., Gage, F.: Neuroplasticity in old age: sustained fivefold induction of hippocampal neurogenesis by long-term environmental enrichment. Ann. Neurol. 52, 135– 143 (2002)

102

P. Marrow and J.-P. Mano

41. Krause, J., Ruxton, G.: Living in Groups. Oxford University Press, Oxford (2002) 42. Lawton, J.: Food webs. In: Cherrett, J. (ed.) Ecological Concepts: The Contribution of Ecology to an Understanding of the Natural World, pp. 43–73. Blackwell Scientific, Oxford (1989) 43. Levy, O., Appelbaum, L., Leggat, W., Gothlif, Y., Hayward, D., Miller, D., Hoegh-Guldberg, D.: Light-responsive cryptochromes from a simple multicellular animal, the coral Acropora millepora. Science 318, 467–470 (2007) 44. Lewis, P., Marrow, P., Yao, X.: Evolutionary market agents for resource allocation in decentralised systems. In: Rudolph, G.T. (ed.) Parallel Problem Solving from Nature—PPSN X. LNCS, vol. 5199, pp. 1071–1080. Springer, Berlin (2008) 45. Marrone, D., Petit, T.L.: The role of synaptic morphology in neural plasticity: structural interactions underlying synaptic power. Brains Res. Rev. 38, 291–308 (2002) 46. Marrow, P., Hoile, C., Wang, F., Bonsma, E.: Evolving preferences amongst emerging groups of agents. In: Alonso, E., Kudenko, D., Kazakov, D. (eds.) Adaptive Agents and Multi-Agent Systems: Adaptation and Multi-Agent Learning, pp. 159–173. Springer, Berlin (2003) 47. Maynard Smith, J., Harper, D.: Animal Signals. Oxford University Press, Oxford (2003) 48. Michiardi, P., Marrow, P., Tateson, R., Saffre, F.: Aggregation dynamics in service overlay networks. In: Di Marzo Serugendo, G., Martin-Flatin, J., Jelasity, M., Zambonelli, F. (eds.) Proc. 1st Intl. Conf. on Self-Adaptive and Self-Organizing Systems (SASO 2007), pp. 129– 140. IEEE Computer Society, Los Alamitos (2007) 49. Morowitz, H.: The Emergence of Everything: How The World Became Complex. Oxford University Press, Oxford (2002) 50. Murray, J.: Mathematical Biology, 2nd edn. Springer, Berlin (1993) 51. Nagano, S.: Modeling the model organism Dictyostelium discoideum. Dev. Growth Differ. 42, 541–550 (2000) 52. Nowak, M.: Evolutionary Dynamics: Exploring the Equations of Life. Belknap Press of Harvard University Press, Cambridge (2006) 53. Pantev, C., Oostenveld, R., Engelien, A., Ross, B., Roberts, L., Hoke, M.: Increased auditory cortical representation in musicians. Nature 392, 811–814 (1998) 54. Poli, R., Langdon, W., Marrow, P., Kennedy, J., Clerc, M., Bratton, D., Holden, N.: Communication, leadership, publicity and group formation in particle swarms. In: Dorigo, M., Gambardella, L., Birattari, M., Martinoli, A., Poli, R., Stützle, T. (eds.) Proc. 5th Intl. Workshop on Ant Colony Optimization and Swarm Intelligence (ANTS 2006), pp. 132–143. Springer, Berlin (2006) 55. Ramchurn, S., Jennings, N., Sierra, C., Godo, L.: Devising a trust model for multi-agent interactions using confidence and reputation. Appl. Artif. Intell. 18, 833–852 (2004) 56. Rauschecker, J., Korte, M.: Auditory compensation for early blindness in cat cerebral cortex. J. Neurosci. 13, 4538–4548 (1993) 57. Shalizi, C., Crutchfield, J.: Computational mechanics: pattern, prediction, structure and simplicity. J. Stat. Phys. 104, 816–879 (2001) 58. Skarda, C., Freeman, W.: How the brain makes chaos in order to make sense of the world. Behav. Brain Sci. 10, 161–195 (1987) 59. Sutherland, W.: From Individual Behaviour to Population Ecology. Oxford University Press, Oxford (1996) 60. Tateson, R.: Self-organising pattern formation: fruit flies and cell phones. In: Proc. 5th Intl. Conf. on Parallel Problem Solving From Nature (PPSN-V), pp. 732–742. Springer, Berlin (1998) 61. Thompson, W.: On Growth and From, 2nd edn. Cambridge University Press, Cambridge (1966) 62. Tudge, C.: The Variety of Life: A Survey and a Celebration of All the Creatures that Have ever Lived. Oxford University Press, Oxford (2000) 63. Turing, A.: On the chemical basis of morphogenesis. Philos. Trans. R. Soc. Lond. B, Biol. Sci. 237, 37–72 (1952) 64. Waring, R.: Ecosystems: fluxes of matter and energy. In: Cherrett, J. (ed.) Ecological Concepts: The Contribution of Ecology to an Understanding of the Natural World, pp. 17–42. Blackwell Scientific, Oxford (1989)

4 Self-organisation in Natural Systems Inspiring Self-organising Software

103

65. Watts, D.: Small Worlds: The Dynamics of Networks Between Order and Randomness. Princeton University Press, Princeton (1999) 66. Watts, D.: Six Degrees: The Science of a Connected Age. Vintage, London (2004) 67. Yao, X., Lui, Y., Li, J., He, J., Frayn, C.: Current developments and future directions of bioinspired computation and implications for ecoinformatics. Ecol. Inf. 1, 9–22 (2006)

Chapter 5

Agents and Multi-Agent Systems Marie-Pierre Gleizes, Valérie Camps, Anthony Karageorgos, and Giovanna Di Marzo Serugendo

Objectives This chapter details the main concepts of the multi-agent systems domain. After reading this chapter, the reader will: • • • •

Know what is an agent and what are its properties, Understand what is a MAS and what are its properties, Be able to explain what is the environment of a MAS, Understand how MASs represent a solution for artificial self-organising systems design.

5.1 Introduction Complexity of near future and even nowadays applications can be characterised as a combination of aspects such as the great number of components taking part in the applications, the fact that knowledge and control have to be distributed, the presence of nonlinear processes in the system, the fact that the system is more and more M.-P. Gleizes · V. Camps () IRIT, Université Paul Sabatier, Toulouse, France e-mail: [email protected] M.-P. Gleizes e-mail: [email protected] A. Karageorgos Technological Educational Institute of Larissa, Larissa, Greece e-mail: [email protected] G. Di Marzo Serugendo Birkbeck College, University of London, London, UK e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_5, © Springer-Verlag Berlin Heidelberg 2011

105

106

M.-P. Gleizes et al.

often open, its environment dynamic and the interactions unpredictable. In order to tackle the design of such complex systems, being able to engineer self-organising systems is a promising approach providing the required robustness. MASs are one of the most representatives among artificial systems dealing with complexity and distribution [20, 22]. The MAS paradigm appeared in the 1980s, and its specificity concerns collective behaviour. The first generation of works (1970–1980) focuses on distributed problem solving; in general, in these systems, knowledge and processing are distributed, but the control is centralised; furthermore, these systems are ad hoc ones with no reusability potential. The second generation (1980–1990) studies systems with decentralised control and generic systems to increase reusability. Interactions were the heart of most of the works on MASs during the third generation (1990–2000). The current generation is interested in the environment, the openness and the dynamics of these systems and the MAS technique is a way to design self-organising systems.

5.2 Agents This section defines the agent concept and its main properties.

5.2.1 Agent Definition An agent is a physical or software (virtual) entity [8, 24] which is: • • • • • • • •

autonomous, situated in an environment and able to act in/on it, having a partial representation of its environment, able to communicate with others agents, having an individual objective/satisfaction function, having resources, able to perceive its environment, having skills and offering services.

Its behaviour is the consequence of its perceptions, knowledge, beliefs, skills, intentions, interactions, . . . . An agent has the following life cycle: • perception: the agent perceives new information coming from its environment, • decision: the agent chooses the action(s) it has to do, • action: the agent acts, it performs the action(s) chosen during the previous step. Some examples of agents are: a user assistant in a system of information retrieval, an expert in a diagnosis system, a player in a soccer team, an ant in an anthill, . . . .

5 Agents and Multi-Agent Systems

107

5.2.2 Agent Properties 5.2.2.1 Agent Autonomy Autonomy is the most important property of an agent. This autonomy may qualify the existence of the agent with the property of viability: an agent is able to maintain some of its parameters in a given range. For example, a robot can maintain its energy at a given level in order to continue to “live”. Autonomy may be autonomy of life: an agent is able to live independently of the other agents’ existence. For example, in a collective of identical robots which have to sort boxes in a room, even if one robot is out of service, the others can continue to do their task. But for an artificial agent, the most important significance of autonomy is the autonomy of control of decision. An agent can say “NO”, which means that it can decide alone what its next action will be and to do or not an action. The agent is able to make its decision in an internal way according to its perceptions, knowledge and belief. Note that the agent must at least be able to perceive and to act to be qualified as autonomous. Depending on the application, an agent can be more or less autonomous, so the answer to the question “is this agent autonomous?” is not “yes” or “no” but “more” or “less”. For example, in the army a common soldier is less autonomous than a military officer, but he has some kind of autonomy. What is important is that without autonomy, an entity cannot be qualified as an agent. For example, a chair is not an autonomous agent because it is a passive entity without perception and action abilities. For an agent which can move inside its environment, its autonomy can be to decide alone if it has to move or not: the agent can say “GO”.

5.2.2.2 Reactive Agent A reactive agent [24] is an agent which is able to react to changes inside its environment and can maintain interactions with it. In general, it reacts in order to reach its own objective. The behaviour of this kind of agent is typically called a reflex behaviour. For example, an ant is the typical example of a reactive agent. Its behaviour is a set of conditions-actions rules such as when the conditions of a rule are satisfied by the environment perception and/or by its internal state, the actions are launched. An ant will collect food if it perceives food near itself inside its environment.

5.2.2.3 Proactive Agent A proactive agent is able to generate and reach goals. He is not only event-driven but also goal-driven, as he can initiate means to achieve its goals [24]. For example, an assistant agent in an E-commerce system, may decide to drop the goal “buy the object O” because the price is too high and may decide to buy another object.

108

M.-P. Gleizes et al.

5.2.2.4 Social Agent A social agent is an agent being aware of other agents and able to reason about them and to interact with them. In a MAS, all agents have to be social.

5.2.2.5 Communicating Agent A communicating agent is an agent which interacts with other agents by sending and receiving messages. A lot of works have been carried out to study languages such as KQML [12], ACL-FIPA [7] and protocols [14].

5.2.2.6 Situated Agent A situated agent is an agent which interacts with other agents through the environment and interacts with the environment by modifying it (e.g., by moving, by collecting resources, . . .).

5.2.2.7 Adaptive Agent An adaptive agent is an agent which can modify its behaviour during its life. Such an agent needs to have some memory in order to learn its future behaviour. Learning can be done with classical methods such as bucket brigade, reinforcement, genetic algorithms, . . . . In general, in MAS, the learning phase in an agent takes into account the other agents.

5.2.2.8 Mobile Agent The different agents which can be designed may possess some of these properties. These properties are not always exclusive; for example, a situated agent may also send messages and may also be a communicating agent. Therefore, the different kinds of agents to be designed are very large.

5.2.3 Agent Architectures Agents always have three main modules: • The perception module is in charge of taking into account events coming from the agent’s environment, which depends on the way the agent interacts with its environment. For communicating agents, it may be a mailbox to receive messages from other agents, or, for situated agents, it may be results of sensors detection.

5 Agents and Multi-Agent Systems

109

• The decision module is composed of knowledge and decision support tools for enabling the agent to execute its task or to reach its objective. For example, the decision module of a reactive agent is a program composed of condition–action rules. For a cognitive agent, this module can be an expert system with an inference engine and a knowledge base. • The action module manages all the activities to be done according to the results of the reasoning process. This is what can be externally observed as the behaviour of this agent. In systems where agents exchange messages, an action consists in messages sending. In systems where the agents have effectors, actions may be the results of actions done by an effector on its environment. Depending on the properties of an agent, these modules may differ. Three main classes of agent architectures exist: • Reactive architectures, an example of such an architecture is the eco-resolution architecture of reactive agents [10]. • Cognitive architectures, an example of this kind of architecture is the BeliefDesire-Intention architecture of intentional agents [17]. • Hybrid architectures, architectures which merge cognitive and reactive levels in the same agent, an example is the system. More details about these architectures can be found in [20]. It is not easy to find the frontier between reactive and cognitive. Some architectures are clearly reactive, some others are cognitive, but some are more or less reactive or cognitive.

5.2.4 Agents and Objects Objects are programming entities which have obvious similarities with agents but also main differences. Like objects, agents have an internal state and modular behavioural units (methods for objects, competencies or skills for agents). Both objects and agents may act to modify their state, and they communicate by message passing. One of the most important differences concerns the autonomy of control. In the agent case, it is the receiver of a request (messages) which decides to execute or not the received request. It is the receiver which has the power to decide, whereas in the object case, it is the sender which takes the decision: the object which receives a method call, executes it. Environment is a first-class concept in MAS and is less important for objects. In object programming, no guide is available to implement properties of reactivity, proactivity, sociability of the agent, as interaction complexity and system dynamics. This capability to help the designer for embedding agents with these properties is called flexibility by Wooldridge and Ciancarini [23]. The object-oriented methodologies—because of the differences between objects and agents—are not directly usable for MASs development. It is why new models, new methodologies and new tools adapted to the agent concept have been developed. The relations (like “is-a” or “is-part-of”) are insufficient to model the organisational relations of

110

M.-P. Gleizes et al.

a complex system. Agents have a very significant social part, which affects their behaviour. It is not the case with objects. Because of their sociability, agents communicate. In object-programming, communications consist only in calling a method. In MASs, interactions between agents are richer (with messages typology, ontology) and complex to implement (with protocols). Moreover, an agent analyses messages and decides of its behaviour. Finally, interactions between objects are defined in a more rigid way. They cannot evolve with time, what is essential in some multi-agent applications. MASs have shown their adequacy for designing (logically or physically) distributed, complex and robust applications. The agent concept is currently more than an efficient technology, it represents a new concept for software design where the agent is an autonomous software which pursues an objective, which is situated inside an environment and interacts with other agents with protocol languages. Often, an agent is considered as an “intelligent” object or as an abstraction level above objects and components. Agents and objects are paradigms to help software systems designers. Because the agent concept is at a higher level of abstraction than the object one, it facilitates the specification and the design of complex applications. Programming of these systems is generally done by using common object languages, but some agentprogramming languages are now well studied.

5.3 Multi-Agent Systems This section presents the concept of multi-agent system and its main properties.

5.3.1 Definition A Multi-Agent System (MAS) is defined as a set of interacting agents in a common environment in order to solve a common, coherent task. These agents try to achieve individual objectives which are sometimes conflicting. There are three main classes of MAS: • MAS in interaction with users. In general, in these systems, an agent is the user assistant and represents the user. The difficulties lie in having the right model of the user and in facilitating the interaction between the system and the user. • Simulation with MAS. Simulation aims at modelling natural phenomena in order to predict and to understand them. Some entities in a simulation are designed as agents, and they interact with one another and with the environment. The global task of the system is not precisely specified, the system has to model the real system and to behave as the real system. The most difficult task of a designer is to adjust the parameters of the agents in order to observe the desired global behaviour.

5 Agents and Multi-Agent Systems

111

• Collective solving problem. Agents in this kind of systems have to solve a common global task. For doing this, they usually have local knowledge and representations, partial skills and/or resources. They therefore need to cooperate in order to solve the global problem. Note that sometimes problem solving can be done by simulation. Therefore, for some applications, the two last classes have the same purpose. MASs are well adapted for applications with inherent distribution such as: • Spatial distribution. Agents must be situated at different locations in an environment. For example, in traffic control, if there is an agent per vehicle, these agents are situated at different locations. • Functional/logical distribution. Agents represent different skills and play different roles. For example, in an E-commerce application, some agents play the customers, and the others play providers. • Semantic distribution. Agents use different vocabularies and ontologies. For example, on the Internet, agents from different communities use different languages and have to interact. There are several representative applications of MASs such as: prey–predator systems, E-commerce, information retrieval, manufacturing control, soccer team (Robocup), crisis management systems, social animal simulations, . . . .

5.3.2 Multi-Agent Systems Properties 5.3.2.1 Autonomous System A MAS is autonomous means that there is no external entity which controls this system. This property is enforced because agents inside the system are autonomous.

5.3.2.2 Distribution, Decentralisation Inside a MAS, data (knowledge) are distributed inside all its agents. The main kinds of knowledge are: • knowledge related to skills (needed for the application), • knowledge related to world representation, belief that is: representation of the system environment, representation of the agents. Moreover, the control is decentralised, there is no supervisor.

5.3.2.3 Asynchronism, Parallelism Asynchronism and parallelism characterise the execution of the agents inside the system. Asynchronous means that agents do not wait an answer to their request,

112

M.-P. Gleizes et al.

they may go on working; otherwise they are synchronous. Parallel means that all agents can act simultaneously and not one after the other in a predetermined order; otherwise, it is sequential. Inside a MAS, the execution of the agents is generally asynchronous and parallel. At this point, note that it is important to highlight that there are two levels for specifying a MAS: • the conceptual level: the MAS model must be done by designers asynchronous and parallel, • the implementation level: the MAS can be program in a simulated asynchronous and parallel way or in real asynchronous and parallel programming.

5.3.2.4 Open/Closed System An open MAS is a MAS in which agents can be removed (killed or suicide committed) at runtime. Moreover, these new agents can be designed by different designers. Otherwise the system is qualified as closed. For example, in an E-commerce system composed of customers and providers, customers (represented by agents) may occur or disappear in the system while the system continues to satisfy the other customers and providers. So, this kind of system is open. An example of closed MAS is an anthill simulation where the ants are created at the beginning of the simulation and cannot be created during the simulation.

5.3.2.5 Heterogeneous/Homogeneous Heterogeneity and homogeneity of a system can be defined at different levels; in this chapter they will be defined at the agent architecture level. If in a system all agents have the same architecture, then the system is homogeneous. For example, an anthill, where the ants inherit of the same skills and perceptions means, is a homogeneous system. If the architecture of the agents inside the system differs, then this system is heterogeneous. For example, if in a same system a humanoid robot and an artificial ant have to cooperate, the system is qualified as heterogeneous. In a heterogeneous system, interoperability becomes one of the most important challenge.

5.4 Environment The environment clearly appears as an essential concept to manage self-organisation of MASs. “Without an environment, an agent is effectively useless. Cut off from the rest of its world, the agent can neither sense nor act” [15]. This citation is also true for a system; the system needs to interact with its environment. Because the environment disturbs the system, this one can react by self-organising.

5 Agents and Multi-Agent Systems

113

Fig. 5.1 Reciprocal influences between a system S and its environment ES

5.4.1 Environment Definition As Weyns et al. [21] underlined in their state-of-the-art and research challenges of environment of MAS, a consensual definition of the environment term in relation to MAS does not exist. Nevertheless, we will propose a positioning and give a general one. Let W be the world in which we are situated. W is both composed of the system to conceive S and the environment of the system ES: W = S ∪ ES. S is plunged into an environment ES with which it interacts and from which it cannot be dissociated. The environment ES is a common “space” for agents composing S. It can be defined as being all what is outside the system; that is, ES is the complementary of the system S in the considered world W , that is ES = W \ S. ES can be totally or partially perceived by the system S. S and ES are strongly coupled and have an influence on their respective behaviour: in that sense, we can speak about the activity of a system or an environment. More precisely, S locally perceives and acts in ES. Its behaviour modifies the state of ES which reacts by applying a pressure on S. For adapting to these constraints, the system makes a new action in the environment which in turn applies a new pressure which can be seen as a feedback (sometimes implicit), by the system in reference to its previous actions, etc. In fact, the environment in which a system is plunged, applies constraints to which the system must adapt. These interactions coupling leads to reciprocal influences that enable mutual fittings between the system and its environment. ES is composed of objects that can be situated (their position can be determined) and/or passive or active. In accordance with the previous definition, the environment S of an agent A belonging to the MAS is defined by all that is not A. More precisely, it corresponds to the environment of S as well as the other agents of the system S. An interaction coupling between an agent and its environment exists too (see Fig. 5.1). According to the application, an agent can perceive locally of entirely its environment, but it also has to be able to act in this environment.

5.4.1.1 Types of Environments One can distinguish: • The physical environment which consists of material resources (active and passive objects such as pheromone, food item, obstacle, . . . in the ants application) which are in the perception field or the effectors of an agent. In that case, the

114

M.-P. Gleizes et al.

physical environment can be an interaction medium: agents can interact in an indirect way, through the environment. For example, ants communicate by depositing pheromonal marks in the environment. The physical environment has more or less importance according to the application domain. Indeed, it is neglected in software agents (a message is supposed to reach its receiver) but is essential for situated agents such as, for example, robots moving in a discrete environment to achieve their objectives and ant colony foraging the environment to gather food. • The social environment which is composed of known agents. In a MAS, agents constituting the system, simultaneously evolve in a same environment. They maintain more or less strong relations with other agents, depending on whether they have or not the necessary skills to achieve their individual objectives. To efficiently interact, agents need to have representations of other known agents, that is of agents with which they already interacted. These representations can be more or less complex for an agent. They can concern its own skills, skills of known agents, available resources or intentions or commitments of others agents, . . . . The more these representations are right, the more interactions are relevant and enable agents to rapidly achieve their objectives. For example, if an agent needs particular information, it asks help to agents it knows and which, from its point of view, are likely to have the required information. This point of view depends on representations owned by the agent. Thanks to these representations, agents can interact with other agents to collaborate, cooperate or coordinate. Odell et al. [15] elaborate this previous variation of environments. According to them, “an environment provides the conditions under which an entity (agent or object) exists”. More precisely, they distinguish: • The physical environment which provides those principles and processes that govern and support a population of entities. • The communication environment which provides those principles, processes, and structures that enable an infrastructure for agents to convey information. • The social environment which is a communication environment in which agents interact in a coordinated manner. These notions of social and physical environments have also been added in the AGR model [1, 9] in order to represent not only the physical part of an interaction but also its social aspect. Agents interact only with the environment which will react according to agent’s influences and to the rules of change defined at both physical and social levels of interaction.

5.4.2 Environment Properties Russell and Norvig [18] had defined and associated different properties with environments. These properties have been adopted by most of researchers in the multiagent field:

5 Agents and Multi-Agent Systems

115

• Accessible vs. inaccessible: an environment is accessible to an agent if its sensors detect all aspects that are relevant to the choice of the action. For example, in a cheese game where the agents of the MAS are the pawns of a same color, the environment of a pawn agent is accessible because for playing, this agent may observe all the chessboard. In soccer, where the agents of the MAS are players of a same team, the environment of a player is inaccessible because a player has a limited perception of its environment. • Deterministic vs. non-deterministic: the environment is deterministic if the next state of the environment is completely determined by the current state and the actions selected by the agents. For example, in the chess MAS presented previously, the environment of a pawn agent is deterministic because when the pawn plays, it is alone to play, and it may forecast the next state of the game. In soccer, the environment of a player is non-deterministic because when the player acts, the other players can act simultaneously. • Static vs. dynamic: the environment is dynamic for an agent if the environment can change while an agent is deliberating. For example, in the chess MAS, the environment of a pawn agent is static because when the prawn deliberates, nobody can play. In soccer, the environment of a player is dynamic because when the player deliberates, the other players can act. • Discrete vs. continuous: the environment is discrete if there is a limited number of distinct, clearly defined percepts and actions. For example, in the cheese MAS, the environment of a pawn agent is discrete because there is a limited number of percepts and actions in a chess game. The real world is an example of continuous environment.

5.5 Multi-Agent Systems and Self-organisation On the one hand, MASs are one of the most representative among artificial systems dealing with complexity and distribution [20, 22] and enable to deal with systems where the global behaviour emerges from the local interactions of the agents. On the other hand, self-organisation is a process which can lead to these emergent phenomena. Self-organisation is defined as a set of dynamical interactions whereby structures appear at the global level of a system from interactions among its lowerlevel components. The rules specifying the interactions are executed on the basis of purely local information, without reference to the global pattern [2]. So, naturally, self-organisation has represented an inspiration source for MASs designer. The objective of most researchers in self-organising MASs is to embed selforganisation inside the MASs. This consists in finding relevant mechanisms to guide the agent behaviour at the micro-level, helping the agents to self-organise and to obtain at the macro-level, the behaviour of the system the designer expects. Therefore, designing such MAS requires to find rules to make the system achieve the required collective behaviour, that is “functions that are useful to the system’s stakeholders”

116

M.-P. Gleizes et al.

[16], “the required macroscopic behaviour” [5], “a functionally adequate function” [3], . . . . But the previous definition framework needs to be carefully instantiated with specific techniques enabling this self-organisation while allowing emergent functionalities to appear. The main question is: “How does this produce a complex system with the right behaviour at the global level?” The environment plays here its key role by constraining the system, and the system needs to be able to adapt to these constraints. There is an apparent antinomic situation in the idea of engineering applications with emergent functionalities. On the one hand, emergent behaviour is a behaviour which occurs and in a certain manner cannot be under control. On the other hand, a software designer wants the system he is building to achieve a desired function. Therefore, we can conclude by saying that we want to control the emergent behaviour of systems. The solution is then to better understand relations between micro- and macro-levels and to build a system able to self-organise and self-adapt to environmental dynamics. Currently, some self-organising MASs have been implemented taking inspiration from the following three main domains: the biologic and natural one [13], the social one [11], and the artificial one [6] and most of the time using techniques based on: stigmergy, cooperation, gossip, natural selection, attraction and repulsion, potential fields, social relationships, trust, . . . (most of them are detailed in this book). Selforganisation enables to design underspecified MAS because the system can selfadapt to new constraints not forecasted at the specification phase. The bottom-up approach of self-organisation simplifies the MAS design and reduces costs development. The next challenge concerns the validation of these systems. Validation is not yet obvious because self-organising systems lead to new challenges not yet taken into account by classical methods. In large-scale dynamic and adaptive systems such as self-organising systems, the methods, techniques and tools for validation are still in a research phase [19]. In general, formal methods [4] for certification, such as model checking, theorem proving, . . . , are adequate for checking/proving desired properties of the system when the code is showing the following properties: it is static, and it runs in well-known environments. A static code is a code which does not evolve, and there is no learning at this level. A well-known environment means that the system does not face unexpected events or unexpected scenarios. Engineered complex systems verification and validation can only be achieved using simulation-based approaches. Nowadays, the most reliable way consists in iteratively improving the designed system using mathematical tools (statistical analysis, behavioural parameters optimisation) or semi-autonomous adaptive programming (Living Design).

5.6 Conclusion In this chapter, we have introduced three main concepts for MASs understanding: agent, MAS and environment. A MAS is composed of a set of interacting agents, and it is plunged into an environment. The agent concept has a lot of meaning, the

5 Agents and Multi-Agent Systems

117

elicitation of its properties leads to clarify and specify it more. Autonomy, local perceptions and its ability to perceive and act are the main features of an agent. The properties always verified of the studied MASs in this book are their autonomy, their control decentralisation and their knowledge distribution. A MAS evolves in discrete, dynamic and indeterministic environments and sometimes inaccessible ones. This chapter ends by highlighting how self-organisation and MASs are joined in order to design more complex systems.

5.7 Problems–Exercises 5.1 For each case, justify whether the entity is an agent or not; if necessary, you can define more precisely the system and its environment: (a) (b) (c) (d) (e)

A door in a classroom, An automatic door in a big store, A thermometer in an hospital room, A bee in a simulated beehive, An engineer expert in motors of cars in a car design firm.

5.2 The air space traffic controller is the entity which centralises data in order to manage a given air space area. This controller is a bottleneck, and to overcome this problem, we wish to agentify all planes to enable them to manage the air space traffic. (a) Characterise this kind of agent with the properties given in Sect. 5.2.2. (b) Characterise the MAS with the properties given in Sect. 5.3.2. (c) Define the environment and characterise it with the properties given in Sect. 5.4.2. 5.3 Define and characterise the environment in the following cases: (a) For a MAS of manufacturing control system composed of the following agents: the machines, the pieces to be manufactured and the workmen (each workman works on a machine). (b) For a machine agent of the previous system described in (a). (c) For a MAS composed of the players of a handball team. (d) For a player of the handball team.

5.8 Further Reading Multi-Agent Systems, A Modern Approach to Distributed Artificial Intelligence. A book surveying multi-agent systems. (G. Weiss, MIT Press, 1999.)

118

M.-P. Gleizes et al.

Multi-Agent Systems: An Introduction to Distributed Artificial Intelligence. This book presents developments in the field of multi-agent systems. (J. Ferber, Addison Wesley Longman, 1999.) An Introduction to MultiAgent Systems. Introduction to agents and multi-agent concepts. (M. Wooldridge, John Wiley and Sons Limited: Chichester, 2002.)

References 1. Baez-Barranco, J., Stratulat, T., Ferber, J.: A unified model for physical and social environments. In: Environments for Multi-Agent Systems III. LNCS, vol. 4389, pp. 41–50 (2007) 2. Bonabeau, E., Dorigo, M., Théraulaz, G.: Swarm Intelligence: From Natural to Artificial Systems. Oxford University Press, London (1999) 3. Capera, D., Georgé, J.P., Gleizes, M.P., Glize, P.: The AMAS theory for complex problem solving based on self-organizing cooperative agents. In: 1st International TAPOCS Workshop at IEEE 12th WETICE, pp. 383–388. IEEE Press, New York (2003) 4. Clarke, E.M., Wing, J.M.: Formal methods: state of the art and future directions. ACM Comput. Surv. 28(4), 626–643 (1996) 5. De Wolf, T.: Analysing and engineering self-organising emergent applications. Ph.D. thesis, Department of Computer Science, K.U. Leuven, Leuven, Belgium (2007). http://www.cs. kuleuven.be/tomdw/phd/PhD-TomDeWolf-29-May-2007.pdf 6. Di Marzo Serugendo, G.: Self-organisation in MAS. In: Di Marzo Serugendo, G., Gleizes, M.P., Karageorgos, A. (eds.) Tutorial at the 4th International Central and Eastern European Conference on Multi-Agent Systems (CEEMAS’05), Budapest, Hungary, 15 September 2005 7. Document Title Fipa: FIPA communicative act library specification (2001). http://www.fipa. org/specs/fipa00037/ 8. Ferber, J.: Multi-Agent Systems: An Introduction to Distributed Artificial Intelligence. Addison-Wesley, Reading (1999) 9. Ferber, J., Gutknecht, O.: Alaadin: a meta-model for the analysis and design of organizations in multi-agent systems. In: ICMAS, pp. 128–135. IEEE Computer Society, Los Alamitos (1998) 10. Ferber, J., Jacopin, E.: The framework of eco problem solving. In: Decentralized Artificial Intelligence, vol. II (1991) 11. Hassas, S., Castelfranchi, C., Di Marzo Serugendo, G., Karageorgos, A.: Self-organising mechanisms from social and business/economics approaches. Informatica 30(1), 63–71 (2006) 12. Labrou, Y., Finin, T.: KQML as an Agent Communication Language. MIT Press, Cambridge (1994) 13. Mano, J., Bourjot, C., Lopardo, G., Glize, P.: Bio-inspired mechanisms for artificial selforganised systems. Informatica 30(1), 55–62 (2006) 14. Odell, J., Parunak, H., Bauer, B.: Representing agent interaction protocols in UML. In: OMG Document ad/99-12-01. Intellicorp Inc, pp. 121–140. Springer, Berlin (2000) 15. Odell, J., Parunak, H., Fleisher, M., Brueckner, S.: Modeling agents and their environment. In: AOSE 2002. LNAI, vol. 2585 (2003) 16. Parunak, H.V.D., Brueckner, S.: Engineering Swarming Systems, pp. 341–376. Kluwer Academic, Dordrecht (2004) 17. Rao, A.S., Georgeff, M.P.: BDI agents: from theory to practice. In: ICMAS’95, pp. 312–319 (1995) 18. Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach. Prentice Hall Series. Prentice Hall, New York (1995) 19. Slaby, J., Welch, L., Work, P.: Toward certification of adaptive distributed systems. In: Realtime and Embedded Systems Workshop (2006)

5 Agents and Multi-Agent Systems

119

20. Weiß, G.: Multiagent Systems, a Modern Approach to Distributed Artificial Systems. MIT Press, Cambridge (1999) 21. Weyns, D., Parunak, H.V.D., Michel, F., Holvoet, T., Ferber, J.: Environments for multiagent systems: state-of-the-art and research challenges. In: Environments for Multiagent Systems. LNAI, vol. 3477 (2005) 22. Wooldridge, M.: An Introduction to Multi-Agent Systems. Wiley, New York (2002) 23. Wooldridge, M., Ciancarini, P.: Agent-Oriented Software Engineering: the State of the Art. LNAI, vol. 1957 (2002) 24. Wooldridge, M., Jennings, N.: Intelligent agents: theory and practice. Knowl. Eng. Rev. 10(2), 115–152 (1995)

Part II

Self-organisation Mechanisms

Chapter 6

Stigmergy Christine Bourjot, Didier Desor, and Vincent Chevrier

Biological background on stigmergy and three computer models of stigmergy-based phenomena: stigmergy by pheromones, by silk and through activity.

Objectives After reading the chapter the reader will: • Understand stigmergy as a type of coordination mechanism; • Be acquainted with computer modelling of three natural phenomena involving stigmergy; • Become familiar with the design of stigmergy-based, self-organising software.

6.1 Introduction: Biological Background The first observations of stigmergy-based coordination in biological societies were made in 1956 by Chauvin [6], who studied termite societies and was amazed by the accuracy that termites re-builded a new vault around the queen after its experimental destruction: C. Bourjot · V. Chevrier () LORIA, Université Nancy, Nancy, France e-mail: [email protected] C. Bourjot e-mail: [email protected] D. Desor URAFPA, Université Nancy-UHP, Nancy, France e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_6, © Springer-Verlag Berlin Heidelberg 2011

123

124

C. Bourjot et al.

From time to time, when we have a look on the work in progress, we can see that it is very slow. We only can see irregular pillars, with sometimes the beginning of a vault, separately built by groups of workers on the two sides of the queen. But ten hours later, a very regular vault completely covers the queen. This vault is only a few millimetres thick, but nevertheless the two halves gather exactly above the queen: is it necessary to remind that the termite workers are completely blind? Nothing is known about the senses which are involved in the coordination of these tiny animals which are pasting like by chance their pellets of excrement (their only building material) on the pillars; but nevertheless, the two halves of the vault will gather with a millimetre scale accuracy.

This was also emphasised by Grassé [12], who in 1967 wrote: . . . one of the main, perhaps the greatest issue raised by social insects is how can the coordination of individual works lead to a coherent global work, logical, could we say, in accordance with the needs of the society.

This has since been known as the coordination paradox. It refers to reconciling individual level behaviour, which generally seems chaotic and without any higherlevel collective goal, with that of the societal level, which is characterised by the generation of highly organised structures indicating that either some collective plan is carried out, or that some superior entity is responsible for centrally organising the individual activities. Grassé raised a hypothesis, which formed the basis of the stigmergy theory [12]: . . . the intentional coordination of individual tasks is an illusion; all is only a strict automatism. The adaptation to circumstances holds first in the reactions of the workers to various stimuli, and these reactions to a given stimulus are always identical if the internal state of the subjects does not change.

The integration of individual behaviours into a coherent result is ensured by an indirect communication affected by influences from the environment: . . . the worker acts for its part, without any coordination with the behaviours of other workers, but in all cases its automatic reactions fit with what was previously built by the worker itself, or by its conspecifics.

Consequently, coordination of individual termite worker tasks is controlled by information previously embedded in the nest structure. In this respect, the environment can be considered as a form of distributed memory [12]: . . . stigmergy explains the logic of actions, which is not due to the actors, but is linked to previous works acting as significant stimuli that peremptorily control the reactions of the workers. So we understand the realisation piece by piece of the whole work, and the possibility, for workers, to pass from a given work to another, when they encounter, by chance, significant stimuli.

The word “stigmergy” derives from the Greek words “stigma” (cue, sign) and “ergon” (work), and carries the meaning that the work of individuals depends on work done previously, either by the individuals themselves or by other members of the same system. Wilson in [20] identified two kinds of stigmergy: • Sematectonic stigmergy, which denotes communication via modification of the physical environment, for example by carving out unofficial trails on the grass of

6 Stigmergy

125

public parks. The subject of communication is the current state of the solution to a problem, such as a partial path leading to a particular destination in a public park. By following and attempting to further improve the current solution, for example by attempting to follow and perhaps shorten the aforementioned partial path, individuals further modify the environment; for instance, they create additional foot trails on the grass. • Sign-based stigmergy, which denotes communication via some signaling mechanism, for example by using pheromones.1 It can be noted that sign-based stigmergy does not make any direct contribution to a given task. An example of stigmergic process is the formation of trails by foraging ants. In such a highly variable environment, involving increased inter- and intra-group competition, an efficient mechanism for coordinating food transportation from the food source to the nest is of critical importance. Coordination via stigmergic trail formation offers many advantages, such as rapid recruitment of foragers towards the richest food source when a choice is possible, and selection of the shortest route between food source and nest. Beckers et al. demonstrated the above concepts with experiments using a population of Lasius niger ants and a Y-maze leading to two sources of glucose, each with different concentration. The experiment results show that foragers choose the path to the source with higher glucose concentration (1 M) instead of the source with lower concentration (0.05 or 0.1 M) [1]. In a similar manner, Beckers et al. demonstrated that colonies of Lasius niger ants can select the shortest path to a food source [1]. This is achieved using a highly adaptive, collective decision-making mechanism, which is based on sensing and depositing volatile pheromones. That mechanism enables ants to select the most profitable food sources, minimising thus the food-to-nest carrying time. Bonabeau et al. [3] distinguish between two stigmergy classes: • Quantitative stigmergy generates stimuli that have no qualitative differences between each other, but they affect the action selection probabilities by means of positive feed-back. This is demonstrated in the construction of pillars around termite nests which is common in various termite species, such as the Macrotermes. Pillars are constructed by soil pellets, each impregnated by an amount of pheromone, which are carried and appropriately arranged together by worker termites. The total pheromone concentration at each location is proportional to the number of pellets found in that location, and it therefore tends to decrease by distancing from pillars that are completed or in progress. Termite workers carrying pellets initially move randomly, but soon they are attracted by and move towards locations with higher pheromone concentration, if such locations exist in their 1A

pheromone is a mixture of chemical substances that is released by an organism into its environment and causes specific behavioural or physiological reactions to other organisms of the same species that receive it [14]. The Greek roots of the term “pheromone” originally meant “to transfer excitement”.

126

C. Bourjot et al.

proximity, in a way that they eventually deposit their pellet to a location with locally maximum pheromone concentration. This approach results in competition between pillars-in-progress to attract pellet carrying termite workers [12]: . . . each pillar tends to grow, and competes with next sites to attract the carrying pellets workers. The only pillars which will survive are those which are at the right distance. So we understand that the regularity of the resultant structure spontaneously appears in the system, without being explicitly coded in individual termites, and that no measure of distance between the pillars is necessary.

• Qualitative stigmergy involves qualitatively different stimuli-responses which correspond to selection of different reactions. For example, in nest building of some social wasp species, such as the Polistes wasps, different types of stigmergy result in different construction decisions, such as constructing a new nest cell adjacent to an existing site. In conclusion, the existence of stigmergic processes indicates that simple rules, without any individual global knowledge, can lead to very efficient collective behaviours by achieving decentralised coordination. Positive feedbacks amplify slight differences and result in very accurate and adaptive collective behaviours. The feedback is mediated by cues left in the environment. More recently, it was demonstrated that stigmergy can be used for other purposes in addition to foraging in social insects, such as nest defence, swarming and exploration. In the same way, it was shown that the communication processes involved are not restricted to chemicals, but can use visual, or vibratory stimuli. Finally, as Camazine et al. state in [5], stigmergy is but one element of an overall set of mechanisms which a group of insects can use to build structures. In particular, stigmergy explains the information sources (the stimuli) that each builder uses to determine its subsequent actions.

6.2 Modelling and Implementing Stigmergy for Simulation Systems This section discusses design issues of stigmergy-based computational models using examples of processes found in natural systems. Initially, the core concepts and model design principles are presented, followed by descriptions of exemplar models of stigmergy-based natural processes. The examples cover foraging routing and brood sorting in ant colonies, and collective web weaving in social spider colonies.

6.2.1 Computational Modelling Principles Stigmergy is a way to achieve coordination of simple living entities through their environment. In particular, stigmergy can be viewed as a means to realise information flow in decentralised systems that involve information exchange between entities and a shared environment [5]. The examination of the definition of stigmergy emphasises the following features:

6 Stigmergy

127

• an environment which is modified by the activities of the living entities, • simple behaving individuals that react according to their own state and to the state of their surrounding environment, • a positive feedback loop in the dynamics of the system. To model our examples, we use reactive multiagent systems since they fit the above definition. An agent represents a living entity, and the agent environment is used to model the natural environment.2 Reactive agent behaviour is based on stimulus/response items, representing the reaction of the agent to a given situation in the environment. The agents have no (or limited) representation of their environment or of the other agents. Their behaviour is only based on internal states and on perception of environmental stimuli. Agent actions are selected stochastically aiming to capture both natural variability in agent behaviour and the level of positive feedback resulting from past behaviours and the current system state. In particular, to accurately model positive feedback the probabilities of agent action selection must reflect the selection reinforcement caused by stigmergy: the more a particular stimulus, the higher the respective response. The environment includes the key characteristics necessary for stigmergic processes, such as pheromones and cues for instance. Since stigmergy is a dynamical process, we need to experiment when developing stigmergic models and to find appropriate parameter values enabling the system to reproduce the essential natural dynamics.

6.2.2 Stigmergy by Pheromones: The Double Bridge Experiment 6.2.2.1 Overview In this section, we discuss stigmergy implemented by pheromones. This is the most popular type of stigmergy met in natural systems, such as in the foraging activity in ant colonies. Ant foraging has provided inspiration for numerous studies in computer science (such as Manta [10]), and its principles have been widely reused to design coordination mechanisms for artificial software systems (see for example [17–19]). Here we focus on the double bridge experiment [1], and we discuss how it can be modelled and simulated by software. Consider a nest (N ) (see Fig. 6.1) and a food source (S) connected by two paths of different lengths (we assume that path A is longer than B). Ants in the system can move towards either direction, N or S, but in both cases to reach their destination, they have to follow one of the two paths, A or B. During experimentation with real ant colonies, it can be observed that when the number of ants in the system is high, ants mainly follow the shortest path, whilst a single ant (or a relatively low number of ants) selects any of the two branches equiprobably. 2 In

the remaining, we will use environment to denote the model of the environment.

128

C. Bourjot et al.

Fig. 6.1 Double bridge experiment: ants have the choice between two paths (A and B) to go from the nest (N ) to the food source (S)

6.2.2.2 Implementation in a Computer Model In the following we present a simple model of the double bridge experiment based on the mathematical formulae proposed in [7]. In this model, an agent can be in different states representing its activities. The state durations are not equal (for example, the trip on the short path lasts less than the trip on the long one). This justifies an event-driven implementation: the simulation evolves from event to event (instead of cycles of fixed duration). An event represents a state change for an agent. Environment The environment includes two paths, A and B, each characterised by the time needed to cross it, which is denoted by duration(PathName). It also includes the respective pheromone densities of each path, denoted by densityAtNest(PathName) and densityAtFoodSource(PathName), and finally the two points of choice (the nest and the food source). Agent Behaviour Each agent corresponds to an ant and is characterised by two attributes. The first one (denoted by state) represents the behavioural state of the agent, and its possible values are listed below. The second one (denoted by P ) corresponds to the pathname being used, and it is valued only when state = GoToNest or state = GoToFoodSource. These definitions allow us to represent agent behaviour at any given time point (the agent state): • being in the nest (state = Nest); • going to the nest (state = GoToNest) and being in a path P = A or P = B; • going to the food source (state = GoToFoodSource) and being in a path P = A or P = B; • being at the food source (state = FoodSource). An agent remains in a state for a certain duration: a constant duration for Nest and FoodSource and a duration that depends on the length of the path when being in GoToNest or GoToFoodSource. State changes are triggered by the timeout corresponding to the respective duration. A state change involves an update of the agent state, and scheduling of a new event. Depending on the current state of the agent, the possible agent state updates are:

6 Stigmergy

129

• Nest: agent chooses a path PathName (A or B) to follow according to probabilities Prob(A) =

(20 + densityAtNest(A))2 (20 + densityAtNest(A))2 + (20 + densityAtNest(B))2

and Prob(B) = 1 − Prob(A). The updates correspond to what happens at the transition between the current and the next state: state ← GoToFoodSource, P ← PathName, densityAtNest(P ) ← densityAtNest(P ) + δ, where δ corresponds to the amount of pheromone dropped on the path. A new event is posted at time = currentTime + duration(P ). • GoToFoodSource: there is no choice. Updates are: state ← FoodSource, densityAtFoodSource(P ) ← densityAtFoodSource(P ) + δ, P ← Null. A new event is posted at time = currentTime + TimeInFoodSource. • FoodSource: the agent chooses a path PathName (A or B) to follow according to probabilities Prob(A) =

(20 + densityAtFoodSource(A))2 (20 + densityAtFoodSource(A))2 + (20 + densityAtFoodSource(B))2

and Prob(B) = 1 − Prob(A). Updates are: state ← GoToNest, P ← PathName, densityAtFoodSource(P ) ← densityAtFoodSource(P ) + δ. A new event is posted at time = currentTime + duration(P ). • GoToNest: There is no choice. Updates are: state ← Nest, densityAtNest(P ) ← densityAtNest(P ) + δ, P ← Null. A new event is posted at time = currentTime + TimeInNest. Stigmergy at Work Initially, it is assumed that agents are located at the nest and that there is no pheromone on the paths. As the first agents reach the first selection point, they arbitrarily select one of the two paths. Thus the population of agents on each path is approximatively equal. The agents that chose the shortest path reach the food source first and can subsequently return. They can then choose between two paths, the shortest one being overlaid with a higher density of pheromones. The shortest path is therefore likely to be chosen more often (and thus additionally reinforced with pheromones). For subsequent choices, the attractivity of the shortest path still grows, and hence the shortest path is reinforced with pheromones even further. In this simple model, the pheromone density is infinitely growing. To avoid this unrealistic feature, some pheromone reduction mechanism needs to be added to the model, for example a linear pheromone evaporation factor.

130

C. Bourjot et al.

6.2.3 Stigmergy Through Activity: Brood Sorting 6.2.3.1 Overview In this section, we describe a model of brood sorting in a species of ants (Leptothorax). The same modelling principles can also be used to model other natural system processes, such as wood chip aggregation, ant cemetery construction and termite pillar formation. Results of research work concerning such phenomena were initially published in [8, 11], and the fundamental principles of such models can be found in [2]. In ant species, brood sorting refers to the process in which ants sort different kinds of brood, such as eggs, larvae, cocoon and nymphea, and classify them into separate groups according to their type. A simple model of brood sorting involving stigmergic communication can be based on the following principles: 1. ants wander around their environment performing a movement covering a specific small distance at each time step; 2. if an ant (not carrying a brood element) encounters a brood element after a movement, then it picks it up with a probability positively linked to the isolation of that element; 3. after a movement an ant carrying a brood element drops it with a probability positively linked to the density of elements of the same type found in the neighbourhood of the ant location. In this model stigmergy is realised by considering the existence of brood elements in the neighbourhood of each ant. 6.2.3.2 Implementation in a Computer Model We consider a simplified version of the brood sorting phenomenon in which only two brood types exist in the environment, termed ‘black’ and ‘white’ respectively. Furthermore, we focus on modelling the actual sorting process, and for simplicity, we do not take ant movements into account. Environment The environment is assumed to be a rectangular grid of cells that can contain at most one brood element and one ant. Agent Behaviour Agents correspond to ants and are considered to be continuously moving in the environment. Furthermore, agents are assumed to possess a short-term memory (denoted by STM) which records the number of brood elements that each agent has encountered during the N last movements (N takes a constant integer value), and an internal state indicating whether agents are carrying a brood element or not. In the former case, the internal agent state also represents the type of brood element being carried by the agent. The short-term memory is managed in a FIFO manner, and it is used to compute contextual probabilities of brood collection or deposition. More formally, an agent possesses:

6 Stigmergy

131

Fig. 6.2 Example of agent Short-Term Memory contents: during the last 10 moves, the agent has encountered two white and three black brood elements

• An internal state Carrying which indicates whether the agent is carrying a brood element or not (values can be black, white or null). • A FIFO queue STM representing the brood elements encountered during the N last agent movements. The FIFO queue can be used to calculate the occurrence frequencies ftype of the two types of brood elements. For example, if N = 10, and STM has the form depicted in Fig. 6.2, then the occurrence frequencies of white and black brood elements are: fwhite = 20% (2/10) and fblack = 30% (3/10). Agent behaviour is then defined by the following action rules: 1. Take: if not Carrying (agent is empty-handed) and if the current agent location contains a brood element of type type, then the probability of the agent collecting K1 that brood element is ( (K1 +f )2 with K1 being a constant. type ) 2. Drop: if the agent is carrying a brood element of type type and the current location does not contain any brood elements, then the probability of the agent ftype )2 with K2 being a constant. dropping the brood element it carries is ( (K2 +f type ) 3. Wander: otherwise do nothing. In [2], it is suggested to use 0.1 as the value of the two constants (K1 = K2 = 0.1). Stigmergy at Work Model execution involves initially placing brood elements at random locations in the rectangular grid. Subsequently, agents start moving randomly, and the associated system dynamics resulting from agent states and the respective agent behaviours come into force. This simple model includes only two basic dynamic processes concerning brood element collection and deposition respectively. Both processes highly depend on the environmental conditions, and primarily on the brood element density found at the various locations around the environment. When wandering agents encounter a brood element at some location, they decide to collect it with some probability. This action will slightly reduce brood element concentration in the area surrounding that location, which will in turn result to subsequent agents collecting the remaining brood elements from that area with higher probability (see agent behavioural rules above). In a similar manner, agents carrying brood elements will deposit them at locations with sufficient brood element density, which according to the agent behavioural rules will reinforce the probability of other agents subsequently depositing additional brood elements in the surrounding area. In this manner, isolated brood elements quickly turn into brood element piles, and furthermore small piles tend to be merged into larger ones.

132

C. Bourjot et al.

Fig. 6.3 A real web of Anelosimus eximius

6.2.4 Stigmergy by Silk: Collective Weaving 6.2.4.1 Overview Anelosimus eximius are small social spiders (5 mm). Amongst different collective behaviours, they are able to build silky structures larger than 10 m3 [13]. Despite their apparent individual simplicity, these spiders exhibit an interesting collective behaviour during web weaving. While moving on vegetation or on a web, these spiders always fix silk at the top of the vegetable environment. Biologists have observed that such webs always exhibit particular architectural properties regardless of the type of the surrounding biological environment. Webs are not shaped geometrically, but instead their shape is twofold, comprising an horizontal hammock and an aerial network of silk lines. An example of a real spider society and the web it has constructed is presented in Fig. 6.3. Research efforts on web construction in social spider colonies concluded that the construction process is highly dependent on stigmergic coordination [15, 16], as well as it is the case in numerous collective social insect processes. The fundamental element of this stigmergic mechanism is silk, to which spiders are highly attracted. This attraction affects spider movement, and hence decisions taken during web construction. In particular, silk “draglines”, fixed between the “stakes” of the vegetable environment, are used by spiders to calculate directions to which they move during web construction, something which influences the web construction result. For example, when spiders have to decide between following a silk “dragline” and simply moving to a nearby “stake”, they select the direction towards which they sense a higher amount of silk. Subsequently, they move along the selected direction and construct new web parts using additional silk quantities. As a result, the route they follow is further reinforced with silk, and therefore other spiders are more likely to be attracted and perform web construction along directions pointing to that route which points to higher silk concentration. This mechanism sufficiently describes the construction of finite webs and aerial silk networks in spider colonies without

6 Stigmergy

133

Fig. 6.4 A simulated spider web (compare it with the real one depicted in Fig. 6.3)

assuming any explicit coordination between spiders. The main idea is summarised in that previous web construction actions leave amounts of traces (inscriptions) in the environment, and these traces proportionally influence selection of particular actions over others in subsequent construction decisions.

6.2.4.2 Implementation in a Computer Model Environment The environment is modelled as a square grid where each cell includes a stake that can take different height values aiming to model diversity in a vegetation environment. In this simple model, stakes can have a height value of low, medium or high. At their upper edge, stakes can be linked with each other with lines representing silk draglines (see Fig. 6.4). Agent Behaviour Agents correspond to spiders, and they are placed on stakes (cells) in the grid environment. In this simple model agents are not associated with an internal state. Furthermore, agents can move around stakes, and their behaviour consists of only two independent actions: constructing a silk dragline linking two stakes and moving between stakes. Agent actions can be executed concurrently when required; for example, at each time cycle an agent can either simply move to another stake, or it can move and at the same time construct a silk dragline to the destination stake. Agent actions are described as follows: • Dragline construction: agents construct draglines by attaching silk on stakes they visit while moving around the environment. The decision to attach silk on a stake is taken randomly. Each time an agent attaches silk to a stake, a silk dragline is constructed between the current stake and the last stake to which the agent had attached silk earlier. A dragline is a direct line linking two stakes. • Agent movement: agents can move to a stake either adjacent to their current stake or to one connected to their current stake with a silk dragline. The choice of the new stake is influenced by the current silk attraction tendency, and it reflects the propensity of agents to follow silk draglines. Each path to an adjacent stake is weighted by a value of 1, and each path that follows one or more silk draglines

134

C. Bourjot et al.

is weighted according to the number of draglines and to the attraction of the silk. This provides a probability distribution among the possible paths that can be issued from the current stake. To summarise: – nbDL(Astake) is the total number of draglines fixed on the stake AStake; – nbDLTo(Astake, stakeB) is the number of draglines fixed between the two stakes AStake and the StakeB; – FSilk is the attraction factor of a single silk dragline; – nbAdj is the number of adjacent stakes. If we assume that an agent is located at the stake Astake, then the probability of moving to an adjacent stake AdjacentStake is P (AdjacentStake) =

1 . nbDL(Astake) · FSilk + nbAdj

The probability of moving to a stake StakeB, which is linked to the current stake with silk draglines is P (StakeB) =

nbDLTo(AStake, StakeB) · Fsilk . nbDL(Astake) · Fsilk + nbAdj

At each cycle step, agents calculate the probability distribution of their movement to stakes accessible from their current stake. The destination stake is then chosen according to the calculated distribution, and subsequently agent movement takes place. When a stake is both adjacent and linked to the current stake with draglines, the probability of moving to that stake equals to the sum of the probabilities of both types of move (namely it equals to P (AdjacentStake) + P (StakeB)).

6.2.4.3 Experiments and Results A model of collective behaviour based on collective weaving can be executed after specifying three parameters: silk attraction Fsilk, number of spiders N and number of cycles T . In various studies, experiments were conducted to check biological assumptions relative to the stigmergic mechanism. For example, to assess stigmergy, the authors conducted several experiments using N = 5 spiders and for different values of silk attraction (ranging from 0 to some upper limit). Results were mainly qualitative: they were related to the behavioural process that led to the weaving. Their assessment was mainly based on qualitative properties: finite aspect of the web, visual assessment. Furthermore, appropriate metrics were used to estimate a number of these properties, such as density, web surface and mean dragline height. The first step was to obtain visual results in 3D. Simulation results depicted in Fig. 6.4 show such visual resemblance which was assessed as satisfactory by biologists. Silk attraction needs to be well “chosen” in order to induce the collective building of the web; otherwise if attraction is too strong, several separate webs are built as it is

6 Stigmergy

135

Fig. 6.5 Web obtained with 5 spiders and (a) null attraction for silk, (b) a strong attraction for silk and (c) a very strong attraction for silk

shown in Fig. 6.5c. We can observe that the web surface is related to silk attraction: when it is null like in Fig. 6.5a, the environment is largely occupied by the web (spiders are not trapped in the web). If it is well chosen like in Fig. 6.5b, there is a unique collective web. Finally, experimental results have revealed that there is a relationship between the silk attraction and web size and that varying spider numbers do not have any qualitative effects on the overall stigmergic process. An example of such experimental results together with relevant discussion can be found in [4].

6.3 Conclusion A large number of natural phenomena are based on stigmergic mechanisms. Such phenomena have been successfully reproduced in artificial systems while aiming to validate particular hypotheses or to study parameter influence as was done in the case of spider web weaving. The most interesting aspect of stigmergy is that coordination can arise without any direct communication. Furthermore, stigmergy can be used as the mechanism to realise information flow in decentralised systems involving information gathering from a shared environment [5]. Adaptive stigmergic mechanisms are good candidates for providing self-organisation in problem solving systems [3]. A classical example is the Ant Colony Optimisation (ACO) metaheuristic [9]. Finally, a more specific example of using the stigmergy by silk model to design a self organising application for region detection in grey level images will be provided in Chap. 17.

136

C. Bourjot et al.

6.4 Problems–Exercises 6.1 What kind or what class of stigmergy is involved in broodsorting? In the double bridge experiment? In web building? 6.2 Find among Models Library in NetLogo some examples of stigmergic processes. 6.3 The double bridge experiment (a) Implement the model of the double bridge experiment (or reuse the code provided in the companion CD). As initial value for parameters, you can use 100 and 50 as lengths of the paths, δ a random value between 1 and 10, time in nest and food source is constant (for example 2) since time in path is linearly proportional (factor 3) to the length. (b) Study the influence of parameters: for example, by varying the relative lengths of the paths. Is it possible to predict their influence without computer simulation? 6.4 Brood sorting (a) Implement the model of the brood sorting. (b) Study the influence of parameters: for example, by varying the size of the short term memory. Is it possible to predict their influence without computer simulation? Key Points • Stigmergy, a biological concept, is a coordination mechanism without involving any direct communication; • In stigmergy the indirect communication is mediated by modifications of the environment, which in turn influence the actions of individuals; • Reactive Multi-Agent Systems are particularly suitable for modelling and implementing stigmergy.

6.5 Further Reading Netlogo. An easy to use tool to model and simulate collective phenomena from a multi-agent perspective (http://ccl.northwestern.edu/netlogo/). Swarm Intelligence: From Natural to Artificial Systems. A comprehensive book on self-organising algorithms drawn from natural systems. (E. Bonabeau, M. Dorigo and G. Theraulaz, 1999, Oxford University Press.)

6 Stigmergy

137

Self-organization in Biological Systems. A detailed presentation of self-organisation mechanisms in biological systems. (S. Camazine, J.-L. Deneubourg, N.R. Franks, J. Sneyd, G. Theraulaz and E. Bonabeau, 2001, Princeton University Press.) Artificial Life, Special issue on stigmergy. A special issue on stigmergy with examples and application of stigmergic processes. Artificial Life Volume 5, Issue 2, Spring 1999, MIT Press. Acknowledgement Christine Bourjot and Vincent Chevrier wish to thank Bertrand Krafft and Alexandre Bernard for their fruitful collaboration about the modeling and simulation of web building.

References 1. Beckers, R., Deneubourg, J., Goss, S.: Trails and U-turns in the selection of a path by the ant Lasius Niger. J. Theor. Biol. 159, 397–415 (1992) 2. Bonabeau, E., Théraulaz, G.: Auto-organisation et Comportements Collectifs: La Modélisation des Sociétés D’insectes. Hermès, Paris (1997) 3. Bonabeau, E., Dorigo, M., Théraulaz, G.: Swarm Intelligence: From Natural to Artificial Systems. Oxford University Press, London (1999) 4. Bourjot, C., Chevrier, V.: Multi-agent simulation in biology: application to social spider case. In: Kermarrec, A.M., Bougé, L., Priol, T. (eds.) Proceedings of Agent Based Simulation Workshop II, Passau, Germany. Lecture Notes in Computer Science (2001) 5. Camazine, S., Deneubourg, J.L., Franks, N.R., Sneyd, J., Théraulaz, G., Bonabeau, E.: SelfOrganization in Biological Systems. Princeton University Press, Princeton (2001). 2nd edn. (2003) 6. Chauvin, R.: Vie et Mœurs des Insectes. Payot, Paris (1956) 7. Deneubourg, J., Aron, S., Goss, S., Pasteels, J.: The self-organizing explanatory pattern of the Argentine ant. J. Insect Behav. 3, 159–168 (1990) 8. Deneubourg, J.L., Goss, S., Franks, N., Sendova-Franks, A., Detrain, C., Chretien, L.: The dynamics of collective sorting robot-like ants and ant-like robots—simulation of animal behaviour. In: Meyer, J., Wilson, S. (eds.) First International Conference on Simulation of Adaptive Behaviour, pp. 356–363. MIT Press, Cambridge (1991) 9. Dorigo, M., Di Caro, G.: The Ant Colony Optimization Meta-Heuristic. McGraw-Hill, New York (1999) 10. Drogoul, A., Corbara, B., Lalande, S.: Manta: new experimental results on the emergence of (artificial) ant societies. In: Artificial Societies: The Computer Simulation of Social Life, pp. 190–211 (1995) 11. Franks, N.R., Sendova-Franks, A.B.: Brood sorting by ants: distributing the workload over the work-surface. Behav. Ecol. Sociobiol. 30, 109–123 (1992) 12. Grassé, P.: Nouvelles expériences sur le termite de muller (macrotermes mulleri) et considérations sur la théorie de la stigmergie. Insectes Soc. 14(1), 73–102 (1967) 13. Krafft, B.: Eco-ethology and evolution of social spiders. University of Paris XIII (1982) 14. McFarland, D.: The Oxford Companion to Animal Behaviour. Oxford University Press, London (1987) 15. Saffre, F., Krafft, B., Deneubourg, J.: What Are the Mechanisms Involved in the Emergence of Cooperativity? The “Spider” Model. Hermès, Paris (1997) 16. Saffre, F., Furey, R., Krafft, B., Deneubourg, J.: Collective decision-making in social spiders. J. Theor. Biol. 198, 507–517 (1999)

138

C. Bourjot et al.

17. Sauter, J., Parunak, H., Van Dyke, H., Brueckner, S., Matthews, R.: Tuning synthetic pheromones with evolutionary computing, workshop on evolutionary computation and multiagent systems. In: GECCO, pp. 321–324 (2001) 18. Topin, X., Fourcassié, V., Gleizes, M., Théraulaz, G., Régis, C.: Theories and experiments on emergent behaviour: from natural to artificial systems and back. In: European Conference on Cognitive Science, Siena, Italy (1999) 19. Valckenaers, P., Hadeli, K., Saint Germain, B., Verstraete, P., Van Brussel, H.: Multi-agent system coordination and control based on stigmergy. Comput. Ind. 58, 621–629 (2007) 20. Wilson, E.O.: Sociobiology: The New Synthesis. Harvard University Press, Cambridge (1975/2000)

Chapter 7

Gossip Márk Jelasity

Anyone can start a rumor, but none can stop one. (American proverb)

Objectives The main goal of this chapter is to: • Explain the basic properties of gossip-based information dissemination; • Show how the gossip approach can be used for another domain: information aggregation; • Discuss example systems which are based on gossip or which apply components based on gossip.

7.1 Introduction 7.1.1 Gossip Like it or not, gossip plays a key role in human society. In his controversial book, Dunbar (an anthropologist) goes as far as to claim that the primary reason for the emergence of language was to permit gossip, which had to replace grooming— a common social reinforcement activity in primates—due to the increased group size of early human populations in which grooming was no longer feasible [5]. Whatever the case, it is beyond any doubt that gossip—apart from still being primarily a social activity—is highly effective in spreading information. In particular, information spreads very quickly, and the process is most resistant to attempts to M. Jelasity () University of Szeged and Hungarian Academy of Sciences, P.O. Box 652, 6701 Szeged, Hungary e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_7, © Springer-Verlag Berlin Heidelberg 2011

139

140

M. Jelasity

stop it. In fact, sometimes it is so much so that it can cause serious damage; especially to big corporations. Rumors associating certain corporations to Satanism, or claiming that certain restaurant-chains sell burgers containing rat meat or milk shakes containing cow eyeball fluid as thickener, etc., are not uncommon. Accordingly, controlling gossip has long been an important area of research. The book by Kimmel [14] gives many examples and details on human gossip. While gossip is normally considered to be a means for spreading information, in reality information is not just transmitted mechanically but also processed. A person collects information, processes it and passes the processed information on. In the simplest case, information is filtered at least for its degree of interest. This results in the most interesting pieces of news reaching the entire group, whereas the less interesting ones will stop spreading before getting to everyone. More complicated scenarios are not uncommon either, where information is gradually altered. This increases the complexity of the process and might result in emergent behaviour where the community acts as a “collectively intelligent” (or sometimes perhaps not so intelligent) information processing medium.

7.1.2 Epidemics Gossip is analogous to an epidemic, where a virus plays the role of a piece of information, and infection plays the role of learning about the information. In the past years we even had to learn concepts such as “viral marketing”, made possible through Web 2.0 platforms such as video sharing sites, where advertisers consciously exploit the increasingly efficient and extended social networks to spread ads via gossip. The key idea is that shocking or very funny ads are especially designed so as to maximise the chances that viewers inform their friends about it, and so on. Not surprisingly, epidemic spreading has similar properties to gossip and is equally (if not more) important to understand and control. Due to this analogy and following common practice, we will mix epidemiological and gossip terminology, and apply epidemic spreading theory to gossip systems.

7.1.3 Lessons for Distributed Systems Gossip and epidemics are of interest for large-scale distributed systems for at least two reasons. The first reason is inspiration to design new protocols: gossip has several attractive properties like simplicity, speed, robustness and a lack of central control and bottlenecks. These properties are very important for information dissemination and collective information processing (aggregation) that are both key components of large-scale distributed systems. The second reason is security research. With the steady growth of the Internet, viruses and worms have become increasingly sophisticated in their spreading strate-

7 Gossip

141

gies. Infected computers typically organise into networks (called botnets), and, being able to cooperate and perform coordinated attacks, they represent a very significant threat to IT infrastructure. One approach to fighting these networks is to try and prevent them from spreading, which requires a good understanding of epidemics over the Internet. In this chapter we focus on the former aspect of gossip and epidemics: we treat them as inspiration for the design of robust self-organising systems and services.

7.1.4 Outline We discuss the gossip communication model in the context of two application domains: information dissemination (Sect. 7.2) and information aggregation (Sect. 7.3). For both domains, we start by introducing key approaches and algorithms, along with their theoretical properties, where possible. Subsequently we discuss applications of these ideas within each section, where we pay more attention to practical details and design issues. In Sect. 7.4 we briefly mention application domains that we did not discuss in detail earlier but that are also promising applications of gossip. Finally, we list some key conclusions in Sect. 7.5.

7.2 Information Dissemination The most natural application of gossip (or epidemics) in computer systems is spreading information. The basic idea of processes periodically communicating with peers and exchanging information is not uncommon in large-scale distributed systems and has been applied from the early days of the Internet. For example, the Usenet newsgroup servers spread posts using a similar method, and the IRC chat protocol applies a similar principle as well among IRC servers. In many routing protocols we can also observe routers communicating with neighboring routers and exchanging traffic information, thereby improving routing tables. However, the first real application of gossip that was based on theory and careful analysis and that boosted scientific research into the family of gossip protocols was part of a distributed database system of the Xerox Corporation and was used to make sure each replica of the database on the Xerox internal network was up-todate [4]. In this section we will employ this application as a motivating example and illustration, and at the same time introduce several variants of gossip-based information dissemination algorithms.

7.2.1 The Problem Let us assume that we have a set of database servers (in the case of Xerox, 300 of them, but this number could be much larger as well). All of these servers accept

142

M. Jelasity

updates; that is, new records or modifications of existing records. We want to inform all the servers about each update so that all the replicas of the database are identical and up-to-date. Obviously, we need an algorithm to inform all the servers about a given update. We shall call this task update spreading. In addition, we should take into account the fact that whatever algorithm we use for spreading the update, it will not work perfectly, so we need a mechanism for error correction. At Xerox, update spreading was originally solved by sending the update via email to all the servers, and error correction was done by hand. Sending emails is clearly not scalable: the sending node is a bottleneck. Moreover, multiple sources of error are possible: the sender can have an incomplete list of servers in the network, some of the servers can temporarily be unavailable, email queues can overflow, and so on. Both tasks can be solved in a more scalable and reliable way using an appropriate (separate) gossip algorithm. In the following we first introduce several gossip models and algorithms, and then we explain how the various algorithms can be applied to solve the above-mentioned problems.

7.2.2 Algorithms and Theoretical Notions We assume that we are given a set of nodes that are able to pass messages to each other. In this section we will focus on the spreading of a single update among these nodes. That is, we assume that at a certain point in time, one of the nodes gets a new update from an external source, and from that point we are interested in the dynamics of the spreading of that update. When discussing algorithms and theoretical models, we will use the terminology of epidemiology. According to this terminology, each node can be in one of three states, namely: • susceptible (S): The node does not know about the update; • infected (I): The node knows the update and is actively spreading it; • removed (R): The node has seen the update but is not participating in the spreading process (in epidemiology, this corresponds to death or immunity). These states are relative to one fixed update. If there are several concurrent updates, one node can be infected with one update, while still being susceptible to another update, and so on. For the purpose of theoretical discussion, we will formulate our algorithms assuming that there is only one update in the system but assuming that the nodes do not know that there is only one expected update. This will allow us to derive the key theoretical properties of update propagation while keeping the algorithms simple. In realistic applications there are typically many updates being propagated concurrently, and new updates are inserted continuously. In such scenarios additional techniques can be applied to optimise the amortised cost of propagating a single update. In Sect. 7.2.3 we discuss some of these techniques. In addition, nodes might

7 Gossip

143

loop wait(Δ) p ← random peer if push and in state I then send update to p; end if pull then send update-request to p; end end procedure onUpdate store m.update; // means switching to state I end procedure procedure onUpdateRequest if in state I then send update to m.sender; end end procedure Algorithm 7.1: SI gossip

know the global list or even the insertion time of the updates, as well as the list of updates available at some other nodes. This information can also be applied to reduce propagation cost even further. The allowed state transitions depend on the model that we study. Next, we shall consider the SI model and the SIR model. In the SI model, nodes are initially in state S and can change to state I. Once in state I, a node can no longer change its state (I is an absorbing state). In the SIR model, we allow nodes in state I to switch to state R, where R is the absorbing state.

7.2.2.1 The SI Model The algorithm that implements gossip in the SI model is shown in Algorithm 7.1. It is formulated in an asynchronous message passing style, where each node executes one process (that we call the active thread), and, furthermore, it has message handlers that process incoming messages. The active thread is executed once in each Δ time units. We will call this waiting period a gossip cycle (other terminology is also used such as gossip round or period). In line 2 we assume that a node can select a random peer node from the set of all nodes. This assumption is not trivial. We will discuss random peer sampling briefly in Sect. 7.4. The algorithm makes use of two important Boolean parameters called push and pull. At least one of them has to be true, otherwise no messages are sent. Depending on these parameters, we can talk about push, pull and push–pull gossip, each

144

M. Jelasity

having significantly different dynamics and cost. In push gossip, susceptible nodes are passive, and infective nodes actively infect the population. In pull and push–pull gossip, each node is active. Notice that in line 2 we do not test whether the node is infected. The reason is that we assumed that nodes do not know how many updates are expected and do not exchange information about which updates they have received. Obviously, a node cannot stop pulling for updates unless it knows what updates can be expected; and it cannot avoid getting known updates either unless it advertises which updates it has already. Therefore, in the simplest case, we pull for any update all the time. Optimisations to mitigate this problem are possible, but solving it in a scalable way is not trivial. For theoretical purposes, we will assume that messages are transmitted without delay, and for now we will assume that no failures occur in the system. We will also assume that messages are sent at the same time at each node, that is, messages from different cycles do not mix, and cycles are synchronised. None of these assumptions are critical for practical usability, but they are needed for theoretical derivations that nevertheless give a fair indication of the qualitative and also quantitative behaviour of gossip protocols. Let us start with the discussion of the push model. We will consider the propagation speed of the update as a function of the number of nodes N . Let s0 denote the proportion of susceptible nodes at the time of introducing the update at one node. Clearly, s0 = (N − 1)/N . Let st denote the proportion of susceptible nodes at the end of the tth cycle; that is, at time tΔ. We can calculate the expectation of st+1 as a function of st , provided that the peer selected in line 2 is chosen independently at each node and independently of past decisions as well. In this case, we have 1 N (1−st ) E(st+1 ) = st 1 − ≈ st e−(1−st ) , (7.1) N where N (1 − st ) is the number of nodes that are infected at cycle t, and (1 − 1/N ) is the probability that a fixed infected node will not infect some fixed susceptible node. Clearly, a node is susceptible in cycle t + 1 if it was susceptible in cycle t and all the infected nodes picked some other node. Actually, as it turns out, this approximative model is rather accurate (the deviation from it is small), as shown by Pittel in [16]: we can take the expected value E(st+1 ) as a good approximation of st+1 . It is easy to see that if we wait long enough, then eventually all the nodes will receive the update. In other words, the probability that a particular node never receives the update is zero. But what about the number of cycles that are necessary to let every node know about the update (become infected)? Pittel proves that in probability, SN = log2 N + log N + O(1)

as N → ∞,

(7.2)

where SN = min{t : st = 0} is the number of cycles needed to spread the update. The proof is rather long and technical, but the intuitive explanation is rather simple. In the initial cycles, most nodes are susceptible. In this phase, the number of infected nodes will double in each cycle to a good approximation. However, in the

7 Gossip

145

last cycles, where st is small, we can see from (7.1) that E(st+1 ) ≈ st e−1 . This suggests that there is a first phase, lasting for approximately log2 N cycles, and there is a last phase lasting for log N cycles. The “middle” phase, between these two phases, can be shown to be very fast, lasting a constant number of cycles. Equation (7.2) is often cited as the key reason why gossip is considered efficient: it takes only O(log N) cycles to inform each node about an update, which suggests very good scalability. For example, with the original approach at Xerox, based on sending emails to every node, the time required is O(N ), assuming that the emails are sent sequentially. However, let us consider the total number of messages that are being sent in the network until every node gets infected. For push gossip, it can be shown that it is O(N log N ). Intuitively, the last phase that lasts O(log N ) cycles with st being very small already involves sending too many messages by the infected nodes. Most of these messages are in vain, since they target nodes that are already infected. The optimal number of messages is clearly O(N), which is attained by the email approach. Fortunately, the speed and message complexity of the push approach can be improved significantly using the pull technique. Let us consider st in the case of pull gossip. Here, we get the simple formula of E(st+1 ) = st · st = st2 ,

(7.3)

which intuitively indicates a quadratic convergence if we assume that the variance of st is small. When st is large, it decreases slowly. In this phase the push approach clearly performs better. However, when st is small, the pull approach results in a significantly faster convergence than push. In fact, the quadratic convergence phase, roughly after st < 0.5, lasts only for O(log log N) cycles, as can be easily verified. One can, of course, combine push and pull. This can be expected to work faster than either push or pull separately, since in the initial phase push messages will guarantee fast spreading, while in the end phase pull messages will guarantee the infecting of the remaining nodes in a short time. Although faster in practice, the speed of push–pull is still O(log N), due to the initial exponential phase. What about message complexity? Since in each cycle each node will send at least one request, and O(log N) cycles are necessary for the update to reach all the nodes, the message complexity is O(N log N). However, if we count only the updates and ignore request messages, we get a different picture. Just counting the updates is not meaningless, because an update message is normally orders of magnitude larger than a request message. It has been shown that in fact the push–pull gossip protocol sends only O(N log log N) updates in total [9]. The basic idea behind the proof is again based on dividing the spreading process into phases and calculating the message complexity and duration of each phase. In essence, the initial exponential phase—that we have seen with push as well— requires only O(N ) update transmissions, since the number of infected nodes (that send the messages) grows exponentially. But the last phase, the quadratic shrinking phase as seen with pull, lasts only O(log log N) cycles. Needless to say, as with the other theoretical results, the mathematical proof is quite involved.

146

M. Jelasity

7.2.2.2 The SIR Model In the previous section we gave some important theoretical results regarding convergence speed and message complexity. However, we ignored one problem that can turn out to be important in practical scenarios: termination. Push protocols never terminate in the SI model, constantly sending useless updates even after each node has received every update. Pull protocols could stop sending messages if the complete list of updates was known in advance: after receiving all the updates, no more requests need to be sent. However, in practice not even pull protocols can terminate in the SI model, because the list of updates is rarely known. Here we will discuss solutions to the termination problem in the SIR model. These solutions are invariably based on some form of detecting and acting upon the “age” of the update. We can design our algorithm with two different goals in mind. First, we might wish to ensure that the termination is optimal; that is we want to inform all the nodes about the update, and we might want to minimise redundant update transmissions at the same time. Second, we might wish to opt for a less intelligent, simple protocol and analyse the size of the proportion of the nodes that will not get the update as a function of certain parameters. One simple way of achieving the first design goal of optimality is by keeping track of the age of the update explicitly, and stop transmission (i.e. switching to the removed state, hence implementing the SIR model) when a pre-specified age is reached. This age threshold must be calculated to be optimal for a given network size N using the theoretical results sketched above. This, of course, assumes that each node knows N . In addition, a practically error- and delay-free transmission is also assumed, or at least a good model of the actual transmission errors is needed. Apart from this problem, keeping track of the age of the update explicitly represents another, non-trivial practical problem. We assumed in our theoretical discussions that messages have no delay and that cycles are synchronised. When these assumptions are violated, it becomes rather difficult to determine the age of an update with an acceptable precision. From this point on, we shall discard this approach and focus on simple asynchronous methods that are much more robust and general, but are not optimal. To achieve the second design goal of simplicity combined with reasonable performance, we can try to guess when to stop based on local information and perhaps information collected from a handful of peers. These algorithms have the advantage of simplicity and locality. Besides, in many applications of the SIR model, strong guarantees on complete dissemination are not necessary, as we will see later on. Perhaps the simplest possible implementation is when a node moves to the removed state with a fixed probability whenever it encounters a peer that has already received the update. Let this probability be 1/k, where the natural interpretation of parameter k is the average number of times a node sends the update to a peer that turns out to already have the update before stopping its transmission. Obviously, this implicitly assumes a feedback mechanism because nodes need to check whether the peer they sent the update to already knew the update or not.

7 Gossip

147

loop wait(Δ) p ← random peer if push and in state I then send update to p end if pull then send update-request to p end end procedure onFeedback(m) switch to state R with prob. 1/k end procedure procedure onUpdate(m) if in state I or R then send feedback to m.sender end else store m.update; // now in state I end end procedure procedure onUpdateRequest(m) if in state I then send update to m.sender end end procedure Algorithm 7.2: An SIR gossip variant

As shown in Algorithm 7.2, this feedback mechanism is the only difference between SIR and SI gossip. The active thread and procedure ON U PDATE R EQUEST are identical to Algorithm 7.1. However, procedure ON U PDATE sends a feedback message when the received update is known already. This message is processed by procedure O N F EEDBACK , eventually switching the node to the removed state. When in the removed state, procedure ON U PDATE R EQUEST will no longer deliver the update. Mathematical models of SIR algorithms are more complicated than those of the SI model. A typical approach is to work with differential equations, as opposed to the discrete stochastic approach we applied previously. Let us illustrate this approach via an analysis of Algorithm 7.2, assuming a push variant. Following [2, 4], we can write ds = −si, dt

(7.4)

148

M. Jelasity

1 di = si − (1 − s)i, (7.5) dt k where s(t) and i(t) are the proportions of susceptible and infected nodes, respectively. The nodes in the removed state are given by r(t) = 1 − s(t) − i(t). We can take the ratio, eliminating t: di k+1 1 =− + , ds k ks

(7.6)

which yields k+1 1 s + log s + c, (7.7) k k where c is the constant of integration, which can be determined using the initial condition that i(1 − 1/N ) = 1/N (where N is the number of nodes). For a large N , we have c ≈ (k + 1)/k. Now we are interested in the value s ∗ where i(s ∗ ) = 0: at that time sending the update is terminated, because all nodes are susceptible or removed. In other words, s ∗ is the proportion of nodes that do not know the update when gossip stops. Ideally, s ∗ should be zero. Using the results, we can write an implicit equation for s ∗ as follows: s ∗ = exp −(k + 1)(1 − s ∗ ) . (7.8) i(s) = −

This tells us that the spreading is very effective. For k = 1, 20% of the nodes are predicted to miss the update, but with k = 5, 0.24% will miss it, while with k = 10 it will be as few as 0.00017%. Let us now proceed to discussing message complexity. Since full dissemination is not achieved in general, our goal is now to approximate the number of messages needed to decrease the proportion of susceptible nodes to a specified level. Let us first consider the push variant. In this case, we make the rather striking observation that the value of s depends only on the number of messages m that have been sent by the nodes. Indeed, each infected node picks peers independently at random to send the update to. That is, every single update message is sent to a node selected independently at random from the set of all the nodes. This means that the probability that a fixed node is in state S after a total of m update messages has been sent can be approximated by m 1 m ≈ exp − . (7.9) s(m) = 1 − N N Substituting the desired value of s, we can easily calculate the total number of messages that need to be sent in the system: it is m ≈ −N log s.

(7.10)

If we demand that s = 1/N , that is we allow only for a single node not to see the update, then we need m ≈ N log N . This reminds us of the SI model that had an O(N log N ) message complexity to achieve full dissemination. If, on the other hand, we allow for a constant proportion of the nodes not to see the update (s = 1/c), then

7 Gossip

149

we have m ≈ N log c; that is a linear number of messages suffice. Note that s or m cannot be set directly, but only through other parameters such as k. Another notable point is that (7.9) holds irrespective of whether we apply a feedback mechanism or not, and irrespective of the exact algorithm applied to switch to state R. In fact, it applies even for the pure SI model, since all we assumed was that it is a push-only gossip with random peer selection. Hence it is a strikingly simple, alternative way to illustrate the O(N log N) message complexity result shown for the SI model: roughly speaking, we need approximately N log N messages to make s go below 1/N . Since m determines s irrespective of the details of the applied push gossip algorithm, the speed at which an algorithm can have the infected nodes send m messages determines the speed of convergence of s. With this observation in mind, let us compare a number of variants of SIR gossip. Apart from Algorithm 7.2, one can implement termination (switching to state S) in several different ways. For example, instead of a probabilistic decision in procedure ON F EEDBACK, it is also possible to use a counter and switch to state S after receiving the kth feedback message. Feedback could be eliminated altogether, and moving to state R could depend only on the number of times a node has sent the update. It is not hard to see that the counter variants improve load balancing. This in turn improves speed because we can always send more messages in a fixed amount of time if the message sending load is well balanced. In fact, among the variants described above, applying a counter without feedback results in the fastest convergence. However, parameter k has to be set appropriately to achieve a desired level of s. To set k and s appropriately, one needs to know the network size. Variants using a feedback mechanism achieve a somewhat less efficient load balancing, but they are more robust to the value of k and to network size: they can “self-tune” the number of messages based on the feedback. For example, if the network is large, more update messages will be successful before the first feedback is received. Lastly, as in the SI model, it is apparent that in the end phase the pull variant is much faster and uses fewer update messages. It does this at the cost of constantly sending update requests. We think in general that, especially when updates are constantly being injected, the push–pull algorithm with counter and feedback is probably the most desirable alternative.

7.2.3 Applications We first explain how the various protocols we discussed were applied at Xerox for maintaining a consistent set of replicas of a database. Although we cannot provide a complete picture here (see [4]), we elucidate the most important ideas. In Sect. 7.2.1 we identified two sub-problems, namely update spreading and error correction. The former is implemented by an SIR gossip protocol, and the latter by

150

M. Jelasity

an SI protocol. The SIR gossip is called rumor mongering and is run when a new update enters the system. Note that in practice, many fresh updates can piggyback a single gossip message, but the above-mentioned convergence properties hold for any single fixed update. The SI algorithm for error correction works for every update ever entered, irrespective of age, simultaneously for all updates. In a naive implementation, the entire database would be transmitted in each cycle by each node. Evidently, this is not a good idea, since databases can be very large and are mostly rather similar. Instead, the nodes first try to discover what the difference is between their local replicas by exchanging compressed descriptions such as checksums (or lists of checksums taken at different times) and transmit only the missing updates. However, one cycle of error correction is typically much more expensive than rumor mongering. The SI algorithm for error correction is called anti-entropy. This is not a very fortunate name: we should remark here that it has no deeper meaning than to express the fact that “anti-entropy” will increase the similarity among the replicas thereby increasing “order” (decreasing randomness). So, since entropy is usually considered to be a measure of “disorder”, the name “anti-entropy” simply means “anti-disorder” in this context. In the complete system, the new updates are spread through rumor mongering, and anti-entropy is run occasionally to take care of any undelivered updates. When such an undelivered update is found, the given update is redistributed by re-inserting it as a new update into the database where it was not present. This is a very simple and efficient method, because update spreading via rumor mongering has a cost that depends on the number of other nodes that already have the update: if most of the nodes already have it, then the redistribution will die out very quickly. Let us quickly compare this solution to the earlier, email-based approach. Emailing updates and rumor mongering are similar in that both focus on spreading a single update and have a certain small probability of error. Unlike email, gossip has no bottleneck nodes and hence is less sensitive to local failure and assumes less about local resources such as bandwidth. This makes gossip a significantly more scalable solution. Gossip uses slightly more messages in total for the distribution of a single update. But with frequent updates in a large set of replicas, the amortised cost of gossip (number of messages per update) is more favourable (remember that one message may contain many updates). In practical implementations, additional significant optimisations have been performed. Perhaps the most interesting one is spatial gossip where, instead of picking a peer at random, nodes select peers based on a distance metric. This is important because if the underlying physical network topology is such that there are bottleneck links connecting dense clusters, then random communication places a heavy load on such links that grows linearly with system size. In spatial gossip, nodes favour peers that are closer in the topology, thereby relieving the load from long-distance links but at the same time sacrificing some of the spreading speed. This topic is discussed at great length in [12]. We should also mention the removal of database entries. This is solved through “death certificates” that are updates stating that a given entry should be removed.

7 Gossip

151

Needless to say, death certificates cannot be stored indefinitely because eventually the databases would be overloaded by them. This problem requires additional tricks such as removing most but not all of them, so that the death certificate can be reactivated if the removed update pops up again. Apart from the application discussed above, the gossip paradigm has recently received yet another boost. After getting used to Grid and P2P applications, and witnessing the emergence of the huge and often geographically distributed data centers that increase in size and capacity at an incredible rate, in the past years we had to learn another term: cloud computing [6]. Cloud computing involves a huge amount of distributed resources (a cloud), typically owned by a single organisation and organised in such a way that, for the user, it appears to be a coherent and reliable service. Examples include storage service or Grid-style computing services. Recently big players in the IT sector have introduced their own cloud computing solutions, for example, Google and IBM [15] and Amazon [1]. These applications represent cutting edge technology at the time of writing, and it is not always clear how they work due to corporate secrecy, but from several sources it seems rather evident that gossip protocols are involved. For example, after a recent crash of Amazon’s S3 storage service, the message explaining the failure included some details: . . . Amazon S3 uses a gossip protocol to quickly spread server state information throughout the system. This allows Amazon S3 to quickly route around failed or unreachable servers, among other things.1 . . .

In addition, a recent academic publication on the technology underlying Amazon’s computing architecture provides further details on gossip protocols [3], revealing that an anti-entropy gossip protocol is responsible for maintaining a full membership table at each server (that is, a fully connected overlay network with server state information).

7.3 Aggregation The gossip communication paradigm can be generalised to applications other than information dissemination. In these applications some implicit notion of spreading information will still be present, but the emphasis is not only on spreading but also on processing information on the fly. This processing can be for creating summaries of distributed data; that is computing a global function over the set of nodes based only on gossip-style communication. For example, we might be interested in the average or maximum of some attribute of the nodes. The problem of calculating such global functions is called data aggregation or simply aggregation. We might want to compute more complex 1 http://status.aws.amazon.com/s3-20080720.html

152

M. Jelasity

functions as well, such as fitting models on fully distributed data, in which case we talk about the problem of distributed data mining. In the past few years, a lot of effort has been directed at a specific problem: calculating averages. Averaging can be considered the archetypical example of aggregation. It is a very simple problem and yet very useful: based on the average of a suitably defined local attribute, we can calculate a wide range of values. To elaborate on this notion, let us introduce some formalism. Let yi be an attribute value at node i for all 0 < i ≤ N . We are interested in the average y¯ = N i=1 yi /N . If we can calculate the average, then we can calculate any mean of the form N i=1 f (zi ) −1 , (7.11) g(z1 , . . . , zN ) = f N where f () is a suitable function. For example, f (x) = log x generates the geometric mean, while f (x) = 1/x generates the harmonic mean. Clearly, if yi = f (zi ), then we can easily calculate g(z1 , . . . , zN ) = f −1 (y). ¯ In addition, if we calculate the mean of several powers of yi , then we can calculate the empirical moments of the distribution of the values. For example, the (biased) estimate of the variance can be expressed as a function over averages of yi2 and yi : σN2 = y¯2 − y¯ 2 .

(7.12)

Finally, other interesting quantities can be calculated using averaging as a primitive. For example, if every attribute value is zero, except at one node, where the value is 1, then y¯ = 1/N , so the network size is given by N = 1/y. ¯ In the remaining parts of this section we focus on several gossip protocols for calculating the average of node attributes.

7.3.1 Algorithms and Theoretical Notions The first, perhaps simplest, algorithm we discuss is push–pull averaging, presented in Algorithm 7.3. Each node periodically selects a random peer to communicate with and then sends the local estimate of the average x. The recipient node then replies with its own current estimate. Both participating nodes (the sender and the one that sends the reply) will store the average of the two previous estimates as a new estimate. Similarly to our treatment of information spreading, Algorithm 7.3 is formulated for an asynchronous message passing model, but we will assume several synchronicity properties when discussing the theoretical behaviour of the algorithm. We will return to the issue of asynchrony in Sect. 7.3.1.1. For now, we also treat the algorithm as a one-shot algorithm; that is we assume that first the local estimate xi of node i is initialised as xi = yi for all the nodes i = 1, . . . , N , and subsequently the gossip algorithm is executed. This assumption will also be relaxed in Sect. 7.3.1.2, where we briefly discuss the case where the

7 Gossip

153

loop wait(Δ) p ← random peer send push(x) to p end procedure onPushUpdate(m) send pull(x) to m.sender x ← (m.x + x)/2 end procedure procedure onPullUpdate(m) x ← (m.x + x)/2 end procedure Algorithm 7.3: Push–pull averaging

attributes yi can change over time, and the task is to continuously update the approximation of the average. Let us first look at the convergence of the algorithm. It is clear that the state when xi = y¯ for all i is a fixed point, assuming that there are no node failures and message failures, and that the messages are delivered without delay. It is not very difficult to convince oneself that the algorithm converges to this fixed point in probability. We omit the technical details of the proof; the trick is to first observe that the sum of the approximations remains constant throughout. More precisely, N

xi (t) = N y¯

(7.13)

i=1

for any t. This very important property is called mass conservation. We then look at the difference between the minimal and maximal approximations and show that this difference can only decrease and, furthermore, that it converges to zero in probability, using the fact that peers are selected at random. But if all the approximations are the same, they can only be equal to y¯ due to mass conservation. The really interesting question, however, is the speed of convergence. The fact of convergence is easy to prove in a probabilistic sense, but such a proof is useless from a practical point of view without characterising speed. The speed of the protocol is illustrated in Fig. 7.1. The process shows a diffusion-like behaviour. Recall, that diffusion is executed on top of a random network over the pixels and not on the twodimensional grid. The arrangement of the pixels is for illustration purposes only. It is possible to characterise convergence speed by studying the variance-like statistic σN2 =

N 1 (xi − y) ¯ 2, N i=1

(7.14)

154

M. Jelasity

Fig. 7.1 Illustration of the averaging protocol—pixels correspond to nodes (100 × 100 pixels = 10,000 nodes) and pixel colour to the local approximation of the average

which describes how accurate the set of current estimates is. It can be shown (see [7, 11]) that

1 E σN2 (t + 1) ≤ σN2 (t), (7.15) 2 where the time index t indicates a cycle. That is variance decreases by a constant factor in each cycle. In practice, 10–20 cycles of the protocol already provide an extremely accurate estimation: the protocol not only converges, but it converges very quickly as well.

7.3.1.1 Asynchrony In the case of information dissemination, allowing for unpredictable and unbounded message delays (a key component of the asynchronous model) has no effect on the correctness of the protocol; it only has an (in practice, marginal) effect on spreading speed. For Algorithm 7.3 however, correctness is no longer guaranteed in the presence of message delays. To see why, imagine that node j receives a PUSH U PDATE message from node i and as a result it modifies its own estimate and sends its own previous estimate back to i. But after that point, the mass conservation property of the network will be violated: the sum of all approximations will no longer be correct. This is not a problem if neither node j nor node i receives or sends another message during the time node i is waiting for the reply. However, if they do, then the state of the network

7 Gossip

155

loop wait(Δ) p ← random peer send (x/2, w/2) to p x ← x/2 w ← w/2 end procedure onUpdate(m) x ← m.x + x w ← m.w + w end procedure Algorithm 7.4: Push averaging

may become corrupted. In other words, if the pair of push and pull messages are not atomic, asynchrony is not tolerated well. Algorithm 7.4 is a clever modification of Algorithm 7.3 and is much more robust to message delay. The algorithm is very similar, but here we introduce another attribute called w. For each node i, we initially set wi = 1 (so the sum of these values is N ). We also modify the interpretation of the current estimate: on node i it will be xi /wi instead of xi , as in the push–pull variant. To understand why this algorithm is more robust to message delay, consider that we now have mass conservation in a different sense: the sum of the attribute values at the nodes plus the sum of the attribute values in the undelivered messages remains constant for both attributes x and w. This is easy to see if one considers the active thread which keeps half of the values locally and sends the other half in a message. In addition, it can still be proven that the variance of the approximations xi /wi can only decrease. As a consequence, messages can now be delayed, but if message delay is bounded, then the variance of the set of approximations at the nodes and in the messages waiting for delivery will tend to zero. Due to mass conservation, these approximations will converge to the true average, irrespective of how much of the total “mass” is in undelivered messages. (Note that the variance of xi or wi alone is not guaranteed to converge to zero.)

7.3.1.2 Robustness to Failure and Dynamism We will now consider message and node failures. Both kinds of failures are unfortunately more problematic than asynchrony. In the case of information dissemination, failure has no effect on correctness: message failure only slows down the spreading process, and node failure is problematic only if every node that stores the new update fails. In the case of push averaging, losing a message typically corrupts mass conservation. In the case of push–pull averaging, losing a push message will have no effect,

156

M. Jelasity

but losing the reply (pull message) may corrupt mass conservation. The solutions to this problem are either based on failure detection (that is, they assume that a node is able to detect whether a message was delivered or not) and correcting actions based on the detected failure, or they are based on a form of rejuvenation (restarting), where the protocol periodically re-initialises the estimates, thereby restoring the total mass. The restarting solution is feasible due to the quick convergence of the protocol. Both solutions are somewhat inelegant, but gossip is attractive mostly because of the lack of reliance on failure detection, which makes restarting more compatible with the overall gossip design philosophy. Unfortunately, restarting still allows for a bounded inaccuracy due to message failures, while failure detection offers accurate mass conservation. Node failures are a source of problems as well. By node failure we mean the situation where a node leaves the network without informing the other nodes about it. Since the current approximation xi (or xi /wi ) of a failed node i is typically different from yi , the set of remaining nodes will end up with an incorrect approximation of the average of the remaining attribute values. Handling node failures is problematic even if we assume perfect failure detectors. Solutions typically involve nodes storing the contributions of each node separately. For example, in the push–pull averaging protocol, node i would store δj i , the sum of the incremental contributions of node j to xi . More precisely, when receiving an update from j (push or pull), node i calculates δj i = δj i + (xj − xi )/2. When node i detects that node j failed, it performs the correction xi = xi − δj i . We should mention that this is feasible only if the selected peers are from a small fixed set of neighboring nodes (and not randomly picked from the network); otherwise all the nodes would need to monitor an excessive number of other nodes for failure. Besides, message failure can interfere with this process too. The situation is further complicated by nodes failing temporarily, perhaps not even being aware of the fact that they have been unreachable for a long time by some nodes. Also note that the restart approach solves the node failure issue as well, without any extra effort or failure detectors, although, as previously, allowing for some inaccuracy. Finally, let us consider a dynamic scenario where mass conservation is violated due to changing y values (so the approximations evolved at the nodes will no longer reflect the correct average). In such cases one can simply set xi = xi + yinew − yiold , which corrects the sum of the approximations, although the protocol will need some time to converge again. As in the previous cases, restarting solves this problem too without any extra measures.

7.3.2 Applications The diffusion-based averaging protocols we focused on will most often be applied as a primitive to help other protocols and applications such as load balancing, task allocation or the calculation of relatively complex models of distributed data such as

7 Gossip

157

spectral properties of the underlying graph [8, 10]. Sensor networks are especially interesting targets for applications, due to the fact that their very purpose is data aggregation, and they are inherently local: nodes can typically communicate with their neighbours only [18]. However, sensor networks do not support point-to-point communication between arbitrary pairs of nodes as we assumed previously, which makes the speed of averaging much slower. As a specific application of aggregation, we discuss a middleware system called Astrolabe [17] in more detail, which is rumored to be in use at Amazon in some form. Astrolabe is all about aggregation, but the basic idea is not diffusion. Rather, aggregates are propagated along a virtual hierarchical structure. However, Astrolabe is based on gossip in many ways, as we will see, and it is the most “real-world” application of the aggregation idea we are aware of, so it is useful to devote a few paragraphs on it. Let us start with the hierarchical structure Astrolabe is based on. First of all, each node is assigned a name (when joining) similar to a domain name, which consists of zones. For example, a node could be called “/Hungary/Szeged/pc3”, which assumes four zones: the root zone “/” and the three successively more specific zones. Each zone has a descriptor: a list of attributes. The leaf zones (for example, “pc3”) have the raw system attributes in their descriptor such as storage space, stored files, and so on. These attributes will be the input for aggregation. Descriptors of higher-level zones contain summaries of lower-level zones; accordingly, the root zone descriptor contains system wide aggregates. The key idea is that fresh replicas of the zone descriptors are maintained at nodes that belong to the given zone and at nodes that are siblings of the given zone. For example, staying with the previous example, the descriptor of zone “Szeged” will be replicated at all the nodes in the same zone, and in addition, at all the nodes in the zones under “Hungary”. This way, any node under “Hungary” can compute the descriptor of “Hungary” (they will all have a replica of the descriptors of all the zones under “Hungary”). The descriptor of “Hungary” will in turn be replicated in each zone under the root zone, etc. Eventually every single node is able to compute the descriptor of the root zone as well. It should be clear by now that the implementation boils down to replicating databases, as we saw in the previous sections on information dissemination, since each node has a set of zone descriptors that need to be replicated. Indeed, Astrolabe uses a version of anti-entropy for this purpose; only the situation is a bit more delicate, because not every node is interested in all the descriptors. As we explained, each node is interested only in the descriptors of its own zones (for example, “/”, “Hungary/”, “Szeged/”, and “pc3/”) and the descriptors that are siblings of some of its own zones. This way, the system can avoid replicating the potentially enormous complete set of descriptors, while still being able to calculate the descriptors for its own zones locally. This twist has two important implications: first, not all nodes will need a list of all other nodes, and second, nodes can learn more from nodes that are closer to them in the hierarchy, so they will want to communicate to such nodes more often. This requires a non-trivial peer selection approach. Astrolabe itself uses the descriptors

158

M. Jelasity

themselves to store contacts for all zones of interest. These contacts can then be used to gossip with (running the anti-entropy over the common interest of the two nodes). This has the advantage that as long as a node has at least one contact from any of the zones it is interested in (which can be the root zone as well) it will eventually fill in the contact list for each relevant zone automatically. As usual, there are a number of practical issues, such as detecting failed nodes (and zones), the joining procedure, and other details involving asynchrony, the timestamping of updates, and so on. These can be solved without sacrificing the main properties of the system, although at the cost of losing some of its original simplicity. Finally, we should point out that Astrolabe is an extremely generic system that supports much more than average calculation. In fact it supports an SQL-like language for queries on the virtual database that is defined by the set of descriptors of all the zones in the system. Astrolabe behaves as if it was a complete database, while in fact the nodes replicate only a small (roughly logarithmic) proportion of the complete set of data and emulate a complete database via progressive summaries.

7.4 What is Gossip After All? So far we have discussed two applications of the gossip idea: information dissemination and aggregation. By now it should be rather evident that these applications, although different in detail, have a common algorithmic structure. In both cases an active thread selects a peer node to communicate with, followed by a message exchange and the update of the internal states of both nodes (for push–pull) or one node (for push or pull). It is rather hard to capture what this “gossipness” concept means exactly. Attempts have been made to do so with mixed success [13]. For example, periodic and local communication to random peers appears to be a core feature. However, in the SIR model, nodes can stop communicating. Besides, we have seen that in some gossip protocols neighboring nodes need not be random in every cycle but instead they can be fixed and static. Attempting to capture any other aspect appears to lead to similar philosophical problems: any characterisation that covers all protocols we intuitively consider gossip seems to cover almost all distributed protocols. On the other hand, restrictions invariably exclude protocols we definitely consider gossip. Accordingly, we do not intend to characterise what this common structure is, but instead we propose the template (or design pattern) shown in Algorithm 7.5. This template covers our two examples presented earlier. In the case of information dissemination the state of a node is defined by the stored updates, while in the case of averaging the state is the current approximation of the average at the node. In addition, the template covers a large number of other protocols as well. One notable application we have not covered in this chapter is the construction and management of overlay networks. In this case the state of a node is a set of

7 Gossip

159

loop wait(Δ) p ← selectPeer() send push(state) to p end procedure onPushUpdate(m) send pull(state) to m.sender state ← update(state,m.state) end procedure procedure onPullUpdate(m) state ← update(state,m.state) end procedure Algorithm 7.5: The push–pull gossip algorithm skeleton

node addresses that define an overlay network. (A node is able to send messages to an address relying on lower layers of the networking stack; hence the name “overlay”.) In a nutshell, the state (the set of overlay links) is then communicated via gossip, and nodes select their new set of links from the set of all links they have seen. Through this mechanism one can create and manage a number of different overlay networks such as random networks, structured networks (like a ring) or proximity networks based on some distance metric, for example semantic or latency-based distance. These networks can be applied by higher-level applications or by other gossip protocols. For example, random networks are excellent for implementing random peer sampling, a service all the algorithms rely on in this chapter when selecting a random peer to communicate with. Apart from overlay management, other applications include load balancing, resource discovery and assignment, and so on.

7.5 Conclusion In this chapter we discussed two applications of the gossip communication model: information dissemination and aggregation. We showed that both applications use a very similar communication model, and both applications provide probabilistic guarantees for an efficient and effective execution. We should emphasise that gossip protocols represent a departure from “classical” approaches in distributed computing where correctness and reliability were the top priorities and performance (especially speed and responsiveness) was secondary. To put it simply: gossip protocols—if done well—are simple, fast, cheap and extremely scalable, but they do not always provide a perfect or correct result under all circumstances and in all system models. But in many scenarios a “good enough” result is

160

M. Jelasity

acceptable, and in realistic systems gossip components can always be backed up by more heavy-weight but more reliable methods that provide eventual consistency or correctness. A related problem is malicious behaviour. Unfortunately, gossip protocols in open systems with multiple administrative domains are rather vulnerable to malicious behaviour. Current applications of gossip are centered on single administrative domain systems, where all the nodes are controlled by the same entity, and therefore the only sources of problems are hardware, software or network failures. In an open system nodes can be selfish, or even worse, they can be malicious. Current secure gossip algorithms are orders of magnitude more complicated than basic versions, thus losing many of the original advantages of gossipping. All in all, gossip algorithms are a great tool for solving certain kinds of very important problems under certain assumptions. In particular, they can help in the building of enormous cloud computing systems that are considered the computing platform of the future by many, and provide tools for programming sensor networks as well.

7.6 Problems–Exercises 7.1 We have seen that in some protocols it is useful to keep track of the age of the update, that is to know how long ago a given update was inserted. If clocks are synchronised, this is easy: one needs only a timestamp. Try to think of algorithms for the various algorithms we discussed for keeping track of or at least approximating the age of the update if the clocks are not synchronised! To make the problem harder, suppose that the cycles are not synchronised either (that is, all the nodes have the same period Δ, but they start the cycle at a random time). Can you do anything if messages have a random unpredictable and unbounded delay? 7.2 We assumed that the updates arrive unexpectedly and that nodes that do not have the update do not know that they miss it. What difference does it make if we know the list of updates that are expected? What optimisations are possible for push, pull and push–pull? What kind of running times and message complexities can we reach? 7.3 We discussed the speed of spreading under strong synchrony assumptions: we assumed that cycles are synchronised and messages are delivered without delay or errors. Think about the speed of spreading if we drop the synchrony assumption. Is it slower or faster? 7.4 We said that if messages can be delayed, then push–pull averaging can get corrupted. Work out an example that proves this!

7 Gossip

161

Key Points • Gossip protocols are based on periodic information exchange, during which all nodes communicate with randomly selected peer nodes. This gossip communication model supports the implementation of a number of functions such as information dissemination, aggregation or overlay topology construction. • Gossip protocols represent a probabilistic mindset where we trade off deterministic guarantees of success (replacing it with strong probabilistic guarantees) for a significant increase in flexibility and scalability. • Push–pull gossip-based broadcasting of an update can reach a time complexity of O(log N) and a message complexity of O(N log log N ), which approaches the performance of tree-based broadcasting, while being extremely robust and scalable to benign failure. • Gossip protocols are a key component in the emerging cloud computing systems, as well as in sensor networks and other fully distributed dynamic systems, which makes them highly relevant in practical applications.

7.7 Further Reading Gossip-Based Networking. Captures very well the state-of-the-art of the field at the date of publication. Contains mostly overview and position papers evolved during a Lorentz Center workshop in Leiden, a rich source of references and ideas. (A.-M. Kermarrec and M. van Steen, editors, ACM SIGOPS Operating Systems Review 41, 2007.) Epidemic algorithms for replicated database maintenance. This is a citation classic that is perceived as the work that initiated this area of research. It is still very well worth reading; our discussion was partly based on this paper. (Demers, A., Greene, D., Hauser, C., Irish, W., Larson, J., Shenker, S., Sturgis, H., Swinehart, D., and Terry, D., In: Proceedings of the 6th Annual ACM Symposium on Principles of Distributed Computing (PODC’87), pp. 1–12. ACM Press, 1987.) The Mathematical Theory of Infectious Diseases and Its Applications. An old book that is still a good starting point for the relevant theoretical aspects. (N.T.J. Bailey, Griffin, London, second edition, 1975.) Acknowledgement While writing this chapter, M. Jelasity was supported by the Bolyai Scholarship of the Hungarian Academy of Sciences.

References 1. Amazon Web Services. http://aws.amazon.com

162

M. Jelasity

2. Bailey, N.T.J.: The Mathematical Theory of Infectious Diseases and Its Applications, 2nd edn. Griffin, London (1975) 3. De Candia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: Amazon’s highly available key-value store. In: SOSP’07: Proceedings of Twenty-First ACM SIGOPS Symposium on Operating Systems Principles, pp. 205–220. ACM, New York (2007). doi:10.1145/1294261.1294281 4. Demers, A., Greene, D., Hauser, C., Irish, W., Larson, J., Shenker, S., Sturgis, H., Swinehart, D., Terry, D.: Epidemic algorithms for replicated database maintenance. In: Proceedings of the 6th Annual ACM Symposium on Principles of Distributed Computing (PODC’87), Vancouver, British Columbia, Canada, pp. 1–12. ACM, New York (1987). doi:10.1145/41840.41841 5. Dunbar, R.: Grooming, Gossip, and the Evolution of Language. Harvard University Press, Harvard (1998) 6. Hand, E.: Head in the clouds. Nature 449, 963 (2007). doi:10.1038/449963a 7. Jelasity, M., Montresor, A., Babaoglu, O.: Gossip-based aggregation in large dynamic networks. ACM Trans. Comput. Syst. 23(3), 219–252 (2005). doi:10.1145/1082469.1082470 8. Jelasity, M., Canright, G., Engø-Monsen, K.: Asynchronous distributed power iteration with gossip-based normalization. In: Kermarrec, A.M., Bougé, L., Priol, T. (eds.) Euro-Par 2007. Lecture Notes in Computer Science, vol. 4641, pp. 514–525. Springer, Berlin (2007). doi:10.1007/978-3-540-74466-5_55 9. Karp, R., Schindelhauer, C., Shenker, S., Vöcking, B.: Randomized rumor spreading. In: Proceedings of the 41st Annual Symposium on Foundations of Computer Science (FOCS’00), pp. 565–574. IEEE Computer Society, Washington (2000). doi:10.1109/SFCS.2000.892324 10. Kempe, D., McSherry, F.: A decentralized algorithm for spectral analysis. In: STOC’04: Proceedings of the Thirty-Sixth Annual ACM Symposium on Theory of Computing, pp. 561–568. ACM, New York (2004). doi:10.1145/1007352.1007438 11. Kempe, D., Dobra, A., Gehrke, J.: Gossip-based computation of aggregate information. In: Proceedings of the 44th Annual IEEE Symposium on Foundations of Computer Science (FOCS’03), pp. 482–491. IEEE Computer Society, Los Alamitos (2003). doi:10.1109/SFCS. 2003.1238221 12. Kempe, D., Kleinberg, J., Demers, A.: Spatial gossip and resource location protocols. J. ACM 51(6), 943–967 (2004). doi:10.1145/1039488.1039491 13. Kermarrec, A.M., van Steen, M. (eds.): ACM SIGOPS Oper. Syst. Rev. 41 (2007). Special Issue on Gossip-Based Networking 14. Kimmel, A.J.: Rumors and Rumor Control: A Manager’s Guide to Understanding and Combatting Rumors. Lawrence Erlbaum Associates, Mahwah (2003) 15. Lohr, S.: Google and IBM join in ‘cloud computing’ research. The New York Times (2008) 16. Pittel, B.: On spreading a rumor. SIAM J. Appl. Math. 47(1), 213–223 (1987). doi:10.1137/ 0147013 17. van Renesse, R., Birman, K.P., Vogels, W.: Astrolabe: a robust and scalable technology for distributed system monitoring, management, and data mining. ACM Trans. Comput. Syst. 21(2), 164–206 (2003). doi:10.1145/762483.762485 18. Xiao, L., Boyd, S., Lall, S.: A scheme for robust distributed sensor fusion based on average consensus. In: IPSN’05: Proceedings of the 4th International Symposium on Information Processing in Sensor Networks, p. 9. IEEE Press, Piscataway (2005). doi:10.1109/ IPSN.2005.1440896

Chapter 8

Trust and Reputation for Successful Software Self-organisation Jean-Marc Seigneur and Pierpaolo Dondio

More and more software is reused, mixed and mingled. How to obtain a trustworthy software mix?

Objectives After reading this chapter, the reader will: • Understand what computational trust and reputation are; • Know how computational trust and reputation can be used for new software piece selection, even when trustors have different perspectives regarding trust definition and estimation methods; • Understand the practical issues of applying computational trust and reputation, especially in two major applications domains: open-source software quality assessment and open collaborative authoring software protection.

8.1 Background Context In the human world, trust can exist between two interacting entities, and it can be very useful when there is uncertainty involved with the interaction result. The requested entity uses the level of trust it has developed for the requesting entity as a J.-M. Seigneur () University of Geneva, Geneva, Switzerland e-mail: [email protected] P. Dondio Trinity College Dublin, Dublin, Ireland e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_8, © Springer-Verlag Berlin Heidelberg 2011

163

164

J.-M. Seigneur and P. Dondio

means to cope with uncertainty, for example to engage in actions that involve risks of a harmful outcome. There are numerous definitions of the human notion trust in a wide range of domains, with different approaches and methodologies such as sociology, psychology, economics and pedagogy. These definitions may even change when the application domain changes. However, it has been convincingly argued that these divergent trust definitions can fit together [30]. Romano’s recent definition [37] tries to encompass the previous work in all domains: . . . trust is a subjective assessment of another’s influence in terms of the extent of one’s perceptions about the quality and significance of another’s impact over one’s outcomes in a given situation, such that one’s expectation of, openness to, and inclination toward such influence provide a sense of control over the potential outcomes of the situation.

Interactions with uncertain result between entities also happen in the online world. For example, if one provides open Wi-Fi network access to passing by strangers, there is a possibility that they will use the Wi-Fi connection maliciously. Therefore, it can be useful to take decisions based on trust in the online world as well. However, the terms “trust”, “trusted”, “trustworthy” and the like, which appear in the traditional computer science literature, have rarely been based on comprehensive multidisciplinary trust models, and they often correspond to an implicit trust concept, a limited view of the faceted human notion of trust. Blaze et al. [4] coined the term decentralised trust management because their approach separates trust management from application: their PolicyMaker model introduced the fundamental concepts of policy, credential and trust relationship. Tversky et al. [46] argued that decentralised trust management still relies on an implicit notion of trust because it only describes . . . a way of exploiting established trust relationships for distributed security policy management without determining how these relationships are formed.

8.2 Theoretical Notions A computational model of trust based on social research was first proposed in [27]. Usually, a trustor is the user who considers trusting another user, a trustee. A third user may recommend the trustee to the trustor based on a recommendation previously received from elsewhere. Sometimes it may not be known who have made recommendations about a trustee. In such cases, where the identity of the recommenders is not exactly known, the resulting overall recommendation that can be taken into account by a trustor is referred to as the reputation of the trustee. In social research, there are three main types of trust: interpersonal trust, based on past interactions with the trustee; dispositional trust, provided by the trustor’s general disposition towards trust, independently of the trustee; and system trust, provided by external means such as insurance or laws [30, 35]. Trust concerning a particular situation or context is termed trust context. Each trust context is assigned an importance value in the range [0, 1] and utility value in the range [−1, 1]. The utility is the utility of the action to be allowed if the trustor chooses to trust the trustee and allow her to carry out the action. Any trust value is in the range [−1, 1). In addition, each

8 Trust and Reputation for Successful Software Self-organisation

165

virtual identity is assigned a general trust value, which is based on all the trust values with this virtual identity in all the trust contexts. Dispositional trust appears in the model as the basic trust value: it is the total trust values in all contexts in all virtual identities with whom the trustor has interacted so far. Risk is used in calculating a threshold for trusting decision making. Generally, a computed trust value of an entity can be seen as the digital representation of the trustworthiness or level of trust of that entity. The trustcomp online community (“Trustcomp”) [9] defines entiTrust (to emphasise that it cannot correspond exactly to real-world trust and avoid that users abstract it to their real-world expectation of trust) as a non-enforceable estimate of the entity’s future behaviour in a given context based on past evidence. The EU-funded (SECURE) [40] project represents an example of a trust engine that uses evidence to compute trust values in entities and corresponds to evidence-based trust management systems (see Fig. 8.1 below). Evidence encompasses outcome observations, recommendations and reputation. A trust metric consists of the different computations and communications which are carried out by the trustor (and his/her network) to compute a trust value in the trustee. Furthermore, Sabater and Sierro [38] remark that . . . direct experiences and witness information are the “traditional” information sources used by computational trust and reputation models.

Depending on the application domain, particular types of evidence may be more weighted in trust computation than others. For example, in a scenario where all potential recommenders are known by the user in advance, such as when friends in a social network have been manually specified by the user, recommendations can be weighted based on the user’s direct observations. When recommendations are processed, the social network can be reconstructed. Along this line, [19] studied the problem of propagating trust value in social networks and proposed an extension to the FOAF vocabulary1 together with suitable algorithms for propagating userestimated trust values instead of only computer-calculated ones. A more efficient algorithm for trust and recommendations propagation in peer-to-peer networks has later been proposed in [11]. Recent approaches base trust value computation on new types of evidence. For example, [50] have discovered an interesting correlation between similarity and trust between social network users, indicating that similarity may be evidence of trust. However, as is the case for trust values that are manually set, it is difficult to accurately estimate user similarity based on a specific and generally applicable set of pieces of evidence. Therefore, trust values are quite often computed from evidence of different types depending on the application domain. Although most work in this direction has so far focused on counting interaction outcomes, other types of evidence may also be found. This approach does not contradict the high-level view of the trust engine depicted in Fig. 8.1 because any type of evidence can be stored in the evidence store for future trust calculation. 1 FOAF

(Friend of a Friend) is a project devoted to linking people and information using the Web. The FOAF vocabulary is a machine-readable ontology describing persons, their activities and their relations to other people and objects, which is defined using RDF and OWL technologies.

166

J.-M. Seigneur and P. Dondio

Fig. 8.1 High-level view of a trust engine

8.2.1 Evidence-Based Trust Engine In Fig. 8.1, the decision-making component can be invoked when a trusting decision has to be made. The most common scenario considered in the relevant research works includes a requested entity having to decide the next action to be taken after a request made by another entity, the requesting entity. For this reason, a specific module termed Entity Recognition (ER) [40] is required to recognise any entities and their associated virtual identities, and to handle their requests. The decisionmaking component comprises two sub-components: • A trust module that can dynamically assess the trustworthiness of the requesting entity based on the trust evidence of any type stored in the evidence store. • A risk module that can dynamically evaluate the risk involved in the interaction, again based on the available evidence in the evidence store. A common decision-making policy is to select (or recommend to the user) an action that would maintain an appropriate cost/benefit ratio. The Evidence Manager component is responsible for gathering evidence, such as recommendations and comparisons between expected outcomes of the chosen actions and real outcomes. This evidence is used to update risk and trust evidence. Thus, trust and risk follow a managed life-cycle. Given that new types of trust evidence may still be found, it is challenging to go beyond this high-level view of a trust engine, that is to propose a generic implementation of a trust engine that would work for any application domain. The SECURE trust engine has been an attempt in this direction, but evidence such as similarity between users or manually defined user trust values without a clear count of evidence have not been considered yet. Other trust engines have been designed for specific application domains. For example, the TriQL.P Trust Architecture [3] aims at supporting users in their decision whether to trust or to distrust information found on the Semantic Web. The main types of evidence are the context, which includes who and when, and content, which is related to similarity, such as the inferred main topic of two Web pages. An additional example is Jøsang’s computational trust approach termed “subjective logic” [23]. That approach integrates the elements of ignorance and uncertainty, which cannot be reflected by mere probabilities and are part of the human aspect of trust. To represent imperfect knowledge, an opinion is modelled as a triplet whose elements are belief (b), disbelief (d) and uncertainty(u).

8 Trust and Reputation for Successful Software Self-organisation

167

The subjective logic provides more than ten operators to combine opinions. For example, the recommendation operator adjusts the recommended opinion based on the recommending trustworthiness (RT) parameter. Jøsang’s approach can be used in many applications since the trust context is open. However, it is still limited to few trust evidence types, such as direct observations of outcomes or recommendations. In addition, there is no risk component. Castelfranchi et al. [6] argue for a trust engine based on cognitive science where the main trust evidence types originate from the entity’s belief and goals structure instead of probabilistic quantitative views, economics or game theory. Furthermore, [22] claim to have built a generic open rating system, which means that anybody is allowed to rate anything in the system, including the ratings of contents. In addition, [24] highlighted the impact of trust in expert systems advice. Subsequently, [2] proposed an expert system containing knowledge about the factors taken into account to compute trust in certification authorities participating in public key infrastructures. In that case, the trust engine is merely mapped to a generic expert system where emphasis is on the knowledge of the particular application domain incorporated in the system by human experts.

8.2.2 Computational Trust Methodology Overview Based on the previous work surveyed above, the informal methodology that has generally been used to apply computational trust is as follows: 1. A model of trust from previous multi-disciplinary work on the human notion of trust is reused or refined to be turned into a computational model. 2. The main types of trust evidence relevant to the application domain are given more weight. 3. A computational version of the trust model is deployed, and evidence is collected and fed in the model using the calculated trust to handle uncertainty. The first of the above steps has successfully been applied in specific application domains such as peer-to-peer file sharing, and the basic types of trust evidence considered, such as positive and negative downloads count, provided satisfactory results for simple cases. As [38] underlined, . . . game theoretical models have given good results in simple scenarios but may be too limited for more complex scenarios.

Further to the above-mentioned informal methodology for computational trust, [21] have proposed a formal methodology for computational trust development which is limited to the business application domain. Olsson [32] has been working on a clearer engineering methodology for building systems that use trust as a component in decision making. Furthermore, [45] proposes a trust engineering methodology which provides design guidance on where and how developers can incorporate trust models into decentralised applications. However, that methodology is too focused on the peer-to-peer application domain, and it does not leave room for the multi-disciplinary aspect of trust models and their future extensions. To sum

168

J.-M. Seigneur and P. Dondio

up this section, there are still types of trust evidence and application domains that have not been considered in computational trust engines. For example, the relation between similarity and trust still requires further research [50].

8.2.3 The Problem of Trust Transferability and Its Solutions Self-organising software can be described as a network of autonomous components that cooperate with each other sharing information, and providing and using services. During the life and evolution of such systems, the problem of selecting trustworthy components among the ones available is crucial. End-users experience similar problems when needing to select a trustworthy open software of sufficiently high quality. As we mentioned above, computational trust offers a technique to support this decision-making process. Techniques such as recommendation systems and social networks could be effective even in open software domains. For example, the reputation of software authors, the feedback shared by users community or rating values interchanged between autonomic software components are all important factors for identifying reliable services. A basic requirement of such systems is the ability of its components to effectively communicate with others, receiving and sending evaluation messages that can be interpreted correctly. Unfortunately, in open systems it is not possible to postulate a common agreement about the representation of a rating, its semantic meaning, the cognitive and computational mechanisms behind a rating formation. It may be the case that agents (end-users or autonomic software components) may have different evaluation metric for software quality. Rating software can be a function of many factors such as programming language used, clarity of the comments, number of bugs reported, tests performed (and by who), number of versions, efficacy and so forth. Differences in evaluation method may invalidate ratings shared by the agents’ community. Therefore, the problem is to understand if the parties involved in the exchange of information can actually be considered compatible. In this section we analyse this central problem that affects all the trustbased systems based on the sharing of information. The analysis of this crucial and well-studied problem and its proposed solution is a useful and comprehensive way to describe in depth a generic trust solution. We propose a practical study case on an eBay-like scenario, leaving in the exercise section the study of a parallel situation within an open-software scenario. We will see how the need for a common evaluation language and a correct translation mechanism is the key for making effective use of such trust systems.

8.2.3.1 Trust is Subjective but Worth to be Shared Before asking how we can effectively transfer trust, a first question is whether trust has a degree of objectivity. Studies in social science seem to agree about the subjective nature of trust. In the classical definition by Gambetta [17], this is represented

8 Trust and Reputation for Successful Software Self-organisation

169

by the subjective probability that the trustor assigns to the trustee, that varies according to the trustee, the situation and the level of perceived risk. Any attempt at objective measurement can dangerously mislead agents into thinking that the value is transferable and be used by another trustor, which is not true for trust. In other words, trust is not transitive, which has also been formally shown in [7]. As Luhmann wrote [26]: Trust is not transferable to other objects or to other people who trust.

To say that one trusts another without further qualification of that statement is meaningless. But, on the contrary, the success and the diffusion of systems like Social Networks or Ratings Systems make the problem worth to be investigated. Therefore, the problem is to qualify correctly trust judgements and build a mechanism to translate values produced by two different systems to allow meaningful communications. Jøsang and Pope [23] investigated the transferability of trust by analysing under which formal condition trust may be considered transitive. Their conditional transitivity construct adds conditions for considering trust values, propagated by transitivity, more plausible. The concept is present also in Abdul-Rahman and Hailes distributed trust model [1]. The conditional transitivity (simplified) requires that: • • • •

A has direct knowledge of B, B has direct knowledge of C, A has knowledge of B as a recommender, A can use B’s trust value in C.

Using Jøsang words, a transitive trust path therefore stops . . . when there are no more outgoing referral trust, where referral trust is the trust in an agent as a recommender. These works clearly show that trust transferability is not a valid concept, but a plausible one that deserves to be investigated. Still, however, transferred trust values can be useful in the decision-making process if additional conditions are properly respected and appropriate semantic annotations are added.

8.2.3.2 A Generic Model of a Trust-Based System In Fig. 8.2 we provide a high-level view of a trust interoperability problem scenario where agents have to select trustworthy open software components. The scenario involves two trust-based reasoning systems with parts arranged at conceptual levels based on their primary functionality and purpose. At each level, possible differences between the respective parts of the two systems may cause lack of trust transferability. Each system is depicted as a multilayer pile with the following components: • Domain Perception. It includes all information an agent has acquired about its domain of interaction. It generally consists of a domain ontology and a facts database (or evidence DB). – Domain Ontology: agent’s representation and understanding of the domain it is interacting in. We model it as an ontology that describes terminology and

170

J.-M. Seigneur and P. Dondio

Fig. 8.2 Trust interoperability problem scenario

relationships related to the domain under analysis (the open software domain in the scenario under discussion). – Evidence DB: the facts collected based on the agent experience which are used to ground trust-based decisions. For example, a factual representation of a piece of evidence can have the form: The component 1 by developer XY Z crashed twice in the last two weeks. • Trust System. It contains the agent notion of trust (referred to as “trust ontology” or “trust model”), its computational model representation by a trust metric, a trust value, a decision making process and a satisfaction function. – Trust Value: the quantification of the trust judgement. – Trust Ontology (model of trust): the definition of trust in the application context. It defines the elements composing the notion of trust for the particular domain. We can represent a trust model as an ontology. An example description of such a trust model for open software component selection could be: trust is a combination of software component stability, programming techniques used and programmer experience or, in absence of past evidence, the overall recommendation received from other users. The trust metric specifies how these elements are converted into numerical values and how each of them concurs to the final aggregated trust value. – Trust Metric: the actual computation performed over the available inputs to generate a trust value. Each concept described in the trust ontology has a computational representation in the trust metric, which quantifies and aggregates all concepts in the trust ontology to produce an overall trust value.

8 Trust and Reputation for Successful Software Self-organisation

171

– Satisfaction Function: a mechanism used by trustor agents to evaluate the quality of interactions carried out with trustee agents. This mechanism can be modelled with a function such as the following: S : O → [0, 1].

(8.1)

Function S maps the set O of possible outcomes to [0, 1] assigning a numerical value representing the agent’s satisfaction level to each outcome. A satisfaction function models an essential concept in trust computation: the trustor should have a mechanism to understand if the trustee fulfilled its expectations. Using function S, a trustee’s trust value can be automatically updated according to the trustor’s level of satisfaction. For example, an agent may be satisfied if a piece of software code has clear comments and it does not crash in the first two months. Therefore, the agent can decide to assign to that piece of software a high trust rating. – Decision Making Process: this component, also referred to as “trust management”, describes how the agent exploits the computed trust values to support its decision-making process. Generally, based on the agent status and the situation context, a threshold T is defined, which specifies the minimum trust value required to start an interaction. The decision-making process can be further articulated and is commonly represented by a set of policies (see the SECURE policy language [5] for instance) or a set of (fuzzy) rules as is the case in the REGRET trust model proposed by Sabater [38].

8.2.3.3 Source of Trust Differences In any conceptual level of the trust model described above, differences between the respective parts of trust-based systems may reduce the transferability of trust. These differences include: • Trust value differences. Trust-based systems may have different trust value representations. For example, a system may represent trust values with real numbers in the range [0, 1], while another may classify trust at discrete levels represented with integer numbers, for example with numbers ranging from 0 to 4 for the case of five trust levels. An additional problem in this respect is the identification of the owner of each representation. This is generally not a trivial problem since different systems may use different names or labels for the same concept or entity respectively. • Trust metric differences. Even when two entities utilise the same trust representation, that is they have adopted the same trust terminology, they may use different trust computation methods. For example, an entity may base 90% of its calculated trust value on the number of bugs reported (for instance assigning it with a weight of 0.9 in a weighted average calculation), while another one may consider number of bugs as a trivial trust parameter.

172

J.-M. Seigneur and P. Dondio

• Trust model differences. Different systems may have different understandings of what trust is. For example, an agent may perceive trust as a prediction of software quality based on the quality of software previously produced by the same developers (a past experience-based prediction), while another agent may consider other factors as well, such as developer popularity and software stability and persistence. • Threshold differences. An agent A can be more optimistic, for example about utilising new software components, and it may therefore have a lower trust threshold than another agent B which may have stricter cooperation requirements. This implies that an exchanged trust value from agent A to agent B could be sufficient for A to start an interaction, even if that would not be the case for B, making thus hard to transfer trust between these two agents. • Satisfaction function differences. Agents can make different assessments of interactions between each other, for instance because of different goals and expectations. For example, in the open software component domain an agent may assess component quality based only on non-faulty operation, without taking clarity of code and number of comments into account. In contrast, another agent may consider confusing source code and lack of comments as significant negative factors affecting trust assessment. • Domain representation differences. Different entities may have different knowledge of the domain structure in which they interact. This can be expressed as differences in domain concept representation, in definition of domain element relationships and in representation of domain element aggregations, resulting thus in differences in evidence descriptions in the domain ontologies of system entities. • Evidence database differences. System entities maintain evidence databases which are dynamically evolvable based on entity interactions, knowledge and recommendations received. In a distributed scenario this is a common situation: each entity has a limited set of experiences and a partial vision of the world. In such cases entities judge trust differently based on different evidence sets and previous experiences. 8.2.3.4 Designing a Solution: Mechanisms to Exchange Trust Values Without loosing generality, we can assume that a solution to the trust interoperability problem would require entities to partially disclose their trust models to enable trust similarity comparisons between different systems. The degree of similarity produced as a result of such comparisons can be used to weight exchange of trust judgements. For example, if the compared trust-based systems differ significantly, the transmitted trust values can be ignored. Trust Value Translation To enable comparisons, mechanisms for translating trust values to a language understandable by the entities receiving trust recommendations are needed. Referring again to Fig. 8.1, a comparison between two systems can be performed at three levels: the trust value level, the trust model level and the evidence level:

8 Trust and Reputation for Successful Software Self-organisation

173

Trust Value Matching At the level of trust value representation, translations are performed to convert trust values to a common representation. An important aspect of such translations is the estimation of information loss during the trust value conversion process. Pinyol et al. [34] classify trust value representations in four common categories: 1. Boolean Representation, where trust values belong to the set {0, 1}. 2. Bounded Real Representation, typically a real number in the interval [0, 1]. 3. Discrete Representation, where the trust value belongs to a set containing discrete elements, such as {VeryBad, Bad, Neutral, Good, VeryGood}. 4. Probabilistic Representation, where trust values are modelled using discrete or continuous probability distributions instead of only crisp numerical values. Pinyol et al. further proposed a method to convert values between these representations which takes into account the uncertainty involved in the conversion outcomes. For example, to convert a boolean representation to a bounded real representation, a boolean value of “false” could correspond to any real number less than or equal to 0.5, while a boolean value of “true” could be represented as any value greater than 0.5. An entity using a real number representation would not be able to accurately treat the converted value, and this would represent the uncertainty involved in the conversion process. Pinyol et al. propose to estimate this uncertainty based on the entropy of trust values when considered as random variables. Generally, conversions to richer trust value representations incur higher levels of uncertainty, while no uncertainty is associated with the opposite conversions. Trust Model Matching El Messery [31] proposes to enhance trust values by declaring the expectations a trustee has. In this way if two agents share the same expectations, it is likely that they will judge situations in similar ways, and consequently trust values can be transferred. The proposed approach requires that trust systems of both agents be able to understand the terminology and the semantics of each trust model. A radical solution implies the matching of the two ontologies describing the trust model, a problem that has not been so far investigated in trust studies. Another possible solution is the definition of a generic ontology for trust representation which will be the starting point for application-specific trust ontologies. In trust systems research, the trust model matching problem has so far been studied using high-level concepts which are compared in taking trust-based decisions. The European project eRep [13] defines a set of terms to represent reputation concepts. The aim of that effort was to define a reputation ontology that all project partners can use as a consensual starting point. That ontology describes in detail all the elements participating in social evaluations, as well as the processes of transmitting them. It also defines the main decisions concerning reputation that agents may take. That ontology is based on the cognitive theory of reputation defined in the book by Conte and Paolucci [10]. Furthermore, Punyol and al. propose a common ontology as a possible common base for mapping different trust models. They describe a generic belief about an entity as a combination of SimpleBelief, a belief

174

J.-M. Seigneur and P. Dondio

that a holding agent acknowledges as true, and MetaBelief, a belief about others’ belief. Finally, an interesting problem concerning trust management in open software is the definition of a common ontology for describing software quality, which generally has the form of a common dictionary for expressing software component ratings. Elements of that ontology may include various software quality parameters such as software stability (represented by variables such as number of crashes, and numbers of modifications, patches and versions), age, clarity of comments, complexity of the code, reliability of the developing language used, portability, reputation of the authors, tests passed and reliability of the testers. Evidence Matching and Unsupervised Similarity Learning If agents act in the same or similar domains, it is likely to encounter similar situations. A degree of similarity can be deduced by simply matching evidence and their corresponding trust values. This could be performed in several ways, but the common idea is that interacting agents disclose sufficient information to compute a degree of compatibility based on common data or behaviours in specific situations. This class of solutions does not consider the elements of the trust model, but instead it focuses on the comparison of common data aiming to understand the similarity between different trust systems. Therefore, this approach limits the need for a common ontology. However, it still requires that interacting agents communicate using a commonly understandable language. A representative example of this solution approach is the collaborative filtering paradigm. Collaborative filtering assumes that ratings originating from agents with similar preferences are the most accurate. The main aspects of collaborative filtering mechanisms concern storing and exchange of user profile and recommendations. Upon entering the system, new users provide information about their profile and a number of trust recommendations. All new user input is stored in a central database. Subsequently, when a user requests for a recommendation, the system will compare the requesting user’s profile with those stored in the database, and it will suggest trust recommendations from similar users. The efficiency of such systems depends on the number of recommendations they receive, the number of users in the system and the level of detail of each user profile. Collaborative filtering systems are typically centralised, commonly based on a central database that stores user profiles and past user recommendations, and therefore they are not always a feasible solution. Alternative solutions usually extend the basic idea of collaborative filtering, namely searching for similarities among user related information, in a dynamic and distributed fashion. For example, Wang [48] proposes a recommendation system where idle agents “gossip” periodically with each other, exchanging information about their assessment of interactions they had previously held with other agents. Interaction assessment is commonly done using probability functions which reflect agent opinions about interaction participants. The result is a similarity factor in the range [0, 1] used to weight agent recommendations. Trust-Based System Comparisons In general, automatic comparison and matching of trust-based systems can be performed based on the satisfaction function S, the

8 Trust and Reputation for Successful Software Self-organisation

175

trust value T and the evidence database DB. In all cases the aim is to provide each agent with an estimate of either the trust metric T or the satisfaction function S of the agents with which it interacts. T and S are generally, but not always, correlated. In particular, the values of T are commonly calculated based on past values of S, except in rare cases, for instance when no information about relevant past interactions is available. Therefore, knowing T does not generally guarantee the expected S. For example, the fact that a person has been reliable in interactions with a specific person does not guarantee the same reliability in interactions with other persons. Similarly, a high satisfaction value S does not generally guarantee the quality of a recommended trust value, since in the computation of T the role of S is unknown. Trust comparison approaches are generally based on sharing trust values, on sharing past interaction evidence, on directly comparing satisfaction function S and trust metric T values using an agreed common domain ontology, and on approximating function S and metric T values based on stereotype scenarios. More specifically: • Sharing Trust Values: In this approach agents share trust values concerning other agents which they have calculated previously. The idea is that interacting agents check for common acquaintances and they use the respective trust values to compute a compatibility degree, which they subsequently use to weight trust-based decisions. In calculating compatibility, statistical indicators such as correlation can be used, and supplemental information such as number of accepted/rejected interactions with an agent can make the computation more plausible. Furthermore, it is assumed that agents have universally known IDs, although this hypothesis is not always valid. In addition, if agents have different trust value representations, a conversion may be performed, for example using the method described in [34]. That method predicts trust metric T without requiring any knowledge of the agent trust model, and for sufficient number of acquaintance agents, it can be a quite accurate indicator. However, it suffers from poor privacy and communication overload due to the high amount of information exchanged. • Sharing Interaction Evidence: This approach is also based on information sharing, but here sharing concerns assessments of single interactions instead of overall trust values. In other words, the goal is to predict function S instead of metric T . In this approach, communication overload is even higher, but no significant privacy issues are involved since the exchanged information concern scenario snapshots instead of agent private data. • Direct Comparison of Function S: When there is a common ontology describing system facts, each agent can easily map its function S over that common ontology and directly compare it with others. The simplest case is where the function S of all interacting agents has the same basic form (for example a linear combination of factors). An example of such an ontology is the recent evolution of the eBay feedback system (see Fig. 8.3), where four fixed criteria have been introduced to assess the validity of an item sold, representing a first common base for comparing feedback. By directly comparing the two functions, the agents compute an accurate degree of compatibility, without disclosing sensible information about other agents or personal experience, and with low communication overload unlike the previous two approaches.

176

J.-M. Seigneur and P. Dondio

• Predicting S and T using Stereotype Situations: When there is no common ontology, but the agents at least partially can understand each other, a solution can be build by using stereotype situations. Here we describe the prediction of function S, but the method can be applied to the prediction of trust metric T using trust values instead of values of satisfaction and stereotype agents instead of stereotype situations. The aim of this approach is to accurately predict the function S using the least number of messages. In the general case function S is assumed to be defined from a set of domain concepts, represented by multiple variables, to the value S1 : f (X1, X2, . . . , Xn),

(8.2)

S2 : f (Y 1, Y 2 . . . , Y m).

(8.3)

We assume that if agents use different value representations, they translate them using the technique described in [34]. Agents send stereotype situations, which they consider meaningful, to other agents and wait for them to evaluate the situation. For instance, an agent considering the low shipping time essential for being a good eBay seller may propose two situations where this factor varies drastically. Agents can propose situations where only one key-factor changes, in order to understand the importance of that specific factor, with the drawback of not understanding the mutual dependence of the factors in the formula. In general, agents need a strategy to generate the appropriate next situation after having received feedback from other agents. The strategy should indicate when the process should stop, that is when enough information has been collected to understand other agent model. In general, agents may employ an unsupervised learning system or adopt statistical tools such as regression and correlation to understand the reasoning model of other agents, performing an on-the-fly negotiation of their preferences. This solution appears to be a good trade-off over the previous ones: using stereotype situations, sensible data are not disclosed, and communication overload is relatively small, varying from the perfect situation of the evidence sharing approach to the case where a large number of messages will have to be exchanged to understand other agents. The required number of messages will generally depend on how close the interacting agent representations are and on the number of situations proposed that are relevant and fully understood.

8.2.3.5 Privacy Issues The disclosure of extra information instead of a trust value raises an issue of privacy. Interacting parties may not want to share information about their trust judgements or reasoning models, even in open software evaluation. The problem was described by Seigneur [42] as the trade-off between trust and security: There is an inherent conflict between trust and privacy: the more knowledge a first entity knows about a second entity, the more accurate should be the trustworthiness assessment; the more knowledge is known about this second entity, the less privacy is left to this entity. . . . A solution should allow the benefit of adjunct trust when entities interact without too much privacy loss.

8 Trust and Reputation for Successful Software Self-organisation

177

Fig. 8.3 eBay feedback system

Referring to the proposed solution, it becomes evident that a matching performed at trust model level discloses less information than the explicit sharing of evidence and the comparison of trust values of common acquaintances. Several systems have been implemented to add an extra-layer of security based on trusted computing and encryption key policies to guarantee the confidentiality of the shared information. A recent work done by Cissee and Albayrak [8] described an approach based on multiagent systems for preserving privacy in information systems, where all communication takes place in a trusted environment and a third entity is elected as referee, collecting all required encrypted information, performing the matching and sending the results to the entity involved.

8.2.3.6 Practical Case Study: Enhancing the eBay Feedback System In this section we briefly describe the eBay feedback system, whose evolution represents an excellent case-study for the above discussion. eBay provides a feedback system aiming to provide buyers with an additional decision support tool for selecting the most reliable sellers. Introduced in 1998, feedback systems evolved adding interesting features from the perspective of the above discussion. The eBay feedback system has a boolean representation for each feedback, where +1 is the value of a positive feedback and −1 of a negative one. The feedback score is simply computed by summarising all feedback messages received. In addition, a buyer may add a comment in natural language. eBay has introduced several new features to support a better decision-making process and add more semantic value to the original unqualified value. Figure 8.3 depicts the information displayed to a single user. Since March 2007, eBay introduced detailed ratings, where users rate the buyer according to four categories: item as described, communication, dispatch time, postage and packaging charges. These subcategories disclose more information to support a more accurate trust value semantic, resulting in a type of common ontology for representing buyers/sellers ratings. For each criterion, a discrete-level feedback is proposed with five possible levels ranging from very accurate to very inaccurate. Other interesting information used to define trust values includes the number of ratings of each category, the temporal distribution of the feedback, the number of feedback messages, the feedback score of the rating entities and the number of feedback removed. We see the eBay feedback system evolution as an answer to the problem of unqualified ratings, a way for defining a common ontology based on which different ratings can have a clearer and more objective meaning for the buyers. It is therefore a good template that many other recommendation-based systems, including the ones used for self-organising software, could refer to.

178

J.-M. Seigneur and P. Dondio

8.3 Applications This section applies the above approach to trust computation and transferability in two application domains: the selection of open-source software components, for example, to be used at the time of software self-organisation, and in development of open collaborative software such as Wikipedia.

8.3.1 Selection of Open-Source Software Pieces Based on Their Trustworthiness and Reputation A major asset of OSS projects derives from their collaborative and community values [44]. On one hand, it has been argued that the more OSS actors participate, the higher software quality is reached. In this respect Raymond argued that “. . . given enough eyeballs, all bugs are shallow . . . ” [36]. On the other hand, recent investigations have argued that few actors review OSS code and that this practise may not be enough to sustain high quality in OSS projects: “. . . the vast majority of ‘eyeballs’ apparently do not exist . . . ” [39]. As the collaborative and community values are deeply anchored in the OSS ecosystem [44], the root of the problem is related to a lack of enabling technical mechanisms. The OSS community is present and willing to contribute, but the tools needed to contribute are missing or too cumbersome to use. A few repositories aggregating OSS datasets and dashboards, such as Web portals presenting the information, have recently emerged, but there is a lack of automated exchange and interoperability: the first workshop trying to harmonise them was run in June 2006 [20]. Fortunately, with the advances in Web technologies, often termed Web 2.0, it becomes increasingly easier to build tools for networking the OSS community. The EU-funded EDOS project [41] has built a formal information model, termed the Project Management Interface (PMI), [33] which describes OSS artefacts, such as Actor and Platform or Maintainer, and OSS activities, for example SubmitPatch activity and SubmitTestReport activity. In EDOS, a number of tools have been deployed to gather and distribute information about OSS projects in a peer-topeer manner, by reusing Web 2.0 building blocks such as XML, RSS feeds and REST/Web services [15]. For example, a Quality Assessment (QA) Web portal, termed the EDOS dashboard, has been put in place to easily inform OSS actors of the quality of OSS projects. The quality of these projects is estimated based on information reported by other OSS actors who can deploy user-friendly tools on their platform when needed to easily evaluate specific OSS projects. Therefore, such actors and their platforms play an important role in the EDOS-powered community process with collaborative OSS quality assessment and improvement. However, relying on external information submitted from an open ecosystem, decentralised by nature and populated by possibly competing actors, introduces a number of trust issues, for example competing actors, including actors outside the OSS community, may try to submit false reports about the quality of projects of their competitors. Furthermore, cheating is facilitated when actors can enter and leave the system without

8 Trust and Reputation for Successful Software Self-organisation

179

any control from a centralised authentication service; and in the QA application domain, “trust in the accuracy of any test data . . . depends on . . . trust in the providence of the testers” [14]. It seems fair to assume that in small projects all OSS QA team actors may know face-to-face all the involved testers. However, this may not be possible in larger projects because the OSS ecosystem is an open environment where actors and their digital identities can come and go. Nevertheless, due to the need of more actors carrying out test tasks, as long as testers are trustworthy, even not personally known testers are welcome to contribute to the project quality improvement process. It is especially relevant in the EDOS project that relies on a peer-to-peer storage layer for OSS information. There is the need for security/trust metrics to select the most trustworthy peers for efficient and safe distribution of QA information. If we assume that there is a sufficient number of trustworthy testers, tests and platforms, the correlation of all test reports and defects should allow us to detect untrustworthy tests and platforms, for example tests that always fail despite that the software as such would work or compromised platforms sending false defect reports. More precisely, if one out of fifty platforms with the same hardware and software configuration reports that numerous different tests crashed the platform whereas all other platforms report that all these tests passed, there is a chance that the platform itself is buggy, and therefore it should be considered as untrustworthy. Another reason for deviant reports may be that testers are untrustworthy, for example because they are involved in competing projects or lack necessary testing skills. Recent work on test quality has emphasised the importance of tester quality: “your trust in the accuracy of any test data will depend on your trust in the providence of the testers/quality of the testers” [14]. Although that related work is a first step towards taking the importance of tester quality into account, it uses only fixed, manually configured, subjective trust levels to characterise tester quality. An example is the average testing quality which is estimated by the QA manager. The EDOS QA information model and approach allow us to easily collect and distribute tester quality and trust objective evidence. There are different aspects that can be considered regarding tester quality and trustworthiness. To start with, there are differences between end-user actors that are only willing to run specific tests on their platforms and tester actors who have designed tests in the first place. Secondly, a tester should be rewarded when a test carried out before the software release detects bugs, especially critical bugs. Furthermore, if the released software contains many critical bugs, the testing quality should be questioned. Since there are chances that testers make mistakes on purpose due to their participation in competing projects, we use the term tester “trustworthiness” instead of tester “quality”. Figure 8.4 depicts our initial decomposition of the testing trust contexts of testers. We assume that each rectangular trust contexts in Fig. 8.4 is associated with a counter counting the number of times a tester has achieved a positive outcome and a negative outcome in this trust context. We use the term “test report trustworthiness” to refer to the number of times a tester has reported the same test outcome with the majority of testers with similar platform configuration. The context test contribution captures the number

180

J.-M. Seigneur and P. Dondio

Fig. 8.4 Example of the trust contexts of a tester

Fig. 8.5 Example update of tester trust

of times the tester has contributed and spent time for the OSS community on QA tasks. Figure 8.5 depicts an example of an activity diagram to update these counters. Depending on the criticality of the bug found by an end-user actor when using the software, we can modulate the trustworthiness counters of test quality based on bug criticality. For example, given: • a tester u, • N independent tests carried out by u, • crit, the criticality of a bug found to be linked to a specific test that ranges within [0, 1], an estimation of test quality trustworthiness of tester u, denoted by tqt(u), can have the form tqt(u) =

Sum of (1 − crit) for all tests carried out by u . N

(8.4)

8 Trust and Reputation for Successful Software Self-organisation

181

8.3.2 Trust in Open Collaborative Authoring Applications such as Wikipedia The emerging self-organising nature of the Web 2.0 shows similarities with the production process of an open-software component. With the proliferation of new tools such as wikis and blogs that simplify and democratise publications, the Internet appears today as the collaborative work result of an open community. The Wikipedia project, under analysis in this section, represents one of the most successful and discussed examples of this trend, with more than 1.2 million registered users (English version) and more than 2 million articles [49]. In a smaller scale, a self-organising open software is also the product resulting from cooperation of a community of developers, where the code added by each member represents a contribution. Therefore, it is not absurd to presume that the two domains may partially share problems and solutions. In this section, we analyse how computational trust techniques have been applied to wiki-based applications to provide users with a tool for supporting more effective decisions. We describe an autonomic trust model that was originally designed for the Wikipedia project and has lately been generalised for collaborative self-organising information systems. We envisage the possible application of such a computation in open collaborative software.

8.3.2.1 Wikipedia Internal Quality Mechanism The Wikipedia project represents the most successful and discussed self-organising information system. It is regarded as one of the best examples of collective knowledge, a concept that is often lauded as the next step towards truth in on line media. The problem of article trustworthiness is central to Wikipedia future development as a recent extraordinary case has brought to attention. Nevertheless, the growing interest and utilisation of such types of source remains unquestionable. In 2006, a study by the magazine Nature showed how, for a subset of scientific-related articles, Wikipedia was actually of comparable quality with Encyclopedia Britannica, with an average of 4 inaccuracies in Wikipedia against 3 in Britannica [18]. To address this concern, Wikipedia has defined an internal rating mechanism to classify article quality. The mechanism is an articulated centralised recommendation system, where any decision is taken based on collected ratings and opinions of Wikipedia users. Using this system, each Wikipedia article may receive a quality classification or a review. In particular, there are two highest quality article classification levels, The first one concerns is the “featured article” status, which means that an article has been identified as one of the best articles produced by the Wikipedia community, and it is particularly well written and complete. Only 0.1% of articles are featured articles. The second one is the “good article” status: the article contains excellent content, but it is unlikely to become featured in its current state. For example, it may be too short, or involving a too specific topic, or a topic on which there is a shortage of information.

182

J.-M. Seigneur and P. Dondio

The Wikipedia internal mechanism is a robust tool that proves how an efficient recommendation-based approach can increase the quality of the information shared. Anyway, it is not the ultimate tool, and it is not free from drawbacks. The rating system can be slow to react in comparison to the fast-changing nature of Wikipedia, making its suggestions out-of-date. New strategies could be coupled to this approach. An interesting Wikipedia feature makes it possible to define complementary strategies to compute articles trustworthiness. Wikipedia has been designed so that it keeps a completely transparent database of all the past contributions. The history of each page is accessible, providing information regarding authors and differences with previous versions. The accessibility of this information provides a valuable base that, if properly processed, can ground a solid decision making process, as shown in the next section.

8.3.2.2 Alternative Approach: Computing Trust Using Domain Elements, from Heuristics to Generic Trust Patterns Wikipedia has been the target of computational trust experiments only recently. A common factor of such efforts was the idea of defining strategies for trust value computation further to the recommendation approach. The common idea was to extract evidence of trust presence directly from application elements. This presumes that the majority of trust information is represented, even implicitly, in elements and dynamics of the particular application. For trust computation to be effective, the identification of these elements and their trust meaning should be plausible and justified in the particular application context. This approach to trust computation is a subclass of evidence-based trust where the evidence used are internal application elements or dynamics. Such computation results are application-contained and thus non-invasive, not requiring any infrastructure added to the application for assessing trust, unlike recommendation systems. A first limited experimentation was performed by McGuiness [29]. To assess the trustworthiness of a Wikipedia article, the author applied heuristics based on a version of the famous PageRank citation-based algorithm. Many authors, notably Massa [28], identified in Google PageRank all the elements of an applicationcontained trust metric. In that specific case, the application is the whole Web, seen as an interconnection of mutually linked web sites. PageRank considers the outgoing and ingoing links of a web page as trust evidence. In Wikipedia, the authors considered the relative number of times the name of an article appears as a link in the whole encyclopedia. The formula proposed is the following: Trust doc(d) =

occurrences[[d]] , (occurrences([[d]]) + occurrences(d))

(8.5)

where d is an occurrence of an article name as a plain text, and [[d]] an occurrence of the article name as a link. In other words, in that approach the fact of being a link was selected as evidence for the trustworthiness of an article. However, its applicability in that context may be severally argued: an expert Wikipedia user may

8 Trust and Reputation for Successful Software Self-organisation

183

argue that in Wikipedia there are automatic procedures that link articles, or that an author may link articles independently from their quality, for example for the sake of completeness. This example, similar to many others, shows several concerns regarding how to perform a selection of evidence directly from domain elements: the selected heuristics cannot be directly applied without critical analysis of their plausibility and trust-related meaning in the application context. The McGuiness experiment does not go beyond definition of ad hoc heuristics, lacking objectivity and systematicity. Its lack of plausibility for expert-users decreases drastically the meaning of trust computation. In the next section we will describe a trust model developed to address these issues.

8.3.2.3 DANTE: A Trust Model for Collaborative Self-organising Information Systems An open and collaborative self-organising system follows an evolution from start-up to a mature status, where the system auto-corrects its errors and changes in response to external stimuli in order to survive and continue to provide its service. During this evolution, some internal dynamics and elements emerging from system interactions reveal information about the health status of the system. This status information is an indicator of the ability of the system to cooperate as expected, its reliability and the probability for it to fulfill other expectations. Therefore the correct identification and quantification of this information is a valuable and relevant source about system’s trustworthiness. This is the idea that grounds the trust model for collaborative selforganising information systems described in this section. The trust model, termed DANTE (Domain Analysis and Trust Extraction) [12], defines a set of trust factors which are indicators of the health status of the system. Their presence is evidence of system reliability and trustworthiness, while their absence is a warning for the agent taking a trust-based decision. That trust model was originally designed for an experiment in the context of the Wikipedia project, but it was later generalised by the authors to cover any collaborative self-organising information system emerging from the collaboration of different authors. Anyway, the model is not regarded as a completely domain-independent solution. The DANTE model consists of a set of guidelines that can be used to design a trust-based solution, where domain-specific expertise plays a limited supportive role. Each trust factor can be described as a generic trust pattern which elements of the application can match. The selection of an element is therefore justified by the corresponded trust factor. Trust factors are grounded on social science theory of trust which guarantees that the computation performed has a meaning for trust, while the fact that the mechanisms are generic provides a valid basis for keeping the computation less domain dependent. Trust factors should therefore limit the space for unsystematic and unjustified heuristics. Among the benefits of this approach lies the fact that it does not require any additional explicit data to be added to the application and, by performing computations over system elements, it is a feasible non-invasive solution. Moreover, that approach can be used to enhance the decision-making process of an autonomic system.

184

J.-M. Seigneur and P. Dondio

It is essential to specify the meaning of trust factors. Trust factors are not intended as definitive evidence of system trustworthiness, and their absence is not intended as a definitive proof of system unreliability. On the contrary, trust factors are considered as plausible trust indications. The core of the computation of the proposed model aims to quantify the plausibility of the selected system elements and the uncertainty of the respective conclusions. Each factor used requires a number of critical questions assessing its plausibility when applied in a specific context. These critical questions require an investigation of the specific situation context to be answered, and they are not dependent on the overall trust domain.

8.3.2.4 Trust Factors for a Self-organising System The trust factors identified are divided in the following categories: pluralism, temporal factors, stability, activity degree, similarity and categorisation. Each category interacts with the others to strengthen or weaken their conclusions. In the rest of this section we further describe each trust factor, while in the next section we show how they were applied in the context of the Wikipedia project. • Pluralism: A collaborative self-organising system is characterised by contributions of many actors whose input affects system dynamics. According to the fundamental trust pluralism principle, in a purely open collaborative environment there should be an appropriate balance between actor contributions. In particular, the cases where the system depends on contributions from only too few or too many actors should be avoided. The former case has the disadvantage that the resulting system can be highly biased, while the latter can lead to a fragmented system with increased complexity and low coherence and consistency among its parts. This is clearly demonstrated in the collaborative document authoring approach that is followed in Wikipedia. In the Wikipedia approach the authoring status of an article can range from having been edited by only one or a very limited number of contributors leading to a lack of pluralism to being the collaborative authoring result of a large number of authors, each providing a small contribution, resulting in highly fragmented information. Fragmentation may lead to inconsistent editing or even conflicting ideas, while lack of pluralism may lead to biased information. Therefore, an article is considered as highly trustworthy only if it has been edited by a sufficient number of authors who have each contributed significantly resulting in high article pluralism. Of course, it may be the case that an article written by a single author be of exceptionally high quality, but this case is less plausible and more risky because if that single author is fallacious, the article will not be trustworthy. On the other hand, if an article has sufficient pluralism, it is more plausible that a solid checking mechanism has been used. For pluralism to be properly assessed, contributions should come from recognisable authors. More specifically, it is only required to distinguish between different contributors and not necessarily characterise them based on their identity or trustworthiness. Furthermore, the pluralism principle is more plausible when

8 Trust and Reputation for Successful Software Self-organisation

185

the total number n of contributors is relatively high, the contributors can be considered as independent or not explicitly related, and the number m of contributors with a non-negligible contribution is significant. Finally, contributions should occur with a relatively high frequency to enable for sufficient contributions to be made in any period of observation. • Stability: Self-organising systems take auto-corrective actions to improve their offered services when they perceive themselves as not exhibiting the behaviour intended for their surrounding environment. During such state transitions, system components may reach states where their integration and functionality had not previously adequately evaluated. On the other hand, self-organising systems that have reached a high degree of maturity have less reasons to radically change their core behaviour apart from perhaps adding some new functionalities. Therefore, they are considered as stable. Stability implies that system components are well established and reliable and that their evolution has reached a mature stage. In contrast, instability indicates the existence of components that still need to evolve to correct their behaviour and abilities. The role of stability in trust evaluation has been widely examined in the literature. Frewer and Miles [16] show that temporal stability is directly linked to perceived trustworthiness. Different bodies (public and private) were asked to release the same information concerning a number of food hazards. The sample of people involved in the test tended to consider the provided information more trustworthy if they had been released by a body with higher temporal stability. For example, hospitals had a higher trustworthiness characterisation than governmental bodies. The Stanford Persuasive Lab Guidelines [43] attributes to the permanence and stability of the information on the Web one of the main five sources of credibility and trust. In the context of Wikipedia, an article stability is considered as evidence that the article contains information with a high degree of completeness and acceptance, without being the source of contentions and editing revisions. • Activity: This trust factor concerns the use of system activity as an indicator of system trustworthiness. In this view, inactive systems should be trusted only after detailed further investigations. In contrast, the trustworthiness of active systems should be considered as plausible, and its degree should be further analysed using appropriate plausibility tests. Examples of parameters that should be included in such plausibility tests include the quality of system activities, the number of competitors providing similar system services and the number of requests for each provided system service. In Wikipedia, the numbers of edits and visits received were considered as article activity indicators. In particular, it was considered that stable articles with high numbers of edits and visits received could be characterised as trustworthy. This assertion can be further strengthened by considering article importance. In Wikipedia, the number of links pointing to an article were used as an indicator of its importance, and that value was further used to normalise the expected activity of that article. For example, based on the number of existing links, an article such as “Garda Lake” would be expected to demonstrate less activity than an article such as “USA”.

186

J.-M. Seigneur and P. Dondio

• Temporal Factors: Temporal factors [25] are deduced exclusively by processing the time distribution of the activity of the actors/components that participate in a self-organising system. Therefore, a number of them partially overlaps with the activity and stability factors described above. Temporal factors are defined as follows: – Regularity–Persistency. This factor examines whether system activity is persistent over time. It is considered to have a positive value if for a given time interval p, at least one interaction takes place within the system for each interval p over the selected observation period. – Frequency. The factor checks the average period between two interactions and the variance of this interval. – Presence. This factor examines the amount of time the system has been in operation, that is the difference between the first and last system activities. Temporal factors consider the constant system operation as evidence of the system ability to fulfil user expectations. For example, if a system is relatively new or it has experienced long periods of inactivity, frequent service interruptions, high service provision variance and low interaction frequency, then its trustworthiness would be characterised as low, and it would require further investigation. Temporal factors do not examine the overall system stability, but only the temporal stability of system activities. This means that the system may change its properties and functionality as needed, but if the system activity remains stable over time, temporal factors contribute positively to system trustworthiness. In Wikipedia, the activity whose temporal factors are taken into account in trustworthiness estimation is the action ‘edit’ carried out by article authors. • Similarity and Categorisation: This trust factor has a statistical nature with clear human related meaning. The assumption is that entities showing properties significantly different from the average of their category, that is they are ‘outliers’, should not be granted trust without further investigation. For example, if a system does not comply to standards, this would be an indicator that the system was produced by non-experts in an unsystematic manner and without proper testing and taking state-of-the-art into account. Tversky [47] have extensively studied similarity and categorisation as cognitive mechanisms used to make judgements under uncertainty. Furthermore, in the context of computational trust similarity has been investigated by several authors, such as Ziegler and Goldbeck [50], while Castelfranci and Falcone [6] in their cognitive trust model consider categorisation as a first form of trust-based reasoning. The plausibility of the trust factor increases if the category set has relatively large cardinality and low variance. Moreover, plausibility further increases if the categories used are well defined, the system entities have comparable lifetime in the environment and the population is stable, meaning that it is not in the process of evolving. In Wikipedia, this approach has been applied based on the Wikipedia article categorisation mechanism. Articles of comparable topics and importance are expected to comply to the standards emerged by the collaborative system dynamics. If an article is not compliant to such a standard, this implies that the article would

8 Trust and Reputation for Successful Software Self-organisation

187

Table 8.1 Trust factors in Wikipedia context Trust factors

Comments

Pluralism

Number of reverted edits A reverted editing is a roll back to a previous version of an article, rejecting any intermediate modifications

Activity

Number of links to the article

Activity

Number of visits

Activity

Number of edits

Temporal factors: regularity

Average and standard deviation of the time intervals between consecutive edits

Temporal factors: persistency

Number of intervals in which there has been at least one interaction

Temporal factors: presence

% Time between last and first edit

Stability

Percentage of article edits from time t to present time

Stability

Percentage of text differences between article versions at time t and current article versions

Pluralism

Average number of article edits per user

Pluralism

Standard deviation of article edits per user

Pluralism

% of edits produced by the n most active article users

Pluralism

% of edits produced by users with more than n edits for a given article

Pluralism

Number of discussions (talk edit)

Categorisation

Computed by relying on Wikipedia categorisation of articles. Among the elements considered to evaluate similarity: length of the text, images, variance of sections, references

require extra editing work, or that it was not edited by expert authors aware of Wikipedia emerging trends and guidelines. Examples of trust factor computation in Wikipedia context are depicted in Table 8.1. To identify the distribution properties of the calculated variables, statistical quantities such as average and standard deviation were used.

8.3.2.5 The Wikipedia Experiment In this section we present the results of computing the above-described trust factors in Wikipedia. The experiment was conducted on 7718 Wikipedia articles. These articles included all 846 featured articles, namely special articles considered by

188

J.-M. Seigneur and P. Dondio

Fig. 8.6 Trust calculation results in Wikipedia context

Wikipedia as having the best quality, and the most visited pages having at least 25 edits. These articles represent the 65% of the editing activity of Wikipedia and the high majority of its access, and thus they can be considered as a significant set. The results are summarised in Fig. 8.6. The graph represents the article distribution based on their trust values. We have isolated the featured articles (grey line) from standard articles (black line), and the hypothesis was that featured articles should show higher trust values than standard articles. The results obtained clearly showed a trust value difference between featured and standard articles, which had trust values of around 45–50% and 75% respectively. Furthermore, 77.8% of featured articles is distributed in the region having trust values greater than 70%. In addition, 42.3% of standard articles are distributed in the region having trust values less than 50%, which does not include any featured articles. Only 23 standard articles are in the region >85%, where there are 93 featured ones. The experiment, covering articles from different categories, was conducted on an absolute scale, and it shows a minimal imprecision if compared with a previous experiment conducted on a set of 200 articles, all taken from the same category “nations” [12], where we could rely on relative comparisons of similar articles. This shows that the method has a general validity.

8.4 Conclusion There are different computational trust models and metrics. Instead of forcing all the trustors to change their trust model and metric to one yet-to-be-found-and-adopted trust model, it seems more feasible to provide means for trustors to obtain an understanding of trust models used by others and to translate the trust values and received recommendations to their own trust model. Finally, another use of trust models in self-organising software is to take trust into account when characterising software testers and developers.

8.5 Problems–Exercises 8.1 How would you design a system that would automatically select the most trustworthy open-source software building blocks needed for a particular application?

8 Trust and Reputation for Successful Software Self-organisation

189

8.2 Consider an eBay-like decentralised database. Agents have different trust models applied to the available system data, but they have the same domain representation, that is they understand each other regarding eBay related issues. Since the environment is decentralised, agents have knowledge only about their past interactions. In the above context, provide examples of all possible differences between two agent systems. Consider all trust system layers. 8.3 For the scenario context described in Problem 8.2, consider that each buyer utilises the same domain representation described by the eBay database structure. Moreover, each agent has a different trust model represented by an ontology that is unknown to others. The outcome of an interaction is based on a 4-tuple f1 , f2 , f3 , f4 representing the four eBay criteria: • • • •

f1 : Item as described; f2 : Communication; f3 : Dispatch time; f4 : Posting and packaging charges.

Furthermore, each agent has the following capabilities: • A trust value representation by a real number in the range [0, 1]. • A trust metric to produce trust values which has the form T = a·A + (a − 1)·B, where A is the trust value derived from past interactions that are saved in the agent database, and B is the value collected from received recommendations. In absence of interactions, the trust value A is set at the half of the scale (equal to 0.5). After each interaction the value of A is updated using the function S to evaluate the quality of the interaction. The value a is a coefficient that determines the relative importance of received recommendations versus previous agent interactions. If a = 0, then only recommendations are taken into account in trust calculation. The value of A after an interaction is given from the formula A = (b·A + (1 − b)·S), where b represents the importance of past agent history in comparison to the last interaction. The value b is a coefficient that describes how the trust value changes after a new interaction. If b is close to 1, the impact of new interactions over the stored trust value is negligible, and if b is close to 0, newest interactions strongly affect calculation of new trust values. • A set of evidence derived from agent past interactions. Evidence can be stored as tuples of the form buyerID, f1 , f2 , f3 , f4 , date_of _transaction, price. • A database with the trust values for each buyer stored as tuples of the form buyerID, trust_value, number_of _interactions. • A trust threshold T used by agents to drive purchasing decisions based on the respective buyer trust values. • A satisfaction function S, defined as S(item) = c1 ·f1 + c2 ·f2 + c3 ·f3 + c4 ·f4 , where S is a value in the range [0, 1]. The values a, b, cn and T are random for each agent to model the differences between trust systems of different agents. Consider a group of N sellers and M buyers. To decide whether to buy an item from a particular seller, an agent A can use the respective trust value stored in its

190

J.-M. Seigneur and P. Dondio

internal database if such a value had been previously stored, or ask an agent B for a recommendation. After a purchasing interaction has taken place, the satisfaction function S is used to assess the quality of the purchasing outcome and characterise the trust quality of the respective buyer. (a) How can a buyer evaluate if it would be preferable to act in isolation or to share the trust values calculated from previous interactions? (b) Design an automatic strategy to translate trust values from one trust system to another so that the overall quality of interactions is improved. How can this strategy be evaluated? (c) Can you think of a similar scenario in a self-organising software system? Which trust metric and which satisfaction function S would you suggest to assess software quality? Which are the possible differences between these two metrics? Can you define a common ontology to describe both software quality and trustworthiness? 8.4 Identify critical questions that can be asked regarding the stability trust factor. Do you consider open software stability as plausible evidence of its trustworthiness? 8.5 What is the meaning of the trust factor “pluralism” in a self-organising opensoftware scenario?

Key Points • Computational trust and reputation can assist in improving safety and quality of a self-organising software ecosystem; • There is a need to consider trustworthiness of testers and developers as well as the approaches followed by trustors to model and estimate trust and reputation.

8.6 Further Reading Reputation Management Services. A book chapter in the “Computer and Information Security Handbook”, edited by John Vacca, which goes into details of reputation management services. The other chapters of that book will remind the reader of other security aspects for successful software self-organisation (J.-M. Seigneur, 2009, Elsevier, ISBN:9780131426436.) Collaborative Computer Security and Trust Management. A book that covers the social engineering aspects of collaborative software (J.-M. Seigneur and A. Slagell, 2009, Information Science Publishing, ISBN:978-1605664156.)

8 Trust and Reputation for Successful Software Self-organisation

191

References 1. Abdul-Rahman, A., Hailes, S.: Supporting trust in virtual communities. In: HICSS (2000) 2. Ball, E., Chadwick, D.W., Basden, A.: The implementation of a system for evaluating trust in a PKI environment. In: Proceedings of Trust in the Network Economy, Evolaris (2003) 3. Bizer, C., Cyganiak, R., Gauss, T., Maresh, O.: The TriQL.P browser: filtering information using context, content and rating-based trust policies. In: Proceedings of the Semantic Web and Policy Workshop (2005) 4. Blaze, M., Feigenbaum, J., Lacy, J.: Decentralized trust management. In: Proceedings of the 17th IEEE Symposium on Security and Privacy (1996) 5. Bryce, C., Couderc, P., Seigneur, J.M., Cahill, V.: Implementation of the secure trust engine. In: iTrust, pp. 397–401 (2005) 6. Castelfranchi, C., Falcone, R., Peluzzo, G.: Trust in information sources as a source for trust: a fuzzy approach. In: Proceedings of the 1st Conference on Autonomous Agent and MultiAgent Systems (AAMAS). ACM, New York (2003) 7. Christianson, B., Harbison, W.S.: Why isn’t trust transitive. In: Security Protocols Workshop, pp. 171–176 (1996) 8. Cissee, R., Albayak, S.: An agent-based approach for privacy-preserving recommender systems. In: Proceedings of the Sixth Conference on Autonomous Agent and Multi-Agent Systems (AAMAS). ACM, New York (2007) 9. Computational trust community. http://www.trustcomp.org (2004) 10. Conte, R., Paolucci, M.: Reputation in Artificial Societies: Social Beliefs for Social Order. Kluwer Academic, Norwell (2002) 11. Despotovic, Z., Aberer, K.: Maximum likelihood estimation of peers performance in P2P networks. In: Proceedings of the Second Workshop on the Economics of Peer-to-Peer Systems (2004) 12. Dondio, P., Barrett, S., Weber, S., Seigneur, J.M.: Extracting trust from domain analysis: a case study on the Wikipedia project. In: ATC, pp. 362–373 (2006) 13. erep:social knowledge for e-governance. http://megatron.iiia.csic.es/eRep (2006) 14. Fenton, N., Neil, M.: Combining evidence in risk analysis using Bayesian networks. Tech. Rep., Agena (2004) 15. Fielding, R.T.: Architectural styles and the design of network-based software architectures. Ph.D. Dissertation, University of California, Irvine (2000) 16. Frewer, L., Miles, S.: Temporal stability of the psychological determinants of trust: implications for communication about food risks. Health, Risk Soc. 5, 259–271 (2003) 17. Gambetta, D.: Can we trust. In: Gambetta, D. (ed.) Trust: Making and Breaking Cooperative Relations, pp. 213–237 (2000). Published Online, Chap. 13. http://www.sociology.ox.ac.uk/ papers/gambetta213-237.pdf 18. Giles, J.: Special report: Internet encyclopedias go head to head. Nature 438, 900–901 (2005). doi:10.1038/438900a 19. Golbeck, J., Parsia, B.: Trusting claims from trusted sources: trust network based filtering of aggregated claims. In: Proceedings of the 3rd International Semantic Web Conference. LNCS, vol. 3298. Springer, Berlin (2004). http://www.mindswap.org/papers/Jen-ISWC04.pdf 20. Gonzalez-Barahona, J.M., Conklin, M., Robles, G.: Public data about software development. In: Proceedings of the International Conference on Open Source Software (2006) 21. Gordijn, J., Tan, Y.H.: A design methodology for trust and value exchanges in business models. In: Proceedings of the 16th Bled Electronic Commerce Conference (2003) 22. Guha, R.: Open rating systems. Techn. Rep., Stanford University (2004) 23. Jøsang, A.: The right type of trust for distributed systems. In: Proceedings of the New Security Paradigms Workshop. ACM, New York (1996). http://citeseer.nj.nec.com/47043.html 24. Lerch, J., Prietula, M., Kulik, C.: The Turing effect: the nature of trust in expert systems advice. In: Expertise in Context. AAAI, MIT Press, Cambridge (1997) 25. Longo, L., Dondio, P., Barrett, S.: Temporal factors to evaluate trustworthiness of virtual identities. In: Proceedings of SECOVAL (2007), Third International Workshop on the Value of Security through Collaboration (2007)

192

J.-M. Seigneur and P. Dondio

26. Luhmann, N.: Familiarity, confidence, trust: problems and alternatives. In: Gambetta, D. (ed.) Trust: Making and Breaking Cooperative Relations, pp. 213–237 (2000). Chap. 13. Published Online http://www.sociology.ox.ac.uk/papers/gambetta213-237.pdf 27. Marsh, S.: Formalising trust as a computational concept. Ph.D. dissertation, University of Stirling, Department of Mathematics and Computer Science (1994). http://citeseer.nj.nec.com/ marsh94formalising.html 28. Massa, P., Avesani, P.: Controversial users demand local trust metrics: an experimental study on Epinions.com community. In: AAAI, pp. 121–126 (2005) 29. McGuiness, D., Zeng, H., da Silva, P.P., Ding, L., Narayanan, D., Bhaowal, M.: Investigations into trust for collaborative information repositories: a Wikipedia case study. In: Proceedings of the WWW2006 Workshop on the Models of Trust for the Web (MTW’06). ACM, New York (2006) 30. McKnight, D.H., Chervany, N.L.: What is trust? A conceptual analysis and an interdisciplinary model. In: Proceedings of the Americas Conference on Information Systems (2000) 31. Messery, A.E.: Expectations enhanced trust value. In: Ninth Workshop on Trust in Agent Societies, pp. 70–77 (2007) 32. Olsson, O.: Privacy protection and trust models. Ercim News (2002) 33. Pawlak, M.: Project management interface (pmi). Techn. Rep., University of Geneva (2005) 34. Pinyol, I., Jordi, S.M., Guifre, C.: How to talk about reputation using a common ontology: from definition to implementation. In: Ninth Workshop on Trust in Agent Societies, pp. 90– 102 (2007) 35. Rahman, A.F.: A framework for decentralised trust reasoning. Ph.D. dissertation, University of London (2005) 36. Raymond, E.S.: The Cathedral and the Bazar (1997) 37. Romano, D.M.: The nature of trust: conceptual and operational clarification. Ph.D. dissertation, Louisiana State University (2004) 38. Sabater, J., Sierra, C.: Reputation and social network analysis in multi-agent systems. In: First International Conference on Autonomous Agents and Multiagent systems (AAMAS-02), pp. 475–482 (2002) 39. Schach, S.R.: Colloquium presentation (2004) 40. Seigneur, J.M.: Trust, security and privacy in global computing. Ph.D. dissertation, Trinity College Dublin (2005) 41. Seigneur, J.M.: Security evaluation of free/open source software powered by a peer-to-peer ecosystem. In: Proceedings of the Workshop on Evaluation Frameworks for Open Source Software, OSS International Conference (2006) 42. Seigneur, J.M., Jensen, C.D.: Trading privacy for trust. In: iTrust, pp. 93–107 (2004) 43. Stanford guidelines. http://credibility.stanford.edu/guidelines 44. Stewart, K.J., Gosain, S.: An exploratory study of ideology and trust in open source development groups. In: Proceedings of the International Conference on Information Systems (2001) 45. Suryanarayana, G., Erenkrantz, J.R., Taylor, R.: An architectural approach for decentralised trust management. IEEE Internet Comput. (2005) 46. Terzis, S., Wagealla, W., English, C., McGettrick, A., Nixon, P.: The secure collaboration model. Techn. Rep., Trinity College Dublin (2004). http://secure.dsg.cs.tcd.ie 47. Tversky, A., Kahneman, D.: Judgment under uncertainty: heuristics and biases. Science, New Ser. 185(4157), 1124–1131 (1974) 48. Wang, Y., Vassileva, J.: Bayesian network trust model in peer-to-peer networks. In: AP2PC, pp. 23–34 (2003) 49. Wikipedia encyclopedia. www.wikipedia.org 50. Ziegler, C.N., Golbeck, J.: Investigating interactions of trust and interest similarity. Decis. Support Syst. 43(2), 460–475 (2007)

Chapter 9

Cooperation Jean-Pierre Georgé, Marie-Pierre Gleizes, and Valérie Camps

Great discoveries and improvements invariably involve the cooperation of many minds. Alexander Graham Bell Thank you for your cooperation and vice versa. Eugene Ormandy

Objectives This chapter aims at providing the reader with a thorough understanding of the notion of cooperation and its use in artificial systems. In this chapter the reader will: • Understand cooperation both on an intuitive level and as a definition; • See an illustration of cooperation in natural systems and understand its importance; • Learn about the AMAS (Adaptive Multi-Agent Systems) theory which states how to use cooperation as an engine for self-organisation, effectively building adaptive multi-agent systems;

J.-P. Georgé () · M.-P. Gleizes · V. Camps IRIT, Université Paul Sabatier, Toulouse, France e-mail: [email protected] M.-P. Gleizes e-mail: [email protected] V. Camps e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_9, © Springer-Verlag Berlin Heidelberg 2011

193

194

J.-P. Georgé et al.

• Be guided as he will try to apply it in artificial systems and discover how it has been done in existing applications; • See how cooperation can be used as a self-organising mechanism in artificial system to produce emergent functionalities.

9.1 Introduction The new applications that software engineers have to develop become more and more complex (see Chap. 3). The different events encountered by the system cannot all be known at the specification phase of the system design. Therefore, designers need new approaches to design adaptive systems, i.e. enabling the system to adapt itself to unexpected events. One powerful way to obtain this is to rely on the emergence of the required functionalities in a given environment or context. To obtain emergent phenomena, different methods or mechanisms have been studied by researchers (see Chap. 3 and the chapters of Part II). In the approach presented in this chapter, we assume that to change the function of a system, the system only has to change the organisation of its agents. For example, a common definition of agent organisation in the agent literature is that it is defined by the lines of communication of the agent components, the authority relationships between them, and the individual agent functionality. Based on this definition, if any if these three aspects changes for some reason, then the agent organisation also changes.The behaviour rules enabling this self-organisation are based on cooperation, which is the heart of this emergence-based bottom-up approach. Everybody has an intuitive understanding of what cooperation is about. Readers can see an illustration in Fig. 9.1. This chapter aims at giving a better understanding of this simple notion so that it can be applied in complex artificial systems when needed to improve their functioning or for facilitating their design. Cooperation is classically defined by the fact that two agents cooperate if they need to share resources or competences [7, 31]. We add to this definition the fact that an agent tries, on one hand, to anticipate cooperation problems and, on the other hand, to detect cooperation failure and try to repair these non-cooperative situations [28]. To anticipate, the agent always chooses the actions which perturb other agents it knows as little as possible (another agent is perturbed when the action hinders it for its own goals or if the action results in any king of cost for it). In this chapter, we present an approach to design self-organising systems based on multi-agent [31] systems (see Chap. 5) and cooperation (see also Chap. 12, describing the ADELFE methodology). The next section clarifies the concept of cooperation by giving a background context, explaining related notions and defining it. In Sects. 9.3 and 9.4, the theoretical notions and the resulting technology con-

9 Cooperation

195

Fig. 9.1 Cooperation increases mutual benefit of parties involved. Even for mules!

stituting the adaptive multi-agent systems (AMAS) theory are expounded. In this theory, systems’ self-organisation capabilities are based on the social inspired notion of cooperation. To illustrate that approach, two applications are then presented, one concerning a service providing multi-agent system and one elaborating on a multi-robot resource transportation problem. The focus is on the implementation of cooperative behaviours and how they enable the self-organisation which solves the problems constituting each application. Finally, the readers will be able to test and train their skills with practical exercises asking to implement cooperation in various situations.

196

J.-P. Georgé et al.

9.2 Understanding Cooperation This section will rapidly clarify the underlying concepts which support cooperation such as the notions of collective activity in social systems, interaction and communication. It will then give an intuitive understanding of cooperation and definitions illustrated by examples in natural systems, both animal and human. It will show that multi-agent systems are quite fond of cooperation, that computer science provides a more formal definition of cooperation for artificial systems and that it seems to be a promising mechanism for achieving self-organisation and emergence.

9.2.1 Underlying Concepts As soon as an activity involves more than one lone entity, interaction is bound to appear eventually. This interaction can take multiple forms, from simple pushing, racing, giving something, to more elaborate exchanges, negotiations, deals, and finally complex organised social structures as seen with social insects or human groups. Communication (directly of indirectly through the environment) plays of course an essential role in enabling this interaction, as well as a representation of the others, their nature and their goals. Computers and software are build as powerful Input/Output systems with wellknown communication and interaction means when considering computer–peripheral or computer–computer interaction. Things get more complicated as complexity grows, entities get mode heterogeneous, resources get scarce, goals vary, and the whole is of course expected to produce relevant and optimal results. At this point, an agent model facilitates further analysis of the systems. Ferber [7] produced a first clear interaction typology of situations depending on the goals of the agents (compatible or not), resources availability (sufficient for all or not) and competences of the involved agents (does each agent possess all its needed competences?). This is summarised in Table 9.1. A brief explanation of the different situation types is presented below: • Situation 1. Since each agent is self-sufficient, there is no need for cooperation. Agents can still benefit from cooperating to get optimised results. Example: “I can set up my tent on my own, but if you help me and then I help you, we might set both up before the storm. Or at least, there will be one already usable when the rain hits and we can share it. Isn’t it nice?” • Situation 2. The agents do not have all the competences needed to be selfsufficient. This is the classic case, for instance, where robots have to move boxes which are to heavy for one robot to carry. Two robots need to cooperate to move one box together, then another. • Situation 3. Resources are insufficient, and the agents have to share these resources. For instance, a one-way bridge situation is optimally handled when agents agree to wait to let some agents cross on one direction, then on the other and so on.

9 Cooperation

197

Table 9.1 Interaction types depending on goals, resources and competences as done by Ferber Goals 1 Compatible

Resources

Competences Situation type

Interaction category

Sufficient

Sufficient

Independence

Indifference

Insufficient

Simple collaboration

2 3

Insufficient Sufficient

Cluttering

4

Insufficient

Coordinated collaboration

5 Incompatible Sufficient

Sufficient

Pure individual competition

6

Insufficient

Pure collective competition

7 8

Insufficient Sufficient Insufficient

Individual conflicts for resources

Simple cooperation

Antagonism (requires negotiated cooperation)

Collective conflicts for resources

• Situation 4. Here both resources and competences are limited. As an example, we can mix both previous examples and have plenty of our robots needing to cross the bride in one direction or another (see also the application presented in Sect. 9.5.2). • Situations 5, 6, 7 and 8. These situations have the same characteristics as the previous ones with one huge exception: the agents do not have the same goals. For instance, each robot aims at gathering all the boxes. They can start attacking each other, avoid each other if possible or, more rationally, negotiate. Such negotiation commonly originates from the belief that acquiring at least some of the available boxes is still preferable to acquiring none. These situations usually require that agents either actively communicate to reach an agreement or have some means to exclude or punish agents which are not cooperating, at least if the aim is to reach a global optimum or to limit risk for an individual agent.

9.2.2 What is Cooperation? The reader certainly already has at least an intuitive understanding of cooperation, for basically, one only has to look at their own everyday cooperation. The above table, explanations and examples also give an intuitive understanding. Dictionaries simply define it as: Definition 9.1 Cooperation: the act of cooperating, or of operating together to one end; joint operation; concurrent effort or labor [1913 Webster]. Definition 9.2 Cooperate: To act or operate jointly with another or others; to concur in action, effort, or effect [1913 Webster].

198

J.-P. Georgé et al.

Furthermore, the following explanation can be found on Wikipedia:1 “Cooperation is the process of working or acting together, which can be accomplished by both intentional and non-intentional agents. In its simplest form it involves things working in harmony, side by side, while in its more complicated forms, it can involve something as complex as the inner workings of a human being or even the social patterns of a nation. It is the alternative to working separately in competition. [. . . ] cooperation may be coerced (forced), voluntary (freely chosen), or even unintentional, and consequently individuals and groups might cooperate even though they have almost nothing in common qua interests or goals. Examples of that can be found in market trade, military wars, families, workplaces, schools and prisons, and more generally any institution or organisation of which individuals are part (out of own choice, by law, or forced).” We can find a multitude of cooperation examples in nature and social systems, whatever the size of the group or its aims. A usual example is how an anthill manages to quite efficiently gather food, or how termites build complex structures using stigmergy [14]. The ant, by leaving pheromones on the ground when returning to the nest after finding food, ensures that other ants looking for food have a better chance to quickly find food sources. Moreover, these pheromones accumulate and evaporate in a way that enables the emergence of collective patterns; for instance, one can observe in specific laboratory set-ups that after a while all ants take the shortest path [3]. The most notorious and well-studied social experiment involving cooperation is the prisoner’s dilemma.2 Two prisoners face specific different sentences depending on if they denounce each other or not as having committed the crime, and they cannot speak with each other. If they both denounce each other, they face heavy sentences, if they both deny, they face light sentences, and if only one denounces the other, he walks while the other pays for the crime alone. The rational action is to both deny (they cooperate) so that as a team, the cost is small. A more realistic set-up is the iterated prisoner’s dilemma where the situation occurs more than once and where trust and reputation enter the game. It is important to note that not all activity involving at least two agents can be seen as cooperation. All prejudicial activity of one agent on another is of course the contrary of cooperation. But even a fully altruistic behaviour is not cooperative in the sense that when some agents sacrifice themselves for the others, it might not be the best for the group as a whole. Cooperation implies that both parties benefit from the activity, at least in the long run. It is also interesting to note that agents can act cooperatively but with very different social strategies. The most simple strategy, which could be called benevolent cooperation, is to suppose that every agent is also cooperative and so, the agent always spontaneously cooperates when asked. This is an assumption which can easily be taken when building an artificial system where each agent is designed to be 1 www.wikipedia.org 2 http://en.wikipedia.org/wiki/Prisoner’s_dilemma

9 Cooperation

199

cooperative. Open heterogeneous systems where agents are free to enter and no normative structure exists to enforce cooperation forces the agents to be more prudent. The agents rely on negotiation, trust, reputation and so on, leading to a tit-for-tat cooperation.

9.2.3 Using Cooperation in Artificial Systems Cooperation was extensively studied in computer science by Axelrod [1] and Huberman [18] for instance. “Everybody will agree that cooperation is in general advantageous for the group of cooperators as a whole, even though it may curb some individual’s freedom” [16]. Relevant biological inspired approaches using cooperation are for instance Ant Algorithms [6] which give efficient results in many domains. Multi-Agent Systems [31] are a perfect paradigm to apply cooperation: several to numerous possibly heterogeneous entities, each with its own local view, knowledge and goals striving to achieve a collective function as effectively as possible. The need to cooperate is inherent in these kind of systems, not surprisingly since they are essentially a social systems metaphor. All imaginable means to implement cooperation can be used since the only limit is what functionality a designer can put into an agent. Well-known and studied mechanisms that can be used to realise cooperation include negotiation protocols, trust, reputation, gossip, normative structures, stigmergy, etc. (refer to the corresponding chapter for more detail). The next section will give a more formal and specific definition of cooperation when used in MAS. It is intended as a guide for the design of the behaviour of the agents. The main aim of this guide is to ensure that the system as a whole, by having the agents locally and cooperatively self-organise, behaves as expected (or as best as it is possible) in any situation.

9.3 The Philosophy of the AMAS Theory This section presents an informal and intuitive approach of the AMAS theory in order to show the process followed to construct this theory and to highlight the motivation that lead to the development of this theory. First, the objectives of the AMAS theory are expounded. For this, we start by presenting examples of the kind of complex adaptive systems we want to develop and the characteristics tackled by this theory. Then, three main concepts of the AMAS theory are presented. The first concerns the adaptation and explains how a system can adapt. The second concept focuses on emergence and describes why we can qualify the global behaviour of the artificial system as emergent. The last concept is about cooperation, which plays a fundamental role in the AMAS theory.

200

J.-P. Georgé et al.

9.3.1 Objectives Example systems are briefly presented here to give the reader a better idea of the kind of systems this approach is used on. Specificities about the agents and the MAS they constitute are given, as well as the more general or philosophical aim of this approach. Examples of Targeted Adaptive Complex Systems The adaptive multi-agent systems we want to design are dedicated to solve complex problems. These problems exclude simple brute force solvers and lead us to design complex systems in order to solve them. Some illustrative examples include: • A time tabling with numerous and dynamic constraints in which when a new constraint is changed, the new solutions must minimise the numbers of changes in the time tabling. • Manufacturing control which is a problem of production planning and control. Its concern is the application of scientific methodologies to problems faced by production management, where materials and/or other items, flowing within a manufacturing system, are combined and converted in an organised manner to add value in accordance with the production control policies of management [35]. • Molecule folding which consists in finding the organisation between the atoms constituting the molecule that minimises the global energy of the molecule. It is a difficult problem because of the lack of knowledge about the inter-atom influences (see the TFGSO website).3 Many more of these kind of problems exist, which generally share one or more of the characteristics presented in the following sub-section. Context and Scope of AMAS The agents have to collectively solve the problem, and thus all the agents participate in the solving without deliberatively lying or being malicious. The agents can be compared to sub-programs which contribute to the design of the global program in classical computer science. In this case, designers do not conceive a sub-program which would hide a result or provide a false result when they find the right one. This approach focuses on MAS in which autonomous agents have to solve a common task or reach a common objective. By consequence, the AMAS theory cannot be fully used to design all the adaptive complex systems and all kinds of simulation of these systems in the same MAS. For example, a system used to simulate in a same system an economic system with layers and malicious agents would require other approaches. The application field is concerned by applications with the following characteristics: • The application is complex in the sense of complex systems. • The control and the knowledge can (and often has to) be distributed. 3 www.irit.fr/TFGSO

9 Cooperation

201

• There is a problem to solve. The problem can be expressed as a task or a function to realise, a structure to be observed, . . . . • The application objective can be very precise such as the optimisation of a function or more diffuse such as the satisfaction of the system end-users. • The system has to adapt to an endogenous dynamic (with the add-ons or removing of parts of the system) or exogenous (with the interaction with its environment). • The system is underspecified. In this case the adaptation is a mean to design it (see Chap. 12). Objectives of the AMAS Theory Specifying a priori an organisation for a system that will have to deal with unexpected events constrains (maybe inopportunely) the space of possibilities. It is commonly admitted that it is very difficult to predict how a real complex system will behave in a dynamic environment. Because there is a huge number of states in the space search, it is not possible to explore all of them in a reasonable time. A goal of the AMAS theory is to provide means to enable the system to find its right configuration in a given environment in order to be in adequacy with this environment (this will be explained in the next sub-section). In these terms, the AMAS theory can be used to produce systems with the same aims as those produced with optimisation methods such as simulated annealing [19], genetic algorithms [13, 17] and swarm algorithms [3, 6]. Since von Bertalanffy [33], many authors [5, 15, 21, 23] have studied systems of different order that cannot be apprehended by studying their parts taken separately: “We may state as characteristic of modern science that this scheme of isolable units acting in one-way causality has proven to be insufficient. Hence the appearance, in all fields of science, of notions like wholeness, holistic, organismic, gestalt, etc., which all signify that, in the last resort, we must think in terms of systems of elements in mutual interaction” [33]. By consequence, scientists who are interested in these complex systems, must propose new models and new approaches to study and design such systems. The AMAS theory is one of these theories which aim to assist in designing adaptive complex (multi-agent) systems. Complying to the previous point of view, it focuses on the elements of the system that are the agents. The main concern is to design interacting autonomous agents with local knowledge which would collectively provide the global system with a coherent behaviour. The question is: what are the local rules of behaviour for these agents? Cooperation is the key as the reader could easily guess.

9.3.2 Adaptation Adaptation is commonly defined by the capability a system has to change its internal structure in order to modify its behaviour to reach adequacy with its environment [10, 36] (see Chap. 5). Adequacy means that the behaviour of the system fits well inside its environment. For example, if we observe a crisis management system

202

J.-P. Georgé et al.

Fig. 9.2 Simple calculus agents in a first configuration representing a function with a result of 120

Fig. 9.3 Simple calculus agents after reorganising, representing a function with a result of 210

during a forest fire, the system is composed of the rescue team (such as fire men, doctors. . . ) and autonomous artificial resources (such as robots). The environment of the system is the forest and the people present in the forest (wounded or not). We can say that the system is adequate from the environment point of view if it can stop the fire, rescue the wounded and save the other people. A multi-agent system is a system composed of several interacting and autonomous agents. The relationships between the agents provide the organisation of the global system. These relationships can be expressed in terms of: • tasks relations between the agents, for example an agent A provides to an agent B the result of a task or an information; • beliefs relations about others agents; for example, an agent can have a point of view and a confidence on others agents; • physical relations between agents, for example two atoms are linked in a molecule by a covalent link. For example, the EPE (Emergent Programming Environment) system [8] shows an organisation between the following agents: the two “+” (“plus”) and “*” (“multiply”) operators and three numbers (120, 2 and 10) which can also be seen as simple agents. The aim of these agents is to form an organisation to exchanges values and calculate a final result. With the organisation in Fig. 9.2 the global behaviour of the system is to compute and provides the value 120. The system can then be required to produce a different value. An intuitive idea (applied in the AMAS theory) to realise the adaptation of a complex system composed of several interacting and autonomous agents is to change its organisational structure. Continuing with the example, you can see in Fig. 9.3 a new organisation of the EPE system. This new organisation provides a new final result which is now 210. The adaptation capability of this system lies on its capability to change and find its right organisation. In the AMAS approach, we consider that each part Pi of a system S achieves a partial function fPi of the global function fS (see Fig. 9.4). fS is the result of

9 Cooperation

203

Fig. 9.4 Adaptation: changing the function of the system by changing the organisation

the combination of the partial functions fPi , noted by the operator “◦”. The combination being determined by the current organisation of the parts, we can deduce fS = fP1 ◦ fP2 ◦ · · · ◦ fPn . As generally fP1 ◦ fP2 = fP2 ◦ fP1 , by transforming the organisation, the combination of the partial functions is changed, and therefore the global function fS changes. This is a powerful way to adapt the system to the environment. A pertinent technique to build this kind of systems is to use adaptive MAS. As in Wooldridge’s definition of multi-agent systems [34], we will be referring to systems constituted by several autonomous agents, plunged in a common environment and trying to solve a common task. Since we want the system to find by itself its organisation, we characterise it as self-adaptive. Note that we have to distinguish a system which is adapted because the designer stops it, changes something in it and launches it again, from systems which adapt themselves to react to their environment. For example, in the first case, a car which does not start is not well adapted, but after a mechanic has repaired it, the car starts and is adapted to the need of its owner. In the second case we can take an example in natural systems with the Darwinian evolution where animals evolve to adapt to their environment. This book is dedicated to the second type of systems. In the AMAS approach, the organisation changes are done autonomously by the agents of the system, and it is a process of self-organisation (as it has been defined in Chap. 3). This self-organisation is the origin of emergent global properties at the system level which cannot be predicted when only the agents behaviours are given.

9.3.3 Emergence As you can see in Chap. 3, emergence is a widely studied concept, and numerous definitions exist. Because you are computer scientists, we provide a “technical” definition of emergence, relying on computer science concepts. It is based on three points:

204

J.-P. Georgé et al.

1. The subject. The goal of a computational system is to realise an adequate function, judged by an external observer. It is this function, which may evolve during time, that has to emerge. The term function must be taken in a general meaning and not in a strict mathematical sense. A global function here can be a problem solving, a coherent behaviour, a structure, . . . . In software engineering it is simply what the system has to do. 2. The condition. This function is emergent if the coding of the system does not depend in any way of the knowledge of this function. Still, this coding has to contain the mechanisms allowing the adaptation of the system during its coupling with the environment, so as to tend any time towards the adequate function. 3. The method. To change the function in the AMAS theory, the system only has to change the organisation of its components. The mechanisms which allow the changes are specified by cooperative self-organisation rules providing autonomous guidance of the components behaviour without any knowledge on the collective function. The condition is perhaps the more difficult part to understand. As a designer, we know what the system has to do, and we want to “control” what the system will do. Designers want to control the emergence which seams completely antinomic with the meaning of emergence. Therefore, most of the time, the phenomenon observed is not a surprise for the designer. But we can qualify it as emergent from an engineering point of view, because if only the code of the agents is accessible and can be studied, the study cannot explain and predict the global function realised by interacting agents. In this sense, the global function is emergent. It is the reason why we need to code local criteria to guide the agents behaviour and the agents do not know the global function. The way to obtain this global function is not coded inside the agent code. In the AMAS theory, these local criteria are based on the aim to maintain cooperation.

9.3.4 Cooperation as the Engine for Self-organisation For a MAS, implementing this adaptation with an emergent global function implies that the designer only has to take care of the agent by giving it the means to decide autonomously to change its links with the other agents. As you have seen, we start from the principle that, to have a relevant behaviour, the elements that constitute a system have to be “at the right place, at the right time” in the organisation. To achieve this, each agent is programmed to be in a cooperative situation with the other agents of the system. Only in this case does an agent always receive relevant information for it to compute its function, and always transmit relevant information to others. The designer provides the agents with local knowledge to discern between Cooperative and Non-Cooperative Situations (NCS).4 4 The

concept of NCS will be precisely defined in Sect. 9.4.1, Definition 9.4.

9 Cooperation

205

Cooperation can be summarised in the following attitude: an agent tries to help and not hinder the other agents. So an agent has to detect and eliminate NCS he encounters, and it has to avoid creation of new NCS. This behaviour constitutes the engine of self-organisation. The reader can compare this with how stigmergy (described in Chap. 6) is used in ant colonies (even if stigmergy can also be seen as a means to cooperate). Depending on the real-time interactions the MAS has with its environment, the organisation between its agents emerges and constitutes an answer to the aforementioned difficulties in complex systems (see Sect. 9.1); indeed, there is no global control of the system. In itself, the emergent organisation is an observable organisation that has not been given first by the designer of the system. Each agent computes a partial function fPi , but the combination of all the partial functions produces the global emergent function fS . By principle, the emerging purpose of a system is not recognisable by the system itself; its only criterion must be of strictly local nature (relative to the activity of the parts which make it up). By respecting this, the AMAS theory aims at being a theory of emergence.

9.4 The AMAS Theory: Underlying Principles and Implementation The two first parts of this section expound the main definitions and theorem the AMAS theory is based on. This theory can be applied at the system and also at the agent level. Because the global system behaviour is the result of interacting agents, the architecture and the general algorithm of an agent are described in the two last parts.

9.4.1 The Theorem of Functional Adequacy In order to show the theoretical improvement coming from cooperation, the AMAS (Adaptive Multi-Agent System) [11] theory has been developed, which is based upon a specific theorem which is described bellow. This theorem describes the relation between cooperation in a system and the resulting functional adequacy5 of the system. For example, let us consider a car as a system and the driver plus the real world as its environment. If the driver wants to move forward with the car and acts on the car to do this and if the car goes backward, the system behaviour is not adequate from the environment point of view. 5 “Functional”

refers to the “function” the system is producing, in a broad meaning, i.e. what the system is doing and what an observer would qualify as the behaviour of a system. And “adequate” simply means that the system is doing the “right” thing, judged by an observer or the environment. Therefore, “functional adequacy” can be seen as “having the appropriate behaviour for the task”.

206

J.-P. Georgé et al.

Theorem 9.1 For any functionally adequate system, there exists at least one cooperative internal medium system that fulfils an equivalent function in the same environment. Definition 9.3 A cooperative internal medium system is a system where no NonCooperative Situations exist. Note that the cooperative internal medium system can be either built so as to be functionality adequate or this system is given the capabilities to reach the adequacy on its own. In a cooperative internal medium system, the components composing the system (which are in the internal medium) are always in cooperative situations. For example, if we consider a manufacturing control problem (briefly described in Sect. 9.3), it has to be constantly in a solved state inside. This means that the products are always made with all the constraints satisfied. From the environment point of view the system is functionally adequate because all constraints are satisfied. Inside the system, this means that the products, the work stations and the operators have no problems to work together, they cooperate well and they have no NCS. In other terms, we can say that all components are in the right location at the right time. The configuration of the system is the right one. We can apply this theorem to design complex adaptive systems composed of agents. To obtain an internal cooperative medium, we have to guarantee cooperative states between the agents inside the system and between the agents and the system environment (if the agents can perceive it of course). We call this kind of agents cooperative agents. To design cooperative agents, it is necessary to provide for each of them a behaviour to be continually in cooperative interactions. But in a dynamic environment and open systems, this status cannot be always guaranteed. The objective is by consequence to design systems that do the best they can when they encounter difficulties. These difficulties can be viewed as exceptions in traditional programming. From an agent point of view, we call them Non-Cooperative Situations (NCS, see the definition below) or cooperation failures. The designer has to describe not only what an agent has to do in order to achieve its goal, but also which locally detected situations must be avoided, and if they are detected, how to suppress them (in the same manner that exceptions are treated in classical programs). The agent design concerns on one part to provide the agent its nominal behaviour (the capabilities to play its role in the system) but also a cooperative behaviour to avoid and/or to remove the NCSs. The NCSs are defined in a very general and high-level way with the above meta-rules at each step of the agent life cycle (perception, decision, action). These meta-rules have to be instantiated in each application by the designer. Definition 9.4 An agent is in a Non-Cooperative Situation (NCS) when: (¬cperception ) a perceived signal is not understood or is ambiguous; here, signal is a general term to point out something received or perceived by the agent (a message, a video feed, . . . );

9 Cooperation

207

(¬cdecision ) perceived information does not produce any new decision; (¬caction ) the consequences of its actions are not useful to others. Let us give examples from everyday life to better understand these situations. For the first situation, a signal is not understood when a person speaks to you in Chinese and if you do not understand Chinese. For the second situation, in the human world, quite often a person overhears a conversation between other people, and this conversation does not concern him, or he already knows the information. In the third situation, if in a robot world, robots have to clean a room and if a robot Ra prevents another robot Rb from moving, Ra does not act cooperatively. We can identify seven NCS subtypes that further specify these situations: • incomprehension (¬cper ): the agent cannot extract the semantic contents of a received stimulus, • ambiguity (¬cper ): the agent extracts several interpretations from a same stimulus, • incompetence (¬cdec ): the agent cannot benefit from the current knowledge state during the decision, • unproductiveness (¬cdec ): the agent cannot propose an action to do during the decision, • concurrency (¬cact ): the agent perceives another agent which is acting to reach the same world state, • conflict (¬cact ): the agent believes that the transformation it is going to operate on the world is incompatible with the activity of another agent, • uselessness (¬cact ): the agent believes that its action cannot change the world state or it believes that the results for its action are not interesting for the other agents.

9.4.2 Consequence of the Functional Adequacy Theorem This theorem means that we only have to use (and hence understand) a subset of particular systems (those with cooperative internal mediums) in order to obtain a functionally adequate system in a given environment. We concentrate on a particular class of such systems, those with the following properties [11]: • The system is cooperative and functionally adequate with respect to its environment. Its parts do not know the global function the system has to achieve via adaptation. • The system does not use an explicitly defined goal, rather it acts using its perceptions of the environment as a feedback in order to adapt the global function to be adequate. The mechanism of adaptation is for each agent to try and maintain cooperation using their skills, representations of themselves, other agents and environment. • Each part only evaluates whether the changes taking place are cooperative from its point of view—it does not know if these changes are dependent on its own past actions.

208

J.-P. Georgé et al.

Fig. 9.5 The different modules of a cooperative agent

This approach has been successfully applied in the engineering of self-organising agent-based systems in various application contexts with different characteristics, such as autonomous mechanisms synthesis [4], flood forecast [9], electronic commerce and profiling [12]. On each, the local cooperation criterion proved to be relevant to tackle the problems without having to resort to an explicit knowledge of the goal and how to reach it.

9.4.3 Architecture and Behaviour of an AMAS Agent A cooperative agent in the AMAS theory has the four following characteristics. First, an agent is autonomous in its decision taking: an agent can say “no” or “go” (start some activity). Secondly, an agent is unaware of the global function of the system; this global function emerges (of the agent level towards the multi-agent level). Thirdly, an agent can, on one hand, try to avoid NCS and, on the other hand, detect NCS and acts to return in a cooperative state. And finally, a cooperative agent is not altruistic in the meaning that an altruistic agent always seeks to help the other agents. It is benevolent, i.e. it seeks to achieve its goal while being cooperative. More formally, the behaviour of an AMAS agent can be described with an algorithm based on the one found in Sect. 9.4.4. Cooperative agents are equipped with several modules representing a partition of their “physical”, “cognitive” or “social” capabilities (see Fig. 9.5). Each module represents a specific resource for the agent during its “perceive-decide-act” life cy-

9 Cooperation

209

cle. The first four modules are quite classical in an agent model [34]. The novelty comes from the Cooperation Module which contains local rules to solve NCS. Interaction Modules Agents’ interactions are managed by two modules. The Perception Module represents the inputs the agent receives from its environment. Inputs may have different levels of complexity and types: integer, boolean for simple agents or even symbolic messages in a mail box for high-level agents. The Action Module represents the output and the way the agent can act on its physical environment, its social environment or itself (considering learning actions for example). Similarly to the perceptions, actions may have different granularities: simple effectors activation for a robot or semantically complex message sending for social agents. Skill Module Even if cooperative agents mainly try to avoid NCS, they have several tasks to complete. The ways to achieve their goals are expressed in the Skill Module. Skills are knowledge about given knowledge fields and allow agents to realise their partial function—as a part of a MAS that produces a global function. No technical constraints are required to design and develop this module. For example, skills can be represented as a classical or fuzzy knowledge base of facts and rules on particular domains. It also can be decomposed into a lower-level MAS to enable learning, as in the ABROSE online brokerage application [12], where skills were decomposed into a semantic network. Representation Module As for the Skill Module, the Representation Module can be implemented as a classical or fuzzy knowledge base, but its scope is the environment (physical or social) and itself. Beliefs an agent possesses on another agent, as well as all information the agent possesses on its environment, are considered as representations. Like skills, representation can be decomposed into a MAS when learning capabilities on representation are needed. Aptitude Module Aptitudes represent the capabilities to reason on perceptions, skills and representation—for example, to interpret messages. These aptitudes can be implemented as inference engines if skills and representations are coded as knowledge bases. Considering a given state of skills, representations and perceptions, the Aptitude Module chooses an action to do. Cases where there is zero or several proposed actions must be taken into account too (cf. Cooperation Module). Cooperation Module The cooperative attitudes of agents are implemented in the Cooperation Module. As the Aptitude Module, this module must provide an action for a given state of skills, representations and perceptions if the agent is in an NCS. Therefore, cooperative agents must possess rules to detect NCS. Several types of NCS have been identified. For each NCS detection rule, the Cooperation Module associates one or several actions to process to avoid or to solve the current NCS. Internal Functioning of an AMAS Agent Considering the described modules, the nominal behaviour of a cooperative agent is defined as follows. During the perception phase of the agents’ life cycle, the Perception Modules updates the values

210

J.-P. Georgé et al.

of the sensors. These data directly imply changes in the Skill and Representation Modules. Once the knowledge updated, the decision phase must result on an action choice. During this phase, the Aptitude Module computes from knowledge and proposes action(s) or not. In the same manner, the Cooperation Module detects if the agent is in an NCS or not. In the former case, the Cooperation Module proposes an action that subsumes the proposed action by the Aptitude Module. In the latter case, the only action6 proposed by the Aptitude Module is chosen. Once an action chosen, during the action phase, the agent acts by activating its effectors or changing its knowledge.

9.4.4 The Cooperative Algorithm The algorithm in Fig. 9.6 may be viewed as a formal representation of the cooperative attitude of the agents described previously: according to the AMAS theory, agents have to be able to detect when they are in an NCS and in which way they can act to come back in a cooperative situation. In the algorithm, there are two main states to which the decision process may be confronted: either the agent is in a cooperative situation (depending on its perceptions) (the function called inCooperativeSituation returns the value “true”), or it is in an NCS (the function called inCooperativeSituation returns the value “false”). In the first state, the agent simply chooses the action with the highest priority among those which can be executed (function called executeUtilityAction). Theses actions are said to be utility actions in the algorithm, meaning that they are useful for the goal of the agent, for another agent or for the system. If the agent can prevent NCS, these actions also avoid to generate new NCS. In the other state, the agent is in an NCS, and, in addition to some possible utility action, it will choose the corrective action with the highest priority depending on its perceptions (function called executeNcsCounteredAction). Following such an algorithm, agents always try to stay in a cooperative situation, and so the whole system converges to a cooperative state within and with its environment. This leads—according to the theorem of functional adequacy—to an adequate system. Thus, this algorithm describes the typical decision process of a generic AMAS agent. But the NCS and the actions which could be applied to solve them are not generic: designers have to write their own- and so specific-NCS set and related actions for each kind of agent they wish the system to contain. This work must be performed during the design of the agents: the designer must exhaustively find all the NCS which could occur for each kind of agent and, for each one, find the relevant actions which could solve the lack of cooperation. Methods (like ADELFE) can help for this (see Chap. 12). 6 There

is only one action possible, otherwise an NCS is detected.

9 Cooperation

211

/* Definitions and notation */ P : Set of possible percept sets (perceptions, representations, or skills) at a given time for the agent. A: Set of possible action sets for the agent. (p, a): The couples (p:P , a:A) are the rules of possible behaviours for the agent, i.e. the actions to be executed for a given percept set. NCSR: Non-Cooperative Situation Rules, set of behaviours rules (p:P, a:A) corresponding to the detection of a non-cooperative situation and the associated corrective actions. SR: Skill Rules, set of behaviour rules (p:P, a:A) corresponding to the possible actions depending only on percept sets (without having to refer to the beliefs of the agent). BR: Belief Rules, set of behaviour rules (p:P, a:A) corresponding to the possible cooperative actions for given percept sets and by referring to the beliefs (to evaluate the cooperation). (p1, a1) (p2, a2): “”expresses a priority relationship of the behaviour (p1, a1) over (p2, a2). Procedure action(p:P ) { /* p is the current percept set of the agent */ if (inCooperativeSituation(p)) executeUtilityAction(p) else { /* The agent is in a non cooperative situation */ executeNcsCounterAction(p); executeUtilityAction(p); } } Function inCooperativeSituation(p:P ) Return Boolean { for each (p , a ) ∈ NCSR if (p ⊆ p) return false return true; } Function executeUtilityAction(p:P ) { for each (p , a ) ∈ SR ∪ BR | p ⊆ p if ( (p , a ) ∈ SR ∪ BR | p ⊆ p and (p , a ) (p , a )) do a } Function executeNcsCounterAction(p:P ) { for each (p , a ) ∈ N CSR | p ⊆ p if ((p , a ) ∈ N CSR | p ⊆ p and (p , a ) (p , a )) do a } Fig. 9.6 Procedure and functions for a cooperative agent

212

J.-P. Georgé et al.

9.5 Applications One does only truly understand a theory after taking two additional steps: study a comprehensive example of the application of the theory and applying it to himself or herself. For the later, we strongly encourage the reader to tackle the exercises of this chapter as they are intended to provide material to acquire practical knowhows. The first step will be taken here as this section presents no less than two different applications illustrating and detailing the use of cooperation conforming to the AMAS theory. The first is a dynamic and open service providing MAS where all the providers and customers (the agents) need to be put in relation with one another. This relationship needs to be constantly updated to ensure the most relevant social network (by being cooperative one with another). The second is a multi-robot resource transportation problem where the robots (the agents) have to share the limited routes to efficiently transport the resources (by choosing cooperatively how to move). Each description focuses on how cooperation can be applied, what NonCooperative Situation are for the agents and how it enables them to self-organise towards the adequate emergent function.

9.5.1 A Service Providing MAS 9.5.1.1 Instantiation of the AMAS Approach to a Case Study The first chosen case study to illustrate the AMAS approach consists in designing a system which enables end-users and service providers to get in touch when they share common points of interest. The application is made for the electronic commerce field, an open, dynamic and distributed context [12]. The main requirement of such an electronic information system is to enable (i) end-users to find relevant information for a given request and (ii) service providers to have their information proposed to relevant end-users. In concrete terms, the system has to provide: • • • •

personalised assistance and notification for the end-users, propagation of requests between the actors of the system, propagation of new information only to potentially interested end-users, acquisition of information about end-users’ real interests, in a general manner, and about providers’ information offers.

In such a system, every end-user and service provider has an individual goal: to answer the request he/she has to solve. Each end-user and service provider does not know the global function realised by the system. The system is open and strongly dynamic because a great number of appearances or disappearances of end-users and/or service providers may occur. Moreover, an a priori known algorithmic solution does not exist. In this context, classical approaches to tackle such a problem cannot be applied, and the use of AMAS is then clearly relevant.

9 Cooperation

213

9.5.1.2 Environment Definition and Characterisation The environment of the system consists of real end-users and service providers who have subscribed to the system. They exert pressure on the system (by submitting requests in order to find relevant service providers or to seek potential customers), and the system has to adapt itself to these constraints. The reorganisation of interaction links between agents representing end-users and service providers is a way for the system to adapt to its environment which can be described as inaccessible, continuous, non deterministic and highly dynamic. We can say that such a system is functionally adequate when a satisfied enduser wants to use services of the system again and when each service is fully used, namely in the most profitable way for the supplier.

9.5.1.3 Agent Design An end-user seeks a relevant service provider according to his/her centres of interest, while a provider tries to find potentially interested end-users according to his/her proposed services. These two actions are totally symmetric, and we will only focus on the search for a service. In that system, entities are autonomous, have a local goal to pursue (to find a relevant service provider or to make targeted advertisement), have a partial view of their environment (other active entities) or may interact with others to target the search more effectively. They are then potentially cooperative. Furthermore, since the system is open, new entities may appear or disappear, and they may not be able to communicate as they should (e.g. an entity does not understand requests from a new one). Therefore, such an entity is prone to cooperation failures and can be viewed as a cooperative agent. Each end-user (or service provider) is then represented within the system by an agent called Representative Agent (see Fig. 9.7). A representative agent (RA) aims at finding relevant RAs according to the request that its associated end-user or service provider submitted for solution. In accordance with the agent model given in Sect. 9.4.3, an RA consists of the following components: • The skills of an RA are those of the entity it represents. • Representations that an agent possesses about itself or about other RAs may evolve at runtime, and they have then to be adjusted. Because of this dynamics, we can use an AMAS to implement them (more information about this point can be found in [12]). When an RA receives a request, it has to query its representations on itself to know if it is relevant to solve this request. If it is not, it has then to query its representations on other known agents to identify if it knows an agent able to solve the received request. • The aptitudes of an RA enable it to modify its representations and to interpret a received request. For example, when an end-user makes a request, his/her RA has to update its representations to learn the new centres of interest of its end-user.

214

J.-P. Georgé et al.

Fig. 9.7 A service providing multi-agent system architecture

• Messages exchanged between RAs concern the requests to be solved. Physical exchanges of these requests can be made using the mailbox concept, a buffer enabling asynchronous communication.

9.5.1.4 Non-cooperative Situations Determination at the RA Level NCS have to be instantiated to the current problem and actions to be done when an agent is faced with an NCS have to be defined. An RA is situated at the right place in the organisation of the system if the three meta-rules given in Sect. 9.4 are checked. An RA may encounter four non-cooperative situations during its perception/decision/action cycle: 1. Total incomprehension Description: An agent faces total incomprehension when it cannot extract any informative content from the received message: this may be due to an error in transmission or if the transmitter gets a wrong belief about it. This NCS is detected during the interpretation phase when the agent compares the received request with its own representation (words matching) and cannot extract any informative content from the message; it has not the necessary competence. Actions: Because the agent is cooperative, the misunderstood message is not ignored; the agent will transmit the message to an agent that seems to be relevant according to its representations on others. 2. Partial incompetence Description: An agent is faced with partial incompetence when can extract an informative content from only one part of the received message. This NCS is detected during the interpretation phase when the agent compares the received request with its own representation (words matching) and can extract an informative content from only a part of the message.

9 Cooperation

215

Actions: The receiving agent sends back the partial answer associated with the understood part of the message. It sends the other part of the request to a more relevant agent. 3. Ambiguity Description: An ambiguity occurs when the an agent can extract several informative content from the received message it is faced to an ambiguity. This NCS is detected during the interpretation phase when the agent compares the received request with its own representation (words matching) and can extract several informative contents from the message. Actions: An agent is supposed to intentionally and spontaneously send understandable data to others. Therefore, the receiver of an ambiguous message sends back all its interpretations of the received request. The initial sender is then able to choose the most pertinent one and update its representation about the receiver’s skills. 4. Concurrence Description: A situation of concurrence occurs when two agents have similar skills for a given task. This NCS is detected during the interpretation phase, when the agent compares the received request with its own representation (words matching). If it can extract an informative content from only a part of the request, the agent compares this request with the representation it has about other agents to find rival agents. An agent A competes with an agent B, from B’s point of view, if A can extract informative content from the same part of the request as B. Actions: Redundancy is beneficial when an agent has not been able to reach its aim or to accept a task it has been asked to undertake. In these cases, it refers the problem to its rival(s). NCS being now instantiated to the current problem, the cooperative behaviour of an RA is the following: • When it detects an NCS, it acts in the world to come back to a cooperative state. • When it does not detect an NCS, it follows its own goal. The simple continued cooperative activity of the agents ensures that the global organisation (the agents of which are in relation with each other) is always the most relevant to satisfy every involved entity. By studying the situations in which cooperation can occur and identifying corresponding behaviours, the application has been enhanced. The analysis of cooperation provides the needed self-organising mechanisms and can be seen as a guide for obtaining the desired emergent functionality.

9.5.2 Multi-Robot Resource Transportation 9.5.2.1 Resource Transportation Problem The resource transportation problem is a classical task in Collective Robotics [29], and it was proposed as a relevant benchmark for robotic systems by [3]. Robots

216

J.-P. Georgé et al.

Fig. 9.8 The environment of the resource transportation problem is composed of a claim room (at left), a laying room (at right) and two narrow corridors (at top and bottom). Robots pick boxes against the left wall of the claim room (claim zone) and drop them against the right wall of the laying room (laying zone)

must transport resources (boxes) as fast a possible from a zone A to a zone B, separated by a constrained environment. In Picard’s work [25, 28] presented here, these zones are linked by two corridors too narrow for robots to cross one another side by side (see Fig. 9.8). This environment leads to a spatial interference problem, e.g. robots must share common resources, the corridors. Once engaged in a corridor, what must a robot do when facing another robot moving in the opposite sense? Spatial interference has been tackled by [30] in the case of robots circulating in corridors and having to cross narrow passages (doors). Their solution is to solve conflicts by aggressive competition (with explicit hierarchy), similarly to eco-resolution by [7]. Reference [20] propose to solve such problems thanks to attraction–repulsion mechanisms based on altruistic behaviours triggering—a reverse vision of the ecoresolution. In the application described here, Picard expounds a viewpoint halfway between the two firsts, in which robots are neither altruistic nor individualist and cannot directly communicate any information or intention. Moreover, no planifier system will anticipate trajectories because the use of planification in multi-robot domain remains inefficient, considering the high dynamics of a robot’s environment.

9.5.2.2 Cooperative Model Instantiation This section shows the instantiation—i.e. fulfilling each module—of the cooperative agent model in order to design robots able to realise the transportation task. This work appears in the ADELFE process (Chap. 12) in the Design Work Definition and more precisely in the Design Agents Activity [27]. ADELFE process is an extension to the Rational Unified Process (RUP) and consists in four work definitions, which are specifically adapted to agent-oriented software engineering: preliminary

9 Cooperation

217

requirements, final requirements, analysis and design. Requirements define the environmental context of the system. Analysis identifies the agents within other object classes. Modules Fulfilling The Perceptions Module represents inputs for agents. Concerning robots, they can know positions of the two zones (claim and laying). Indeed, this example only focuses on adaptation to a circulation problem instead of a foraging one. For example, we consider that the task of robots is to transfer boxes between rooms and that robots do not get involved in identifying box locations. Here is a possible list of perceptions for transporter robots: position of the claim zone, position of the laying zone, a perception cone in which objects are differentiable (robot, box or wall), proximity sensors (forward, backward, left and right), a compass and the absolute spatial position. The environment is modelled as a grid whose cells represent atomic parts on which a robot, a box or a wall can be situated. The Perceptions Module also defines limit values of perceptions (e.g. five cells). The Actions Module represents outputs of agents on their environment. Possible actions for transporter robots are: rest, pick, drop, forward, backward, left and right. Robots cannot drop boxes anywhere in the environment but only in the laying zone. They cannot communicate directly or drop land marks on the environment. In the case of social agents that are able to communicate, communication acts are specified in this module. The Skills Module contains knowledge about the task the agent must perform. Skills enable robots to achieve their transportation goals. Therefore, a robot is able to calculate which objective it must achieve in terms of its current state: if it carries a box, then it must go to the laying zone, otherwise it must reach the claim zone. Depending on its current goal, the Skills Module provides an appropriate action to process to achieve it. Robot’s goals are reach claim zone and reach laying zone. Moreover, robots have intrinsic physical characteristics such as their speed, the number of transportable boxes or the preference to move forward rather than backward—as ants have. Such preferences are called reflex values. The Representations Module contains knowledge about the environment (physical or social). Representation a robot has on its environment is very limited. From its perceptions it cannot identify a robot from another, but can know if it is carrying a box or not. It also can memorise its past absolute position, direction, goal and action. The Aptitudes Module enables an agent to choose an action in terms of its perceptions, skills and representations. Concerning transporter robots, a design choice must be taken at this stage. In terms of the current goal, the Skills Module provides preferences on each action the robot may do. The Aptitudes Module chooses among these actions what will be the next action to reach the goal. Many decision functions can be considered; e.g. an arbitrary policy (the action having the highest preference is chosen) or a Monte Carlo method-based policy that is chosen for our example. Therefore, the Aptitudes Modules can be summed up in a Monte Carlo decision function on the preference vector (the list of action preferences for an agent) provided by the Skills Module. In the same manner, the Cooperation Module provides preference vectors in order to solve NCS described in Sect. 9.5.2.3.

218

J.-P. Georgé et al.

Action Choosing At each time t, a robot has to choose between different actions that are proposed by the two decision modules (skills and cooperation). At time t, each action actj of the robot ri is evaluated. For each action, this value is calculated in terms of perceptions, representations and reflexes in the case of a nominal behaviour: (t, actj ) = wpri (t, actj ) + wmri (t, actj ) + wrri (actj ), Vrnomi i where: • • • •

(t, actj ) represents the value for the action actj at time t for the robot ri , Vrnomi i wpri (t, actj ) represents the calculated value in terms of perceptions, wmri (t, actj ) represents the calculated value in terms of memory, wrri (t, actj ) represents the calculated value in terms of reflexes.

As for aptitudes, an action preference vector is generated by the Cooperation coop Module Vri (actj , t). Once these values are calculated by the two modules for each action of a robot, the vector on which the Monte Carlo drawing will process is a combination of the two vectors in which the cooperation vector subsumes the nominal vector: coop

(t) ≺ Vri Vri (t) = Vrnomi i

(t).

9.5.2.3 Cooperative Behaviours Study In the previous section, the different modules of a robot and its components have been detailed, except the Cooperation Module. This section aims at discussing cooperation rules to establish in order to enable the multi-robot system to be in functional adequacy with its environment. Cooperative Unblocking Beyond the limited number of only two robots acting to transport boxes in a same environment, the nominal behaviour cannot be sufficient. Indeed, a robot owns skills to achieve its tasks, but not to work with other robots. In this very constrained environment, spatial interference zones appear. If two robots, a first one carrying a box and moving to the laying zone and a second one moving to the claim zone to pick a box, meet in a corridor, the circulation is blocked—because they cannot drop boxes outside the laying zone. Then, it is necessary to provide cooperative behaviours to robots. Two main NCSs (non-cooperative situations) can be reactively solved: A robot is blocked. A robot r1 cannot move forward because it is in front of a wall or another robot r2 moving in the opposite direction.7 In this case, if it is possible, r1 must move to its sides (left or right). This corresponds to increasing valcoop ues of the cooperative action vector related to side movements Vr1 (t, right) and 7 If r moves in another direction than the opposite direction of r , it is not considered as blocking 2 1 because it will not block the traffic anymore.

9 Cooperation

219

Table 9.2 Example of specification of the “a robot is returning” uselessness NCS Condition

Action

ret ∧ freeR

Vri

ret ∧ freeL

Vri

ret ∧ ¬(freeL ∨ freeR) ∧ ant ∧ toGoal ∧ cGoal

coop Vri (t, backward) coop Vri (t, forward) coop Vri (t, backward) coop Vri (t, forward)

ret ∧ ¬(freeL ∨ freeR) ∧ ant ∧ toGoal ∧ ¬cGoal ret ∧ ¬(freeL ∨ freeR) ∧ ant ∧ ¬toGoal ret ∧ ¬(freeL ∨ freeR) ∧ ¬ant With: • ret: ri is returning • freeR: right cell is free • freeL: left cell is free • ant: in front of an antinomic robot

coop

(t, right)

coop

(t, left)

• toGoal: ri is moving to goal • cGoal: ri is closer to its goal than its opposite one • : increasing

coop

Vr1 (t, left). If r1 cannot laterally move, two other solutions are opened. If r2 has an antagonist goal, the robot which is the most distant from its goal will move coop backward (increasing Vri (t, backward)) to free the way for the robot which is coop the closest to its goal (increasing Vri (t, forward) even if it may wait). If r2 has the same goal as r1 , except that if r1 is followed by an antagonist robot or if r1 moves away from its goal (visibly it moves to a risky8 region), r1 moves backward; else r1 moves forward, and r2 moves backward. A robot is returning. A robot r1 is returning9 as a consequence of a traffic blockage. If it is possible, r1 moves to its sides (and is no more returning). Else, r1 moves forward until it cannot continue or if encounters another robot r2 which is returning and is closer to its goal than r1 . Table 9.2 sums up the behaviour in this situation. If there is a line of robots, the first returning robot is seen by the second one that will return too. Therefore, the third one will return too, and so on until there are no more obstacles. These rules correspond to resource conflict (corridors) or uselessness when a robot must move backward and away from its goal. In the case of robots, situations will not be specified as incomprehension because robots are unable to communicate directly. These rules, which are simple to express, ensure that robots cannot block each other in corridors. However, this cooperation attitude only solves problem temporarily, creating returning movement, and then implies time loss to transport boxes. Cooperative Anticipation By taking into account the previous remark, it seems possible to specify cooperation rules to anticipate blockage situations in order to 8 It

is risky in the sense that it may occur a lot of non-cooperative situations such as conflicts.

9A

robot is considered as returning until it has no choice of side movements.

220

J.-P. Georgé et al.

make the collective more efficient. We call this optimisation cooperation rules. Previous rules enable robots to extract from blockage. A robot is in such a situation because it was crossing a zone frequented by antinomic robots.10 So as to prevent this situation, robots must be able to avoid such risky zones, zones from which antinomic robots come. In accordance, an anticipation rule can be specified: A robot sees an antinomic robot. If a robot r1 perceives a robot r2 having an antinomic goal, if r1 can move to its sides, it does, else it moves forward.

Nevertheless, this reactive anticipation presents a major problem: once a robot has avoided the risky zone, no mechanism ensures that it will not go in it again, led by its goal. In order to tackle this difficulty, robots can be equipped with a memory of the risky zones (in the Representations Module). Each time t a robot ri experiments an anticipation situation facing a robot rj , it adds to its memory a tuple (or virtual marker) posX(rj , t), posY(rj , t), goal(ri , t), w in which posX(ri , t) and posY(ri , t) represent the coordinates of rj at the moment t, goal(ri , t) represents that the goal ri was achieving at time t, and w represents a repulsion value. The higher the value is, the more the robot will try to avoid the zone described by the marker when it is achieving another goal than goal(ri , t). Therefore, the robot inspects all its personal markers11 whose distances are inferior to the perception limit (to fulfil the locality principle). A marker with weight w and situated in the direction coop dir at a distance d induces that Vri (t, diropp ) will be increased of w (diropp is the opposite direction to dir). As the memory is limited, tuples that are added must disappear during simulation run-time. For example, the weight w can decrease of a given value δw (called forgetting factor) at each step. Once w = 0, the tuple is removed from the memory. This method corresponds to the use of virtual and personal pheromones. Finally, as ants do, robots can reinforce their markers: a robot moving to a position corresponding to one of its marker with another goal re-initialises the marker. In fact, if the robot is at this position, it might be a risky zone when it tries to achieve another goal.

9.5.2.4 Cooperation and the Emergence of Corridor Dedication Figure 9.9 shows the corridor-usage after a while: their cooperative behaviour guides the robots to use one corridor when trying to reach the boxes and the other to bring them back. The anticipative behaviour is mainly responsible for this observance of the emergence of a sense of traffic. Robots collectively dedicate corridors to particular goals. In fact, markers of all the agents are positioned only at one corridor entry for one direction as shown in Fig. 9.9. This view is only for monitoring purpose: robots do not perceive all the markers, only their own. We can assign the emergent property to this phenomenon because robots do not handle any notion of corridor— unlike some previous works [26]. Thus, just with local data, robots established a 10 Robots with an antinomic behaviour to the considered robot, for instance going in the opposite direction in a corridor. 11 Robots

cannot share their memory as they cannot communicate.

9 Cooperation

221

Fig. 9.9 The robots self-organise, and a corridor dedication emerges. We can also see the positioning of all the virtual markers (dark squares) for all the robots and the two goals

coherent traffic behaviour that leads to an optimisation of the number of transported boxes. In this application, readers can see the relevance of cooperation as a local criterion for agents to self-organise to be more adapted to a task. Considering the ignorance of the global task and the environment, the self-organising collective reaches an emergent coherent behaviour, which is then more robust to environmental risks. This example tackles a simple problem in a simple static environment in which the collective achieves its global task. Other simulations in difficult and dynamic environments confirm the relevance of cooperative self-organising collectives.

9.6 Conclusion To understand how cooperation can be used to build complex artificial systems, a theory of self-organising MAS, called AMAS has been presented in this chapter. This approach considers groups of cooperative agents which modify their interaction when non-cooperative situations occur, to reach a functional adequacy with the environment. This approach has been illustrated by two different example applications: a service providing MAS and a multi-robot resource transportation problem. In both, the global or macro-level functionality of the system emerges from the cooperative interactions between micro-level entities, the agents. It is mainly because the uniqueness of a generic mechanism of learning or selforganising is rarely admitted in the scientific community, the research in this field explores many different directions. This opinion is clearly stated by Minsky [24]: “I doubt that in any one simple mechanism, e.g., hill-climbing, will we find the means to build an efficient and general problem-solving machine. Probably, an intelligent machine will require a variety of different mechanisms. These will be arranged in

222

J.-P. Georgé et al.

hierarchies, and even in more complex, perhaps recursive, structures. And perhaps what amounts to straightforward hill-climbing on one level may sometimes appear (on a lower level) as the sudden jumps of “insight”.” But some others, like Inhelder [22], consider that a general theory of leaning might be feasible: “in the contrary to Chomsky’s assertion, who states that no general theory of cognitive learning is possible, we strongly believe that knowledge learning is to a very general process, be it logico-mathematical knowledge, physics, or natural language learning.” From the point of view exposed here, a general theory can not be tributary of constricting presuppositions like the knowledge of the global function to achieve, of specific attributes of the environment, or of an explicit environment-system feedback. This explains that the notions of emergence and self-organisation are deeply embedded in this work. Cooperative self-organisation does not need any presupposition about the finality of the system. Component agents only follow their own individual objective (to be the most cooperative possible) rather than try to adapt by individual learning to external perturbations. Even if such a system does not possess any programmed finality at the agent level (except being cooperative), selforganisation leads to the required collective result by emergence. The AMAS theory is a guide to design adaptive groups of agents in a simplified manner since designing parts of the system is of a constructive nature. Instead of starting from the global collective function and decomposing it in more elementary functions, we start by designing agents (their elementary functions) as well as the local criteria and behaviours that will guide their collective reorganisation. This is the detection and treatment of non-cooperative situations. This theory strongly relies on emergence: the agents learn in a collective and non-predefined way. They modify their organisation in relation to disturbing interactions with the environment and thus, their global function. The two illustrating examples show the usability and relevance of this approach. Quite a few other applications showed the adequacy of this emergent approach for managing adaptation and complexity in artificial systems, but there is still a lot of theoretical work to be done to explore its properties. For instance, we never seemed to observe local attractors. If this method of search space exploring seems unconcerned by a complex search space, it may be because the agents just ignore this search space and explore another one, the one constituted of the cooperative organisations of the system. But this will have to be proved. Another main objective is the diffusion of an AMAS design method among developers in both academic and industrial worlds. For this, readers can refer to Chap. 12, where the ADELFE method is presented, a toolkit to develop software with emergent functionality [2, 27]. Classic learning and adaptation techniques, which mostly have need of the knowledge of a cost function associated with the global function, cannot pretend to produce emergent phenomena. This is why they fit the scope of the limitations enunciated by the “no free lunch theorem” of Wolpert and MacReady [32]. This theorem stipulates that all search space exploration algorithms (both deterministic and stochastic), which make use of a search function to optimise a cost, globally have equivalent performances. Indeed, each one of them is only efficient given the bias introduced by the knowledge of the cost function. On a sufficiently vast corpus, they have the same performance: very efficient algorithms for a specific class

9 Cooperation

223

of problems are the worst in another, and those with less efficiency are nevertheless average for all classes. On the contrary, any effective theory of emergence does not fit in the scope of this theorem since, by principle, the function to reach is unknown, and so, there cannot exist an associated cost function. The AMAS theory could well be efficient for any class of problems. The glimpse of this possibility is one of the fascinating reasons to explore emergence in artificial systems and thus the ability to manage complex adaptive systems.

9.7 Problems and Exercises 9.1 Carrying heavy boxes. There are two sorts of boxes (light and heavy) lying around in a depot, and they need to be carried away. A human can only lift a light box, and a heavy box is two times the weight of a light one. (a) How can the problem be solved by cooperation (ok, this is really easy). Do the humans need to be able to communicate to cooperate? (b) We now have robots to carry the boxes but with the same limitations as the humans. On top of that, because of expenses, only basic perception means are installed on the robots (they only perceive the boxes, not the other robots), and no communication device is available. Is it still possible to solve the problem? What simple tweak to the behaviour of the robots would ensure that even the heavy boxes will be carried away given enough time? 9.2 Foraging Ants and Cooperation. When ants forage for food, they leave a chemical substance on the ground when returning to their nest carrying food. This substance is called pheromones and is a mark which other ants can detect. Pheromones can accumulate on a given spot or path and evaporate over the course of time. Readers can find plenty of on-line resources explaining how pheromones work in different species and comprehensive descriptions of ant behaviours. Readers can also take a look at the “Methodologies” chapter which uses an artificial ant foraging application as a case study. But a basic understanding is quite enough to tackle this exercise. (a) Explain how this use of pheromones can be qualified as “cooperative”. (b) Could the behaviour of natural ants be enhanced to be even more cooperative? Imagine designing an artificial ant for a simulation or building an ant-like robot using pheromone-like marks and basic perceptions. Describe a few enhancements to their basic behaviours (using pheromones or simple perceptions) which would make them more cooperative and thus produce better results. At least six enhancements can be found. 9.3 Cooperation in an Open System. Benevolent or selfless agents spontaneously cooperate whatever the cost to themselves. This is not the case for most agents in open systems.

224

J.-P. Georgé et al.

(a) What can two agents do to ensure that the cost of cooperation is evenly distributed? (b) What is one of the most efficient ways to have plenty of cooperation going on in an open system and still ensure that rogue or malevolent agents do not profit of the cooperative attitude of others? 9.4 Classifying Cooperation Means. List as many means as you can which enable cooperation. 9.5 Colour Cubes Game. Consider the following game: in a house with several rooms connected by doors there are cubes of different colours dispersed among the rooms and several robots. We want all the cubes of the same colour in the same room. A robot can carry up to four cubes and has three available actions: pick up cube, drop a cube, go to another room. There are no communication means. Your client wants to build a simulation which would solve the problem. For this, he asks you to design a multi-agent system in which each agent controls a virtual robot which can act once per simulation step. Describe the algorithm of the agents so that a solution is efficiently reached and does not depend on the number of robots, rooms, cubes or colours.

Key Points • Cooperation is inherent to social interactions, structures and organisations. • Cooperation is present in numerous natural systems, ranging from social insects to human societies. • Cooperation can take place in different forms and at different degrees. • Cooperation is a powerful mechanism to implement self-organisation in artificial systems.

References 1. Axelrod, R.: The Evolution of Cooperation. Basic Books, New York (1984) 2. Bernon, C., Camps, V., Gleizes, M.P., Picard, G.: Engineering self-adaptive multi-agent systems: the ADELFE methodology, pp. 172–202. Idea Group Publishing (2005). Chap. 7 3. Bonabeau, E., Dorigo, M., Théraulaz, G.: Swarm Intelligence: From Natural to Artificial Systems. Oxford University Press, London (1999) 4. Capera, D., Gleizes, M.P., Glize, P.: Mechanism type synthesis based on self-assembling agents. J. Appl. Artif. Intell. 18(9–10), 921–936 (2004) 5. de Rosnay, J.: The Macroscope: A New World Scientific System. Harper & Row, New York (1979) 6. Dorigo, M., Di Caro, G.: The Ant Colony Optimization Meta-Heuristic. McGraw-Hill, New York (1999) 7. Ferber, J.: Multi-Agent Systems: An Introduction to Distributed Artificial Intelligence. Addison-Wesley, Reading (1999)

9 Cooperation

225

8. Georgé, J.P., Gleizes, M.: Experiments in emergent programming using self-organizing multiagent systems. In: Multi-Agent Systems and Applications IV, 4th International Central and Eastern European Conference on Multi-Agent Systems, CEEMAS 2005, Budapest, Hungary, 15–17 September 2005. LNCS, vol. 3690, pp. 450–459. Springer, Berlin (2005) 9. Georgé, J., Gleizes, M., Glize, P., Régis, C.: Real-time simulation for flood forecast: an adaptive multi-agent system staff. In: Kazakov, D., Kudenko, D., Alonso, E. (eds.) Proceedings of the AISB’03 Symposium on Adaptive Agents and Multi-Agent Systems (AAMAS’03), SSAISB, University of Wales, Aberystwyth, pp. 109–114 (2003) 10. Georgé, J., Edmonds, B., Glize, P.: Making Self-Organizing Adaptive Multi-agent Systems Work, pp. 321–340. Kluwer Academic, Dordrecht (2004). Chap. 16 11. Gleizes, M.P., Camps, V., Glize, P.: A theory of emergent computation based on cooperative self-organization for adaptive artificial systems. In: Fourth European Congress of Systems Science, Valencia, Spain (1999) 12. Gleizes, M.P., Glize, P., Link-Pezet, J.: An adaptive multi-agent tool for electronic commerce. In: The Workshop on Knowledge Media Networking IEEE Ninth International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE 2000), Gaithersburg, Maryland (2000) 13. Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning. AddisonWesley, Reading (1989) 14. Grassé, P.P.: La reconstruction du nid et les coordinations interindividuelles chezbellicositermes natalensis etcubitermes sp la théorie de la stigmergie: Essai d’interprétation du comportement des termites constructeurs. Insectes Soc. 6(1), 41–80 (1959). doi:10.1007/BF02223791 15. Haken, H.: Synergetics: An Introduction. Springer, Berlin (1978), reedited (1983) 16. Heylighen, F.: Evolution, selfishness and cooperation; selfish memes and the evolution of cooperation. J. Ideas 2(4), 70–84 (1992) 17. Holland, J.H.: Adaptation in Natural and Artificial Systems. MIT Press, Cambridge (1975). 2nd edn. (1992) 18. Huberman, B.: The Performance of Cooperative Processes. MIT Press, Cambridge (1991) 19. Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983). doi:10.1126/science.220.4598.671 20. Lucidarme, P., Simonin, O., Liégeois, A.: Implementation and evaluation of a satisfaction/altruism based architecture for multi-robot systems. In: Proceedings of the 2002 IEEE International Conference on Robotics and Automation, ICRA 2002, 11–15 May 2002, Washington, DC, USA, pp. 1007–1012. IEEE Press, New York (2002) 21. Maturana, H.R., Varela, F.J.: Autopoiesis and Cognition: The Realization of the Living. Boston Studies in the Philosophy of Science. Springer, Berlin (1991). 2nd edn. (1980), 1st edn. (1973) 22. McCarthy Gallagher, J., Reid, K.: The Learning Theory of Piaget and Inhelder. Authors Choice Press, New York (2002) 23. McLaughlin, B.P.: Emergence and supervenience. Intellectica 2(25), 25–43 (1997) 24. Minsky, M.: Steps toward artificial intelligence. In: Feigenbaum, E.A., Feldman, J. (eds.) Computers and Thought, pp. 406–450. McGraw-Hill, New York (1963) 25. Picard, G.: Agent model instantiation to collective robotics in ADELFE. In: Gleizes, M.P., Omicini, A., Zambonelli, F. (eds.) Fifth International Workshop on Engineering Societies in the Agents World (ESAW’04), Toulouse, France, 20/10/04–22/10/04. LNCA, vol. 3451, pp. 209–221. Springer, Berlin (2004). http://www.irit.fr/ESAW04 26. Picard, G., Gleizes, M.P.: An agent architecture to design self-organizing collectives: principles and application. In: AISB’02 Symposium on Adaptive Multi-Agent Systems (AAMASII). LNAI, vol. 2636, pp. 141–158. Springer, Berlin (2002) 27. Picard, G., Gleizes, M.P.: The adelfe methodology: designing adaptive cooperative multi-agent systems. In: Bergenti, F., Gleizes, M.P., Zambonelli, F. (eds.) Methodologies and Software Engineering for Agent Systems. The Agent-Oriented Software Engineering Handbook, vol. 11. Kluwer Academic, Amsterdam (2004)

226

J.-P. Georgé et al.

28. Picard, G., Gleizes, M.P.: Cooperative self-organization to design robust and adaptive collectives. In: 2nd International Conference on Informatics in Control, Automation and Robotics (ICINCO’05), 14–17 September 2005, Barcelona, Spain, vol. I, pp. 236–241. INSTICC Press, Setubal (2005) 29. Vaughan, R., Støy, K., Sukhatme, G., Matari´c, M.: Blazing a trail: insect-inspired resource transportation by a robotic team. In: Proceedings of 5th International Symposium on Distributed Robotic Systems (2000) 30. Vaughan, R., Støy, K., Sukhatme, G., Matari´c, M.: Go ahead make my day: robot conflict resolution by aggressive competition. In: Proceedings of the 6th International Conference on Simulation of Adaptive Behaviour (2000) 31. Weiß, G.: Multiagent Systems, a Modern Approach to Distributed Artificial Systems. MIT Press, Cambridge (1999) 32. Wolpert, D.H., MacReady, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1, 67–82 (1997) 33. von Bertalanffy, L.: General System Theory. Braziller, New York (1968) 34. Wooldridge, M.: An Introduction to Multi-Agent Systems. Wiley, New York (2002) 35. Wu, B.: Manufacturing Systems Design & Analysis: Context & Techniques. Springer, Berlin (1994) 36. Zwirn, H.: Les limites de la connaissance. Éditions Odile Jacob, coll. Sciences (2000)

Chapter 10

Immune Systems Vincent Hilaire, Abderrafiâa Koukam, and Sebastian Rodriguez

This chapter introduces an architecture for MAS which is based upon a specific interpretation of the human immune system.

Objectives After reading the immune systems chapter the reader will: • Understand what aspects of immune systems are relevant for self-organising software; • Understand technical and theoretical issues which are important for using such systems in various applications.

10.1 Introduction Complex systems are characterised by a large number of interacting components. The behaviour of the system as a whole is the result of nonlinear aggregation of local behaviours of these components. Multi-Agents Systems (MASs) (see Chap. 5) have become a natural tool for modelling, simulating and programming complex V. Hilaire () · A. Koukam UTBM, Belfort, France e-mail: [email protected] A. Koukam e-mail: [email protected] S. Rodriguez CITAT, Universidad Tecnológica Nacional—Facultad Regional Tucumán, San Miguel de Tucumán, Argentina e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_10, © Springer-Verlag Berlin Heidelberg 2011

227

228

V. Hilaire et al.

systems. Indeed, MASs are composed of autonomous, reactive, proactive and interacting entities termed agents, which are engaged in the realisation of a joint goal. Both types of systems are studied in respect with their organisation dynamics and the emergence of organisational structures. However, in Complex Systems we usually find a large number of entities in interaction, acting at different levels of abstraction. Analysis and modelling techniques able to represent several levels of abstraction and computation models that are capable to self-organise and adapt to environmental adversities are needed to overcome the large entity numbers issue. Among the possible solutions, Holonic Multi-Agent Systems (HMAS) seem to be a promising paradigm (see Chap. 11). HMAS are based upon self-similar entities, termed holons, which define an organisational structure termed holarchy. HMAS have shown to be a convenient way to engineer complex and open systems in multiple application domains. Artificial Immune Systems (IS) appear in this area as an auspicious selforganising mechanism for MAS (see Chap. 4). The advantages of immune systems in this case are twofold. Firstly, immune systems have learning capabilities, and they are able to adapt to environmental changes. Secondly, immune systems can be easily used as a recursive scheme for decomposing complex decision-making problems. Indeed, an immune system is an entity that can be considered as a whole, but it is composed from smaller entities, in our case antibodies. The immune system that served as an inspiration for this work is based upon the Jerne’s Idiotypic Network [8]. The metaphor used considers stimulations/requests as antigens and selected antibodies as reactions/answers. This mechanism enables agents to select among a set of possible behaviours the one which seems the most suitable for a specific context. Each agent has a set of antibodies, representing behaviours, designed to respond to specific antigens representing stimuli. There are numerous advantages of this approach. First of all, agents maintain their autonomy, and the coordination towards the accomplishment of goals is ensured by utilising the right stimuli. Furthermore, MAS can pursue several, possibly interdependent, goals at the same time. Finally, similarly to the immune system, learning is supported by means of a reinforcement learning mechanism. The natural immune system has received significant research interest because it provides a powerful and flexible information processing capability as a decentralised intelligent system. The immune system provides an excellent model of adaptive operation at local level and of emergent behaviour at global level. There exist several theories for explaining immunological phenomena and numerous software models for simulating various components of the immune system [10]. In this chapter we explore how the artificial immune system can be applied to MAS as a self-organising mechanism. Moreover, we introduce an organisationbased model aiming to use IS as a multi-layer self-organising mechanism. We achieve this by defining a MAS architecture based on the IS concepts and model. As this architecture can be recursive, it can be compared to the Holonic MAS architecture described in Chap. 11.

10

Immune Systems

229

10.2 Background Context The human immune system is composed of several layers of defence. The immune system protects organisms from infection with layered defences of increasing specificity. Most simply, physical barriers prevent pathogens such as bacteria and viruses from entering the organism. If a pathogen breaches these barriers, the innate immune system provides an immediate, but non-specific response. Innate immune systems are found in all plants and animals. However, if pathogens successfully evade the innate response, vertebrates possess a third layer of protection, the adaptive immune system, which is activated by the innate response. Here, the immune system adapts its response during an infection to improve its recognition of the pathogen. This improved response is then retained after the pathogen has been eliminated, in the form of an immunological memory, and allows the adaptive immune system to mount faster and stronger attacks each time this pathogen is encountered.1

In this chapter, we are specifically interested in the third layer, namely the Adaptive Immune System (AIS). It is the layer that handles antigens, which are considered as substances that bind to specific antibodies and exhibit self-organisation features. Indeed, the adaptive immune system has four key functionalities: recognition, adaptation, learning and memory. The adaptive immune response recognises specific antigens and generates responses that are tailored to specific pathogens. The ability to exhibit these tailored responses is maintained in the body by “memory cells”. Should a pathogen infect the body more than once, these specific memory cells are used to quickly eliminate it. The cells of the adaptive immune system are special types of leukocytes, termed lymphocytes. B cells and T cells are the major types of lymphocytes, and they are derived from cells in the bone marrow. Both B cells and T cells carry receptor molecules which recognise specific targets. T cells recognise a “non-self” target, such as a pathogen, only after appropriate antigens have been processed. B cells have an antigen-specific receptor: an antibody molecule on the B cell surface. Each type of B cell corresponds to a different antibody, and hence the complete set of B cell antigen receptors represent all antibodies that the body can manufacture. Antibodies recognise and bind to specific types of antigens, namely foreign substances invading a human body. The key part of an antigen recognised by an antibody is termed epitope, which is the antigen determinant. Paratope is the antibody part that corresponds to a specific antigen type. Once an antibody matches with an antigen via their epitope and paratope, killer T cells start eliminating the antigen. Recognition Figure 10.1 illustrates the antigen recognition mechanism. An antibody Ab manufactured by a B-cell recognises and binds to an antigen Ag if Ag matches Ab’s structure. The region of antibodies which matches antigens is termed paratope. The counterpart region on antigens is termed epitope. In Fig. 10.1 antibody Ab1 recognises only antigens similar to Antigen 1. 1 Article

from Wikipedia http://en.wikipedia.org/wiki/Immune_system.

230

V. Hilaire et al.

Fig. 10.1 Antibody and antigen recognition and binding mechanism

Adaptation The immune system continuously generates new sort of antibodies by cloning and muting existing ones. The goal is to produce antibodies that will match antigens. It is the second functionality of the immune system, adaptation. If new antigens appear, the immune system may be capable of producing matching antibodies. Memory When an antibody matches an antigen Agi , it proliferates in order to bind to the antigen. This selection mechanism augments the concentration of effective antibodies. Even if all Agi antigens are destroyed, the immune system is able to keep some antibodies that could destroy them for some time. This is the third functionality of the immune system: memory. If Agi antigens reappear, matching antibodies may already exist. Learning Immune systems continuously learn from the results of past behaviours. Their learning mechanism can handle noise, it is inherently unsupervised, and it does not require negative examples to function. Furthermore, an important advantage is that what is learnt is explicitly represented. From a computing perspective the human immune system can be viewed as a parallel, distributed and decentralised system, which has the capability to control a complex system over time [4]. Mobile agents (B/T cells) permanently roam the system for detecting and destroying antigens. Several immune system models have been originated from these general concepts, a number of which is described in the following subsections.

10

Immune Systems

231

10.2.1 Jerne Idiotypic Network Among the numerous theories which try to explain the human immune system, Nobel Laureate N.K. Jerne proposed a model for immune system regulation based on communications between antibodies [8]. These communications take the form of stimulation and inhibition. That theory is known as Jerne’s Idiotypic Network. The network is defined by stimulation/inhibition links between antibodies. The region in which antibodies stimulate or inhibit other antibodies is termed idiotope. It is idiotopes that play the roles of antigens for other antibodies. The network is termed idiotypic network, and it is built on the basis of idiotope recognition with the stimulation and suppression chains among antibodies. This mechanism enables AIS to regulate antibodies as if they were antigens, to maintain an effective set of cells and to self-organise to deal with antigens.

10.2.2 Other Models Another well-known model of AIS is the negative selection algorithm of Forrest [6]. That algorithm aims at detecting foreign (non-self) antigens. It generates detectors randomly, and it eliminates the ones that detect self enabling the remaining T-cells to detect any non-self. This maturation process is done through a genetic rearrangement mechanism. This negative selection algorithm was used for several applications such as tool break point detection [3] and anomaly detection in datasets [2]. Another approach [5] consists of exploiting the learning capabilities of an immune system which is aware of neither all antibodies nor all antigens. However, the immune system is able to associate an answer to each encountered antigen. This type of approach was often compared to classifier systems [1], and it was applied to pattern recognition [5]. In this chapter we are focusing on the idiotypic network since it enables maintaining behaviours capable of handling situations modelled by antigens. For a detailed survey on AIS models, the interested reader can refer to [2].

10.3 Theoretical Notions 10.3.1 Computational Model of Immune System An AIS has to reproduce the four main characteristics of natural immune systems which are: recognition, adaptation, learning and memory. These four aspects can be modelled in different ways [2, 5, 6]. This chapter details the idiotypic network and its links with agents and multi-agent systems. The first step before modelling any of the above-mentioned natural IS characteristics is to analyse the problem at hand and decide of the following issues:

232

V. Hilaire et al.

Table 10.1 Antibody description Paratope

Agent specification

Idiotope

Precondition under which this B-agent is stimulated

Attributes, codes, data, behaviour and intern idiotypic network

References to stimulating Bagents and the degree of the stimuli (affinity)

• What will be the environment? • What are the antigens? This refers to the phenomenon (collective behaviour) associated with antigens. Furthermore, how are antigens encoded? • What perception functions will feed the system with antigens? • What actions constitute the answers (global goals) of the system? How are they encoded? The following subsection provides a description of an agentified idiotypic network and a framework aiming to provide answers to the above questions.

10.3.2 Artificial Immune System Architecture The Jerne’s idiotypic network has already been implemented as agent architecture, see for example [11]. We use the concepts concerning the single agent case that are described in [11] as a basis for the approach presented in this chapter. First of all, antibodies are represented by agents.2 This analogy is depicted in Table 10.1. An antibody is divided into three parts. Precondition The first part is the precondition. It describes the circumstances under which an antibody is stimulated. This refers to the context in which an antibody can execute its associated behaviour. This is an analogy to the real antibody paratope which tries to match antigen epitopes to recognise them. The preconditions part is linked to agent perceptions, and it determines the agent stimulation conditions with respect to agent internal state and the state of the environment. This part is commonly implemented by matching detector strings for instance, and by requiring a specific code for the representation of the pertinent characteristics of the environment. A stimulation condition is more general, and it can be easily applied in problems where predicates can be defined for the environment. Behaviour The second part specifies the antibody behaviour, which is executed when the antibody is selected. The behaviour of the real antibody is to contribute to the elimination of antigens. 2 This

is a design choice made for simple agents. If agents are complex one can make the choice of modelling part of the agent behaviour as an antibody.

10

Immune Systems

233

Fig. 10.2 Idiotypic network

Idiotope The third part is composed of references to other antibodies associated with different degrees of stimulation (affinities). The functionality offered by the idiotope part enables the antibody to be recognisable and carry out interactions (stimulation/inhibition) with other antibodies. For the agent case, this part can be modelled using a mapping from antibodies to numeric values, positive for stimulation and negative for inhibition. The system initially starts with no affinities as these values are learned at runtime. According to Jerne’s theory, antibodies consider other antibodies as antigens, something which enables antibody regulation. A Jerne’s idiotypic network is then defined by a number of antibodies and their respective affinities. An antibody paratope is recognised by epitopes (stimulation and increase of concentration) of other antibodies. Furthermore, an antibody is removed from the system when its own epitope is recognised by other antibodies (inhibition and decrease of concentration). Figure 10.2 shows an example of an idiotypic network. Let us examine Antibody i at the centre of the figure. The antibodies at the left part of the figure recognise Antibody i, namely their paratope recognises Antibody i’s epitope, and their epitope is recognised by Antibody i’s paratope). The antibodies at the right part of the figure are those who are recognised by Antibody i as antigens, namely their epitope is recognised by Antibody i’s paratope. Thus, Antibody i is stimulated, or inhibited, by Antibody 1 to Antibody N , and in turn it stimulates, or inhibates, Antibody 1 to Antibody M. The value mj i denotes the amount of stimulation/inhibition of i by j , while value mik represents the amount of stimulation/inhibition of k by i. These values are termed “affinities” or “affinity values”, and they can change over time. Furthermore, at any time, new antibodies stimulating or inhibiting Antibody i can appear, and existing antibodies can disappear from the system. The state of a Jerne’s idiotypic network at any point in time is a directed graph with nodes representing antibodies and labelled arcs representing affinity values.

234

V. Hilaire et al.

10.3.3 Artificial Immune System Dynamics The antibody population is represented by the concept of concentration. For any antibody type, the higher the concentration, the larger the number of corresponding antibodies. The concentration is the result of the immune system self-organisation actions including natural death and cell interactions, as well as antigen stimulations. In [4] authors propose equations to compute antibody concentrations. The concentration of the ith antibody, denoted by ai , based on its stimulation/inhibition links is defined by N M dAi (t) 1 1 mj i aj (t) − α mik ak (t) + βmi − ki ai (t). (10.1) = α dt N M j =1

k=1

In (10.1), the first and second right-hand side terms denote the weighted sums of stimulations and inhibitions that Antibody i has received from other antibodies. Values mj i and mik are the affinity values which are positive values between 0 and 1. The weights used are computed from the number of stimulating and inhibiting antibodies. N is the number of stimulating antibodies, and M is the number of inhibiting antibodies. The third term, mi , is 1 when antibody i is stimulated directly by an antigen and 0 otherwise. The forth term, ki , denotes the dissipation factor representing the antibody’s natural death. The two coefficients α and β are used to weight antibody stimulation/inhibition against antigen stimulation. These two coefficients are application-specific. ai (t) =

1 . 1 + exp(0.5 − Ai (t))

(10.2)

Equation (10.2) is the function used to normalise the parameter Ai (t + 1), calculated by (10.1), between 0 and 1. In this model not all antibodies are active at the same time. The selection of the active antibody, the one that will execute its behaviour, depends on other antibody concentrations. Different strategies can be used for active antibody selection, for example selecting the antibody with higher concentration or the one with concentration probabilistically proportional to the current concentration (a strategy termed “roulette-wheel selection”). Learning The system initiates from a state where antibody affinities, forming antibody idiotopes, are undefined. The process of construction of these affinities is based upon learning mechanisms such as reinforcement learning. Idiotopes are progressively computed using reinforcement signals. The idiotypic network structure rearranges itself on run-time by changing the affinity values. This is achieved by having affinity values initialised to 0 and subsequently updated, using reinforcement learning, after each execution of an antibody behaviour. The antibody selected to carry on its behaviour is monitored, and the results of its actions are analysed. If the analysed results constitute a correct answer to the stimulations received, then the antibody/agent is rewarded. In contrast, if the analysed results do not correctly

10

Immune Systems

235

Fig. 10.3 Antibody behaviour

respond to the received stimulation, the agent/antibody is penalised. This process constructs the idiotypic network. The affinity m12 between antibody 1 and antibody 2 can be calculated from the following equation: m12 =

TpAb1 + TrAb2 Ab

TAb12

,

(10.3)

where: • TpAb1 is the number of times penalty reinforcement signals were received when Ab1 was selected. • TrAb2 is the number of times reward reinforcement signals were received when Ab2 was selected. Ab2 is the number of times both Ab1 and Ab2 have reacted to specific antigens. • TAb 1 This approach to affinity calculation models the fact that immune networks learn from the results of their past behaviour.

10.3.4 Agent Behaviour and Interactions We have identified two agent roles which are required for implementing AIS behaviour. The first role corresponds to antibody behaviour and is represented by the statechart shown in Fig. 10.3. The second role encapsulates network management behaviour, and it is modelled with the statechart presented in Fig. 10.4. The statechart of Fig. 10.3 is composed of two parallel states which are simultaneously active. The first upper state, above the dashed line, specifies the behaviour

236

V. Hilaire et al.

Fig. 10.4 Idiotypic network behaviour

of the role concerning antigen stimulation of other antibodies. Antibodies are idle by default. If an antibody detects and is stimulated by an antigen that matches its paratope, then it becomes active. This matching process is application-specific. It depends on the analysis of the environment and the truth value of the antibody’s precondition. In the activated state an antibody can be either fully stimulated by other antibodies, in which case it computes a new concentration value as described in (10.1), or it can have only received reinforcement signals, and in that case it computes affinities as described by (10.3). If an activated antibody perceives an antigen which does not match its paratope, then it returns to the idle state. The second parallel state, below the dashed line, concerns the selection process. By default, antibodies are not selected. When the selection signal is broadcasted with the identity of the winner, the winner becomes executable and fires its behaviour. If an antibody was executing its behaviour and receives a selection signal with a non-self identity, then it becomes idle. The statechart of Fig. 10.4 is composed of three states, with the idle state being the default. When a change event is detected in the idiotypic network, for example due to an antibody activation, the respective antigen is broadcasted, and the idiotypic network waits for the antibodies to self-organise. This means that the antigen broadcast triggers a reorganisation of antibody concentrations, which is carried out by antigen stimulation and subsequent inhibition/stimulation of the available antibodies. After a specific time period, a winner is chosen among the selected antibodies, and an appropriate winner signal is sent. The idiotypic network then waits for execution of the winner antibody behaviour. After a certain time, the idiotypic network analyses the results of the winner antibody actions and sends it appropriate reinforcement signals. Figure 10.5 details the interactions resulting after perception of an antigen by the immune system. The perception and encoding of antigens is done by analysis of the environment. The perceived antigens are sent to all antibodies in the network. Each antibody checks if it is stimulated by the particular antigen, and if so, it broadcasts its affinity values. By doing so, it stimulates or inhibits other antibodies. These stimulations/inhibitions correspond to the first and second components of (10.1) respectively, and they are used by antibodies to compute their concentration. The degree of stimulation an antibody receives from an antigen is modelled as mi in the third component of that equation. Subsequently, antibodies send their concentrations to the immune system to carry out the winner selection process. The antibody with the greater concentration is chosen and executes its behaviour. The execution

10

Immune Systems

237

Fig. 10.5 A complete step of the immune system interactions

results are analysed by the immune system, which in turn sends reward or penalty reinforcements. Antibodies then utilise the received reinforcements to update their affinities based on (10.3).

10.3.5 From Agent to Multi-Agent The idiotypic network model presented in previous sections enables selection and execution of only one agent behaviour at a time. However, this restriction to a single executing behaviour is a significant limitation since many real problems need parallel execution of several behaviours to be optimally, or even feasibly, solved. An approach to overcome this issue is to use an idiotypic network as part of the agent architecture. In other words, agents will integrate an idiotypic network, and antibodies will represent possible agent actions or behaviours. The idiotypic network will arbitrate among these possible actions/behaviours. Agents designed following this approach will exhibit a self-organising architecture. A problem raised in such a solution is that agent interactions will not be taken into account and selforganisation will occur at agent level instead of system level.

238

V. Hilaire et al.

Fig. 10.6 Holarchy with an idiotypic network

Another approach involves simultaneously using both the holonic (see Chap. 11) and the AIS paradigms in MAS design (see Chap. 5 for more details on MAS). As detailed in Chap. 11, a holon is a self-similar structure composed of other holons as sub-structures. Such a hierarchical structure of holons is termed holarchy. A holon can be seen, depending on the level of observation, either as an autonomous “atomic” entity or as an organisation of holons. AIS can be easily used inside holarchies since they have the same duality as holons. Indeed, an immune system is an entity that can be considered as a whole but is composed of smaller entities, in our case antibodies. Therefore, we can apply this recursive scheme and draw a parallel between an immune system and its principles of operation and a holarchy and its associated decision-making mechanism. Figure 10.6 presents a three-level holarchy. Each holon consists of sub-holons playing the two roles defined in the previous sub-

10

Immune Systems

239

sections, namely the immune network and the antibody roles. A holon plays the antibody role with respect to its upper-level and the idiotypic role with respect to its lower-level. In other words, a holon represents behaviour which results from interactions of behaviours at its lower-level. In that architecture the selected antibody provides a result that will be fed into the lower-level immune system as an antigen. The next section presents an application of an idiotypic network coupled with a holarchy which addresses the problem of managing a robot soccer team playing a robot soccer game.

10.4 Applications 10.4.1 Robot Soccer The FIRA Robot soccer competitions began in 1996 using real robots and simulators [9]. Robot soccer is an example where real-time coordination is needed. Indeed, the principle is that two teams of autonomous robots play a game similar to human football. It constitutes a benchmark for several research fields, such as MAS, image processing and control. The architecture used for controlling the robot team is based upon idiotypic networks and holons. Specifically, a three-layer holarchy has been designed to coordinate a robot soccer team. Each layer handles a different level of abstraction, as shown in Fig. 10.7. The behaviour of each layer is described below. Strategy Selection Firstly, the high-level immune system, termed team level, decides on a team strategy and assigns appropriate roles to robots based on the current game state. These roles (such as defender, attacker and midfielder) are defined by the strategy followed by the team as a whole. That immune system is then embedded in each team holon. Role Selection Secondly, the medium level immune system handles robot roleplaying. It computes a target point for every robot based on the role assigned to the robot by the high-level immune system. Robot Trajectories Thirdly, the low-level immune system computes the robot trajectories. The use of an immune system in the above holonic levels enables each robot team to adapt to environmental changes, from ball and opponent movements to opponent strategies. Such changes to environmental states are interpreted as antigens. The immune system learns in time how to select a suitable behaviour for each particular context. That behaviour is part of the selected antibody. Furthermore, each antibody represents a specific strategy. In the above model, new behaviours can be generated to answer new needs and regenerate the antibody populations. Furthermore, even if the general structure of

240

V. Hilaire et al.

Fig. 10.7 Holarchy with idiotypic network for robot soccer control

the holarchy remains the same, its composition in terms of holon behaviours changes according to the immune system dynamics. These changes can be considered as reorganisations.

10.4.2 High-Level Immune System The above described high-level network has been designed to coordinate a team of robots in a robot soccer environment. This means that the network has to choose a suitable strategy and to assign the roles defined by that strategy to the robots. The choice of strategy depends on the current game state. The main concepts of this immune network are described below.

10

Immune Systems

241

Fig. 10.8 Antigen example: which team is in control of the ball

Fig. 10.9 Antigen example: at which zone the ball is located

Antigen The game state obviously defines the environment of this idiotypic network. The environment is defined by several parameters which act as antigens or antibody preconditions. The first parameter defines which team controls the ball (see Fig. 10.8). It is determined by the robot which is closest to the ball. The team of the robot closest to the ball is the one that is in control of the ball. The second parameter represents the zone of the field the ball is (see Fig. 10.9). To reduce the set of preconditions, the field has been split into nine equal zones. The high-level immune system assigns roles to robots according to the current game state and the chosen strategy. As described above, each robot plays its own role such as attacker, goalkeeper and defender for instance. The concept of zone is used to assign the appropriate role to each robot. For example, {goalkeeper, neardefender, midfielder, left-attacker, right-attacker} and {goalkeeper, near-defender, near-defender, far-defender, midfielder} are two strategies for five-robot teams. The

242

V. Hilaire et al.

Table 10.2 High-level immune system antibody example Precondition

Behaviour

TeamInControl and (BallZone = 1 or BallZone = 2 or BallZone = 3)

goalkeeper, near-defender, midfielder, left-attacker, right-attacker

Affinities

Fig. 10.10 Example of strategy roles

first strategy is oriented towards attack, and the second one towards defence. The duplicate role, near-defender, in the second strategy means that two robots will play this role. Figure 10.10 illustrates another example of strategy. Adaptation for this idiotypic network may consist in generation of new strategies. Learning A strategy can be evaluated with criteria such as number of goals taken, number of goals scored and team ball possession time. A particularly important parameter is the inter-evaluation time, that is the time between states waitingSO and analyseAction depicted in Fig. 10.4. Furthermore, for strategy evaluations to be meaningful, their duration should be minimal. An example of a high-level immune system antibody is presented in Table 10.2. In that example, the precondition takes into account two of the aforementioned evaluation parameters. The first one concerns the team that controls the ball, and the second one tracks the ball position on a nine-zone square grid. The network behaviour defines an attack strategy with three roles dedicated to attack. Antibody affinities are computed using a reinforcement mechanism.

10.4.3 Medium-Level Immune System The main concepts of the medium-level immune system are antigen, adaptation and learning, which are described below. Antigen For each robot, the medium level system takes as input: (1) a role produced by the high-level immune system and (2) the game context. The game context consists of all robots and ball positions.

10

Immune Systems

243

Fig. 10.11 Examples of medium-level antibodies

Table 10.3 Medium-level immune system antibody example

Precondition

Behaviour

Defence-far & ball-ahead & nohelp

NorthEast

Affinities

Roles define general behavioural principles. For example, the purpose of the goalkeeper role is to prevent opponent robots from scoring goals. This role consists of trying to move so that the robot be on the ball trajectory. Roles act as antigens for antibodies, and hence after execution of each role behaviour there may be several stimulated antibodies. The behaviour of each antibody basically involves specification of an aim point. This is because in the robot soccer game, robots are not supplied with actuators, and their actions can only be movements in a way to either push the ball or block an opponent. Each aim point defines a trajectory linking it with the current robot position, and it is provided as input to the lower-level immune system of the respective robot. Adaptation The selected approach involves generating a fixed number of antibodies. For example, Fig. 10.11 describes an immune system consisting of eight antibodies. Each antibody defines a direction, and its aim point is generated based on a circle centred on the robot and having a given radius. For the defence-far role, there are eight antibodies representing different trajectories. If the defence-far role is assigned to a robot, then one of these trajectories will be selected by the medium level immune system. The role of that robot will then consist of moving to the aim point defined by that antibody. To differentiate between game contexts, robot and ball positions are considered. In particular, this differentiation enables determining the ball position relative to each robot, and the possible assistance that each robot could receive from partner robots in the particular game context. The antibody behaviour presented in Table 10.3 corresponds to Ab2 in Fig. 10.11. The antigen part is an “and conjunction” meaning that all antigens must be present simultaneously to stimulate an antibody. Learning Several aim points are generated for each role, and one of them is selected by the medium-level network. Subsequently, the result of the corresponding

244

V. Hilaire et al.

Fig. 10.12 Goalkeeper reinforcement criterion

antibody execution is analysed, and the fitness of the selected action is evaluated. Based on that evaluation an appropriate reward or penalty, reinforcement is assigned to the respective antibody. The evaluation criteria and the appropriate reinforcement level depends on the role actively pursued. For example, different evaluation criteria and reinforcement levels should be used to assess and characterise defender and attacker role behaviours. For the goal-classic role, the criterion shown in Fig. 10.12, where the idea of rr maintaining a ddblrl = ddbr relationship was considered. This relationship requires only the robot goal and current ball positions to be calculated, and therefore it enables robots to be on the ball trajectory without having to be aware of actual, possibly complex, ball movements. If the relationship is satisfied, then the selected antibody receives a reward reinforcement signal, otherwise the antibody receives a penalty signal. These signals are required for computing affinities as defined in (10.3). Various other more “aggressive” roles considering additional details can be similarly defined.

10.4.4 Low-Level Immune System Antigen The low-level idiotypic network implements primitive robot capabilities such as movement to selected positions and collision avoidance. Robots perceive information about their environment, and hence they can detect obstacles. In this study, the low-level immune system antigens represent a robot aim point, generated by the medium level IS, and the different obstacles, such as other robots and robot area boundaries. For simplicity, movement directions are categorised as front, right, left and back. Preconditions are defined as {ObstacleFront, ObstacleLeft, ObstacleRight, ObstacleBack, NoneObstacle}. The direction towards the aim point is encoded as {AimFront, AimLeft, AimRight, AimBack}. Finally, the behaviours corresponding to the

10

Immune Systems

Table 10.4 Low-level immune system antibody example

245 Precondition

Behaviour

ObstacleFront & AimRight

MoveRight

Affinities

Fig. 10.13 Example of trajectory computation

ObstacleFront precondition are defined as {MoveLeft, MoveRight, MoveBack} (see Table 10.4). Adaptation It is possible to easily add or remove antibodies from this immune system to improve or restrict robot behaviour. Learning In this immune system antibody affinities are computed using the respective reinforcement mechanism when the system evolves. Robots receive reward signals when their current distance to their aim point, measured as a hamming metric, is shorter than the respective distance that was measured in the previous system execution step. Furthermore, robots receive penalty signals either when they collide with an obstacle or when their distance to their aim point is longer than the respective distance they had in the previous execution step. Figure 10.13 shows an example of such a robot movement trajectory. An obstacle, consisting of another robot, is placed between the selected robot and its aim point. The selected robot has therefore to move to avoid that obstacle.

10.4.5 The Simulator A robot soccer simulator and an immune system API has been developed using the MadKit platform [7]. The interface of that simulator during a game is shown in Fig. 10.14. The developed simulator enables testing of various immune systems and particularly those described in the previous sections.

246

V. Hilaire et al.

Fig. 10.14 Robot soccer simulator

10.5 Conclusion In this chapter we have presented Adaptive Immune System concepts that can be utilised to design adaptive software systems based on software agents. To this purpose, the Idiotypic Network immune system theory of Jerne [8] was selected due to its wide applicability and described in detail, together with techniques for applying this theory for the design of agent-based Artificial Immune Systems. Major advantages of the proposed approach are the offered self-organisation and learning capabilities. A considerable limitation of that approach is that only one behaviour can be executed at a time. To address this issue, possible solutions were discussed, and an improved solution approach, based on a multi-level architecture tailored for Immune Systems and taking inspiration from holonic systems, was introduced and described in detail. Subsequently, a case study illustrating application of the holonic immune systems approach to the robot soccer problem domain was described. More specifically, the presented exemplar application involves an agent-based software system managing a robot soccer team. The approach comprises different immune system levels, each dedicated to a specific objective, for example the upper immune system level selects the overall playing strategy of the robot soccer team. A significant part of the proposed architecture is application-dependent, and each time it has to be adjusted for the application on hand.

10

Immune Systems

247

Fig. 10.15 Example of environment for the Predator/Prey problem

10.6 Problems–Exercises 10.1 This problem is inspired from the well-known predator–prey pursuit. In a closed digital environment we have four predators and a prey. The environment can be represented as a two-dimensional matrix as depicted in Fig. 10.15. Each upper-case P represents a predator. The lower-case p represents the prey. In this environment predators and prey can freely move to neighbouring cells in all directions, namely up, down, left and right. The goal of the prey is to escape predators, and the goal of the predators is to capture the prey. This goal is reached when the prey is surrounded by predators and has no escape move possible. In this context we want to define an immune system architecture for the predators. A “good” architecture should enable the predators to collaborate in order to capture the prey. (a) Define suitable antigens describing the different situations that may occur in the described environment. Antigens should take into account the positions of the different actors of this problem. (b) Define different antibodies for the predators. Each antibody should define a behaviour. (c) Propose a reinforcement function. This function should reward behaviours that lead to the prey capture or at least to a better situation for the predator. 10.2 Transportation on demand (TOD) is concerned with transportation of passengers and goods on specific routes at user request. Common TOD examples include dial-a-ride transportation services for the elderly and the disabled, urban courier services, aircraft sharing, and emergency vehicle dispatching. In all such systems, users formulate transportation requests from an origin point to a destination point. These requests are then served by capacitated vehicles, which often provide a shared service in the sense that several passengers or goods may be carried by a vehicle at the same time.

248

V. Hilaire et al.

We focus on passenger transportation by a set of vehicles of limited capacities (from 2 to 7). We assume that user requests are served by an immune system composed of two levels. The top/strategic level handles long-term strategic vision and global answers to requests. The bottom/vehicle level is encapsulated by each vehicle. (a) Define suitable antigens describing the possible user requests. (b) Define suitable antibodies for vehicles and the strategic level itself. (c) Propose reinforcement functions for both levels.

Key Points • Idiotypic networks designed as agent architecture can be used for engineering self-organising software; • Combining the holonic metaphor with the immune system theory enables concurrent selection and execution of agent behaviours; • The main components of an agent-based Artificial Immune System are antigens, roles and reinforcement mechanisms.

10.7 Further Reading Proceedings of the International Conference on Artificial Immune Systems. This conference is dedicated to artificial immune systems, all models are presented and discussed. Published in the LNCS series Springer since 2001. Theoretical advances in artificial immune systems. Theoretical concepts related to AIS. (J. Timmis, A. Hone, T. Stibor and E. Clark. Theoretical Computer Science 403(1), 2008.) Artificial immune systems—today and tomorrow. Insights about research on AIS past and future. (J. Timmis. Natural Computing 6(1), 2007.) Artificial immune systems as a novel soft computing paradigm. How AIS can be used to solve difficult problems. (L. Nunes de Castro and J. Timmis. Soft Computing 7(8), 2003.) That special issue includes additional relevant papers as well.

References 1. Bersini, H., Varela, F.J.: Hints for adaptive problem solving gleaned from immune networks. In: PPSN, pp. 343–354 (1990) 2. Dasgupta, D., Attoh-Okine, N.: Immunity-based systems: a survey. In: Proc. of the IEEE Int. Conf. on Systems, Man and Cybernetics. IEEE Press, Piscataway (1997). citeseer.ist.psu.edu/ dasgupta97immunitybased.html

10

Immune Systems

249

3. Dasgupta, D., Forrest, S.: Tool breakage detection in milling operations using a negativeselection algorithm. Tech. Rep. CS95-5 (1995) 4. Farmer, J.D., Packard, N.H., Perelson, A.S.: The immune system, adaption and machine learning. Physica D 22, 187–204 (1986) 5. Forrest, S., Javornik, B., Smith, R.E., Perelson, A.S.: Using genetic algorithms to explore pattern recognition in the immune system. Evol. Comput. 1(3), 191–211 (1993). doi:10.1162/ evco.1993.1.3.191 6. Forrest, S., Perelson, A., Allen, L., Cherukuri, R.: Self-nonself discrimination in a computer. In: Proceedings IEEE Computer Society Symposium on Research in Security and Privacy, pp. 202–212 (1994) 7. Gutknecht, O., Ferber, J.: Madkit: a generic multi-agent platform. In: Sierra, C., Gini, M., Rosenschein, J.S. (eds.) Proceedings of the Fourth International Conference on Autonomous Agents, Barcelona, Catalonia, Spain, pp. 78–79. ACM, New York (2000) 8. Jerne, N.: Towards a network theory of the immune system. Ann. Inst. Pasteur., Immunol. 125C, 373–389 (1974) 9. Kim, Y.H.: Micro-robot world cup soccer tournament. KAIST (1996) 10. Suzuki, J., Yamamoto, Y.: A decentralized policy coordination facility in openwebserver. In: Proceedings of SPA2000 (2000) 11. Watanabe, Y., Ishiguro, A., Uchkawa, Y.: Decentralized behavior arbitration mechanism for autonomous mobile robot using immune system. In: Books Artificial Immune Systems and Their Applications, pp. 186–208. Springer, Berlin (1999). ISBN 3-540-64390-7

Chapter 11

Holonic Multi-Agent Systems Sebastian Rodriguez, Vincent Hilaire, Nicolas Gaud, Stephane Galland, and Abderrafiâa Koukam

This chapter presents holonic concepts and how they can be used to model MAS with multiple granularity levels.

Objectives After reading this chapter the reader will: • Understand what holons and holarchies are and on what type of applications they can be successfully applied; • Have obtained an overview of the primary issues concerning engineering of holonic self-organising systems; • Be aware of an engineering framework which eases the analysis, design and deployment of such systems.

S. Rodriguez CITAT, Universidad Tecnológica Nacional—Facultad Regional Tucumán, San Miguel de Tucumán, Argentina e-mail: [email protected] V. Hilaire () · N. Gaud · S. Galland · A. Koukam UTBM, Belfort, France e-mail: [email protected] N. Gaud e-mail: [email protected] S. Galland e-mail: [email protected] A. Koukam e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_11, © Springer-Verlag Berlin Heidelberg 2011

251

252

S. Rodriguez et al.

11.1 Introduction In the physical world, Complex Systems (CS) are characterised by large numbers of interacting entities exhibiting self-organisation features and emergent behaviours (see also Chap. 3). Taking inspiration from physical complex system operation and properties, an important research issue has long been concerned with the design of software featuring controlled complex system properties. Complex systems are characterised by large numbers of interacting entities exhibiting self-organisation features and emergent behaviours (see also Chap. 3). In this respect, Multi-Agent Systems (MAS) have proved to be a quite appropriate paradigm for complex software system development, primarily because of the flexible and effective strategies for system analysis, modelling and implementation they offer. The elementary constituents of MAS are termed “agents” referring to software entities that exhibit autonomous and flexible behaviours (see also Chap. 5). Further to complex system modelling, Nobel Laureate Herbert Simon states [18]: Empirically a large proportion of the complex systems we observe in nature exhibit hierarchic structure. On theoretical grounds we could expect complex systems to be hierarchies in a world in which complexity had to evolve from simplicity.

The above statement raises the question of whether artificial complex systems should be modelled in a hierarchical manner as well. In this respect, Arthur Koestler coined in 1967 the term holon as an attempt to reconciliate holistic and reductionist visions of the world. A holon represents a whole-part construct which can be viewed either as a component of a higher-level system or as a whole composed of other holons as substructures. That elegant idea was later adopted by the Distributed Artificial Intelligence Community (DAI) under the term Holonic Multi-Agent System (HMAS) [11]. The underlying idea of HMAS is that a number of agents can be created and coalesce to form other higher-level agents, generating thus a hierarchic system. However, although that view has been adopted by numerous MAS researchers, most widely used agent-based models still consider agents as atomic entities. Indeed, as [10] points out, almost all proposals for agent architectures have not addressed the general problem of treating collections of “agents” as higher-order entities, for example by treating agent organisations as higher-level agents.

11.2 Background Context 11.2.1 Hierarchies and Holons in MAS An issue of high interest in agent research is the representation of collective behaviour exhibited by groups of agents often termed agent organisations. The problem primarily concerns representing a group of interacting agents that exhibit a specific behaviour and which, at a certain level of abstraction, behave as if they were a single entity.

11

Holonic Multi-Agent Systems

253

Several models have been proposed for representing agent organisations based on experiences drawn from different domains. The common view is to consider agent groups as agents composed of other agents. These composite agents are referred to with different terms in the literature. For example, Ferber [6] discusses individual and collective agents, while Holland [13] uses the term meta-agents instead. In [15] agent groups are referred to as intermediate agents, and they are used to detect earthquakes. Furthermore, Correa [3] proposes the terms recursive or intermediate agents to discriminate between reactive (lower-level) and cognitive (upper-level) composite agents. As another example, Odell [16] refers to agent groups with the term Agentified Groups. The list of terms researchers use to refer to such “aggregated” agent groups composed of lower-level agents is far from complete. Based on the holon concept, we use the term Holonic Agents to refer to agent groups consisting of coalesced agents. The term Holon was coined by the Hungarian philosopher Arthur Koestler in 1967 while trying to explain social phenomena in human societies and formed the basis for the Holonic Systems concept. Since then, the holonic approach has been used in a wide range of domains, including Philosophy [23], Manufacturing Systems [21], and Multi-Agent Systems [11]. In particular, the combination of holonic and MAS concepts gave rise to what is currently known as Holonic Multi-Agent Systems. The main difference between the holon and agent concepts concerns their granularity. Holons are, by definition, composed of other holons, referred in this work as super- and sub-holons (or member holons) respectively, while agents are not necessarily composed of other agents. This does not mean that agents cannot be composed of agents, but that “agents are atomic entities” is the general assumption. Indeed, as pointed out by [10], almost all proposals for agent architectures have not addressed the general problem of how to treat collections of “agents” as higher-order entities, for example how to treat organisations as agents. Holonic MAS represent an attempt to tackle this problem.

11.2.2 Organisational Metaphor for MAS The parallel between human organisations and computational systems has been first drawn in the eighties [9]. Since then, organisational approaches have become the subject of increasing interest in the research community. In MAS research in particular, several agent system modelling approaches inspired from social systems and using terms such as “role”, “group” and “community” have been proposed. The usefulness of organisational concepts in modelling MAS collective and individual behaviour resulted in their inclusion in many MAS engineering methodologies, such as GAIA [24] and MESSAGE [2], as well as in various MAS (meta-)models, such as AGR [7], RIO [12] and MOCA [1]. This is because adopting an organisational view in agent-based system engineering offers significant advantages, as Ferber points out in [7, 8]:

254

S. Rodriguez et al.

• Heterogeneity of Languages: Each organisational group can be viewed as a separate interaction space inside which different communication languages, such as KQML and ACL,1 can be used without modifying the overall system architecture. • Modularity: Organisations can be seen as modules producing specific behaviours, which are achieved by providing appropriate behavioural instructions to organisational members. Therefore, organisations can be used for defining clear visibility rules that facilitate the design of MAS. • Multiple Architectures: An organisational approach makes no assumptions about the internal agent architecture, enabling the use of various models and implementations. • Application Security: Unrestricted inter-agent communication raises several security issues. This problem can be mitigated by enabling controlled role access to each agent organisational group. In this way, the security level can be increased without imposing “global” centralised control. • Reusability: Finally, considering organisations as blueprints defining solutions to particular problems, allows reusing them in multiple contexts and problem domains, increasing thus software reusability. Given the above context, the RIO agent organisational framework [12] was selected to form the basis of the Holonic MAS framework described in this chapter.

11.3 Theoretical Notions 11.3.1 A Generic Framework for Holonic Systems Modelling A holon is a self-similar structure composed of holons as sub-structures. The relationship between holons and sub-holons can be extended recursively giving rise to hierarchical structures termed holarchies. Therefore, depending on the level of observation, a holon can be viewed either as an autonomous individual entity or as a hierarchical organisation of sub-holons. This duality is often referred to with the term Janus Effect2 in reference to the two “faces” of a holon, that is the two perspectives from which a holon can be viewed. Several examples of holarchies can be found in everyday life, the most common one probably being the human body. The human body cannot be viewed as a whole in an absolute sense since it consists of organs, which in turn are composed of smaller organic constructs. This decomposition can continue up to the level of cells and molecules. Hundreds of other examples of holarchies can be mentioned such as enterprises, cities and even galaxies. In another example, we can consider a university as a holon consisting of departments and laboratories. Furthermore, we can consider that a department is composed 1 Knowledge

Query Manipulation Language and Agent Communication Language are two agent communication languages based on the speech acts theory.

2 Janus

was a Roman god having two faces.

11

Holonic Multi-Agent Systems

255

Fig. 11.1 University example holonic structure

of academic staff and students, and a laboratory from researchers respectively. This holarchy is depicted in Fig. 11.1.3 At the highest level (level n + 2) there is the university holon which, in this example, is composed of three sub-holons: Computer Science (CS) Department, Mechanics Department and a Laboratory (level n + 1). The lowest level includes lecturers, students and researchers (level n). The role RP at the intersection of the CS Department and the Laboratory holons is simultaneously a lecturer and a researcher. Holonic structures offer a powerful abstraction for modelling large Complex Systems. However, suitable representational formalisms, which must be generic enough to represent the different aspects of holons and holarchies, are required. To this purpose, explicit modelling organisational constructs can play a significant role as described in the following sections.

11.3.2 Overview of the Approach Our framework adopts an organisational view aiming to minimise the impact of the underlying architecture. Furthermore, to increase the framework generality, we need to distinguish between two largely overlapping holon aspects. The first one concerns the relationship between holons and sub-holons, and the second one the purpose of 3 The

representation of holon composition has been inspired by [6].

256

S. Rodriguez et al.

the holonic system, that is the problem the holonic system aims to solve. We term the former aspect holonic since every holon, regardless of the application domain, is always composed of sub-holons. In addition, we use the term application to characterise the latter aspect. For example, in the Laboratory holarchy of Fig. 11.1 the holonic aspect refers to the fact that a laboratory group consists of researchers. On the other hand, the application aspect refers to that a laboratory group is created aiming to fulfill a specific purpose and achieve a number of particular goals, such as for instance to carry out a research project. The manner in which holon members organise and interact to enable the super-holon to achieve its goals is application-specific. In particular, members of two different super-holons may even follow different interaction patterns to achieve the same result. We provide more details on modelling the holonic and application aspects of a holarchy in Sects. 11.3.4 and 11.3.8 respectively. Our framework is further concerned with the specification of three important Holonic MAS aspects: • Holon Structure and Management: A super-holon is an entity in its own right, but it is composed of its members. This part of the framework focuses on how sub-holons organise and manage super-holons. • Goal-Dependent Interactions: Super-holons are created with the purpose of fulfilling particular goals and carry out specific tasks. For super-holon goals to be achieved, sub-holons must interact and coordinate their actions. The proposed framework offers adequate means to model goal-dependent functionality of super-holons. • Dynamics: Dynamic behaviour is an inherent MAS characteristic. In the proposed framework two particularly important dynamic Holonic MAS operations are considered. The first one is Merging, referring to sub-holons creating and joining a super-holon, and the second one is Self-organisation. The above issues are tackled separately throughout this chapter. In Sect. 11.3.4 an approach to model super-holons considering only their structure and management and the roles that holon members play are discussed. Section 11.3.8 illustrates a method to model the goal-dependent behaviour of holon members and to introduce elaborated coordination mechanisms. Finally, Sects. 11.3.9 and 11.3.10 focus on dynamic holonic MAS aspects.

11.3.3 Definitions and Terminology At a certain level of observation, a holon can be viewed as a number of interacting sub-holons. Sub-holon interactions shape the structure and behaviour of their superholon. To model these interactions in an orderly and modular manner, we can group them by means of organisations. Hence, a holon can be modelled as follows: Definition 11.1 A holon of level n, Hn , is defined as a tuple Hn = Hn−1 , OP, ψ,

11

Holonic Multi-Agent Systems

257

where Hn−1 is the set of sub-holon members, OP is the set of organisations affecting the super-holon Hn functionality, and ψ : Hn−1 → 2roles(OP) is a function relating sub-holons with the roles played in Hn . The roles function returns the set of roles defined in OP organisations such that for all hi ∈ Hn−1 , ψ(hi ) = ∅. The above definition involves certain implications concerning holons and holonic systems: 1. A set of entities can generate a new super-holon only if they interact. 2. A holon is defined not only by its member sub-holons but also by their interaction patterns. As a result, different super-holons can be generated from the same set of sub-holons in case sub-holon interactions determining super-holon behaviours differ. In other terms, different super-holons can be formed if sub-holons play different roles in each super-holon. 3. The structure of super-holons is determined by the social commitments and interactions of their sub-holon components. This is consistent with Ferber’s discussion about individual and collective agents [6]. 4. The level of sub-holon commitment towards super-holons depends on the obligations arising from the roles that sub-holons play within these super-holons. Hence, sub-holons need to interact to create super-holons. In the proposed framework, sub-holon interaction is not limited only to exchanging direct communication messages. Examples of other interaction types include indirect communication, such as communication carried out via the environment, and interaction based on commitments that modify super-holon states. Furthermore, different super-holons may result from the same sub-holons. For example, let us consider the group of employees of some company. At work, meaningful interactions for them are considered only those related to work tasks and business activities. This means that if they go out for a drink, the business operations and overall behaviour of their company are not affected. Now let us assume that the company employees found a sports association. In that case, the athletic activities of that association will be influenced only by those employee interactions and behaviours which are relevant to sports. In other terms, two super-holons, business company and sports association have been created from the same persons. The super-holon behaviour is defined by different member interactions, and they can evolve separately. When holons undertake roles4 in a super-holon context, they commit to execute the behaviours, including obligations, capabilities and conformance to certain organisational rules, associated with these roles. These commitments are the binding forces that keep super-holon members together. The requirements for undertaking and relinquishing roles are included in the respective role descriptions.

4 Roles

represent behavioural abstractions of explicit behaviours within a certain context.

258

S. Rodriguez et al.

11.3.4 Holon Structure and Management The first issue to address concerns organising holons as needed to generate and manage super-holons. To this purpose, three organisational structures targetting Holonic Multi-Agent Systems organisations have been proposed in [11]: Autonomous Agents Federation, Moderated Group and Fusion. In our approach we have adopted the moderated group structure. This decision has been motivated by the multiple configurations that are possible by differentiating sub-holon commitments towards their super-holons. In a moderated group, we generally consider two main roles for group members. The first one is the moderator or representative role, who acts as an interface to external holons, and the second one describes represented sub-holons who interact with the outside world only via their representatives. This organisational structure does not support further relations between sub-holons, for example based on authority or subordination. Therefore, in our approach we extend this organisation by providing representatives with specific authorities based on application requirements and constraints.

11.3.5 Modelling Holon Members To represent a moderated group completely, we need to identify appropriate roles and specify their behaviours. In the proposed approach we have selected four generic roles enabling description of a moderated group as an organisation: Head, Part, Multi-Part and Stand-Alone. The first three roles describe sub-holon status in a super-holon. The Stand-Alone role represents holons that are external to a given holon. In this section we discuss how the holonic aspect can be represented as an organisation in a super-holon. As mentioned above, our approach considers that super-holon members can play three different roles: Head, Part and Multi-Part. The Head role is the representative, or moderator, of the group. Represented members can play either the Part or the Multi-Part role depending on whether they belong to only one or to multiple super-holons respectively. We term such roles holonic roles. The adjective “holonic” is used to distinguish these organisational roles from roles modelling applicationdependent behaviours. A number of holonic roles is present in every holon and guarantees its organisational structure. In the aforementioned university example if we isolate the components of the CS Department and Laboratory holons and add holonic roles, we obtain Fig. 11.2. In the Laboratory holon the Part role represents researchers working only in the laboratory, such as full-time researchers. On the other hand, some researchers may, in addition to their laboratory activities, also give lectures in the Computer Science department. In that case, the respective sub-holons, such as holon RP in Fig. 11.2, belong to both CS Department and Laboratory holons simultaneously, and therefore they play the Multi-Part role in each of them. In this example, the department and

11

Holonic Multi-Agent Systems

259

Fig. 11.2 CS Department and Laboratory holons

laboratory directors would be the Head roles of the CS Department and Laboratory holons respectively. Let us now consider the members of a super-holon. The super-holon can then be viewed as a number of sub-holons in interaction. This organisation, termed Holonic Organisation, abstracts away all application-dependent interactions and concentrates solely on the status and behaviour of sub-holons from the super-holon point of view. A Holonic Organisation instance is termed holonic group. Interactions between sub-holons in a holonic group are described by holonic roles. Every superholon member must play at least one holonic role belonging to a holonic group. This ensures that super-holons contain at least one holonic group whose roles determine the status of their sub-holons. The four holonic roles considered in our approach are described in more detail below. The Head role is played by super-holon representatives. This means that the head constitutes an interface between sub-holons and the outside world. The head role can be played by more than one sub-holon at a time, and it represents a privileged status in the super-holon. Heads will generally be conferred with a certain level of authority, but they receive additional administrative load. This load can vary according to the sub-holon organisational configuration. It is important to emphasise that when a number of holons merge into a superholon, a new entity appears in the system. In that case, super-holon members are not merely a group of interacting holons as indicated in “traditional” MAS theories. A super-holon is an entity in its own right, having certain skills and being capable of undertaking roles in other super-holon organisations. Despite that Head sub-holons have certain powers over super-holon skills and services, it must be stressed that heads are not equivalent to super-holons. For example, consider a holon representing an army composed of sub-units such as divisions, battalions and platoons. The army head could be a General with full authority over all army units. However, the overall army capabilities, although influenced, cannot be considered as capabilities of its commanding General (head). For example, if the General is removed, due to retirement for instance, the army will still continue to exist. Furthermore, since Heads constitute the super-holon interface to the external world, they are in charge of redistributing the externally arriving information and

260

S. Rodriguez et al.

therefore of “triggering” the internal super-holon processes that will produce the desired result. We will discuss this issue further when we introduce techniques to model task-related interactions taking place between sub-holons. The Part role identifies members of a single super-holon that are represented to the outside world by a number of sub-holons playing the Head role. As long as a holon belongs to a single super-holon, it will play the Part role. However, when that holon requirements are no longer satisfied by its participation in the current superholon, then there are two possibilities. The first one is for the holon to abandon its current super-holon and seek another holon with which to merge and collaborate. The second possibility involves the holon to attempt to merge with another superholon while still remaining a member of the first super-holon. In the latter case, the holon will necessarily change its holonic role to Multi-Part. The Multi-Part role is an extension to the Part role. It emphasises that a sub-holon is shared by more than one super-holon. For example, the RP holon in the aforementioned university example represents persons that are both researchers and lecturers at the laboratory and the CS department respectively (see Fig. 11.2). There are several reasons to differentiate between the Part and Multi-Part roles, the most important being that shared holons are represented by more than one head. For example, imagine that a holon offers services that can be conflicting, such as a mechanism to create/access/destroy a resource. If that holon is shared by several super-holons, it may receive contradictory requests from its heads, such as to simultaneously access and destroy the same resource. More specifically, we can discriminate between three conflict types that may arise in sub-holons playing the Multi-Part role: • Interest conflicts: The super-holons do not share the same goal or have contradictory objectives. • Authority Conflicts: The super-holon representatives request contradictory actions from the shared members. • Unbalanced Authority Conflicts: A super-holon head has more power on a shared sub-holon than other super-holons sharing the same sub-holon. Several problems can arise as a result of these conflicts. For example, in open systems a self-interested head could use its authority on a shared member to prevent the progress of other holons. Furthermore, shared holons can cause bottlenecks and raise performance issues. These possibilities must be analysed in detail to ensure coherence and stability in the designed software systems. Despite the above-mentioned weaknesses, Multi-Part holons offer significant advantages as well. One such capability is Message Forwarding. This involves enabling Multi-Part holons to forward messages to members of different super-holons. For example, to delegate a task to a student, a laboratory researcher could ask a shared member, for example an RP in Fig. 11.2, to search for a suitable student at the university and delegate the task on her behalf. This could decrease the administrative load of head and reduce the number of “formal requests” between the laboratory and the CS department. An additional advantage is the possibility to implement trust mechanisms that would assess holon trustworthiness taking into account recommendations of shared holons. Finally, shared holons can facilitate access to

11

Holonic Multi-Agent Systems

261

holons positioned at different levels in a holarchy. In the hierarchical structure of a holarchy Multi-Part holons can be scattered at different levels, and they can produce feedback and shortcuts for other holons, positioned at their level, with which they interact directly.

11.3.6 Super-Holon Management From an internal perspective, super-holons can be viewed as communities of holons that cooperate to achieve commonly agreed objectives. Therefore, certain rules governing holonic structure evolution need to be defined at super-holon creation time. Since it is not practical to attempt to enumerate all possible issues requiring specific evolution rules, in the following we will concentrate only on a significant subset of evolution rules that are applicable in most cases. A fundamental operation necessary for super-holon management is the Member Inclusion/Exclusion. After super-holon creation, additional holons may request to join for various reasons. Furthermore, the super-holon itself may invite other holons to join if it considers that they will positively contribute to its goals. In this respect, two important aspects need to be analysed in detail. The first one concerns specification of the decision-making algorithm used, such as voting of sub-holons playing the Head role, and identification of entities with authority to take decisions. The second aspect concerns specification of the process handling join and leave requests. We use the term Merging to refer to the sub-holon inclusion/exclusion process, and we discuss it further in Sect. 11.3.9. Another important issue is the management of super-holon goals and tasks. More specifically, this involves detailing the manner in which super-holons can handle goals and tasks, for example introducing new goals and modifying existing ones, and specifying which entities should be authorised to approve such modifications. A similar issue concerns holon destruction. The most obvious destruction criterion can be goal achievement or completion of a particular task. However, other criteria can also be applicable, for example reaching appropriate fitness or member satisfaction thresholds. The most important issue is probably adding/modifying rules of sub-holon behaviour. This means that we provide holons with the possibility to evolve and modify their existing configuration as needed. We will elaborate further on this issue in Sect. 11.3.9 together with our discussion of the Merging process.

11.3.7 Decision Making: Defining a Government As mentioned in the previous section, decisions need to be taken quite frequently during holonic systems operation. For example, when an external holon requests membership in a super-holon, existing super-holon members can utilise a voting

262

S. Rodriguez et al.

mechanism to take the respective decision. A number of representative decisionmaking approaches applicable to super-holon management (see [22] for more details) is outlined below. A quite common approach to super-holon decision-making is to organise subholons in a federation. In that case, all members are equal with respect to taking decisions. In contrast, in a dictatorship organisation, heads are omnipotent, and their decisions are treated as indisputable by other sub-holons. Furthermore, in a dictatorship configuration, members abolish most of their autonomy since they need to always request head for permission to provide a service or to initiate a collective action. The above-mentioned approaches have proven useful in a number of specific cases. However, an intermediate approach will generally be most efficient. An example of such an approach involves attributing a voting mechanism to each functionality. Furthermore, to fully define the voting mechanism, three aspects need to be determined, that is the Requesters, the Participants and the Adoption Mechanism. Vote requesters are sub-holons authorised to initiate a voting process when deem necessary. Participants are those sub-holons entitled to participate in voting, and the adoption mechanism specifies the decision-making method and relevant criteria for accepting or rejecting a proposed action. Vote requesters, as well as vote participants, can be either all sub-holons, sub-holon heads only, or a specific group of sub-holons. Furthermore, various adoption mechanisms can be used, such as adopting based on voting consensus. Considering only the number of voters and the percentage of heads and parts involved in the decision-making process, we can distinguish between four configurations: • Monarchy: In this case a single head controls the entire decision-making process. The head selection process needs to be explicitly specified. • Oligarchy: A small group of heads share the command without referring to the Part members. • Polyarchy:5 A small group of heads generally shares the command, but they consult Parts when taking certain types of decisions. • Apanarchy:6 In this case the decision-making authority is completely shared between all super-holon members, and they participate in every decision taken. The decision making process of a super-holon can be highly decentralised to an extent where heads are simply interfaces to the outside world having no authority over the super-holon functionality or its members. Intuitively, apanarchy represents the highest level of decentralisation. The characteristics of the above-mentioned government structures are summarised in Table 11.1.

5 Borrowed

from Robert A. Dahl, who coined it to describe a specific democratic government type.

6 Originates

from the Greek terms Apan, meaning “all” or “every”, and archein, meaning “to rule”.

11

Holonic Multi-Agent Systems

Table 11.1 The characteristic forms of super-holon government

Name

263 Configuration

Monarchy

one Head, one Voter, no Parts vote

Oligarchy

n Heads, n Voters, no Parts vote

Polyarchy

n Heads, n + k Voters, k Parts vote

Apanarchy

Everybody votes, All Parts vote

11.3.8 Goal-Dependent Interaction Modelling In addition to member status and overall functionality, a super-holon aspect that also needs to be described is goal-driven sub-holon actions. To achieve their objectives, super-holons need to accomplish a number of tasks. Therefore, sub-holons need to organise themselves as needed to distribute sub-tasks and exchange information concerning sub-task execution. Tasks are commonly application dependent, and they vary between sub-holons according to sub-holon capabilities. These domaindependent holon organisations are termed Internal Organisations. Consequently, a holon model comprises a set of sub-holon organisations, denoted by OP in holon definition. One such organisation is the Holonic Organisation defining the status of super-holon members, such as Head, Part and Multi-Part. Additional organisations, that is Internal Organisations, determine sub-holon interactions required to achieve the super-holon goal. Therefore, a non-atomic holonic agent instantiating this model includes: • A unique Holonic Group, that is an instance of Holonic Organisation, which defines how sub-holons are organised. All super-holon members must belong to this group. • A set of organisational groups, that is instances of Internal Organisation, which are aimed to coordinate sub-holon interactions. These groups are created based on sub-holon individual goals/tasks and each one can comprise only a subset of available sub-holons. To illustrate this idea, let us consider a University Department modelled using two Internal Organisations. The first organisation represents the Council, defining how administrative decisions are taken and who is involved in the decision-making process. The second organisation represents a specific Lecture, describing the interactions between students and their professor. Multiple instances of the Lecture organisation may be present in a Department at a time. Using the above described approach, the behaviour and interactions of subholons can be described independently of their holonic roles, which they play being super-holon components. The main advantages of the proposed approach are: • Clear separation between Holonic-related (Holonic Organisation) and domainspecific (Internal Organisation) behaviours. • Modularity in organisational descriptions. We can associate an organisation to each task/goal without modifying existing ones.

264

S. Rodriguez et al.

• Facilitation of reusable modelling through the use of organisations as primary modelling constructs. • Modelling encapsulation of the intrinsic recursivity of holons in the modelling phase. Designers can describe sub-holon interactions without the need to explicitly consider whether sub-holons are atomic or not. • Ease of design, since the hierarchical holon structure facilitates incorporating complex organisational mechanisms, concerning task distribution, decisionmaking and cooperation for instance. To summarise, a holon model specification involves detailing a number of subholon organisations, from which the Holonic Organisation that describes sub-holon status is mandatory. The remaining sub-holon organisations are optional, and they determine sub-holon application behaviours.

11.3.9 Holon Dynamics In this section we discuss new super-holon creation and integration of additional components into existing super-holons. This process is termed Merging. In Merging new super-holons can be created either from a number of existing holons that merge into a super-holon or by decomposing a holon into appropriate subcomponents. In holon decomposition super-holons are capable of centrally defining the intentions and behaviours of the resulting sub-holons. Furthermore, they can predefine and control sub-holon interactions, and they can even impose particular sub-holon architectures. Therefore, in this chapter we will not further discuss holon decomposition, and we will instead concentrate on super-holon creation from existing individual holons. We can distinguish between two types of Merging: creation of a new superholon and incorporation to an existing super-holon. To support the integration of new members, we need to provide external holons with a “standard” interface so that they can request their admission to a super-holon when needed. In this respect, non-member holons can be represented by the Stand-Alone role. Given that from a super-holon perspective external holons are viewed as Stand-Alone role players, super-holon interfaces to external holons typically comprise only Heads. Therefore, other holon members, namely those playing Part and Multi-Part roles, are generally not visible from external holons. This arrangement can be modelled as a specific holonic organisation. In this organisation, Stand-Alone external holons may interact only with Heads of the available super-holons. Acting as autonomous entities, holons can interact with each other without necessarily creating/merging into a higher-level entity. This highlights an important concept: Holons can have different independent faces. In other words, a holon can be playing the Stand-Alone role at one level, while it can be viewed as composed by substructures at another level. We use the term Face in accordance with the Janus Face holonic feature introduced by Koestler. Along this line, a holon can have two faces: The first one—looking up—represents holons as autonomous entities, and

11

Holonic Multi-Agent Systems

265

the second one—looking down—considers holons as groups of sub-holons in interaction. An interesting characteristic of this approach is that the Stand-Alone role realises a standardised view of non-members of a super-holon. The merging process commonly results in new holonic entities (super-holons). In this case, all rules governing super-holon operations need to be defined in detail. From an engineering point of view, a number of different approaches can be followed for this purpose: • Predefined rules: In this case the rules governing super-holon operations are known by member holons in advance. This approach can be useful for developing closed and relatively small applications. However, any adaptability of such applications is limited to only anticipated cases, and therefore it is generally quite hard to maintain it in large and open software environments. • Negotiation: This approach involves mechanisms for negotiating super-holon configurations throughout the merging process. This perspective allows for a wider range of applications and improved adaptation capabilities. However, the negotiation process may induce considerable overheads. An effective way to mitigate such overheads is to combine negotiation with the predefined rules approach described above. Furthermore, additional issues involved in this approach include determining the communication language to be used in the negotiations, and handling the significant workload inherent in determining all possible rules describing super-holon behaviours. • Evolutive: In this approach, operations of newly created super-holons initially require minimum involvement of member holons. In time, members can increase their participation in super-holon operations should they deem it as appropriate. The minimal rule set specifying holon behaviour can contain only one rule: Add new rules. Using this rule and a voting mechanism, any new rule, or variations of the existing ones, can be generated and added to the rule set. A representative example of such a task-oriented system is the FORM framework which has been proposed in [17].

11.3.10 Self-organisation for Holonic Multi-Agent Systems One of the most prominent characteristics of holonic systems is their selforganisation capability, which is primarily realised in the merging process. In this respect, an additional component of our framework concerns a generic engine regulating holon merging and decomposition. This engine is based upon the holonic roles described in Sect. 11.3.5, and it realises an effective approach for implementing selforganisation in HMAS. The underlying assumption is that Head role players have all the necessary authority to accept or refuse new members. Furthermore, to avoid possible conflicts of interests, Heads cannot be shared holons, namely they cannot play the Multi-Part role. An alternative approach to implementing self-organisation capabilities in HMAS is discussed in Chap. 10.

266

S. Rodriguez et al.

Fig. 11.3 Generic engine for self-organisation

The roles described in Sect. 11.3.5 represent member status in super-holons. Based on these roles, we can define a set of inter-role transitions modelling evolution of holon behaviour within a super-holon. Furthermore, by adding conditions to role transitions, enclosed in square brackets in our notation, we can construct an automaton consistently describing evolution of super-holon member behaviours. Each state in this automata represents a role that a holon can play within a super-holon. An additional state has been added to represent the start the Merging interactive process. Despite that Merging is commonly a fairly atomic and simple operation, it can often prove to be quite complex and time-consuming. An example of the latter case is when an elaborated negotiation protocol, such as the Contract Net Protocol [20] for instance, is utilised to recruit new holon members. Figure 11.3 presents an exemplar automaton highlighting the transition conditions between behavioural states of super-holon members. The following conditions regulating inter-state transitions are considered: • • • • • • • •

CSM (Stand-Alone to Merging) CMH (Merging to Head) CMP (Merging to Part) CPS (Part to Stand-Alone) CPH (Part to Head) CHS (Head to Stand-Alone) CPMP (Part to Multi-Part) CMPP (Multi-Part to Part).

When specific self-organising properties are desired, the above automaton can be refined according to the overall objectives of the holonic application. For example, in the university holonic system described in Sect. 11.3.5, CSM would represent the criteria for merging different university departments. Similarly, CPH would represent the conditions for an entity playing the Part role, such as a university staff member, to start playing the Head role, for example to become Head of the Department after departmental elections. An interesting specialisation of the above self-organisation engine can be constructed based on the concepts of holon affinity and satisfaction. The notion of

11

Holonic Multi-Agent Systems

267

Affinity was inspired from the work done in Immune Systems [5]. Furthermore, the term Satisfaction has commonly been used to represent agent gratification resulting from agent internal states and progress made towards fulfillment of goals/tasks [19]. The affinity between holons is dependent on the holonic application domain. A general definition of affinity is the following: Definition 11.2 Holonic affinity is the degree of holon compatibility with respect to collaborating towards achieving shared objectives in a particular application domain. A number of holons are considered as compatible if they are capable of providing assistance to each other towards achieving a common goal. Considering the overall application objectives, it is possible to define application-specific criteria for evaluating holon compatibility. The satisfaction, on the other hand, can be defined as follows: Definition 11.3 Holon satisfaction is the degree of accomplishment of the holon’s current goals in a particular application domain. To facilitate specification of the transition conditions of the generic selforganisation engine described above, we can classify holon satisfaction as follows: • Self Satisfaction (SSi ). This refers to satisfaction of holon i which has resulted from own efforts. • Collaborative Satisfaction (CSH i ). This type of satisfaction results from collaboration of holon i with other members of super-holon H . This satisfaction can be either positive, meaning that other holons have assisted holon i in carrying out its tasks, or negative, indicating that other holons have imposed obstacles to the efforts of holon i. • Accumulative Satisfaction (ASi ). Accumulative satisfaction results from collaboration of holon i with members of multiple super-holons. This type of satisfaction can be defined only when holons are allowed to play the Multi-Part role, for example when holon i can belong to more than one super-holon. When a holon is unsatisfied from its membership in a super-holon, it has two options: it can either quit its current super-holon and join a new one, or it can additionally join a second super-holon without leaving the first. This decision is driven by the accumulative satisfaction produced in each case. Accumulative satisfaction is calculated from the following formula: ASi =

p

CSi

∀p ∈ superholon(i),

p

where the superholon function returns the super-holons of holon i.

(11.1)

268 Table 11.2 Transition conditions in terms of holon satisfaction

S. Rodriguez et al. CSM

IS < NS

CPS

IS < NS and SS > CS

CHS

IS < NS

• Instant Satisfaction (ISi ). The current satisfaction of holon i is given from the following formula: ⎧ H ⎨ CSi + SSi , if Ri = Part ∨ Ri = Head, (11.2) ∀i ∈ HMAS ISi = ASi + SSi , if Ri = Multi-Part, ⎩ if Ri = Stand-Alone, SSi , where Ri is the role played by holon i, and H is the respective super-holon. Furthermore, we define the function Necessary Satisfaction (NS). NS estimates the minimum satisfaction required by a holon to complete its tasks while observing all relevant constraints. The NS value can be viewed as a feasibility threshold that depends on the application domain and must be each time adapted to match the problem under consideration. The transition conditions of the automaton presented in Fig. 11.3 can be expressed in terms of holon satisfaction as described in Table 11.2: The above table does not include conditions CMP and CMH which determine whether a holon will play the Head or the Part role. Both these conditions, as well as the CMPP and CPMP ones, should be defined based on application-specific criteria. This self-organisation engine is particularly useful when the organisation of holons into a holarchy consists itself a sufficient solution to the problem addressed by the holonic application. In these cases, we only define the holon merging/decomposition process, and we do not need to further specify any particular interactions among super-holon members.

11.4 Applications To illustrate the approach described in Sect. 11.3.10, we consider the Adaptive Mesh Problem concerning dimensioning and positioning of antennas in a radiomobile network. A distinguishing feature of cellular radio mobile networks is the possibility of rapid increase of consumer demand and the ensuing complexity in their design and management. The adaptive meshing problem for antenna dimensioning considers communication traffic statistics as a predefined resource that must be attributed to numerous adaptive and low-power Base Transceiver Stations (BTS). This means that the amount of expected traffic must be approximated and attributed to BTS. This is a fundamental step in radiomobile network design [4]. The system takes as input a digitised environment comprising the communications inside each region of the area considered. This information is obtained by digitising a specific geographical area, as shown in Figs. 11.4, 11.5 and 11.6, which

11

Holonic Multi-Agent Systems

269

Fig. 11.4 Sample area to be covered

present a map of the area to cover, the digitisation grid used and the resources of each grid cell respectively. A regular grid is used to produce a matrix where each element represents the communication traffic in the respective region. The grid values are obtained either by field measurements, possibly corrected by taking future requirements into account, or by empirical estimations. This matrix represents the input of adaptive meshing, and it is termed Resource Matrix. The above process generates an Adaptive Mesh comprising a number of cells. Each cell covers a certain amount of communication traffic. The cell size depends on the sum of communication traffic of the resources covered. This process is illustrated in Fig. 11.7. The generated mesh must cover the resources appearing in the resource grid while respecting a number of constraints. The first constraint requires that each cell has a specific geometrical shape, or geometry. In our example we consider a rectangular geometry, but cells can generally have various geometries. Given their shape, BTS are placed inside cells to support the respective communication traffic. The second constraint concerns the maximal communication traffic covered by a cell. This should not exceed the respective BTS capacity. To address the above problem using a holonic MAS approach, each resource of the Resource Matrix is associated with a holon (Resource holon) whose unique goal is to become Part of a cell (Cell holon) covering the required communication traffic. The reader should bear in mind that this model does not aim to provide a final solution to the adaptive mesh or to the antenna positioning problems. We simply intend it to be an introductory example to the application of the framework “out of the box”. Many works, see [14] for instance, emphasise the complexity of the

270 Fig. 11.5 Regular grid used to digitise the sample area

Fig. 11.6 Communication Traffic Digitisation

S. Rodriguez et al.

11

Holonic Multi-Agent Systems

271

Fig. 11.7 Mesh composition Fig. 11.8 Holarchy for the Adaptive Mesh problem

adaptive mesh problem and suggest that a large number of parameters need to be considered to obtain a realistic approach to the subject.

11.4.1 Holonic Model The holonic system of our adaptive mesh example has simple objectives, and thus it comprises only one holarchy. We model application behaviour using what we term Behavioural Approach since what we know in advance is the intended system behaviour. Our behavioural approach principles can be summarised as: System resources must be grouped into cells while observing any geometrical and maximal communication constraints. The mesh finally formed by these cells will be a solution to the adaptive mesh problem considered.

We then need to define criteria specifying when resources are allocated to cells, taking extra care for the cases where several alternatives are possible. In particular, to incorporate the intended behaviour in the holons associated with resources, we will use the automaton introduced in Sect. 11.3.10. An overview of the resulting holarchy is given in Fig. 11.8. At the lowest level the holarchy contains Resource holons which have two attributes:

272

S. Rodriguez et al.

• a geographical position expressed as (x, y) coordinates • a communication traffic value obtained from the Resource Matrix. Resource holons are atomic, namely they are not composed of any simpler subholons. As mentioned previously, the unique goal of resource holons is to ensure that their associated communication traffic is covered by a BTS. To fulfil this goal, resource holons group together into cells, forming Cell holons located at the intermediate holarchy level. Upon formation, each Cell holon establishes a link to an antenna (BTS) so that any specific problem constraints, namely those concerning maximal communication load and geometrical shape, are satisfied. Cell holons have two attributes as well. The first one concerns the communication traffic covered by their associated cells. For each Cell holon, this value is the sum of communication traffic of all its members (which are Resource holons). The second attribute refers to the geometrical shape of the associated cell of each Cell holon. This shape is calculated based on the coordinates of the Cell holon members. Cell holons ensure that any constraints are observed and that their members playing the Head role are granted the authority to include or exclude Resource holons as needed to achieve a better overall configuration. All remaining members play the Part role since no Multi-Part roles are allowed in the simple example considered. This reflects the assumption that a resource must be covered by one and only one BTS.7 Resource holons playing the Head role can be placed at the centre of the respective cell to facilitate recruitment and management of Part members. Finally, there is only one Mesh holon in the system, and all Cell holons are considered to be its members playing the Part role. The Mesh holon is responsible for determining when the system has arrived to a stable state and thus to a solution, a decision taken by its Head members. There are several Head role configurations, and decision mechanisms that can be used in a Mesh holon, for example a simple configuration can be to allow all participating Cell holons play the Head role and take collective decisions using a voting mechanism.

11.4.2 Satisfaction In the example holonic system addressing the adaptive mesh problem, the aim of Resource holons is to ensure coverage of their communication traffic by joining a suitable cell and participating in the respective Cell holon. This results in a specialisation of the Merging Organisation (cf. Sect. 11.3.9), which we term Cell Merging Organisation. Since our system is a closed one, we do not need to employ a dynamic merging process, for example a process based on holon negotiations, but instead we can use a Predefined Merging Process. In other words, our holons are developed so that to know in advance how to request their admission to a super-holon. In our 7 Allowing

resources to be covered by more than one antennas can be a first extension of this exemplar model.

11

Holonic Multi-Agent Systems

273

model the merging process will be triggered based on holon satisfaction, and admission of Resource holons to Cell holons will be regulated using holon affinity. In our holonic approach we use communication traffic to define Resource holon satisfaction. More specifically, the satisfaction definitions introduced in Sect. 11.3.10 are adapted in our example as follows: • Self Satisfaction (SSi ). In our model, self-satisfaction SSi represents the communication traffic of Resource holon i. • Collaborative Satisfaction (CSH i ). In the proposed example system collaborative satisfaction CSH of Resource holon i is a function of the communication traffic i of the other members of Cell Holon H in which Resource holon i participates. • Accumulative Satisfaction (ASi ). This type of satisfaction results from collaboration of holon i with members of multiple holons. Since in this example holons are not allowed to play the Multi-Part role, no accumulative satisfaction can be considered. • Instant Satisfaction (ISi ). In our example instant satisfaction ISi is defined in terms of the roles that holon i plays: CSi + SSi , if Ri = Part ∨ Ri = Head, (11.3) ∀i ∈ HMAS, ISi = if Ri = Stand-Alone, SSi , where Ri is the role played by holon i. In the automaton considered in our example, we define the Necessary Satisfaction (NS) as the instant satisfaction required to accomplish the Resource holon objective. More specifically, we define the necessary satisfaction as the maximal communication traffic a BTS can cover. In this way, we not only direct Resource holons to participate in cells, but we also mandate that cells should be as close as possible to the maximum capabilities of the respective BTS. As can be seen from (11.3), when Resource holon i plays the Stand-Alone role, it holds that IS = SS. As a result, if traffic is not sufficient for the Resource holon to be solely assigned to an antenna, it will try to join a suitable Cell holon (IS < NS). When a number of Resource holons merge into a Cell Holon, one of them is designated as the Head of the cell. Generally, various approaches can be used to appoint a Head of a super-holon. To avoid unnecessary communication among members, and since we are operating in a closed system, in our example we select Resource holons that will play the Head role based on predefined simple rules. For example, the Resource holon that is closest to the centre of the respective cell undertakes the Head role. In the adaptive mesh problem, the holon that plays the Head role guarantees that the respective Cell holon observes the constraints of its members. To achieve this, Heads are granted the authority to accept or refuse joining requests from non-member holons based on the current Cell holon member constraints. Furthermore, Heads can remove any Cell holon member if that will increase the overall Cell holon satisfaction. Finally, a Head may decide to destroy the Cell holon if after attempts to improve the Cell holon’s overall satisfaction, the resulting satisfaction is still insufficient to obtain a BTS. Such a situation may occur when the communication traffic covered by a cell is far below the maximum capacity of an antenna and all surrounding resources are already covered by other cells.

274

S. Rodriguez et al.

Any Resource holon can request admission to an existing Cell holon. If accepted, it becomes a Part of that Cell holon.8 A Resource holon remains in a Cell holon if that increases its satisfaction level. To calculate Resource holon satisfaction, we consider the collaborative satisfaction (CSH ) defined as the sum of Cell holon H member resources. A Resource holon attempts to establish and maintain membership in a Cell holon if it assesses that such membership would contribute towards accomplishment of its goal. In particular, a Resource holon terminates its membership in a Cell holon in the following cases: 1. Explicit removal: When a Cell holon Head considers that a Resource holon reduces the overall Cell holon satisfaction, it can require the Resource holon to terminate its participation in that Cell holon and exit. In that case, the Resource holon is forced to depart the Cell holon and restart the merging process. 2. Satisfaction decrease: Any Cell holon member, including the Cell holon Head, can decide to discontinue its membership and withdraw from its current Cell holon at any time. This is usually the case where the resources covered by the respective cell fall below a minimum threshold. The selection of a suitable Cell holon to merge with is of primary importance in Resource holon merging. To this purpose, the affinity concept measures the “compatibility” of two holons. We describe an approach to holon affinity calculation in Sect. 11.4.3 below.

11.4.3 Affinity In an HMAS, holons commonly alter the roles they play and seek to merge with other holons to better achieve their goals. The suitability of two holons for merging together to work collaboratively is expressed in the concept of Affinity, and it can be estimated based on holon goals and services they provide. All relevant information concerning holon goals and provided services are included in an identifier associated with each holon. Therefore, the affinity of two holons can be determined by comparing the respective identifiers. The involved holons can then use the estimated affinity values to decide whether to merge with each other or not. In our meshing example each holon identifier contains the position of the respective resource, expressed in (x, y) coordinates, and the amount of traffic associated with that resource. Furthermore, Resource holons have all the same goal, namely to achieve coverage of their resources. Therefore, the main issue concerning possible merging of Resource holons to a Cell holon is to ensure that any geometrical constraints are respected. To consider geometrical constraints in affinity estimation, we introduce two affinity types that form the overall holon affinity: 8 The criteria for accepting or rejecting admission requests submitted by Resource holons will be discussed in the next section.

11

Holonic Multi-Agent Systems

275

Fig. 11.9 Acceptance distance example

• Distance affinity: This type of affinity refers to observance of the geometrical constraints involved. Since we consider square meshes, we define distance affinity using two parameters. The first one is boolean indicating whether a Resource holon lies within the acceptance distance of a Cell holon (see Fig. 11.9). If the Resource holon is located outside the specified acceptance distance, it will not be granted permission to join the Cell holon. The second parameter is the actual distance affinity which is defined as the number of resources already included in the Cell holon mesh and having unitary distance from the resource of the Resource holon under consideration. For example, let us consider three different Cell holon configurations and the respective Resource holon joining requests demonstrated in Fig. 11.10. Resource holons in cases (a) and (b) will be accepted into the Cell holon if the maximum traffic that an antenna can handle has not been reached. The distance affinity is higher in case (a) which is preferred over case (b), especially if the resulting Cell

Fig. 11.10 Distance affinity examples

276

S. Rodriguez et al.

holon traffic is close to the maximum antenna capacity. The Resource holon in case (c) will be rejected in all circumstances. • Resource affinity: This type of affinity concerns the level of antenna capacity that will be required after a possible merge. As a new BTS is positioned in a cell, the communication traffic covered should not exceed the maximum antenna capacity. In this respect, resource affinity can be used to ensure that antenna capacity limits are not exceeded. All parties involved in a merging process, such as Resource holons and Cell holons, estimate the respective affinities, but each party does not necessarily obtain the same affinity values. The Cell holon Heads constantly re-evaluate the affinity between the Cell holon and its members. In this way, when a new request arrives, Heads may decide to remove an existing member and accept a new one aiming to increase the overall Cell holon satisfaction. To summarise, Resource holons group into Cell holons based on affinity and satisfaction. Each Cell holon has a Head ensuring that the relevant constraints are observed. One of those constraints concerns the communication traffic that must be covered to obtain an antenna. A Cell holon is said to be “stable” when the covered resources are within the limits of the respective cell capacity and the geometrical constraints are observed. In that case, all new merging requests are refused, and therefore the Cell holon members remain stable in time. The Mesh holon constantly examines the state of its Cell holon members. If all Cell holons are stable, then the overall holarchy structure can be used to obtain a solution to the adaptive mesh problem.

11.5 Conclusion In this chapter we have presented the main concepts underlying holonic systems and a framework which allows the engineering of such systems. Our framework focuses on modelling and representation of three important aspects of a Holonic MAS: • Holon Structure and Management: A super-holon is an entity in its own right, but it is composed of other holons. This part of the proposed framework focuses on how members organise and manage a super-holon. • Goal-Dependent Interactions: To better achieve their objectives, holons must interact and coordinate their actions. Our framework also offers constructs for modelling holon interactions. • Dynamics: Our framework particularly focuses on two of the most attractive characteristics of Holonic MAS, namely Merging, that is creating and joining a superholon, and Self-Organisation. The proposed framework is illustrated in the adaptive meshing problem. Holons self-organise following a given mechanism, and the structure of the resulting stable holarchy can be used to obtain a satisfactory solution to this problem.

11

Holonic Multi-Agent Systems

277

11.6 Problems–Exercises 11.1 This problem concerns the design of a pedestrian traffic simulator based on the holonic paradigm. The simulator should be able to handle large numbers of pedestrians in a specified environment. Each pedestrian will have as main goal to reach a destination point in the environment. The pedestrian goals will be able to change even if not satisfied. In other words, pedestrians will be able to arbitrarily decide to set course for different destination points which will then become their main goals. Pedestrians will be represented by atomic holons, namely holons that do not include any other holons as substructures. Composite holons will represent groups of pedestrians currently directed to the same destination point. (a) Propose a simple model of the pedestrian environment. (b) Propose a holarchy for the above-described problem scenario. (c) Define an affinity function based on the distance between holons and their respective destination points. (d) Define an overall satisfaction criterion based on the satisfaction of individual holon goals. 11.2 The following problem is extracted from a benchmark for self-organisation techniques discussed in the Self-Organisation Special Interest Group which operated in the context of the EU AgentLink III Network of Excellence. Consider a number of processes executing on an open network of workstations. The system satisfies the following constraints and requirements: • dynamic load balancing of processors; • minimisation of inter-communication costs of highly communicating applications; • optimal data and resource sharing between same application processes; • mutual exclusion of applications accessing the same set of resources. The openness of the environment implies that workstations can dynamically join and leave the network at any time. Furthermore, new applications can be launched at arbitrary time points, and existing applications can dynamically suspend or resume their execution. Finally, the network can be subject to various perturbations such as workstation breakdowns. Different system aspects can be considered in this benchmark, such as dynamic routing of processor traffic, network topology evolution and constraint satisfaction prioritisation (for example assigning different weights to different constraints). Furthermore, it is assumed that data files concerning system operation aspects such as network topology, application characterisation, application interrelations and relations between applications and resource access can be available. Finally, breakdowns, including correlated occurrences of disturbances, are assumed to occur according to given stochastic distributions. (a) Propose a holarchy for this problem. (b) For each holon type, define affinity functions and a satisfaction criterion.

278

S. Rodriguez et al.

(c) Analyse the consequences of affinities and satisfaction criteria on the relationships of holons and holonic roles. Key Points • Holonic systems comprise holons which can be both individual entities and groups of other holons; • Holons play roles and interact with each other to coordinate and better fulfill their goals; • Holons merge to form super-holons structured in holarchies and selforganise by changing their roles according to their objectives.

References 1. Amiguet, M.: Moca: un modèle componentiel dynamique pour les systèmes multi-agents organisationnels. Ph.D. thesis, Université de Neuchâtel (2003) 2. Caire, G., Coulier, W., Garijo, F.J., Gomez, J., Pavón, J., Leal, F., Chainho, P., Kearney, P.E., Stark, J., Evans, R., Massonet, P.: Agent oriented analysis using message/uml. In: Wooldridge, M., Weiß, G., Ciancarini, P. (eds.) Agent-Oriented Software Engineering II, Second International Workshop, AOSE 2001, Montreal, Canada, 29 May 2001. Lecture Notes in Computer Science, vol. 2222, pp. 119–135. Springer, Berlin (2002). Revised Papers and Invited Contributions 3. Correa e Silva, F., Kelly, C.: Systèmes multi-agents hybrides: une approche pour la conception de systèmes complexes. Ph.D. thesis, Université Joseph Fourier-Grenoble 1 (2001) 4. Creput, J.C., Koukam, A., Lissajoux, T., Caminada, A.: Automatic mesh generation for mobile network dimensioning using evolutionary approach. IEEE Trans. Evol. Comput. 9(1), 18–30 (2005) 5. Dasgupta, D.: Artificial Immune Systems and Their Applications. Springer, Berlin (1998) 6. Ferber, J.: Multi-Agent Systems: An Introduction to Distributed Artificial Intelligence. Addison-Wesley, Reading (1999) 7. Ferber, J., Gutknecht, O.: A meta-model for the analysis and design of organizations in multiagent systems. In: Demazeau, Y., Durfee, E., Jennings, N. (eds.) ICMAS’98 (1998) 8. Ferber, J., Gutknecht, O., Michel, F.: From agents to organizations: an organizational view of multi-agent systems. In: Agent-Oriented Software Engineering IV 4th International Workshop (AOSE-2003@AAMAS 2003), Melbourne, Australia. LNCS, vol. 2935, pp. 214–230 (2003) 9. Fox, M.S.: An organizational view of distributed systems. IEEE Trans. Syst. Man Cybern. SMC-11(1), 70–80 (1981) 10. Gasser, L.: Boundaries, identity, and aggregation: plurality issues in multiagent systems. SIGOIS Bull. 13(3), 13 (1992). doi:10.1145/152683.152696 11. Gerber, C., Siekmann, J., Vierke, G.: Holonic multi-agent systems. Research Report RR-9903, DFKI (1999). ftp://ftp.dfki.uni-kl.de/pub/Publications/ResearchReports/1999/RR-99-03. ps.gz 12. Hilaire, V., Koukam, A., Gruer, J.P., Müller, J.P.: Formal specification and prototyping of multi-agent systems. In: Omicini, A., Tolksdorf, R., Zambonelli, F. (eds.) Engineering Societies in the Agents’ World. Lecture Notes in Artificial Intelligence, vol. 1972. Springer, Berlin (2000) 13. Holland, J.H.: Hidden Order: How Adaptation Builds Complexity. Addison-Wesley, Longman, Redwood City (1995)

11

Holonic Multi-Agent Systems

279

14. Lissajoux, T., Koukam, A., Renaud, D., Caminada, A., Créput, J.: Evolutionary meshing for mobile network dimensioning. La Rochelle, France (2000) 15. Marcenac, P., Calderoni, S.: Self-organisation in agent-based simulation. In: Magnus Broman, W.V.D.V., Hägg, S. (eds.) Poster Proceedings of the 8th European Workshop of Modelling Autonomous Agents in a MultiAgent World, pp. 116–131. Springer, Berlin (1997). citeseer.ist. psu.edu/17993.html 16. Odell, J., Nodine, M., Levy, R.: A metamodel for agents, roles, and groups. In: Odell, J., Giorgini, P., Muller, J. (eds.) Agent-Oriented Software Engineering (AOSE) IV. Lecture Notes in Computer Science. Springer, Berlin (2005) 17. Schillo, M.: Multiagent robustness: autonomy vs. organisation. Ph.D. thesis, Department of Computer Science, Universitat des Saarlandes (2004) 18. Simon, H.A.: The Science of Artificial, 3rd edn. MIT Press, Cambridge (1996) 19. Simonin, O., Ferber, J.: Modélisation des satisfactions personnelle et interactive d’agents situés coopératifs. In: JFIADSMA’01: 9èmes Journées Francophones d’Intelligence Artificielle Distribuée et Systèmes Multi-Agents, pp. 215–226 (2001) 20. Smith, R.G.: The contract net protocol: high-level communication and control in a distributed problem solver. In: Distributed Artificial Intelligence, pp. 357–366 (1988) 21. Van Leeuwen, E., Norrie, D.: Holons and holarchies [intelligent manufacturing systems]. Manuf. Eng. 76(2), 86–88 (1997) 22. Weiß, G.: Multiagent Systems, a Modern Approach to Distributed Artificial Systems. MIT Press, Cambridge (1999) 23. Wilber, K.: Sex, Ecology, Spirituality. Shambhala, Halifax (1995). http://207.44.196.94/ ~wilber/20tenets.html 24. Zambonelli, F., Jennings, N., Wooldridge, M.: Developing multiagent systems: the Gaia methodology. ACM Trans. Softw. Eng. Methodol. 12(3), 317–370 (2003)

Part III

Engineering Artificial Self-organising Systems

Chapter 12

Engineering Self-organising Systems Carole Bernon, Marie-Pierre Gleizes, Frédéric Migeon, and Giovanna Di Marzo Serugendo

Designing self-organising systems represents a challenge, and new tools have to be provided for helping designers; agent-oriented methodologies are now focusing on this issue.

Objectives The objectives of this chapter are to highlight the difficulties of designing self-organising systems, and after reading it, the reader will: • Understand what are the main challenges of a self-organising systems design; • Know the most important frameworks and methodologies (existing and in progress); • Have a clear idea of the ADELFE main activities. The given description of the design process helps the reader to start from an application and to achieve the system design phase. Of course, at the end of the chapter, the reader has to practise this method in order to gain more expertise; • Know the future needs for self-organising systems engineering.

C. Bernon () · M.-P. Gleizes · F. Migeon IRIT, Université Paul Sabatier, Toulouse, France e-mail: [email protected] M.-P. Gleizes e-mail: [email protected] F. Migeon e-mail: [email protected] G. Di Marzo Serugendo Birkbeck College, University of London, London, UK e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_12, © Springer-Verlag Berlin Heidelberg 2011

283

284

C. Bernon et al.

12.1 Introduction Complexity of near future and even nowadays applications is exponentially increasing. A promising approach for tackling complexity, providing robustness and adaptation, consists in following self-organising principles. Designers of self-organising systems have been taking inspiration from natural systems in which complex structures or behaviours appear, at the global level of a system, as a result of the interactions among its lower-level components. The phenomenon observed at the macrolevel emerges by self-organisation of the micro-level components making up the system. From an engineering point of view, the potential of this approach is important because the individual components should be simple to design and develop while the system as a whole exhibits a complex behaviour. In order to develop a complex system, designing its individual components (called agents), providing them with means to self-organise and enabling them to interact with the environment is sufficient. But doing this is not so easy, as Van Parunak and Zambonelli [28] have claimed: “Such behaviour can also surface in undesirable ways”. Indeed, systems can reach undesirable states because the main difficulty lies in controlling the global behaviour while designing the micro-level. Three main challenges have to be overcome when designing self-organising applications. The first consists in answering the question “how to control emergence” or, in other terms, “how to control the system behaviour at the macro-level by only focusing on the design of agents at the micro-level?” in order to obtain the expected functionality while avoiding harmful global phenomena. The second challenge, on which this chapter focuses, is to provide tools, models and guides for developing such systems. Because the goal of engineering self-organising systems is to deliver systems with a global behaviour which meets the requirements or realises the expected function. The third challenge is about how to validate and to verify these systems: did we build the intended system and did we build it without errors or undesired behaviour? The first and obvious problem software designers encounter when trying to engineer complex systems lies of course in their nature, complexity. How can we build something we do not even fully understand? Since the years 2000, the agent-oriented methodology field is in full rise; numerous new methodologies devoted to particular problems have appeared [15], but very few of them are devoted to designing multi-agent systems generating emergent functionalities. In this chapter, we present a series of methods specifically tackling the development of self-organising systems, and we discuss in more details ADELFE, the most mature and complete one.

12.2 Background and Context Before discussing some methods to help designing self-organising systems, the reader must be familiar with notions related to agent, multi-agent, and self-organisation concepts which are defined in Chap. 2. The reader is expected to have some general knowledge of software engineering methods, of object-oriented ones such as Rational Unified Process (RUP) [17], of the Unified Modeling Language (UML)

12

Engineering Self-organising Systems

285

Fig. 12.1 Customised unified process methodology [7]

notation [24] and some meta-model notions. Furthermore, the expounded method, ADELFE, is based on the AMAS (Adaptive Multi-Agent Systems) theory, and consequently Chap. 9 must be read before discovering this chapter.

12.3 Engineering Self-* Systems: An Overview As already said in the introductory section, research on self-organising systems is in rapid development.1 Engineering such systems is not an easy task, and only a few agent-oriented development methodologies are provided for the time being. This section briefly reviews some recent works in this domain.

12.3.1 The Customised Unified Process The Customised Unified Process (CUP) [7] is an iterative process that provides support for the design of self-organising emergent solutions in the context of an engineering process. It is based on the Unified Process (UP) [16] and is customised to explicitly focus on engineering macroscopic behaviour of self-organising systems (see Fig. 12.1). During the Requirement Analysis phase the problem is structured into functional and non-functional requirements, using techniques such as use cases, feature lists and a domain model that reflects the problem domain. In particular, macroscopic requirements that the system at the global level must display are identified. In order 1 As

suggested by the SASO conference, www.saso-conference.org.

286

C. Bernon et al.

to determine the actual suitability of a self-organising solution, these requirements are analysed on the basis of characteristics such as autonomy of behaviour, need of decentralised information, and high dynamics (failures or frequent changes). The Design phase is customised into two steps: Architectural Design and Detailed Design addressing microscopic issues. The second step proposes to use general guidelines or references architectures or decentralised mechanisms that allow coordination. Two main concepts are introduced in support of the design: Information Flows (a design abstraction), which traverse the system and form feedback loops; and Locality, which is ‘that limited part of the system for which the information located there is directly accessible to the entity’ [7]. Activity diagrams are used to determine when a certain behaviour starts and what its inputs are. Information flows are enabled by decentralised coordination mechanisms, defined by provided design patterns. During the Implementation phase, the design is realised by using a specific language. When implementing, the programmer focuses on the microscopic level of the system (agent behaviour). In the Testing and Verification phase, an empirical approach based on iterative development feedback is proposed. This consists in testing the system obtained after the implementation and verifying that the performances with respect to the macroscopic properties are achieved. If it is not the case, this feedback enables to adapt the design at the design phase. The design can be broken up into early and later iterations. In the early iterations, the design focuses on coarse-grained decisions at the architecture level. The later iterations can be viewed as parameters tuning. Verification is performed through agent-based simulations combined with numerical analysis algorithms for dynamical systems verification at the macro-level. The CUP approach has been applied to autonomous guided vehicles and document clustering [7].

12.3.2 MetaSelf MetaSelf proposes both a system architecture and a development process [10]. It addresses the development of dependable and controllable self-* systems (both self-organising and self-managing). MetaSelf considers a self-organising system as a collection of loosely coupled autonomous components (agents or services). Metadata (data about the running system) describes the components’ functional and non-functional characteristics, such as availability levels, and environment-related metadata (e.g. artificial pheromones). The system’s behaviour (self-organisation and self-management, e.g. reconfiguration to compensate for component failure) is governed by rules for self-organisation and policies for control, dependability and selfmanagement that describe the response of system components to detected conditions and changes in the metadata. When the system is running, both the components and the run-time infrastructure exploit dynamically monitored and updated metadata to support decision-making and adaptation in accordance with the rules

12

Engineering Self-organising Systems

287

Fig. 12.2 MetaSelf development process

and policies. The MetaSelf system architecture involves autonomous components, repositories of metadata, rules for self-organisation and policies for dependability and self-adaptation, and reasoning services which dynamically enforce the policies on the basis of dynamically updated metadata values. Metadata may be stored, published and updated at run-time by the run-time infrastructure and by the components themselves, both of which can also access rules and policies at run-time. The MetaSelf development process consists of four phases (Fig. 12.2). The Requirement and Analysis phase identifies the functionality of the system along with self-* requirements specifying where and when self-organisation is needed or desired. The required quality of service is determined. The Design phase consists of two sub-phases: D1—the designer chooses architectural patterns (e.g. autonomic manager or observer/controller architecture) and self-* mechanisms (governing the interactions and behaviour of autonomous components (e.g. trust, gossip or stigmergy). Generic rules for self-organisation and control and dependability policies are defined. In the second part, D2—the individual active autonomous components (services or agents) and the passive ones (artefacts) are designed. The necessary metadata, rules and policies are selected and described. The self-* mechanisms are simulated and possibly adapted or improved. The Implementation phase produces the run-time infrastructure including agents, artefacts, metadata and executable policies. In the Verification phase, the designer makes sure that agents, the environment, artefacts and mechanisms work as desired. Potential faults arising on one of these design elements and their consequences are identified. Corrective measures (redesign or dependability policies) to avoid, prevent or remove the identified faults are taken accordingly [8]. The MetaSelf development process has been applied to dynamically resilient Web services [6] and to self-organising industrial assembly systems [11].

288

C. Bernon et al.

Fig. 12.3 Methodology steps, adapted from [13]

12.3.3 A General Methodology The General Methodology [13] provides guidelines for system development. Particular attention is given to the vocabulary used to describe self-organising systems. This is a domain-independent methodology addressing the design and control of systems intended for complex problems solving. This iterative and incremental methodology includes five steps or phases depicted in Fig. 12.3. In the Representation phase, according to given constraints and requirements, the designer chooses an appropriate vocabulary, the abstraction level, granularity, variables, and interactions that have to be taken into account during system development. Then the system is divided into elements by identifying semi-independent modules, with internal goals and dynamics, and with interactions with the environment. The representation of the system should consider different levels of abstractions. In the Modeling phase, a control mechanism is defined, which should be internal and distributed to ensure the proper interaction between the elements of the system, and produce the desired performance. However, the mechanism cannot have strict control over a self-organising system; it can only steer it. To develop such a control mechanism, the designer should find aspects or constraints that will prevent the negative interferences between elements (reduce friction) and promote positive interferences (promote synergy). The control mechanism needs to be adaptive, able to cope with changes within and outside the system (i.e. be robust) and active in the search of solutions. It will not necessarily maximise the satisfaction of the agents, but rather of the system. It can also act on a system by bounding or promoting randomness, noise and variability. A mediator should synchronise the agents to minimise waiting times. In the Simulation phase, the developed model(s) is (are) implemented, and different scenarios and mediator strategies are tested. Simulation development proceeds in stages, from abstract to particular. The models are progressively simulated, and based on the results, the models are refined and simulated again. The Application phase is used to develop and test model(s) in a real system. Finally, in the Evaluation phase, the performances of the new system are measured and compared with the performances of previous ones. This methodology was applied to traffic lights, self-organising bureaucracies and self-organising artefacts [13].

12.3.4 A Simulation Driven Approach The Simulation Driven Approach (SDA) [12] to build self-organising systems is not a complete methodology, but rather a way of integrating a middle phase into ex-

12

Engineering Self-organising Systems

289

Fig. 12.4 Design phases—adapted from [12]

isting methodologies, such as for example Gaia [29]. To describe the environment, suitable abstractions for environmental entities are necessary: the Agent & Artefact metamodel [21] considers agents as autonomous and proactive entities driven by their internal goal/task. Artefacts are passive, reactive entities providing services and functionalities to be exploited by agents through a user interface. To overcome many methodologies’ limitations regarding the environment, environmental agents are introduced. They are responsible for managing artefacts to achieve the targeted self-* properties. Environmental agents are different from standard agents (user agents), which exploit artefact services to achieve individual and social goals. SDA is situated between the analysis and the design phase, as an Early design phase (Fig. 12.4). It assumes that system requirements have just been collected and the analysis has been performed, identifying the services to be provided by environmental agents. To design environmental agents, a model of agents and environmental resources is needed. This model is analysed using simulation, with the goal to describe the desired environmental agent behaviour and a set of working parameters. These are calibrated in a tuning process. SDA consists of three iterative phases. During the Modeling phase, strategies are formulated to make the system behaviours explicit. To enable further automatic elaborations and reduce ambiguity, these descriptions should be provided in a formal language (not specified in [12]). The model is expected to provide a characterisation of user agents, artefacts and environmental agents. Feedback loops are necessary in the entire system. In the Simulation phase, the created specifications are used in combination with simulation tools, to generate simulation traces. These will provide feedback about the suitability of the created solution. This step uses stochastic simulation to study the system dynamics through statistical analysis of results, considering that proper parameters are provided for artefacts and agents. In the Tuning phase, the model has to be tuned until the desired qualitative dynamics is reached, which depends on initial conditions. The tuning process may provide unrealistic parameter values or may not reach the required behaviour. This means that the chosen model cannot be implemented in a real scenario. The designer then needs to return to the modelling phase and start again with a new model. This methodology was applied to collective sorting and to plain diffusion [12].

290

C. Bernon et al.

12.3.5 Tropos4AS Tropos4AS [18], an ongoing extension of the Tropos methodology [5], is a framework for designing self-adaptive systems. Characteristics enabling self-adaptation at run-time are captured during the design phase. The adopted approach is goaloriented and guides the designer through different modelling steps: conceptual modelling for self-adaptivity (by extending the Tropos goal model with goal types and relationships), failure modelling (by complementing the Tropos goal model with undesired states and recovery actions) and modelling of alternatives (by proposing a design pattern; alternative configurations used by the system to adapt are identified as well as their possible constraints). Although nothing here self-organises, Tropos4AS enables prevention of failures by modelling errors and recovering actions at design time. For dealing with the modelling of organisations that are able to self-organise in order to reach their goals in an unknown, changing environment, we combine TAS, an extension of Tropos for adaptive systems, with concepts, guidelines and modelling steps from the Adelfe methodology, which provides a bottomup approach for engineering collaborative multi-agent societies that exhibit emergent behaviours [19]. Other approaches are more especially interested in MAS such as the ones presented in [18] and [12] in which agent-oriented development methodologies are involved. The remaining of this paper has then been chosen to present ADELFE,2 a more mature and equipped agent-oriented methodology dedicated to engineering selforganising MAS according to the AMAS (Adaptive Multi-Agent Systems) theory introduced in Chap. 9.

12.4 ADELFE Methodology For several years now, the AMAS theory and its related concepts have been used to develop many systems in various application areas exploiting self-organisation by cooperation (see Chap. 9). However, this kind of programming is very specific, and tools were needed to ease and promote it among engineers; a dedicated methodology, ADELFE [1, 3], was thus designed for helping and guiding designers when developing self-organising multi-agent systems. In a general way, a methodology is made up of a process, some notation and tools to support the notation and/or help the developer [26]. ADELFE provides a specific process, supports UML and AUML3 notation and offers several main tools [2, 23]. An interactive tool describes the process and helps designers in applying it; the AMAS adequacy tool analyzes answers given by the designer to tell her if using the AMAS technology is useful 2 ADELFE is a French acronym meaning “toolkit to develop software with emergent functionality”.

It was a French RNTL-funded project (2000–2003). 3 http://www.auml.org/

12

Engineering Self-organising Systems

291

to implement the target system; a graphical modelling tool supports the adopted notation, and a model-driven engineering tool enables automatic transformations between models or from models into code. The aim of the following sections is to illustrate, through a case study, how the development process of ADELFE may be applied.

12.4.1 Case Study Overview For illustrating how ADELFE may be applied, a rather simple and well-known case study is followed through this chapter: the simulation of foraging ants [27], this study will be called ANTS hereafter. Natural ants, like all social insects, are animals with limited cognitive abilities, and yet all together they may perform difficult and complex tasks without any centralised control. Structures they build or collective tasks they perform are emerging phenomena coming from their individual behaviours. Designing the collective behaviour of an anthill is a complex thing for many reasons: ants are generally numerous, the environment in which ants operate is also evolving, the individual behaviours are not especially related to the global emergent one, and there is no a priori algorithm to express this collective behaviour. The aim of ANTS is to simulate the behaviour of an anthill, based on entomological data. According to ethologists, an ant possesses several distinct behaviours. An ant leaves its nest for foraging in a random way in the environment and picking up food when it finds some. Once loaded, an ant brings back the food it found to its nest, while dropping some pheromones along its way in order to connect this nest to the last found food area. When finding a pheromone trail, an ant tends to follow it. Therefore, this chemical substance enables other ants to find food and to continue exploiting it. Ants move in a bounded arena in which several ant nests may be located as well as obstacles that are inaccessible areas. Some food items are also randomly distributed within the arena. An ant has a local perception of its environment and is able to distinguish between other ants, food items, pheromone items and obstacles. Since obstacles are also spread in the environment, an ant has also to avoid them. As in natural environments, an ant communicates with others in an indirect way by dropping pheromone when it brings back some food. The pheromone can be piled up and evaporates with time. Simulating foraging ants aims at studying how much food is globally brought back to the nest. Furthermore, ants are endowed with a cooperative social attitude, in the AMAS sense, in order to evaluate whether the resulting cooperative ants behave in a more efficient way than natural ones, namely: do cooperative ants bring back more food to the nest than natural ones?

292

C. Bernon et al.

Fig. 12.5 Process of ADELFE, based on the RUP

12.4.2 ADELFE Design Process The primary objective of ADELFE is to cover all the phases of a classical software design, from requirements to deployment. ADELFE provides a specific process adapted from an interpretation of the RUP (Rational Unified Process [17]) according to the Neptune project,4 this process can be seen on Fig. 12.5. Phases are called WorkDefinitions (WDi), Activities (Aj) or Steps (Sk) following the vocabulary of the OMG’s SPEM (Software Process Engineering Metamodel [20]), which has been used to express the ADELFE process [14]. RUP has been tailored to take into account specificities coming from the design of adaptive multiagent systems. Accordingly (preliminary and final) requirements, analysis and design work definitions were modified. For example, some additions have been done such as the characterisation of the environment of the system or the identification of cooperation failures. The implementation phase is currently under study, and the following phases remain the same. For lack of space and because the non-AMAS-specific parts are based on objectoriented software engineering principles, only the stages or steps that are closely related to the self-organisation concepts used in the AMAS theory will be described hereafter. Their description is sequential, but, of course, a designer may back track between the different stages, like in the RUP.

12.4.3 WD1 and WD2—Preliminary and Final Requirements With respect to an object-oriented methodology, ADELFE adds nothing to preliminary requirements (WD1) as described by the RUP. The aim still consists in studying 4 http://www.neptune.irit.fr

12

Engineering Self-organising Systems

293

the customer needs for producing a document on which both customer and designer agree. This is done by: • defining (A1) and validating (A2) user requirements about the system-to-be considering both functional and non-functional points of view, • defining consensual requirements on which all stakeholders agree (A3), • establishing the set of keywords and concepts that describe the system and its environment (A4), • extracting the limits and constraints of this system (A5). Final requirements (WD2) complete preliminary ones for giving a description of the functionalities the system has to fulfil relative to a specific domain. The activities involved in this work definition are the following: • characterise the environment in which the system operates (A6), • determine use cases (A7), • define (A8) and validate (A9) user interfaces that will be required. The last two activities are classical ones; however, the former one was modified, and AMAS specific features are detailed hereafter. Characterising the Environment Since the adaptation process of a system depends on the interactions between this system and its environment, this latter is central in the AMAS theory. Therefore, during the final requirements study, the environment is characterised by the designer (WD2-A6) before determining use cases. The first step in this activity (A6-S1) is to determine entities and qualify them as passive or active. Entities are active if they are able to act in an autonomous and dynamic way in the environment; they are passive if they can be considered as mere resources. This distinction is essential because agents in ADELFE are not known or given a priori, and they will be found later among active entities. However, depending on the nature of the system designers have to build, finding these entities may be done in two different ways: • If the system has to solve a problem (e.g. constraint satisfaction problems, optimisation problems), its environment generally involves the human beings or the objects that may interact with it during its execution. The designer has therefore to determine the entities among them, i.e. actors in the UML sense, along with the constraints on these interactions. An example of such a system may be found in the guide associated with ADELFE5 in which the problem to be solved is a timetabling one. • If the system has to simulate some behaviour (e.g. the ANTS problem, social simulation, modelling of a biological process), this system may be considered as the collective for which the behaviour has to be defined. Its environment involves therefore all the entities that may interact with this collective. In order to determine them, the designer may study the documents established during the domain analysis activity (A4). 5 See

the web site of ADELFE, http://www.irit.fr/ADELFE.

294

C. Bernon et al.

In the case study, the considered system is therefore the anthill (the collective of ants), and its environment is what may be found in the arena in which ants move. Therefore, ants and pheromone may be considered as active entities (because ants perceive their environment in which they may move, and pheromone may evaporate and then evolve) and the nest, food items, and obstacles are viewed as passive ones (because they just exist and do not evolve, they are resources). Defining the context (A6-S2) studies then interactions between entities and the system which are expressed through UML collaboration or sequence diagrams. Finally, the environment must be described (A6-S3) using terms inspired from [25]. The environment may therefore be dynamic or static, accessible or not, deterministic or not, or discrete or continuous. In ANTS, the environment of the target system is characterised as being: • Dynamic: the evolution of the environment during the simulation cannot be predicted by the system (anthill) because ants continuously modify its state. • Inaccessible: the system (anthill) is unable to obtain all the information about the state of the environment even if this information is distributed among ants. • Non-deterministic: the system is unable to know what the effects of its actions on the environment are, ants’ actions may have different effects. • Discrete: the number of interactions is finite, ants only have limited possible actions on the environment. Identifying Cooperation Failures In order to focus on cooperative agents (see Chap. 9) that operate in AMAS, a step was added in the next activity (A7); its aim is to identify use cases and Non-Cooperative Situations (NCS). The designer has then to think about the kind of situations which may prevent future agents to remain always cooperative and that could lead to future Non Cooperative Situations. These “cooperation failures” that may occur between the system and its environment, have to be highlighted on the previous identified use cases (A7-S2). Considering the case study ANTS, nothing unexpected may occur between the system and its environment, however in a system such as an E-commerce application problems may appear when agents communicate. For example, requests coming from customers may be ambiguous or unintelligible for providers, and this will may lead to harmful situations for agents the role of which is to answer these requests. To conclude, the requirements workflow provides an environment model which is composed of a set of actors, a use-case model and a characterisation of the system’s environment.

12.4.4 WD3—Analysis The analysis phase has the same objective as a classical object-oriented analysis, understanding the domain in which the system-to-be has to evolve and structuring it into interacting components. Therefore the first activity analyses the domain (WD3-A10) by identifying components and clustering them into a preliminary class

12

Engineering Self-organising Systems

295

Fig. 12.6 Interface of the AMAS adequacy tool

diagram, and the last activity studies the system dynamics through the interactions between its components (A13); this activity remains the same as in the RUP (WD3A10). However from a multi-agent point of view, analysis has also to identify whether the agent technology is useful and what components may be viewed as agents. Two activities mainly related to AMAS are then added (A11–A12). Verifying AMAS Adequacy As it was said before, not every system requires to be implemented using the AMAS theory. The first added step in WD3 helps therefore designers verifying whether using the principle of self-organisation by cooperation is useful (A11). A tool is provided to enable this verification by answering a certain number of questions related to both the system (or global) level and the entities (or local) level (see Fig. 12.6). To decide if implementing an AMAS is useful at the global level, eight criteria were defined, let us apply them to ANTS: 1. Is the global task completely specified? Is an algorithm a priori known? In the case study, the designer cannot determine an efficient algorithm in advance for she does not know where the food items take place at run-time; the research space is too big to know a solution.

296

C. Bernon et al.

2. If several components are needed to solve the global task, do they need to act in a certain order? The global task is performed by a great number of components which interact: the foraging ants which form a collective. Their activities are correlated in the sense of common foraging tasks. 3. Is the solution generally obtained by repetitive tests, are different attempts required before finding a solution? In this case, it is quite sure that a solution is not usually found by repetitive tries; ‘No’ can be answered. 4. May the environment of the studied system be dynamic? By nature, the environment is dynamic due to the appearance of new food items and obstacles. 5. Is the system functionally or physically distributed? Are several physically distributed components needed to solve the global task? Or is a conceptual distribution needed to solve it? By nature, ants act autonomously to solve a global task. So they are geographically and functionally distributed. 6. Is a large number of components needed? The number of ants in a nest may be in the order of several hundreds entities, which represents quite a relatively large number. 7. Is the studied system nonlinear? May a little modification of a local behaviour have a great impact on the global result? The changes an ant does in the environment, by dropping pheromones, may have a great impact on the collective behaviour. 8. Finally, is the system open or evolutionary? Can new components appear or disappear dynamically? No, ant or nest addition or removal is not specified in the primary requirement set. After having given these different answers, the designer can obtain the verdict at the bottom part of the interface window. This result can be seen in Fig. 12.6: an AMAS is adequate to the ANTS system at the global level. The local level has therefore to be examined to know whether some components require being recursively designed as AMAS. Three criteria were defined to try and answer this question: 9. Does a component have a limited rationality only? An ant does not have a complete knowledge of its environment. 10. Is the component “big” or not? Is it able to do many things, to reason? Or does it need simple abilities to perform its own task? At first sight, an ant is a very simple organism that does not require complex cognition. Ants are perfect examples of reactive organisms. 11. May the behaviour of a component evolve? Ants do not have any learning or adaptation capabilities. All these points come to the conclusion that the AMAS approach is well adapted to the ANTS problem. Identifying Agents and Their Relationships Therefore, cooperative agents have to be identified in the next activity (A12). Its aim is to study the active entities (A12S1) that were identified in WD2-A6-S2 (ants and pheromone) in order to evaluate which ones may be considered as potential cooperative entities (A12-S2). A cooperative entity exhibits specific properties: autonomy, local goal to pursue, ability to

12

Engineering Self-organising Systems

297

interact or to negotiate with others, partial view of its environment. For instance, in ANTS, a pheromone drop is an active entity because it can appear and evaporate in the environment; however, it cannot be considered as a potentially cooperative one because it has no autonomy (it is produced by an ant) or interaction aptitude. On the contrary, an ant is autonomous (it roams in a random manner in the environment, picks up food and drops pheromone), has a goal (picking up food and bringing it back to the nest) and is able to interact with other ants (even if an indirect way). An ant can therefore be considered as a potentially cooperative entity. An active entity that is not considered as being cooperative remains a mere object, while a cooperative one may become an agent. To actually be an agent, this entity has to be prone to cooperation failures, and this is determined by studying its interactions with the environment and with other entities (A12-S3). For example, the environment of an ant is dynamic; food and pheromone items appear and disappear, other ants are also moving in this environment. Therefore, an ant may try to access a food item that does not exist anymore, or several ants may try to access the same food item. In this case study, ants will then be considered as cooperative agents. In systems in which entities interact in a direct way, some other interaction issues may occur. For instance, an entity does not understand what another one “tells” it; this kind of situation is a priori unexpected and may be harmful for the cooperative state of a potential cooperative agent. Once passive entities, active entities and agents are identified, the designer has to study their relationships using AUML protocol diagrams (A13-S3). For doing this, she has to describe some scenarios with sequences diagrams in which she highlights each potential cooperation failure by a dotted line associated with a descriptive comment. In the ANTS example, agents communicate through the environment, in a indirect way, and such a protocol is therefore useless. Let us consider an E-commerce application for exemplifying this point. Figure 12.7 shows a protocol diagram which expresses how agents may interact during a room booking. The scenario chosen on this figure expresses only one of the many possible behaviours agents may have. A customer asks its representative agent (BookingAgent) to book a room in a hotel by submitting a request including some details about what he wants. This agent forwards this request to a mediator agent (MediatorAgent). This MediatorAgent knows at least one agent representing a hotel (in this case, HotelAgent1) which is able to fulfil this request and therefore, forwards this request to that agent. Unfortunately, in this scenario, this HotelAgent is not able to provide it with the requested service (e.g. no room available, hotel under repair, etc.). However, this HotelAgent has some representations (beliefs) on other HotelAgents, and as it is cooperative, it is going to forward the received request towards another HotelAgent it believes relevant to fulfil it (here, HotelAgent2). The latter is actually relevant and answers directly to the BookingAgent which forwards this proposal to its customer. Once the customer has accepted the proposed service, its BookingAgent sends this confirmation to the involved HotelAgent2 as well as the MediatorAgent in order to let it update its knowledge about both HotelAgents. In order to keep the diagram light, some behaviours are not shown, e.g. what a MediatorAgent does when it is not competent. This additional behaviour should be expressed using further sequence diagrams.

298

C. Bernon et al.

Fig. 12.7 Example of interaction protocol between agents

Cooperation failures (incomprehension) may take place when messages are received and not understood by the receiver agents. Possibly cooperation failures (incompetence) may also appear when agents are not relevant for doing something asked by others (for instance, a HotelAgent cannot provide some service requested by a MediatorAgent). These reminders will be used when agents are designed and non-cooperative situations they may encounter have to be exhaustively listed by the designer (WD4-A16).

12.4.5 WD4—Design The aim of the design (WD4) is to define entities and mechanisms which provide the behaviour defined during the analysis (WD3) and the detailed architecture of the system. This design enables to understand how the system will be built from the analysis phase. Design may start as soon as a model of the system is carried out. This model will be progressively enriched during the design. At the end of this phase, software components to be coded must be defined. Activities involved in this work definition are the following: • Study detailed architecture and multi-agent model (A14). This activity of the design phase does not change with respect to the RUP. The activity A14 aims at identifying and describing software components. The result is a set of packages, classes, agents and interactions.

12

Engineering Self-organising Systems

299

• Study interaction languages (A15). In this activity the way in which agents interact in a direct manner is studied. For scenarios defined in A7 and A13, information exchanges between agents are described by AUML protocol diagrams. Languages that enable these interactions are implemented (set of classes, design patterns, FIPA ACL implementation). This activity is not studied in the ANTS application because ants do not interact directly, they only deposit pheromone items that could be perceived by other ants. • Design agents (A16). This activity is specific to ADELFE and enables designing the agents that were previously identified during the analysis phase. This activity will be detailed below. • Fast prototyping (A17). This activity is specific to ADELFE too and enables a designer to test the behaviour of the previously designed agents. This activity will be detailed below. • Complete design diagrams (A18). This last activity of design is a classical one in which the detailed architecture is finalised by enriching class diagrams (A18S1) and developing state-chart diagrams that are required to design dynamic behaviours (A18-S2). Designing Agents The AMAS meta-model was elaborated from the ADELFE meta-model [4, 22], enriched by three distinct logical points of view. Each of them represents a specific part of the AMAS theory on which emphasis is put: • System point of view: it is devoted to the description of the system and its surroundings in terms of entities which populate it (discernible objects of the “world”). • Agent point of view: this part of the meta-model represents agent internal characteristics. • Cooperation point of view: it represents taxonomy of non-cooperative situations an AMAS agent is likely to encounter. We focus on the agent point of view because it is the relevant part for cooperative agent design. An AMAS agent is made up of various modules and parts that are managing a sector of its activities and life cycle as explained previously. Typically, the AMAS agent life cycle is defined according to three phases: perception, decision and action. From these phases and the needs they imply in terms of environmental interactions, knowledge representation, non-cooperative situations avoidance, etc. the following meta-model concepts are determined: • Environmental interactions are represented by Perception and Action on the Entities as well as the means to carry them out (Actuator, Sensor). CommunicationAction consists in direct interaction with other agents by the means of messages (Message) whose protocol is defined in the System point of view (CommunicationProtocol). • DecisionModule gathers the Aptitudes and the CooperativeRules enabling an agent to determine the next actions to carry out. This decision is taken according to the agent knowledge and aptitudes as well as its cooperation rules which propose actions to overcome possibly detected NonCooperativeSituations (NCS).

300

C. Bernon et al.

In the absence of NCS detection, an agent carries out its local and nominal function determined by its aptitudes. • Knowledge represents what an agent possesses. It has Representations of itself and Representations of the medium surrounding it (environment entities, agents of the system). It also possesses Skills and Characteristics that are possibly discernible by other agents of the system (isPerceptible). According to the cooperative agent paradigm, an agent is made of several parts and modules which have to be filled up from steps A16-S1 to A16-S5. They represent the main abilities or specificities of an agent such as: Skills, Aptitudes, Representations and Characteristics; they also represent the way this agent interacts with its environment: Action Module, Perception Module, Action, Perception and the means that are involved, Actuators and Sensors. • First, skills represent knowledge about a domain allowing an agent to execute actions. They are described using rules behaviours (A16-S1). In the ANTS case study, for instance, an ant knows how to determine its next position in the environment. An ant lays down pheromone tracks in order to communicate the location where food items were discovered. This task involves the use of a specific action dropPheromone() which is fulfilled thanks to the ExocrynGland actuator. This information is useful for the “extraction” of the agent architecture, it indicates which part of the agent is responsible for performing an action. • Aptitudes are the capabilities an agent has in order to reason. They are based on its knowledge about the domain or on its representation of the world (A16-S2). Usually aptitudes are generic tools used by an agent. In ANTS, an ant fills an interpretedSurroundings grid with integer values (the more appealing a position is, the bigger the integer value), and then it may apply a Monte Carlo method to find its actual next position. This Monte Carlo method represents an aptitude for an ant. This method was chosen because it is a generic one; however, an ant could use something else for finding its position. • The aim of the A16-S3 step is to choose, among those previously defined in A15, an interaction protocol for an agent. In the ANTS example, an ant does not use direct communication and therefore does not use an interaction protocol. • The perception consists in filling the different Representations with fresh values. In the case study, an ant may be able to know whether or not it is positioned on the nest, food or pheromone. It also knows whether it is nor far away from another ant, food, pheromone or an obstacle. For instance, the food array corresponds to the position where food has been perceived. Furthermore, agents possess some intrinsic or physical characteristics. An ant, for instance, knows whether it is loaded or not with food, for how long it has been out of the nest, or where is the nest location. • Finally, the most important step in this activity is A16-S5 which aims at enumerating and describing the situations that a cooperative agent judges being harmful for its cooperative social attitude. These non-cooperative situations (NCS) belong to different types (such as ambiguity, or conflict—see Chap. 9).They depend on some conditions that are fulfilled or not when an agent performs a certain phase

12

Engineering Self-organising Systems

301

Table 12.1 Description of NCS an ant-agent may encounter Name State

Description

Conditions

Actions

NCS1 Exploration

An ant meets another No pheromone and no Move in a different diant which does not carried food rection than the met carry any food ant’s one (for not exploring an already explored area)

NCS2 Exploration

Two food sources are Two food resources are Move towards the food perceived perceived, one is al- that is not exploited by ready exploited by sev- a lot of ants eral ants, and the second is not

NCS3 Pheromone_ An ant perceives a food A food item is per- Move towards the perFollow_Up item when it follows a ceived ceived food item and pheromone track drop a greater amount of pheromone NCS4 Go_Back

An ant perceives a food A food item is per- Increase the amount of item while coming back ceived, and the maxi- pheromone dropped to the nest mum quantity of food an ant can carry is reached

NCS5 Go_Back

An ant comes back to The maximum quantity Do not drop pheromone the nest without a com- of food is not reached plete amount of food

NCS6 Rest

A resting ant tries to recruit nest mates depending on the quantity of food perceived while returning to the nest

The quantity of carried Increase recruitment of food is not empty, and nest mates some food is remaining in the environment

in its life cycle (perceive, decide, act). All the potential NCSs an agent may encounter have to be enumerated for every cooperative agent already identified. For ANTS, all the NCSs an ant agent may encounter are described in Table 12.1 with the following information: the state of the involved agent when detecting this NCS, a textual description of this NCS, conditions describing the different elements permitting local detection of this NCS, and the actions describing what the agent has to do to remove this NCS. So when designing an agent, we express the skills and the cooperative attitude with two kinds of rules (see Fig. 12.9): the Standard Behaviour rules which constitute the local function or goal assigned to an agent and the Cooperative Behaviour rules which are intended to manage Non Cooperative Situations. At this step in the development process, we are designing these rules as being triggered from an agent state, which is itself characterised by a logical expression over the Representations, Characteristics and Skills of the agent. A rule results in a set of actions, or skills that have to be accomplished in order to achieve a local goal or to regain a cooperative

302

C. Bernon et al.

Fig. 12.8 AMAS Agent Structural Diagram: detailed design for a cooperative ant-agent

state. Figure 12.9 shows an example of a graphical representation of those rules. It presents the avoidance of concurrence that could appear when ants are lusting for the same food item and the standard ant behaviour that consists in going back to the nest while dropping a pheromone trail when food has been collected. Fast Prototyping The last activity of design (A17) is specific to ADELFE and enables a designer to test the behaviour of the agents designed in the previous steps using simulation of state-machines. Therefore a protocol diagram can be automatically transformed into a state-chart in order to detect, for instance, inconsistencies and deadlocks in protocols, or sufficiency of methods.

12.4.6 WD5—Implementation According to the specific activities of the ADELFE method, we designed a specific modelling language. This graphical language, called AMAS-Modelling Language (AMAS-ML), which is domain-specific, improves the expressiveness of the modelling, compared to the use of general-purpose modelling language such as UML. AMAS-ML has been designed based on the AMAS meta-model presented in Sect. 12.4.5. AMAS-ML enables graphical diagrams such as agent structural diagram (shown in Fig. 12.8) or agent behavioural rules diagram (shown in Fig. 12.9). From the design phase and the AMAS-ML agent diagram, we propose to automatically generate an abstract agent architecture. This architecture is made up with micro-components which specification could be edited and modified with μADL

12

Engineering Self-organising Systems

303

Fig. 12.9 Graphical representations of behavioural rules of ant-agents

Fig. 12.10 Phases and diagrams in ADELFE

(micro-Architecture Description Language) [23]. The result of this step is called the “Agent model”; it is used to generate a specific API which is given to the developer in order to complete the following stages of this phase. Figure 12.10 recalls the place of the implementation phase in the whole process and the use of diagrams all along the phases. In this section we describe the use of activities and the model transformations from the AMAS-ML model to the final source code.

304

C. Bernon et al.

12.4.6.1 Micro-architecture Generation (A19) The goal is to obtain an API which is specific to the application business domain. The result is a kind of abstract agent architecture which conforms to the flexible agent principles on which we base our developments [23]. This software architecture promotes a clear separation between operational concerns, which are more related to the environment and are more or less independent of the business domain, behavioural concerns ant foraging in our case. The μ-architecture obtained for an ant-agent can be seen on Fig. 12.11. Micro-architecture Extraction This activity is automated and carried out by means of a model-to-model transformation where source models are written in AMAS-ML and target ones are μADL diagrams. Our whole toolkit for implementation is based on Eclipse Modelling Framework (EMF)6 and we used the Atlas Transformation Language (ATL)7 as a model-driven tool. However, every other model transformation language like openArchitectureWare (oAW)8 could be used. Micro-architecture Detailed Specification The diagram obtained at the previous step can be manually edited in order to introduce different choices for the separation of concerns or to introduce more hierarchical components. It is also possible to use round-tripping process in order to extract, from existing code, a description of components that can be used in μADL diagrams.

12.4.6.2 Code Generation (A20) The three activities in this code generation part are embedded in a tool called MAY.9 The goal is to produce the complete API source code that will be used in the final implementation. These activities are processed under the control of a Java developer who is responsible of decision making for the various alternatives. Abstract Micro-architecture Code Generation This first step of code generation produces Java code for implementing the abstract micro-architecture. The code is mainly made up of Java interfaces and a few classes that are generated according to the flexible agent principle [23]. In this step, there is no implementation of microcomponents, which explains why code generation concerns mainly interfaces. The software architecture which is obtained mixes research results on design patterns, component programming and reflection or aspect-oriented programming. 6 http://www.eclipse.org/modeling/emf/ 7 http://www.eclipse.org/m2m/atl/ 8 http://www.openarchitectureware.org/ 9 Make

Agents Yourself, http://www.irit.fr/MAY.

Fig. 12.11 μ-architecture for an ant-agent

12 Engineering Self-organising Systems 305

306

C. Bernon et al.

Micro-components Services Implementation The implementation of the microcomponents specified in the μADL diagrams can be obtained either by re-using previous ones or by a brand new development, which is often used because of the specific constraints of new environmental conditions. API Generation This final step generates the Specific Cooperative Agent API by merging interfaces and classes obtained in the two previous steps into a jar file that will be used in the last implementations steps that are described below.

12.4.6.3 Behavioural Implementation (A21) This part embeds the last two activities of coding. They deal with behavioural/business part of the application. The first one extracts as much as possible information from the model; the resulting source code can be enriched manually during the second one. Behavioural Code Generation This final model transformation allows one to generate code for the behavioural part of the agent, i.e. the business logic of our foraging ants. This step uses the rules that have been described in the design phase and generates code skeletons that ease the developer’s task. The goal is to provide the basic mechanisms for decision making in every agent. Behavioural Code Implementation Behavioural code can be manually modified in order to complete the implementation details that were not described in the model.

12.4.7 ADELFE Conclusion Half a decade of experiments, several projects (e.g. an Intranet system design, a timetabling problem, a mechanical self-design system, etc.) and its use by engineers of UPETEC Ltd10 have shown good results when using a methodology like ADELFE. ADELFE is a multi-agent-oriented methodology especially suited to self-organising multi-agent systems based on the AMAS theory. It is a complete methodology covering all phases from requirements to implementation phases. By using Model-Driven Engineering techniques, ADELFE reduces even more the development effort. In this chapter, only the activities specific to self-organising systems were detailed, while the other activities of the RUP used for classical systems engineering were not described; however, a complete description of ADELFE is available on its web site.11 10 http://www.upetec.com 11 http://www.irit.fr/ADELFE

12

Engineering Self-organising Systems

307

The reader must have in mind that ADELFE is not the only methodology to design self-organising systems. Since several ways exist for producing selforganisation, several methodologies have to be available in order to guide and help developers with crucial tasks (like, for instance, the ones that deal with cooperation in the ADELFE context). However, research in Agent-Oriented Software Engineering is still going on, and Agent-Oriented methodologies are continuously enhanced. We will see in the next section what can be the future trends in this huge field. But to stay focused on ADELFE, let us mention the work done on behaviour design. Whereas agent design has originally been defined as a single activity, experiments and studies have shown that nominal behaviour design and cooperative behaviour design should be executed iteratively.

12.5 Conclusion In this chapter, a synthetic view about existing methodologies for self-organising systems design was first provided. Then, the ADELFE methodology, which is currently the most complete methodology for self-organising systems engineering, was detailed. From an engineering perspective, if we look at what happened in the Multi-Agent Systems domain, now a huge number of development methodologies exist and the difficulty, from a designer’s point of view, is to know which one to use. For selforganising systems engineering, it seems that in a very near future, the number of methods will increase. So, we must think to provide, for each technique and associated method, a means to choose the most appropriate one for solving a specific problem or for developing a particular application. To conclude, we review the three main challenges to overcome in order to design self-organising applications. The first one concerns production and control of selforganisation. To answer to this challenge, there are many techniques (described in this book) such as stigmergy, cooperation, gossip, natural selection, attraction and repulsion, potential fields, social relationships, trust, etc. The second challenge is to provide frameworks and methodologies for helping designer reducing the development time. The works presented in this chapter are devoted to take up this challenge. The third challenge is about validation and verification of the designed systems. Given that emergence and self-organisation have not seriously been studied as hard science subjects, classical formal methods are not suitable. Verification and validation of engineered complex systems can only be achieved by using simulation-based approaches. Nowadays, the most reliable way consists in iteratively improving the designed systems using mathematical tools (statistical analysis, behavioural parameters optimisation) or semi-autonomous adaptive programming (Living Design). It is quite obvious that the software validation phase, requested by industrials and endusers, is required before its marketing. So, validation of self-organising applications is, even more, a mandatory step during development. The research of self-organising systems community must now focus on this main issue.

308

C. Bernon et al.

12.6 Problems–Exercises 12.1 Manufacturing Control Problem. Within a factory, several types of management and control are needed in order to deliver the right product to the right customer at the right time. This exercise focuses on the part which controls and coordinates the internal logistics of a manufacturing department. This manufacturing problem consists in delivering a detailed schedule that describes how available resources are allocated to tasks that have to be performed for manufacturing expected products and how these tasks are scheduled on these resources while respecting some constraints (e.g. precedence or temporal constraints). This complex problem may also be seen as a satisfaction and optimisation one. In this exercise, the problem considered is based on a case study proposed in the “Self-Organization in Multi-Agent Systems” Technical Forum Group of AgentLinkIII by Paul Valckenaers [9]. Specifications of the problem are as follows. The system-to-be is a department of a factory which processes parts of products that have to be delivered by a given due date for assembly in another department. Parts are carried into containers which capacities may vary. The system comprises a certain number of workstations with varying properties and capabilities. A human operator who has also specific capabilities, picks parts out of one container, processes them in a pipeline fashion on the machines in her workstation and places processed parts in another container at her workstation. An automated conveyor transports the container. In order to simplify the problem, parts are supposed to be similar in one container and the rail on which the conveyor moves is not handled. Depending on the processing that parts must receive, a container has to be handled by a certain number of workstations. We assume that containers know this process, a machining sequence. The allocation of a human operator to a workstation depends on her qualifications and constraints (breaks, working time and holidays). Data of the problem are: the number of workstations in the workshop and their competences description; the number of human operators as well as their qualifications and constraints; and the number of containers with their associated process. Perturbations may occur: machines and therefore workstations may break down; human operators may also be unavailable for some reasons. Question: Following the ADELFE methodology, identify which agents composing the adaptive MAS are able to solve this manufacturing problem. Use a selforganisation-by-cooperation approach. 12.2 Molecule Folding. Usually a drug is made up of a ligand, an active principle, and a protein that transports it. Finding the right protein for a given ligand amounts to finding the functional and stable conformation of the protein (folding problem) and studying how both entities match (docking problem). The problem considered here is to find the functional three-dimensional structure of a molecule. A molecule is an assembly of atoms connected by strong (or covalent) and weak bonds. Covalent bonds define quasi-stable interatomic distances for a particular pair of atoms that share outer-electrons. Weak bonds exist between neighbouring atoms and result

12

Engineering Self-organising Systems

309

from attraction and repulsion forces that induce a potential energy; a balance is established at a certain distance (van der Waals’ ray). In this exercise, we consider that these forces follow a law such as Lennard-Jones’ potential. Finding one of the stable conformations of a molecule consists in minimising the energy level of the whole molecule while respecting its covalent bonds (a variation of 1 percent is tolerated). The problem is to design an adaptive MAS for finding the right conformation for a molecule. It is supposed that ADELFE was applied in order to identify limits and constraints of this system and cooperative agents within it (A12). This AMAS represents the studied molecule; cooperative agents are atoms. Indeed, an atom has a local goal (to decrease its energy level), is autonomous (it is able to move for changing its location in space in order to achieve its goal), has a partial view of the environment (it knows atoms it has covalent or weak bonds with), it may interact with other atoms through its bonds and it could be imagined that it may negotiate its location. Atom-agents are going to perceive their environment, decide and then act in order to decrease their local energy, and, globally, through their interactions, the whole energy of the molecule-system is also going to decrease. These agents were studied during activity A16. An atom-agent is characterised by its nature, its position and its energy level. Representations it has of itself or others involve these features. Depending on its location, it may perceive some neighbours (atom-agents with which it shares some covalent or/and weak bonds) and their features. It is able to evaluate the error on its covalent bonds. The only way an atom-agent has to act on its environment is to move in space; by changing its location it also changes its error on covalent bonds, its energy level and the energy level of its neighbours. Question: Considering the features already identified for atom-agents, identify non-cooperative situations (activity A16, step 5) encountered by these agents and how these agents may avoid or repair them.

Key Points • In order to engineer trustworthy self-organising systems, it is important to have methods that support the development of such systems; • Some methods are available, but they are still in their infancy. The ADELFE methodology is the most mature so far. It includes a complete development process and a series of tool; • Recurrent design elements include: (active) agents, (passive) artifacts modified by the agents and/or their environment, a set of self-organising rules for agent coordination; • Simulations are an essential tool for tuning parameters, verifying algorithms and observing global behaviour.

310

C. Bernon et al.

12.7 Further Reading Methodologies for Self-organising Systems: a SPEM Approach. This paper provides more information about the methods described in this chapter. The main characteristics of these methods are identified, expressed using SPEM fragments and illustrated through a common example. (M. Puviani, G. Di Marzo Serugendo, R. Frei and G. Cabri, 2009, IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT09), Milan, Italy.) Methodologies and Software Engineering for Agent Systems. A book surveying some agent-oriented methodologies (Gaia, Tropos, MaSE, ADELFE, MESSAGE, SADDE, Prometheus), tools (AUML, FIPA, infrastructures) and nontraditional approaches to AOSE. (F. Bergenti and M.-P. Gleizes and F. Zambonelli, 2004, Kluwer Publishing.) Agent-Oriented Methodologies. This book presents some agent-oriented methodologies (Tropos, MAS-CommonKADS, PASSI, Prometheus, Gaia, ADELFE, MESSAGE, INGENIAS, AOR), compares them and introduces the OPEN framework. (B. Henderson-Sellers and P. Giorgini, Idea Group Pub, NY, USA, 2005.) Agent-Oriented Software Engineering IX. Selected papers of the AOSE 2009 workshop. (M. Luck and J. Gomez-Sanz, Springer, LNCS 5389, 2009.) Software Engineering for Self-adaptive Systems. For software engineering techniques (architectures, verifications, etc.) related to self-adaptation—not specifically focusing on self-organisation. (B. Cheng, R. de Lemos, H. Giese, P. Inverardi, and J. Magee, 2009, LNCS 5525, Springer.)

References 1. Bernon, C., Gleizes, M.P., Peyruqueou, S., Picard, G.: ADELFE: a methodology for adaptive multi-agent systems engineering. In: Petta, P., Tolksdorf, R., Zambonelli, F. (eds.) Third International Workshop on Engineering Societies in the Agents World (ESAW-2002), Madrid, Spain. LNAI, vol. 2577, pp. 156–169. Springer, Berlin (2002) 2. Bernon, C., Camps, V., Gleizes, M.P., Picard, G.: Tools for self-organizing applications engineering. In: Di Marzo Serugendo, G., Karageorgos, A., Rana, O., Zambonelli, F. (eds.) Engineering Self-Organising Systems, Nature-Inspired Approaches to Software Engineering. Lecture Notes in Artificial Intelligence (LNAI), vol. 2977, pp. 283–298. Springer, Berlin (2004) 3. Bernon, C., Camps, V., Gleizes, M.P., Picard, G.: Engineering self-adaptive multi-agent systems: the ADELFE methodology, pp. 172–202. Idea Group Publishing (2005). Chap. 7 4. Bernon, C., Gleizes, M.P., Peyruqueou, S., Picard, G.: A study of some multi-agent metamodels. In: Giorgini, P., Mueller, J., Odell, J. (eds.) Fifth International Workshop on AgentOriented Software Engineering (AOSE-2004). LNCS, vol. 3382. Springer, New York (2004) 5. Castro, J., Giorgini, P., Kolp, M., Mylopoulos, J.: Tropos: a requirements-driven methodology for agent-oriented software. In: Henderson-Sellers, B., Giorgini, P. (eds.) Agent-Oriented Methodologies, pp. 20–45. Idea Group Pub., Hershey (2005) 6. Chen, Y., Romanovsky, A.: Improving the dependability of web services integration. IT Prof. 10(3), 29–35 (2008) 7. De Wolf, T.: Analysing and engineering self-organising emergent applications. Ph.D. thesis, Department of Computer Science, K.U. Leuven, Leuven, Belgium (2007). http://www.cs. kuleuven.be/tomdw/phd/PhD-TomDeWolf-29-May-2007.pdf

12

Engineering Self-organising Systems

311

8. Di Marzo Serugendo, G.: Robustness and dependability of self-organising systems—a safety engineering perspective. In: Guerraoui, R., Petit, F. (eds.) The 11th International Symposium on Stabilization, Safety and Security of Distributed Systems (SSS 2009), Lyon, France. LNCS, vol. 5873, pp. 254–268. Springer, Berlin (2009) 9. Di Marzo Serugendo, G., Gleizes, M.P., Karageorgos, A.: Self-organisation in multi-agent system. Research Report IRIT/2005-18-R, IRIT, Paul Sabatier University, Toulouse (2005) 10. Di Marzo Serugendo, G., Fitzgerald, J., Romanovsky, A.: Metaself—an architecture and development method for dependable self-* systems. In: ACM Symposium on Applied Computing (SAC’2010), Sierre, Switzerland, pp. 457–461. ACM, New York (2010) 11. Frei, R., Di Marzo Serugendo, G., Barata, J.: Designing self-organization for evolvable assembly systems. In: Second IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO’08, pp. 97–106 (2008) 12. Gardelli, L., Viroli, M., Casadei, M., Omicini, A.: Designing self-organising environments with agents and artefacts: a simulation-driven approach. Int. J. Agent-Oriented Softw. Eng. 2(2), 171–195 (2008) 13. Gershenson, C.: Design and control of self-organizing systems. Ph.D. thesis, Vrije Universiteit Brussel (2007) 14. Gleizes, M.P., Millan, T., Picard, G.: ADELFE, using SPEM notation to unify agent engineering processes and methodology. Research Report IRIT/2003-10-R, IRIT, Paul Sabatier University, Toulouse (2003) 15. Henderson-Sellers, B., Giorgini, P.: Agent-Oriented Methodologies. Idea Group Pub, Hershey (2005) 16. Jacobson, I., Booch, G., Rumbauch, J.: The Unified Software Development Process. Addison Wesley, Reading (1999) 17. Kruchten, P.: The Rational Unified Process: An Introduction. Addison Wesley, Reading (2000) 18. Morandini, M., Penserini, L., Perini, A.: Modelling self-adaptivity: a goal-oriented approach. In: Second IEEE International Conference on Self-Adaptive and Self-Organizing Systems, 2008, SASO’08, pp. 469–470 (2008) 19. Morandini, M., Migeon, F., Gleizes, M.P., Maurel, C., Penserini, L., Perini, A.: A goaloriented approach for modelling self-organising MAS. In: The 10th Annual International Workshop on Engineering Societies in the Agents World (ESAW). Springer, Berlin (2009). (online) http://www.springerlink.com 20. OMG: Software Process Engineering Metamodel Specification (2002) 21. Ricci, A., Viroli, M., Omicini, A.: Programming MAS with artefacts. In: Bordini, R.P., Dastani, M., Dix, J., El Fallah Seghrouchni, A. (eds.) Programming Multi-Agent Systems. LNAI, vol. 3862, pp. 206–221. Springer, Berlin (2005) 22. Rougemaille, S., Migeon, F., Maurel, C., Gleizes, M.: Model driven engineering for designing adaptive multi-agent systems. In: The 8th Annual International Workshop on Engineering Societies in the Agents World (ESAW). Springer, Berlin (2007). (online) http://www. springerlink.com 23. Rougemaille, S., Arcangeli, J., Gleizes, M., Migeon, F.: ADELFE design, AMAS-ML in action: a case study. In: The 9th Annual International Workshop on Engineering Societies in the Agents World (ESAW). Springer, Berlin (2008). (online) http://www.springerlink.com 24. Rumbaugh, J., Jacobson, I., Booch, G.: The Unified Modeling Language Reference Manual. Addison Wesley, Reading (1999) 25. Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach. Prentice Hall Series. Prentice Hall, New York (1995) 26. Shehory, O., Sturm, A.: Evaluation of modeling techniques for agent-based systems. In: Press, A. (ed.) Proceedings of the Fifth International Conference on Autonomous Agents, pp. 624– 631 (2001) 27. Topin, X., Fourcassié, V., Gleizes, M., Théraulaz, G., Régis, C.: Theories and experiments on emergent behaviour: from natural to artificial systems and back. In: European Conference on Cognitive Science, Siena, Italy (1999)

312

C. Bernon et al.

28. Zambonelli, F., Parunak, H.V.D.: Signs of a revolution in computer science and software engineering. In: Petta, P., Tolksdorf, R., Zambonelli, F. (eds.) Engineering Societies in the Agents World 3rd International Workshop, ESAW 2002, Madrid, Spain, September 2002. Lecture Notes in Computer Science, vol. 2577, pp. 120–125. Springer, Berlin (2003) 29. Zambonelli, F., Jennings, N.R., Wooldridge, M.: Multiagent systems as computational organisations: the Gaia methodology. In: Henderson-Sellers, B., Giorgini, P. (eds.) Agent-Oriented Methodologies, pp. 136–171. Idea Group Pub., Hershey (2005)

Chapter 13

Middleware Infrastructures for Self-organising Pervasive Computing Systems Matteo Casadei, Marco Mamei, Cynthia Villalba, Mirko Viroli, and Franco Zambonelli

Exploiting field-inspired and chemical-inspired middleware to develop service ecosystems supporting self-organisation, self-adaptation and evolvability in pervasive settings.

Objectives The main goal of this chapter is to: • Present the requirements for middleware systems in upcoming distributed and pervasive computing scenarios; • Introduce a nature-inspired architecture addressing the above requirements and illustrating the fundamental role of middleware infrastructures in this vision; • Survey the possibly approach towards nature-inspired middleware infrastructures;

M. Casadei · M. Viroli Università di Bologna, 47023 Cesena, Italy M. Casadei e-mail: [email protected] M. Viroli e-mail: [email protected] M. Mamei · C. Villalba · F. Zambonelli () Università di Modena e Reggio Emilia, 42100 Reggio Emilia, Italy e-mail: [email protected] M. Mamei e-mail: [email protected] C. Villalba e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_13, © Springer-Verlag Berlin Heidelberg 2011

313

314

M. Casadei et al.

• Detail two concrete examples of nature-inspired middleware (TOTA and Chemical-TuCSoN) fulfilling some of the above aspects. In particular, the programming models of the two proposed middleware infrastructures will be presented.

13.1 Introduction The distributed computing scenario is radically changing. While Telecom and Internet networks are rapidly converging and computing is becoming intrinsically mobile, pervasive sensing and actuating devices are increasingly populating our everyday environments and will be tightly integrated with such networks. This is paving the way for the deployment of innovative pervasive services, i.e. context-aware services for interacting with the physical and social worlds around us [11, 19]. In this context, an innovative open software platform—i.e. a middleware infrastructure—will have to be provided to host and orchestrate the execution of general-purpose pervasive services and the organisation of large masses of contextual data. The platform should take into account the increasing needs of users [31] and must be able to flexibly tolerate evolutions over time. In addition, such an infrastructure should deal with the inherent nature and situatedness of components and services, taking into account the spatial and social contexts in which they are deployed. To reach the goal, we should no longer conceive services and their interactions as common in standard SOA architectures and middleware [18]. There, services are simply considered as “loci” of functionalities, whose activities are orchestrated according to specific pre-defined patterns with the support of middleware services such as discovery, routing and context services. As a consequence, situatedness, spatial concepts, and self-* features are not intrinsic properties of the system, but typically enforced via ad hoc one-of solutions, e.g. via the introduction of specific control tools as it is the concept of autonomic manager in the IBM approach to autonomic computing [22]. The long-term evolvability is simply not ensured and most likely can be achieved only at very high re-engineering costs. The most promising direction towards the actual realisation of adaptive pervasive service environments is that of taking inspiration from natural systems, where spatial concepts, self-* features, and long-lasting evolvability are inherently there because of the basic “rules of the game”. We are aware that nature-inspired solutions have already been extensively exploited in the area of distributed computing (see [2, 25] and Chaps. 4 and 6 of this book). However, most of these proposals exploit the natural inspiration only for the effective implementation of specific algorithmic solutions or for the realisation of specific distributed services. Similarly, a number of initiatives recognise that the complexity and dynamics of modern ICT systems is comparable to that of natural systems, thus requiring very flexible and

13

Middleware Infrastructures for Self-organising Pervasive Computing

315

innovative solutions (see [17, 32]). However, the idea that natural metaphors can become the foundation on which to re-think the whole architecture of those systems is far from being metabolised. It should be noted that natural systems can be thought of and modelled at different abstraction levels, e.g. at the physical, chemical, biological or social levels. Whatever the case, one can always recognise the following characteristics: above a spatial environmental substrate, individuals (or particles, or components) of different kinds interact, compete and combine with one another in respect of the basic laws of nature. Starting from such a simple consideration, in this chapter we present a general innovative architecture inspired by natural systems. In particular, we focus on innovative middleware systems that are aimed at supporting a new architectural model inherently promoting situatedness, self-organisation, self-adaptation and evolvability at the application/service level. These middleware infrastructures are conceived as the space in which bringing to life a sort of “natural ecosystem of services”, the latter intended as sorts of spatially situated individuals whose computational activities, while serving to own specific purposes, are subject to some (a limited set of) basic natural laws. There, the idea is that the dynamics of the system (as determined by the enactment of its laws, enforced by the middleware) will provide for spontaneously enforcing features of self-organisation and self-adaptation as endogenous properties enacted by the infrastructure rather than peculiar characteristics of its individuals [20]. More in particular, the remainder of this chapter is structured as follows: Sect. 13.2 presents a general architecture for nature-inspired service-ecosystem approaches. Section 13.3 introduces a taxonomy of middleware models and infrastructures that can be defined for such an architecture. Section 13.4 introduces a case study that will be used to ground the discussion throughout the chapter. Section 13.5 analyses the surveyed middleware abstractions with regard to the presented application case study. Section 13.6 presents a first example of middleware supporting self-organising applications: the middleware is TOTA, which has been designed to support physically inspired computing models. Section 13.7 presents another example of middleware (namely TuCSoN), supporting a chemically inspired computing model. Section 13.8 discusses some general related and open issues, and Sect. 13.9 concludes providing for final remarks.

13.2 A Reference Architecture Let us start our analysis by introducing a general reference architecture aimed at providing a common and uniform conceptual model for future nature-inspired pervasive computing systems and for properly framing the concepts revolving around innovative middleware infrastructures for such scenarios. Figure 13.1 provides a pictorial representation of such a conceptual architecture. The lowest level is the concrete physical and digital ground on which the middleware will be deployed, i.e. a network of computing devices and information sources. Depending on the application at hand, this may be a complex, dense infrastructure

316

M. Casadei et al.

Fig. 13.1 A conceptual architecture for pervasive service ecosystems

(ideally, a pervasive continuum), including e.g. all the hardware devices that are going to increasingly pervade all our everyday environments, such as PDAs, smart phones, tags, sensors and actuators. At the top level, service developers, service producers and consumers of services and data access the open framework for using/consuming data or services, as well as for producing and deploying new services and new data components in the framework or making new devices available. At both the lower and top levels of the architecture openness and its dynamics stand: on the one hand, new devices can join/leave the system at any time, like perturbations of the network structure; on the other hand, new users can interact with the framework and deploy new services and data items on it, and possibly new devices too. The abstract computational components of the pervasive ecosystem architecture reside in between these two levels. The “Species” (i.e. components) level includes a variety of individuals. There are individuals such as: physical and virtual devices of the underlying network, digital and network resources of any kind, persistent and temporary knowledge/data, contextual information, events and information requests, software service components, as well as software components/agents to represent users, consumers and producers. The species are all provided with a uniform abstract view and represent the entities (or the individuals) populating the natural system. Although such individuals are expected to be modelled and computationally rendered in a uniform way, they will have specific characteristics very different from one another, i.e. they will be of different “species”. In general terms, a system is expected to be populated with a set of individuals physically deployed in the environment (e.g. physical and network resources, static information, initialisation data and services, and so on) and situated in some portion of the network. Yet, the overall population is far from being static. In the short term,

13

Middleware Infrastructures for Self-organising Pervasive Computing

317

the set of individuals is subject to changes due to the mentioned factors of dynamics and openness that typically affect the overall infrastructure. In the longer term, technological evolution and social evolution may notably affect the characteristics of the species that populate the ecosystem. The various species of the ecosystem have to be brought to life into some “World” where some “Laws” exist to rule their activities and interactions. The “World” and “Laws” levels are indeed those that need be realised by some innovative middleware infrastructure. The “World” level provides the spatial fabric supporting individuals, their spatial activities and interactions, as well as their life-cycle. From a conceptual viewpoint, the “World” level gives shape to and defines the structure of the virtual world. This level should consider that individuals exist in a specific portion of some metric space and that their activities and interactions are directly dependent on their position in space and on the shape of the surrounding space. What the actual structure and shape could be might depend on the specific abstractions adopted for modelling the system. From a more practical viewpoint, a middleware infrastructure will have to realise the World level by supporting the execution and life cycle of individuals, and by enforcing (rather than hiding) concepts of distance, locality, local interactions and mobility, coherently to a specific structure of the space. The way whereby individuals live and interact (which may include how they produce and diffuse information, how they move in the environment, how they selfcompose and/or self-aggregate with one another, how they can spawn new individuals and how they decay or die) is determined by the set of fundamental “Laws” regulating the self-organising and self-adaptive system model, again expected to be enforced by the middleware infrastructure. Laws are expected to act on the basis of spatial locality principles, as in real laws of nature: the enactment of the laws on individuals will typically affect and be affected by the local space around them and by other individuals. Although such laws are expected to be always the same, they can have different effects on different species. In middlewares strongly supporting openness, the enactment of the laws may require the presence of some meaningful description of the information/service/structure/goals of each species and of their current context and state. These descriptions, together with proper “matching” criteria, define how laws apply to specific species in specific conditions of the space. With this regard, we emphasise that, in the proposed architecture, the concept of “semantic description”—adopted in SOA architectures to facilitate service discovery—turns into a concept of “alive semantic description” to rule the enactment of the laws. The proposed architecture—in its simplicity—represents a total deconstruction of the traditional perspective on middleware. There are no hardwired and heavyweight middleware services and concepts such as that of “vertical layering”, which indeed limits flexibility and adaptability. The middleware is deconstructed and made minimal. Between the network and the users, there is a single universe of components, all of which underlying the same model and obeying the same eco-laws. Such laws and the spatial structure in which they apply are the only concepts hardwired into the system. Such a radical deconstruction is very important to ensure depend-

318

M. Casadei et al.

ability, diversity and evolution: nothing is there to stay, everything can change and evolve, without undermining at the foundations the overall stability of the ecology. The dynamics of a system running over such an architecture will be overall determined by having individuals acting based on their own internal goals, yet being subject to the laws as far as their actions and interactions are concerned. The fact that the laws may be affected by the presence and state of other individuals provides for closing the feedback loop which is a necessary pre-condition to enable self-organisation features. Indeed, the typical evolution patterns include forms of self-organisation like service aggregation or service orchestration, where the eco-laws can play an active role in facilitating individuals to interact with one another and orchestrate their actions, but also in self-adaptation and decentralised self-management.

13.3 A Taxonomy of Middleware Models and Infrastructures The key differences in the possible middleware that can be devised according to this conceptual reference architecture stand in the metaphor adopted to model the overall ecosystem and in particular its individuals and laws. In particular—without excluding the existence of other useful natural metaphors or the possibility of conceiving interesting non-natural metaphors—the main metaphors that can be adopted are: physical metaphors [12, 24], chemical metaphors [3, 34, 37] and biological metaphors [2, 4], together with metaphors focusing on higher-level ecological/social metaphors (e.g. trophic networks [1, 33]). Let us now come to the distinguishing characteristics of each metaphor and analyse the role of middleware infrastructures in each of them (see Fig. 13.2 for a concise summary). Physical metaphors consider that the individuals of the ecosystem are sorts of computational particles, living in a world of other particles and virtual computational fields, the latter acting as the basic interaction means. Middleware infrastructures adhering to a physical metaphor will typically model all the activities of particles as driven by laws that determine how particles should spread fields, how such fields propagate and get re-shaped upon changing conditions, and how particles should be influenced by the local gradients and shape of the computational field (those whose description “matches” some criterion). Fields spreading and propagation will be supported by the middleware itself. In this context, particles can change their status based on the perceived fields and move or exchange data by navigating over such fields (i.e. by having particles that move following the gradient descent of a field or by making them spread sorts of data particles to be routed according to the shape of fields). The world in which such particles live and fields spread and diffuse can be either a simple (euclidean) metric world mapped in the physical space or a network space mapped on the technological network, or in addition, it could also be a sort of relativistic world, in which shapes and distances in the environment are not “inherent” but rather shaped by fields themselves (as in gravitational space–time). Chemical metaphors consider that the individuals of the system are sorts of computational atoms/molecules. Middleware infrastructures adopting this view model

13

Middleware Infrastructures for Self-organising Pervasive Computing

319

Fig. 13.2 Metaphors for middleware supporting self-organisation

the properties of individuals by some sort of semantic descriptions, as the computational counterpart of the description of the bonding properties of physical atoms and molecules, yet made dynamic by the fact that such descriptions reflect the current state and context of individuals. In this scenario, the laws driving the overall behaviour of the system (as enforced by the middleware itself) can be regarded as chemical laws which dictate how chemical reactions and bonding between components take place (e.g. relying on some forms of pattern matching between the semantic description of components). Furthermore, the laws can lead to the production of aggregates (e.g. aggregated distributed components), new components (e.g. composite components or components spawned by existing ones) and to the growth/decay of a set of components. The world in which components live is typically formed by a set of localities, intended as the “solutions” where chemical reactions can occur. Biological metaphors typically focus on biological systems at the small scale, i.e. at the scale of individual organisms (e.g. cells and their interactions) or colonies of simple organisms (e.g. ant colonies). The individuals are therefore either simple cells or very simple animals, both of which can be seen as acting on the basis of very simple goal-oriented rules (e.g. move, find food, find mate). Similarly to physical systems, these middleware infrastructures consider that interactions take place by means of signals of various flavours (i.e. chemical pheromones) spread by components in the environment and to which components can react. However, unlike physical systems, components are not necessarily passively subject to the pheromones around: the reaction of components to the sensed pheromones, as well

320

M. Casadei et al.

as their own spreading of pheromones, may depend on their current “mood” (which reflects their current state towards the achievement of a goal). In the context of the biological metaphor, the laws of the system are only aimed at determining how such pheromones, depending on their specific flavours, should diffuse and/or evaporate. With this regard, we emphasise that pheromones, unlike fields, are typically more persistent than fields (i.e. they slowly evaporate and diffuse), which makes it easier for the middleware infrastructure to support them, making it also possible to exploit them as a sort of environmental memory, other than to reflect the current state of things in the environment. The world in which components live is typically a virtual computational landscape, mapped on either the network topology or the physical space, that can influence the way pheromones diffuse and the way components can move over it. Social metaphors focus on biological systems at the level of animal species and their interactions, e.g. at the level of trophic interactions. There, components of the system are sorts of goal-oriented animals (i.e. agents) belonging to specific species (i.e. agent classes) searching for “food” resources to survive and prosper (e.g. specific resources or other components). Pure data items and resources can be abstracted as sorts of passive life-forms (i.e. vegetables). In this context, the laws of the system determine how the resulting “web of food” should be realised, that is they determine how and under which conditions animals are allowed to search for food, eat, and possibly produce and reproduce, thus influencing and ruling the overall dynamics of the system and the interaction among individuals. Similarly to chemical systems, this kind of middleware considers that the shape of the world is typically organised around a set of localities, i.e. ecological niches (think of a set of local pervasive computing environments), yet enabling interactions and diffusion of individuals across niches.

13.4 The Case Study To clarify the concepts expressed, a simple case study (that will be used to evaluate the various metaphors) may be of help. It is a matter of fact that we are increasingly surrounded by digital displays, wherever and at any time: from the small ones of our wearable devices and our domestic hardware to the wide wall-mounted ones increasingly pervading our urban and working environments. Currently, in most of the cases, such displays are conceived to show information in a hardwired or manually configured manner, being completely insensitive to the context in which they operate and to the specific needs of users. For instance, advertising displays typically cycle a number of pre-defined (and pre-paid) commercials independently of who is around. For such a costly and intrusive hardware infrastructure to be highly effective and favourable for both users and information providers, a totally different approach should be undertaken. Information should be displayed based on the current state of the physical environment (e.g. an advertiser would not be happy to pay for a hot chocolate commercial to be displayed in a warm day, while advertising about ice

13

Middleware Infrastructures for Self-organising Pervasive Computing

321

tea would lead to much more impact), possibly by exploiting information coming from surrounding sensors and by coordinating actions among displays in physical proximity (i.e. to avoid users irritated by seeing the same advertisement over and over as they pass by). Displays could adapt the shown information to the classes of users around (e.g. showing a soda commercial rather than a hot chocolate one in the presence of a teens’ majority) and to what they have already seen in the recent past. The overall display infrastructure should be able to automatically re-distribute and re-shape the information displayed upon changing conditions (e.g. when new displays are deployed or some get out-of-order). It should be possible for users to upload information to enrich the overall informative offer or simply adapt the offer to their own needs. Finally, the overall infrastructure should tolerate in a seamless way changes related to the technology of displays themselves and of the sensors around (e.g. the introduction of new classes of sensors to improve the information offer), as well as changes in the technological and business models according to which information and advertisements are displayed. In order to become a ground for useful self-organising information services, the envisioned display infrastructure must consider interactions with the various pervasive and personal devices around, must accordingly exhibit situatedness and adaptivity in its behaviour and must also be prosumption- and diversity-oriented. When modelling and deploying such a case study relying on a middleware based on the Physical metaphors, one would model displays (better, the corresponding display services) as masses that tend to emit sorts of gravitational fields (in the form of broadcast events or spanning trees). Such fields may also have different “flavours” (determined by, e.g. the species of users around and the specific environment conditions) and an intensity proportional to either their dimension or the available display slots. Advertiser agents can then simply behave as masses attracted by gravitational fields with specific flavour (on the basis of some matching criteria), to eventually get in touch with suitable displays for their commercials. Upon changing conditions, the structure and characteristics of diffused fields will change, providing for dynamically re-assigning commercials to displays. Similarly, we can easily conceive the definition of whatever user-level service by applying the simple basic concepts of the physical metaphor (e.g. by having data items and processes move in the network by following the gradients of specific computational fields). When modelling and deploying such a case study relying on a middleware based on the Chemical metaphors, one would model display services, information services (concerning ads and news) and user and environmental data as molecules. Displays represent different localities in which components react. Chemical rules dictate that when the preferences of a user enter the locality of a display and match an advertisement service, then a new composite component is created which is in charge of actually displaying that service in that display. Concurrently, in each locality, catalytic components can be in charge of re-enforcing the concentration of specific information or of specific advertisements, reflecting the current situation of users. Also, localities can be opened to enforce chemical bonds across displays, so that high-activity of advertisement reactions on a display can eventually propagate to neighbour displays.

322

M. Casadei et al.

When modelling and deploying such a case study relying on a middleware based on the Biological metaphors, one would model users as simple agents roaming around and spreading chemical signals with a flavour reflecting their personal interests. Displays can locally perceive such pheromones and react by emitting some different pheromones to express their availability for showing some commercials of some specific nature, a choice which can also rely on the past history. Advertising agents can sense the concentration of such pheromones and be attracted towards the displays where the concentration of users with specific interests is maximised. Both displays and advertising agents by their side, depending on which advertisements they have displayed so far, can also emit additional flavours of pheromones, to keep memory of past events in the environment. The temporary persistence of pheromones can also be effectively exploited by additional classes of components that, by moving around from display to display, can create proper pheromone trails as a basis for more global strategies (e.g. to create sorts of routing paths for enabling advertiser agents to globally find the best displays to exploit). When modelling and deploying such a case study relying on a middleware based on the Social metaphors, one would model users—after assuming that each display is associated with an ecological niche—as species of herbivores agents that roam around from niche to niche, so as to possibly eat those vegetable-like life-forms which are represented by information. Advertisers can be sorts of carnivores (i.e. eating other active life-forms) needing to find proper users (e.g. users with matching interests) to survive. For both cases, the primary effect of an eating act is the feeding of displays with something to be displayed (information or commercials). Possible secondary effect of eating acts can be the reproduction and diffusion in the environment of the best-fit species (e.g. most successful commercial). Those species that do not succeed in eating at a niche can either die or move to other niches to find proper food. Concurrently, one can think of the presence of background agents acting as sorts of bacteria that digest the activity logs in the various niches to enforce specific forms of distributed control over the whole system (e.g. by affecting the way information is propagated across niches).

13.5 Critical Analysis The analysis of which metaphor (and thus of which type of middleware model) best suits the characteristics of modern pervasive scenarios at hand cannot abstract from the identification of the key features that we want to achieve from it and that— consequently—we want our self-organising system to express. All the proposed metaphors are inherently “situated”, i.e. based on and inspired by concepts and entities that have a precise and limited situation in “space”. The suitability of the different metaphors should be analysed in terms of how a middleware infrastructure, developed according to one of such metaphors, can make it possible to enforce the following desirable properties of the system:

13

Middleware Infrastructures for Self-organising Pervasive Computing

323

• Space: the capability of the system to autonomously express various forms of spatial self-organisation among the distributed (i.e. spatial) activities of the components, so as to autonomously adapt the overall structure and behaviour of the system to specific contingencies. • Time: the capability of the system to tolerate changes and adaptation over time, which includes the capability of adaptively accommodating new individuals and of accommodating very diverse and composite behaviour with the same limited set of laws. • Control: the need of allowing humans (e.g. system administrator and users) to exert control over the behaviour of the system (or some of its parts), i.e. the need of directing its activities and behaviour over space and time, at both design-time and run-time. All of these features that must be supported by the middleware should be enforced without paying the price of dramatically increasing the complexity of the middleware and the system design. Clearly, the analysis of the extent to which the presented metaphors are able to accommodate (and how easily and naturally) the above features is very complex and would require much more room and experience with real-world systems than it is currently available.

13.5.1 Metaphors Peculiarities Figure 13.3 summarises the peculiar characteristics of each metaphor, which are described here in turn. Middleware infrastructures based on Physical metaphors have been extensively studied due to their spatial self-organisation features and, in particular, due to their capability of facilitating the achievement of coherent behaviours even in large-scale systems (e.g. for load balancing, data distribution, clustering and aggregation, differentiation of behaviours). Also, the conceptual tools available for controlling the spatial behaviour and the dynamics of such systems are well developed, most of them related to properly acting on the way fields propagate and dynamically change. In fact, by acting on fields, it is possible to exert control over the overall behaviour of the ecosystem or some of its part. Unfortunately, the physical metaphors fall short in properly tolerating diversity, evolution and time adaptation. In fact, if one wants to preserve the simplicity of laws and of the supporting middleware infrastructure (which is in charge of enforcing the eco-laws also by supporting field propagation and updates), it is necessary to limit the amount of different fields in the system. However, to support very diverse components and behaviours (at a time and over time), laws must become complex enough to tolerate a wide range of different fields and propagation rules, with an overall increase in the conceptual complexity of the model and in the burden of the infrastructure.

324

M. Casadei et al.

Fig. 13.3 Characteristics of each metaphor

Middleware infrastructures based on Chemical metaphors can effectively lead to local self-organising structures (e.g. local composite services and local aggregates), although they may have problems in dealing with more distributed self-organisation (as explained below). Also, as in real chemistry, chemical computational metaphors can accommodate an incredible amount of different components and composites, yet with the same set of simple basic laws. In practice, this means that it can be possible to tolerate an increasingly diverse and evolving set of semantic description for components without affecting the basic eco-laws and increasing the burden of the infrastructure. As far as control is concerned, one can think of using sorts of catalyst or reagent components to engineer and control (in a fully decentralised way) the dynamics and the behaviour of a chemical ecosystem. The key drawback of the chemical approach is that it typically relies on activities taking place within a locality, making it hard to enforce distributed self-organised behaviours. To this end, one should rely on complementing this metaphor with physically or biologically inspired mechanisms to rule the diffusion of chemical substance across localities and the creation of bonds between components in different localities. Middleware infrastructures based on Biological metaphors appear very flexible at enabling the spatial formation of both localised and (by exploiting diffusion of pheromones and movements of individuals) distributed morphological and activity patterns. This has been shown to have notable applications in a variety of distributed systems scenarios [2]. As in physical metaphors, the number of patterns that can be enforced by the spread of chemical pheromones and by the reactions of simple

13

Middleware Infrastructures for Self-organising Pervasive Computing

325

individuals seems quite limited. Also in this case, the problems can be somehow circumvented by increasing the variety of pheromone flavours and characteristics. This solution, as regards biological metaphors, can be more easily supported by the infrastructure since pheromones (unlike fields) rely on local diffusion and slow evaporation dynamics. A problem of biological metaphors lies in the inherent difficulty of understanding how to properly engineer and control the overall behaviour of such systems. In fact, as the mechanisms of morphogenesis and biological self-organisation in actual biological systems have not been fully understood by scientists so far, one cannot hope to be able to properly enforce any desired form of control in complex biologically inspired systems. Middleware infrastructures based on Social metaphors, such as that previously described and based on trophic interaction, promise to be very suitable for local forms of self-organisation, e.g. think of self-organised equilibria of web food patterns in ecological niches. However, unlike chemical metaphors and more similarly to biological metaphors, social metaphors can be exploited also to enforce distributed forms of self-organisation, by exploiting the capability of individuals of moving from locality to locality to achieve their goals. In addition, social metaphors are particularly suited for modelling and tolerating evolution over time (think about how biodiversity has increased over the course of evolution, without ever mining the existence of life in each and every place on earth). However, understanding how to control the overall behaviours, dynamics and local/global equilibria of real ecological systems is a very difficult task. Accordingly, this would probably be very difficult also with respect to their computational counterparts.

13.5.2 Discussion In summary, it is very difficult to assess once and for all which of the metaphors is the best for building general middleware supporting self-organisation, adaptation over time and ways to enforce decentralised control. Some exhibit suitable features for certain aspects but fall short for others. The general lessons that we can learn from the performed analysis is that: • For domain-specific scenario, a metaphor should be adopted depending on the specific characteristics of the domain and on the relative importance, in that domain, of the general requirements. For instance, in the case study, one can evaluate that the capability of supporting adaptation and evolution locally at each display is more important than the capability of supporting distributed coordination of activities among display, thus making a chemical approach preferable over, e.g. a physical or biological one. • For general scenarios, none of the presented metaphors is directly applicable as it is, each of them falling short in meeting some of the requirements. Some new synthesis, based on incorporating some features of a metaphor into a different one

326

M. Casadei et al.

to produce a sort of hybrid metaphor, should be possibly studied to overcome this limitation. By our side, we have extensively worked in the past with middleware systems based on physically inspired metaphors [24] and with middleware systems based on chemically inspired metaphors [34, 37], and we have already developed usable prototypes. The study of middleware systems inspired by social metaphors have been undertaken too, but it is at a very preliminary stage [33]. In the following sections, we dive into the details of the two existing physically inspired and chemically inspired middleware systems, also with the goal of showing (with the use of the introduced case study) what actually means to program natureinspired pervasive applications.

13.6 A Physically Inspired Field-Based Middleware: TOTA The TOTA (Tuples On The Air) middleware [23] supports the creation and maintenance of distributed data structures resembling the physical concept of “fields”. In this section, we first describe how fields can support the coordination of agents in a distributed computing environment. Then, we expound an exemplary case study to ground the discussion. Finally, we present the TOTA middleware highlighting how to use it to program self-organising distributed system.

13.6.1 The Physical Metaphor in TOTA The activities of coordinating agents in dynamic network environments by traditional models and infrastructures are not easy. It is of great importance to rely on models and middleware infrastructures explicitly designed to take care of all the issues implied in network dynamics. In particular, the key problem that drove the definition of the TOTA middleware was the support of agents’ coordination by providing them with an effective representation of their operational environment (hereafter referred to as context). Such representation, in fact, defines what the agents see about their environment and thus implicitly conditions their behaviour. In particular, such context representation must: (i) be simple and cheap to be obtained; (ii) be expressive enough to facilitate the definition and the execution of agents’ activities; (iii) continuously reflect the actual state of the dynamically changing context. With this regard, physically inspired approaches based on the concept of fieldbased coordination [21, 23] appear very suitable. In these approaches, and along the lines previously discussed in this chapter, the agents context is abstracted by means of abstract “fields” expressing forces driving agents’ activities, resembling the way the gravitational field drives mass particles in our universe. In these approaches, a middleware infrastructure is required to create and support field representation. Moreover, to offer an up to date representation of the agent context, fields must adapt

13

Middleware Infrastructures for Self-organising Pervasive Computing

327

and self-maintain their value to reflect possibly changing environmental conditions. Agents can locally perceive these fields and decide what to do on the basis of the sensed fields and the corresponding magnitude. This field-based representation of the context is (i) simple and cheap to be obtained, since fields are spread in the environment and agents need only to perceive them locally; (ii) expressive enough to facilitate the coordination activities, since agents need only to sense fields and be driven by them; (iii) robust and adaptive to the above-mentioned environmental dynamism due to fields self-maintenance.

13.6.2 Modelling the Case Study in TOTA A central issue in the development of any kind of distributed application is to provide components (i.e. agents) with suitable and effective mechanisms to access distributed information. In its broader meaning, information access is at the basis of data sharing, context-awareness (i.e. access to context-related information) and also interaction (i.e. access to communication partners). Location-based mechanisms to access distributed information enable an agent to access resources within suitable locality constraints (e.g. find all the printers on this floor, or find the closest gas station). In general, this kind of mechanism can be very useful in a wide range of application scenarios. For example, in a B2C (Business to Consumer) scenario, a customer may want to limit the scope of a query for a specific item to the city where (s)he lives, to save shipping cost and time. Considering mobile computing scenarios, location-based access to information and services becomes even more important [8]. Most mobile computing applications become really useful when it is possible to limit the scope of queries to those resources and services actually reachable by the mobile users (e.g. mobile users ask their PDA to retrieve nearest restaurants while roaming through a city). When even the resources and the services are mobile, location-based access enables new and exciting applications. Focusing on the case study previously described, we can consider that a display might want to get information about the presence of users with a given profile in the environment. This information can help an advertisement service by allowing to pre-fetching important advertisements. In general, we think that field-based concepts are very useful in this scenario. In fact, fields spread in the environment can easily define regions that can be used to constrain the scope of the information access. As a simple example, we can envision that a display looking for some information will create and spread across the mobile network a QUERY field carrying on the information about what the display is looking for (e.g. a user with a given profile) and that propagates the field within a specified locality scope. In more details, the QUERY field can be flooded across the network infrastructure but stopping propagating once a specified critical distance (e.g. 300 m) from the source has been reached. Moreover, the QUERY field, by incrementing one of its values by one at every propagation hop, can create a routing structure to collect back the answers. Users can look for incoming QUERY fields

328

M. Casadei et al.

Fig. 13.4 (a) An agent propagating a QUERY field to look for a specific information. (b) A suitable bus propagates an ANSWER field that diffuses following downhill the QUERY field

and possibly answer by propagating an ANSWER field carrying suitable information (e.g. User: Marco, interested in BMW Cars). The ANSWER field would propagate following the QUERY field’s routing structure downhill to reach the enquiring display without flooding the network. Once the ANSWER field has been received, the display can pre-fetch a suitable advertisement for the user (see Fig. 13.4). It is worth noting that fields are well suited to such highly mobile scenarios since they automatically update to reflect changes in the environment situation. Thus, ANSWER fields are able to route back to the enquiring display, even if the user moves after issuing the QUERY field, or if the network dynamically changes.

13.6.3 The TOTA Middleware The idea of fields can potentially be implemented on any distributed middleware providing basic support for data storing (to store field values), communication mechanisms (to propagate fields) and event-notification mechanisms (to update fields and notify agents about changes in fields’ values). However, we think that a set of distributed tuple spaces is a particularly fertile ground upon which building such an idea. A set of logically bounded tuples, one in each of the distributed tuple spaces, naturally matches the idea of a field spread across the infrastructure. The content of a tuple would represent field physical properties (e.g. magnitude). This is the approach adopted by our middleware TOTA (Tuples On The Air) [23]. TOTA is composed of a peer-to-peer network of possibly mobile nodes, each running a local version of the TOTA middleware. Upon the distributed space identified by the dynamic network of TOTA nodes, each component is capable of locally storing tuples and letting them diffuse across the network. Tuples are injected in the system from a particular node and spread hop-by-hop accordingly to a specified propagation rule. Specifically, in TOTA, fields have been realised by means of distributed tuples T = (C, P, M).

13

Middleware Infrastructures for Self-organising Pervasive Computing

329

The content C is an ordered set of typed elements representing the information carried on by the tuple. The propagation rule P determines how the tuple should be distributed and propagated across the network (i.e. it determines the “shape” of the tuple field). Propagation typically consists in a tuple cloning itself, being stored in the local tuple space and moving to neighbouring nodes recursively. However, different kinds of propagation rules can determine the “scope” of the tuple (i.e. the distance at which such tuple should be propagated and possibly the spatial direction of propagation) and how such propagation can be affected by the presence or the absence of other tuples in the system. In addition, the propagation rule can determine how tuple content C should change during propagation. Tuples are not necessarily distributed replicas: by assuming different values in different nodes, tuples can be effectively used to build a tuple field expressing some kind of contextual and spatial information. In the end, unlike traditional event based models, propagation of tuples is not driven by a publish-subscribe schema, but encoded in the propagation rule of tuples. Moreover, tuples can change their content during propagation. The maintenance rule M determines how a tuple field should react to events occurring in the environment or simply to time elapsing. On the one hand, maintenance rules can preserve the proper spatial structure of tuple fields (as specified by the propagation rule) despite network dynamics. To this end, the TOTA middleware can support tuples propagation actively and adaptively: by constantly monitoring the local topology of the network and the income of new tuples, the middleware can automatically re-propagate or modify the content of tuples as soon as appropriate conditions occur. For instance, when new nodes get in touch with a network, TOTA automatically checks the propagation rules of the already stored tuples and eventually propagates the tuples to the new nodes. Similarly, when the topology changes due to node movements, the tuple field can automatically change to reflect the new topology. On the other hand, tuples (i.e. their content and their distributed structure) can be made variable over time, for instance to support temporary tuples or tuples that slowly “evaporate”.

13.6.4 Programming with TOTA TOTA is provided with a simple set of primitive operations to interact with the middleware. Most of the operations take as inputs and provide as outputs objects of the class TotaTuple. This is the base abstract class for all the TOTA tuples. Concretely, this class will implement the data and methods associated to a specific content (C), propagation rule (P) and maintenance rule (M) [23]. TOTA tuples have been designed by means of objects: the object state models tuple content, the propagation rule is encoded by means of a specific propagate method, while maintenance rule can be implemented by properly coding a specific react method (see code in Fig. 13.5). The TOTA middleware is provided with a predefined hierarchy of classes implementing the abstract TotaTuple for a number of purposes (e.g. GradientTuple is

330

M. Casadei et al.

abstract class TotaTuple { protected TotaInterface tota; /* instance variables represent tuple fields */ ... /* this method initialises the tuple, by giving a reference to the current TOTA middleware */ public void init(TotaInterface tota) { this.tota = tota; } /* this method codes the tuple propagation */ public void propagate() {} /* this method enables the tuple to enforce maintenance rules */ public void react(String reaction, String event){} }

Fig. 13.5 The main structure of the TotaTuple class. The TOTA middleware calls methods init and propagate once the tuple enters a node (whether because it has been injected by an agent or it moved there because of propagation). The middleware calls the react method if the tuple subscribe to a condition matching a particular event. In particular, we defined a ReactiveComponent interface exposing the react method. Tuples (and also agents) implement that interface

public void inject (TotaTuple tuple); public Vector read (TotaTuple template); public Vector readOneHop (TotaTuple template); public TotaTuple keyrd (TotaTuple template); public Vector keyrdOneHop (TotaTuple template); public Vector delete (TotaTuple template); public void subscribe (TotaTuple template, ReactiveComponent comp, String rct); public void subscribeOneHop (TotaTuple template, ReactiveComponent comp, String rct); public void unsubscribe (TotaTuple template, ReactiveComponent comp);

Fig. 13.6 The TOTA API. TotaTuple arguments are called ‘templates’ in read-like operations because they are used in pattern matching and can contain formal (i.e. null) elements

a tuple having an integer value that increases with the distance from the source, DownhillTuple is a tuple that propagates by following downhill the gradient laid out by another tuple). The class hierarchy contains many tuples that can be used in several applications. It is also possible to create new tuples from scratch. In this chapter, we do not deal with tuple programming, but only with programming services on top of TOTA. Further details on the tuple class hierarchy and on tuple programming can be found in [23]. The operations provided by the TOTA middleware are listed in Fig. 13.6. Inject The inject primitive is used to inject in the TOTA network the tuple passed as a parameter. First the tuple object is passed to the TOTA middleware. Then, the middleware calls the tuple propagation method that will specify whether the tuple will be stored in the local tuple space and if it will be sent to neighbouring nodes to be propagated. Once the middleware finishes executing the tuple propagation code on the current node, the inject method returns. Read and readOneHop The read primitive accesses the local tuple space and returns a collection of the tuples locally present in the tuple space that match the

13

Middleware Infrastructures for Self-organising Pervasive Computing

331

template tuple passed as a parameter. A template is a TotaTuple in which some of the content elements can be left uninitialised (null). In the implemented OO-patternmatching, a template tuple TMPL and a tuple T match if and only if: • TMPL is an instance of either the class of T or one of its superclasses; • the non-null elements of TMPL that represent primitive types (integer, character, boolean, etc.) have the same value of the corresponding elements in T; • the non-null non-primitive elements (i.e. objects) of TMPL are equal—in their serialised form—to the corresponding ones of T (two Java objects assume the same serialised form only if each of their instance variables have equal value, recursively including enclosed objects). The readOneHop primitive returns a collection of the tuples present in the tuple spaces of the node’s one-hop neighbourhood and matching the template tuple. The TOTA middleware sends the template tuple TMPL to neighbouring nodes. Neighbouring nodes compute pattern matching operations and return the tuples matching TMPL to the requestor node. All the tuples are collected in a vector that is finally returned to the application agent. Both read and readOneHop operations are synchronous and non-blocking. They return all the tuples matching the given template, or an empty vector if no matching tuples are found. Keyrd and keyrdOneHop The keyrd and keyrdOneHop non-blocking methods are analogous to the former two read methods, but instead of performing a pattern matching on the basis of tuple content, they look for tuples with the same middleware-level ID of the tuple passed as argument via a fast hash-based mechanism. These two methods are useful in many cases. For example, to evaluate the local shape of a tuple (i.e. to evaluate the gradient of specific values in the content of the tuple), the agent needs to retrieve the instances of the same tuple in its one-hop neighbourhood: keyrdOneHop is a fast method to achieve such a functionality. Delete The delete primitive extracts from the local middleware all the tuples matching the template and returns them to the invoking agent. In this context, it is worth noting that the effect of deleting a tuple from the local tuple space may be different depending on both the maintenance rule for the tuple and the invoking agent. In particular, if a tuple field has a maintenance rule specifying to preserve its distributed structure in reaction to events then: (i) deleting it from the source node induces a recursive deletion of the whole tuple structure from the network; on the opposite, (ii) deleting it from a different node may have no effect, in that the tuple will be re-propagated there if the maintenance rule specifies so. Subscribe and Unsubscribe The subscribe and unsubscribe primitives are defined to manage reactive programming. An agent (or any object implementing the ReactiveComponent interface—the second parameter of the method) can subscribe to the insertion and removal of specific kinds of tuples in the local tuple space, and have a suitable react method to be called when such an event happens. The subscribeOneHop method allows one to subscribe to the insertion and removal of specific kinds of tuples in tuple spaces at one-hop distance.

332

M. Casadei et al.

public class DisplayAgent implements AgentInterface { private TotaMiddleware tota; private Object descriptiontemplate []; /* agent body */ public void run() { /* inject the query */ descriptiontemplate = new Object [] {"BMW Profile", null, null, null}; Gradient query = new Gradient(descriptiontemplate); tota.inject(query); /* subscribe to the answer: the answer will be conveyed in a DownhillTuple */ DownhillTuple answer = new DownhillTuple(descriptiontemplate); tota.subscribe(answer,this,"PRE-FETCH"); } /* code of the reaction that simply prints out the result */ public void react(String reaction, String event) { if(reaction.equals("PRE-FETCH")) { prefetch_advertisement(answer); }}}

Fig. 13.7 DisplayAgent code. This agent runs on a screen and pre-fetches a BMW advertisement if an interested user is found

Any event occurring in TOTA—including connections and disconnections of peers, and sensor readings—can be represented as a tuple. For example, each TOTA node broadcasts in its one-hop neighbourhood a tuple of the class PresenceTuple to express its presence. An agent can subscribe to the insertion or removal of these tuples to actually react to peer connections and disconnections. In general, an agent can notify other agents about any kind of event by representing it via a suitable tuple. The unsubscribe primitive removes all the matching subscriptions. It is worth noting that, despite the fact that all the TOTA read methods are nonblocking, it is very easy to realise blocking operations by exploiting the event-based interface: an agent willing to perform a blocking read has simply to subscribe to a specific tuple and wait until the corresponding reaction is triggered to resume its execution. The presented operations are exploited by agents using the TOTA middleware. For example, the display agent in the case study can be realised by the following code. The agent injects a query TOTA tuple and subscribes for an incoming answer tuple. If the answer tuple is received, the display pre-fetches a proper advertisement (see Fig. 13.7).

13.7 A Chemical Middleware Based on TuCSoN This section presents a concrete middleware for the development of self-organising applications. The idea underlying the middleware is based on the adoption of chemistry as a reference metaphor, and the TuCSoN infrastructure for tuple centres is adopted as a concrete tool to develop the actual chemical-like middleware. A case study, based on the scenario depicted in Sect. 13.4, is also exploited as a reference example to show a concrete application.

13

Middleware Infrastructures for Self-organising Pervasive Computing

333

13.7.1 The Chemical Metaphor in TuCSoN When conceiving a self-organising middleware where interaction is managed by rules that transform and move data in a distributed setting, it is quite natural to find connections with chemistry. Accordingly, we envision the idea of a “chemical” middleware for self-organising systems, where data and processes can be regarded as chemical reactants/molecules whose interaction with other reactants is ruled by chemical-like laws. To model the topological structure induced by this middleware, we consider the chemistry of life, namely biochemistry, in which different compartments exist, each hosting a different chemical solution and possibly transferring some chemical substance in a neighbouring compartment. Hence, the self-organising system is modelled as a network of such compartments that can be easily identified with a network of computing nodes—chemical transfer would hence model relocation of data/processes in a different node. Though the adoption of compartments in chemistry might be seen as relying on a combination of chemistry and biology, we actually see it as a natural chemicaloriented mechanism: first of all, biochemistry is actually to be considered as a subset of chemistry, and secondly, biochemistry has a level of abstraction similar to that of chemistry, which is rather lower than the one of biological metaphors. Among the many sources, the work presented here draws inspiration by previous approaches adopting chemical abstractions for coordination—e.g. the Chemical Abstract Machine [5] or Gamma [7]. However, they have never come to a full realisation of the chemical metaphor, as they do not handle the exact dynamics of chemical reactions—hence, they do not support the intrinsic self-organisation of chemistry [30]. As pointed out in several previous works, distributed systems might be suitably designed as biochemical systems: mobile data items spread in the network can be seen as floating molecules and be used to model knowledge, data, events, as well as the state of processes: the coordination system manipulates such data items so as to enact chemical-like laws that aggregate and transform them into other data. Moving to the middleware setting, we foster the idea of a self-organising middleware providing the application level with an interaction space hosting a biochemical-like dynamics and evolution: accordingly, the components of a self-organising system built upon the middleware exploit the interaction space as a coordination service, where shared knowledge and reified interactions flow in it and are managed according to chemical-like coordination rules. On the one hand, agents at the application level can get coordinated through this biochemical space by inserting new data and retrieving chemically transformed ones; on the other hand, agents can also truly influence the biochemical system and behaviour by inserting e.g. data items acting as catalysts for certain reactions. Moreover, the number of data items of the same kind—i.e. the concentration of such a chemical substance—can be used as a dynamic characterisation of an “activity” level, i.e. the activity level of an agent in the system: the higher the concentration, the higher the probability for the agent to interact with others. As far as chemical reactions are concerned, a computational model has been given by Gillespie [14], who basically provides a characterisation of discrete

334

M. Casadei et al.

chemistry—number of molecules are considered instead of concentrations—as a continuous-time stochastic transition system à la Markov that can be used as a r blueprint for implementing the middleware [36]. Consider a chemical rule A + B − → C + D, meaning that a molecule A joins a molecule B, forming molecules C and D. Rate r is called reaction constant and expresses the likelihood of the reaction between two pairs of molecules A and B. That reaction can be given a Markovian rate that is proportional to r and to the concentration of each reactant in the left-hand side, and a proper algorithm can be used to select which reaction is to be executed and at which “speed”, following the approach of stochastic simulation algorithms [14]. This approach is standard for stochastic computational models used to simulate biological systems, such as stochastic π -calculus [29]: consequently, it seems as a feasible one also for the biochemical-like model presented here. In [14, 35] some examples of natural and synthetic chemical systems are described, which could be of some interest for the proposed chemical middleware, r → 0) by which chemical substances fade; (ii) including: (i) radioactive decay (X − r Lotka reactions [6] (e.g. X + Y − → 2X) modelling the dynamic of prey–predator r → X ) by which a chemical (X–Y) populations; and (iii) diffusion rules (e.g. X − substance gets transferred to a neighbouring compartment, ultimately supporting concepts related to field diffusion as discussed in Sect. 13.6.2—the notation X stands for a molecule X to be sent out of the compartment.

13.7.2 The Case Study with TuCSoN In order to put the presented chemical metaphor in a concrete setting, we consider the scenario of advertising displays in a public place—such as a train station, an airport or a stadium—introduced in Sect. 13.4. In this case, the overall display infrastructure should be able to coordinate what advertisement has to be shown on a specific display (or set of displays) by relying on chemical rules, as those described in Sect. 13.7.1. Imagine now the display infrastructure conceived as a network of spatially distributed nodes, where each node hosts a display in the real world. The network is populated by many data chunks representing information on available advertisements to be potentially displayed. Also, other data is injected in the network by users provided with mobile devices and wandering the public place (e.g. in an airport, people waiting for a flight at the gate or in the line for the check-in, and so on): they reflect user’s needs, inclinations, profile, and so on, which can be turned into personal preferences for specific categories of consumer goods. Turned to a chemical context, such data chunks can now be seen as molecules diffusing and interacting with one another according to chemical laws. In other words, such laws would play the role of coordination rules for the overall display infrastructure, managing the way whereby displays in the environment are shared among the many advertisements available in the network.

13

Middleware Infrastructures for Self-organising Pervasive Computing

335

In this context, it is undoubtedly essential to provide a management of the infrastructure that aims at maximising the performance and appropriateness of user-ad match over time: put it simply, this can be measured as the number of users actually watching an advertisement and being interested in it according to personal inclinations and needs. The decision about what to show on a specific set of displays has to be made locally by definition (based on a context-aware access to available information) and should autonomously adapt to unforeseen evolution of ads, users and their preferences. As a consequence, decentralised, local and dynamic approaches, such as those promoted by biochemistry, appear to be truly promising in order to achieve the desired requirements in an emergent and self-adaptive way. To make things more concrete, suppose now that data chunks representing advertisements (hereafter referred to as ADV) are diffused in the network by providing each node with proper chemical laws. Like a molecule, each ADV in a node is characterised by a concentration, specifying its activity level, i.e. its probability to be displayed next in the display managed by the node. Other than diffusion, ADVs are also subject to decay, so that less popular advertisements can fade and possibly disappear. Besides ADVs, the network contains also data chunks related to users’ preferences (hereafter called PREF), injected in the nodes by software agents running on the mobile devices of the users. Such information is subject to decay but does not diffuse: indeed, it is important for data related to users to remain as close as possible to the area where they have been generated; furthermore, as we suppose that users are mobile, the data injected in a node should remain on that node for a limited amount of time in order to be consistent with the actual state of the environment. In order to devise a solution for the infrastructure to self-adaptively choose the commercial to be displayed on a specific display, we provide every node with a chemical law resembling that of Lotka modelling. Such a reaction would look like this: c

→ ADV + ADV + SHOW. ADV + PREF − In a node, whenever an ADV and a PREF match—i.e. the advertisement (ADV) meets the needs of a given user (PREF) according to some semantic matching algorithm— the reaction executes so as to increase ADV concentration/activity level by 1. In other words, through the reaction, the presence of PREFs in the node acts as a positivefeedback mechanism for matching ADVs, resulting in an increased concentration of the most popular advertisements, which will then fade less quickly in the display managed by the node. Additionally, the reaction also spawns a SHOW data item, which feeds the display so as to schedule the visualisation of ADV. Along the same line, diffusion regulates the extent to which advertisements may be displayed: the higher advertisement concentration and diffusion rate, the wider the extent of the field generated by diffusion. As such, the choice of a specific value for diffusion rate should be mainly driven by the specific structure of the physical environment, as well as by the spatial coverage one wants to provide advertisements with. In general, the behaviour of the overall infrastructure can be tuned by choosing appropriate values for each reaction rate, so as to meet application-dependent requirements. Examples of behaviour tuned via reaction rate are shown e.g. in [35].

336

M. Casadei et al.

13.7.3 Towards the Notion of “Chemical Tuple Spaces” This biochemical metaphor can be designed in terms of a suitable coordination abstraction, combing chemical behaviour with facilities to share a “space of interaction”, namely, in terms of a so-called (bio-)chemical tuple space [35]. The basic idea underlying the concept of chemical tuple space is to conceive a tuple space as a compartment and attach each tuple an integer value called concentration, precisely measuring the pertinency/activity of the tuple. Concentration of tuples is dynamic (as pertinency/activity typically is) and evolves using a chemicallike behaviour, namely, chemical rules can be installed into the tuple space, affecting concentrations over time in the same way chemical substances evolve into chemical solutions. Primitive out can now be used to inject a tuple with any initial concentration: if the same tuple is already present in the space, the two tuples will join together, and their concentrations summed—chemically speaking, out resembles the injection of a chemical substance into a solution. Primitive in can be used to either entirely remove a tuple (if no concentration is specified) or decrease the concentration of an existing tuple—in amounts to removing (partially or entirely) a chemical substance from a solution. Primitive rd is similar to in, but it just reads tuples instead of removing them—rd amounts to observing a chemical substance into a solution, in order to know its concentration. Note that, if t is a tuple specified by in or rd operations, a tuple existing in the space is sought for that match t: differently from standard tuple spaces—but analogous to related works like e.g. SwarmLinda [9, 26]—matching function is application-dependent and can be continuous—stronger matching implies higher probability of finding a certain tuple. In open systems, most likely, such a match would be semantic [15]. According to the above description, a self-organising middleware for the chemical metaphor can be conceived as a network of biochemical tuple spaces. Interaction between tuple spaces is achieved through a special kind of chemical law which, other than just changing tuple concentration, fires some tuple from a tuple space to one of its neighbours—probabilistically picked. This mechanism fits the concept of compartment and chemical transfer as described in previous subsections.

13.7.4 Chemical Tuple Spaces in TuCSoN TuCSoN has been conceived as a coordination infrastructure providing Java agents with coordination media spread over the network, called tuple centres. A centre is a tuple space abstraction augmented with the possibility of programming coordination rules, which are fired as response to interaction events, and which can transform the set of tuples and execute coordination primitives on other tuple centres in the neighbourhood. ReSpecT (Reaction Specification Tuples) is the logic-based language by which such coordination rules can be programmed [10, 28]. Agents act on a tuple centre according to the original Linda [13] tuple-space model by exchanging logic tuples, i.e. first-order terms: the corresponding matching mechanism is based

13

Middleware Infrastructures for Self-organising Pervasive Computing

337

on unification. A comprehensive description of the TuCSoN infrastructure and the ReSpecT language is of no interest here: the interested reader can refer to [27] for further details. TuCSoN tuple centres can be specialised to act as basic building blocks of a biochemical middleware for self-organising systems, where tuple spaces model compartments, while tuples model reactant molecules: they combine, transform and move to other tuple spaces over time as in true chemical systems. Agents perceive such transformations through usual tuple retrieval. Letting m be a molecule kind, we assume that the tuple centre holds one tuple of kind r(m,N), where N is the activity level (concentration) of m. Agents keep inserting and removing molecules one per time, and a ReSpecT specification (implementing Gillespie’s algorithm) is adopted to define the chemical behaviour of the tuple-centre informally described in Sect. 13.7.3, e.g. to extend the semantics of agent operations with the chemical metaphor—for the sake of brevity, the actual ReSpecT specification in not reported here; the interested reader can however refer to [36]. In addition, we assume that chemical laws are expressed as tuples of the kind law(InputList,Rate,OutputList). For example, the reactions presented in Sect. 13.7.1 would be expressed by the tuples law([x],10,[]). law([x,y],0.01,[x,x]). law([x],1,[move(x)]).

and any other chemical system can be as well represented in the same way. The ReSpecT implementation of Gillespie’s algorithm makes the state of the tuple centre evolve according to the defined chemical laws, either simulating an actual chemical system or defining chemical-like services for self-organising systems built on top of the resulting middleware. Correspondingly, agents at the application level perceive the current state of the system by executing primitives or affect the chemical behaviour by properly inserting/removing reactants. As a very simple system behaviour, consider a TuCSoN implementation of a chemical tuple space, with the decay rule above installed in it, meaning that a single unit of concentration of tuple x is removed with average frequency 10—hence overall concentration will have negative exponential distribution. After inserting such a tuple with concentration 1000 by operation out (chemical(x,1000)), an agent continuously reading the tuple concentration by operation rd(chemical(x,Conc)) would perceive it as decreasing over time (Conc/1000, Conc/999, Conc/998, Conc/997, and so on). Of course, the magnitude of decrease perceived by the agent depends on the frequency of observations with respect to decay rate (10 in the example). Other interesting interaction patterns can be developed that are not reported here for the sake of brevity: the interested reader can however refer to [35] for further details.

13.7.5 Programming with Chemical Tuple Spaces in TuCSoN As a concrete case study to illustrate how chemical tuple spaces implemented in TuCSoN can be actually exploited, we show how to program a chemical middleware to support self-organisation for the pervasive display infrastructure described

338

M. Casadei et al.

in Sect. 13.7.2. The aforementioned scenario can be concretely realised by disseminating the pervasive network with chemical tuple spaces powered by proper, general-purpose chemical rules: each tuple space coordinates a specific display in the infrastructure. Advertisements, users, and displays—seen as the individuals of the self-organising system—interact with one another through such distributed middleware, e.g. advertisements compete for the displays placed in the physical environment, and the competition itself is driven by chemical rules and users’ preferences. Each of these individuals, located in a region of the network, is reified through proper tuples placed in the spatially closest tuple spaces, which are subject to an evolution of concentration (i.e. activity/pertinency level) ultimately resulting in the predominance of certain advertisements according to the chemical laws introduced in Sect. 13.7.2—also composition, aggregation and evolution patterns can be envisioned, though. In the display infrastructure, we cannot assume any kind of knowledge about which advertisements will be injected and how many times they will be shown on the displays—i.e. whether they will successfully meet customers’ needs. In this context, semantic matching becomes crucial to dynamically bind advertisements and customers [15, 16]. Semantic matching can be implemented in TuCSoN chemical tuple spaces by adopting the Prolog language, as this is the reference language for the TuCSoN infrastructure, provided with a Prolog engine developed in Java. The middleware presented here is provided with chemical laws so that: (i) advertisements that do not attract customers’ need fade until eventually disappearing from the system, (ii) successful advertisements get their concentration/activity level increased over and over, and accordingly (iii) advertisements willing to use the same set of displays compete with one another. An advertiser agent declares its role by publishing its advertisement description through an out on a specific node of the infrastructure, namely operation out(publish(ad(Ida,Desc)))—this may be repeated again on the same or other nodes. Dually, any customer agent—e.g. the software agent running on the mobile device of the guest of a public place—simply keeps inserting preference tuples in the tuple spaces of the infrastructure by operation out (pref(Idc,Desc)). Note that the tuple space is charged with the role of matching a user’s preference with an advertisement, so as to correspondingly execute chemical rules. The strategy by which this matching is performed will ultimately determine successful advertisements—some advertisements might end up being never exploited, others may become intensively displayed. As a result, the adoption of an appropriate approach based on semantic matching, to bind users’ preference and advertisement descriptions, is crucial for delivering an implementation significant in real scenarios. The described behaviour can be enacted by installing in each tuple space the rules shown in the following: law([Decay], R_dec, []).

% DECAY

law([ad(Ida,Desc_a),pref(Idc,Desc_p)], R_feed, [ad(Ida,Desc_a),ad(Ida,Desc_a)]).

% USE

law([ad(Ida,Desc_a)], R_diff, move(ad(Ida,Desc_a))]). % DIFFUSE

13

Middleware Infrastructures for Self-organising Pervasive Computing

339

Chemical rule (DECAY) states that any tuple possibly fades with negative exponential dynamics: since Decay is a variable, it matches any tuple—it gives perfect match to tuples that should more quickly decade like e.g. those of users’ preferences. In order to define a mechanism to sustain advertisement concentration, rule (USE) provides a positive-feedback approach along with advertisement-customer’s preference binding: it takes an advertisement and a preference tuple that (semantically) match, and increases advertisement concentration by one—this rule mimics the law shown in Sect. 13.7.2. Semantic matching function should be built to give this rule higher rates if an advertisement and a preference strongly match from an actual semantic viewpoint. Since our display infrastructure is composed of a network of chemical tuple spaces, in addition to the laws already described—responsible for competition— we also need to adopt the diffusion law (DIFFUSE), allowing advertisement tuples to diffuse: this is achieved by moving advertising tuples, which will then diffuse on a step-by-step basis. Note that according to the proposed set of rules, as soon as an advertisement starts fading, it is unlikely to be shown on a specific display (or set of displays), until possibly getting completely unused. In other words, advertisement tuples can be thought of as a reification of advertisement state: more generally, the whole set of tuples in the network represents the current self-organised state of the system. In spite of the very simple chemical laws, the emergence of self-organisation properties can be clearly recognised by observing the evolution of chemical concentrations as shown in [35]. Indeed, it is quite easy to observe properties such as self-adaptation (the most requested advertisements, according to users’ needs, are those more frequently displayed), competition (ads for users with similar preferences compete for survival), spatial sensitiveness (advertisement diffusion starts in a neighbourhood) and openness (the arrival of new advertisements is not known in advance at design-time) that are all enacted as intrinsic properties of the infrastructure.

13.8 Related Issues Of course, innovative proposals such as the ones we have presented in this chapter, to be practically applicable, require facing additional many additional issues. Here, without the ambition of being exhaustive, we shortly discuss some of them.

13.8.1 Integration with Legacy A crucial issue of middleware supporting self-organisation is the extent to which it can support legacy, which can be of different sorts. A first possibility is to deal with existing “actors” of Internet, like information sources that produce data and knowledge that is to be sensibly used by the middleware (like contextual information),

340

M. Casadei et al.

or information seekers that might be interested in observing data and knowledge managed by the middleware (like when the Internet receives information from selforganising services seen as an information source). In both cases, as in many other new technologies, wrapping can be a viable solution to tackle legacy. An external source/seeker is to be encapsulated into a skeleton agent of the system, which properly marshals external interactions and turns them into the insertion/observation of proper individuals living in the World supported by the middleware. Another interesting issue would be the possibility of seeing existing and immutable data and knowledge available in the Internet as information that can be exploited by laws to properly drive the self-organising system’s evolution. Again, this can be achieved by a wrapping technique, namely a synthetic world’s location can be constructed that provides one virtual individual for any piece of information externally available, ready to be managed by existing laws. Of course, in this case it should be guaranteed that laws do not aim at changing such individuals.

13.8.2 Security Another relevant issue concerns security: how is it possible to tackle typical security concerns like authorisation, access control, and prevention of denial-of-service attacks? Given the self-organisation character of our framework, we should be able to: (i) prevent unauthorised users to read any information from the system or affect it, (ii) prevent certain users from affecting in given ways the systems, and finally (iii) prevent malicious users from injecting individuals that would affect the system by breaking useful emerged patterns—namely, disrupting its balance. We believe that these issues can be tackled in a uniform way within the model instead of relying on separate mechanisms, after each individual is tagged with a unique identifier of the (authenticated) user. On the one hand, this can be achieved at the level of laws by writing security laws that make certain (or all) individuals of a given user to fade quickly before sensibly affecting the system—like changing the world’s laws so that some material is unstable and decay—like a pheromone. On the other hand, this can be achieved by security individuals, whose goal is to match unwanted individuals and destroy them—like in organisms’ immune systems.

13.8.3 The Social World Another interesting issue is the relationship between the framework presented in this paper and the many social systems that are rapidly emerging in the Internet nowadays, like Facebook. Such systems typically create a sort of virtual world under which people live, share information and interact; such worlds sometime also provides facilities to ease the connection of people based on their data or interests, as well as artefacts like online games, quizzes and other services. In a sense, a world of

13

Middleware Infrastructures for Self-organising Pervasive Computing

341

this kind can be seen as a sort of “ecology” where people, data and services live and self-organise, and where each of them has a dynamic pertinency that can make them being either more and more popular (people with increasing connections, news with increasing clicks, services with increasing clients) or unused until becoming like garbage to be removed for efficiency. Considering this perspective, we believe that social systems very much resemble sort of virtual self-organisation environments, and as such they can be seen as an interesting application area for an infrastructure as described in this paper.

13.9 Conclusion The peculiar characteristics of emerging and future network scenarios challenge current service frameworks and architectures, calling for novel service models and associated innovative middleware infrastructure capable of supporting selforganisation, self-adaptation, and evolvability, of services and applications. In this paper, we have elaborated on the idea of getting inspiration from natural ecosystem, i.e. of conceiving future service frameworks as an ecology of data, services and resources. There, services are modelled and deployed as autonomous individuals in an ecosystem of other services, data sources and pervasive devices, whose spatial activities and interactions are to be supported by a minimal middleware infrastructure. Such middleware infrastructure, which can be inspired by different natural metaphors, enforces the components of the ecosystems in obeying to a simple set of well-defined “laws of nature”. In this way, it is possible to deliver self-organisation, self-adaptation and evolution as inherent properties of the service framework, rather than as complicated ad hoc solutions. The specific examples of the TOTA and TuCSoN middleware infrastructures have shown that such innovative concepts can be indeed realised and that their usage is, in the end, simple and intuitive. In any case, the road towards the widespread deployment of usable and effective “eco-inspired” open service frameworks still requires answering to several challenging questions, as we have tried to identify.

13.10 Problems–Exercises 13.1 Middleware systems like TOTA and TuCSoN, by embedding the necessary behaviours and functionalities to support self-organisation, makes it possible to have much lighter and simpler application components than when relying on traditional middleware architectures. To prove this, refer to a traditional service-oriented or event-based middleware of your knowledge. Then, with reference to the case study introduced in this chapter, try to design (with pencil and paper) some of the services discussed in this chapter with respect to the case study. Finally, compare the complexity of these components with those resulting from having TOTA and TuCSoN as supporting middleware infrastructures.

342

M. Casadei et al.

13.2 Is it possible to realise TOTA-like models of interactions above the TuCSoN middleware? And viceversa? How can this be done? 13.3 Download the TOTA middleware (http://polaris.ing.unimo.it/tota/download. html) and its simulation environment, and try to reproduce some of the selforganising physical coordination patterns presented in this chapter. 13.4 Download the TuCSoN middleware (http://alice.unibo.it/xwiki/bin/view/Tu CSoN/Download) and try to reproduce some of the chemical patterns of coordination presented in this chapter. Key Points • Need for innovative open pervasive middleware infrastructures to support self-organisation, self-adaptation and evolvability in distributed applications; • The middleware infrastructure can be nature-inspired; • TOTA is an example of physically inspired field-based middleware; • TuCSoN is an example of chemical inspired middleware.

13.11 Further Reading Design patterns from biology for distributed computing. A detailed description of a conceptual framework for capturing several biological processes to be used as design patterns for distributed computing. (O. Babaoglu et al., 2006, ACM Transactions on Autonomous and Adaptive Systems, 1(1):26–66.) A survey of autonomic communications. A comprehensive survey of the current autonomic communication research. (S. Dobson et al., 2006, ACM Transactions on Autonomous and Adaptive Systems, 1(2):223–259.) Programming pervasive and mobile computing applications: The TOTA approach. An overall description of the TOTA middleware, including application examples. (M. Mamei and F. Zambonelli, 2009, ACM Transactions on Software Engineering Methodology, 18(4).) A biochemical approach to adaptive service ecosystems. An introduction to biochemical mechanisms to develop innovative service frameworks for self-adaptive and long-lasting evolvability applications. (M. Viroli and F. Zambonelli, 2010, Information Sciences, 180(10):1876–1892.)

References 1. Agha, G.: Computing in pervasive cyberspace. Commun. ACM 51(1), 68–70 (2008). doi:10. 1145/1327452.1327484

13

Middleware Infrastructures for Self-organising Pervasive Computing

343

2. Babaoglu, O., Canright, G., Deutsch, A., Caro, G.A.D., Ducatelle, F., Gambardella, L.M., Ganguly, N., Jelasity, M., Montemanni, R., Montresor, A., Urnes, T.: Design patterns from biology for distributed computing. ACM Trans. Auton. Adapt. Syst. 1(1), 26–66 (2006). doi:10. 1145/1152934.1152937 3. Barros, A.P., Dumas, M.: The rise of web service ecosystems. IT Prof. 8(5), 31–37 (2006). doi:10.1109/MITP.2006.123 4. Beal, J., Bachrach, J.: Infrastructure for engineered emergence on sensor/actuator networks. IEEE Intell. Syst. 21(2), 10–19 (2006). doi:10.1109/MIS.2006.29 5. Berry, G., Boudol, G.: The chemical abstract machine. Theor. Comput. Sci. 96(1), 217–248 (1992). doi:10.1016/0304-3975(92)90185-I 6. Berryman, A.A.: The origins and evolution of predator–prey theory. Ecology 73(5), 1530– 1535 (1992) 7. Bonâtre, J.P., Le Métayer, D.: Gamma and the chemical reaction model: ten years after. In: Coordination Programming, pp. 3–41. Imperial College Press, London (1996) 8. Cabri, G., Leonardi, L., Mamei, M., Zambonelli, F.: Location-dependent services for mobile users. IEEE Trans. Syst. Man Cybern., Part A, Syst. Hum. 33(6), 667–681 (2003) 9. Casadei, M., Menezes, R., Viroli, M., Tolksdorf, R.: A self-organizing approach to tuple distribution in large-scale tuple-space systems. In: Hutchison, D., Katz, R. (eds.) Self-Organizing Systems. LNCS, vol. 4725, pp. 146–160. Springer, Berlin (2007). doi:10.1007/978-3-540-74917-2. http://www.springerlink.com/content/f602r041350x5u8v/. 2nd International Workshop on Self-Organizing Systems (IWSOS 2007), The Lake District, UK, 11–13 Sep. 2007. Proceedings 10. Casadei, M., Omicini, A., Viroli, M.: Prototyping A&A ReSpecT in Maude. Electron. Notes Theor. Comput. Sci. 194(4), 93–109 (2008). doi:10.1016/j.entcs.2008.03.101 11. Castelli, G., Rosi, A., Mamei, M., Zambonelli, F.: A simple model and infrastructure for context-aware browsing of the world. In: Pervasive Computing and Communications, 19–23 March 2007, pp. 229–238 (2007). doi:10.1109/PERCOM.2007.4 12. Crowcroft, J.: Toward a network architecture that does everything. Commun. ACM 51(1), 74–77 (2008). doi:10.1145/1327452.1327486 13. Gelernter, D.: Generative communication in Linda. ACM Trans. Program. Lang. Syst. 7(1), 80–112 (1985). doi:10.1145/2363.2433 14. Gillespie, D.T.: Exact stochastic simulation of coupled chemical reactions. J. Phys. Chem. 81(25), 2340–2361 (1977) 15. Giunchiglia, F., Shvaiko, P.: Semantic matching. Knowl. Eng. Rev. 18(03), 265–280 (2003). doi:10.1017/S0269888904000074 16. Giunchiglia, F., Yatskevich, M., Shvaiko, P.: Semantic matching: algorithms and implementation. J. Data Semant. 9, 1–38 (2007) 17. Herold, S., Klus, H., Niebuhr, D., Rausch, A.: Engineering of it ecosystems: design of ultralarge-scale software-intensive systems. In: ULSSIS’08: Proceedings of the 2nd International Workshop on Ultra-Large-Scale Software-Intensive Systems, pp. 49–52. ACM, New York (2008). doi:10.1145/1370700.1370714 18. Huhns, M.N., Singh, M.P.: Service-oriented computing: key concepts and principles. IEEE Internet Comput. 9(1), 75–81 (2005). doi:10.1109/MIC.2005.21 19. Jain, R.: Eventweb: developing a human-centered computing system. Computer 41(2), 42–50 (2008). doi:10.1109/MC.2008.49 20. Jazayeri, M.: Species evolve, individuals age. In: IWPSE’05: Proceedings of the Eighth International Workshop on Principles of Software Evolution, pp. 3–12. IEEE Computer Society, Washington (2005). doi:10.1109/IWPSE.2005.27 21. Johansson, S., Saffiotti, A.: Using the electric field approach in the RoboCup domain. In: RoboCup 2001: Robot Soccer World Cup V. LNAI, vol. 2377. Springer, Berlin (2002) 22. Kephart, J.O., Chess, D.M.: The vision of autonomic computing. Computer 36(1), 41–50 (2003). doi:10.1109/MC.2003.1160055 23. Mamei, M., Zambonelli, F.: Programming pervasive and mobile computing applications with the TOTA middleware. In: Pervasive Computing and Communications, pp. 263–273. IEEE Press, New York (2004). doi:10.1109/PERCOM.2004.1276864

344

M. Casadei et al.

24. Mamei, M., Zambonelli, F.: Field-based Coordination for Pervasive Multiagent Systems. Springer, Berlin (2006) 25. Mamei, M., Menezes, R., Tolksdorf, R., Zambonelli, F.: Case studies for self-organization in computer science. J. Syst. Archit. 52(8–9), 443–460 (2006) 26. Menezes, R., Tolksdorf, R.: Adaptiveness in Linda-based coordination models. In: Engineering Self-Organising Systems: Nature-Inspired Approaches to Software Engineering. LNAI, vol. 2977, pp. 212–232. Springer, Berlin (2004). doi:10.1007/b95863 27. Omicini, A.: Formal ReSpecT in the A&A perspective. Electron. Notes Theor. Comput. Sci. 175(2), 97–117 (2007). doi:10.1016/j.entcs.2007.03.006. 5th International Workshop on Foundations of Coordination Languages and Software Architectures (FOCLASA’06), CONCUR’06, Bonn, Germany, 31 Aug. 2006. Post-proceedings 28. Omicini, A., Denti, E.: From tuple spaces to tuple centres. Sci. Comput. Program. 41(3), 277– 294 (2001) 29. Priami, C.: Stochastic π -calculus. Comput. J. 38(7), 578–589 (1995) 30. Prigogine, I., Steingers, I.: The End of Certainty: Time, Chaos, and the New Laws of Nature. Free Press, New York (1997) 31. Ramakrishnan, R., Tomkins, A.: Toward a peopleweb. Computer 40(8), 63–72 (2007). doi:10.1109/MC.2007.294 32. Ulieru, M., Grobbelaar, S.: Engineering industrial ecosystems in a networked world. In: 5th IEEE International Conference on Industrial Informatics, 23–27 June 2007, pp. 1–7. IEEE Press, New York (2007). doi:10.1109/INDIN.2007.4384717 33. Villalba, C., Rosi, A., Viroli, M., Zambonelli, F.: Nature-inspired spatial metaphors for pervasive service ecosystems. In: 1st International SASO Workshop on Spatial Computing, Venezia, Italy. IEEE Comput. Soc., Los Alamitos (2008) 34. Viroli, M., Casadei, M.: Biochemical tuple spaces for self-organising coordination. In: Proceedings of the International Conference on Coordination Models and Languages, Lisboa, Portugal. LNCS. Springer, Berlin (2009, to appear) 35. Viroli, M., Casadei, M.: Exact biochemical tuple spaces for self-organising coordination. In: 11th International Conference on Coordination Models and Languages (Coordination 2009), Lisbon, Portugal. Springer, Berlin (2009, to appear) 36. Viroli, M., Casadei, M., Omicini, A.: A framework for modelling and implementing selforganising coordination. In: Shin, S.Y., Ossowski, S., Menezes, R., Viroli, M. (eds.) 24th Annual ACM Symposium on Applied Computing (SAC 2009), Honolulu, Hawai’i, USA, vol. III, pp. 1353–1360. ACM, New York (2009) 37. Viroli, M., Zambonelli, F., Casadei, M., Montagna, S.: A biochemical metaphor for developing eternally adaptive service ecosystems. In: Shin, S.Y., Ossowski, S., Menezes, R., Viroli, M. (eds.) 24th Annual ACM Symposium on Applied Computing (SAC 2009), Honolulu, Hawai’i, USA, vol. II, pp. 1221–1222. ACM, New York (2009)

Part IV

Applications of Self-organising Software

Chapter 14

Self-organisation in Constraint Problem Solving Pierre Glize and Gauthier Picard

In tackling the problems of tomorrow with yesterday’s organizations, we gather the dramas today. Michel Crozier

Objectives After reading this chapter, the reader will: • Appreciate self-organisation as a mechanism to tackle distributed and dynamic combinatorial problems; • Understand the differences between self-organisation and classical approaches to combinatorial problem solving; • Know how to implement a self-organising system to tackle CSPs.

14.1 Introduction Self-organisation is a mechanism that appears in natural systems as a way to adapt and to respond to the environmental dynamics, as emphasised in previous chapters (see Part I: Main Concepts and Background). Such systems often adapt in the sense that they change their internal organisation to fit better the new environmental state. P. Glize () CNRS/IRIT, Université de Toulouse III, Toulouse, France e-mail: [email protected] G. Picard École Nationale Supérieure des Mines de Saint-Etienne, Saint-Etienne, France e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_14, © Springer-Verlag Berlin Heidelberg 2011

347

348

P. Glize and G. Picard

For instance, a collective of foraging ants will easily adapt to a environmental disaster that break routes to food sources by finding other paths, which can be easily explained by the stigmergic mechanisms (see Chap. 5: Stigmergy). If we now consider that such a system aims at solving a problem (for example finding food) with respect to given constraints (for example in a minimum time), self-organisation can be viewed as a mechanism to adapt to problem changes and dynamics, in a manner that the system always aims to perform a function as optimally as possible, given the disturbances of the environment (for example resources disappearance) or within the system itself (for example agent (dis)appearance). Moreover, self-organisation can also be considered as the mechanism that leads to the collective and decentralised problem solving, thanks to cooperative interactions between the parts of the system. For instance, stigmergic mechanisms are the means for a collective to achieve a global goal (for example finding the shortest path to a food source) without any global controller. It is these phenomena that are here regrouped into the notion of self-organising problem solving or optimisation. Therefore, the resulting question is whether self-organisation is a suitable approach to solving dynamic and distributed constraint problems.

14.1.1 Contents This chapter aims to provide an understanding of the suitability of using selforganisation to solve distributed and dynamic problems. Therefore it describes how self-organisation can be applied to constraint problem solving and what are the benefits we get from that. It is structured as follows. We first precise the context background of this chapter, which is the multi-agent approaches for problem solving, in Sect. 14.2. Since problem solving is here expressed as constraint-based combinatorial problem solving, Sect. 14.3 expounds the CSP theoretical notions and the existing approaches to tackle such problems. We also analyse the limitations of these methods in distributed and dynamic environments. Section 14.4 presents an illustration of how a self-organising multi-agent system solves a constraint-based problem. Finally, summary, synthesis and some exercises conclude this chapter in Sect. 14.5.

14.2 Background Context As to automatise hard and large problem solving, one possible way to represent problems is the CSP (Constraint Satisfaction Problem) formalism, which considers a problem as a set of variables to assign a given set of constraints between the possible values for these variables. Solving a problem is finding an assignment for each variable that respects the constraints (or minimises the number of violated constraints). Finding such an assignment (therefore a solution) is a difficult problem (NP-complete). There exist numerous methods to solve completely or approximately CSP [6], in a centralised manner, that will not be discussed in this chapter, since we will address more specifically distributed problems (DisCSP).

14

Self-organisation in Constraint Problem Solving

349

DisCSP considers CSP where variables (which represents the goals of the system) and constraints are distributed among a population of agents1 that have to collectively solve the CSP. Each agent is responsible for assigning one or more variables by interacting with other agents. This does not imply that the solving process is decentralised. In fact there exist distributed versions of classical algorithms that still remain centralised, in the control point of view—even if the execution and the solving process is concurrent, for instance several sub-processes solve sub-parts of the whole problem in a parallel manner but are coordinated by a single entity that may fail or dysfunction. A way to make this process decentralised is to implement self-organising mechanisms where agents are able to change by themselves the organisation without any central controller. For more details and background on DisCSP, we redirect the reader to the Yokoo’s book [35]. Our purpose is consequently oriented to approaches where software agents manage locally a distributed algorithm and where the solving process depends heavily on their organisation.

14.2.1 Locality There are many types of algorithms to solve constraint-based problems. Notably, we can identify two main classes: constructive algorithms and algorithms with complete assignment. The first class considers algorithms that build a solution, step by step, by assigning variables in a given order (using tree structure, for example). The second class considers algorithms in which each variable is assigned at any time, even if this assignment is not a solution for the problem. These approaches then try to enhance the given assignment using techniques like hill-climbing [6]. The system has to move from state to state, by following a particular exploration policy, in order to converge to a solution [9]. Since we will consider systems where variables are controlled by agents, the approach discussed in this chapter undoubtedly refers to the second class of algorithms. Generally speaking, the principle of locality refers to the fact that agents have a limited amount of information and a limited reasoning capability, or bounded rationality, and are only influenced by their immediate surroundings. Classically, agents in CSP are impoverished: they have no explicit beliefs associated to the constraints, no communication languages or protocols, no autonomy or even no creation/suppression capabilities. As we will see, these characteristics are required for full self-organisation mechanisms allowing dynamicity, robustness and openness to the solver. This solver is here viewed as a “living” system of autonomous agents managing variables that always has a complete state (assignment) and that evolves as to provide a solution or an optimum. However, contrary to local search, selforganising systems, and more precisely the agents composing a self-organising system, do not reason on the global state of the system (for example, foraging ants are 1 The

agent terminology is used from more than twelve years in the CSP domain.

350

P. Glize and G. Picard

unaware of the paths lengths). Despite the lack of guarantee to find a global optimum, self-organisation enacts the richness of possibility in the exploration of the state space. Thus, the solving process is an evolution of the system (due to changing interactions between agents) towards a solution, which represents a stable state that can be detected at the local level by agent themselves.

14.2.2 Agents Agents described in this chapter are compliant with notions presented in Chap. 2, Software Agents, by following guidelines from Part III, Engineering Artificial SelfOrganising Software. Agents can represent several concepts. In fact, several modeling choices can be made to develop a multi-agent system able to solve constraintbased problems: (i) Agents encapsulate one or more variables and aim at finding assignments for them. The solving process consists of cooperating to collectively find a global solution. Such models are mainly used in DisCSP framework (see Sect. 14.3.1). Here the solution is the state of the whole system. (ii) Each agent represents one point in the search space (a complete assignment). Agents concurrently explore the search space, by moving from point to point, and therefore by changing their state. Such models are used in some local search and population-based approaches (see Sects. 14.3.2 and 14.3.3). Here the solution is the state of one or more agents assigned with a particular value. (iii) Agents represent domain-specific entities that do not directly manipulate constraint-based concepts. For instance, modelling resource assignment in manufacturing control can be done by modelling agents representing machines, containers or operators. These approaches often mix up the two previous visions. For example, an agent is responsible for a set of variables and/or a region of the search space. Others approaches make also cooperate several solvers as to improve the solving process convergence. Some examples of such ad hoc approaches are presented in Sects. 14.3.4 and 14.4.

14.2.3 Organisation In the context of this chapter, the notion of organisation can take two main meanings: (i) As the positioning within the CSP framework will manipulate the notion of constraint network, an organisation of the agents can be structured around the constraints between agents; for example, agents sharing constraints are connected within the organisation, forming therefore neighbourhoods. So, there is a possible mapping between the problem structure and the multi-agent organisation (see Sect. 14.3.1).

14

Self-organisation in Constraint Problem Solving

351

(ii) Agents can play different roles during the solving process, which can be interpreted as an organisation; for example, in ABT (Asynchronous Backtracking, see Sect. 14.3) agents are organised with a complete order using priorities, whereas in DBA (Distributed Breakout Algorithm) agents are equal. In this case, there is a mapping between the solving process and the multi-agent organisation (see Sect. 14.3). When this organisation (structure or process) is likely to change, the system requires reorganisation capabilities. Reorganisation means here that the organisation based on the problem structure (i.e. the constraint network representing the variables with nodes and the constraints with edges between nodes) can change by adding or removing constraints or variables, or the organisation based on the solving process (i.e. functional assignment of tasks to agents) can change by assigning new roles or responsibilities to agents. When these changes are initiated by agents themselves, at run-time, we talk about self-organisation. In this case adaptation is the consequence of the self-organising capabilities of the system.

14.3 Theoretical Notions This section presents the main concepts related to self-organising multi-agent problem solvers, which mainly consist of the inspiring methods to classically solve problems expressed as constraint satisfaction/optimisation ones. In this section, we will classify constraint problem solving approaches and evaluate each category with respect to the characteristics presented below so that to identify commonalities and differences between categories. Problem distribution: in multi-agent systems, it relates to the manner and the degree of distribution of the problem among agents (for instance one variable per agent). Decision decentralisation: in multi-agent systems, decentralisation means that no agent has the power and the capability to decide for the others, or to solve the whole part of the problem, at a given time. This dimension is not binary, but a continuous spectrum from totally centralised to totally decentralised or distributed. Bounded rationality and local actions: in multi-agent systems, an agent cannot know the values of other agents, and its actions are limited to its own limited neighbourhood. Robustness to dynamics: in self-organising systems, the impact of environmental disturbances occurring at run-time is minimised by the peer-to-peer propagation. Non-determinism: in most self-organising approaches, agents behave or are initially set non-deterministically. Global state is unknown: in self-organising systems, micro-level entities are not conscious of the global state of the system, which is however evaluable at macrolevel. We will further use these points as an analysis matrix for determining whether or not an existing multi-agent approach can be considered as self-organising, or can be

352

P. Glize and G. Picard

used as a starting point for designing a self-organising solver. These keypoints are analysed in sections entitled “Self-Organising Multi-Agent Viewpoints” at the end of each presented approach. We will present three families of methods which constitue a rich inspiration sources for developing self-organising problem solvers, even if they are partially relevant in a dynamic and distributed context: • Complete and asynchronous solvers for combinatorial problems, within the DisCSP framework, such as Asynchronous Backtracking (ABT) or Asynchronous Weak-Commitment Search (AWCS). • Distributed local search methods, such as Distributed Breakout Algorithm (DBA) or Environment, Reactive rules and Agents (ERA) approach. • Population-based methods, such as Particle Swarm Optimisation (PSO) or Ant Colony Optimisation (ACO). • The section ends with the presentation of self-organising methods inspired by the previous ones.

14.3.1 DisCSP Framework and Extensions The DisCSP (Distributed Constraint Satisfaction Problem) framework has been introduced by [35]. As we will see, the algorithms presented here are inspired by classical centralised ones, like the backtracking procedure. The main advantages of these algorithms are that they are complete (but time consuming) and that they propose an agent modeling which is directly derived from the CSP formulation of the problem to solve. However, they are still centralised since they need a total order among agents, which may reduce the robustness in dynamic environments. Classical constraint-based decision problems can be expressed by using the CSP formalism. A CSP is a triplet X, D, C where X = {x1 , . . . , xn } is the set of variables to instantiate, D = {D1 , . . . , Dm } is the set of domains, each variable xi is related to a domain of value, and C = {c1 , . . . , ck } is the set of constraints, which are relations between some variables from X that constrain the values the variables can be simultaneously instantiated to. Therefore, making a decision consists in finding a solution, for instance a complete and consistent assignment of X. Constraint satisfaction is NP-complete in general. In DisCSP, distribution can affect either variables or constraints. Most approaches consider the first kind of distribution by defining a function φ (also defined by a predicate belongs) that bounds variables to stakeholders (agents for example): φ(ci ) = j (or belongs(ci , j )) means that the constraint ci belongs to stakeholder j . In most approaches, agents concurrently behave during a loop consisting in waiting for messages and reacting to received messages. Such messages contain information about the chosen values, the conflictual values, the violated constraints or even organisational information such as priorities.The topology of a constraint-based problem can be represented by a constraint network, in

14

Self-organisation in Constraint Problem Solving

353

i←0 Di ← Di while 0 ≤ i < n do xi ← null ok? ← false while not ok? and Di not empty do a ← a value from Di remove a from Di if a is in conflict with {x0 , . . . , xi−1 } then xi ← a ok? ← true end end if xi is null then backtrack i ←i −1 else i ←i +1 Di ← Di end end Algorithm 14.1: A classical centralised backtracking search method

which vertexes represent variables, and edges represent binary constraints2 between variables.

14.3.1.1 Asynchronous Algorithms for DisCSP As mentioned before, there exist numerous centralised and efficient methods to tackle constraint-based problems. Therefore, it is not surprising to utilise them as inspiration for developing distributed constraint solvers. The most famous are the classical backtracking, inspired by Algorithm 14.1 and the weak-commitment search [34], which led to the first distributed constraint solvers, ABT (Asynchronous Backtracking) and AWCS (Asynchronous Weak-Commitment Search) [35]. These algorithms mainly rely on an ordered organisation, in which agents only communicate with agents with lower priority (maintaining variables with higher index i) for informing value changes and with the agent with direct higher priority for informing that there is a conflict (recorded as a nogood) in their variable assignments. The agent with the highest priority (for example x0 ) is responsible for initiating the termination procedure. This priority order is fixed in ABT, whereas it changes at every conflict detection in AWCS, not to keep a part of the organisation 2 One

can note that not all the constraints are binary; but for many problems, n-ary constraints can be transformed into binary constraint by adding new constraints and variables [2].

354

P. Glize and G. Picard

that is known to be incorrect. For more details on these algorithms, we point the reader to the Yokoo’s book on DisCSP [35]. These previous algorithms were the first to tackle the solving of distributed problems, using distributed solvers. Nevertheless, they cannot be characterised as selforganising, since the organisation among agents is fixed or globally predefined using some static or dynamic order. The APO algorithm [22] was the first to introduce concepts close to reorganisation, since agents can play the role of mediator during the solving process. This mechanism is called cooperative mediation and mixes distributed and centralised problem solving. In APO, agents have priorities (depending on the size of their neighbourhood) and cooperate during mediation sessions. When an agent does not manage to find a consistent value that does not violate constraints with higher priority agents (for instance, x2 checks consistency only with x0 and x1 ), it initiates a mediation session, otherwise it changes its value and informs its neighbours. During mediation session, contacted agents which accept to enter the session inform the mediator about their conflicts for each value in their domain. The mediator then chooses an assignment for every mediating agent and informs them. The mediator also inform agents that are not directly connected by constraints, but for which it knows that current assignments may trigger constraint violation. Despite that the solving process is most of the time decentralised, mediators centralise the solving procedures for sub-problems. Adding links between these agents diminish the decentralisation in the sense that an agent may completely connect the network of agents if the constraint network is completely connected, and therefore the mediator may solve the entire problem alone, as a centralised classical solver. In all these algorithms, agents can be deployed among a physically distributed environment without explicit shared resources. Agents only need to know the addresses of their neighbours to directly send messages.

14.3.1.2 Optimisation Issues Since in dynamic and complex environments not all constraints can be satisfied completely, distributed CSPs are often handled as optimisation problems. This formulation is called a distributed constraint optimisation problem (DCOP). Finding an optimised solution is equivalent to finding a solution in which, for example, the sum of all the non-satisfied weighted constraints is minimal. But all problems cannot be viewed as in an utilitarian point of view, and objective functions can be more complex than a sum. Examples include a weighted sum observing hard constraints and multiple levels of soft constraints, leximin, absolute max, Nash equilibrium, etc. For distributed combinatorial optimisation, the ADOPT (Algorithm for Distributed Constraint Optimisation) algorithm is directly inspired by ABT and branchand-bound (B&B) algorithms and uses a pseudo-tree structure to order agents [24]. Each constraint is associated with a cost. Agents aims at minimising a global objective function (sum of the costs of the violated constraints). Each agent chooses a value minimising the total cost, which impacts the search for agents with lower priority (deeper in the tree organisation) by modifying bounds as in B&B algorithm.

14

Self-organisation in Constraint Problem Solving

355

Another notable optimisation algorithm is OptAPO, which is a simple extension of APO [21]. Both ADOPT and OptAPO are complete and terminate. Nevertheless, it is difficult to characterise them as self-organising since their structures are really rigid, but they still serve as a good inspiration for future algorithms.

14.3.1.3 Handling Dynamics “A problem is known as a dynamic one if it varies with time, i.e. the optimal solution does not have the same characteristics during the time of optimisation. These problems give rise to specific difficulties, owing to the fact that it is necessary and possible to approach the best solution at each instant of time” [9]. In a DisCSP context, dynamics are often only represented as adding or removing constraints. For this reason, in the majority of solving approaches suitable behaviours are commonly added to agents to respond to such changes by specifying “add Constraints” and “remove Constraints”. The most illustrating algorithms for tackling such problems are DynAPO [20], a dynamic version of APO and DynAWCS [11], an extension of AWCS.

14.3.1.4 Self-organising Multi-Agent Viewpoints DisCSP or DCOP do not imply any specific problem configuration and are generic frameworks. Nevertheless, under certain conditions, these previous algorithms may lead to a centralised problem solving or optimisation, in which only one top-level agent solves directly or indirectly the whole problem. For instance, in APO, if the constraint network is complete (for example, a constraint specifies that all the variable values must be different), the problem will be solved by only one mediator knowing all the values of all the agents. Distribution in DisCSP mainly concerns the decision and the solving process which is spread among multiple agents. However, it does not automatically imply that the solving process is really decentralised. Another viewpoint for the distribution is the intrinsic distribution of the problem to solve. Distribution can be intrinsically embedded within the constraint network. For instance, a complete graph restricts distributiveness, since it requires complete constraint-checking at each change. Therefore distribution is not a good solution. At the contrary, graphs with several quasi-isolated cliques can easily be distributed, in which several solving sites can appear. The fixed organisation in the previous algorithms reduces drastically the principle of locality (see Sect. 14.2.1) and therefore the robustness to agent disappearance and dysfunctions. In fact, organisations such as total order or pseudo-tree organisation may lead to a centralised failure point. However, the actions of the agents are local: agents only change their own values and directly communicate using asynchronous messages. In these complete algorithms, the only source of non-determinism is the order of agents’ actions, but not the actions themselves, due to the asynchronous execution. However, AWCS is a first step towards self-organisation: the order changes

356

P. Glize and G. Picard

choose an initial assignment s(0) while s(t) not terminal do select an acceptable move m(t) to another assignment apply move m(t) to reach s(t + 1) t := t + 1 end Algorithm 14.2: A generic centralised local search algorithm

during the solving process with respect to the sequence of events. In APO approaches, the mediator role depends also on the sequence of events and the problem topology.

14.3.2 Distributed Local Search Approaches Local search (LS) algorithms explore the search space from state to state, from complete assignment to complete assignment. They mainly behave as presented in Algorithm 14.2. The main advantage of this anytime behaviour is that it can naturally handle dynamics (adding constraints, changing values) because it always tends to improve the current state of the system, and more specifically, when the state has been altered by environmental disturbances. Even if often time efficient, they are not complete and require some subtle parameter tuning.

14.3.2.1 Classical Centralised LS Algorithms Many methods or meta-heuristics exist to implement LS, which involve different termination criteria and acceptable moves. Such termination criteria can, for instance, be time, number of iterations or a distance to the solution. In general, such algorithms are used for global optimisation and are really efficient but not complete. In tabu search [10], acceptable moves are moves that diminish the cost of the solution (for example the number of violated constraints) but that are not in a tabu list consisting of the n last visited states as to avoid local minima. The size n of the list strongly depends on the search space configuration and is often difficult to set. Simulated annealing [17] specifies acceptable moves using an analogy to thermodynamics. At each state, a neighbourhood is computed following stochastic rules inspired by physics. Selecting an acceptable move only consists in randomly selecting one neighbour. Here again, determining the neighbourhood of a state is strongly problem-dependent. For its part, the iterative improvement breakout method [25], coupled with the min-conflict heuristic [23], uses a penalty to escape from local minima by adding a weight to violated constraint during exploration. Therefore, when a constraint is violated during several consecutive steps, its weight increases, and thus the algorithm escapes from this state to another state with a lower constraint cost.

14

Self-organisation in Constraint Problem Solving

357

All these approaches are very efficient, but they still remain centralised and mainly focus on global optimisation of an objective function, which is evaluated at each step. We will now focus on techniques, inspired by such approaches, but with some distribution and decentralisation. However, we will not discuss about parallel local search approaches (PLS), which proposes to launch several instances of LS algorithms with different parameters as to concurrently explore the search space. We redirect the interested readers to the book edited by Talbi [31]. Naturally, as to tackle distributed problems in distributed environments, some recent approaches took inspiration from the aforementioned local search methods and techniques. However, the main matter is to provide distributed mechanisms that do not use a global knowledge or a unique objective function, due to the distributed context, contrary to those global optimisation methods which compute next assignment using the global evaluation of the state. We present here two different approaches, DBA and ERA, which differ in the way they coordinate the solving process.

14.3.2.2 Distributed Breakout Algorithm (DBA) DBA (Distributed Breakout Algorithm) is a distributed version of the breakout algorithm [12]. As in ABT or AWCS, DBA proposes a concurrent execution of agents looking for variable assignments, with respect to their constraints, but contrary to these two approaches, there is no order over the agents society, which positions this approach within the self-organising ones. Generally, distributing a centralised algorithm mainly raises two difficulties: (i) if two neighbouring agents concurrently change their values, the system may oscillate; (ii) detecting the fact that the whole system is trapped in local minimum requires the agents to globally exchange data. Therefore, Yokoo introduces the following answers to these difficulties: (i) for a given neighbourhood, only the agent that can maximally improve the evaluation value is given the right to change its value; (ii) agents only detect quasi-localminimum, which is a weaker local-minimum that can be detected only by local interactions. In fact, each agent knows its constraints and their associated weights, and can compute its evaluation (the sum of the weight of the violated constraints) and its potential improvement. The core outline of DBA is presented in Algorithm 14.3. Even if more detailed, Algorithm 14.3 mainly follows Algorithm 14.2, but in a distributed and decentralised manner. Every agent follows a two-mode behaviour alternating between exchange of potential improvement and exchange of assignments. The system halts if a solution is found3 (all the agents have no conflict) or if the weight of constraints have reached a predefined upper bound, which represents the only difficult parameter to set. This algorithm, even if non-complete, is able to detect the termination or a global solution only by reasoning on local data. 3 For

more details on how to detect that a solution is found by using propagation only, we redirect the reader to [12].

358

P. Glize and G. Picard

foreach agent i do concurrently randomly set an initial assignment set the weight of all constraints to 1 t_counter ← 0 round ← 0 send the assignment to i’s neighbours while t_counter < predefined upper bound do wait for ok? messages from i’s neighbours if constraint violations then t_counter ← 0 end LC ← local change that can reduce the cost send t_counter and LC to i’s neighbours wait for improve messages from neighbours t_counter ← minimum t_counter from i’s neighbours if neither i nor some of its neighbours have constraints violation then t_counter ← t_counter + 1 end if i detects a quasi-local-minimum then increased the weights of violated constraints end if LC does not involve potential conflicts then apply changes from LC else apply changes from LC that are still valid after conflict resolution end send the assignment to i’s neighbours end end Algorithm 14.3: DBA Agent’s behaviour outline (extracted from [12])

14.3.2.3 Environment, Reactive Rules and Agents (ERA) ERA (Environment, Reactive rules and Agents) [18] proposes a multi-agent system in which agents evolves in an discrete grid environment that is used as a communication medium, as is the case in stigmergic approaches (see Chap. 5, Stigmergy). Similar to ABT or AWCS, agents in ERA are responsible for assigning a value to a variable. An agent evolves in a separate environment from the other ones, which consists in a row with one cell by possible value for its variable. An agent cannot move in the row of another agent, but can mark it by using artifacts indicating the number of potential conflicts, as in the min-conflict heuristic [23]. Contrary to ABT, AWCS or DBA, agents move synchronously as presented in Algorithm 14.4. Each agent, when it chooses a position within its environment, increments the conflict values of the cells with which it is in conflict, with respect to the constraints of the DisCSP, and decrements the values of the cells with which it is not in conflict

14

Self-organisation in Constraint Problem Solving

359

t ←0 initialise the grid to 0 violation in each cell; foreach agent i do randomly move to a cell of row i end while t < tmax and no solution do foreach agent i do select a move behaviour compute new position decrease markers in all cells with past violations increase markers in all cells with new violations end t ←t +1 end Algorithm 14.4: ERA outline (extracted from [18])

anymore. These values represent therefore the number of violated constraints if an agent chooses the marked cell. Agents move by reasoning as in local search methods. There are three different possible move behaviours: least-move, better-move and random-move, each of them being associated with a probability, as in simulated annealing. A decision consists in a random Monte Carlo choice of the action to perform. The least-move action moves the agent to the cell with minimum cost. The better-move action randomly chooses a cell; if its cost is better than the current one, the agent moves, else it rests. Finally, the random-move action aims at exiting from local minima in which there is no better cell, by accepting to move even if it degrades the current solution. When every agent is on a cell with a cost equal to 0, the solution is found. As the environment is used to communicate and coordinate the agents, there is no asynchronous mechanisms and message handling. In counterpart, the environment often represents a synchronisation point that can lead to high synchronous solving process with no benefit from distribution, in case of highly connected constraint networks. ERA quickly finds assignments close to the solution, which can be interesting for repairing issues in optimisation problems. One major flaw of this self-organising approach is the redundant usage of random choices (for choosing the action to do, and then to choose a cell), which produce a non-guided method, close to random walk, and non-complete. Contrary to all the distributed algorithms that have been presented, ERA requires a resource shared by the agents: the grid. It can be split, but it may require a lot of messages for update and consistency check. Finally, concerning termination, ERA requires a time limit (tmax ) which is as difficult to set as the upper bound of DBA, since it is strongly problem-dependent.

14.3.2.4 Self-organising Multi-Agent Viewpoints In the DBA case, the decision is completely decentralised, and this is why a global termination parameter is required (the upper-bound), contrary to aforementioned al-

360

P. Glize and G. Picard

gorithms, which centralise the termination decision in a predefined (ABT) or a dynamically chosen agent (AWCS, APO). As for these algorithms, DBA is completely distributed since agents directly exchange messages with agents in their neighbourhood.

14.3.3 Population-Based Approaches In population-based approaches, agents use simple local rules to govern their actions, and via the interactions of the entire group, the population achieves its objectives. The cooperative individual behaviour leads to an emergent collective one. From an engineering point of view, population-based algorithms are largely applied to optimisation problems. Each agent has the capability to find autonomously the solution of the global problem, without following central orders or some global plan. This class contains—but is not limited to—evolutionary algorithms, genetic algorithms (GA), particle swarm optimisation (PSO) and ant colony optimisation (ACO) which are presented here in a simplified way. We analyse in this part how some algorithms are able to go to an optimum satisfying local constraints and driven by local information. The advantage of these methods is their inspiration from natural phenomena adapted to dynamic environments, which may ease their applicability to dynamic problem solving contexts. However, they require more memory than the previous approaches since the search space is explored concurrently at several states.

14.3.3.1 Ant Colony Optimisation (ACO) ACO is a meta-heuristic initially proposed by M. Dorigo, V. Maniezzo and A. Colorni as a method for solving combinatorial optimisation problems [8]. ACO is a stigmergic approach (see Chap. 5, Stigmergy) that uses an environment on which information is stored and read to coordinate the search. The basic four steps of an ACO algorithm is the following:

set parameters initialise trails while termination conditions not met do construct ant solutions apply local search; // optional update pheromones end Algorithm 14.5: Ant Colony Optimisation outline

14

Self-organisation in Constraint Problem Solving

361

Some parameters must be initialised such as the number of artificial ants to work into the search space, the formula to choice the current partial solution, the termination conditions, the initial value of the pheromone and the pheromone decay. A termination condition generally occurs when a solution is completed or when a maximum number of steps is reached. The solutions are constructed incrementally from partial solutions. Each ant decides to add a feasible solution component from the set of feasible neighbours with respect to the current partial solution. The choice of a solution component is done probabilistically close to a Monte Carlo approach using the amount of pheromone for each solution component. A considerable number of optimisation problems include heuristic information useful to increase additional hints for the decision: this can be used in the apply local search step. When available, this coupling with local search improves the quality of the solution. Several variants of pheromone update exist, but the basic approach consists first in reducing the pheromone values by a fixed proportion and second in increasing the pheromone values corresponding to better solutions. Consequently, pheromone guides ants near the paths to good solutions and avoiding older paths by the evaporation process. ACO algorithms have demonstrated their performance on numerous applications such as scheduling, vehicle routing, quadratic assignment, travelling salesman, graph colouring, frequency assignment, generic constraint satisfaction and sequential ordering. The algorithm principle indicates that ACO are well suited for parallel implementations in having several populations which work on separate CPUs. In counterpart, this requires intensive communication between them in order to exchange information about better solutions. The global performance depends on the balance between the number of populations and the number of ants. We could conclude that the constraints implied by the distribution are more directly the consequences of the algorithm itself (distribution of the search process among populations) than the variable dependencies themselves (for instance, the constraint network can be completely connected or can have several cliques); thus, distribution is not really effective in ACO. Nevertheless, they become inappropriate for continuous problems when the domain of variable is large. This is due to their inefficiency in the case where the size of the problem is particularly large and the necessity to split each variable domain by an a priori number of intervals. The algorithm tuning depends on the number of ants in the colony and the pheromone decay. Thus, the solution accuracy is sensitive to the application dynamics (number of variables and constraints); the more the current problem configuration is far from the initial one, the more the result is far from the optimum.

14.3.3.2 Particle Swarm Optimisation (PSO) PSO is a stochastic global search optimisation technique for an inherently continuous variable domain, originally developed by Kennedy and Eberhart [15, 16]. The particle population—called swarm—searches for the whole solution space, where each particle represents a candidate solution of the problem being solved. The particle swarm algorithms find optimal regions of complex search space through the

362

P. Glize and G. Picard

foreach particle i do initialise position xi and velocity vi initialise the local best value: lbest ← xi initialise the global best value: gbest ← argminf (xi ) end while termination conditions not met do foreach particle i do update velocity: vi = w.vi + cg .rand(gbest − xi ) + cl .rand(lbest − xi ) update position dimension: xi = xi + vi update the local best value: lbest ← argmin[f (lbest ), f (xi )] update the global best value: gbest ← argmin[f (gbest ), f (xi )] end end Algorithm 14.6: Particle Swarm Optimisation outline

interaction of individual particle in the population. PSO shares many similarities with evolutionary computation techniques such as Genetic Algorithms. However, unlike GA, PSO has no evolution operators such as crossover and mutation. In ndimensional spaces, the position xi of a particle and its velocity vi are represented as vectors. Algorithm 14.6 presents the outline of a classical PSO. Initialisation concerns the population size, the inertia weight as a quite high value, the random positions and velocities of each particle. The fitness function f determines which particle has the best value in the swarm and also determines the best position of each particle over time. The process ends usually when a maximum number of iterations is reached, when there is no improvement during a given number of iterations or when the best solution is less than a predefined threshold. The inertia weight w (typically setup to vary linearly from 1 to near 0) provides a suitable balance between global (best global population value gbest ) and local exploration (best local values lbest ). Values cg and cl represent self-confidence of the particle on global and local influence. Each particle keeps track of its coordinates in the problem space which are associated with the best solution (fitness) it has achieved so far. The local best value is obtained so far by any particle in the neighbours of the particle. The particle swarm optimisation concept consists of, at each time step, changing the velocity of each particle towards its local and global locations. This means that if a particle discovers a promising new solution, all the other particles will move closer to it, exploring the region more thoroughly in the process. 14.3.3.3 Genetic Algorithms (GA) This approach is inspired from natural Darwinian evolution using mutation and selection. It was first used by John Holland and applied for finding solutions to optimisation and search problems [14]. In essence, a GA follows the standard procedure shown in Algorithm 14.7.

14

Self-organisation in Constraint Problem Solving

363

foreach chromosome i do initialise xi by the set of genes xi .fitness ← f (xi ) end while termination conditions not met do compute the fitness xi .fitness of the overall new population foreach individual xi of the population do select two individuals (xi , xj ) (xi , xj ) ← crossover(xi , xj ) (xi , xj ) ← mutation(xi , xj ) xi .fitness ← f (xi ) xj .fitness ← f (xj ) insert offspring in new generation population end replace the current population with the new population end Algorithm 14.7: A Genetic Algorithm outline

An individual is a particular chromosome and is referred to a genotype. The fitness function (devised for each problem to be solved) computes the phenotype of a given chromosome from its genotype. In a realistic problem this fitness combines several measures to optimise, and consequently is not so obvious to define. Parents xi and xj are selected using the Monte Carlo method, leading statistically to reward with more opportunities to reproduce the most fit individual. The two offspring inherit some genes from each parent, and the simplest approach uses only one crossover point (multiple choices can de done). Then mutation is applied to each child for altering each gene with a small probability (typically 0.001). Crossover and mutation give the balance between exploitation of a given location of the search space and the exploration of new ones. The procedure leads to the evolution of the population over successive generations, and the convergence is assumed when the entire population shares globally the same gene pool. 14.3.3.4 Self-organising Multi-Agent Viewpoints The distribution property can be seen at two levels. First, the distribution is complete concerning the ability of each agent to autonomously progress into the whole search space. However, the need to synchronise at the global level in order to share the best results (selecting the parents for crossover in GAs or updating pheromone in ACO) reduces distribution. Because every agent in a population contains the global description of the problem—and potentially the solution—decentralisation is not the strong point of these algorithms. Nevertheless, some works propose to split the problem into dependent sub-problems where a population is devoted to solve each sub-problem, and synchronisation is done by the constraints they share.

364

P. Glize and G. Picard

Table 14.1 Viewpoints analysis for all the overviewed methods ABT et al.

DBA

ERA

ACO

PSO

GA

Distribution

yes

yes

yes

yes

yes

yes

Decentralisation

no

yes

no

no

no

no

Locality

no

yes

yes

yes

no

no

Dynamics

limited

limited

yes

yes

yes

yes

Non-determinism execution

execution

behaviours

behaviours

behaviours

mechanisms

Global state

unknown

unknown

known

known

known

known

Except the synchronisation point in the algorithms for taking into account the best individuals, each agent has strictly local activity for exploring the search space. During the initialisation phase of population-based algorithms, some parameters must be tuned to obtain efficient search strategy. When problem description evolves slightly (the number of variables, their value domain and the constraints between them), the parameters could be left unchanged. When a great change occurs, the algorithm must be re-initialised because it is sensitive to high dynamics. The agent behaviour is non-deterministic for two reasons. First, each algorithm uses at some level a random choice (such as Monte Carlo) to decide the current action in order to keep an exploration/diversification activity inside the population. Second, there is no predefined strict order of agent activation and interaction leading to some uncontrolled process of the collective.

14.3.4 Decentralised Self-organising Multi-Agent Solving Considering the aforementioned methods, this section presents a synthesis of their characteristics and limitations as to understand why defining explicit self-organising cooperation mechanisms can provide more robust solvers.

14.3.4.1 Self-organising Multi-Agent Viewpoints Analysis Table 14.1 sums up the properties of the overviewed methods with respect to our viewpoints. We can consider that all the methods are really distributed and can easily be deployed in a network. These methods all lie on the distribution of the solving process among parts (ants, agents, particles or genes). Nevertheless, only one method can be considered as decentralised, DBA. In fact, DBA is the only algorithm that does not centralise some process or some data within one part of the system, even if there exist some global parameters that every agent knows (for example the time limit). For instance, the global order in ABT centralises the consistency of the solution within the highest priority agent. In ERA or ACO, some valuable information is stored within the environment. In an artificial context, this environment is

14

Self-organisation in Constraint Problem Solving

365

represented as an entity that therefore centralises information. In PSO and GA, the best solution and the fitness which is computed at each step is known from all the particles/genes. In GA, the scheduler that drives the algorithm step by step knows all the agents and processes the different mechanisms such as mutation and crossover. Nevertheless, not all the methods ensure the principle of locality (see Sect. 14.2.1), and furthermore they may require information on the global state of the system. In fact, higher-priority agents in ABT know the values of the lower-priority agents, even if they do not share constraints. However, in AWCS or DBA, agents only communicate with their “constraint” neighbours, but with a loss in efficiency in finding that a problem has no solution. In ERA, variable agents mark their domain and can check conflicts with variables involved in shared constraints as well. In ACO, virtual ants are localised within an environment which ensures locality. In PSO and GA, all the parts know a global piece of information—the global optimum—at each step to change direction or to choose next generation. Since many methods are based on local search, they are easily usable in a dynamics context, but with some limitations. For instance, DBA lies on correct information, and if an agent disappears or changes its value at runtime without informing neighbours (due to communication failure), the process will fail. For other approaches based on stochastic search (ERA, ACO, PSO, GA), a failure or a change will be forgotten or avoided based on random choices. Therefore, all methods do not lie on randomness in the same manner. For instance, in ABT or DBA, the only randomness that appears is the order of agent execution. It can impact the time to solve but not the way to cooperatively explore the solution space. In ERA, randomness is used to choose the behaviours at each step and is the way the ensure diversification of the search. In ACO or PSO, randomness is used by ants to choose directions. Finally, in GA, randomness is used to choose gene for mutation, crossover and next generation—which makes GA the most stochastic method. Finally, we can claim that all the presented approaches possess some interesting characteristics for designing self-organising systems. Furthermore, each method lies on a set on interacting entities that cooperate as to solve a common (for example PSO) or a distributed problem (for example ABT or ERA). However, they do not handle all the intrinsic properties of distributed and dynamic problems and environments. Therefore, in the remainder of this chapter we present a purely selforganising approach, based on the cooperation mechanism and Adaptive MultiAgent Systems (see Chap. 9), and inspired by aforementioned models, as to provide solvers more robust to dynamics and sustainable for large-scale problems.

14.3.4.2 Cooperation to Solve CSP The seminal work of Axelrod [1] produced a great interest about cooperating mechanisms in social and computer sciences. He initiates a lot of works in problem solving, and a general definition is expressed in [5] as “Cooperation involves a collection of agents that interact by communicating information to each other while solving a problem”, which fits well with the general statement of MAS. Further works on

366

P. Glize and G. Picard

cooperative search consider cooperation [3, 7, 32, 33] as a way for increasing the efficiency of parallel heuristic methods; which was early stated by Hogg and Huberman [13]. In distributed and local search algorithms for DisCSP (ABT, AWCS, ERA and DBA), it is assumed that agents only reason on a neighbourhood, even if sometimes it may represent the whole society (as in ABT, for instance). However, we can identify the following cooperation mechanisms: • The nogoods (conflictual configurations) and potential solutions communicated by agents to their neighbourhood in ABT or AWCS help agents to cooperatively solve a DisCSP. • The min-conflict heuristic used in AWCS or ERA is a mean to represent the fact that agents cooperatively act by minimising the negative impact of their actions. In population-based algorithms each individual is able to find alone the global solution. Nevertheless information from other individuals could be very useful for itself and reciprocally: • The pheromone deposited by ants in ACO gives relevant information about the region of the search space and modifies later the behaviour of the other ants. • The particle in PSO is influenced by the velocity and position of the local and global bests: this is the cooperative information exchange allowing efficient exploration phase. • The function fitness of GA determines at a time the better individuals which will share their genes with other members of the population to produce new relevant offspring. This can be seen as an altruistic behaviour because they do not have anything in turn, except sharing genes later with the new betters. According to these works, some sort of cooperative behaviour seems required to achieve an efficient result between concurrent methods, “but is unknown how to model the impact of correlated interactions on the performance of cooperating search heuristics, consequently the design of cooperative search algorithms remains mostly intuitive and highly empirical” [29]. This is a reason we developed a meta-heuristic method only based on cooperative decisions called AMAS (see also Chap. 9, Cooperation). By assuming that cooperation can be viewed as a generic concept manipulated by problem solvers, since it transcends to all the aforementioned methods, we propose the following algorithm. This algorithm derives from the study of problem solving in multi-agent systems as presented in the previous sections. It states that an agent is unable to find alone the global solution and consequently it has to interact locally with its neighbours (a significant subset of the MAS cardinality) in order to find its current actions able to reach its individual goals and help its neighbours (ν). The basic algorithm is Algorithm 14.8. The parameters to be settled are mainly the critical level (or criticality) κ (usually initialised to zero), the list of neighbours for the agents ν(xi ) and the termination conditions. κ expresses the individual difficulty an agent has to satisfy its own objectives, which could be roughly given, for example, by the portion of non-satisfied

14

Self-organisation in Constraint Problem Solving

367

foreach agent i do set an initial assignment to xi xi .κ ← 0 end while termination conditions not met do concurrently order the possible solutions according to their κ value xworst ← argmax {xj .κ | xj ∈ ν(xi )} assign cooperative value to xi such as xworst .κ decreases compute xi .κ send xi .κ to neighbours in ν(xi ) end Algorithm 14.8: An algorithm outline based on critical level (κ) in AMAS

constraints (we will see another concrete example in the next section). The agent’s neighbourhood is the set of other agents sharing constraints, as in DBA, AWCS or ERA. In the usual approach, an agent manages only one variable, even several ones can be associated to it, but it can be extended to multi-CSP context as in classical DisCSP algorithms [35]. For each feasible solution s, the agent computes its κ value according to the information received from its neighbours, noted κ(s | ν(xi )). These solutions are then ordered using argmin{κ(s | ν(xi ))}, and the variable assignment corresponds to the element with the minimum κ. For a continuous variable domain, an agent manages an increment value δ which is itself adjusted: increased when the direction variation is maintained between two steps, or decreased. At the last step, the agent sends its own κ value to the neighbours in order to choose their next own assignment in the most cooperative way. The cooperative behaviour leads therefore to two situations: 1. If the agent has the greatest κ in its neighbourhood, it chooses an assignment reducing its value without becoming higher than the current greatest κ. 2. If not, it chooses an assignment able to reduce the greatest κ in its neighbourhood, without becoming higher than the current greatest κ. Because no agent has knowledge about the global problem, the termination condition (usually the max number steps or the solution quality, as in ERA, DBA, PSO or GA) must be evaluated at the macro level. Each agent notifies to this global evaluator when its ending condition is satisfied. This approach was successfully applied in many domains such as timetabling [27], n queens [26], manufacturing control [4] or frequency assignment [28]. This algorithm pre-supposes the capability to have a global scale to evaluate the critical level of each agent, which cannot be necessarily an easy work. Often it is based on an aggregation of criteria, such as the number of conflicts (as in the case of the min-conflict heuristic for instance), the elapsed time with no solution and more classical metrics of CSP and graph theory such as the number of constraints in the problem, the connectivity, etc. These criteria are evaluated using a weighted sum or product, or a leximin or a leximax order, for instance. AMAS does not add new dependencies or links between agents to the constraints defined by the problem itself, leading to a complete distribution and decentralisa-

368

P. Glize and G. Picard

tion approach. As indicated by Algorithm 14.8, the decision procedure is only based on local information obtained from the neighbourhood. The indeterminism is given by the complete asynchrony of the agent’s execution. Another particular—and extremely rare—case of non-determinism comes from the indecision occurring for two assignations having exactly the same critical level at a time: the agent can choose randomly between them.

14.4 Applications 14.4.1 A Sample Application to the n-Queens Problem To illustrate the cooperative algorithm (Algorithm 14.8) presented before, we will discuss its utilisation for a classical combinatorial problem: the n-queens problem. This problem is classically solved using centralised complete or local search-based methods based on heuristics like the min-conflict one [23]. The problem is now to find a solution in which queens are agents that must cooperate to find a solution (no queen can attack another one) in a decentralised manner (without central control or scheduler).

14.4.1.1 Multi-Agent Model One way tho model the n-queens problem with agents is to represent each queen as an agent having to find a cell in the chessboard without being able to attack another queen. The model we propose here is both inspired by DisCSP solvers and stigmergic approaches like ACO or ERA, since queens will be able to mark the chess board with some information about conflicts, and utilize a critical level measure. The basic agent behaviour derives from local capabilities. An agent (qi ∈ A) can perceive all attackable cells (pCells(qi )) and the cell it occupies (cell(qi )). Concerning actions, an agent can move on a perceived free cell (moveTo(cj )), reserve (or mark) the cells to inform other agents and identify conflicts (mark(cj ) or mark({cj , . . . , ck })), and simply rest on the current cell (rest). The markers affect the common environment—the grid—and not the MAS which is composed of distributed autonomous agents. These marks are deleted once the agent moves. To decide which next cell to occupy, agents must be able to perceive other agents (pAgents(qi ) with qj ∈ pAgents(qi ) ≡ (∃k((ck ∈ pCells(qi )) ∧ (ck = cell(qj ))))). Moreover, when reserving a cell, an agent puts extra-data about the less conflictual cell it perceives, and therefore an agent can know the number of agents which can potentially attack a given cell (cost(cj ) or cost(qi ))4 and their constraint degrees. During its lifetime, an agent follows the behaviour presented in Algorithm 14.9. This algorithm represents a nominal behaviour which only leads the agent to a local i ) ≡ (cost(cj )

4 cost(q

with (cj = cell(qi ))).

14

Self-organisation in Constraint Problem Solving

369

while alive do updatePerceptions(); // removing previous marks and adding new ones if (cost(qi ) == 0) then not occupying a conflictual cell rest() else moveTo(minConflictCell(qi )) mark(pCells(qi )) end end Algorithm 14.9: A basic behaviour for an agent (qi )

minimum without taking care of neither the global state of the system nor the other agents. In order to avoid local minimum resting, this behaviour must be enriched by cooperation rules.

14.4.1.2 Cooperation Model The first way to inject cooperation into the basic agents’ behaviour is to define exception rules. This may also imply defining a critical level κ to sort multiple actions. In the n-queens problem, considering the given perceptions and actions, one exception can be identified, a conflict between two agents that can respectively attack themselves. Two different solutions are presented in Algorithm 14.10. The first one is called less-altruistic-as-possible behaviour and depends on the other agents. Here, agents only act if they are less constrained than their neighbours (current perceived agents). In other words, if an agent is more constrained than another one, it will wait until the other moves. As the other agent respects the same rules, it will detect this situation and then will move. The second possible action is performed if an agent detects is more constrained than its neighbourhood. Here, agents are cooperative because they are less-selfish-as-possible. In the two cases, agents must all respect the same cooperation rule, and if moving is necessary, they will move to the less conflictual cell (lessCriticalCell). This cell is determined using a critical level measure. This measure must take into account the constraint degree of the other agents. The idea is to limit the impact of a movement by analysing the worst constrained agents that see a given cell, so as to choose the cell that does not increase the cost for an agent. Definition 14.1 Let P qi be the set of cells perceived by agent qi : P qi ≡ pCells(qi ) ∪ cell(qi ) . Definition 14.2 Let the cooperation measure coop : {c1 , . . . , cn } → N , where coop is defined as coop(ci ) ≡

max

q∈{q |ci ∈P q }

cost(q).

370

P. Glize and G. Picard

// (14.10(i)) less-altruistic-as-possible behaviour: if ∃j ((j = i) ∧ (qj ∈ pAgents(qi ))) then if (qi = qj ) ∨ (cost(ci ) > cost(qj )) then rest else moveTo(lessCriticalCell(qi )) end end // (14.10(ii)) less-selfish-as-possible behaviour: if ∃j ((j = i) ∧ (qj ∈ pAgents(qi ))) then if (qi = qj ) ∨ (cost(ci ) < cost(qj ) then rest else moveTo(lessCriticalCell(qi )) end end Algorithm 14.10: Examples of cooperative behaviours

q

i the set of cells with a minimum cost from the point of Definition 14.3 Let Cmin view of an agent qi : qi ≡ c ∈ P qi |c = c, cost(c ) < cost(c) . Cmin

q

i Definition 14.4 Let Ccoop the set of most cooperative cells from the point of view of an agent qi : qi qi qi ≡ c ∈ Cmin |c ∈ Cmin , c = c, coop(c ) ≤ coop(c) . Ccoop

Therefore, the set of the less critical cells to choose is the set of cells with a minimum cost and a minimum impact for the other agents. These two measures are thus aggregated within the critical level, as presented in Sect. 14.3.4.2. One can qi remark that the contents of Ccoop cannot be empty since it contains at least the current cell (see Definition 14.1). We can also notice that the data an agent needs for evaluating a cell is the following: the number of markers on the cell, and the cost of each agent having marked the cell (see Definitions 14.2 and 14.3). Therefore, a marker for a cell only contains the current cost of its owner. Cooperative agents are then able to determine the set of cooperative cells to move to. But, how can the next cell be chosen without leading to a local minimum? Here is the main decisional challenge for a cooperative agent. By now, AMAS theory does not give any guidance. Therefore, any kind of method can be considered: random, tabu, etc. In the next examples, good results are obtained with a very simple selection criterion. The three main choices before encoding agents’ behaviours and launching the solving process are the following:

14

Self-organisation in Constraint Problem Solving

371

Fig. 14.1 Solving trace for four queens, with a less-altruistic-as-possible cooperative behaviour, in 4 steps and 6 moves

Fig. 14.2 Solving trace for four queens, with a less-selfish-as-possible cooperative behaviour, in 8 steps and 19 moves

• The initial positioning: all the agents are initially positioned at the left border of the grid (see Figs. 14.1 and 14.2). • The selection function (lessCriticalCell(qi )) for choosing the most cooperative cell: since implementation implies to choose an order in perceived cells (from closest to farthest, from east to north east, clockwise), this order is used to choose the next cell. Moreover, a limited memory of one cell is added to avoid the previous visited cell (very simple tabu implementation). • The only termination criterion is time: agents always try to find a better solution until a deadline is reached. 14.4.1.3 Sample Execution At the beginning of the solving process, all the agents are positioned on the left side, and the environment is not marked. Numbers in cells represents the numbers of markers.

372

P. Glize and G. Picard

Fig. 14.3 Global cost in terms of solving steps for 512 less-altruistic-as-possible cooperative agents with random disturbances

Figure 14.1 shows a trace for the 4-queens problem solving. The system finds a collective solution in 4 steps (during which every agent acts), which are delimited by dashed rectangles. Only 6 moves are performed to reach a solution. Some agents do not act during some steps, since they are in cooperative situations, and are not situated on conflictual cells. Contrary to classical CSP solving methods with global knowledge, the agents move to different lines, as shown in the figure when q3 moves to the position (1, 3). Even if it is not a solution, it represents an intermediary stage to a more adequate state. Figure 14.2 shows the solving trace for a less-selfish-as-possible cooperative behaviour. The system reach a collective solution in 8 steps (twice than less-selfishas-possible cooperative agents) and 19 moves. This is due to the fact that agents prefer moving when they detect conflict rather than resting until another one moves. However, experiments on larger problems shows that the two behaviours tend to be equivalent with size of the problem growing [26].

14.4.1.4 Adaptation This chapter mainly focuses on the need to provide solutions to dynamic problems which require adaptivity and robustness. To show how relevant the presented approach tackles dynamics, we can look at a set of experiments that has been performed with the same experimental setup than before, but with 512 agents and random disturbances: each 10 steps, 10% of randomly chosen agents are moved to random positions in the grid. Figure 14.3 shows the global cost ( q∈A cost(q)) in terms of steps. The collective, after every disturbance, quickly repairs the organisation and finds a solution: the system is adaptive and robust to environmental disturbances. Moreover, these results show that a solution is found with a random initial agent positioning. Finally, the first steps of the solving process (see Fig. 14.1) are also a particular case of perturbation since the constraint degrees associated to cells are erroneous. Experiments, with less-selfish-as-possible cooperative agents and the same setup, shows similar results. Here again, the result of the solving process is a spatial organisation of the system. Since this organisation is found without any global knowledge, such as agent

14

Self-organisation in Constraint Problem Solving

373

positions and constraint degrees, it is a self-organising process, within which cooperation guarantees the evolution towards a collective solution.

14.4.2 Problems Requiring Self-organisation In many contemporary applications systems have to face openness, when new stakeholders appear or disappear, or perturbations when in a large distributed environment, like the Internet, possibilities and sources of failures are numerous and unpredictable. Moreover, with more and more users implied in the decision process, possibilities of parameter modifications increase too. Therefore, we can identify self-organising solvers as being relevant for peer-to-peer applications, for ubiquitous computing in which several devices produce a common global behaviour in a physically distributed environment, for socio-technical application, like in ad hoc networks, ambient intelligence or home automation, in which humans interact with ambient systems in a multi-modal manner. These systems often require selfoptimisation for providing high quality of service to users and are representative of this class of problems where the components themselves are mobile and consequently the organisation is not the result but the problem. More precisely, each component has to decide its own right behaviour (including information exchange with its neighbours) according to its contextual perceptions of the problem. There is no explicit global function to optimise, but only local and heterogeneous ones. In these situations the global solution is really emergent in the sense that it cannot be evaluated by a unique entity. Some environmental constraints may limit the capabilities of devices (for example in ambient intelligence or mobile) like PDA, cellular phones or netbooks or typically concerned by swarms of interrelated electronic components. This is the case of ubiquitous computing or (pervasive computing) in which small, cheap and inter-communicating processing devices have been thoroughly integrated into everyday objects and activities. These limitations may concern communication capabilities due to limited energy or can be non-functional requirements coming from the end-users (for example, a hospital may limit wireless signal in some areas). Such limitations may encourage developers to design self-organising collective of devices rather than centralising control and coordination within few entities with large computation power. For instance, multi-satellite scheduling aims to generate operations to each satellite considering a set of constraints such as satellite capabilities, ground station configuration, satellite priorities and payload operations. This problem is quite dynamic according to new tasks to schedule, deadline modification and atmospheric conditions. Satellites can reconsider their works assignment without central control of a ground station. This is a true self-organising process because satellites have to negotiate frequently, and the result is a new operations assignment. We found a similar self-organising process in manufacturing if we consider breakdowns, machining changes, deadline and others unexpected events. The timetabling schedule provides information about a series of dependent events: in particular, the

374

P. Glize and G. Picard

time at which it is planned these events will take place. When changes about problem description occur, the events must be reorganised. In this class of problems, where timetabling manufacturing control or multi-satellite scheduling are examples, the organisation of the agents is an image of the solution, and when the problem description is changing, the organisation must self-adapt. More generally, self-organising solvers are more relevant for CSP displaying specific characteristics like bounded vertex degrees (for instance limited neighbourhoods) or large constraint graph diameter (for instance numerous neighbourhoods in the same system). Good examples of this kind of problems are frequency affectation or information routing problems. The n-queens problem presented in Sect. 14.4.1 is a particular case of affectation problem, and the proposed approach can be generalised as expounded in [26].

14.5 Conclusion In this chapter, we presented a set of approaches to solve combinatorial problems such CSP focusing on distributed and decentralised algorithms and multi-agents systems. For fifteen years, the agent terminology is used in this domain because it seems quite natural to consider a local process trying to assign values to variables according a set of constraints with its neighbours as an autonomous entity (such as an agent). Consequently, MAS technologies are suitable for problems where selforganisation is appropriate, for instance those involving high dynamical environment, limited computation capacities of the agents having to solve a large problem, limited size or umber of communications. However, centralised solutions suit better if the problem to solve is static, not distributed, and thus solvable by an optimised algorithm. Moreover, we have also analysed algorithms from the cooperation point of view: an agent has a sub-problem to solve (and it must cooperate with the others in order to find the global solution), or it knows the entire problem (and it must cooperate to other for exploring efficiently and collectively the search space). As a synthesis of the utilisation of self-organisation for problem solving in dynamic and open environments, we present the AMAS algorithm where cooperation is considered as the engine for self-organising the collective and solving efficiently the problem.

14.6 Problems–Exercises Exercise 14.1 How do ABT and DBA behave if an OK? or a nogood message is lost? Exercise 14.2 List all the parameters that should be initialised before initiating a solving process for the overviewed methods? Classify the methods from the less problem-dependent to the most problem-dependent.

14

Self-organisation in Constraint Problem Solving

375

Exercise 14.3 Propose a solution to add dynamics handling, such as adding new constraints, new agents or new domains in ABT. List the advantages and weaknesses of your solution. Exercise 14.4 How should the behaviour of agents solving the n-queens problem be modified to solve the (n2 /2)-knights problem (positioning n2 /2 knights on a board of size n)? In the next exercises, we will consider the k-colouring problem5 (or kCP), which concerns finding the colour of a number of regions so that neighbouring regions have not the same colour. This problem can be represented as a constraint graph, with vertices representing regions (and their value/colour) and edges representing constraints between regions (in this case, constraints are only differences of colours between adjacent regions). Exercise 14.5 Propose a model for solving the kCP using one of the DisCSP approaches (ABT, AWCS, ERA or DBA). Exercise 14.6 Propose a model for solving the kCP using one of the populationbased approaches (ACO, PSO or GA). Exercise 14.7 What are the main differences between the models of Exercises 14.5 and 14.6? What types of constraints are handled efficiently by each of these approaches? Exercise 14.8 What happens in each algorithm when the problem parameters are dynamically modified, for example the number of nodes, vertices and colours? Exercise 14.9 Propose a cooperative self-organisation model based on either Exercise 14.5 and 14.6.

14.7 Further Reading For a complete view of the classical DisCSP algorithms (ABT, AWCS, DBA), we encourage the reader to have a look at the Yokoo’s book [35]. For an overview of the general CSP framework and formalisation, we point the reader to the Rina Dechter’s book [6]. For implementing distributed algorithms and adding termination procedure to the previous algorithms, the Nancy A. Lynch’s book is a great source of solutions [19]. For more details on the social criteria and collective problem solving from the game-theoretic point of view, we advise the reader to consult the book from Yoav Shoham and Kevin Leyton-Brown [30]. Finally, the following book gives a lot of details and examples of meta-heuristics such as ACO and GA [9]. 5 A detailed description of the colouring problem can be found at http://www.cs.ualberta.ca/~joe/ Coloring/index.html.

376

P. Glize and G. Picard

References 1. Axelrod, R.: The Evolution of Cooperation. Basic Books, New York (1984) 2. Bacchus, F., van Beek, P.: On the conversion between non-binary and binary constraint satisfaction problems. In: Proceedings of the 15th National Conference on Artificial Intelligence (AAAI-98) and of the 10th Conference on Innovative Applications of Artificial Intelligence (IAAI-98), pp. 311–318. AAAI Press, Menlo Park (1998) 3. Bachelet, V., Talbi, E.: Cosearch: a co-evolutionary metaheuristics. In: Proceedings of Congress on Evolutionary Computation (CEC’00), pp. 1550–1557 (2000) 4. Clair, G., Gleizes, M.P., Kaddoum, E., Picard, G.: Self-regulation in self-organising multiagent systems for adaptive and intelligent manufacturing control. In: Second IEEE International Conference on Self-Adaption and Self-Organization (SASO 2008), Venice, Italy, 20–24 October 2008. IEEE Computer Society, Los Alamitos (2008) 5. Clearwater, S.H., Hogg, T., Huberman, B.A.: Cooperative problem solving. In: Computation: The Micro and Macro View, pp. 33–70. World Scientific, Singapore (1992) 6. Dechter, R.: Constraint Processing. Morgan Kaufmann, San Mateo (2003) 7. Denzinger, J., Offerman, T.: On cooperation between evolutionary algorithms and other search paradigms. In: Congress on Evolutionary Computation (CEC’99), pp. 2317–2324 (1999) 8. Dorigo, M., Stützle, T.: Ant Colony Optimization. MIT Press, Cambridge (2004) 9. Dréo, J., Pétrowski, A., Siarry, P., Taillard, E.: Metaheuristics for Hard Optimization: Methods and Case Studies. Springer, Berlin (2005) 10. Glover, F., Laguna, M.: Tabu Search. Kluwer Academic, Dordrecht (1997) 11. Hattori, H., Ito, T.: A quick adaptation method for constraint satisfaction in a real-time environment. Int. J. Comput. Sci. Netw. Secur. 6(78), 107–112 (2006) 12. Hirayama, K., Yokoo, M.: The distributed breakout algorithms. Artif. Intell. 161(1–2), 89–115 (2005) 13. Hogg, T., Huberman, B.: Better than the best: the power of cooperation. In: Lectures in Complex Systems. SFI Studies in the Sciences of Complexity, vol. V, pp. 165–184. AddisonWesley, Reading (1993) 14. Holland, J.H.: Adaptation in Natural and Artificial Systems. MIT Press, Cambridge (1975). 2nd edn. (1992) 15. Kennedy, J., Eberhart, R.C.: Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks, Piscataway, NJ, pp. 1942–1948 (1995) 16. Kennedy, J., Eberhart, R.C.: Swarm Intelligence. Morgan Kaufmann, San Mateo (2001) 17. Kirkpatrick, S., Gellat, C., Vecchi, M.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983) 18. Liu, J., Jing, H., Tang, Y.Y.: Multi-agent oriented constraint satisfaction. Artif. Intell. 136(1), 101–144 (2002) 19. Lynch, N.: Distributed Algorithms. Morgan Kaufmann, San Mateo (1997) 20. Mailler, R.: Comparing two approaches to dynamic, distributed constraint satisfaction. In: Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS’05), pp. 1049–1056. ACM, New York (2005) 21. Mailler, R., Lesser, V.: Solving distributed constraint optimization problems using cooperative mediation. In: Proceedings of Third International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS’04), pp. 438–445. IEEE Computer Society, Los Alamitos (2004) 22. Mailler, R., Lesser, V.R.: Asynchronous partial overlay: a new algorithm for solving distributed constraint satisfaction problems. J. Artif. Intell. Res. 25, 529–576 (2006) 23. Minton, S., Johnston, M., Philips, A., Laird, P.: Minimizing conflicts: a heuristic repair method for constraint satisfaction and scheduling problems. Constr. Bas. Reason. 58(1–3), 161–205 (1994) 24. Modi, P.J., Shen, W., Tambe, M., Yokoo, M.: ADOPT: asynchronous distributed constraint optimization with quality guarantees. Artif. Intell. 161(2), 149–180 (2005) 25. Morris, P.: The breakout method for escaping from local minima. In: AAAI, pp. 40–45 (1993)

14

Self-organisation in Constraint Problem Solving

377

26. Picard, G., Glize, P.: Model and analysis of local decision based on cooperative selforganization for problem solving. Multiagent Grid Syst. 2(3), 253–265 (2006) 27. Picard, G., Bernon, C., Gleizes, M.P.: ETTO: emergent timetabling by cooperative selforganization. In: Engineering Self-Organizing Applications—Third International Workshop (ESOA) at the Fourth International Joint Conference on Autonomous Agents and MultiAgents Systems (AAMAS’05), July 2005, Utrecht, Netherlands. Lecture Notes in Artificial Intelligence (LNAI), vol. 3910, pp. 31–45. Springer, Berlin (2005) 28. Picard, G., Gleizes, M.P., Glize, P.: Distributed frequency assignment using cooperative selforganization. In: First International Conference on Self-Adaptive and Self-Organizing Systems, 2007. SASO’07, pp. 183–192. ACM, New York (2007) 29. Prakasam, P., Toulouse, M., Crainic, T.G., Qu, R.: A multilevel cooperative search algorithm for the graph colouring problem. In: Learning and Intelligent Optimization Conference (LION 3) (2009) 30. Shoham, Y., Leyton-Brown, K.: Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, Cambridge (2008) 31. Talbi, E.G. (ed.): Parallel Combinatorial Optimization. Wiley, New York (2006) 32. Toulouse, M., Crainic, T., Sansò, B.: An experimental study of the systemic behavior of cooperative search algorithms. In: Voß, S., Martello, S., Osman, I., Roucairol, C. (eds.) MetaHeuristics: Advances and Trends in Local Search Paradigms for Optimization, pp. 373–392. Kluwer Academic, Dordrecht (1999) 33. Toulouse, M., Thulasiraman, K., Glover, F.: Multi-level cooperative search: a new paradigm for combinatorial optimization and application to graph partitioning. In: Proceedings of the 5th International Euro-Par Conference on Parallel Processing. LNCS, pp. 533–542. Springer, Berlin (1999) 34. Yokoo, M.: Weak-commitment search for solving constraint satisfaction problems. In: Proceedings of the 12th National Conference on Artificial Intelligence (AAAI-94), pp. 313–318. AAAI Press, Menlo Park (1994) 35. Yokoo, M.: Distributed Constraint Satisfaction: Foundations of Cooperation in Multi-Agent Systems. Springer, Berlin (2001)

Chapter 15

Adaptive Trust Management Jean-Marc Seigneur, Gabriele Lenzini, and Bob Hulsebosch

The world constantly changes. How trust management can adapt to these changes?

Objectives After reading this chapter the reader will: • Be aware of context-aware trust management concepts and relevant issues; • Understand the meaning of the term “adaptive trust management”; • Understand the practical issues of applying adaptive computational trust in authentication.

15.1 Theoretical Notions Research in context-aware applications often emphasises the dynamic short-term situational aspect of context. Dey and Abowd [9] proposed a definition of context which has been widely accepted in the research community: J.-M. Seigneur () University of Geneva, Geneva, Switzerland e-mail: [email protected] G. Lenzini University of Luxembourg, Luxembourg-Kirchberg, Luxembourg e-mail: [email protected] B. Hulsebosch Telematica Instituut, Enschede, The Netherlands e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_15, © Springer-Verlag Berlin Heidelberg 2011

379

380

J.-M. Seigneur et al.

. . . context is any information that can be used to characterise the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and the applications themselves.

Dimmock, who took care of the risk module in the SECURE project [36], concludes in his Ph.D. thesis that more work concerning situational risk must be done, especially concerning the time element of risk since it has not been addressed by the SECURE framework [10]. Regarding the trust context, most trust engines currently consider trustworthiness within a particular application domain. Examples include examining trustworthiness in writing good security books, in recommending good doctors [30] and in characterising virtual identities of requesters/candidates. Apparently a type of trust originating from social research, described as: situational decision to trust [. . . , which means that the trust engine’s owner] has formed an intention to trust every time a particular situation arises . . .

has been overlooked, even if it can be considered as imbricate with dispositional trust. Guha [13] argues that his work is limited in respect with time: These [content rating systems] are dynamic systems which can change quite rapidly. Understanding the time-dependent properties of such systems and exploiting these properties is another potentially useful line of inquiry.

In the TriQL.P trust engine, although the policy language allows programmers to specify time-dependent policies, no trust metric covering the notion of time is supported [4]. Furthermore, Marsh [24] similarly underlines the role of time as being relevant to all variables used in his trust model, but again no specific time-dependent trust metric is provided. The method applied in the SECURE trust engine for computing trust values utilises a number of parameters, in addition to the identity context, but no specific time-sensitive trust metric implementation is produced. From a time perspective, most previous trust metrics consider that a trust value is updated only when a user manually resets trust values in an entity [12], or as a result of interactions carried out and recommendations received by that entity. The few trust metrics that take time into consideration simply propose that trust values decay over time, even if there is no interaction. For example, Mezzetti [25] considers that trust values decay over time at a rate given by a transitive aging function. That function can be used to calculate trust values at any time point tn based on a given trust value at a previous time point tk . Furthermore, in recent Bayesian-based trust metrics [5], trust values are also considered as aging with time and converging towards their initial bootstrapping value if not reinforced. However, the choice of the aging factor is still arbitrary to a large extent. The above approaches can be sufficient for continuous application domains, but they reveal weaknesses when applied to dynamically changing applications, for example to those handling dynamic information that needs to be frequently updated. In these cases information trustworthiness must drop sharply when its validity limit has been reached, deeming gracefully decreasing metrics as unsuitable for that purpose. An approach towards handling trustworthiness in dynamic contexts is to assign timestamps to interactions carried out and trust evidence produced. Trust estimation

15

Adaptive Trust Management

381

methods can then use timestamp information to conduct complex trust analyses involving time. For example, patterns detected in trust evidence updates can be used in subsequent trust value calculations, as is the case in calculations of time-patterned trust metrics. Another approach towards dynamic trust management can be to allow trust engines to use discrete time-based functions, instead of arbitrary continuous decay ones, which would allow more flexibility in representation of trust value variations.

15.1.1 Adaptive Trust for Context-Aware Security Traditional system security mechanisms are pre-configured to have a static behaviour. They are designed to protect systems against certain typologies of threats and cannot be dynamically adapted, for instance, to emerging situational constraints. This static limitation is due to the partial absence of two important features, namely (1) adaptivity and (2) context-awareness.

15.1.1.1 Adaptivity Adaptive secure behaviours are possible only if security services are designed to be flexible and able to cope with a multitude of situations that can arise throughout system operation. An approach to ensure security adaptation is parameterisation [8, 11, 22, 28]. Security parameterisation is based on the ability of identifying security levels for aspects such as authentication, authorisation and data security. Each security level depends on the outcomes of an associated security function. For example, a security function can return, given a set of observable qualities of the system, a real-valued number; a security level can be defined in terms of statistical or absolute thresholds over the range of the security function. Each security level indicates the extent to which security can be guaranteed. In adaptive security, each security level is guarded by level-specific protection mechanisms. Those mechanisms are assumed to protect the system against potential attacks which are known that can occur at that security level. At run-time, a change of security level triggers the activation of the correspondent security mechanisms. A complementary approach to parameterisation consists of adapting the security level by means of structural changes of the system. In that case, adaptive security mechanisms are able to dynamically respond to environmental changes by reconfiguring their security functions accordingly [35]. For example, a system controlling access to business transactions can adapt its security functions (in that context, they are termed “policies”) according to the sensitivity of the access requests and a suspicion level associated with the requesters. When request sensitivity is low, the server can allow customers to purchase goods with weaker proofs of payment, for example using a one-time credit card. This provides better customer protection, but it increases the on-line store financial risk. When transaction sensitivity increases,

382

J.-M. Seigneur et al.

as is the case where, for instance, a customer attempts to make a large purchase, the server requires higher trustworthiness credentials, such as a valid user identity or a sufficiently high user credit rating [32]. Security effectiveness increases when different security levels with respect to various security aspects, such as quality assurance, availability and execution efficiency, are introduced for both users and systems. Furthermore, adaptive security solutions result in high levels of usability; for example, absolute user identity verification can be avoided. Additional advantages of adaptive security approaches include increased realism; for example, user access control mechanisms are fine-grained, and high sensibility to external constraints, for instance power limitations, may influence the choice of encryption algorithms. Finally, adaptive security mechanisms increase system ability to handle unexpected situations, such as emergencies. Related Work Several adaptive security approaches exist in the literature, and a representative number of them is surveyed here. Parameterised authentication is described in detail in Covington et al. [8]. In that work the authors illustrate how parameterisation can ensure functionality in a damaged, compromised or fundamentally inaccurate system. Furthermore, they provide examples of security functions which they term “authentication metrics”. Schneck and Swan have developed Authenticast [35], a dynamically configurable user-level communication protocol which offers variable levels of security and reliability in packet transmission. Ryutov et al. [32] describe a system that adapts its security policies based on the sensitivity of access requests and a suspicion level associated with requesters. In a similar line, Ganger [11] proposes the concept of “authentication confidence” aiming to parameterise security. Authentication confidence refines authentication decisions by allowing systems to grant partial access rights to authenticated users with suspicious identities. Another representative work is that of Noble and Corner [28] who propose a transient authentication model. In that model system users wear small hardware tokens that constantly authenticate them to other devices over short-range wireless links. Finally, Levin et al. [22] propose a “quality of security service” mechanism for modulating and provisioning predictable security service levels.

15.1.1.2 Context-Awareness To address context-aware security, services need the support of a context-aware infrastructure that detects and interprets context information. But what is a context? A precise definition of context depends on the specific application, and it cannot be fully predicted. In traditional security, for example, a security context consists of information concerning the security infrastructure of users and their application environment [27]. In that view, a security context can include, for instance, information concerning user roles, user group memberships and user capabilities [2]. In context-aware security, contextual information can include any type of data that can be useful for improving and customising security solutions. Examples of contextual information are human factors (for example user habits, mental state, social environment and task-related activities), the physical environment (for example

15

Adaptive Trust Management

383

location, network connectivity and battery power), business data (for example goaldirected activities and trust) and time. In that way, contextual information adds new elements to the traditional notion of security context. Context-awareness and adaptivity are closely related because context can include the necessary information for effective security adaptation when needed. Coping with context requires particular attention to user privacy. In fact, privacy concerns often conflict with the requirement for secure authentication, for example when private data such as PIN number, credit card number and other sensitive private information, such as, for instance, user location, need to be revealed. While service providers commonly require registration of user personal data to protect their services from misuse, these same users prefer to avoid releasing personal information when not strictly necessary. In these cases, the use of indirect user information or the use of certain public critical types of contextual data can often be a sufficient alternative. For example, many services require a credit car number as an identity credential despite that no payment is actually involved. Therefore, using user contextual information instead of credit card data can assist in strengthening privacy in these cases. In the last years context-awareness has gained an increasing attention from the computer science research community. In certain cases dedicated projects have supported the effort of universities and industries in designing infrastructures for context-aware applications, as, for instance, is the Dutch programme Freeband AWARENESS [14]. In the AWARENESS project, contextual information that networking environments are able to provide about users is utilised to improve service provisioning in a personalised, secure and privacy-sensitive manner. Related Work A detailed survey of the research carried out on context-awareness is out of scope in this chapter. Therefore, only a small subset is surveyed here. A representative work is that of Marcus [23], who describes the logical foundations of an adaptive security infrastructure concept which takes the environment into account. Kouadri et al. [27] propose a conceptual model for context-based authorisation that offers fine-grained control over access to protected resources. In particular, the control model is based on state and relevant information of users and the environment. Furthermore, the specific problem addressed in that work concentrates on the tradeoffs between security, performance and efficiency of current and proposed security protocols. In addition, the work of Kouadri et al. focuses on various contextual aspects affecting these trade-offs, such as constrained network capacity and limited mobile nodes. In another approach, Yee and Korba [44] propose a context-aware security policy agent that is responsible for selecting security mechanisms for mobile Internet services. The selection is based on user preferences and location, and on mobile device power. Furthermore, Bhatti et al. [3] underline the importance of contextual information for access control and describe an access control scheme, which is an extension of their XML Role Based Access Control (X-RBAC) framework, to incorporate trust domains and (temporal and non-temporal) contextual conditions into access control policies. Finally, Montanari et al. [26] recognise context as an

384

J.-M. Seigneur et al.

important factor in the specification of dynamic and adaptable security policies in ubiquitous environments [39].

15.1.2 Adaptive Trust for Context-Aware Authentication In computer security, access to resources is usually protected by access-control mechanisms that require users to authenticate themselves before granting them permission of use. Authentication is the process of attempting to verify the digital identity of a user, that is to convince the access control engine that the requester has the identity, the role, or the privileges [33, 34], which allow her to use the resource/service according to a particular usage policy. Authentication factors can typically be classified in three categories: 1. who a user is, 2. what the user knows, 3. what the user has. The first category includes all types of biometric measurements, for instance fingerprints, retina scanning and face recognition. Authentication solutions count on the fact that biometric measurements are (or should be) able to identify each individual unambiguously. The second category includes all types of secret information such as PINs, passwords, question/answer pairs that users are expected to keep private. The third category comprises user-names and personal devices, such as RFID badges, smart cards and mobile-phones, that users are assumed to carry with them almost always, and whose presence can be detected by a network of sensors. Current available authentication solutions are not completely reliable (cf. [2]), and hence the expectation of complete trustworthy user identification is unrealistic. Even in a situation where someone enters the right username and password combination, there is still a possibility that the combination has been eavesdropped and that it is currently typed by someone else. The solutions based on biometric identification are attractive in theory but not so accurate in practice. Biometric features are hard to measure up, and there is always a chance for a false positive or a negative result. The detection of personal user devices or identity tokens, although technologically more precise than the biometric counterpart, is also affected by false positive and false negative errors. Moreover, such detection provides only partial pieces of information about the user. For example, a detected Bluetooth device indicates where the device, not the user, is located. Finally, since badges, mobile phones, smart phones, and RFID cards can be stolen, forgotten, or lost, the risk of identity theft is quite elevated in this category. Although user identity cannot be verified with absolute certainty in most cases, its trustworthiness can increase with the adoption of clever strategies. One such strategy, known as “multiple-factor authentication”, is based on combining different authentication methods. A typical example is given by current Internet banking solutions where authentication is the result of combined use of an automated teller

15

Adaptive Trust Management

385

machine, a bankcard and a PIN. Alternative solutions require multiple applications of a particular authentication method. For example, most cash dispensers ask users for their PIN both at login time and before each requested transaction. Generally, the more solutions used to authenticate the user, the stronger the system confidence in the particular user identity. Formally, if L1 , L2 , . . . are confidence values (ranging in the real interval [0, 1]) associated with different authentication methods, then, under the assumption that all authentication methods yielded positive outcomes, the overall confidence L associated with the composite authentication solution can be calculated from the following probability-based formula [31]: L = 1 − (1 − L1 )(1 − L2 ) · · · (1 − Ln ).

(15.1)

As can be seen from (15.1), increasing or decreasing confidence values Li respectively increase or decrease the overall confidence value L of the composite authentication method. For example, if the authentication confidence of an RFID badge is LRFID = 0.80, a Bluetooth (BT) device confidence level of LBT = 0.60 will result in a combined confidence of L = 0.92. A Bluetooth device confidence level of LBT = 0.80 will result in L = 0.96 respectively. Equation (15.1) assumes that credential items are independent, which is a reasonable assumption if they are validated in a common context. Although multiple-factor authentication result in significant improvements as compared to single source authentication, it also introduces new problems. For example, the determination of meaningful authentication confidence values for each authentication technique (that is the values Li in (15.1)) proves difficult, and it is strongly application specific [1]. Moreover, L depends only on those credentials that are actually shown by the requester. Generally, however, the context in which each authentication method validates the corresponding credentials is not the same for all methods. For example, in cases where the network infrastructure is able to perceive which user items are available in the environment, that is in addition to those provided by the user at the authentication point, there is the need to resolve possible inconsistencies. An example of such inconsistency is when the RFID badge and the Bluetooth device of the same user are almost simultaneously used at two completely different locations. An additional example is the case where the time intervals between composite authentication methods for a particular user are longer than an accepted threshold. To answer such questions, it has recently been proposed to analyse the authentication process context, and in particular location and time. Location and time can be used to define a fourth class of authentication factors, namely: 4. where the user is, and when. By exploiting information about the location of a device in time, user authentication can significantly enhance the confidence strength of identification services. For instance, if an RFID reader at a building entrance has identified a user via her RFID badge and, at the same time, a Bluetooth dongle senses the Bluetooth-enabled Personal Device Assistant (PDA) of the same user at the third floor of the same

386

J.-M. Seigneur et al.

building, then granting the user electronic access to confidential files should be restricted. This restriction is motivated by the contradiction of location information of the two identification measures, and it results in lower credibility of the identified user. On the contrary, if the respective locations match, then the confidence in the authenticated user identity should be higher. In other words, in this case the user is implicitly authenticated based on the location of her identity tokens in time. Positional and temporal data can also improve adaptability in authentication. For example, authenticated users that carry accredited items in the proximity of a particular location can be allowed to access resources available at that location without the need of explicit login. Thus, in addition to combined identification inputs, the use of sensor information enables reasoning about the validity of composite authentication information aiming to achieve higher authentication status. Unfortunately, the analysis of multiple contextual data involves additional difficulties. Managing multiple contextual information sources requires appropriate sensor data analysis algorithms (cf. [43]). For example, RFID sensors can be capable of analysing user location within a two-meter radius, while Bluetooth sensors can be able to do location analysis in a ten-meter radius. Such differences must be taken into account when analysing sensor data. Similarly, the analysis should give priority to the relative positions of the sensor scan areas. For example, if the scan areas of an RFID and a Bluetooth sensor overlap only partially, the confidence on the authentication results should be lower than the results obtained in the case of a complete overlap between the two scan areas. Generally speaking, contextual discrepancies can be analysed with a measure of trust that applies to the process of merging the contextual credential items. Contextual information can be modelled in trust-based security as an additional set of information elements, I , that is managed by the system together with the traditional set of signed credentials, C [20]. Data in I are not necessarily accredited, and they are commonly used to improve traditional system security solutions. In context-aware authentication, for example, I can be related to contextual information of the identity items in C, as proposed in [21]. Mobile devices, in particular, are both identity credentials, so it is possible to check if a mobile phone belongs to Bob for instance, sources of location and temporal information, and the mobile phone links to the network cell where it is detected at a certain time. The elements of set I are used to evaluate the trustworthiness of security-related statements concerning user positions. Authentication is then re-evaluated in light of trust which emerges from the given context. In case of low trustworthiness, the user can be asked to provide additional, context-related or context-independent credentials. Following the previous example, if Bob is indeed the accredited requester, by switching on his Bluetooth device at the appropriate login location he may be able to reach the authentication level required to use the requested resources. Furthermore, this action strengthens the confidence in Bob standing at the right location. In addition, Bob can select (or be asked) to provide confidential information, for example his PIN, if the achieved authentication level and trust still do not suffice to grant his full access. Context-aware trust has been proposed as a back-end stage of a primary authentication process [38]. Therefore, if trust(u ∈ l) is a measure of trust in user u that

15

Adaptive Trust Management

387

submits a request from location l, the context-aware overall degree of access is L trust(u ∈ l). L represents the authentication level according to (15.1), and is an appropriate operator. For example, if trust values range in the real interval [0, 1], where 0 and 1 indicate distrust and complete trust respectively, can be implemented as real number multiplication. In case of complete trust the overall authorisation level remains unchanged. In all other cases, it is reduced based on the calculated trust. A solution studied in [15] suggests to combine the authentication confidence L with the probability of the user u being at a given location l at a particular time. Here trust(u ∈ l) is calculated as P (u ∈ l | s1 = 1, . . . , sn = 1), namely the conditional probability that a user stands at cell l given that sensors s1 , . . . , sn (which control cells l1 , . . . , ln respectively) have detected the presence of that user. Here the composite pieces of location information originate from different sensed identity tokens associated to user u. An alternative solution, proposed in [21], uses Subjective Logic [18] to estimate trust(u ∈ l). Intuitively, trust that can be reposed on the statement u ∈ l is evaluated by analysis of different types of location information related to a requester. To evaluate trust(u ∈ l), different sources of location information are considered as recommenders giving an opinion on the statement u ∈ l. Location information is then considered as a special type of recommendation. Data location sources are prepared in the form of opinions that these sensors have about the statement u ∈ l from their local point of view. For instance, if a user identifies himself as Bob using an accredited GPS-enabled device at location l, then a sensor detecting Bob’s badge as being at a cell l , far from the identified login place, distrusts the predicate u ∈ l. Both proposals require support of an infrastructure where contextual information and digital identities are appropriately managed, such as the infrastructure developed in the Dutch Freeband AWARENESS project [14]. Figure 15.1 shows the basic elements of that context-sensitive and adaptive security architecture. The proposed architecture supports authentication and privacy/confidentiality which are contextaware and based on user detection input. Sensor information input is provided by the respective context management infrastructure. For instance, RFID readers, Bluetooth dongles and WLAN access points provide information regarding detected user devices, such as RFID badges, Bluetooth-enabled phones and laptops respectively, and their associated locations. The Context-Aware Trust Evaluation Component of AWARENESS receives the locations of sensed user devices and evaluates their trust(u ∈ l). The Authentication Level Assignment and Access Control components utilise the outcomes of trust evaluation and user identification and assign a specific authentication status to the user. Furthermore, the Privacy Enforcement component ensures user privacy when handling contextually required data.

15.1.3 Recognition-Based Adaptive Trust Management The entification framework proposed in [36] bridges the gap between authentication/identity management and trust management. To increase the level of auto-

388

J.-M. Seigneur et al.

Fig. 15.1 Location detectors such as RFID readers, Bluetooth dongles and WLAN access points provide information regarding detected user devices and associated locations. The Trust Management Component receives the location of sensed user devices and evaluates the trust the system has in the user standing at a certain location. The output of trust evaluation is subsequently communicated to the authentication process that assigns a certain authentication status to the user

configuration in smart homes, the entification trust engine is integrated into the smart appliance access control mechanism to manage interactions between previously unknown users or smart appliances [36]. Instead of having to set up an access control list for individual or groups of entities, such as other users or devices, device owners need only to set up the level of trust required before interactions with newly encountered entities can take place. The trust engine takes care of trust management, namely of the evolution of trust values based on new observations, recommendations and reputations, on behalf of the user. In doing so, the access control at the entity level, which may be overwhelming for home users if done manually, is implicitly managed. In recognition-based trust management, there is still the issue of achieving dynamic, with minimum human intervention, enrolment of not previously known users, including home tenants when they first join the system. A solution can be to utilise a “concierge” process monitoring the system environment [36], capable of recognising various user types and discriminating between acquaintances and unknown users. A general obstacle in this respect is that existing user enrolment methods are not satisfactory for ubiquitous computing because they require significant manual administrative work. This issue contradicts the fundamental “calm technology” ubiquitous computing principle, as stated by Weiser [42], according to which “technology should create calm”.

15

Adaptive Trust Management

389

15.1.3.1 Enrolment in the ER Process To enable dynamic user enrolment, the entification framework introduced in [36] proposes an Entity Recognition (ER) process which supports enrolment of unknown users and devices. Furthermore, an exemplar message-based ER scheme, termed “A Peer Entity Recognition” (APER), is included in that framework, together with a scheme based on vision techniques, termed Vision Entity Recognition (VER) [36]. Generally, a number of different sensing, recognition and retention strategies can be envisaged for entity recognition schemes. In VER, the context at time of retrieval is used to optimise system responsiveness. Depending on the technologies involved, different types of evidence may be analysed to identify the examined entities. Recognition evidence analysis, which we term “detective work”, depends on the recognition scheme used. For example, analysis in APER may consist of sending challenge/response or signature verification messages. Another example is the scheme proposed by Jain et al. [16], in which the term “recognition” encompassing verification/authentication and identification is used. However, that approach does not constitute a full recognition process; for example, it is not applicable in nonbiometric systems. In contrast, the ER process is perfectly applicable to authentication schemes which are not based on human biometrics. An entity in ER can initiate the recognition process to identify potential surrounding entities as needed, a feature termed “self-triggering”. For example, self-triggering may start a recognition scheme involving network monitoring and selectively carrying out detective work on the entities observed. In the ER scheme, clues of past recognitions need not be retained. When an entity needs to refer to a virtual identity, it provides recognition clues for other entities to recognise the virtual identity under scrutiny. Each local computing entity interprets these clues in its own way. For example, in the VER scheme the main clues consist of sequences of images of people passing in front of cameras. In that case, the smart concierge software carries out its own local detective work based on the provided images. To cope with scalability, past information is not maintained after a certain time threshold. For example, entities do not store information of virtual identities with which they have not actively collaborated for a certain time.

15.1.3.2 Discriminative Retention in the ER Process The tremendous number of entities expected in a pervasive computing environment raises the issue of scaling entity recognition to cover billions of entities with potentially different distinguishing characteristics. A common approach to address this issue is to store and later examine past recognition information, with the disadvantage that retrieval becomes inefficient after a certain time. There are also scalable peer-topeer schemes that would allow trustworthy entities to share the load of recognition clues. To take into account that different authentication schemes are relatively difficult to compromise, as, for instance, is the case in Gaia framework [41], a level of confidence can be associated with the authentication scheme used. In this respect,

390

J.-M. Seigneur et al.

differences in strength of recognition schemes obviously raise the issue of trust in the underlying technical infrastructure. Trust in a virtual identity cannot be accurate if the information used at the recognition level is imprecise or invalid. Therefore, technical trust in the infrastructure used must be taken explicitly into account. Furthermore, applications requiring strong security should not be executed with weak ER schemes. Consequently, we can consider a layered view of trust, with the two primary layers being trust in the underlying technical infrastructure and trust in an interacting entity. This chain of trust layers gives rise to an end-to-end overall trust. In [29] it has been reported that: . . . information security measures reside in the physical layer of the trust model and have interaction with the personal layer.

Similarly, Golbeck et al. [12] notice that: . . . a security measure builds trust about the authenticity of data contained in the network but does not describe trust between people in the network.

The overall trust level can be estimated based on the trust values of the abovementioned trust layers. However, whether that overall trust level is acceptable or not is a separate issue. Some benefits of autonomous applications make it worth relying on not-so-trustworthy underlying technologies. There is a trade-off between what can be obtained and what can be lost. That trade-off has to be acknowledged and specified. To get the full potential of autonomous computing, the risks of using notreally-trustworthy environments have to be considered explicitly, as is indeed done in the risk analysis component of trust engines. In this respect, the following generic function can be used to calculate the overall trust value, commonly termed end-toend trust value [36]: End-to-endTrust = f ·(TechnicalTrust, EntityTrust). There are different functions f that can be used to compute the final end-to-end trust value. For example, in Jøsang’s framework [17] the conjunction operator could be used. Another example may be that the two trust values are on a scale between 0 and 1, where trust may be interpreted as the probability that an entity behaves in the expected manner for the intended purpose. Assuming that these trust values are independent, their multiplication would limit the overall trust value. Beyond a simple level of confidence in recognition, manually set by an expert, the recognition scheme can be associated with a technical trust value, which can be based on direct observations and recommendations. The recognition scheme is seen as an entity whose trust value varies dynamically interaction after interaction, which is an improvement compared to static confidence values. In the remainder of this chapter, the underlying technical infrastructure is abstracted to the technical trust of the recognition scheme. However, recognition is only a small part of the underlying technical infrastructure. Other technical elements could be considered, such as secure communication over networks after authentication. In fact, Jøsang, in his metric for public keys in a web of trust [17] did not consider technical trust at the recognition scheme level, such as considering differences between key sizes. Instead, he focused on the link between real-world identities and public keys corresponding to virtual identities. To summarise, with respect to computation of end-to-end trust the ER process includes four steps:

15

Adaptive Trust Management

391

1. Triggering of the recognition mechanism. 2. Detective work to recognise an entity using the available recognition scheme(s); this provides the level of confidence, or technical trust, in recognition. 3. Discriminative retention of information relevant for possible recall or improved future recognition; this is the equivalent of enrolment and constitutes the first main difference from authentication. 4. Upper-level action based on the outcome of recognition, which includes technical trust; this constitutes the second main difference from authentication. False alarms can be sources of problems, especially for system administrators that lack the necessary administration skills, as is, for instance, the case for smart home tenants. Adaptability can make life of users easier in this respect. Therefore, assuming that the ER module can be used in a plethora of contexts, for example based on different types of hardware, and considering the lack of skilled administrators and the requirement for minimal user intervention, an autonomic model for the ER module becomes mandatory [7].

15.1.3.3 Autonomic Adaptive ER Module The most common architectural pattern of autonomic elements comprises a management unit and a functional unit [7]. When we apply this pattern to our ER process, its four steps (namely triggering, detective work, discriminative retention and upper-level action) become parts of the functional unit, as depicted in Fig. 15.2. The management unit is responsible for monitoring and tuning the ER module. As stated in [7], management unit types can vary significantly: . . . unlike conventional computing systems, which behave as they do simply because they are explicitly programmed that way, the management unit of an autonomic element will often have a wide range of possible strategies . . .

The management unit is open to a broad panel of policies and decision-making mechanisms. In the entification framework proposed in [36] decision-making is naturally based on computational trust. Furthermore, access control of the autonomic element pattern can be enforced by tuning the level of attention with lower attention levels corresponding to executing less recognition rounds. In addition, changes to attention level increase the sensitivity of the ER module and accordingly affect the time spent for entity recognition. This is particularly useful when management of trust computation is required. Moreover, differences in detective work levels imply differences in time and mechanisms applied by the ER module to recognise surrounding entities. Similarly, differences in the level of discriminative retention imply differences in the number of recognition clues that the ER module retains for later recognition. Exceptions during entity recognition should update the environmental context as needed to react to potential attacks, such as denial-of-service due to large numbers of triggerings and trial-and-error attacks over a number of possible observable attributes. Each time a triggering occurs without providing sufficient recognition

Fig. 15.2 Autonomic entity recognition

392 J.-M. Seigneur et al.

15

Adaptive Trust Management

393

clues, the ER module can log an optional piece of evidence summarising the exception. This type of logging is represented by the Monitor loop leading back to the management unit at the bottom of Fig. 15.2. Other optional useful pieces of evidence include the level of confidence in recognition reached, the ER current workload and the in-level of clues (that is, an estimation of the level of clues used by other virtual identities) utilised for reaching that particular confidence level. All these pieces of evidence can be used by the management unit, in addition to external information, to select the most appropriate input level for the ER Module. For example, by knowing that the system is under denial-of-service attack ER process, triggering can be made less sensitive to decrease resources used for recognition. If more security is required, the level of detective work can be increased in spite of spending more resources. A parallel can be drawn between intrusion tolerance [40] and the need of dynamic enrolment provided by the ER module, which is followed by the formation and evolution of entity trust levels. Generally, the door must be open to stranger entities, but if they are detected to misbehave, their level of trust should decrease and prevent them from generating major security failures. In intrusion tolerance, a different attack reaction mechanism is used, but both approaches provide adaptability. In particular, in security intrusion-tolerant architectures [40], it is assumed that security is not perfect and therefore security faults always exist. However, faults are mitigated by the presence of error detection mechanisms, which are triggered when a fault is detected, and error handling mechanisms, whose goal is to avoid failures. For example, an error handling action can be to close a network connection that has been detected to lead to a remote fault source. The final element that the management unit can configure is the level of clues to be exhibited to other entities, which can include selection of the local pseudonym to use. This is further discussed in the applications described in Sect. 15.2 below.

15.2 Applications 15.2.1 Adaptive Trust Management with Vision-Based Entity Recognition Vision is a quite popular mechanism for recognising people in various contexts. Among other things, it has been widely used for authentication based on visual biometrics, such as fingerprint, face and gait recognition [37]. Generally, these techniques are used in controlled environments where enrolment is mandatory, that is persons to be enrolled have their visual biometrics entered in the security system in advance. In home environments, enrolment cannot always involve human intervention, for example from a system administrator. Furthermore, in public environments there is no list of known people to be enrolled since people can move freely between

394

J.-M. Seigneur et al.

Fig. 15.3 Vision-based entity recognition (VER)

places. In this respect, the Vision-based ER (VER) scheme addresses the requirement for smooth dynamic enrolment. For example, the door should not be completely closed to strangers, but instead completely identified strangers might become acquaintances. From an evaluation point of view, the VER scheme provides empirical technical trust estimation of different vision techniques. In addition, it investigates ER forgetfulness and scalability issues. Particular emphasis is given to methods for improving indexing and retrieval of previously recorded imagery based on content and context, such as time and weekday. In this respect, VER combines different image processing and retrieval techniques for recognising people entering and leaving the system space. The VER mechanism is illustrated in Fig. 15.3. The ER process allows recognition of previously observed/encountered entities based on visual recognition clues, that is, imagery. In VER, different vision schemes can be implemented, for example face matching and clothes colour. Each time someone moves in front of the camera, the ER process is initiated. We use the term selftriggering to refer to this mechanism because the system itself takes the initiative and starts the entity recognition process. The different VER steps are presented in Fig. 15.4. In step 2 of the ER process, the Detective Work involves carrying out various visual analyses to obtain a level of confidence for each recognition. Retrieval of previous imagery is based on content and context. Step 3 is closely related to step 2 because Discriminative Retention of recognition must be based on previously stored imagery. A difficult question arising at this step concerns defining whether the person entering the room is new and how the ER scheme can converge to the real number of different persons monitored until that time. In the ER process, there is no mandatory initial enrolment, but instead enrolment occurs at step 3 when recognition information for a new entity is stored for the first time. Step 4 of the ER process concerns further actions to be taken according to what person is recognised. In this respect, context is used to adjust the level of Detective Work. For example, if an unidentified person is detected at 2 am, the concierge module should increase its level of suspicion and perhaps send a warning message to the security guards. In addition, it should increase the level of Detective Work and Discriminative Retention, which may increase the chance to later recognise the potential thief.

Adaptive Trust Management

395

Fig. 15.4 VER process

15

396

J.-M. Seigneur et al.

Castro and Muntz [6] pioneered the use of context for multimedia object retrieval. We apply the concept within our ER process, which enables the concierge module to adapt retrieval and recognition based on context and level of suspicion without the input of an administrator. We especially make use of time and date to index and retrieve imagery. Concerning indexing, the first time the VER scheme is started in a new space, the list of faces and associated visual and temporal attributes is empty. As soon as someone comes in front of the camera, a sequence of faces is extracted from the video. Our proof-of-concept implementation involves storing the time and the day of the week. For each sequence, height and colour information are computed as well. A database is used to store the recognition clues extracted from each face sequence. These recognition clues are indexed at specific rows which are assumed to correspond to different persons. We can then dynamically index the different rows based on context similarity. For example, we can order the rows in decreasing order starting from the row containing images that are most often seen on Monday mornings at around 8 am. Unfortunately, the face template matching process is too expensive to be run in parallel during the capture of the sequence. Thus, for performance reasons, each image sequence is processed for face template matching after the end of the sequence when nobody is moving in front of the camera.

15.2.2 Trust-Enhanced Location-Based Authentication The concepts that underlie the Adaptive Trust approach to Security and Authentication, introduced in Sects. 15.1.1 and 15.1.2, can be illustrated into a scenario that sees an intelligent coffee corner as in Fig. 15.5a. A user approaching the coffee corner identifies herself by means of different identity tokens, such as RFID badges, Bluetooth devices, GPS and WiFi-enabled PDAs. A camera with face recognition capabilities is also present at the corner. The camera can be activated when face recognition feedback is needed. On a wall screen, the user can see the position of her colleagues (see Fig. 15.5b), but this information is available only if the colleagues have expressed, in terms of policy, their approval to be located by that user. Users can also request that their position will be visible only if the identification level of a particular user is above a certain value. As a use case, we imagine Bob standing in front of a public video display. A BT dongle associated to display detects Bob’s BT-enabled mobile phone. This allows Bob to make use of an application that allows Bob to see, on the screen, the location of his colleagues. However, not all locations are shown because Bob has not a sufficiently high identification level. On the screen, Bob sees that he has been authenticated with an accuracy of 50%. The display also shows several additional identification measures that Bob can use to increase his authentication level. One of them is RFID. Bob swipes his RFID badge in front of a reader. Almost immediately, the application authentication meter increases to 90%, and, subsequently, the location information of all Bob’s colleagues appears. When Bob walks away from the terminal, his authentication level drops, and the application logs him off. The

15

Adaptive Trust Management

397

Fig. 15.5 (a) The intelligent coffee corner scenario. Different sensors accept different identity tokens to authenticate user. (b) A screen-shot from the screen of the coffee corner scenario. Depending on the authentication level of the user, the position of a list of colleagues of hers is displayed on the screen. The authentication level is driven by the number of identity tokens that are shown at the coffee corner Fig. 15.6 Overall architecture for context aware adaptive security

next day Bob tries to view the location of his colleagues again. This time, however, he has left his BT phone at his office desk, so he only uses RFID to authenticate himself. The authentication meter does not come higher than 30%, and no location information is shown. Only by typing in a username and password combination Bob is able to see his colleagues locations again. The architecture that supports the intelligent coffee corner scenario is depicted in Fig. 15.6. It is an instance of the general architecture introduced in Fig. 15.1, and it constitutes the infrastructure that allows security services, such as authentication, privacy and confidentiality, to be context aware and adaptive. It allows identity token-based security measures and the biometric measure based on face recognition to be enhanced with context information. The sensor information required by these security applications is provided by the context management infrastructure. RFID readers, BT dongles and WLAN access points provide information regarding

398

J.-M. Seigneur et al.

Fig. 15.7 Message flow for probabilistic and adaptive context-aware authentication

the detection of user devices, such as badges, smart-phones and laptops, and their associated locations. The following paragraphs describe the implementation of two solutions for context-aware authentication in reference to the coffee corner scenario. This first one is based on a probabilistic approach, and the other on a trust-based approach to context-aware authentication.

15.2.2.1 Probabilistic Approach to Context-Aware Authentication Figure 15.7 shows the message flow for the implementation of the context-aware authentication mechanism. For authenticating the user, the service has to register to the User Location Probability Component (ULPC). If sensors s1 and s2 have detected the presence of the user (written s1 = 1 and s2 = 1 respectively) then the ULPC calculates the conditional probability P (u ∈ l | s1 = 1, s2 = 1) that expresses the probability of the user u being in l given that sensors s1 and s2 have detected his presence and l stands for location. It requests the ULPC component to determine the probability that the user is at a certain location. This probability is calculated considering the following factors: (a) Which sensor was triggered because of the presence of a user. (b) The probability of false negative and false positive of the sensors. (c) The geometric disposition of the areas that are controlled by the sensors, and the area where the user is expected to stand. The output of the ULPC is used by the application to evaluate the context-aware authentication status of the user. Optionally, a web camera is used for user detection

15

Adaptive Trust Management

399

purposes. Once a user has been detected, a face recognition component can try to identify the user standing in front of the web camera. The output of the face recognition component is also communicated to the ULPC, which combines it with the results obtained from fusing of location information obtained from the sensed BT or RFID user identity tokens. If this level of authentication is not sufficiently high, the application will ask the user to provide stronger identification information, for example by displaying a username and password input window. Due to the time-dependent aspect of location information, the ULPC component calculates the location probability regularly, that is every time it gets new location information events. In particular, the RFID authentication method is highly timesensitive as it has a very accurate location quality. This means that the RFID location probability drops quite rapidly in time. In other words, the coverage area of the RFID sensor becomes larger accordingly to a model of mobility of the user. With dropping probability, the level of authentication will decrease in time as well. Subsequently, the user will automatically loose authorisation to resources that require a higher level of authentication.

15.2.2.2 Trust-Based Approach to Context-Aware Authentication Alternatively to the probabilistic approach, the trust-based approach is able to cope with the uncertainty that emerges in case of missing sensors and of contradictions in the information coming from different sensors. Trust, in the user standing at the authentication point, can be calculated by collecting all available location data that are linked to the identity trying to get access to the restricted resources. Here, each location sensor is considered as a recommender giving its recommendation on the statement u ∈ l standing for “the user u stands at the l point”. As a matter of fact, a recommendation can be a Subjective Logic [19] opinion, that is a triple of values that express belief, disbelief and uncertainty that the sensor has on the statement u ∈ l from its subjective point of view. A sensor’s viewpoint is affected by the geometry of the area that the sensor controls and by the knowledge of whether an identity token has been detected, at a certain time, within that area. For example, the sensor that detects Bob entering the hall hardly believes that Bob can be at the same time at the third floor. During the authentication process, the recommendations manager (a modified version of the ULPC in Fig. 15.7) retrieves the sensor data, calculates the recommendation information on behalf of sensors and composes an overall measure of trust on the user standing at the authentication point. Trust is used to adjust the security level that results from analysis of the identity tokens provided by the requester. High trust levels increase confidence in the chosen security level, whilst low trust levels indicate the need to strengthen the security level. Furthermore, the use of techniques for trust-management makes the locationbased authentication approach extremely scalable. The time complexity is linear in the number of location sensors. The algorithm is flexible with respect to the used location and sensor type. For example, face recognition cameras can also be sources of recommendations.

400

J.-M. Seigneur et al.

Recommendations originating from unreliable sensors can also be appropriately discounted. Moreover, it is easy to model trust decay with time. Trust decay of a single sensor occurs when no new data are available and affects overall trust. The opinion of a sensor regarding a user being at a certain position depends on the user movement model used. Upon detection the sensor is aware that the user is within the area it controls. After certain time, if the user is not detected again, the sensor assumes that the user might have moved. In that case where the sensor is asked for a recommendation, its uncertainty will be higher. Let us consider again the example of the sensor that detects Bob entering the hall. Upon detection at the entrance, the sensor is certain that Bob cannot be at the third floor at the same time. However, after ten minutes the same sensor, will have less disbelief and more uncertainty about Bob being at the third floor. The reason is that it would be possible for Bob to have moved to third floor during that time.

15.3 Conclusion Trust is often context-sensitive. Context encompasses more than identity. The entity recognition process is able to use a large panel of different identity recognition schemes. Recognition is related to authentication. The end-to-end trust must not only take into account trust in the entity but also trust in the underlying technical infrastructure. Authentication mechanisms are only a part of the underlying technical infrastructure. Thus, computational trust and traditional computer security are quite interdependent and both need to be considered in context-aware software selforganisation.

15.4 Problems–Exercises 15.1 In Chap. 8, computational trust has been used to select the most trustworthy pieces of open source software based on software quality information sent by other open-source platforms available in the network. How would you recognise/authenticate the different platforms? How would you adapt the selection of most trustworthy open-source software pieces if you knew the trust values at platform authentication? 15.2 The approaches for context-aware adaptive authentication illustrated in Chap. 8 have characterised contextual information mainly based on location and time. In particular, location and time have been utilised to enhance user authentication. Describe an enriched scenario of the coffee corner application where location and time are utilised to enhance other security aspects as well, such as user privacy.

15

Adaptive Trust Management

401

Key Points • Trust is quite context-dependent, and context-awareness can be taken into account in advanced computational models of trust; • Identity is an important part of context and is taken into account in computational models of trust via entity recognition and the end-to-end trust; • The end-to-end trust highly depends on technical trust; • Technical trust relies on traditional security mechanisms; • Security extended with context-awareness and trust is very promising.

15.5 Further Reading Reputation Management Services. A book chapter of the “Computer and Information Security Handbook” edited by John Vacca that goes into the details of reputation management services. The other chapters of this book will also remind the reader of the other security aspects for successful software self-organisation. (J.-M. Seigneur, 2009, Elsevier, ISBN:9780131426436.) Collaborative Computer Security and Trust Management. A book that covers the social engineering aspects of collaborative software. (J.-M. Seigneur and A. Slagell, 2009, Information Science Publishing, ISBN:978-1605664156.)

References 1. Belovin, S.M.: On the brittleness of software and the infeasibility of secure metric. IEEE Secur. Priv. 4(4), 96 (2006) 2. Benantar, M.: Access Control Systems: Security, Identity Management and Trust Models. Springer, Berlin (2006) 3. Bhatti, R., Bertino, E., Ghafoor, A.: A trust-based context-aware access control model for web-services. Distrib. Parallel Databases 18(1), 83–105 (2005) 4. Bizer, C., Cyganiak, R., Gauss, T., Maresh, O.: The TriQL.P browser: filtering information using context, content and rating-based trust policies. In: Proceedings of the Semantic Web and Policy Workshop (2005) 5. Buchegger, S., Boudec, J.L.: A robust reputation system for P2P and mobile ad-hoc networks. In: Proceedings of the Second Workshop on the Economics of Peer-to-Peer Systems (2004) 6. Castro, P.: Using context to assist in multimedia object retrieval. In: First International Workshop on Multimedia Intelligent Storage and Retrieval Management (1999). citeseer.ist.psu. edu/article/castro99using.html 7. Chess, D.M., Palmer, C.C., White, R.S.: Security in an autonomic computing environment. IBM Syst. J. 42(1), 107–118 (2003) 8. Covington, M.J., Ahamad, M., Essa, I., Venkateswaran, H.: Parameterized authentication. In: Samarati, P., Rayan, P., Gollmann, D., Molva, R. (eds.) Proc. of the 9th European Symposium on Research in Computer Security, Sophia Antipolis, France, 13–15 September 2004. Lecture Notes in Computer Science, vol. 3193, pp. 276–292. Springer, Berlin (2004)

402

J.-M. Seigneur et al.

9. Dey, A.K., Abowd, G.D.: Towards a better understanding of context and context-awareness. Tech. Rep., Georgia Institute of Technology, College of Computing (1999). http://citeseer.ist. psu.edu/context/1190105/0 10. Dimmock, N., Bacon, J., Ingram, D., Moody, K.: Risk models for trust-based access control(tbac). In: Trust Management. Lecture Notes in Computer Science, vol. 3477, pp. 364–371. Springer, Berlin (2005) 11. Ganger, G.R.: Authentication confidences. Tech. Rep. CMU-CS-01-23, Carnegie Mellon University, School of Computer Science (2001) 12. Golbeck, J., Parsia, B.: Trusting claims from trusted sources: trust network based filtering of aggregated claims. In: Proceedings of the 3rd International Semantic Web Conference. LNCS, vol. 3298. Springer, Berlin (2004). http://www.mindswap.org/papers/Jen-ISWC04.pdf 13. Guha, R.: Open rating systems. Techn. Rep., Stanford University (2004) 14. http://www.freeband.nl 15. Hulsebosch, R.J., Bargh, M.S., Lenzini, G., Ebben, P.W.G., Jacob, S.M.: Context sensitive adaptive authentication. In: Proc. of the 2nd European Conference on Smart Sensing and Context, Kendal, UK, 23–25 October 2007. Lecture Notes in Computer Science, Springer, Berlin (2007, to appear) 16. Jain, A.K., Ross, A., Prabhakar, S.: An introduction to biometric recognition. IEEE Trans. Circuits Syst. Video Technol. 14(1), 4–20 (2004). http://ieeexplore.ieee.org/xpls/abs_all.jsp? arnumber=1262027 17. Jøsang, A.: The right type of trust for distributed systems. In: Proceedings of the New Security Paradigms Workshop. ACM, New York (1996). http://citeseer.nj.nec.com/47043.html 18. Jøsang, A.: A subjective metric of authentication. In: Quisquater, J.J., Deswarte, Y., Meadows, C., Gollmann, D. (eds.) Proc. of the 5th European Symposium on Research in Computer Security (ESORICS 98), Louvain-la-Neuve, Belgium, 16–18 September 1998. Lecture Notes in Computer Science, vol. 1485, pp. 329–344. Springer, Berlin (1998) 19. Jøsang, A.: A logic for uncertain probabilities. Int. J. Uncertain. Fuzziness Knowl.-Based Syst. 9(3), 279–312 (2001) 20. Krukow, K.: Towards a theory of trust for the global ubiquitous computer. Ph.D. thesis, Dept. of Computer Science, Univ. of Aarhus, Denmark (2006) 21. Lenzini, G., Hulsebosch, R.J., Bargh, M.S.: Trust-enhanced security in location-based adaptive authentication. In: Proc. of the ESORICS 3rd International Workshop on Security and Trust Management (STM 07), Dresden, Germany, 27 September 2007. Electronic Notes in Theoretical Computer Science. Elsevier, Amsterdam (2007, to appear) 22. Levin, T.E., Irvine, C.E., Spyropoulou, E.: Quality of Security Service: Adaptive Security. Handbook of Information Security. Threats, Vulnerabilities, Prevention, Detection and Management, vol. III. Wiley, New York (2005) 23. Marcus, L.: Local and global requirements in an adaptive security infrastructure. In: Proc. of the Int. Workshop on Requirements High Assurance Systems (RHAS), Monterey Bay, CA, USA, 9 Sept. 2003, pp. 23–29. Carnegie Mellon University Press, Pittsburg (2003) 24. Marsh, S.: Formalising trust as a computational concept. Ph.D. dissertation, University of Stirling, Department of Mathematics and Computer Science (1994). http://citeseer.nj.nec.com/ marsh94formalising.html 25. Mezzetti, N.: A socially inspired reputation model. In: Proceedings of EuroPKI (2004) 26. Montanari, R., Toninelli, A., Bradshaw, J.M.: Context-based security management for multiagent systems. In: Proc. of the 2nd IEEE Symposium on Multi-Agent Security and Survivability (MAS&S 2005), 30–31 Aug. 2005, Philadelphia, USA, pp. 75–84. IEEE Computer Society, Los Alamitos (2005) 27. Mostéfaoui, G.K., Brézillon, P.: A generic framework for context-based distributed authorizations. In: Proc. of the 4th Int. and Interdisciplinary Conference on Modeling and Using Context (CONTEXT 2003), 23–25 June 2003, Stanford, CA, USA. Lecture Notes in Artificial Intelligence, vol. 2680, pp. 204–217. Springer, Berlin (2003) 28. Noble, B., Corner, M.: The case for transient authentication. In: Proc. of the 10th ACM SIGOPS European Workshop: Beyond the PC, Saint-Emillion, France, September 2002, pp. 24–29. ACM, New York (2002)

15

Adaptive Trust Management

403

29. Presti, S.L., Cusack, M., Booth, C.: Trust issues in pervasive environments. Techn. Rep., University of Southampton (2003) 30. Rahman, A.F.: A framework for decentralised trust reasoning. Ph.D. dissertation, University of London (2005) 31. Ranganathan, A., Al-Muhtadi, J., Campbell, R.H.: Reasoning about uncertain contexts in pervasive computing environments. Pervasive Comput. 3(2), 62–70 (2004) 32. Ryutov, T., Zhou, L., Neuman, C., Leithead, T., Seamons, K.E.: Adaptive trust negotiation and access control. In: Proc. of the 10th ACM Symposium on Access Control Models and Technologies (SACMAT05), 1–3 June 2005, Stockholm, Sweden, pp. 139–146. ACM, New York (2005) 33. Sandhu, R.S., Samarati, P.: Access control: principles and practise. IEEE Commun. Mag. 9(32) (1994) 34. Sandhu, R.S., Coyne, E.J., Feinstein, H.L., Youman, C.E.: Role-based access control models. IEEE Comput. 29 (1996) 35. Schneck, P.A., Schwan, K.: Dynamic authentication for high-performance networked applications. In: Proc. of the 6th IEEE/IFIP Int. Workshop on Quality of Service (IWQoS 98), Napa, CA, USA, 18–20 May 1998, pp. 127–136. IEEE Computer Society, Los Alamitos (1998) 36. Seigneur, J.M.: Trust, security and privacy in global computing. Ph.D. dissertation, Trinity College Dublin (2005) 37. Shakhnarovich, G., Lee, L., Darrell, T.: Integrated face and gait recognition from multiple views. In: IEEE Computer Vision and Pattern Recognition (2001). citeseer.ist.psu.edu/ shakhnarovich01integrated.html 38. Toivonen, S., Lenzini, G., Uusitalo, I.: Context-aware trust evaluation functions for dynamic reconfigurable systems. In: Proc. of the Models of Trust for the Web Workshop (MTW’06), held with the 15th International World Wide Web Conference (WWW2006), 22 May 2006, Edinburgh, Scotland, CEUR Workshop Proceedings, CEUR-WS (2006) 39. Toninelli, A., Montanari, R., Kagal, L., Lassila, O.: A semantic context-aware access control framework for secure collaborations in pervasive computing environments. In: Proc. of the Fifth International Semantic Web Conference (ISWC), Athens, GA, 5–9 Nov. 2006. Lecture Notes in Computer Science, vol. 4273, pp. 473–486. Springer, Berlin (2006) 40. Veríssimo, P.E., Neves, N.F., Correia, M.P.: Intrusion-tolerant architectures: concepts and design. In: Architecting Dependable Systems. citeseer.ist.psu.edu/561906.html 41. Viswanathan, P.: Security architecture in gaia. Techn. Rep. 2215, Urbana, Illinois (2001). citeseer.ist.psu.edu/viswanathan01security.html 42. Weiser, M., Brown, J.S.: Designing calm technology. Techn. Rep., Xerox PARC (1995). http://nano.xerox.com/hypertext/weiser/calmtech/calmtech.htm 43. Wu, H., Siegel, M., Ablay, S.: Sensor fusion using Dempster–Shafer theory II: static weighting and Kalman filter-like dynamic weighting. In: Proc. of 20th IEEE Instrumentation and Measurement Technology Conference (IMTC 2003), 20–22 May 2003, Vail, CO, USA, vol. 2, pp. 907–912. IEEE Computer Society, Los Alamitos (2003) 44. Yee, G., Korba, L.: Context-aware security policy agent for mobile Internet services. In: Proc. of the 2005 IFIP International Conference on Intelligence in Communication Systems, Montréal, Québec, Canada, 17–19 October 2005

Chapter 16

Security in Artificial Systems Noria Foukia and Melanie Middlemiss

Natural systems as a source of inspiration for computer security.

Objectives The main goal of this chapter is to: • Present the main concepts needed to understand intrusion detection and intrusion response challenges; • Detail an intrusion detection approach based on immune system; • Detail a response detection approach based on stigmergy; • Propose a completely distributed and decentralised intrusion detection and response system for use in computer networks.

16.1 Introduction In the highly “connected” and mobile world that we live in, organisations rely heavily on computers for their day-to-day processes. From schools and universities to “high-tech” companies, computers and networks are used to process, communicate and store information. The reliance on these systems means that any fault or security breach could cause major disruptions to business and potentially large financial losses. Security risks increase as the number of computers connected to a network increases along with the number of people that have access to the network. Despite N. Foukia () · M. Middlemiss Information Science Department, University of Otago, P.O. Box 56, Dunedin, New Zealand e-mail: [email protected] M. Middlemiss e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_16, © Springer-Verlag Berlin Heidelberg 2011

405

406

N. Foukia and M. Middlemiss

these known weaknesses in computer networks, it is not currently possible to build a completely secure system that still provides users with a suitable level of access and use. A significant reason is that it is difficult to develop completely bug-free software [25]. There will always be one or two “holes” or weaknesses in the software that attackers may exploit so as to gain access to, or manipulate the system. Even with policies and procedures in place, there is no fool-proof method for stopping attackers using social engineering and password cracking. Examples of this interference are the gaining of access to authorised user accounts, or the abuse of access privileges, leading to systems being compromised. Nowadays, many researchers in the field of Intrusion Detection (ID) and Intrusion Response (IR) try to find novel solutions to circumvent previously unencountered intrusive behaviours and to adapt these solutions to new network topologies. One of the principal weaknesses of these systems is the lack of robustness inherent in their centralised nature. Even though most of the existing Intrusion Detection Systems (IDSs) use distributed data collection, many of them continue to perform data analysis centrally, thereby limiting scalability. Moreover, even if the IDS is distributed in the network, its deployed elements generally remain static. With the means available to modern attackers, such as automated intrusion tools, these static and distributed elements are easily accessible. Often, this contributes to decreased reliability and resistance to attacks of such static components. This chapter describes intrusion detection and response realised according to different natural systems metaphors. It will mainly focus on the metaphor taken from the human immune system and foraging ant colonies. The remainder of this chapter is structured as follows. Section 16.2 provides the background necessary to understand the full chapter. It gives basic notions and definitions about computer security and intrusion detection as well as an overview of the immune system and the stigmergy paradigm. Section 16.3 explains how the immune system functions and the ant foraging mechanism is used as inspiration for an Intrusion Detection and Response System (IDRS), and how they have been used to build the conceptual model of an existing IDRS, termed IDReAM (Intrusion Detection and Response executed with Agent Mobility). Section 16.4 presents IDReAM as an application of the conceptual model presented in Sect. 16.3. The concepts presented in this chapter are concluded in Sect. 16.5.

16.2 Background This chapter is at the frontier of two different research domains, namely computer security (intrusion detection and intrusion response) and bio-inspired systems. Thus, the following gives the basic concepts and definitions necessary for a good understanding of the chapter.

16

Security in Artificial Systems

407

16.2.1 Some Definitions About Security Defining computer security is not trivial. The difficulty lies in developing a definition that is broad enough to be valid regardless of the system being described, yet specific enough to describe what security really is. Our definition is based on the definition taken from [5]: in a generic sense, security is the absence of risk or danger. In the context of a computer system, security is the prevention of, or protection against, an illicit action against this computer. These illicit actions are: • Access to information resources by unauthorised entities. • Intentional but unauthorised destruction or alteration of that information. This can be restated as “Security is the ability of a system to protect information and system resources with respect to confidentiality, authentication, integrity and accessibility” [11], where the four core issues are defined as follows: 1. Confidentiality: preventing access to information by unauthorised persons. 2. Authentication: ensuring that users (or the entities) are the persons they claim to be. 3. Integrity: preventing the alteration of information by unauthorised persons in a way that is not detectable by authorised users. 4. Accessibility: preventing detention of information or resources to the detriment of others. Computer security is not restricted to these core concepts. Additional concepts that are often considered part of the taxonomy of computer security include: • Access control: ensuring that users access only those resources and services that they are entitled to access. Ensuring also that qualified users are not denied access to services that they legitimately expect to receive. • Non-repudiation: ensuring that the originators of messages cannot deny that they sent the messages. • Availability: ensuring that a system is operational and functional at any given moment usually provided through redundancy. Loss of availability is often referred to as Denial of Service (DoS). • Privacy: ensuring that individuals maintain the right to control what information is collected about them, how it is used, who uses it, who maintains it and for what purpose it is used.

16.2.2 Intrusion Detection Systems Terminology In the literature different terms and definitions are used for intrusion, intrusion detection and response. We retain the definitions of [4] that are summarised below: • An intrusion is a deliberate or accidental unauthorised access to and/or activity against a communication and information system. It is also a deliberate or accidental abuse of privileges by legitimate users. An intrusion is always relative to

408

• • • •

N. Foukia and M. Middlemiss

the security policy built for the system it is intended to protect because the security policy defines what is allowed and what is denied on the system. An attack is a deliberate intrusion. An intruder is any entity, such as a person, a group or an organisation responsible for the intrusion, and an attacker is any entity responsible for an attack. Intrusion detection is the process of identifying that an intrusion has been attempted, is occurring, will occur or has occurred. Intrusion response is the counteractive measures taken after an intrusion is detected and/or at best, the corrective measures taken to undo the damage that was done or even suspected to be imminent. An IDS is a tool to detect intrusions. Its main goal is to alert the appropriate authority (system administrator or system security officer) before the intruder does any damage.

Intrusions generally fall into two categories as mentioned in [9]: either misuse intrusions or anomaly intrusions. Misuse intrusions are well-defined intrusions (known attacks) on known weak points (vulnerabilities) of a network of computer systems [23]. The assumption is made that all attacks can be described by a pattern of events or “signature”. The common method used to detect misuse intrusions is based on examining the traffic passing though the system and looking for wellknown patterns of attacks or attack signatures. If a match occurs, an alert is produced indicating some form of misuse. The security officers define in their security policy the signatures that they consider as illicit patterns and/or they make arrangements to reduce the system vulnerabilities such as install firewalls and filter addresses. The main drawback of misuse-based IDS is that every new signature has to be entered by the security officer. Often, if done manually, this technique is not very efficient, and new attack signatures can be wrongly identified as being normal event sequences, thus generating false negative patterns. One alternative is to let the IDS itself refer to large databases that contain hundreds or thousands of attack signatures. In certain situations, this implies an authorised access to these databases. Another problem is that of creating the signature itself. Due to the constantly evolving nature of attacks, it is difficult to create a signature that describes an intrusion or attack strategy in such a way that encompasses the specifics of the intrusion, while also distinguishing it from non-intrusive system behaviour. Misuse-based intrusion detection is a widely used approach in commercial IDS [12] as it has the potential to produce a low rate of false positive alerts when the signatures are properly tuned. A false positive alert occurs when the system identifies an intrusion but the activity is not actually intrusive. Anomaly intrusions are based on the observation of deviations from normal system usage or behaviour. They are detected by building up a priori a statistical profile of the system being monitored, and any activity that deviates significantly from this profile is flagged as anomalous and thus intrusive [25], raising an alert. The common way to build the profile is to measure the normal running of the system such as the normal CPU utilisation, disk activity, user logins and file activity. For the security officer, the main task in this model is to define what can be considered as a significant deviation from a normal behaviour. For that, the classical models, for example

16

Security in Artificial Systems

409

the one described in [7], propose metrics computed from available parameters such as average CPU load, number of network connections per minute and number of processes per user. Other more recent models propose new mechanisms based on neural networks [26], learning machines [14] and biological immune systems [12]. The benefit of anomaly-based intrusion detection over misuse-based intrusion detection is that it can detect an anomaly without having to understand an attack before it can be identified as an intrusion. Any deviations from “normal” system behaviour indicate potential intrusions. However, this requires a comprehensive set of training data in order to form an accurate model of normal system activity or behaviour. The main drawback is that this method may also detect unusual behaviours which are not necessary illicit ones (false positives). For instance, it could happen that the system previously accepts abnormal behaviour for a certain user who legitimately changes his behaviour over time. If the current normal user profile does not fit the current user behaviour well, then the IDS will wrongly report an anomaly. As with misuse detection, the level of false negatives is also important. Depending on how the system is tuned, this can arise when the anomalous activity is very similar to activity determined to be normal. The activity would therefore not appear to be anomalous, and no alert would be produced, allowing the intrusive activity to continue. False positives and false negatives are undesirable, and the rate at which they occur is important. For instance, if too many incorrect alerts are produced, the system administrator is likely to start ignoring the alerts or even turn the system off completely [22]. The process of generating false positives can also cause the system to consume resources as it produces unnecessary alerts.

16.2.3 Vertebrate Immune System All living creatures need a way of protecting themselves from harmful stimuli. These stimuli can be endogenous, for example cancerous tumours, or exogenous, for example pathogens. The immune system is a highly complex and distributed system that provides a multilayered form of defence capable of identifying and responding to these harmful stimuli. This section gives an overview of the immune system and some important principles and mechanisms. It provides the basis for understanding the IDS inspired by the immune system itself. There are three main functions of the immune system, namely the recognition, the disposal and the communication [20]: • Recognition: The widely held view of the immune system is that it is designed to recognise the difference between harmless entities called “self” from harmful ones called “non-self”. This difference is believed to be essential for recognising and eliminating the non-self that can cause harm to self. An alternative view is that of the danger model which suggests that not all self is harmless and not all non-self is harmful. For this reason, according to the danger model, the immune system is thought to recognise alarm signals arising from damage to tissues in the

410

N. Foukia and M. Middlemiss

body [17, 18]. This damage could be caused by harmful self (e.g. cancerous cells) as well as harmful non-self (e.g. pathogens). • Disposal mechanisms by which the immune system eliminates the harmful stimuli depend on the part of the immune system that has been activated. These range from the phagocytes (“eating cells”) of the innate immune system which ingest and degrade pathogens, to the more complex mechanisms of the adaptive immune system. • Communication within the immune system is essential to provide coordination between the various recognition and disposal mechanisms. The majority of cells in the immune system act under the influence of signals that they receive from other cells [20]. These signals direct the function of the cells and can be delivered either (1) through direct contact between cells or (2) through specialised molecules called cytokines. To facilitate these three main functions, the immune system comprises multiple levels of protection: • The external defences: the tissues that cover and line the body, along with the various physiological conditions that are present in the body. • The innate immune system: this consists of specialised cells whose self and nonself recognition abilities are “built-in” from birth [21]. There are two theories concerning how these cells determine the need to initiate an immune response. According to the first, antigen presenting cells (APCs) of the innate immune system, such as macrophages and dendritic cells, have pattern recognition receptors (PRRs) on their cell surface. These PRRs recognise pathogen associated molecular patterns (PAMPs), which are found on the cell surface of invading pathogens, and never on the host [13]. These molecular patterns have been conserved through evolutionary time and are common to many pathogens. An alternative view is that of the danger theory proposed by [17, 18]. This theory proposes that APCs, in particular the dendritic cells (DCs), have danger signal receptors (DSRs) which recognise alarm signals sent out by distressed or damaged cells. These signals then stimulate the immune system into initiating an immune response [18]. • When an APC detects danger signals or PAMPs, the associated bacterium is internalised and broken down into smaller protein fragments, termed peptides. These peptides are presented on the surface of the APC in a complex with a specialised molecule called the major histocompatibility complex (MHC). • The adaptive immune system: the adaptive immunity provides specificity and memory capabilities to the system. This involves a particular type of cell, called a lymphocyte, of which there are two kinds, the B-cell and the T-cell. Lymphocytes have several differences from the types of cells involved in the innate immune system. In particular, lymphocytes recognise specific antigens, whereas innate immune cells recognise general pathogen associated molecular patterns. Also, lymphocytes are responsible for developing immune memory, with some cells being retained after an immune response. In the adaptive immune system, lymphocytes circulate through the blood and lymphatic system waiting to encounter antigens. Each antigen has a particular shape

16

Security in Artificial Systems

411

that is recognised by the receptor present on the lymphocyte cell surface. B- and T-cells have slightly different receptors. The B-cell receptor is the surface-bound form of the antibody that is secreted to remove the antigen after recognition. The T-cell receptor (TCR) is similar to that of the B-cell, except that it is not secreted as antibody and does not bind directly to antigen. Rather, it binds to peptides presented in a complex with MHC [13]. There are two ways in which an adaptive immune response can be initiated [3]: 1. Antibody-mediated or humoral immune response. The humoral response is driven by direct B-cell interaction with antigen (in the form of the MHC-peptide complex, as discussed above) and defends against extracellular pathogens. 2. Cell-mediated immune response. The cell-mediated response is driven by indirect activation of T-cells through the recognition of the self MHC-peptide complex and defends against intracellular pathogens and cancer. A more detailed explanation of the immune system can be found in [13, 15, 24].

16.2.4 Ant-Based Systems—Stigmergy Paradigm Social insects organise themselves to ensure the survival of the colony by means of a reactive individual behaviour and a cooperative collective one. Such behaviours can be observed from different activities such as foraging, building nests and sorting larvae. Cooperation in these systems is mediated by an efficient communication mechanism relying on the inscription of task evolution in the environment without direct communication between entities. This paradigm, which is termed stigmergy1 and was introduced for the first time by Paul Grassé in 1959 [10], describes the way social insect communities, such as ants, termites and bees, interact through their environment. To simplify, each entity has an individual local view of its neighbourhood but uses volatile chemical substances called pheromones to mark its environment when achieving a collective task. The deposited pheromone is propagated in the environment and evaporates over time. The deposit of the pheromone creates a gradient field in the environment which tends to attract other insects and to enrol them in a self-catalytic2 behaviour. When the task is finished, no more pheromone is deposited, leading to the disappearance of this information after a period of time through an evaporation mechanism. Stigmergy is used in the particular case of food foraging by ant colonies. At the beginning of the process, when ants are leaving the nest to look for a source of food, they first walk randomly. They deposit pheromone along the path they choose. The amount of deposited pheromone is called the pheromonal gradient, and every ant scanning its neighbourhood will have a great probability to walk up the gradient. When an ant finds a source of food, 1 Refer

to Chap. 3 Self-organisation and Emergence in Natural Systems for more detail about selforganising systems.

2A

self-catalytic reaction is catalysis in which the catalyst is one of the products of the reaction.

412

N. Foukia and M. Middlemiss

it returns to the nest using the same path, reinforcing the pheromonal deposit and gradient. This phenomenon is called self-amplification. This deposit reinforces the probability that other ants will be attracted by the same path to reach the food, and these enrolled ants will also reinforce the pheromonal gradient of the path that they choose. Finally, when the source of food is depleted, more ants will begin to follow other paths3 leading to other sources of food. The pheromone evaporates, and the first path will consequently progressively disappear.

16.3 Taking Inspiration from Natural Systems This chapter describes intrusion detection and response realised according to different natural systems metaphors. It will mainly focus on the metaphors taken from the human immune system and foraging ant colonies. In the following, we will explain to the reader how we anticipate that these two metaphors can inspire some functionality of an IDRS. The immune system and the social insect stigmergic system have been used as an inspiration for the conceptual model of an Intrusion Detection and Response System. In this conceptual model, the IDS maps a number of mechanisms borrowed from the immune system. Autonomous and mobile entities (that can be implemented as software agents)4 travel around the network and work cooperatively to detect local and distributed suspicious patterns, in a manner inspired by immune system cells. The IRS component of the conceptual model maps the collective behaviour of a population of foraging ants (also using autonomous and mobile entities) which mimics the ants’ behaviour and an electronic version of the pheromone. For this conceptual model, there are two populations of agents: • The Intrusion Detection Agents (IDAs) that borrow mechanisms from immune cells to detect attacks happening in the security domain. IDAs responsible for a local domain have to be able to discriminate between normal and abnormal activity. In the immune system that discrimination is done by immune cells distinguishing self from non-self. In the IDS the normal self activity is viewed as a valid sequence of events, whereas the abnormal non-self activity is viewed as an invalid one. • The Intrusion Response Agents (IRAs) that respond to detected intrusions. To achieve this, they must locate the place where the alarm was given by the IDAs and move to this location. To trace the source of the alarm, a mechanism is used which is modelled on the mechanism employed by ants to trace the source of food. Ants use chemical pheromones deposited by other ants in the environment to trace the source of food. The IRAs use an electronic version of pheromone which 3 There

is always a small probability that other paths are created by other ants that keep moving randomly until they find another source of food. This behaviour is crucial for the survival of the ant colony.

4 Refer

to Chap. 4, Software Agents, for more detail about software agents.

16

Security in Artificial Systems

413

indicates the route to the infected node (source of the alarm). This pheromone is built by an IDA when it detects an intrusion and is randomly diffused from the infected node through different nodes of the network. Thus, the pheromone allows a path to be built that the IRAs follow to trace the source of the alarm. Moreover like in ant colonies, an evaporation process inhibits the electronic pheromone to limit the number of IRAs following the same pheromonal trails.

16.3.1 The Immune System Inspiration for the Intrusion Detection System Whether a misuse-based, an anomaly-based or a hybrid approach is taken, the goal when developing an IDS is to construct an adaptable system that can identify intrusive activity in an accurate and timely manner. It is this goal that has led to investigations of how the immune system can inspire systems for intrusion detection. Several desirable characteristics of a good IDS are presented in this section [6, 7, 16, 19]. These characteristics can be matched with properties of the immune system, suggesting the suitability of an artificial immune approach to intrusion detection: • Run continuously without supervision: a good IDS must be reliable enough to be able to run continuously in the background of the system that is being observed without the need for intervention by an administrator. The immune system generally operates unnoticed, constantly providing the body with protection from harmful stimuli. • Fault tolerant: during an attack, the system that the IDS is operating on may be compromised. In this case the IDS should be able to resume without loss of its knowledge-base prior to the attack. If circumstances, such as exhaustion or malnourishment, cause an organism to become weak, its immune system will still function—albeit less effectively. The complementary roles of several immune system components also mean that the system is able to re-allocate tasks to other components, if necessary, providing a fault tolerant system. • Minimal computing load for a system: the level of computational effort required to operate the IDS is important. If the system requires substantial computational effort, it is likely to reduce the performance of the computer with regard to other tasks, and the IDS will not be used. In the immune system a trade-off is made between the space available and the time required to maintain the cell repertoire. Cells are constantly being circulated around the system, and this repertoire of cells is updated through mechanisms such as cell death, immunological memory and cell production. • Recognition: the IDS must be able to successfully detect differences between normal and potentially abnormal behaviour. The primary function of the immune system is to recognise and respond to potentially harmful stimuli.

414

N. Foukia and M. Middlemiss

• Adaptable: IDS can be used in many different situations on differing platforms and hardware architectures. The IDS must be able to be tailored to the system that it is being used to monitor. It must also be able to adapt during its operational lifetime in order to accommodate environmental changes, such as changes to usage patterns. The dynamically changing coverage of the immune system aids in its ability to adapt to new and previously un-encountered pathogens. During response to harmful stimuli, such as pathogens invading the body, the cells of the immune system adapt themselves to enable the system to provide a response to remove the potential harm presented. During this process, the overall immune cell repertoire adapts with changes to cells and the addition of memory cells. • Response: the IDS must respond in a manner appropriate to the level of alert produced. The immune system provides a multilayered defence system. Each layer has different mechanisms by which potentially harmful stimuli can be removed. The combination and coordination of these layers provide an effective overall defence. • Scalability: in a network-based situation, the IDS should be able to scale in accordance with the network it is monitoring without compromising performance. The dynamically changing coverage of the immune system allows it to adapt to changes within an individual. Within a population, the major histocompatibility complex causes individuals to respond differently to the same stimuli and ensures immune response to be diverse within the population. • Resistance to compromise of IDS itself: as far as possible, the IDS should be able to protect itself from attack. While the immune system provides protection against harmful exogenous stimuli, endogenous stimuli can also cause harm, and any cell is susceptible to this harm, even the cells of the immune system itself. Despite this, the ability of the immune system to dynamically adapt itself and continue functioning, even though this maybe in a less effective state, demonstrates the robustness and resilience of the immune system. • Efficiency and reliability: overall the IDS should provide efficient and reliable protection of the system or network it is monitoring. Despite situations of immuno-deficiencies in some individuals, the combined properties of the immune system generally mean that it provides an efficient and reliable protection for its host.

16.3.2 The Ant Foraging System as an Inspiration for an Intrusion Response System Many natural phenomena involve self-organisation, from magnetisation and crystallisation to the collective behaviour of insects’ colonies, etc. As a self-organising system, the ant foraging system has influenced many scientists in computer science,

16

Security in Artificial Systems

415

who have tried to understand it and use it as a metaphor for their conceptual models. Prigogine [8] identified four essential properties that a self-organising system should fulfil: • Mutual interaction: the existence of multiple interactions involving a large number of interacting constituents. • In the IRS many specific IRAS interact with other IRAs in order to offer an adequate response. Each IRA is designed to execute small and simple tasks. The combination of the individual actions induces the behaviour of the whole IRS. • Positive feedback: individuals reinforce the organising effect attracting new individuals which will in turn reinforce the effect. • By diffusing an electronic version of the pheromone, IDAs will attract IRAs. When a specific IRA moves to the location of an intrusion, if it needs help from another IRA (of the same type or a different type), the IRA will reinforce the electronic pheromone at that location in order to get help. • Negative feedback: conversely, some variations or phenomena tend to reduce themselves and progressively disappear. • After a response has been performed by IRAs, they will not reinforce the electronic pheromone which evaporates over time. Pheromones with a higher gradient will be followed by IRAs. • Amplification of fluctuations: the randomness in the behaviour of some constituents allows the system to escape difficult situations that could compromise self-organisation. Self-organisation can then take part at another moment and/or at another location. • Without any pheromonal information, IRAs will exhibit a random walk. When following a pheromonal trace and after providing a response to an intrusion, they will switch back to their random walk. This situation is vital for the IRS since it avoids a situation where an attacker attracts (known as a honey pot in the security domain) all the IRAs to one location.

16.4 Application: Implementation of an IDRS 16.4.1 Description of the Basic Architecture To summarise, the IDS and IRS conceptual models described previously, borrow some functionalities that are exhibited in natural systems. More precisely, the IDS is inspired by the behaviour of the human immune system, and the IRS is inspired by the behaviour of a colony of foraging ants. An existing prototype, named IDReAM (Intrusion Detection and Response executed with Mobility), that draws upon these natural system inspirations is composed of a set of Mobile Agents (MAs) and services that allow specific detection and response tasks to be carried out on the computer network being protected. In IDReAM architecture, each network node has an MA platform, called J-Seal2 [2], running on it. The platform hosts the IDAs, the IRAs and the needed services in protection domains or seals and provides resource control. Besides the platform, the node also houses a monitor that provides

416

N. Foukia and M. Middlemiss

Fig. 16.1 Node architecture

the administrator with interactive access to the IDRS through a graphical user interface. This allows the administrator to consult the current agents’ activity and to send instructions to the agents, such as an order to terminate. The pheromones and the different profiles are stored locally as well as in a database for administrative operations. For each J-Seal2 platform, there are two seals called AgentHosts: one houses and manages the IDAs and their services, and the other houses and manages the IRAs and their services as shown in Figs. 16.1 and 16.2. The roles and functionalities of the different types of MAs are taken from the conceptual model and have been implemented: 1. An IDA follows the sequence of actions below: • It moves randomly. When entering a machine, it probes the incoming events and computes the Suspicion Index (SI), taking into account the information deposited by other IDAs and the information it previously memorised. • If SI > threshold, it then launches an alert, builds and diffuses a pheromone. • Then it continues its random walk to compute a new deviation. • When returning to the AgentHost that created it, it updates the SI in a non-self profile of prohibited events. Different populations of IDAs have been implemented: • The AuditIDA audits logs using an internal state machine for detecting a signature of intrusion. Indeed, a set of states are interconnected by possible actions, and only some are “final”, for example set off an alert. The incoming log events determine which state is considered at any given moment. • The BehaviourIDA compares a sequence of traced system calls against a clean profile. The current implementation uses the Hamming distance and the r-continuous bit distance.

16

Security in Artificial Systems

417

Fig. 16.2 Different agents in a node

• The PortScanIDA is an implementation for detecting port scans. It listens in raw mode on the TCP sockets of the host. In order to limit resources used, it obtains copies of such packets on all ports for a limited time only. 2. An IRA follows the sequence of actions below: • In its normal quiet state, it moves randomly, searching for pheromonal information in the network. • If it finds a pheromone, it switches to a tracking state and follows the pheromone back to its source. Following the trail, it speeds the pheromone’s evaporation on each node belonging to the current trail. • As soon as it reaches the source of the alert, it initiates a response to the original attack and switches to the quiet state. • Then it continues its random walk to discover another pheromone. Different IRAs have been implemented, executing different tracing strategies. An IRATenace always follows the same pheromone until the source of the alert, whereas an IRASI can switch to another trail if it crosses a pheromone with a higher SI. 3. Other helper agents and services needed for the realisation of the conceptual model are described below: • The SimpleHostSeeker: it provides the IDA (or the IRA) with an asynchronous way to obtain a travel destination. While the IDA (or the IRA) is operating

418

N. Foukia and M. Middlemiss

on the current host, its specially created SimpleHostSeeker selects the next neighbour to visit according to the moving strategy and reports back to the IDA (or the IRA). Using the information provided by the SimpleHostSeeker, the IDA (or the IRA) decides autonomously to choose preferably one direction. Each AgentHost maintains a table of the agent visit frequencies which is used by the AgentHost to control the different populations of agents. • The SimplePheromonePropagator: an IDA which detects an intrusion builds the corresponding pheromone to diffuse the alert through its neighbourhood. The pheromone diffusion requires the implementation of the SimplePheromonePropagator. This MA deposits the pheromone in each host of a randomly chosen path of the network. On each host, the intensity of the deposit is governed by an exponential function of t, where t is the time elapsed after the creation of the pheromone. This latter function was chosen because it satisfies the monotonous decreasing requirement of the pheromonal gradient of the pheromone randomly deposited by the SimplePheromonePropagator. This will allow the IRAs to go up the pheromonal gradient in the opposite direction. • The PheromoneStorage: when the pheromone is found by an IRA, this service modifies the pheromone in the local Pheromone conveyor. This service decreases the pheromonal gradient in the case of evaporation and destroys the pheromone in the case of inhibition.

16.4.2 Implementation Details A set of packages and classes for the detection and response tasks has been implemented. A simple description of the main packages and classes is given below: 1. Package MobileHost. This subsystem presents a thin service layer that allows weak autonomy in the movement of agents and inter-agent communication through method calls. This is mainly implemented here by two classes, AgentHost and MobileAgent. AgentHost is the topmost seal instantiated as an agent, an instance of this class is created by the framework at start-up. AgentHost uses three sub-classes for the creation and reception of the agents: (a) Initialiser is launched at the start of AgentHost. Initialiser reads from a configuration file the number of IDAs or IRAs to create, depending on the AgentHost and creates them using MobileAgentStarter. It stops immediately thereafter. (b) MobileAgentStarter creates and activates all agents. (c) NetworkListener, launched in parallel to Initialiser, is permanently listening to the network in order to receive and unwrap the incoming agents. MobileAgent is a common abstraction of all the IDA and the IRA derived classes. MobileAgent receives all the necessary elements for its creation from the AgentHost class, most notably its name. JSeal2 provides synchronous communication

16

Security in Artificial Systems

419

Fig. 16.3 Agent moving mechanism

via channels and also an asynchronous send method for asynchronous communication in a default net service. This method is rather rudimentary; therefore, MobileAgent provides an implementation of the moveTo method which enhances the autonomy of an MA. The send method does not allow an agent to decide when it moves, whereas the moveTo allows an agent to do so. Moreover, moveTo accepts the name of a seal as parameter, whereas send accepts only the content of a channel called a capsule. Figure 16.3 shows how the various objects interact to allow agents to semi-autonomously move around network hosts. Figure 16.4 explains the inter-agent communication (between an IDA and its SimpleHostSeeker), required by the collaborative conceptual model. 2. Package IDSHost. This package provides the implementation of the various services provided by stationary agents related to ID and to IR. These services mainly include the storage of the pheromone, administrative alerts, system logs that IDAs feed, and a network topology information service. For instance, the Neighbourhood service, initiated in the start-up, provides the local view of the network to all agents.

16.5 Conclusion In this chapter we have shown how an artificial system can take its inspiration from natural systems. We proposed using functionalities borrowed from the immune system and stigmergy mechanisms in order to build an IDRS. More precisely, the IDS borrows mechanisms from the immune system that protect the human body against external stimulus. The Intrusion Response System (IRS) borrows mechanisms from the stigmergic paradigm of a colony of ants. The two natural systems exhibit a social life by the organisation of their entities (immune cells and ants) which is not possible without the functionality of mobility. Therefore, MAs have been chosen as good candidates to provide this property of mobility.

420

N. Foukia and M. Middlemiss

Fig. 16.4 Inter-agent communication

The combination of the MA paradigm with the two natural system paradigms is the driving force behind the IDRS. This chapter has also described an implementation named IDReAM as a concretisation of the conceptual model. The IDS and the IRS are totally separated from an architectural point of view but can communicate through primitives and mechanisms derived from the conceptual model, such as communication through pheromones. The implementation of the different agents and services is quite generic, with mechanisms dedicated to the two main populations of agents (IDAs and IRAs). This is the case, for instance, of the moving mechanism. But there is also a specialisation of the agents’ role such as the PortScanIDA’s role. The implementation has been thought to be sufficiently flexible and modular: new populations of agents can be easily derived from the generic classes according to the system requirements.

16.6 Problems–Exercises 16.1 Identify the different steps and reasons that have led to the use of software agents in computer networks and discuss the pros and cons of these in terms of network security. 16.2 Name and describe three main properties of the vertebrate immune system that have been utilised in Intrusion Detection Systems.

16

Security in Artificial Systems

421

16.3 Using Sects. 16.2 and 16.3, draw a figure that shows the mapping between a Network Intrusion Detection System and the Human Immune System. 16.4 In broad terms, a Virtual Organisation (VO) can be defined as a temporary alliance of autonomous and separately owned groups (companies, individuals, . . . ) where the participants pool resources, information, capabilities and knowledge in order to meet common objectives [1]. These groups are linked by information and communication technologies with the purpose of delivering a service or a product. List the security risks associated with the establishment of a VO and explain why establishing trust between VO’s members is necessary for a VO’s creation. 16.5 Snort5 is an open-source Intrusion Detection and Prevention System developed by Sourcefire. Snort performs signatures, protocols and anomalies analysis to detect intrusions. Explain the structure of a snort rule and identify the five necessary steps to write a snort rule. 16.6 Explain the difference between link encryption and end-to-end encryption. For each type of encryption, give an example of existing protocols. In terms of traffic analysis, compare what type of information can or cannot be collected by an attacker in both cases. 16.7 Identify the aspects of Mobile Agent security: from the host’s point of view and from the agent’s point of view and classify the possible types of threats. For each type of threat that you identify, give at least two examples of such threats and explain how they work. 16.8 In a distributed collaborative electronic environment, collaboration can be conducted on the basis of autonomous interactions and information exchange between software agents that may act on behalf of a user or on behalf of another agent. In this context, identify the two main types of privacy threats posed by the use of software agents. For each type of threat, build a plausible scenario illustrating such a threat.

Key Points • • • • •

5 The

Objectives and role of intrusion detection and intrusion response system; The intrusion systems can be nature-inspired; Intrusion detection is based on immune system approach; Response detection is based on stigmergy; IDReAM shows a system based on multi-agent system and natural approaches.

official site for Snort is: http://www.snort.org/.

422

N. Foukia and M. Middlemiss

16.7 Further Reading Network Intrusion Detection—An Analysis Handbook, 3rd Edition. This book provides knowledge and practice on network intrusion detection and shows how to handle intrusion analysis. (S. Northcutt and J. Novak, 2002, New Riders, ISBN 0-7357-1265-4.) An Artificial Immune System Architecture for Computer Security. This paper presents an example of agent-based distributed intrusion detection system inspired by the immune system and exhibiting biological behaviours. (P.K. Harmer, P.D. Williams, G.H. Gunsch, and G.B. Lamont, IEEE Transactions on Evolutionary Computation, 6(3), June 2002.) Self-organisation and Computer Security: a case study in adaptive coordination. In this paper, self-organisation is used to allow adaptive and dynamic protection against malicious code in computer systems. (R. Menezes, Proceedings of the 2005 ACM Symposium on Applied Computing, 2005, ACM Publisher, pp. 467– 468.)

References 1. Axelrod, R.: The Evolution of Cooperation. Basic Books, New York (1984) 2. Binder, W.: Design and implementation of the J-SEAL2 mobile agent kernel. In: Proceedings of the European Conference on Object-Oriented Programming (ECOOP 2000) (2000) 3. Campbell, N.A., Reece, J.B.: Biology, 7th edn. Benjamin-Cummings, San Francisco (2004) 4. Coolen, R., Luiijf, H.A.M.: Intrusion detection: generics and state of the art. Tech. Rep., Research and Technology Organisation, Neuilly sur Seine, France (2002) 5. Daintith, J., Wright, E.: A Dictionary of Computing, 6th edn. Oxford University Press, London (2008) 6. Debar, H., Dacier, M., Wespi, A.: Towards a taxonomy of intrusion detection systems. Comput. Netw. 31(9), 805–822 (1999) 7. Denning, D.E.: An intrusion detection model. IEEE Trans. Softw. Eng. 13(2), 222–232 (1987) 8. Glansdorff, P., Prigogine, I.: Thermodynamic theory of structure, stability and fluctuations. Am. J. Phys. 41, 147–148 (1973) 9. Graham, R.: Frequent asked questions, network intrusion detection systems (2000). http:// www.biblioteca.co.cr/pdf/network-intrusion-detection.pdf. White paper 10. Grassé, P.P.: La reconstruction du nid et les coordinations interindividuelles chezbellicositermes natalensis etcubitermes sp la théorie de la stigmergie: Essai d’interprétation du comportement des termites constructeurs. Insectes Soc. 6(1), 41–80 (1959). doi:10.1007/BF02223791 11. Heady, R., Luger, G., Maccabe, A., Servilla, M.: The architecture of a network level intrusion detection system. Techn. Rep. CS90-20, Department of Computer Science, University of New Mexico, USA (1990). http://www.osti.gov/energycitations/servlets/purl/425295-4IN2Pw/web viewable/425295.pdf 12. Hofmeyr, S.A.: An immunological model of distributed detection and its application to computer security. Ph.D. thesis, University of New Mexico, USA (1999) 13. Janeway, C., Travers, P., Walport, M., Shlomchik, M.: Immunobiology: The Immune System in Health and Disease, 6th edn. Garland, New York (2004) 14. Lane, T., Brodley, C.E.: Temporal sequence learning and data reduction for anomaly detection. In: Proceedings of the 5th ACM Conference on Computer and Communications Security, pp. 150–158 (1998)

16

Security in Artificial Systems

423

15. Lydyard, P.M., Whelan, A., Fanger, M.W.: Immunology, Instant Notes, 2nd edn. Bios Scientific, London (2004) 16. Maiwald, E.: Network Security: A Beginner’s Guide. McGraw-Hill, New York (2001) 17. Matzinger, P.: Tolerance, danger, and the extended family. Annu. Rev. Immunol. 12, 991–1045 (1994) 18. Matzinger, P.: The danger model: a renewed sense of self. Science 296(5566), 301–305 (2002) 19. Pieprzyk, J., Hardjono, T., Seberry, J.: Fundamentals of Computer Security. Springer, Berlin (2003) 20. Playfair, J.H.L., Bancroft, G.J.: Infection and Immunity, 2nd edn. Oxford University Press, Oxford (2004) 21. Playfair, J.H.L., Chain, B.M.: Immunology at a Glance, 8th edn. Blackwell Publishing, Malden (2005) 22. Ranum, M.: Intrusion detection: ideals, expectations and realities. J. Comput. Secur. 15(4), 25–45 (1999) 23. Ryan, J., Lin, M.J., Miikkulainen, R.: Intrusion detection with neural networks. In: Jordan, M., Kearns, M., Solla, S. (eds.) Advances in Neural Information Processing System, vol. 10. MIT Press, Cambridge (1998) 24. Sompayrac, L.: How the Immune System Works, 2nd edn. Blackwell Science, Malden (2003) 25. Sundaram, A.: An introduction to intrusion detection. Crossroads 2(4), 3–7 (1996) 26. Tan, K.M.C.: The application of neural networks to Unix computer security. In: Proceedings of the IEEE International Conference on Neural Networks (1995)

Chapter 17

Region Detection in Images Vincent Chevrier, Christine Bourjot, and Vincent Thomas

How a stigmergic process based on silk in social spiders can be transposed to detect regions in grey-level images.

Objectives This chapter details how the collective weaving in social spiders has been transposed and applied for region detection in grey-level images. More precisely, the reader will: • Know the key concepts of ‘region detection’; • Understand the transposition principles that have been used to adapt the biological behaviour to the region extraction; • Find technical details about the transposed system and how it can be programmed.

17.1 Introduction Natural and collective phenomena are sources of inspiration to design self-organised applications, and some examples can be found in [7]. V. Chevrier () · C. Bourjot · V. Thomas LORIA, Université Nancy, Nancy, France e-mail: [email protected] C. Bourjot e-mail: [email protected] V. Thomas e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_17, © Springer-Verlag Berlin Heidelberg 2011

425

426

V. Chevrier et al.

This chapter focuses on the transposition of an original model inspired by social spiders and applied to extract regions in grey-level images. Since the natural inspiration is in insects societies and since the underlying mechanism is a stigmergic process based on silk, we recommend the reader to have had (at least) a look in Chap. 6 of this book to know a little about the phenomenon and about the basis of this mechanism. The approach we develop in this chapter shares the properties that are usually associated with self-organised software. It is: • generic since it can extract various kinds of features depending on the agents and their behaviour; • flexible since this approach gives rise to several variations (for instance by taking into account interactions among features—see exercises); • robust since the final decision is the consequence of local individual decisions which reinforce mutually; • easy to implement since the extraction is made by agents with simple behaviours and is the result of a stigmergic process. The next section presents the application domain on which we applied the swarm mechanism, region extraction. Section 17.3 focuses on the application of the selforganised mechanism by detailing what corresponds to the agents, what represents their environment and what is the dynamics of the whole. Details of each of these elements are provided before presenting and discussing some experimental results. Finally, we summarise the key ideas presented in this chapter. Some exercises at the end of the chapter will highlight variations of this approach for other features extraction.

17.2 Application Description This section presents what region extraction means in the domain of image analysis. It details the characteristics of region extraction and argues for a self-organised approach.

17.2.1 Image Analysis Image analysis and computer vision is a research issue which has been studied since the 1960s. The general aim of computer vision consists in obtaining information and finding meaningful objects from an image represented as a vector of pixels [11]. This task is very complex because the content of the image is not a priori known, because the image can contain noise and because the information to be extracted is highly task-dependent. Thereby, classical approaches focus at first on extracting relevant features (like edges [6] or regions [9, 11]) and try to use these features as a

17

Region Detection in Images

427

basis to assign a meaning to the picture. That is why feature extraction is a key issue for image analysis. A feature can be defined as an ‘interesting part of an image’ and is also highly task-dependent. It can be considered to be a set of pixels sharing specific properties. These properties are: • local properties associated with the pixel and its neighbourhood, like local radiometric characteristics; • global properties concerning the set itself, like the variance of radiometric properties among the belonging pixels. This trade-off between local and global properties makes feature extraction extremely complex. This process is made even more complex by the noise present in images. For instance, if the process focuses only on the local properties without tolerance (like a specific grey level), it will extract an important number of distinct features whereas some of these should be merged (for instance small connected regions with similar grey level). If the process focuses rather on global properties, the features might not be significant enough to be relevant.

17.2.2 Region Extraction The problem this chapter focuses on consists in extracting meaningful regions from an image. A region is defined by a set of pixels with the following properties: • each pair of points of the region is connected (there is a path of pixels belonging to the region that connects these two pixels); • all the pixels of the region share some radiometric properties. The problem of region segmentation can be formally defined by providing from an image A a partition of pixels (a set of n regions Ri with i ∈ [1, n]) such as [23]: • ∀i ∈ [1, n], Ri = ∅; • ∀i, j ∈ [1, n], j = i, Ri ∩ Rj = ∅; • i∈[1,n] Ri = A. In our case, we limit our purpose to extract several regions from an image. Our aim is to propose an approach that enables the first part of the process (finding a set of regions Ri ) without necessarily respecting the constraints of the segmentation (for example, we do not ensure that Ri ∩ Rj = ∅ for all regions). Moreover, all pixels in a region must share some properties. In our case, we will consider grey-level images. Detecting a region is then equivalent to finding a set of connected pixels that have the same grey level, minor or plus a given tolerance. Generally it can be expected that the regions correspond to features in the image that will be later interpreted as meaningful objects. Thus, the relevance of extracted regions is, in general, task-dependent. This problem raises difficult issues:

428

V. Chevrier et al.

• there is no general solution to this problem since many equivalent partitions can be found; • the relevance of a feature is extremely subjective making the results hard to be analysed; • the features must be considered locally and globally, thus raising the question of the link between local and global points of view.

17.2.3 Classical Algorithms A lot of methods have been proposed to extract regions from an image. This section will present two very famous methods, region growing and watershed. • Region-growing approach [1, 23] consists in defining a set of pixels that will become seeds for region extraction. Iteratively, the algorithm agglomerates the neighbouring pixels of already allocated ones according to local criteria. This algorithm finally stops when there is no more allocation. This algorithm has given birth to several variations (like split and merge where initial regions grow and split according to local and global criteria). • Watershed approach [4] consists in progressively flooding the gradient of the image. The first step consists in computing the gradient of the grey-level image. Then, the flooding begins with the lowest pixels of the gradient image and propagates to form basins. These basins will make contact with each other at watershed lines and will finally create a partition of the grey-level image. In both cases, these algorithms involve local processing. This is one of the reason why multi-agent approaches have been developed to tackle the region extraction. Some examples are provided at the end of the chapter (Sect. 17.5).

17.2.4 Why Use Self-organisation The idea of using multi-agent systems for image processing relies on some properties of such systems (see also [10]): • The extraction of features is based on local characteristics of pixels (rupture, homogeneity); agents which have only local perception and which behave according to local stimuli can be suited for such an analysis. Moreover, since extraction of features requires a more global analysis, the way a multi-agent system organises itself can solve this issue. • Multi-agent approaches exhibit inherent parallelism. Since image processing can have a high cost, using multi-agent systems is a means to propose systems with inherent parallel processing ability [3, 15]. • A multi-agent system is an adequate way of decomposing a problem: each agent is responsible for local assessment and organisation emerges from their interactions.

17

•

• •

•

Region Detection in Images

429

The behaviours of agents are easy to define and to implement contrary to larger systems that must consider the global image [14]. The drawback is that, up to now, it has been difficult to correctly analyse the performance of multi-agent systems and to assess and predict image segmentation results. As pointed out by [18], multi-agent systems are known to be fault-tolerant by distributing information redundantly. Even if an agent makes a bad decision, the interactions among the agents will lead to the final decision, and other decisions might compensate the wrong one. Multi-agent systems exhibit flexibility [18]: since computation is mainly local, the system can adapt itself to real-time variation of images and can then be used efficiently for dynamic image feature capture. The features of an image are correlated: for instance, extracting edges and extracting regions are dual problems. These two problems use different algorithms which face different difficulties. It is possible to take advantage of both algorithms by making agents extracting regions and agents extracting edges interact. Multi-agent systems propose an adequate architecture for implementing such interactions. The problem is hard to define, which makes top-down approaches very difficult. Therefore bottom-up approaches like self-organised ones constitute an interesting alternative.

17.3 Description at the Specification Level From our point of view, applying a self-organised approach to solve this problem requires designing a system as three components: • an environment that represents the problem (its initial conditions and constraints); • individual behaviours at the micro level that generate a pattern from the environmental constraints; • the collective pattern that is interpreted as a solution of the problem at the macro level. The problem of extracting regions shares some similarities with the collective weaving presented in Chap. 6: the process needs to explore a space (the set of pixels, namely the image) that has to be restricted to subsets of pixels (the regions) through a selection mechanism.

17.3.1 Intuition The basic intuition that underlies the transposition of the model is as follows. Initially, the environment corresponds to an image. Each stake corresponds to a pixel of the image whose height is the grey level of the pixel. A spider (or a group of spiders) is in charge of detecting a region specified by a reference grey

430 Table 17.1 Principle of transposition for the environment

V. Chevrier et al. Simulation model (vegetation)

Resolution model (grey-level image)

Square grid of stakes (stakes of various height)

Array of pixels (pixels with their grey level)

Silk draglines between stakes

Silk draglines between pixels

Result: Web

Result: web interpreted as a region (a set of pixels)

level. Each agent is provided with parameters which describe the region it has to detect. The spiders will explore the image and will lay down draglines on some pixels that have local ‘interesting’ radiometric properties (for instance pixels that have a grey level similar to the reference one). Silk fixing is then a way to ensure pixel selection. Finally, the environment will contain collective webs that will be interpreted to deduce regions by considering the pixels on which the web is fixed.

17.3.2 Transposition Transposition consists in adapting each of the elements found in a biological system to the context of the problem, and in preserving the collective response. Concretely, this needs (i) to encode the problem and to relate it to the swarm mechanism and (ii) to interpret the collective result as an exploitable solution in the problem domain: • The environment is modified to model the problem. The initial array of stakes corresponds now to an array of pixels. • The collective response in the problem domain is the same as in natural systems (collective webs); the principles of the dynamics are unchanged and are still based on a stigmergic process where the movement of spiders is influenced by the woven silk. • The agents’ behaviour is adapted to make the link between the environment and the system dynamics: silk fixing is now based on a contextual probability instead of a constant one. • For efficiency purposes, some new behaviours have been added: we introduced the ‘return to web’ behavioural item that makes the spider go back to its web. This item restricts the exploration to pixels in the neighbourhood of the already selected ones. It avoids the selection of pixels that share the same grey level but that are not necessarily connected to previously selected ones. Tables 17.1 and 17.2 summarise these key ideas of the transposition of the collective weaving model into a region detection model.

17

Region Detection in Images

431

Table 17.2 Transposition of agent behaviour Behavioural items

Simulation model

Resolution model

Movement Unchanged

According to silk attraction and draglines number

According to silk attraction and draglines number

Silk Fixing Contextual

Constant probability

According to grey level

Return to web New

Absent

Constant probability

17.3.3 Environment The environment corresponds to grey-level image. It is represented by a twodimensional array whose elements are the pixels of the image. Each pixel is featured by its grey level and by the list of draglines already fixed on it. Initially, the environment contains no dragline, the agents will add draglines during runtime. Each dragline is described by its extremity pixels. • The environment is: Env = Array[X, Y ]

of Pixel.

• A pixel is defined by its grey level and the list of draglines fixed on it: Pixel = grayLevel : 0..255; dl : List(Dragline) . • A dragline is defined by its extremities: Dragline = [start, end : Pixel]. When several regions are simultaneously detected, the dragline is labelled by the group of the spider spiderg that created it: Dragline = [start, end : Pixel; spiderg]. In order to specify the behaviour of agents, we define the following sets: • DL(APixel), the set of draglines on the pixel APixel; • DLTo(APixel, BPixel), the set of draglines between APixel and BPixel; and the number of elements in these sets: • NbDl(APixel), the number of draglines fixed on APixel: NbDl(APixel) = Card DL(APixel) . • NbDLTo(APixel, BPixel) is the number of draglines between two pixels APixel and BPixel: NbDLTo(APixel, BPixel) = Card DLTo(APixel, BPixel) .

432

V. Chevrier et al.

Finally, we define the following perception primitives: • Neigh(APixel), the neighbouring pixels of APixel such as their distance1 are below the perception radius R; • NbNeigh(APixel) is the number of neighbouring pixels of APixel (it is equivalent to the cardinal of elements of Neigh(APixel)); • Scuts(APixel), the set of pixels being accessible by following a dragline fixed on APixel: Scuts(APixel) = {p such as p ∈ DLTo(APixel, p)}; • Access(APixel) is the set of all pixels being accessible from APixel: Access(APixel) = Neigh(APixel) ∪ Scuts(APixel). In case of several groups of spiders, each one detecting a region, some of the previous primitives can be specialised to take into account the different kinds of silk draglines. For example, Scuts(APixel, Spiderg) is the set of pixels being accessible by following a dragline fixed by a spider of Spiderg on APixel; or NbDl(APixel, Spiderg) is the number of draglines of type Spiderg fixed on APixel.

17.3.4 Agents Description As mentioned above, an agent behaves according to three behavioural items— movement, silk fixing and return to web—which are cyclically checked. A behavioural item fires stochastically according to the local environment characteristics and to the agent features. The agents can be distributed into groups. In this case, the agents belonging to the same group are focusing on the same region and will share the same parameters. A distinction between the silk fixed by the members of a group and the silk fixed by the members of other groups is made. In both cases, the principle of the behaviour is the same. We will provide a global picture of the agent model valid for both, and when detailing behavioural items, we will present the differences.

17.3.4.1 Behaviour Parameters The agent model includes two sets of parameters. The first concerns the exploration behaviour and is involved in movement and return to web items. The second characterises the silk fixing behaviour. The first set of parameters includes: • BackProbability, a constant probability that models the tendency of an agent to return to its web; 1 The distance between two pixels of respective coordinates (x , y ) and (x , y ) is x − x + 1 1 2 2 1 2 y1 − y2 .

17

Region Detection in Images

433

• R, the perception radius that defines the neighbourhood of the agent’s position; • Attract, the attraction to silk. In the case of the simultaneous detection of several regions,2 we make a distinction between AttractSelf the attraction of an agent to the silk of its group and AttractOther its attraction to the silk of other groups. The second set is made of: • RefLevel that defines the reference grey level of the region to be searched; and • Selectivity that corresponds to the tolerance of selection. In case of several regions detection, an agent is featured by its group Spiderg.

17.3.4.2 Agent Internal State Each agent possesses two internal states (that evolve during problem solving process): • CurrentP, the current position of an agent; • LastFixed, the position of the last pixel on which the agent has fixed a dragline.

17.3.4.3 Perception Each agent has access to the environment according to some perception primitives that provide information on the basis of which its decision is built. When an agent is on a pixel APixel, it has access to the adjacent pixels Neigh(APixel) and to the pixels that can be reached by following a silk dragline put on APixel Scuts(APixel). All these accessible pixels correspond to Access(APixel). Details on these perception primitives can be found in Sect. 17.3.3.

17.3.4.4 Behaviour The agents are reactive ones. They behave according to their perception, their parameters and their internal state. An agent possesses three behavioural items: • a movement item; • a silk fixing item; and • a return to web item. The basic cycle of an agent consists in: Movement choosing, according to a probability distribution, a pixel p from accessible ones; then carrying out the movement to the selected pixel; 2 From

now on, this case will be referred as ‘several regions detection’.

434

V. Chevrier et al.

Silk Fixing deciding to fix according to a contextual probability; if a decision is made, fix a silk dragline between the current position and the last fixed pixel and exit basic cycle; otherwise. Return to web deciding whether to return to the web, according to BackProbability; if the decision is made, returning to the last fixed pixel. In Sect. 17.3.5, each of these items will be presented by detailing the conditions that trigger their firing, the functions used to compute the probability distribution and the updates caused by the decision.

17.3.5 Detail of the Behaviour of Agents 17.3.5.1 Movement Item Triggering At each cycle, the agent decides which pixel to reach among the pixels it can access from its current position. The agent selects the pixel to move on according to a probability density depending on its silk attraction and on the number of draglines. The main tendencies can be stated as the following: • The higher its attraction to silk, the higher the probability to select a pixel not belonging to its neighbourhood; • the higher the number of draglines leading to a specific pixel, the higher the probability to select this pixel. Equations

The probability to move to a pixel is defined by Weight(Apixel) a∈Access(CurrentP) Weight(a)

P (APixel) = with Weight(APixel) being:

• a constant function when Apixel ∈ Neigh(CurrentP); • a function proportional to the number of draglines leading to Apixel and to the silk attraction when APixel ∈ Scuts(CurrentP). A possible instantiation of the function may be Weight(Apixel) = Attract · f NbDLTo(CurrentP, APixel) , where f is a function expressing how the number of draglines in the path influences the weight.3 3 In our experiments, we used f (x) = min(x, SaturationValue) where SaturationValue

corresponds to a limit from which the number of draglines does not any longer influence the result.

17

Region Detection in Images

435

In case of several regions detection, we make a distinction between the silk draglines of the spider’s group and draglines of the other groups to compute the weight Weight(Apixel) = AttractSelf · f NbDLSelfTo(CurrentP, APixel) + AttractOther · f NbDLOtherTo(CurrentP, APixel) , where NbDLSelfTo (resp. NbDLOtherTo) is the number of draglines between the current position and APixel whose group is (resp. is not) the one of the agent. Updates

Once a pixel p has been chosen, the internal state of the agent evolves CurrentP ← p.

17.3.5.2 Silk Fixing Item Triggering When an agent reaches a pixel, it can fix a dragline on it. This silk dragline fixing depends on the local characteristics of the pixel and on the parameters of the agent. The more the pixel corresponds to the type of pixel the agent is searching for, the higher the probability to fix a dragline of silk between this current pixel and the last fixed pixel. Equation For region extraction, the probability can be computed by a normal law (Gaussian function) whose mean is RefLevel and whose standard deviation is 1/Selectivity. Updates As a result, a new dragline is added in the environment, and the internal state of the agent evolves: d ← newDragline(CurrentP, Lastfixed), DL(CurrentP) ← DL(CurrentP) ∪ d, DL(Lastfixed) ← DL(Lastfixed) ∪ d, Lastfixed ← CurrentP. In case of several groups of agents, the new dragline d is built as follows: d ← new Dragline(CurrentP, Lastfixed, spiderg). 17.3.5.3 Return to Web Item Triggering The agent has a constant probability to return to its originating web to prevent it from getting too far from an interesting region without laying silk. Otherwise, two non-connected sets of pixels (separated by one or more regions) could be detected as a single region. An illustration of that is provided by results of experiments in Sect. 17.3.8.

436

V. Chevrier et al.

Equation This probability corresponds directly to the BackProbability parameter of the agent. Result When this behavioural item is triggered, the agent comes back to the last pixel on which it has woven a silk dragline. As a result, its internal state evolves according to the formula CurrentP ← Lastfixed.

17.3.6 System Dynamics 17.3.6.1 Generalities An execution starts initially with an environment empty of silk representing an image. Each agent or set of agents is initialised with parameters characterising the region to detect through the RefLevel and Selectivity values. All the agents have the same BackProbability, and their attraction to silk is defined as a single parameter in case of unique region detection or as two values (AttractSelf and AttractOther) when considering several regions detection. The starting position of each agent defines also the initial value for lastFixed. The system evolves by cycles. In each one, every agent is successively active and applies its basic cycle. The execution ends after a user-fixed number of cycles.

17.3.6.2 Relationships with the Stigmergy Concept Stigmergy is a way to achieve coordination without any explicit reference to the task being performed: the past actions put traces (inscriptions) in the environment; and these traces favour in return some actions among others. In our case, the behaviour of agents leads to the modification of the environment by the apparition of silk draglines. It provides new possibilities for movement, and these possibilities are favoured by silk attraction: the more draglines between the current position and another one, the more likely this position to be chosen. The ‘interesting’ pixels will be more visited and more woven. As a global consequence, we expect each pixel to be woven with a number of draglines which is proportional to the ‘interest’ for it.

17.3.7 Interpretation of the Collective Result The following explanation made the hypothesis that only a single region has to be detected. In case of several regions, a distinction is made according to the type of silk (the spiderg feature of dragline), but the key principles remain the same. As agents do not have any representation of the task to be achieved (region detection), the only output of their behaviour is a web which is the purpose of the initial simulation model.

17

Region Detection in Images

437

We have to deduce regions from the available information in the environment that consists of a set of silk draglines dropped on pixels. At the local level, a list of draglines is associated with each pixel, and the more the pixel is ‘interesting’, the more draglines. We define a rough region RR as the set of pixels that has been woven. Formally, it is expressed as RR = {p ∈ Env such as NbDl(p) > 0}. To ensure selection of relevant pixels, we define a region as the set of pixels whose number of draglines is above a given threshold ThrR = {p ∈ Env such as NbDl(p) ≥ Thr and Thr > 0}. In case of several regions detection, a pixel can be woven by any type of group of spiders. As each group is dedicated to a region, an ‘interesting’ pixel for a group is a pixel the group has woven on. A rough region for a group Spiderg is then RR(Spiderg) = p ∈ Env such as NbDl(p, Spiderg) > 0 , and a region is then

R(Spiderg) = p ∈ Env such as NbDl(p, Spiderg) ≥ Thr and Thr > 0 .

This approach detects regions that can overlap and, therefore, do not meet the requirements for region extraction.

17.3.8 Results and Analysis 17.3.8.1 Experiments We focus on single region detection. We used grey-level images acquired from a CCD camera that have not been preprocessed (no histogram stretching, no contrast enhancement, etc.). We present two main series of experiments. The first is related to region extraction and aims to assess the results our approach can provide. The second deals with the self-organised process and its dynamical properties. In both cases, we present the webs obtained and the belonging degree of pixels, that is the number of draglines on each pixels (the more, the brighter).

17.3.8.2 Expected Outcomes Two main properties are generally expected when extracting regions in pictures: coverage and homogeneity. An efficient extraction algorithm is first characterised by a good coverage of the extracted regions. When it determines a region, it is expected that the entire region is extracted and no parts of the region are forgotten. In our case, this coverage is the global consequence of the exploration process of the agents.

438

V. Chevrier et al.

Extracted regions have to be relevant. First, the extracted region must be constituted by pixels of homogeneous radiometric properties. Furthermore, we do not want the apparition of artificial boundaries due to small variations of light intensity in a single region. This characteristic will be the consequence of the silk-fixing item or, in other words, the selection process. As the characteristics of region extraction are dependent on the quality of the post processing based on that region and refer to a semantic content, it is quite difficult to define the quality of region extraction per se by a mathematical analysis. This is why we focus on qualitative results rather than quantitative ones to assess the properties of our approach.

17.3.8.3 Results and Discussion Results presented here have been obtained by using the code and images provided as material for students in the accompanying CD. The following pictures bring to light that the self-organised approach can provide satisfying results. They confirm the relevancy of our approach since it is able to properly extract various kinds of regions from real images. One of its major advantages is that the same simple behaviour is used and only the individual parameters determine the extracted region. The main drawback is that the parameters of the spider-model have been accurately and empirically tuned by trials and errors to obtain such results. If we focus on the process itself, we have to illustrate the influence of each behavioural item (and associated parameters) on the results. The ‘return to web’ item prevents agents from building a web linking two nonconnected regions. Figures 17.1a, b, c and d: The spiders are put initially in the bright zone on the right of the image. The RefLevel is the grey level of this zone. As BackProbability is null, the spiders invade all the image and wove any pixel with the ‘same’ grey level. Figure 17.1 a is the original image, Fig. 17.1b corresponds to the web, Figs. 17.1c and 17.1d present the silk density on pixels (null density is viewed as blank in Fig. 17.1d). We can see that the pixels of separate regions are woven as if they are in the same one. A very high BackProbability prevents agents to cross small noisy zones to carry on selection processes afield (as in Figs. 17.2 a–b). As a conclusion on this item, we can state that it has to be set up with a medium value (Figs. 17.2 c–d). The ‘movement’ item is influenced by the silk attraction. As in the simulation model, if it is set too high, the constructed web captures the agents, and they do not explore the entire region; if set low, the region covered is larger, but the silk density is low during the same number of cycles the spider is active (Fig. 17.3). Figures 17.3 show the evolution of the system after 1000 iterations. The first results (a–b) correspond to a Pdragline of 0.5 (max density = 17), the second (c–d) of 0.9 (max density = 23). The ‘silk fixing’ item is dependent on the Selectivity parameter. If set too high, small fluctuations of light intensity are not tolerated (as they would be with a low

17

Region Detection in Images

439

Fig. 17.1 (a) Inital image, (b) Web built and (c)–(d) density obtained with a null BackProbability

value of Selectivity). However, this parameter setting is mainly dependent on the region to be extracted. If it is well separated from the rest of the image, Selectivity should be low to consider small fluctuations of grey level in the region. On the contrary, if the region is not well detached from the image, selectivity must be set high to extract expected borderlines. Figures 17.4 show the influence of Selectivity. Execution starts with the same parameter as in Figs. 17.3 except the selectivity that the value of Selectivity is put higher. Spiders are unable to cover the whole region.

17.3.9 Concluding Remarks Our application currently enables the partial detection of regions. All the ingredients are available in our approach for detecting various regions if the required parameters are well assessed. However, a major drawback has to be solved in order to produce a real application: parameters are until now empirically adjusted. The model we propose detects simultaneously several regions. A basic setting of parameters consists in setting the OtherAttract coefficient to null. In this case, the global process consists in several processes without competition that ignore each other. A positive value of AttractOther introduces competition between groups:

440

V. Chevrier et al.

Fig. 17.2 Effect of BackProbability: density obtained with high (a)–(b) or medium (c)–(d) BackProbability

webs built by a group might be attractive for other groups, which will possibly compete for the selection of the same pixels. As the process is based on a simple model, it can be easily adapted to similar tasks. It could be adapted to detect regions according to their textures. In this case, the adaptation would consist in the modification of the silk fixing item by expressing a condition the pixel neighbourhood has to fulfill in terms of texture. The same kind of adaptation could be considered to detect edges. In this case, the silk fixing item would select pixels such as the grey level of their neighbourhood is significantly different. Our approach could also be adapted to detect region in colour images with a similar principle. The selection process would be expressed with a distance measurement computed between the three values corresponding to the desired colour and a reference level expressed as three values.

17.4 Conclusion This chapter presented a swarm mechanism inspired from social spiders and transposed to a region detection problem. This transposition has been undertaken by adapting each of the elements found in the biological system into the context of the concerned problem and by preserving the collective response (in that case, building collective webs):

17

Region Detection in Images

441

Fig. 17.3 Effect of Pdragline: densities obtained after 1000 iterations with high (a)–(b) and medium (c)–(d) values of Pdragline

Fig. 17.4 Effect of Selectivity: densities obtained with a strong Selectivity

• The environment of the agents corresponds to the problem to be solved: it is an array of pixels; • The pattern that emerges from individual behaviours and interactions is the same and is based on a stigmergic process of the movements based on silk; • Agents’ behaviour makes the link between the environment and the system dynamics: silk fixing is based on a contextual probability depending on the grey level of pixels.

442

V. Chevrier et al.

This approach is easy to implement since it requires simple rules of behaviour to code. When applied on real images (not being preprocessed to enhance their quality), this approach proved to be able to actually extract regions even if it needed to empirically set the parameters values. Results demonstrated the potential of the spider approach, and they show that such a mechanism can provide all the ingredients necessary to detect various regions. Additionally, this approach is flexible enough to give rise to several variations (see exercises) and can be used to extract various kinds of features in images.

17.5 Related Work Since image analysis and feature extractions are based on local features, other multiagent approaches have been proposed to deal with these problems. In this article, we have proposed a mechanism based on stigmergy and inspired by spider colonies, but other, agent-based approaches have been proposed in [2, 3, 5, 8, 12–14, 16–22]. This section details three of these approaches. Liu Jiming et al. [14] have proposed a mechanism based on evolutionary computation. In the same way as spiders, they use populations of autonomous agents, but the number of agents evolves during runtime to adapt the population to the environment. At first, agents are distributed on the image. Each agent fires a behavioural item according to its local perception based on the pixel on which it is and its neighbourhood. If the pixel corresponds to the local specification, the agent marks a pixel and reproduces itself, otherwise the agent diffuses to neighbours and finally dies if it exceeds its lifetime. Evolution adapts also the preferred direction of the agent’s diffusion, leading the agents to explore the most promising regions. Bocchi et al. [5] use an algorithm which simulates the colonisation of the image by several populations which are in competition. Each agent faces two constraints: • an environmental constraint: if the agent does not correspond to the pixel, its chance of surviving is reduced; and • a neighbourhood constraint: its chance of surviving depends also on the presence of other agents of the same type in its neighbourhood. These local behaviours lead to population which competes to occupy all the pixels of the image. At the end, each population is supposed to represent a region, and the result of the competition directly leads to an image segmentation. Ramos et al. [18] have proposed to use artificial ants to extract features in images. The behaviour of the agent is inspired by ants: the agent observes the local pheromones and follows preferentially the most pheromone-marked trail. Like [14], the agents have also preferential directions and tend to follow their previous direction. Agents in return can lay pheromones whose quantity depends on a local measure on the image. This individual mechanism leads to the emergence of a ‘cognitive map’ produced by the colony. This cognitive map can then be analysed as the result of the segmentation after a first collective filtering made by the colony.

17

Region Detection in Images

443

17.6 Problems–Exercises In this chapter, we presented the basic region detection algorithm. In the exercise, we propose to focus on possible extensions of this basic model. 17.1 Results analysis. Propose an application that manages to analyse the results. This application should attribute a region to woven pixels and detect pixels that are not belonging to any region. 17.2 Introduction of groups of spiders. The basic model is considering only one group of spiders. The spiders belonging to this group share the same behavioural properties. Can you precise how this basic model can be extended to consider several spider groups? How would you implement self-reproducing spiders inspired by what Liu Jiming has proposed in his article [14]? What can be the use of reproduction among agents? 17.3 Region extraction in movies. The stigmergic approach is based on the dropping of elements in the environment. These elements in return alter the behaviour of the agents. If we consider two successive images in a movie, how is it possible to use the result of the first region extraction in the second region extraction task? How could you adapt the social spider algorithm to detect and follow regions in movies? One way to do so consists in considering that silk can disappear after several time steps. 17.4 Various features extraction. The algorithm we proposed aims at extracting region in grey-level images. How could you alter the behavioural items, so that the spiders extract edges in an image? How could you make region extraction spiders and edge extraction spiders compete in order to enhance the global result of both regions and edges extraction? 17.5 Parameter adaptation. This chapter has shown that the parameters need to be well tuned in order to extract the right regions. Propose directions to automatically tune them. Key Points • This chapter has presented an original stigmergic approach for region detection in grey-level image. • The result of region extraction is the consequence of silk weaving made by artificial spiders which are also attracted by silk woven by other individuals. • This approach can be used to extract various features in an image (like edges) by changing the behavioural parameter of the agents.

444

V. Chevrier et al.

17.7 Further Reading Digital Image Processing, 3rd edition. A large overview of image processing including a 100-page chapter presenting image segmentation. (R. Gonzalez and R. Woods, Prentice Hall edition, pp. 689–795.) An evolutionary autonomous agents approach to image feature extraction. An article based on self-reproducing agents to perform region extraction in images. (J. Liu, Y.Y. Tang, Y.C. Cao. IEEE Transactions on Evolutionary Computation, 1(2):141–158, 1997.) Artificial ant colonies in digital image habitats—a mass behaviour effect study on pattern recognition. An article presenting pattern recognition based on ants algorithms. (V. Ramos, F. Almeida, Proc. of ANTS’2000—2nd Int. Workshop on Ant Algorithms (From Ant Colonies to Artificial Ants), M. Dorigo, M. Middendoff and T. Stfizle (Eds.), pp. 113–116, 2000.) Acknowledgements We wish to acknowledge all the students who worked on the several versions of the software, especially Aurélien Saint-Dizier, Dominique Marie and Anne Chevreux.

References 1. Adams, R., Bischof, L.: Seeded region growing. IEEE Trans. Pattern Anal. Mach. Intell. 16(6), 641–647 (1994) 2. Anton-Canalys, L., Sanchez-Nielsen, E., Hernandez-Tejera, M.: Swarmtrack: a particle swarm approach to visual tracking. In: VISAPP 2006 Proceedings (2006) 3. Ballet, P., Rodin, V., Tisseau, J.: Multiagent boundary detection system: a way to parallel image processing. In: SPIE Optical Sciences, Engineering and Instrumentation, San Diego, CA, USA, vol. 3166, pp. 316–323 (1996) 4. Beucher, S., Meyer, F.: The morphological approach to segmentation: the watershed transformation. In: Mathematical Morphology in Image Processing, pp. 433–481 (1983) 5. Bocchi, L., Ballerini, L., Hässler, S.: A new evolutionary algorithm for image segmentation. In: Applications of Evolutionary Computing, EvoWorkshops2005, pp. 259–268 (2005) 6. Canny, J.F.: Finding edges and lines in images. MIT AI LAB, TR-720, Master’s thesis (1983) 7. Di Marzo Serugendo, G., Karageorgos, A., Rana, O., Zambonelli, F. (eds.): Engineering Selforganising Systems: Nature-Inspired Approaches to Software Engineering. Lecture Notes in Artificial Intelligence, vol. 2977 (2004) 8. Fledelius, W., Mayoh, B.: A swarm based approach to medical image analysis. In: AIA’06: Proceedings of the 24th IASTED International Conference on Artificial Intelligence and Applications, pp. 150–155 (2006) 9. Gambotto, J., Monga, O.: A parallel and hierarchical algorithm for region growing. In: Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (1985) 10. Garbay, C.: Computer vision: a plea for a constructivist view. In: 12th Conference on Artificial Intelligence in Medicine (AIME’09) (2009) 11. Haralick, R., Shapiro, L.: Survey: image segmentation techniques. In: Computer Vision Graphics and Image Processing, vol. 29, pp. 100–132 (1985) 12. Jones, J., Saeed, M.: Image enhancement—an emergent pattern formation approach via decentralised multi-agent systems. Multiagent Grid Syst. 3(1), 105–140 (2007). Special Issue on Nature inspired systems for parallel, asynchronous and decentralised environments 13. Keshtkar, F., Gueaieb, W.: Segmentation of dental radiographs using a swarm intelligence approach. In: IEEE CCECE/CCGEI, Ottawa, Canada (2006)

17

Region Detection in Images

445

14. Liu, J., Tang, Y., Cao, Y.: An evolutionary autonomous agents approach to image feature extraction. In: IEEE Transactions on Evolutionary Computation, vol. 1, pp. 141–158 (1997) 15. Luckenhaus, M., Eckstein, W.: A multi-agent based system for parallel image processing. In: SPIE Optical Sciences, Instrumentation Parallel and Distributed Methods for Image Processing I, San Diego, CA, USA, vol. 3166, pp. 21–30 (1997) 16. Malisia, A., Tizhoosh, H.: Image thresholding using ant colony optimization. In: CRV 06: Proceedings of the 3rd Canadian Conference on Computer and Robot Vision, Washington, DC, USA, p. 26 (2006) 17. Omran, M., Salman, A., Engelbrecht, A.: Image classification using particle swarm optimization. In: Proceedings of the 4th Asia-Pacific Conference on Simulated Evolution and Learning 2002 (SEAL 2002), Singapore, pp. 370–374 (2002) 18. Ramos, V., Almeida, F.: Artificial ant colonies in digital image habitats—a mass behavior effect study on pattern recognition. In: Dorigo, M.M.M., Stfizle, T. (eds.) Proc. of ANTS’2000— 2nd Int. Workshop on Ant Algorithms (From Ant Colonies to Artificial Ants), pp. 113–116 (2000) 19. Richard, N., Dojat, M., Garbay, C.: Multi-agent approach for image processing: a case study for MRI human brain scans interpretation. In: Artificial Intelligence in Medicine, pp. 91–100 (2003) 20. Rodin, V., Benzinou, A., Guillaud, A., Ballet, P., Harrouet, F., Tisseau, J., Le Bihan, J.: An immune oriented multi-agent system for biological image processing. In: Pattern Recognition, vol. 37, pp. 631–645 (2004) 21. Saatchi, S., Hung, C.: Swarm intelligence and image segmentation, swarm intelligence, focus on ant and particle swarm optimization. In: I-Tech Education and Publishing (2007) 22. White, C., Tagliarini, G.A., Narayan, S.: An algorithm for swarm-based color image segmentation. In: IEEE Southeast Conference, pp. 84–89 (2004) 23. Zucher, S.: Survey region growing: childhood and adolescence. In: Computer Graphics and Image Processing, vol. 5, pp. 382–399 (1976)

Chapter 18

Conclusions Giovanna Di Marzo Serugendo, Marie-Pierre Gleizes, and Anthony Karageorgos

18.1 Motivation The dynamicity of the ICT environments and the large-scale size of certain systems makes them impossible to be controlled completely by human administrators. Future systems evolve towards systems not designed by the same designers team, such as ambient systems. This development has lead many researchers and industrials to look into systems that can “work on their own”, that is that they can take care of specific or managerial tasks as much as possible without human intervention. Nature-inspired techniques, borrowed from living or non-living systems and from societies of humans or animals, provide a convenient solution to this problem. This book focuses on self-organising systems, namely systems made of numerous autonomous components that are able to update their organisation on their own in order to fulfill a certain task. The appealing characteristics of these systems include the relative simple nature of their components and their capability as a whole to solve complex tasks. In general, the global system behaviour emerges from the interaction between the local entities. One of the main objectives of researchers is to provide local behaviours leading to coherent global behaviour. Many definitions and concepts are provided in the literature, but here we highlight the ones that seem appropriate specifically for artificial systems. In summary, self-organisation refers to the ability of a system to re-organise its own structure

G. Di Marzo Serugendo Birkbeck College, University of London, London, UK e-mail: [email protected] M.-P. Gleizes IRIT, Université Paul Sabatier, Toulouse, France e-mail: [email protected] A. Karageorgos () Technological Educational Institute of Larissa, Larissa, Greece e-mail: [email protected] G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6_18, © Springer-Verlag Berlin Heidelberg 2011

447

448

G. Di Marzo Serugendo et al.

under environmental changes without external control. A phenomenon is emergent if none of the autonomous components participating in the system has the exact knowledge and capabilities to produce it and in addition system components are not aware of the final objective of the global system.

18.2 Main Points to Remember The main building blocks of an engineered self-organising system are the software agents. These autonomous active entities are able to take initiatives and decisions based on local knowledge, and, possibly empowered with additional reasoning or mobility capabilities, they are the fundamental elements with which to build engineered self-organising systems. Essential to the coherent coordination of individual agents are the self-organising mechanisms. They come under the form of (relatively simple) rules individually applied by all agents in the system. In a decentralised way and generally requiring the agents to have local knowledge only, these rules direct the agents’ behaviour and lead the whole system towards a coherent functionality. Stigmergy is an indirect coordination mechanism where agents coordinate their work by leaving clues in their environment, later retrieved by other agents. It has been extensively used for solving optimisation problems. Gossip is an epidemic style for disseminating information among peers where information obtained is processed before communicating it further. Cooperation, as a means of recovering failures, is also an important approach applied to numerous application domains. Other popular self-organising mechanisms include those inspired from bird flocking and schooling, human trust and the mammals’ immune system. As with any other type of software, it is important that artificial self-organising systems are developed in a systematic manner following appropriate engineering methods. This is even more important in the case of self-organising software since the expected computation result emerges from the local computations of the individual agents. A certain number of engineering methods have so far been introduced focusing on different self-organising system aspects. All methods emphasise the importance of using simulation to calibrate the resulting self-organising system, an approach which remains the main tool for verifying self-organising system behaviour available today. Furthermore, specific middleware infrastructures have been proposed to ease the task of software developers by providing built-in features supporting the implementation of self-organising mechanisms. Artificial self-organising software has been developed in different application domains. Historically, bio-inspired techniques have provided a useful alternative to traditional techniques in the field of static and dynamic problem solving. More recently, adaptive trust management and network security have shown to be fertile fields where self-organising techniques can prove quite useful.

18

Conclusions

449

18.3 Open Issues Despite the intensity of research and activities led on this topic during the last decade, many questions are still open, and important engineering issues still need to be solved. Among others, a point that raises numerous questions is: “How to engineer the global-to-local behaviour?” In other words, if we know what global functionality we seek our multi-agent system to achieve, what should be the individual agents’ behaviour? There is no universal answer to such questions, it currently depends on the problem at hand and on the skills of the engineer. We have seen that a crucial element in the design of self-organising software is provided by the self-organising mechanisms or self-organising rules. This book essentially focuses on rules designed off-line (by a programmer or in an evolutionary way) and coded into the agents. Some of these rules can be adaptive by dynamically changing parameters. However, other types of rules could be considered, such as self-evolving rules—rules that change themselves at run-time or self-generating rules—rules that are designed while the system is working. Some researchers even consider that a truly self-organising system is one able to generate its own rules. Development techniques and methodologies are still in their infancy, and only few of them are mature enough to incorporate tool support. Besides continuing research on development methods, specific to self-organising systems, additional engineering questions need to be answered, such as “How can we develop testing techniques for self-organising software?” and “Can we formally verify a self-organising system?” Furthermore, designers need to study how to evaluate such systems and which measures are relevant for comparing their performance. Another point that seems relevant to consider relates to real-time issues. It is well known that self-organising systems, because of their need to re-organise when necessary, show some latency in converging towards the desired functionality. This fact directly raises the question: “Is it possible to build self-organising systems that meet real-time constraints?” Self-organising systems, both natural and artificial, provide a fascinating topic for study and research. This book targets postgraduate students and active researchers aiming to provide a comprehensive introduction and open the door to the selforganising systems domain, and trigger curiosity and interest for the reader to pursue the presented research topics even further.

Glossary

Access control The means of ensuring that users access only those resources and services that they are entitled to access; ensuring also that qualified users are not denied access to services that they legitimately expect to receive. Accessibility Preventing detention of information or resources to the detriment of others. Adaptation Process leading to a modification of a system for it to better respond to its environment. Adaptive Immune System The adaptive immunity provides specificity and memory capabilities to the system involving a particular type of cell, called a lymphocyte, of which there are two kinds: the B-cell and the T-cell. Adequacy between a system and its environment State reached when a system flawlessly interacts with its environment. Every action/reaction is relevant. Antibody Antibodies (also known as immunoglobulins) are gamma globulin proteins that are found in blood or other bodily fluids of vertebrates, and are used by the immune system to identify and neutralize foreign objects, such as bacteria and viruses.1 In our case, antibodies represent specific behaviours with associated preconditions and affinities with other antibodies. Antigen An antigen (from antibody generator) originally defined as any molecule that binds specifically to an antibody.2 In our case, it refers to any external stimulation that can influence the system of interest. Authentication Ensuring that users (or the entities) are the persons they claim to be. Availability Ensuring that a system is operational and functional at any given moment usually provided through redundancy. Loss of availability is often referred to as Denial of Service (DoS). Chemical-Based Middleware Middleware—e.g. the chemical extension of the TuCSoN (Tuple Centres Spread over the Network) infrastructure—where data and processes can be seen as reactants or molecules whose reactions are managed via 1 Article

from Wikipedia http://en.wikipedia.org/wiki/Antibody.

2 Article

from Wikipedia http://en.wikipedia.org/wiki/Antigen.

G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6, © Springer-Verlag Berlin Heidelberg 2011

451

452

Glossary

laws resembling those governing actual biochemical systems. As a result, agents are coordinated through chemical-like coordination laws that specify how data items have to be manipulated based on patterns such as data transformation and aggregation. Collective weaving Activity exhibited by social spiders consisting in building spiderweb through coordinated actions. Completeness An algorithm is complete if it is able to find a solution if one exists and if not, to inform that no solution is possible. This characteristic is desired but often costly to implement, depending on the complexity of the problem itself. Confidentiality Preventing access to information by unauthorised persons. Cooperation The act or attitude of interacting in whatever the means with another for mutual benefit. It usually involves sharing of information or competences. Decentralisation In artificial and natural systems, decentralisation means that the decision-making process is spread among the entities composing the system. A process is decentralised when there is no central entity responsible for the implementation of the whole process. Emergence Pattern or function at the global level of a system appears solely from local interactions among the lower-level components of the system. Encapsulation In computer science, encapsulation is a principle for software architecture in order to reduce the interdependencies between components. This is a main idea in object-oriented design, which is extended in the MAS domain in order to use the same public interface between agents in protecting their internal components and facilitating their reuse. End-to-End Trust End-to-end trust emphasises that the trust in the technical infrastructure must be taken into account when the trust in the virtual identity is computed. Evolution in the sense of natural evolution mixes the natural selection process with the DNA mutations upon several generations of a population. At each generation, some individual presenting some phenotypical traits ar favoured, and so are more slightly able to propagate those “good” trait in next generation. False negative A false negative occurs when the system fails to detect that an activity is intrusive. False positive A false positive occurs when the system identifies an intrusion, but the activity is not actually intrusive. Field-Based Middleware Middleware—e.g. the TOTA (Tuples On The Air) middleware—where data structures are created and managed via mechanisms resembling the physical concept of field. Agents in distributed computing environments are then coordinated via abstract virtual fields that drive agents’ activities, in the same way as gravitational field drive mass particles in the universe. To this end, the role of middleware is that of effectively supporting creation and representation of such virtual fields, taking into account dynamic environmental conditions so as to allow agents to base their actions on the locally sensed fields. Holarchy A holarchy is a hierarchy defined by the composition relationship between holons. Holon A holon is an entity that can be composed of other (sub-)holons or that can be part of (super-)holons.

Glossary

453

Human Immune System The human biological system that protects the body against diseases, infections and foreign agents. Idiotypic network N.K. Jerne proposed a model for immune system regulation based on communications between antibodies. These communications take the form of stimulation and inhibition. Immune System The immune system protects organisms from infection with layered defences of increasing specificity. Most simply, physical barriers prevent pathogens such as bacteria and viruses from entering the organism. If a pathogen breaches these barriers, the innate immune system provides an immediate but nonspecific response. Innate immune systems are found in all plants and animals. However, if pathogens successfully evade the innate response, vertebrates possess a third layer of protection, the adaptive immune system, which is activated by the innate response. Here, the immune system adapts its response during an infection to improve its recognition of the pathogen. This improved response is then retained after the pathogen has been eliminated, in the form of an immunological memory, and allows the adaptive immune system to mount faster and stronger attacks each time this pathogen is encountered.3 Innate Immune System Specialised cells whose self- and non-self-recognition abilities are built-in from birth. Integrity Preventing the alteration of information by unauthorised persons in a way that is not detectable by authorised users. Intruder Any entity, such as a person, a group, or an organisation responsible for the intrusion. An attacker is any entity responsible for an attack. Intrusion A deliberate or accidental unauthorised access to and/or activity against a communication and information system. It is also a deliberate or accidental abuse of privileges by legitimate users. An intrusion is always relative to the security policy built for the system it is intended to protect because the security policy defines what is allowed and what is denied on the system. An attack is a deliberate intrusion. Intrusion detection The process of identifying that an intrusion has been attempted, is occurring, will occur or has occurred. Intrusion Detection System A tool to detect intrusions. Its main goal is to alert the appropriate authority (system administrator or system security officer) before the intruder does any damage. Intrusion response The counteractive measures taken after an intrusion is detected and/or at best, the corrective measures taken to undo the damage that was done or even suspected to be imminent. MAS Environment Space shared by the agents which can be modified by the actions of the agents. Mobile Agent A software agent that is able to autonomously move from one computer to another via the network and can continue its execution on the destination computer. 3 Article

from Wikipedia http://en.wikipedia.org/wiki/Immune_system.

454

Glossary

Monte Carlo method As to simulate a decision-making process over a set of possible actions with an associated probability distribution, one can sort a randomly generated action depending on this distribution. This technique is called the Monte Carlo method. Mutation (DNA) are defined by some physical and chemical alteration of the genome on an individual. Mutations spans from single base change to gene duplication or deletion. In monocellular individual, mutation will propagate in the next generation each time the cell divides. In multi-cellular sexuated individual, the mutation will propagate in the following generation only if concerns sexual cells. Natural Selection is the process by which a population of individuals under the pressure of environment will be modified. Less fitted individual will die or at least become less performant in their reproductive behaviour. Non repudiation Ensuring that the originators of messages cannot deny that they sent the messages. Non-self Harmful entity of the human body. NP-Complexity A decision problem is in the class P if it can be solved in polynomial time on a deterministic Turing machine. The P-class problems are considered to be easy to solve. The NP class contains problems solvable in a polynomial time by a non-deterministic Turing machine. Unfortunately, there exist a huge amount of real hard problems (NP-complete problems) for which we do not know polynomial-time algorithms (P = NP is an open question). Pathogen An agent or a microorganism capable of causing disease. Pervasive Display System A system composed of a network of digital displays pervading the physical world, showing advertisements and\or information in urban and working environments. Traditionally, such displays are exploited via manual configuration so that they are insensitive to their operating context, resulting in a low infrastructure effectiveness. The adoption of the service ecosystem metaphor can promote context-awareness for the infrastructure, so that information can be displayed based on the current state of the physical environment and actions among displays can be coordinated depending on their spatial distribution, e.g. their physical proximity. Pervasive Service Infrastructure The growing deployment of pervasive computing technologies has led to an increased availability of devices carrying digital information as well as able to perform actuating/sensing actions in the physical world. This results in a huge amount of data available, regarding personal, social and professional activities as well as the physical world. Accordingly, a pervasive service infrastructure is aimed at providing digital services allowing users to perceive, interact and act on the surrounding physical world so as to open to emerging application scenarios, such as real-time traffic information systems and urban networks. Phagocytes Eating cells of the innate immune system. Pheromonal gradient The amount of pheromone that a colony of insects dispatches in its environment. Pheromone A volatile chemical substance used by insects to mark their environment.

Glossary

455

Rationality Ideally an agent having all the useful data, the relevant heuristics and algorithms, and sufficient time to process on, it could make the more optimal decisions. This is not the case in real multi-agent systems, and consequently an agent has only a bounded rationality; nevertheless, this situation cannot prevent the MAS to find a relevant collective solution. Privacy Ensuring that individuals maintain the right to control what information is collected about them, how it is used, who uses it, who maintains it and for what purpose it is used. Reactive agent Simple behaving individual that reacts according to stimulus response rules based on its own state and to the state of its surrounding environment. Region In image analysis, it is a set of pixels with the properties of connexity (each pair of pixels can be connected by a path of pixels belonging to the region) and homogeneity (all the pixels of the region share some radiometric properties). Region segmentation Task in image analysis consisting in providing from an image a partition of regions (which by definition cannot overlap and whose union reconstitute the whole image). Search space Given p the number of parameters being estimated to define a problem, and having a set of constraints to satisfy, the domain (typically a subset of R p ) where each parameter verifies its constraints is called the search space. Security In the context of a computer system, security is the prevention of, or protection against, an illicit action against this computer. These illicit actions are access to information resources by unauthorised entities or intentional but unauthorised destruction or alteration of that information. Self Harmless entity of the human body. Self-adaptation The property of a system to automatically adapt at runtime to dynamic evolution of the surrounding environment, with no need for external control. Self-organisation Process where a system changes its internal organisation to adapt to changes in its goals and the environment without explicit external control. Self-organisation According to Prigogine, self-organisation involves four essential properties: (a) Mutual interaction: the existence of multiple interactions involving a large number of interacting constituents. (b) Positive feedback: individuals reinforce the organising effect attracting new individuals which will in turn reinforce the effect. (c) Negative feedback: conversely, some variations or phenomena tend to reduce themselves and progressively disappear. (d) Amplification of fluctuations: the randomness in the behaviour of some constituents allows the system to escape difficult situations that could compromise self-organisation. Selforganisation can then take part at another moment and/or at another location. Self-organizing Middleware Middleware that support the construction of systems through architectural models inherently promoting self-organisation at the application level. Senescence is the natural eldering process that leads some cell to loose their repairing potentialities. Service Ecosystem A system where service and data components are conceived as individuals of a sort of virtual ecosystem, in which they interact to serve their own purposes according to a set of “ecological laws”, resulting in self-adaptation and

456

Glossary

eternity coming as inherent properties of the system rather than peculiar features of the individuals. Social spider Spider species whose individuals have a tendency to live in colonies and to exhibit collective behaviour. Stigmergy Stigmergy is a kind of coordination with indirect communication. The etymology of “stigmergy” is from “stigma” (cue, sign) and “ergon” (work): the work of individuals is controlled by previous works, done by itself or by conspecifics. Sematectonic stigmergy denotes communication via a modification of the physical environment. Sign-based stigmergy denotes communication via a signaling mechanism. Stigmergy Paradigm that describes the way social insect communities, such as ants, termites and bees, interact through their environment. Trust Trust is a subjective assessment of another’s influence in terms of the extent of one’s perceptions about the quality and significance of another’s impact over one’s outcomes in a given situation, such that one’s expectation of, openness to, and inclination towards such influence provide a sense of control over the potential outcomes of the situation. Trust Value A trust value, that is the digital representation of the trustworthiness or level of trust in the entity under consideration, is seen as a non-enforceable estimate of the entity’s future behaviour in a given context based on past evidence. Virtual Organisation A temporary alliance of autonomous and separately owned groups (companies, individuals, . . . ) where the participants pool resources, information, capabilities and knowledge in order to meet common objectives. Methodologies • Self-organisation—see Chap. 2. • Self-organising system—see Chap. 2. • Development methodology—in software engineering, a development methodology formally defines the process that is used to express requirements, analyse them, design an application that meets these requirements before implementing and testing this application. A methodology is usually made up of a process, some notations and tools to support these notations and/or help the developer. • Development process/lifecycle—in software engineering, the software development process/lifecycle is a logical model followed by designers in order to create and/or alter a software system. Different models have been proposed such as “waterfall”, “spiral”, “incremental”, “agile” and some others. • Unified Process—UP is an iterative software development process. • Rational Unified Process—RUP is an implementation by Rational Software (IBM) of the Unified Process. • Modelling language—a modelling language is a graphical or textual language which defines a precise notation to express abstract concepts. Such a language enables designers to have a common basis for understanding each other during the design of a system by sharing a common language. • Unified Modelling Language—UML is the OMG’s modelling language that enables designers to express the abstract artefacts that are involved in the objectoriented design of a system (such as use cases, classes, activities and so on).

References

457

• Requirement analysis—this phase defines the requirements of the system and the problem the customer wants to solve. • Functional/Non-functional requirements—in software engineering, a collection of requirements describes the features of the system to be designed. These requirements may be functional (what the system has to do) or not (with which quality of service). According to [4], a functional requirement specifies an action that a system must be able to perform, without considering physical constraints, and is a requirement that specifies input/output behaviours of a system. A non-functional requirement specifies system properties, such as environmental and implementation constraints, performance, platform dependencies, maintainability, extensibility and reliability. It is a requirement that specifies physical constraints on a functional requirement [1–3]. • Analysis—considering the requirements previously gathered, what the system does is specified and expressed considering existing constraints. Problems are abstracted and separated, and a conceptual model is established. • Design—this phase answers how the system has to be done. A modular decomposition is done, and the software architecture is defined to provide a logical model. • Implementation—the models previously defined are implemented using a specific programming language. The different modules written are validated. • Test and verification—software testing aims at evaluating features of a program and proving that this program behaves as expected by meeting its requirements. • Deployment—the software system previously implemented and tested is made available for use. This is a general process which is usually customised depending on the context. • Software Process Engineering Metamodel—OMG’s SPEM is a metamodel used to describe a concrete software development process or a family of related software development processes. • WorkDefinition—in SPEM (1.0), a WorkDefinition describes a work performed in a process. Its main subclass is Activity, but Phase, Iteration and Lifecycle are also subclasses of WorkDefinition. A WorkDefinition can be composed of other WorkDefinitions. A WorkDefinition is related to the work products it uses (as input or output) and is owned by the primary role that performs that WorkDefinition in the process. • Activity—an Activity is a subclass of a WorkDefinition. It is a discrete task, relatively short which can be assigned to one individual playing role. It has input and output work products and can de decomposed into smaller atomic elements called Steps. • Step—a Step is the atomic unit of an Activity and can be described in terms of the workproducts it uses and roles it depends on.

References 1. Georgé, J.P., Gleizes, M.: Experiments in emergent programming using selforganizing multi-agent systems. In: Multi-Agent Systems and Applications IV,

458

Glossary

4th International Central and Eastern European Conference on Multi-Agent Systems, CEEMAS 2005, Budapest, Hungary, 15–17 September 2005. LNCS, vol. 3690, pp. 450–459. Springer, Berlin (2005) 2. Georgé, J.-P., Edmonds, B., Glize, P.: Making Self-Organizing Adaptive Multiagent Systems Work, pp. 321–340. Kluwer Academic, Dordrecht (2004). Chap. 16 3. Georgé, J.-P., Peyruqueou, S., Régis, C., Glize, P.: Experiencing self-adaptive MAS for real-time decision support systems. In: Demazeau, Y., Pavón, J., Corchado, J., Bajo, J. (eds.) 7th International Conference on Practical Applications of Agents and Multi-Agent Systems (PAAMS 2009). Advances in Intelligent and Soft Computing, vol. 55, pp. 302–309. Springer, Berlin (2009) 4. Jacobson, I., Booch, G., Rumbauch, J.: The Unified Software Development Process. Addison Wesley, Reading (1999)

Index

A A service providing MAS, 212 ABT, see Asynchronous Backtracking ACO, see Ant Colony Optimisation Acropora millepora, 90 Adaptation, 372 Adaptive trust, 381, 393 Adaptivity, 381 ADELFE, 290 ADOPT, see Asynchronous distributed constraint optimization Affinity, 232–236, 242, 244, 245 Agent, 106, 349, 350 Agent & Artefact metamodel, 289 Aggregation, 151 AMAS, see Adaptive Multi-Agent Systems AMAS-Modelling Language, 302 Ambiguity, 207 Ant Colony Optimisation, 352, 360, 364–366, 375 Ant foraging, 10 shortest path, 10 Antibody, 88, 228–239, 241–245 Antigen, 228–236, 239, 241–244 APO, see Asynchronous Partial Overlay, 354–356, 360 dynamic, 355 optimisation, 355 Apoptosis, 85 Aptitudes, 300 Artificial evolution, 81 Artificial self-organising systems, 14 Astrolabe, 157 Asynchronous Backtracking, 352–354, 357, 358, 360, 364–366, 374

Asynchronous Weak-Commitment Search, 352, 353, 355, 357, 358, 360, 365–367, 375 dynamic, 355 Autonomic Entity Recognition, 391 Autonomy, 107 AWCS, see Asynchronous Weak-Commitment Search B B-lymphocyte, 87 Bucket brigades, 20 C Cell communication, 85 Cellular systems, 85 Chaotic dynamics, 78 Cloud computing, 151 Collective weaving, 429 Completeness algorithm completeness, 352, 355, 368 Complex adaptive systems, 9 Complex systems, 200 Complexity, 78, 105 Computer security, 98 Concentration, 230, 233, 234, 236 Concurrency, 207 Conflict, 207, 352–354, 358, 365, 367–369, 372 min-conflict heuristic, 356, 358, 366, 368 nogood, 353, 366, 374 Constraint optimisation DCOP, 354, 355 Constraint satisfaction complete assignment, 349, 350, 352, 356 constructive algorithms, 349 CSP, 348–350, 352, 355, 358, 365, 367, 372, 374

G. Di Marzo Serugendo et al. (eds.), Self-organising Software, Natural Computing Series, DOI 10.1007/978-3-642-17348-6, © Springer-Verlag Berlin Heidelberg 2011

459

460

Index

Constraint satisfaction (cont.) DisCSP, 348–350, 352–355, 366–368 domain, 352 variable, 352 Context-aware Authentication, 398, 399 Context-aware Security, 381 Context-awareness, 382 Convergence, 153 speed, 153 Cooperation, 365, 369, 375 Critical level, see criticality Criticality, 366, 369 Customised Unified Process (CUP), 285 Cytoskeleton, 11

G GA, see Genetic Algorithms General Methodology, 288 Genetic Algorithms, 82, 360, 362, 364–366, 375 Genetic programming, 82 Gossip, 95, 139 push averaging, 155 push-pull averaging, 152 push-pull gossip algorithm skeleton, 159 SI Model, 143 SIR model, 146 Grassé, 7 Groups, 89

D Data aggregation, see aggregation DBA, see Distributed Breakout Algorithm Death certificates, 150 Decentralisation, 349, 351, 355, 357, 359, 363, 364, 368 Dictyostelium discoideum, 84 Distributed Breakout Algorithm, 352, 357–359, 364, 365, 367, 374 Distribution, 351, 352, 355, 357, 361, 363, 364, 367 Drosophila melanogaster, 84 Dynamics, 364 DynAPO, see Asynchronous Partial Overlay, dynamic DynAWCS, see Asynchronous Weak-Commitment Search, dynamic

H HMAS, 252, 253, 258 Holarchy, 254, 255, 268, 271, 276 Holon, 252–254, 256–261, 263–269, 271–276 Holon destruction, 261 Holon satisfaction, 266, 273, 274, 276 accumulative, 267, 273 collaborative, 267, 273, 274 instant, 268, 273 necessary, 268, 273 self, 267, 273 Holonic role head, 258–266, 268, 272–274, 276 multi-part, 258, 260, 261, 263, 264, 266–268, 272, 273 part, 258, 260, 262–264, 266, 268, 269, 272–274 stand-alone, 258, 264–266, 268, 273 Human behaviours, 92

E Ecology, 82 Ecosystem adaptation, 82 Ecosystems, 77 Embryology, 85 Emergent phenomena, 9 Entities, 293 Environment, 8, 113, 294 Environment, Reactive rules and Agents, 352, 357–359, 364–367, 375 Epidemics, 94, 140 ERA, see Environment, Reactive rules and Agents Eukaryotic cells, 11 Evolutionary programming, 82 Evolutionary strategies, 82 F Fitness function, 81 Foraging, 89

I Image analysis, 426 Immune system, 87, 409 Incompetence, 207 Incomprehension, 207 Information dissemination, 141 Intrusion, 407 Intrusion detection, 406, 412 Intrusion response, 406, 412 L Local search, 349, 350, 352, 356, 357, 359, 361, 365, 366, 368 distributed, 356 tabu search, 356 Locality, 349, 355, 364, 365 principle of locality, 349 Location-based Authentication, 396

Index M Markets, 96 Mass conservation, 153 Mating behaviour, 90 MAY, 304 Memory cells, 87 MetaSelf, 286 Methodology, 290 Middleware infrastructure, 314, 315, 317 Multi-Agent System, 110, 350, 351, 358, 368, 374 Adaptive Multi-Agent Systems, 366, 367, 370, 374 Multi-robot resource transportation, 215 Mutation, 80, 81 N Natural computing, 10 Natural evolution, 76, 80 Natural selection, 80 Natural self-organising systems, 9 biological systems, 11 business systems, 13 physical systems, 11 social systems, 12 Nature-inspired pervasive computing systems, 315 Neo-neurogenesis, 86 Neural system formation, 86 Neurogenesis, 86 Nogood, see conflict, nogood Non cooperative situations, 204, 300 Non determinism, 364 NP-completeness, 348, 352 O Operational environment, 326 OptAPO, see Asynchronous Partial Overlay, optimisation Organisation, 350, 353–355, 372, 374 Overlay networks, 158 P Particle Swarm Optimisation, 352, 360–362, 364–366, 375 Pattern formation, 84 Perception, 300 Predator avoidance, 89 PROSA, 20 PSO, see Particle Swarm Optimisation Q Quasispecies, 79

461 R Rationality bounded rationality, 349 Recombination, 80 Region segmentation, 427 Representations, 300 Reputation, 95 S Search space, 350, 360, 361, 363, 366, 374 Security, 407 Self-organisation, 8 Self-organising applications, 14 grids and P2P systems, 16 manufacturing control, 20 networks, 17 robots, 19 security, 18 sensors, 21 services, 16 web pages, 17 workflows, 22 Self-organising system, 286 Sexual reproduction, 90 Simulation Driven Approach (SDA), 288 Skills, 300 Social insects, 92 Social networks, 93 Specialisation, 91 Stable state, 350 Stigmergy, 91, 358, 360, 368, 411, 436 Stigmergy, coordination through the environment, stigmergy modelling, double bridge experiment, collective sort, collective weaving, collective phenomena in nature, 123 Synchronisation, 91 T T-lymphocytes, 87 TAS, 290 TOTA, 326, 328 TOTA tuples, 329 Transposition, 429, 430 Tropos, 290 Trust, 95 Trust factors, 184 Trust-Based System, 169 TuCSoN, 336 TuCSoN infrastructure, 332 Tuning parameter tuning, 356, 361 Tuple, 328, 336

462 U Unproductiveness, 207 Uselessness, 207

Index V Variable, 348–351 Vision-based Entity Recognition, 393