Artificial Immune Systems: 9th International Conference, ICARIS 2010, Edinburgh, UK, July 26-29, 2010, Proceedings (Lecture Notes in Computer Science Theoretical Computer Science and General Issues)

Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen

Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen TU Dortmund University, Germany Madhu Sudan Microsoft Research, Cambridge, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max-Planck Institute of Computer Science, Saarbruecken, Germany

6209

Emma Hart Chris McEwan Jon Timmis Andy Hone (Eds.)

Artificial Immune Systems 9th International Conference, ICARIS 2010 Edinburgh, UK, July 26-29, 2010 Proceedings

13

Volume Editors Emma Hart Edinburgh Napier University, Centre for Emergent Computing 10 Colinton Road, Edinburgh, EH10 5DT, UK E-mail: [email protected] Chris McEwan Edinburgh Napier University, Centre for Emergent Computing 10 Colinton Road, Edinburgh, EH10 5DT, UK E-mail: [email protected] Jon Timmis University of York, Departments of Computer Science and Electronics Heslington, York, YO10 5DD, UK E-mail: [email protected] Andy Hone University of Kent, School of Mathematics, Statistics and Actuarial Science Cornwallis Building, Canterbury, Kent, CT2 7NF, UK E-mail: [email protected]

Library of Congress Control Number: 2010930538 CR Subject Classification (1998): I.6, I.2, J.3, F.1, F.2, I.5 LNCS Sublibrary: SL 1 – Theoretical Computer Science and General Issues ISSN ISBN-10 ISBN-13

0302-9743 3-642-14546-9 Springer Berlin Heidelberg New York 978-3-642-14546-9 Springer Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. springer.com © Springer-Verlag Berlin Heidelberg 2010 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper 06/3180

Preface

Artificial immune systems (AIS) is a diverse and maturing area of research that bridges the disciplines of immunology and computation. The original research impetus in AIS had a clear focus on applying immunological principles to computational problems in practical domains such as computer security, data mining and optimization. As the field has matured, it has diversified such that we now see a growing interest in formalizing the theoretical properties of earlier approaches, elaborating underlying relationships between applied computational models and those from theoretical immunology, as well a return to the roots of the domain in which the methods of computer science are being applied to immunological modelling problems. Following the trends in the field, the ICARIS conference intends to provide a forum for all these perspectives. The 9th International Conference on AIS (ICARIS 2010) built on the success of previous years, providing a convenient vantage point for broader reflection as it returned to Edinburgh, the venue of the Second ICARIS in 2003. This time, the conference was hosted by Edinburgh Napier University at its Craiglockhart Campus, recently reopened after extensive refurbishment which has resulted in a stunning building and state-of-the-art facilities. The extent to which the field has matured over the preceding years is clear; a substantial track of theoretical research now underpins the discipline. The applied stream has expanded in its outlook, and has examples of AIS algorithms being applied across a wide spectrum of practical problems, ranging from sensor networks to semi-conductor design. This stream demonstrates a mix of both novel approaches and consolidation of more familiar approaches; new paradigms such as artificial chemistries are inspiring development of new algorithms while effort continues in the refinement of existing algorithms based on clonal selection and danger theory. The dedicated modelling stream further emphasizes the interdisciplinary nature of the field, and is an area which we hope to see grow in the future as immunologists and computer scientists continue to find mutually beneficial common ground. As in the previous conference, the Immune Modelling stream published extended abstracts, rather than full papers, appealing to the format more commonly adhered to in the biological sciences. In addition, ICARIS 2010 was pleased to host an additional workshop jointly organized with PerAda, the European Network in Pervasive Adaptation. The workshop solicited short position statements which identified novel applications of bio-inspired computing to pervasive adaptive systems and brought an interesting and eclectic mix of ideas to the area. All papers underwent a thorough review process, and as in the previous year, a rebuttal system was used that allowed authors to respond directly to reviewers’ comments. Based on the rebuttals, we were able to conditionally accept a number of papers that were revised and checked before full acceptance, resulting in an increased quality of these papers. From 41 submissions, we were pleased

VI

Preface

to accepted 23 high-quality full-length papers and extended immune modelling abstracts for publication, giving us an acceptance rate of 56%. The PerAda workshop attracted nine position statements of which 100% were accepted. ICARIS 2010 was delighted to play host to three fascinating keynote speakers. Derek Smith, Professor of Infectious Disease Informatics in the Zoology Department at Cambridge University. His talk, on “The Evolution of Influenza Viruses” showed how the evolution, and thus antigenic characteristics, of A(H3N2) viruses outside E-SE Asia might be forecast each year based on surveillance within E-SE Asia, and showed the importance of the co-evolution of the virus and population-level immunity to the virus. Falko Dressler from the University of Erlangen spoke on the subject of “Self-Organization”. His talk focused on the behavior and the challenges in networked embedded systems, and described the potential for bio-inspired and nano-scale networking in this area. Serge Kernbach from the University of Stuttgart addressed the PerAda workshop on the topic of self-adaptivity and self-development in cooperative and symbiotic swarms. Three tutorial sessions on a diverse range of subjects illustrated the multidisciplinary nature of ICARIS. The first tutorial was given by Mike Holcombe on the topic of agent-based modelling of biological systems, examining new techniques for simulating complex systems. Hugo Van den Berg gave an excellent insight into how mathematical modelling can be applied to immunology, and Mark Neal discussed the pros and cons of working with industry in applying AIS to real problems. In addition, ICARIS 2010 played host to two industry-sponsored competitions. A DSTL-sponsored workshop on AIS for anomaly detection in real-time spectra was organized by Mark Neal of Aberystwyth University. The workshop included a competition requiring participants to perform anomaly detection on real-time mass-spectrometry data. A second competition was hosted by UReason which looked at rationalizing, analyzing and visualizing alarm data, typical of that generated by an operating plant. We are grateful to The Institute for Informatics and Digital Innovation at Edinburgh Napier University for sponsoring a prize for the best student paper and to support from PerAda for the workshop. We would like to extend our thanks to the keynote and tutorial speakers, all members of the Program Committee, ICARIS Vice and Publicity Chairs, and Mark Neal and Martin Robbins for organizing the competitions with DSTL and UReason. The conference enjoyed excellent behind-the-scenes support from Callum Egan (design and maintenance of the website), Jennifer Willies (organizational matters) and an army of PhD students during the conference itself. The final thanks extends to the authors for their input into creating such a high-quality conference. July 2010

Emma Hart Chris McEwan

Organization

Organizing Committee General Chair General Chair Vice Chair Vice Chair Publicity Organizational Support

Emma Hart (Edinburgh Napier University, UK) Chris McEwan (Edinburgh Napier University, UK) Jon Timmis (University of York, UK) Andy Hone (University of Kent, UK) Callum Egan and Chris McEwan (Edinburgh Napier University, UK) Jennifer Willies (Edinburgh Napier University, UK)

Steering Committee Jon Timmis Emma Hart Leando de Castro Hugues Bersini Stephanie Forrest Christian Jacob Guiussepe Nicosia Mark Neal Peter Bentley Doheon Lee

University of York, UK Edinburgh Napier University, UK McKenzie University, Brazil Univerite Libre de Bruxelles, Belgium University of New Mexico, USA University of Calgary, Canada University of Catania, Italy Aberystwyth University, UK University College London, UK KAIST, Korea

Keynote Speakers Derek Smith Falko Dressler Serge Kernbach

University of Cambridge, UK University of Erlangan, Germany University of Stuttgart, Germany

Tutorials Hugo Van den Berg Mark Neal Mike Holcombe

University of Warwick, UK University of Aberystwyth, UK University of Sheffield, UK

VIII

Organization

PerAda Workshop Organizing Committee Martin Drozda Emma Hart Daniele Miorandi Lidia Yamamoto

Leibniz Universit¨ at Hannover, Germany Edinburgh Napier University, UK Create-Net, Italy University of Strasbourg, France

Competitions Martin Drozda Mark Neal

UReason DSTL

Program Committee Uwe Aickelin Paul Andrews Iain Bate Peter Bentley Ed Clark Martin Drozda Andries Engelbrecht Stephanie Forrest Maoguo Gong Fabio Gonzalez Emma Hart Andy Hone Christian Jacob Thomas Jansen Colin Johnson Henry Lau Chris McEwan Daniele Miorandi Nikolaos Nanas Giuseppe Nicosia

Robert Oates Nick Owens Ben Paechter Mario Pavone Fiona Polack Peter Ross M. Zubair Shafiq Susan Stepney Thomas Stibor Alexander Tarakanov Jon Timmis Andy Tyrrell Neil Urquhart Hugo van den Berg Patricia Vargas Fernando Von Zuben Andrew Watkins Slawomir Wierzchon Lidia Yamamoto

Sponsors PerAda (Pervasive Adaptation Network) Institute for Informatics and Digital Innovation (Edinburgh Napier University)

Table of Contents

Immune System Modelling A Petri Net Model of Granulomatous Inflammation . . . . . . . . . . . . . . . . . . Luca Albergante, Jon Timmis, Paul Andrews, Lynette Beattie, and Paul M. Kaye

1

Defining a Simulation Strategy for Cancer Immunocompetence . . . . . . . . Grazziela P. Figueredo and Uwe Aickelin

4

Theoretical Artificial Immune Systems Clonal Selection from First Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chris McEwan and Emma Hart Density Preservation and Vector Quantization in Immune-Inspired Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alisson G. Azzolini, Ricardo P.V. Violato, and Fernando J. Von Zuben

18

33

Immune Inspired Information Filtering in a High Dimensional Space . . . . Nikolaos Nanas, Stefanos Kodovas, Manolis Vavalis, and Elias Houstis

47

On the Benefits of Aging and the Importance of Details . . . . . . . . . . . . . . . Thomas Jansen and Christine Zarges

61

Classifying in the Presence of Uncertainty: A DCA Perspective . . . . . . . . Robert Oates, Graham Kendall, and Jonathan M. Garibaldi

75

Insights into the Antigen Sampling Component of the Dendritic Cell Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chris. J. Musselle FDCM: A Fuzzy Dendritic Cell Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zeineb Chelly and Zied Elouedi Modular RADAR: An Immune System Inspired Search and Response Strategy for Distributed Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Soumya Banerjee and Melanie Moses

88

102

116

X

Table of Contents

Applied Artificial Immune Systems A Faster Clonal Selection Algorithm for Expensive Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Heder S. Bernardino, Helio J.C. Barbosa, and Leonardo G. Fonseca An Information-Theoretic Approach for Clonal Selection Algorithms . . . . Vincenzo Cutello, Giuseppe Nicosia, Mario Pavone, and Giovanni Stracquadanio Antibodies with Adaptive Radius as Prototypes of High-Dimensional Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ricardo P.V. Violato, Alisson G. Azzolini, and Fernando J. Von Zuben

130 144

158

GAIS: A Gaussian Artificial Immune System for Continuous Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pablo A.D. Castro and Fernando Jos´e Von Zuben

171

An Immune Algorithm for Minimum Interference Channel Assignment in Multi-radio Wireless Mesh Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Su-Wei Tan

185

A Developmental and Immune-Inspired Dynamic Task Allocation Algorithm for Microprocessor Array Systems . . . . . . . . . . . . . . . . . . . . . . . . Yang Liu, Jon Timmis, Omer Qadir, Gianluca Tempesti, and Andy Tyrrell An Immunological Algorithm for Doping Profile Optimization in Semiconductors Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Giovanni Stracquadanio, Concetta Drago, Vittorio Romano, and Giuseppe Nicosia

199

213

QML-AiNet: An Immune-Inspired Network Approach to Qualitative Model Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wei Pang and George M. Coghill

223

Biomedical Article Classification Using an Agent-Based Model of T-Cell Cross-Regulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alaa Abi-Haidar and Luis M. Rocha

237

An Artificial Immune System Approach for Artificial Chemistries Based on Set Rewriting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Daniel Schreckling and Tobias Marktscheffel

250

Further Experimentation with Hybrid Immune Inspired Network Intrusion Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Robert L. Fanelli

264

Table of Contents

XI

Danger Theory and Intrusion Detection: Possibilities and Limitations of the Analogy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mark Vella, Marc Roper, and Sotirios Terzis

276

Electronic Fraud Detection for Video-on-Demand System Using Hybrid Immunology-Inspired Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rentian Huang, Hissam Tawfik, and Atulya Nagar

290

PerAda Workshop on Novel Applications of Bio-inspired Computing to Pervasive Adaptive Systems Converging Bio-inspired Robotics and Socio-inspired Agents for Intelligent Transportation Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jeremy Pitt, Yiannis Demiris, and John Polak On Homeostasis in Collective Robotic Systems . . . . . . . . . . . . . . . . . . . . . . . Jon Timmis and Andy Tyrrell Can a Developmental AIS Provide Immunity to a Multi-cellular Robotics System? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Maizura Mokhtar and Yang Liu

304 307

310

Using Virtual Embryogenesis for Structuring Controllers . . . . . . . . . . . . . . Ronald Thenius, Michael Bodi, Thomas Schmickl, and Karl Crailsheim

312

Towards Self-aware PerAda Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Emma Hart and Ben Paechter

314

Is Receptor Degeneracy Suitable for Automatic Response Decisions in Ad Hoc Networks? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sven Schaust, Martin Drozda, and Helena Szczerbicka

317

Biochemically-Inspired Emergent Computation . . . . . . . . . . . . . . . . . . . . . . Lidia Yamamoto and Thomas Meyer

320

Nature-Inspired Adaptivity in Communication and Learning . . . . . . . . . . Borbala Katalin Benko and Vilmos Simon

323

Symbiotic Cognitive Networks: A Proposal . . . . . . . . . . . . . . . . . . . . . . . . . . Tinku Rasheed, Emma Hart, Jim Bown, and Ruth Falconer

326

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

329

A Petri Net Model of Granulomatous InflammationÆ Luca Albergante1 , Jon Timmis2,3 , Paul Andrews2 , Lynette Beattie4 , and Paul M. Kaye4 1

4

Department of Mathematics, Universit` a degli Studi di Milano 2 Department of Computer Science, University of York 3 Department of Electronics, University of York Center for Immunology and Infection, Hull York Medical School and Department of Biology, University of York

Leishmania donovani is an obligate intracellular parasite responsible for the systemic disease visceral leishmaniasis. During the course of the disease, the parasite is found in the spleen, liver and bone marrow. Characteristic of the liver immune response to leishmaniasis is a type of inflammation (“granulomatous inflammation”) that results in the formation of granulomas, structures comprised of an infiltrate of mononuclear cells surrounding a core of infected macrophages. Granulomas help limit the spread of infection and facilitate the killing of parasites. Liver-resident macrophages (Kupffer cells) are able to spontaneously kill many infectious agents, but L. donovani is capable of reproducing inside these cells. Activation of Kupffer cells is required to turn them from host cell to a cell that is able to kill intracellular L. donovani. This process of activation is regulated by cytokines (notably IFNγ) produced by many different types of leukocytes, including natural killer (NK) cells ([1]), CD4 and CD8 T cells ([2]), and NKT cells ([3]). As activated Kupffer cells can potentially damage the local environment, the extent of activation must be carefully regulated by the immune system. Therefore, it is not surprising that some CD4 IFNγ T cells and NK cells, which produce activating cytokines in the early stages of leishmaniasis, develop an immunoregulatory function as disease progresses and also produce IL-10, a cytokine that deactivates macrophages, including Kupffer cells ([4], [1]). The aim of our model is to study the impact of different cell types and cytokines on the formation and maintenance of liver granulomas, with the ultimate goal of developing better therapies. To this end, we opted for a population dynamics approach based on stochastic petri nets (SPNs). SPNs have been extensively used in biological modeling ([5]), as they provide a visual representation that facilitates the development of the model and its description outside the modeling community. SPNs have a well-developed mathematical characterization and can be transformed into Markov chains. Moreover, a SPN can be approximated by a system of ordinary differential equations ([5]). Finally, place and transition Æ

This research is, in part, funded by EPSRC grant number EP/F032749/1 The TRANSIT Programme - Discipline Bridging at the University of York.

E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 1–3, 2010. c Springer-Verlag Berlin Heidelberg 2010 

2

L. Albergante et al.

(a)

(b)

(c)

Fig. 1. Average number of parasites per granuloma plotted against time (weeks): (a) in vivo in C57BL/6 mice — from [10] — and in silico experiments (50 granulomas); (b) in silico experiments (10, 20, 50 granulomas); (c) in silico experiments (50, 100, 200 granulomas). Standard deviation for in silico experiment refers to 5 runs.

invariants of a petri net provide a way to verify some basic properties of the underlying structure. We considered populations of cellular entities (L. donovani amastigotes, Kupffer cells, NK cells, NKT cells, CD4 T cells, CD8 T cells, and mononuclear phagocytes) and cytokines (IL-2, IL-4, IL-10, IL-12, and INFγ). Activation, deactivation and differentiation of the cells have been considered when applicable. Moreover, given the importance of Kupffer cell activation in outcome of the infection, considerable attention has been devoted to the mechanisms controlling this process: (1) deactivation by cognate signaling via CD47-SIRPα during iNKT cell-Kupffer cell interactions ([6]), (de)activation by paracrine signaling ([7]), and deactivation by Leishmania ([8]). Finally, the evolution of CD4 IFNγ T cells has been modeled ([9], [4]). During the course of leishmaniasis, granulomas at different sizes and stages of maturation are found in the liver, and at the peak of infection, the murine liver may contain about 500,000 granulomas. However, most biological data in the published literature do not directly provide information on parasite burden at the level of the individual granuloma (as the method of parasite counting disrupts these structures), and represent merely the total liver parasite burden. Therefore, as a first step to validating our model, 50 parallel and independent granulomas were simulated, and the mean of the number of entities of a specific population was considered, in relation to biological data in the literature.

A Petri Net Model of Granulomatous Inflammation

3

As shown in Fig. 1a, data generated by in silico experiments (data marked by ­) are qualitatively and quantitatively similar to that for the course of infection in mice (data marked by ¬). Moreover, as we can see from Fig. 1b and 1c, while increasing the number of granulomas from 10 to 50 has a noticeable effect on the standard deviation, increasing further the number of granulomas does not significatively affect the data. This suggests that 50 granulomas are a good approximation of the whole inflammatory process. Moreover, the trend of the standard deviation indicates that the probability of oddly behaving granulomas is low. We are currently validating this model using data collected from experiments in a series of gene targeted mice with selective loss of various cells and cytokines involved in granulomatous inflammation. Should the model stand up to such validation, its predictive power can then be tested by further in vivo experimentation.

References 1. Maroof, A., Beattie, L., Zubairi, S., Svensson, M., Stager, S., Kaye, P.M.: Posttranscriptional regulation of Il10 gene expression allows natural killer cells to express immunoregulatory function. Immunity 29(2), 295–305 (2008) 2. Stern, J., Oca, M., Rubin, B., Anderson, S., Murray, H.: Role of L3T4+ and LyT2+ cells in experimental visceral leishmaniasis. J. Immunol. 140(11), 3971–3977 (1988) 3. Amprey, J.L., Im, J.S., Turco, S.J., Murray, H.W., Illarionov, P.A., Besra, G.S., Porcelli, S.A., Sp¨ ath, G.F.: A subset of liver nk t cells is activated during Leishmania donovani infection by cd1d-bound lipophosphoglycan. The Journal of Experimental Medicine 200(7), 895–904 (2004) 4. Trinchieri, G.: Interleukin-10 production by effector t cells: Th1 cells show self control. The Journal of Experimental Medicine 204(2), 239–243 (2007) 5. Heiner, M., Gilbert, D., Donaldson, R.: Petri nets for systems and synthetic biology. In: Bernardo, M., Degano, P., Zavattaro, G. (eds.) SFM 2008. LNCS, vol. 5016, pp. 215–264. Springer, Heidelberg (2008) 6. Beattie, L., Svensson, M., Bune, A., Brown, N., Maroof, A., Zubairi, S., Smith, K.R., Kaye, P.M.: Leishmania donovani-induced expression of signal regulatory protein alpha on kupffer cells enhances hepatic invariant nkt-cell activation. European Journal of Immunology 40(1), 117–123 (2010) 7. Mantovani, A., Sica, A., Sozzani, S., Allavena, P., Vecchi, A., Locati, M.: The chemokine system in diverse forms of macrophage activation and polarization. Trends in Immunology 25(12), 677–686 (2004) 8. Bhattacharyya, S., Ghosh, S., Jhonson, P.L., Bhattacharya, S.K., Majumdar, S.: Immunomodulatory role of interleukin-10 in visceral leishmaniasis: Defective activation of protein kinase c-mediated signal transduction events. Infection and Immunity 69(3), 1499–1507 (2001) 9. Schulz, E.G., Mariani, L., Radbruch, A., Hfer, T.: Sequential polarization and imprinting of type 1 t helper lymphocytes by interferon-γ and interleukin-12. Immunity 30(5), 673–683 (2009) 10. Murray, H.W., Tsai, C.W., Liu, J., Ma, X.: Responses to Leishmania donovani in Mice Deficient in Interleukin-12 (IL-12), IL-12/IL-23, or IL-18. Infect. Immun. 74(7), 4370–4374 (2006)

Defining a Simulation Strategy for Cancer Immunocompetence Grazziela P. Figueredo and Uwe Aickelin Intelligent Modelling and Analysis Research Group, School of Computer Science, The University of Nottingham, NG8 1BB, UK {gzf,uxa}@cs.nott.ac.uk

Abstract. Although there are various types of cancer treatments, none of these currently take into account the effect of ageing of the immune system and hence altered responses to cancer. Recent studies have shown that in vitro stimulation of T cells can help in the treatment of patients. There are many factors that have to be considered when simulating an organism’s immunocompetence. Our particular interest lies in the study of loss of immunocompetence with age. We are trying to answer questions such as: Given a certain age of a patient, how fit is their immune system to fight cancer? Would an immune boost improve the effectiveness of a cancer treatment given the patient’s immune phenotype and age? We believe that understanding the processes of immune system ageing and degradation through computer simulation may help in answering these questions. Specifically, we have decided to look at the change in numbers of naive T cells with age, as they play a important role in responses to cancer and anti-tumour vaccination. In this work we present an agentbased simulation model to understand the interactions which influence the naive T cell populations over time. Our agent model is based on existing mathematical system dynamic model, but in comparisons offers better scope for customisation and detailed analysis. We believe that the results obtained can in future help with the modelling of T cell populations inside tumours.

1

Introduction

According to estimations, over a million cases of colorectal cancer are diagnosed annually. This type of cancer is also the second most common cause of cancerrelated deaths. Despite many different types of cancer treatments, survival rates are low and remain between 50% and 10%, depending on the cancer stage. However, none of the actual treatments take into account the patient’s immune fitness or the immune system’s capability to respond to cancer. Recent research [1,2] shows success in laboratory stimulation of T cells that kill tumour cells. However, in real patients there are additional factors to be considered before the stimulation. It is necessary to understand the patient’s ability to respond to the treatment. Clinical trials are important, but not necessarily E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 4–17, 2010. c Springer-Verlag Berlin Heidelberg 2010 

Defining a Simulation Strategy for Cancer Immunocompetence

5

the end point. Therefore further investigation is necessary to determine all the factors to be observed before anti-tumour vaccination. We believe that one of the factors that should be considered is immunocompetence with age. Therefore, we want to investigate if we can use computational simulation models to verify if the immune responses to cancer changes in different life stages. With age there is a decay of the immune system’s performance resulting in degenerative diseases and deregulated and ineffective responses. The ageing of the immune system is called immunosenescence. Following an extensive study of the immunosenescence literature, we point out the four most influential theories in section 2. All four theories seem to be good candidates for developing immunosenescence simulation models. The four theories are: lack of naive T cells, immunological space filling with memory cells, innate system up-regulation and accumulation of T-regulatory cells. The objective of our work is to find out if there is the possibility to predict what actually is going to happen if a patient with a certain age is anti-tumour vaccinated. To build such a computer simulation, one of the above theories has to be chosen first, as they are not all mutually compatible. We will discuss later in more detail why we believe that the first theory is the most appropriate one to be used in cancer simulation models: T cell populations are a major contributor to the immune system’s functionality. They also play a very important role in responses to cancer and anti-tumour vaccination. In this paper, we present results of our simulation model showing the interactions which influence the naive T cell populations over time. The model is based on the mathematical equations defined in [3]. In their work, Murray et al. [3] propose a model with a set of equations to fit some observed data and they try to estimate the likely contribution of each of the naive T cell repertoire maintenance methods. Therefore, we believe that the results of this model could be used as an input to a bigger model involving T cells inside tumours. The work is organized as follows. Section 2 presents the immunological concepts related to immune system ageing. Next, in Section 3, there is a summary of the main theories that could be used for computational modelling. In Section 4 we present the model studied and its results. Finally, we draw conclusions and present future steps of this research in Section 5.

2

Background

According to Bulati et al [4], ageing is a complex process that negatively impacts on the development of the immune system and its ability to function. Progressive changes of the innate and adaptive immune systems have a major impact on the capacity of an individual to produce effective immune responses. The decrease of immunocompetence in the elderly can be envisaged as the result of the continuous challenge of the unavoidable exposure to a variety of potential antigens, e.g. viruses, bacteria, food and self-antigens [5]. Antigens are the cause of persistent life-long antigenic stress, responsible for the filling of the

6

G.P. Figueredo and U. Aickelin

immunological space by an accumulation of effector T cells and immunological memory [5]. With age, there is also a significant reduction of naive T cells numbers caused by the involution of the thymus. This situation eventually leaves the body more susceptible to infectious and non-infections diseases [3]. Also, there is evidence that clonotypical immunity deteriorates, while ancestral innate or natural immunity is conserved or even up-regulated [5,6]. 2.1

Some Factors Related to Immunosenescence

According to Franceschi [5], some factors that characterise immunosenescence are the accumulation of memory T cells, the decrease and exhaustion of naive T cells and a marked reduction of the T cell repertoire. Bulati [4], on the other hand, believes that both innate and adaptive immunity are usually involved in the pathogenesis of chronic age-related diseases like arthritis, atherosclerosis, osteoporosis, diabetes and so on. However, the innate immune system appears to be the prevalent mechanism driving tissue damage associated with different age-related diseases [4]. Thus, ageing is accompanied by an age-dependent upregulation of the inflammatory response, due to the chronic antigenic stress that impinges throughout life upon innate immunity, and has potential implications for the onset of inflammatory diseases. Bulati points out some further important factors related to ageing: – There is evidence of neuromuscular degenerative disease and other tissue dysfunction. – Micronutrient inadequacy leads to metabolic consequences, e.g. DNA damage, cancer, severe infection, cognitive dysfunction and accelerated ageing. – Reactivity of dendritic cells to self antigens can be characteristic of ageing. Furthermore, this over-reactivity induces T lymphocyte proliferation with subsequent higher risk of autoimmune diseases. – Hyper activated T cells are possibly involved in bone loss associated with vascular disease in aged mice. – There is a decrease in vaccine responsiveness leading to mortality. De Martinis [7] and Franceschi [5] state that the most important characteristics of immunosenescence are the accumulation of memory and effector T cells, a reduction of naive T cells, shrinkage of the T cell repertoire and filling of immunological space. He points out that: – The filling of the immunological space with memory and effector cells is a consequence of exposure to a variety of antigens over time. – Clonal expansion of peripheral T cells carrying receptors for single epitopes of the herpes viruses Cytomegalovirus and Epstein-Barr virus are common in the elderly and are associated with a loss of early memory cells, an increase of T cytotoxic cells, a gradual filling of immunological space and an immune risk phenotype. The immune risk phenotype is a set of bioparameters associated with poor immune function. These parameters are: low levels

Defining a Simulation Strategy for Cancer Immunocompetence

–

–

–

–

–

3

7

of B cells, increased levels of cytotoxic T cells, poor T cell proliferative response, a T helper - T cytotoxic ratio of less than 1 and cytomegalovirus seropositivity [8]. With the decline of immune function there is an increase in autoantibody frequency. An important result of this may be a loss of the ability to distinguish between self/nonself molecules. The lifelong respiratory burst, i.e. a reactive oxygen species causes damage to important cellular components (lipidic membranes, enzymatic and structural proteins and nucleic acids) during ageing. Oxidative damage is counteracted by several genetically controlled enzymatic and non-enzymatic antioxidant defence systems. All these protective mechanisms tend to become less effective with age. An elderly immune system becomes more predisposed to chronic inflammatory reactions and less able to respond to acute and massive challenges by new antigens. This continuous attrition is caused by clinical and sub-clinical infections, as well as the continuous exposure to other types of antigens, is likely responsible for the chronic innate immune system activation and inflammation. Inflamm-ageing, the peculiar chronic inflammatory status which characterizes ageing, is under genetic control and is detrimental to longevity. It leads to long term tissue damage and is related to an increased mortality risk. The unavoidable chronic overexposure to stress factors determines a highly pathogenic sustained activation of the stress-response system leading to a progressively reduced capacity to recover from stress-induced modifications.

Candidates for Immunosenescence Models

We decided to select the four most influential theories from the above as possible candidates for building computationally predictive systems and will now discuss them in more detail. The four theories are: a lack of naive T cells, immunological space filling with memory cells, innate system up-regulation and accumulation of T-regulatory cells. 3.1

Lack of Naive T Cells

Before an individual reaches the age of 20, the set of naive T cells is sustained primarily from thymic output [3]. However, in middle age there is a change in the source of naive T cells: as the thymus involutes, there is a considerable shrinkage in its T cell output, which means that new T cells are mostly produced by peripheral expansion. There is also a belief that some memory cells have their phenotype reverted back to the naive cells type [3]. However, these two new methods of naive T cell repertoire maintenance are not effective [3] as they do not produce new phenotypic changes in the T cells. Rather, evidence shows that they keep filling the naive T cell space with copies

8

G.P. Figueredo and U. Aickelin

of existing cells. Therefore, the loss of clones of some antigen-specific T cells becomes irreversible. These age-related phenomena lead to a decay of performance in fighting aggressors. 3.2

Space Filling

The immune system deteriorates with age by losing functionality and immunocompetent cells. Moreover, it becomes limited in its use of resources. There is a finite number of T cells in operation at any time and to work properly, the immune system needs a reserve of naive T cells to combat new intrusions, and memory cells for previously encountered antigens. With age, the repertoire of naive T cells shrinks proportionately to the previously faced threats, while memory cell numbers increase [3,7,5]. Late in life the T cell population becomes less diverse and some antigen-specific types of T cell clones can grow to a great percentage of the total T cell population, which takes up the space needed for other T cells, resulting in a less diverse and ineffective immune system. At some point there are not enough naive T cells left to mount any sort of effective defence and the total repertoire of T cells is filled with memory cells. 3.3

Innate Up-Regulation

With age there is a decay in adequate functioning of the main phagocytes, i.e. macrophages, neutrophils [9] and dendritic cells [10]. As a consequence, deregulated immune and inflammatory responses occur in old people. The investigation into the cellular and molecular mechanism underlying these disorders has provided compelling evidence that up-regulated cyclooxygenase and its product, particularly prostaglandin, play a critical role in the age-associated disregulation of the immune and inflammatory responses [6]. Increased prostaglandin production in old macrophages contributes to the suppression of T cell function with ageing. Furthermore, interventions targeted at decreasing prostaglandin production have been shown to enhance T cell-mediated function [6]. Thus, innate immunity and a high capacity to mount a strong inflammatory response, which is useful at younger age can become detrimental later in life. Inflamm-ageing can thus be considered a main phenomenon responsible for major age-related diseases and the evolutionary price to pay for an immune system fully capable of defending against infectious diseases earlier in life. 3.4

Accumulation of Treg Cells

The individual’s ability to mount an effective immune response can be limited by regulatory elements such as significant changes in the number of T regulatory (Treg ) cells [11]. Treg cells act to suppress activation of the immune system and thereby maintain immune system homoeostasis and tolerance. The accumulation of Treg cells in old people inhibits or prevents some immune responses, e.g.

Defining a Simulation Strategy for Cancer Immunocompetence

9

anti-tumoural ones. Also, the reduction of Treg cells might compromise the activation of immune responses in the aged. Therefore, an imbalance in Treg normal functioning can predispose immune dysfunction. This results in a higher risk of immune-mediated diseases, cancer or infections. 3.5

Discussion

A summary of the main characteristics of the candidate models described in the previous sections is presented in Table 1. Table 1. Main characteristics of the candidate models Theories Characteristics Lack of Naive Space Filling Innate Up-regulation Treg Cells Shrinkage of naive cells × × Diversity decrease × × Few clones taking space × × Excessive memory cells × Loss of clones × Inflammation × × Excessive T suppression × × Degeneration × × × × Auto-immunity × × × × Less vaccine response × × ×

Recent research [2] states that the stimulation of high-avidity T cell receptors responses is essential for effective anti-tumour vaccines. High-avidity responses are capable of efficient anti-tumour activity in vitro and in vivo. However, in order to have effective vaccinations, there should be sufficient T cell response around tumours. As we have discussed, when the organism ages it lacks T cells. Thus, we would like to know how critical immunosenescence is for T cell responses to anti-tumour vaccination in cancer treatments. In order to proceed with this study, we have chosen the first candidate model of immunosenescence, because it has the most direct focus on T cells which have a direct relation to cancer immunocompetence. The number and phenotypical variety of naive T cells in an individual represents one of the main factors highlighted as influence in the process of immunosenescence. This number changes with age in quantity and diversity. It is also one of the first immune-components to show signs of ageing. Therefore, we believe that T cell responses to cancer might deteriorate with age. By understanding how T cell populations change over the years, we could get insights of what kind of T cell response there will be inside tumoural sites and towards vaccination. In order to understand T cells dynamics with time, we developed an agent based simulation model presented in the next section. The simulation is based

10

G.P. Figueredo and U. Aickelin

on data and equations obtained in [3]. The future objective is to use this agentbased model as an input for another model involving interactions of T cells and cancer.

4

Model

In this section we present the conceptual model together with the mathematical model and show how we transformed this into the agent based simulation that we performed. 4.1

Naive T Cell Output

A good indicator of thymic contribution to naive T cell output in an individual is the level of a particular biological marker called ’T cell receptors excision circle’ (TREC). TREC is some circular DNA formed during the coding of Tcell receptors. The TREC percentage on a T cell decays with shrinkage of the thymic output and with the activation and reproduction of naive T cells [3]. This means that naive T cells originating from the thymus have a greater percentage of TREC than those originating through other proliferation. Our model proposed here is based on data and equations obtained from [3], which is concerned with understanding naive T cell repertoire dynamics. The objective of Murray’s model is to determine the likely contribution of each source of naive T cells, by comparing estimates of the presence of TREC in these cells (see Figure 1). The dynamics of the sustaining sources, i.e. naive proliferation, TREC and reversal of memory to naive T cells are each modelled mathematically.

Fig. 1. Dynamics of Naive T cells

Defining a Simulation Strategy for Cancer Immunocompetence

4.2

11

The Mathematical Model

The mathematical model proposed in [3] is described by the equations (1) to (7) below. In these equations, N is the total number of naive cells of direct thymic origin, Np is the number of naive cells that have undergone proliferation, A is the number of activated cells, M is the number of memory cells and t is time (in years). The first differential equation is: dN = s0 e−λt t s(Np ) − [λn + μn g(Np )]N, dt

(1)

where: s0 is the thymic output ; λt is the thymic decay rate, s0 e−λt t s(Np ) represents the number of cells that arise from the thymus where s(Np ) is the rate of export of the thymus defined by: s(Np ) =

1 1+

(2)

s¯Np N¯p

As in the beginning of life there are no naive cells from proliferation, i.e. Np , this parameter will have no influence on the output of the s function and therefore also no influence on the death rate of naive cells from the thymus. This can be better understood when we substitute Np by a number close to zero in the equation: S(Np ) = 1/(1 + s¯Np /N¯p ) = 1/(1 + 0(number→0) /392) ∼ = 1. Also, λn N represents the naive cells’ incorporation into the naive proliferating pool, where λn is the naive proliferation rate, μn is the thymic naive cells death rate, μn g(Np )N represents the naive cell death rate where the function g(Np ) is the rate of change between naive TREC-positive and naive TREC-negative, defined as: g(Np ) = 1 +

1

bNp N¯p N + N¯pp

,

(3)

N¯p and s¯ are equilibrium and scaling values respectively. For the simulation, s0 was defined as: 2

2

s0 = 0.82(7024e(−((t−12.02)/3.623) ) + 5.203 × 105 e(−((t+127.8)/64.47) ) + 2

1937e(−((t−7.357)/6.03) ) + 1.259 × 1018 e(−((t−1309)/214.4)

2

))

The second differential equation is: dNp = λn N + [ch(N, Np ) − μn ]Np + λmn M dt

(4)

12

G.P. Figueredo and U. Aickelin

where: c is the proliferation rate, ch(N, Np )Np represents the naive proliferation where h(N, Np ) is the dilution of thymic-naive cells through proliferation defined by: 1 h(N, Np ) = (5) N +Np 1 + N¯p μn Np is the death rate of naive cells originated from proliferation and λmn is the reversion rate from memory into Np . The differential equation for memory cells (M) is: dM = λa A − μm M − λmn M, (6) dt where: λa is the reversion rate into memory and μm is the death rate of memory cells. The final differential equation concerns the active cells (A) and is: dA = λN a N + λN p N p − (λa + μa )A. dt

(7)

where: λN a is the activation rate of naive cells from the thymus (N). As we know, these cells have to proliferate before being activated. Therefore λN a was set to zero. λN pA is the activation rate of the naive cells from proliferation (Np ). μa is the death rate of active cells set to 44.4 given their short life span. All rate values and other parameters for the model can be seen in Table 2. The values for these rates and parameters have been obtained by [3], through sensitivity analysis and parameter optimisation of the model. These parameter values, when used in conjunction with the above mathematical model, produced the results that most closely matched real-life observations of cell numbers[3]. Thus we use the same rates and parameter values in our model. Table 2. Parameter values for the mathematical model parameter value(s) λt λn μn c λmn μm λNa λNpA s λn λmn Np b

log(2) (year −1 ) 15.7

0.003 4.4 μn (1 + 0 0.05 0 0.1 0 0.003 0 392 4.2

700 ) N¯p

Defining a Simulation Strategy for Cancer Immunocompetence

13

In the next section, we will show how we can build an agent-based model representing equations (1) to (7) in order to investigate if it is possible to reproduce and validate the results obtained by [3]. 4.3

The Agent Based Model

To convert the conceptual model into an agent based model, we first have to decide on what are the agents and their states: T cells are the agents and can assume four states, i.e. naive, naive from proliferation, active or memory. The agents’ state changes and their death are defined by the ratios given in the mathematical model. Initially, all the agents are in the naive state. As the simulation proceeds, they can assume other stages according to the transition pathways defined in the conceptual model of figure 1. The simulation scenario studied alters the function g over time by setting the parameter b greater than zero (b = 4.2). This means that the death rate of naive T cells from thymus will increase through the years as the number of naive cells from peripheral proliferation increases. There is no change of the thymic export, no reversion from memory to a naive phenotype and the conversion rate of naive cells from thymus to naive cells from proliferation is low (equal to 0.003). The simulation was run for a period of one hundred years and it considered the impact of thymic shrinkage per mm3 of peripheral blood and 2000 initial naive cells from thymus. The data collected for validation is plotted in the graph shown in Figure 2. The results from the simulation can be seen in Figure 3. The results show three different curves representing naive T cell populations derived from the thymus,

Original data collected for validation 1 Log10 percentage T cells from Thymus Naive % CD4+ data

0.9 0.8

Proportion of cells

0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0

10

20

30

40

50

60

70

Age

Fig. 2. The dataset used for validation was provided by [3] and [12]

14

G.P. Figueredo and U. Aickelin

Results for the ABS model − scenario 3 1 Naive T cells Naive from Proliferation Total T Cells Log10 percentage T cells from Thymus Naive % CD4+ data

0.9 0.8

Proportion of cells

0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0

20

40

60

80

100

Age

Fig. 3. Results for the agent based simulation 4000 3500 3000 2500 2000 1500 1000 500 0

0

20

40

60

80

100

Fig. 4. Estimations of the amount of memory T cells with age

peripheral naive T cells and the total number of naive cells. The curve representing naive cells from the thymus presents decay at the beginning of life followed by some interval of stability. By the age of twenty, the thymic export decreases following an exponential trend. The results show that with the decay of naive cells derived from thymus, the naive repertoire changes from the thymic source to the peripheral proliferation source. And although the numbers of naive cells tend to be stable over time, there is no new phenotypical naive cell entering the system. This phenomenon reinforces the ideas of clone degeneration and space taken over by a few phenotypic clones.

Defining a Simulation Strategy for Cancer Immunocompetence

15

As an extrapolation of the model, if we consider that around ten percent of the total T cell population is active in the organism during its lifetime, we can roughly get an idea of the behaviour of memory cells over time. This behaviour is shown in figure 4.

5

Conclusions

New trials to combat cancer are intending to take into account the individual patient’s tumour biology and immune responses. In vitro stimulation of effector T cells has shown success in tumour treatments. To proceed with anti-tumour vaccinations in vivo it is necessary to determine the patient’s immunocompetence profile. We believe that this profile and the patient’s ability to fight cancer is influenced by immunosenescence and can be computationally modelled and simulated. We have shown in this paper how important factors related to immunosenescence that have a direct influence on T cell responses to cancer and vaccination can be investigated in a simulation model. In our research, the aim is to establish correlations between age, T cell populations and cancer immunocompetence. What we have done to date is to understand how T cell populations change with time. The simulation model that we built and studied is based on mathematical equations converted into an agent based simulation. Results fit the observed data and thus we can estimate the likely contribution of each of the naive T cell repertoire maintenance method. With the decay of naive cells derived from the thymus, the naive repertoire changes from the thymic source to the peripheral proliferation source. The numbers of naive cells tend to be stable over time, but there is no new phenotypical naive cell entering the system. The lack of new T cell phenotypes probably makes vaccination and responses to cancer less effective. We believe that the results of this model could be used as an input to a bigger model, involving the movements of T cells inside tumours. We could have built other types of simulation models based on the mathematical equations, and indeed have done so elsewhere, e.g. we have built a system dynamics model of the same problem for the 2010 Summer Simulation conference. Building a system dynamics simulation is a more straightforward task, as it readily lends itself to simulating differential equations. Agent-based models on the other hand require different conceptual models, including states and their transitions. This is non-trivial, but agent-based models do have the advantage of being more flexible. For instance, in future we intend to extend the model to include notions of space and movement (e.g. inside a tumour). Such additions are not easily possible when using other simulation paradigms. Moreover, agent-based simulations do not only work on an aggregate level, i.e. it is also possible to follow individual cells. We believe this point will hep us not only in our understanding and validation of future models, but also assist us in our explanations to non-computer scientists.

16

G.P. Figueredo and U. Aickelin

In order to move this research forward, the model needs to be extended to address the interactions between T cells and tumoural cells. We want to investigate how T cells would respond to vaccination and fight tumoural cells. We also want to look at the individual behaviour of T cells inside different types of tumours and cancers. Therefore, we believe that the choice of an agent-based model is the most suitable for our future work. Other future questions we intend to answer include the usability of the simulation by clinicians and which model would be more suitable for their understanding of the underlying immunology. Specifically to continue the cancer model, we intend to use the actual mathematical model as part of a bigger set of equations representing T cell responses inside tumours. Data about T cell responses in colorectal cancer has been collected and will be used for a new mathematical model. Once we have established how T cells respond to cancer and stimulation, an additional factor will be added. We will consider the influence of age in quantity and quality of responses. We believe that this model could provide clinicians with information about the suitability and advantages of cancer vaccination in patients with a particular immunocompetence profile.

References 1. Metheringham, R.L., Pudney, V.A., Gunn, B., Towey, M., Spendlove, I., Durrant, L.G.: Antibodies designed as effective cancer vaccines. mAbs 1, 71–85 (2009) 2. Pudney, V.A., Metheringham, R.L., Gunn, B., Spendlove, I., Ramage, J.M., Durrant, L.G.: DNA vaccination with T-cell epitopes encoded within Ab molecules induces high-avidity anti-tumor CD8+ T cells. European Journal of Immunology 40, 899–910 (2009) 3. Murray, J.M., Kaufmann, G.R., Hodgkin, P.D., Lewin, S.R., Kelleher, A.D., Davenport, M.P., Zaunders, J.: Naive T cells are maintained by thymic output in early ages but by proliferation without phenotypic change after twenty. Immunology and Cell Biology 81, 487–495 (2003) 4. Bulatti, M., Pellican, M., Vasto, S., Colonna-Romano, G.: Understanding ageing: Biomedical and bioengineering approaches, the immunologic view. Immunity & Ageing 5 (2008) 5. Franceschi, C., Bonaf, M., Valensin, S.: Human immonosenescence: the prevailing of innate immunity, the failing of clonotypic immunity, and the filling of immunological space. Vaccine 18, 1717–1720 (2000) 6. Wu, D., Meydani, S.N.: Mechanism of age-associated up-regulation in macrophage PGE2 synthesis. Brain, Behavior, and Immunity 18, 487–494 (2004) 7. Martinis, M.D., Franceschi, C., Monti, D., Ginaldi, L.: Inflamm-ageing and lifelong antigenic load as major determinants of ageing rate and longevity. FEBS 579, 2035–2039 (2005) 8. Eric Boren, M.E.G.: Inflamm-aging: autoimmunity, and the immune-risk phenotype. Autoimmunity reviews 3, 401–406 (2004) 9. Butcher, S., Chahel, H., Lord, J.M.: Ageing and the neutrophil: no appetite for killing? Immunology 100, 411–416 (2000)

Defining a Simulation Strategy for Cancer Immunocompetence

17

10. Agrawal, A., Agrawal, S., Tay, J., Gupta, S.: Biology of dendritic cells in aging. J. Clin. Immunol. 28, 14–20 (2007) 11. Sharma, S., Dominguez, A.L., Lustgarten, J.: High accumulation of T regulatory cells prevents the activation of immune responses in aged animals. The Journal of Immunology 177, 8348–8355 (2006) 12. Cossarizza, A., Ortolani, C., Paganelli, R., Barbieri, D., Monti, D., Sansoni, P., Fagiolo, U., Castellani, G., Bersani, F., Londei, M., Franceschi, C.: CD45 isoforms expression on CD4+ and CD8+ T cells throughout life, from newborns to centenarians: implications for T cell memory. Mechanisms of Ageing and Development 86, 173–195 (1996)

Clonal Selection from First Principles Chris McEwan and Emma Hart Edinburgh Napier University {c.mcewan,e.hart}@napier.ac.uk

Abstract. Clonal selection is the keystone of mainstream immunology and computational systems based on immunological principles. For the latter, clonal selection is often interpreted as an asexual variant of natural selection, and thus, tend to be variations on evolutionary strategies. Retro-fitting immunological sophistication and theoretical rigour onto such systems has proved to be unwieldy. In this paper we assert the primacy of competitive exclusion over selection and mutation; providing theoretical analysis and empirical results that support our position. We show our algorithm to be highly competitive with well-established approximation and learning techniques; notably for large, high-dimensional problems. We further argue that it provides concise, operational abstractions of some influential theoretical immunology.

1

Introduction

Burnet’s Clonal Selection principle is the keystone of both mainstream theoretical immunology and immune-inspired computational systems. Briefly, antigen select their responding lymphocyte clones through a cyclic process of receptorligand binding, proliferation, mutation and competitive exclusion. Thus, randomly generated lymphocytes, with receptors proven “fit” in the pathogenic environment of the host, persist and improve. To date, the computational perspective on clonal selection has, overwhelmingly, been based on the genetic algorithm without crossover model proposed almost 20 years ago by Forrest et al. [16]. This work was not itself motivated by solutions to computational problems, but proved to be seminal in inspiring research in that direction (e.g. [10,4]). Despite subtle differences in implementations, these algorithms all have the same basic features: a cloning operator that reproduces solutions and a hypermutation operator that performs mutation. Although such algorithms differ from classical evolutionary algorithms in their lack of a crossover operator and in the use of operators that are a function of solution fitness, the underlying computational model remains essentially the same [30]. A second sizable branch of computational work with clonal selection has been directed at producing unsupervised learning algorithms [11,41]. Much of the seminal research here also built upon the work of Forrest et al. – focusing first on the pattern-matching nature of receptor-ligand binding, then developing these ideas into clustering and classification in RN . The resulting algorithms tend to share commonalities with Self-Organising Maps, Learning Vector Quantization E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 18–32, 2010. c Springer-Verlag Berlin Heidelberg 2010 

Clonal Selection from First Principles

19

or instance-based methods from statistical learning [21]. Stibor and Timmis [35] were the first to provide a negative result in this area, showing why the aiNet algorithm is incapable of achieving its stated goal of data compression. McEwan and Hart more recently elaborated several other flaws that potentially undermine the basic principles underlying this family of algorithms [27]. In contrast to the above, much of the seminal work that preceded the formation of artificial immune systems as a recognised field (e.g. [2,15]) very much focused on applying and exploring computational properties of biologically plausible dynamical systems; rather than re-contextualising the immunology into existing algorithmic approaches. It is our position that this was, and still is, a better way to produce artificial immune systems. In this paper, we demonstrate how such methods (and a little hindsight) improve not only biological plausibility, but theoretical approachability and computational effectiveness. The paper develops as follows: in the next section we motivate and introduce our dynamical system perspective of clonal selection as competitive exclusion. In Sect. 3 we theoretically analyse the approximatory capacity of our model and empirically demonstrate its effectiveness in the domain of sparse approximation. In Sect. 4 we extend this approximatory behaviour to decision making, cast in a statistical learning framework; and again, provide empirical support of our position. In Sect. 5 we briefly discuss the biological plausibility of our model, before concluding in Sect. 6.

2

Clonal Selection Is Competitive Exclusion

To date, the emphasis on clonal selection as algorithm has been on the generateand-filter approach typified by Darwinian selection. This emphasis has been almost exclusively placed on the generate aspect, with much attention given to receptor (cf. genotype) representation and mutation strategies. The and-filter aspect has received very little attention and any attention it has received tends to also be imported from evolutionary computing through selection and retention operators such as elitism, tournament selection and so on. Although the generation of novel and diverse components is a critical aspect of clonal (and natural) selection, here we will we concentrate on filtering. In the particular case of the learning domain, poorly realised interaction mechanisms lie behind several of the negative results already discussed. In contrast to selection operators, biologists have very simple and elegant models of inter-species competitive exclusion [31]. The one we will focus on is paradigmatic, originally formulated by Alfred Lotka and Vito Volterra [24,40], and later developed by many others, notably Levins [23], Roberts [34] and Nowack [32]. This generalised Lotka-Volterra model is as follows: we posit each species has a carrying capacity that the environment can support; each species population grows towards that capacity under the classic sigmoid-shaped logistic equation; reaching this capacity is further hindered by interactions from other species competing for the same resources. More formally, let ρi and ki represent the population and carrying capacity of the i’th species, respectively. The so-called “community matrix” K represents

20

C. McEwan and E. Hart

the competitive effects of species i on species j, that is Kij ≥ 0 and Kii = 1. The population dynamics evolve as follows   ki − σi dρi = (1) ρi dt ki  where σi = j Kij ρj . It is apparent that when ki = σi the capacity is equal to the competitive effects and that species reaches equilibrium. If ki < σi then the species is out-competed and declines. If ki > σi the species grows smoothly towards its, now reduced, capacity. It is straight-forward to add additional factors such as growth-decay rates, immigration-emigration terms and predator-prey interactions, but they add little to the immediate exposition. 2.1

Adding Immunological Detail

When modelling clonal selection, our species will be lymphocyte clones differentiated by receptor configuration. It is intuitive that the receptor configuration will determine both capacity (antigen-receptor potential) and competition (receptorreceptor redundancy) as binding-sites are a limited resource. The common bit-string and geometric models of ligand binding have been criticised on both biological and computational grounds [3,17]. In contrast, we will attempt a general definition, transparent to both biologists and computer scientists, by modelling epitopes as abstract peptides localised on the surface of the tertiary structure of proteins (see [28] for a deeper discussion). Let us assume n such peptides. We then conceptualise receptors ϕi as n-dimensional vectors with most components zero or negligible. Non-zero components indicate some affinity to that particular peptide; but binding is a function of multiple peptides being correlated on the protein surface. We model the surface of a protein as a square-symmetric matrix P where Pij represents the surface correlation of peptides i and j. Our measure of affinity, or binding strength, is then naturally embodied in the product ϕi |P |ϕi  which is the magnitude of ϕi in the subspace of Rn defined by P . We will further assume that surfaces are additive, in which case the entire antigenic environment of the immune system is concisely repre sented as the matrix Q = Pk , the aggregation of individual protein  surfaces. Thus, a clone’s capacity can now be defined as ki = ϕi |Q|ϕi  = ϕi |Pk |ϕi . That is, magnitude in the antigenic environment. Competitive effects between clones are quite naturally embodied in receptor redundancy, which we again represent abstractly as correlation ϕi |ϕj  ∈ [0, 1]. Thus the competitive effect on clone i is an aggregate measure of redundancy and competitor fitness σi = j ϕi |ϕj ρj , which includes a clones “competition” with itself ϕi |ϕi ρi . We collect these correlations in a matrix K that readily satisfies the conditions for Eq. (1).

3

Competitive Exclusion for Approximation

Our first goal is to demonstrate the empirical and theoretical efficacy of our simple model in a non-decision-making setting. Approximation is the perfect

Clonal Selection from First Principles

21

domain for this: on the one hand, it can be a precursor to good decision making; on the other, it is widely applicable in applied and theoretical science. This further illustrates clearly how the proposed method deviates from the state of the art in AIS applied to unsupervised learning. The classical approximation problem formulation is to minimise the metric distance between a given vector or function x and its approximation x ˜ chosen from some set of elements. Of particular interest here will be additive expansions of basis functions ϕi ∈ Φ such that  αi ϕi = Φα (2) x≈x ˜= i

The classic metric of error is the 2 norm, leading to a least-squares problem argmin x − Φα22 α

(3)

which, if Φ is an orthonormal basis, has the particularly convenient solution α = ΦT x, that is, αi = ϕi |x. In this case the approximation is exact and easy to compute. But this convenience comes with two undesirable conditions: 1. The constraint of pairwise orthogonality severely limits the form (and amount) of components in the additive expansion [7]. This makes representing some signals extremely convoluted (e.g. representing a sharp, temporally localised wave with periodic functions). This is also a problem when the coefficients of ϕi are to be interpreted (e.g. representing data as a sum of latent factors). In both cases, it is desirable to expand the number and diversity of columns of Φ, resulting in redundant, overcomplete representations [1]. 2. Any least-squares solution α will be dense, that is, every basis will contribute to the approximation. In many domains, assuming sparsity in the coefficients is either reasonable or highly desirable. For example, in statistics, one might appeal to parsimony of the model (i.e. feature selection); in signal processing, an appropriately chosen basis may induce the representation coefficients to rapidly approach zero, allowing truncation with little perceptible loss in reconstruction (i.e. lossy compression). The ubiquity of these conditions leads to sparse approximation. Stated as an optimisation objective, it is essentially a regularised variant of Eq. (3) that can be used to finesse the over-determined nature of (1) and bias the solution of (2) towards extremal coefficient values argmin α s.t. x − Φα22 <  α

(4)

In principle, the sparsest solution to Eq. (4) can be quantified using the 0 pseudo-norm, which counts the non-zero coefficients in α. Unfortunately, the combinatorial nature of this formulation makes it NP-Hard [29]. Briefly, there have been two major thrusts at attacking this problem. Donoho [12] was the first to show that the 0 and 1 solutions coincide when −1 , where M is the “coherence” of Φ defined as maxi=j ϕi , ϕj . α0 < 1+M 2

22

C. McEwan and E. Hart function mp (x, Φ) r=x α = [] while r2 >  do i = argmaxi ϕi |r αi = ϕi |r r = r − αi ϕi end return α

Algorithm 1. Matching Pursuit. Repeated subtraction of the most correlated atom with the residual error. In the classification and regression setting, variations on this algorithm are Least Angle Regression and 2 -Boosting.

Using the 1 norm, it is (somewhat) straight-forward to relax this combinatorial optimisation into a quadratic program with linear equality constraints (see e.g. [38,5]). In the signal processing literature, this method is known as Basis Pursuit [6]; in statistical learning, it is called the Lasso [37]. Unfortunately, this rigorous approach is prohibitive computationally and scales very poorly. The second approach uses heuristic, greedy algorithms to construct a sparse representation sequentially. Mallat and Zhang’s [25] Matching Pursuit algorithm holds a special place in the literature. It is simple, intuitive, and has a rich history within, and outside of, the field [33,13,20,19]. We outline the procedure in Alg. (1): the residual error r is repeatedly stripped of structure correlated with bases until a stopping criteria is satisfied (e.g. number of chosen bases, norm of the residual etc). In regression and classification problems, this approach is known as Forward Stepwise Regression and 2 -Boosting, respectively. A modern variation on this idea, Least Angle Regression [14], avoids overly-greedy steps based on ϕi |rt , favouring instead to increase αi until ϕi is no longer the most correlated with r; at which point a “competing” predictor is introduced to the representation. It is this notion of competition amongst predictors, bases or classifiers that we wish to develop here, albeit without myopic greediness. 3.1

Competition and Approximation

Returning to our dynamical system of Eq. (1) we now elaborate on the approximatory capacity of the repertoire under our formulation as a basis. If Φ was orthonormal, then by definition there are no competitive effects between clones. The dynamics of ρ smoothly approaches equilibrium where ρi = ki = ϕi |x, as would be expected from any orthonormal system. For a redundant, overcomplete repertoire, there is a trade-off for clones: growth requires maximising correlation with the antigenic environment (capacity) and minimising correlation with other clones (competition). This makes sense in the approximation setting: the former reduces reconstruction error; the latter promotes “almost orthogonality”, penalising representational redundancy and rewarding diversity. In contrast to Donoho’s coherence (and similar measures) we need not expect our

Clonal Selection from First Principles

23

basis to satisfy almost orthogonality a priori; rather the competition dynamics promote satisfaction in the context of individual signal decompositions. We can clarify the approximatory behaviour of the repertoire by simplifying capacity from ki = ϕi |X|ϕi  to ki = ϕi |x1 . It then follows that all numerators in Eq. (1) can be written simultaneously as k − Kρ = Φ x − Φ Φρ

(5)



= Φ (x − x˜) which clearly stabilises when x = x˜. Note that this is essentially a restatement of the least-squares solution ρ = (Φ Φ)−1 Φ x as (Φ Φ)ρ = Φ x. Rather than inverting a matrix we are iterating Φ Φρ . The logistic equation further introducing non-linearity and “lag” into Eq. (1) and the stable configuration is not the least squares solution. The above derivation also shows that the competition vector ˜ is equivalent to receptor correlation with the implicit approxiσ = Φ Φρ = Φ x mation x ˜. Any clone more correlated with the approximation will decelerate and decay, again, penalising redundancy in the representation. Extending to the matrix setting ki = ϕi |X|ϕi  carries several benefits. With vectorial data, squaring the capacity promotes additional sparsity by penalising unfit and boosting fit clones (demonstrated later). Further, when X is an aggregated quantity the matrix representation more correctly represents the sum of correlated “features”, particularly in sparse data environments. Further still, it accommodates a richer matrix-based data representation, while degrading gracefully in the standard vectorial setting. 3.2

Competition and the Greedy/Global Trade-Off

We have shown that, under a suitable formulation, a repertoire of competing receptors can collectively approximate the antigenic environment. We have also highlighted the similar form to the traditional least squares solution and the crucial sparsity inducing non-linearities. We will empirically validate these properties in the next section. Here we elaborate on where this method is positioned in the spectrum of greedy heuristics and global optimisations. To simply our analysis, we will again let ki = ϕi |x and retain Kij = ϕi |ϕj . Let max(·) return the index of the maximum component, rather than the value. Now observe that in Alg. (1) the index i1 in the first iteration will be max(k), simply because r0 = x. On the second iteration i2 = max(Φ r1 ) = max(Φ (x − ϕi1 |xϕi1 )) = max(Φ x − ϕi1 |xΦ ϕi1 ) = max(k − ϕi1 |xKi1 ) where Ki1 refers to the i1 column of K. In general, we have 1

In the traditional vectorial data-analysis setting this is reasonable. Using an outerproduct to represent our “surface” ϕi ||xx||ϕi  visibly reduces to ϕi |x2 . Thus we are simply using the square root of the matrix representation in our current analysis.

24

C. McEwan and E. Hart

it+1 = max(k −

t 

ϕit |xKit )

(6)

j=1

= max(kt ) where kt = kt−1 − ϕit |xKit . What this derivation makes explicit is the implicit role that inter-basis correlation plays in the evolution of the Alg. (1). When a basis ϕit is selected, those correlated with it suffer a drop in their capacity proportional to their correlation with the signal in the subspace of ϕit kj(t+1) = kj(t) − ϕj |ϕit ϕit |x

(7)

Crucially, notice that we are now dealing solely the same quantities used in Eq. (1) – capacity and competition. If we expand Eq. (5) as  ρj K j (8) k − Kρ = k − j∈Φ

then it becomes clear that while Alg. (1) greedily sums over the current selections, weighting by the maximal coefficient values (Eq. 6); in contrast, competitive exclusion sums over all dictionary atoms, weighting by the current coefficient values ρi . The rest of Eq. (1) simply provides an update rule to have ρi → ki , subject to competitive effects. So, in contrast to the myopic selective process of Matching Pursuit, Eq. (1) uses a more informed eliminatory process – evolving coefficients, i.e. population sizes, in parallel as a dynamical system. Clearly, simulating the entire potential repertoire is impractical and implausible. Preliminary results (not shown) suggest that redundancy ensures results are not degraded by simulating a fixed-size repertoire with sampling from the full Φ. Evolutionary search through Φ is an obvious next step in development. 3.3

Approximation Experiments

In the following experiments we follow a standard protocol. First we randomly generate an arbitrary over-complete “basis”. From this basis we generate synthetic signals, each a sparse linear combination of basis vectors, adding a small amount of Gaussian noise. We then compare the approximations found by Matching Pursuit (greedy), Basis Pursuit (global) and competitive exclusion. For each algorithm, we record the summary statistics (max, min, quartiles, mean and variance) averaged over 100 signals for – – – –

Sparsity: number of non-zeros components α0 . CPU: time to produce a representation. Reconstruction Error: x − Φα22 Synthetic Error: β − α22 , described below.

Squared reconstruction error is the de facto metric in these types of experiments. However, reconstruction error is only a proxy measure implying that the algorithm has found a good representation. When using synthetic signals it

Clonal Selection from First Principles

25

is possible to measure the actual error in representation, that is, the error in selected coefficients and their magnitude. We refer to this as Synthetic Error : β − α22 where β is the coefficients used to generate the synthetic datum. In contrast to pure approximation, this metric can be important when the bases have application-specific meaning and their coefficients are to be interpreted. This is illustrated further in Fig. (1). In Figure (2) we graph the performance of the algorithms in approximating 100-dimensional signals, each generated from 10 bases selected at random from a 1000 elements basis. Each population starts with the same initial value (see supplementary code for further details). We also plot two variants of our algorithm to illustrate the trade-off discussed in Sect. 3.1: dp uses the regular dot product as capacity; dp2 squares the dot product. In general, the former is more accurate but slower and denser, the latter is faster and sparser. It is apparent that competitive exclusion is, true to its name, very competitive across metrics. Although simulating dynamics will obviously be computationally more expensive than greedy approximation, the difference is not as large as one might expect. Further it is significantly faster than performing optimisation by linear programming – Basis Pursuit’s CPU time was over 200 seconds and is well outside the bounds of the graph. It is interesting to note that in all other respects dp performs similarly to Basis Pursuit, at a fraction of the computational effort. At the other extreme, we see that although Matching Pursuit achieves very low reconstruction error, it does so at significant cost to synthetic error and sparsity. Recall, each signal is a linear combination of 10 bases. Matching Pursuit, Basis Pursuit and our own algorithm without quadratic capacity all significantly under-estimate the true sparsity, employing between 60 and 90 bases. In contrast, the quadratic capacity version of our algorithm, as suggested in Sect. 3.1, is able to drive the sparsity down to around 20, with negligible effect of synthetic error and a large improvement in time. The cost here is a notable increase in reconstruction error.

4

Competitive Exclusion for Learning

Statistical learning is simply another application of approximation, so we do not have to digress too much. The essential difference is that we are now approximating an unknown function of our signal y = f (x), rather than approximating x itself. However, we may still make use of the coefficients of ϕi in approximating x to weigh its contribution to approximating the decision surface y. In [28], the authors introduce a novel derivation of the linear classifier, with immunological interpretation, based on the spectral decomposition of the data’s feature correlation matrix. The reader is directed to the primary references for the full mathematical and biological justification, here we simply recreate the resultant decision function  ϕi |w ˜ (9) yˆ = f (ˆ x) = ˆ x|XX −1 |Xy = ϕi |ˆ x ϕi |G|ϕi  i

26

C. McEwan and E. Hart

   

















 





















Fig. 1. Illustration motivating the synthetic error metric. Light (yellow) background bars represent each bases correlation with a signal. Heavy (blue) foreground bars represent coefficients recovered by Matching Pursuit. Dots (red) represent generating coefficients. Low reconstruction error does not imply correct recovery of representation.



   




 
 









    













































 
 
 



 










Fig. 2. Approximation results for Matching Pursuit (greedy), Basis Pursuit (global) and variants of our competitive exclusion algorithm. See text for discussion.

Clonal Selection from First Principles

27

 If ϕi were the eigenvectors of G = XX  = |xx| then this would be the ubiquitous linear classifier. For the same reasons given in Sect. 3, we are interested in non-orthogonal, overcomplete bases. Technically, only the eigenvectors are blessed with the property that G = ΦΛΦ and G−1 = ΦΛ−1 Φ because Λ is the diagonal matrix of eigenvalues Λii = ϕi |G|ϕi . Using an argument similar to that for justifying nearest-neighbour decisions, we assert that this technical omission can be ignored if the benefits of expanding Φ overcome any costs in inaccurately approximating G−1 . For classification this is entirely plausible, as approximation errors do not necessarily imply classification errors. The relation to Eq. (1) lies in the denominator ϕi |G|ϕi , which is just the same “surface representation of the antigenic environment” used in Sect. 2.1. Thus we see that the decision function involves two other quantities not present in the purely approximatory setting; but both are still measures of approximax is an intuitive measure of tory capacity, receptor correlation or affinity: ϕi |ˆ correlation with the test datum and is thus not sodifferent to what has come before; ϕi |w ˜ is a measure of correlation with w ˜= yi xi , the vector measuring feature-label correlations; that is, the current bias individual features have towards one decision or the other. The immunological interpretation of this vector is discussed in Sect. 5, here we concentrate on the numerical details. The leap from linear classifier to non-linear dynamical model of an immune response now rests on one simple idea. With a slight abuse of notation, let · be upper-bounded by the dot product. That is, as before, this upper-bound is the capacity; the actual value · takes in Eq. (9) is the equilibrium population emerging from inter-clonal competition dynamics. Thus, according to Eq. (9), the immune response is the integration across the repertoire of three key pieces of information about the fitness of each receptor in competing for (i.e. approximating) different environmental resources: the target of the response x ˆ; the antigenic environment of the host G; and what we will call the context of the response w. ˜ Notice that G is in the denominator, thus large values are penalised. 4.1

Learning Experiments

It is important to note that we do not think it reasonable to expect Eq. (9) to perform well in arbitrary classification tasks; it is designed to exploit the properties of sparse, high-dimensional problems. To validate our algorithm we use a subset of the UCI newsgroups dataset, producing a task of discriminating comp.graphics from alt.atheism postings. The high dimensions, size, sparsity and noise of text data can antagonise classical parametric and non-parametric techniques. In [27], the immune-inspired algorithm AIRS was shown to perform no better than random guessing on a similar dataset. As before we record summary statistics for each algorithm, this time classification accuracy, testing and training CPU time. These are recorded over a 10-fold, unstratified partitioning of 1000 documents. We compare performance of our algorithm against the 3-nearest neighbour classifier and the linear classifier because, much like the comparison in Sect. 3.3, these algorithms represent extremes between which our approach lies. Unlike before, we have no a priori

28

C. McEwan and E. Hart   
































 

  

 











 

















Fig. 3. Learning using competitive exclusion with randomly generated receptors

determined basis to represent the repertoire. In place of more sophisticated methods, we simply generate 10,000 basis vectors, each from a 3-step uniformly random walk on G. Thus each basis has three non-zero components. Note that 10, 000 ≈ 5 × 10−7 of the receptor space and we perform no additional searching. We also do not a priori assess receptors with respect to producing good results. We stress that our goal here is demonstrating the efficacy of competitive exclusion on randomly generated receptors, not optimising performance metrics. We plot our results in Fig. (3). At one extreme, the linear classifier’s low complexity is robust in high dimensions and its parametric model fast to compute decisions. Numerically it is also most accurate, but at the significant cost of inverting either XX  or X  X, neither of which is small. Including the linear classifier is the only reason why the dataset has been limited to 1000 documents. At the other extreme, k-NN is obviously superior in training time (because there is none) but test time suffers due to nearest neighbour calculations. The negative effects of many, noisy dimensions also outweigh any benefit in producing non-linear decision boundaries. Our own algorithm performs comparably to the k-nearest neighbour, albeit with notable variance caused by the simple-minded receptor generation process. However, this variance is not enough to call into question that learning is indeed taking place, even under such severe conditions as uniformly random receptors. Although we generate 10,000 receptors, only about 50-1500 survive the competitive exclusion process. It is perhaps remarkable that so few, 3-dimensional random projections are capable of retaining sufficient representation to compete at all. Notice that, in contrast to the other algorithms, our scaling properties are insulated from data size and, to some

Clonal Selection from First Principles

29

extent, dimension; being dominated by the size of the repertoire (in time) and the density of XX  (in space). Further, our approach directly tackles the curse of dimensionality insomuch as the repertoire is not attempting to fill Rn and the decision function is the aggregation of many low-dimensional decisions.

5

Discussion on Immunological Interpretation

With some exceptions [9,18,22,36] the ecology of lymphocytes has been given little attention in the immunology literature. Our own path to the presented model was driven by a desire to make operational some of the influential rhetoric that has inspired artificial immune systems to date. We discuss this now. Francisco Varela was the original proponent of the cognitive view of the immune system [39]. He posited that the immune system constructs and asserts an internal representation of the immunological self. Quite simply, constructing representations requires building blocks, not prototypical instances, and this is precisely what basis functions are. The approximatory capacity of receptors-asbases is our attempt to quantify these largely philosophical arguments. Similarly, Irun Cohen has promoted cognitive analogies. The central idea behind his co-respondence [8] – that coherent systemic decisions can emerge from the interactions of randomly generated, contradictory and individually weak components – is supported by our results in Sect. 4. Cohen is also a proponent of the blessings of degeneracy and redundancy [42]. Our abstraction of receptors as bases makes these these notions operational too. The benefits of redundant bases is what motivated sparse approximation in Sect. 3. Degeneracy naturally follows from our notion of affinity as magnitude in a projected subspace: all x will intersect with several, but not all, subspaces of the ϕi (i.e. poly-clonality); and each ϕi will intersect with several, but not all, x (i.e. poly-recognition). Degeneracy is impossible when abstracting affinity as pointwise distance in metric shape-space. Lastly, Polly Matzinger’s Danger theory [26] has been contentious in both the immunological and computational literature. Based on our distinction between peptide-fragments and compound epitopes, the so-called context vector w ˜ in Eq. (9) may provide some insight here. In conjunction with epitope binding ϕi |P |ϕi  inducing B-Cells to proliferate and mutate, the so-called second signal that activates B-Cells is triggered, under Matzinger’s hypothesis, by the pro- and anti-response feedback driven by T-Helper and Dendritic cells that, respectively, recognise and sample peptide fragments and chemical indicators of pathology from the tissues. Although we do not model the dynamics of this process, it is easy to see that the feature-class correlations in vector w, ˜ which compete over, fulfil much the same role as the second signal. Certhe ϕi also  tainly, w ˜= xi yi is a gross simplification, but our model is indifferent to how the w ˜ vector is produced. Seen in the light of the decision function Eq. (9), the Danger Theory seems, at least, a pragmatic alternative to more convoluted or teleological explanations of the self-nonself distinction. Of course, we are not immunologists. How well our abstractions may yield to biological realism remains to be seen. But we are not aware of another model that captures so many aspects of the immune response, so concisely.

30

6

C. McEwan and E. Hart

Conclusion

We have reconsidered the foundational principle behind clonal selection, highlighting the primacy of competitive exclusion over selection and mutation. Theoretical and empirical analysis support that our model is computationally effective and formally approachable. In addition, we have argued that it represents a step towards a level of plausibility and sophistication that is lacking in the ad-hoc hybridisation of evolutionary and instance-based algorithms. To be clear, we do not wish to abandon the evolutionary aspect of clonal selection. On the contrary, stochastic search still has a lot to offer in terms of adapting the repertoire which, in both biology and computation, cannot typically be realised whole and evolved in unison. But we think it better that this be an additional feature of artificial immune systems, rather than the foundational principle from which all else follows. In the future we hope to take this work in two directions: integrating evolutionary search back into clonal selection; and developing the dynamical system beyond the rudimentary model presented here.

Reproducible Research All source code used to produce the results and figures in this paper is available from http://www.dcs.napier.ac.uk/∼cs268/mcewan2010principles.tgz

References 1. Aharon, M., Elad, M., Bruckstein, A.: On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them. Linear Algebra and its Applications 416(1), 48–67 (2006) 2. Bersini, H.: Immune Network and Adaptive Control. In: Bourgine, P., Varela, F. (eds.) Toward a practice of autonomous systems, Proceedings of the First European Conference on Artificial Life, pp. 217–226 (1991) 3. Carneiro, J., Stewart, J.: Rethinking Shape Space: Evidence from simulated docking suggests that steric shape complementarity is not limiting for antibody-antigen recognition and idiotypic interactions. J. Theor. Biol. 169, 391–402 (1994) 4. Castiglione, F., Motta, S., Nicosia, G.: Pattern Recognition by primary and secondary response of an Artificial Immune System. Theory in Biosciences 2(120), 93–106 (2001) 5. Cheman, K.M.: Optimization Techniques for Solving Basis Pursuit Problems. Masters, North Carolina State University (2006) 6. Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic Decomposition by Basis Pursuit. SIAM Review 43(1), 129 (2001) 7. Christensen, O.: Frames and Bases: An Introductory Course (Applied and Numerical Harmonic Analysis), Illustrate edn. Birkh¨ auser, Basel (July 2008) 8. Cohen, I.R.: Real and artificial immune systems: computing the state of the body. Nature Reviews Immunology, Group 7, 569–574 (2007) 9. De Boer, R.J., Perelson, A.S.: T Cell Repertoires and Competitive Exclusion. Journal of Theoretical Biology 169, 375–390 (1994)

Clonal Selection from First Principles

31

10. de Castro, L.N., Von Zuben, F.J.: The Clonal Selection Algorithm with Engineering Applications. In: Proceedings of GECCO 2000 (2000) 11. de Castro, L.N., Von Zuben, F.J.: Data Mining: A Heuristic Approach, book chapter/section aiNet: An Artificial Immune Network for Data Analysis, pp. 231–259. Idea Group Publishing, USA (2001) 12. Donoho, D.L.: For Most Large Underdetermined Systems of Linear Equations the Minimal 1 -norm Solution is also the Sparsest Solution. Statistics 40698, 1–28 (2004) 13. Donoho, D.L., Drori, I., Tsaig, Y., Starck, J.L.: Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit (2006) 14. Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. The Annals of statistics 32(2), 407–451 (2004) 15. Farmer, J.D., Packard, N.H., Perelson, A.S.: The Immune System, Adaptation and Machine Learning. Physica 22, 187–204 (1986) 16. Forrest, S., Javornik, B., Smith, R.E., Perelson, A.S.: Using genetic algorithms to explore pattern recognition in the immune system. Evolutionary computation 1(3), 191–211 (1993) 17. Freitas, A.A., Timmis, J.: Revisiting the Foundations of Artificial Immune Systems for Data Mining. IEEE Transactions on Evolutionary Computation 11(4), 521–540 (2007) 18. Freitas, A.A., Rocha, B.: Population Biology of Lymphocytes: The Flight for Survival. Annual Review of Immunology 18, 83–111 (2000) 19. Friedman, J.H.: Greedy function approximation: A gradient boosting machine. Annals of Statistics 29, 1189–1232 (2001) 20. Friedman, J.H., Tukey, J.W.: A Projection Pursuit Algorithm for Exploratory Data Analysis. IEEE Transactions on Computers C-23(9), 881–890 (2006) 21. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, Heidelberg (2001) 22. Leon, K., Perez, R., Lage, A., Carneiro, J.: Modelling T-Cell-Mediated Suppression Dependent on Interactions in Multicellular Conjugates. Journal of Theoretical Biology 207, 231–254 (2000) 23. Levins, R.: Evolution in changing environments, 120 p. Princeton Univ. Press, Princeton (1968) 24. Lotka, A.J.: Analytical note on certain rhythmic relations in organic systems. Proceedings of the National Academy of Sciences of the United States of America 6(7), 410 (1920) 25. Mallat, S.G., Zhang, Z.: Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing 41(12), 3397–3415 (1993), (see also IEEE Transactions on Acoustics, Speech, and Signal Processing) 26. Matzinger, P.: Tolerance, danger, and the extended family. Annual review of immunology 12(1), 991–1045 (1994) 27. Mc Ewan, C., Hart, E.: On AIRS and Clonal Selection for Machine Learning. In: Proceedings of 8th Annual Conference in Artificial Immune Systems (ICARIS). Springer, Heidelberg (2009) 28. Mc Ewan, C., Hart, E.: Representation in the (Artificial) Immune System. Journal of Mathematical Modelling and Algorithms 8, 125–149 (2009) 29. Natarajan, B.K.: Sparse Approximate Solutions to Linear Systems. SIAM Journal on Computing 24(2), 227 (1995) 30. Newborough, J., Stepney, S.: A generic framework for population-based algorithms, implemented on multiple FPGAs. In: Jacob, C., Pilat, M.L., Bentley, P.J., Timmis, J.I. (eds.) ICARIS 2005. LNCS, vol. 3627, pp. 43–55. Springer, Heidelberg (2005)

32

C. McEwan and E. Hart

31. Nowak, M.A.: Evolutionary Dynamics: Exploring the Equations of Life. Belknap Press of Harvard University Press (September 2006) 32. Page, K., Nowak, M.: Unifying Evolutionary Dynamics. Journal of Theoretical Biology 219(1), 93–98 (2002) 33. Pati, Y.C., Rezaiifar, R., Krishnaprasad, P.S.: Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: 1993 Conference Record of the Twenty-Seventh Asilomar Conference on Signals, Systems and Computers, vol. 1, pp. 40–44 (1993) 34. Roberts, A.: The stability of a feasible random ecosystem. Nature 251, 607–608 (1974) 35. Stibor, T., Timmis, J.: An Investigation on the Compression Quality of aiNet. In: IEEE Symposium on Foundations of Computational Intelligence, FOCI 2007, pp. 495–502 (2007) 36. Stirk, E.R., Molina-Paris, C., Van Den Berg, H.A.: Stochastic niche structure and diversity maintenance in the T cell repertoire. Journal of Theoretical Biology 255, 237–249 (2008) 37. Tibshirani, R.: Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 267–288 (1996) 38. Tropp, J.A.: Just relax: convex programming methods for subset selection and sparse approximation (2004) 39. Varela, F.J., Coutinho, A.: Second generation immune networks. Immunology Today 12(5), 159–166 (1991) 40. Volterra, V.: Variations and fluctuations of the number of individuals in animal species living together. ICES Journal of Marine Science 3(1), 3 (1928) 41. Watkins, A., Timmis, J., Boggess, L.: Artificial Immune Recognition System (AIRS): An Immune-Inspired Supervised Learning Algorithm. Genetic Programming and Evolvable Machines 5(3), 291–317 (2004) 42. Wucherpfennig, K.W., Allen, P.M., Celada, F., Cohen, I.R., De Boer, R., Garcia, K.C., Goldstein, B., Greenspan, R., Hafler, D., Hodgkin, P., et al.: Polyspecificity of T cell and B cell receptor recognition. Seminars in immunology 19(4), 216–224 (2007)

Density Preservation and Vector Quantization in Immune-Inspired Algorithms Alisson G. Azzolini1 , Ricardo P.V. Violato1,2 , and Fernando J. Von Zuben1 1

School of Electrical and Computer Engineering - University of Campinas (Unicamp) P.O. Box 6101, 13083-970, Campinas, Brazil {azzolini,vonzuben}@dca.fee.unicamp.br http://www.fee.unicamp.br/ 2 CPqD, Telecommunications Research Center 13086-902, Campinas, Brazil [email protected] http://www.cpqd.com.br/

Abstract. A clustering algorithm may be designed to generate prototypes capable of minimizing the cumulative distance between each sample in the dataset and its corresponding prototype, denoted as minimum quantization error clustering. On the other hand, some clustering applications may require density-preserving prototypes, more specifically prototypes that maximally obey the original density distribution of the dataset. This paper presents a conceptual framework to demonstrate that both criteria are attainable but are distinct and cannot be fulfilled simultaneously. Illustrative examples are used to validate the framework, further applied to produce an adaptive radius immune-inspired algorithm capable of transiting between both criteria in practical applications. Keywords: Immune-inspired algorithm; Data clustering; Vector quantization; Density preservation.

1

Introduction

Data clustering is a powerful data analysis mechanism devoted to revealing some essential aspects of the data structure, particularly subsets of data points with some detectable similarities [5, 7]. These subsets, or clusters, present themselves in number and conformation that can either be set a priori or be left as an outcome of the data analysis. Given that the data space may be of high dimension and a large number of arbitrarily distributed and possibly redundant data points are available, data clustering may be preceded by some compression of the original dataset into a reduced set of prototypes. One way to find good prototypes for representing the data is by using some sort of pattern recognition. That is why immune-inspired algorithms are considered in the literature as proper devices for data preprocessing [1–3, 10]. The data points are fed to these algorithms in the form of antigens. In turn, the algorithm produces a reduced set of antibodies (prototypes) that represent and recognize E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 33–46, 2010. c Springer-Verlag Berlin Heidelberg 2010 

34

A.G. Azzolini, R.P.V. Violato, and F.J. Von Zuben

the set of antigens (data points). The number and location of antibodies is based on shape similarity among antibodies and between antibodies and antigens. There are various ways to measure the performance of such algorithms, and it is not always clear how one relates to the others. Finding correspondences between different criteria, and possibly unifying them, provides a deeper insight into the matter and can lead to the conception of more flexible algorithms. The aim of this paper is to create and test a model for comparing two of these criteria: (i) minimization of the quantization error; and (ii) preservation of the density distribution of the original dataset. The first simply corresponds to minimizing the sum of the distance from each data point to the nearest prototype. The latter, however, has different meanings depending on how density is estimated, and on the method used to compare the estimated density of data and prototypes. Here, we estimate density in a non-parametric fashion: each data point or prototype is considered as an unweighted sample taken from the underlying data and prototype density distributions, respectively. A frequently applied immune-inspired algorithm for data reduction is aiNet (artificial immune network) [2]. In [9], the performance of this algorithm is tested assuming criterion (ii) stated above, with a non-parametric density estimation. The authors showed aiNet does not lead to good density preservation when the antigen distribution is not approximately uniform. Even before this analysis was published, however, the ARIA algorithm had already been proposed [1], in an attempt to generate prototypes respecting the criterion (ii). As an example of application of the model developed here, we present an adaptation of ARIA that can respect one criteria or another, depending on the value of a parameter. The paper is organized as follows: section 2 develops the concepts of density preservation, quantization error and the entropy-based quality measurement; section 3 presents the intuitive difference between minimization of the quantization error and density preservation; section 4 presents a mathematical model that quantifies this difference and gives the density of prototypes which minimizes the quantization error; section 5 presents empirical tests used to validate the model; section 6 shows how the model can be used to adapt ARIA to transit between density preservation and minimization of the quantization error. Concluding remarks and future work are depicted in section 7.

2

Methods to Evaluate the Compression Quality

In this section, we present the mathematical formulation for quantization error and density preservation that will be used in the paper. 2.1

Quantization Error

Given a bounded metric space Ω ⊂ Rl , consider the input dataset N , containing N vectors xi ∈ Ω, and the reduced set M, consisting of M prototypes zi ∈ Ω, where M < N . In possession of M, any vector x ∈ Ω can be approximated by the nearest prototype in M. In other words, M partitions Ω into M Voronoi polytopes, each of which corresponds to the region represented by one prototype.

Density Preservation and Quantization in Immune-Inspired Algorithms

35

Given a metric d(·, ·), that provides a measure of the distortion between two vectors, the quantization error Qi between a vector xi ∈ N and its quantized version q(xi ) ∈ M is defined as Qi = dr (xi , q(xi ))

(1)

In the equation above, the exponent r > 0, proposed in [6] as an extension of the original k-means algorithm, represents the importance given to a certain distance from a data point to its prototype. In most vector quantization tasks, such as the original k-means, r = 2, since this simplifies the algorithms. The mean quantization error QN , which will be used in this paper as criterion (i) (see Introduction), is defined as the arithmetic average distance resulting from representing each point in N by its nearest prototype in M: QN 2.2

N N 1  1  r = Qi = d (xi , q(xi )) N i=1 N i=1

(2)

Density Preservation

We take the definition of density preservation from [9], where it is used to evaluate the quality of the aiNet data compression. The underlying probability distributions of data and prototypes are estimated by a non-parametric method, considering that each point in the sets N and M is independently sampled from its respective probability distribution. Then, a Monte Carlo estimation of the relative entropy (Kullback-Leibler divergence) is used to measure the dissimilarity between the two estimated probability density functions (pdf). Suppose that the pdfs pN (x) of N and pM (x) of M are known. Then, it is possible to measure the dissimilarity H{N , M} between these two data distributions using the relative entropy, suggested in [4] and given by    pN (x) ln (3) H{N , M} = pN (x)dx ≥ 0 pM (x) Ω The more distinct the two datasets, the greater H, being 0 when the two distributions are the same. In possession of N samples xi ∈ N , i = 1 . . . N , the following Monte Carlo approximation can be used [4]:   N pN (xi ) 1  ˆ H{N , M} = ln N i=1 pM (xi )

(4)

However, pN (x) and pM (x) are usually unknown and hence must be estimated. There are two classical approaches to non-parametric density estimation [8]: the k-nearest neighbor (KNN) method and the kernel estimator. The KNN method estimates the density around a point x as: pˆ(x) =

k cl [dk (x)]l

(5)

36

A.G. Azzolini, R.P.V. Violato, and F.J. Von Zuben

where dk (x) is the distance from x to its k-nearest neighbor in a dataset sampled from the distribution being estimated, and cl is the volume of the unit-sphere in Rl . In this paper, the KNN method is preferred over the kernel estimator method because the parameter k is easier to determine than the kernel’s bandwidth, and the smoothness and integrability of the estimator are not crucial in the Monte Carlo approximation of entropy. Moreover, since in this work we only compare densities, cl is taken to be 1. Finally, it should be clear that the results of this paper do not apply to semiparametric models, in which each prototype has additional parameters such as weight and deviation. As an example of this, note that one of the consequences of using non-parametric models is that, in order to preserve density, each prototype must represent the same number of data points, in average. However, this does not hold for semi-parametric models, where the weight of each prototype can be adjusted depending on the number of points it represents. Nevertheless, the results of the paper apply whenever a non-parametric model is necessary or desirable.

3

Quantization and Density Preservation Are Different

Intuitively one can see that the conservation of density leads to a good prototype distribution, at least in the statistical sense, because a peak in the probability density function becomes a high concentrated region, whereas valleys are related to sparser areas in the data space. Hence, positioning prototypes in a way that respects these differences produces a prototype probability density function more similar to the corresponding data function. As it was described in section 2, it is possible to estimate this similarity with the relative entropy. On the other hand, from the point of view of the quantization error, a densitypreserving prototype distribution may lead to a poor result. To illustrate this idea, consider the data distribution showed in Fig. 1. These points were generated from two Gaussian distributions, one of them with mean (0,0) and deviation 1.0 in each axis, and the other with mean (10,10) and deviation 0.5, producing two clearly distinct data clusters. The number of 15

x

2

10

5

0

−5 −5

0

5 x

10

15

1

Fig. 1. Two well defined data clusters, generated from two Gaussian distributions

Density Preservation and Quantization in Immune-Inspired Algorithms

37

points in each one is exactly the same (1000). In this scenario, a good solution from the statistical point of view, restricting ourselves to the non-parametric case, would be to place the same number of prototypes in both clusters, leading to conservation of probability density, and hence minimum relative entropy. It is clear that in the denser region (cluster of smaller variance) the prototypes should be closer to each other when compared to the prototypes in the sparser region, thus guiding to a density preserving solution. However, if the objective is to minimize the quantization error, intuitively, putting more than half of the prototypes in the sparser region seems to be a better approach, since the distance from prototypes to data tends to be greater there. However, there should be a threshold over which the denser region would be too under-represented, increasing the quantization error. Hence, there should be a compromise between the two extremes. In the next section, these intuitive ideas are quantitatively modeled. As a final remark, note that if a semi-parametric model with Gaussian kernel were used instead of a non-parametric model, the data in this illustrative example could be represented by as few as two prototypes, if a different deviation were assigned to each one. However, we restrict the analysis to the non-parametric case, in order to find a mathematical model for comparing the two criteria.

4

Minimization of the Quantization Error

In this section we propose a simplified model that, given the probability density of the data, pN (x), and the probability density of the prototypes, pM (x), propˆ Then, given a data distribution pN (x), erly estimates the quantization error Q. it is possible to find analytically the prototype distribution p∗M (x) that miniˆ This optimum will be given by eq. 15. As this equation shows, following mizes Q. = pN (x), that is, the density of prototypes the model, we generally have p∗M (x)  that minimizes the quantization error will be different from the density of data. We then conclude that density preservation does not lead to minimization of quantization error. In order to develop the model, the whole space Ω will be partitioned into small pieces of infinitesimal volume dV , with constant probability density of prototypes ˆ inside this piece is and of data in its interior. The small quantization error dQ ˆ then estimated and integrated to give the total estimated quantization error Q. ˆ Variational Calculus is used. Finally, in order to find p∗M (x) that minimizes Q, First, let us define the distance metric adopted here. As an abuse of the term, we will use the word distance to denote dr (x, y), that is, the Euclidean distance between the points x and y, raised to the r-th power, as it appears in the definition of the quantization error (eq. 2). The distance is then defined as:  d (x, y) = r

l  i=1

where l is the dimension of the space.

 r2 (xi − yi )

2

(6)

38

A.G. Azzolini, R.P.V. Violato, and F.J. Von Zuben

ˆ inside dV is the product of the mean The infinitesimal quantization error dQ r ˆ distance d between a data point and a prototype in that region, and the mean number of data points dN inside dV . To find dN , we consider that the mass probability of data inside dV is pN (x)dV , where x is any point inside dV . The argument is similar for the number of prototypes dM . Then, dN (x) = N pN (x)dV

dM (x) = M pM (x)dV

(7)

The mean distance dˆr from a data point to its nearest prototype zi depends on the form of the region S represented by the prototype. In general, considering that the region is small enough so that the data density is constant inside S (a simplifying hypothesis), we have:  dr (zi , x)dV r ˆ d (zi , x) = Si  (8) Si dV with dr (·, ·) as defined in eq. 6. We know that the region S is an irregular convex polytope. However, without the exact location of each prototype, there is no way to know the form of each polytope precisely, thus it is impossible to calculate dˆr exactly. As a simplifying assumption, we consider the form of the polytopes to be roughly constant for all prototypes in the whole space Ω. In order to calculate dˆr , suppose two polytopes S1 and S2 of same form, but different volumes V1 and V2 . Since the form does not change, S2 can be obtained from S1 if we make the change of variables x2 = x1 l V2 /V1 . Changing variables r in eqs. 8 and 6 above, after some calculations, we have that dˆr2 = dˆr1 (V2 /V1 ) l . If we take S1 to be a polytope of unit volume, and we define C to be the mean distance from a data point to the prototype in a unit-volume region, we have: r

dˆrzi = CVzil

(9)

where Vzi is the volume of the region represented by the prototype zi . This equation is our estimation of the mean distance dˆr from a data point to its nearest prototype. We see that, in average, not considering the different forms of polytopes, dˆr depends on the volume of the region represented by the prototype. Considering that there are dM prototypes inside dV , the mean volume Vzi of the region represented by a prototype is: Vzi =

dV dM

(10)

We now have everything we need to calculate the infinitesimal quantization error ˆ inside the volume dV . From the definition of quantization error, plus eqs. 7, dQ 9 and 10, we have: r

ˆ = dˆr dN = CVzil N pN (x)dV dQ r

= C[M pM (x)]− l N pN (x)dV = CN M

− rl

r

pM (x)− l pN (x)dV

Density Preservation and Quantization in Immune-Inspired Algorithms

39

r

ˆ inside the Introducing the constant G ≡ CM − l , the quantization error dQ volume dV is: ˆ = GN pM (x)− rl pN (x)dV dQ (11) Integrating all over Ω and dividing by the number of data points N , we have ˆ over Ω: the expected mean quantization error Q  −r ˆ=G Q pMl (x)pN (x)dV (12) Ω

The equation above gives an estimation of the mean quantization error over the space Ω, given only the probability density pM (x) of the prototypes and the probability density pN (x) of the data. This equation cannot be easily used to get ˆ because the constant G is not known. Nevertheless, a reliable estimation of Q, ˆ for a it is possible to use it to find the density of prototypes that minimizes Q given distribution of data. To do this, the following optimization problem must be solved for pM (x):  ⎧ −r ⎪ pMl (x)pN (x)dV ⎪ ⎪ min G ⎨ P1

Ω

⎪ ⎪ ⎪ ⎩ s.t.

(13)

 Ω

pM (x)dV = 1

where the restriction assures that pM (x) is a probability distribution function, that is, its integral over Ω must be equal to 1. The problem P1 can be solved using tools from Variational Calculus [11]. The Lagrangian functional for this problem is r

− f˜(pM (x), x, λ) = pMl (x)pN (x) + λpM (x)

(14)

Since f˜ is strongly convex [11], it can be minimized by solving the resulting EulerLagrange equation. The result is the optimum prototype probability density distribution p∗M (x), given a data distribution pN (x): 

l

p∗M (x)

p l+r (x) = N , K

K≡

Ω

l

l+r pN (x)dV

(15)

and the normalization factor K ≥ 1 is constant over Ω. The above equation is the main result of this section. Clearly, by following eq. 15 with r > 0, there is no conservation of density. A first consequence of this model is that it is not possible to make the prototypes follow the density of the data if the goal is to minimize the quantization error. However, there is a continuum ranging from one objective to the other, obtained by varying r from 2 (minimization of square distance to prototypes) to 0 (conservation of density).

40

A.G. Azzolini, R.P.V. Violato, and F.J. Von Zuben

A related consequence of following eq. 15 is that the relative entropy of eq. 3 is not zero, in general. Indeed,      r  pN (x) l+r H{N , M} = log log KpN pN (x)dV pN (x)dV = pM (x) Ω   Ω r pN (x)dV + pN (x) log pN (x)dV = log K l + r Ω Ω r = log K + H{N } (16) l+r where H{N } > 0 is the Shannon entropy of the data, and K is the normalization factor defined in eq. 15. Since K ≥ 1, it can be shown from this result that H{N , M} ≥ 0 when eq. 15 is followed, with equality only when r = 0. 4.1

Expressing the Results in Terms of a Finite Sample Set

In the first examples of the next section, we will adopt datasets consisting of W data clusters, inside which the density is considered constant, in average. In this ˆ in eq. 12, which leads to a case, it is possible to discretize the expression for Q discrete expression for the optimum prototype distribution of eq. 15. Consider that the i-th cluster has volume Vi , data probability density pdi ≡ pN (xi ) and prototype probability density ppi ≡ pM (xi ), where xi is any point inside the cluster. Then, eq. 12 becomes: ˆ=G Q

W 

−r

ppi l pdi Vi

(17)

i=1

It is possible to express the probability density in terms of the volume Vi and the number of prototypes mi and data ni located inside the i-th cluster: mi ni ppi = pdi = M Vi N Vi Then, the estimated mean quantization error becomes: ˆ = G2 Q

W 

r

l+r

(mi pdi )− l ni l

(18)

i=0 l+r

with G2 ≡ CN − l . Moreover, if the number of data points ni is the same in each cluster, that is, if n1 = n2 = · · · = nW = N/W , then the mean quantization error is W W   r ˆ = G3 Q (mi pdi )− l , mi = M (19) i=0

i=0

l+r l

with G3 ≡ G2 (N/W ) . In this special case, the number m∗i of prototypes that minimizes the above error is: −r

m∗i

p l+r = di , K2

K2 ≡

M −r 1  l+r pdj M j=1

(20)

Density Preservation and Quantization in Immune-Inspired Algorithms

41

The equation above gives the optimum number of prototypes in the i-th cluster for minimizing the quantization error, given the data probability density pdi in that cluster, and considering that the number of data points in every cluster is the same. We can say that this is a discretized version of eq. 15. Before we proceed to the tests, it must be said that the model developed in this section is highly simplified. First of all, the region represented by a prototype was considered to have constant density, what is not always valid, especially for high-dimensional datasets with complex distributions. Also, the form of every polytope is considered to be the same. This is only valid in average, for a great number of prototypes. In the following sections, the model developed here is tested empirically with illustrative, yet relatively high-dimensional datasets.

5

Examples and Tests

Eq. 15, and its discrete counterpart, eq. 20, give the distribution of prototypes that minimizes the quantization error, taking as input only the pdf of the data. In this section, we test the validity of these equations. In each test, we first choose a simple distribution pN (x) for the data, and take a fixed set N of samples from it. Then, various sets Mi of prototypes are composed of samples from N , attributing to each x ∈ N a given probability of being chosen as prototype. Finally, we calculate the resulting relative entropy and quantization error between N and each Mi . The results are compared to the model’s prediction. 5.1

Two Gaussians

In this example, the data distribution is the equally-weighted mixture of two 26 dimensional Gaussians, with means (0, 0 . . . 0) and (10, 10, . . . 10) and with variances 1 and 0.5, respectively. The dataset N is composed of n1 = n2 = 1000 = N/2 samples taken from each of them, forming clusters 1 and 2, respectively. Each set Mi of prototypes is composed of points sampled from the dataset N , in the following way: mi1 points uniformly sampled from the cluster 1, and mi2 from the cluster 2. For each i, we vary mi1 and mi2 , keeping the total number of prototypes constant: Mi = mi1 + mi2 = 100. This way, for each Mi , the relative density of prototypes in each cluster changes. The theoretical quantization error, as estimated by the model developed in last section, is given by eq. 19 with W = 2 and pd2 = 226 pd1 , and is shown in Fig. 2(a) for different number of prototypes m1 and m2 inside each cluster, but always keeping m1 + m2 = 100. The constant G3 is arbitrarily set to 1, hence the value of quantization error shown in the vertical axis of the figure can only be proportional to the experimental one. The experimental mean quantization error, calculated using eq. 2, is shown in Fig. 2(b). The result is the average of five repetitions of the experiment for each Mi . The figure gives also the quantization error resulting after the execution of the Matlab’s k-means and the relative entropy of data and prototypes, with

A.G. Azzolini, R.P.V. Violato, and F.J. Von Zuben 3

6 Quantization Error (r = 2)

Quantization Error (r = 2)

1.3 Model Model’s Optimum

1.2 1.1 1 0.9

0

100

50 m1

QE Before k−means QE After k−means Minimum Relative Entropy (k=8)

5

2.5

4

2

3

1.5

2

0

20

40

60

80

Relative Entropy (KNN estimate)

42

1 100

m

(b) Experiment

(a) Model

Fig. 2. (a) Theoretical quantization error given by eq. 19, assuming G3 = 1, and the theoretical minimum, by eq. 20. (b) Experimental mean quantization error calculated with eq. 2 before and after running Matlab’s k-means. The relative entropy is calculated before running k-means, using k-nearest neighbor for density estimation.

density estimated by the k-nearest neighbors method, with k = 8. The figure shows that the experimental optimum, before running k-means, coincides with the one given by eq. 20. The minimum of the relative entropy, however, is located at the point where the number of prototypes is the same on each Gaussian (m1 = m2 = 50), that is, when density is preserved, as by eq. 16. This is an empirical verification of the model proposed. 5.2

Three Hypercubes

In this example, the dataset N is composed of three clusters: each one has the shape of a 20-dimensional hypercube and is centered in xi = 0 for all dimensions but the first, where the hypercubes are concatenated side by side. Each cluster has a different density of data, as follows: pd1 = 0.520 pd2 = 220 pd3 . The volume of each cluster is inversely proportional to its density, so that the number of data points in each cluster is the same. Each cluster contains n1 = n2 = n3 = 1000 = N/3 uniformly sampled points. Some 2D projections of N are shown in Fig. 3. In order to construct each set Mi of prototypes respecting a given distribution pMi (x), we use the roulette method: a data point xj ∈ N is chosen to be a prototype with probability: Pr[xj ∈ Mi ] =

pMi (xj ) pN (xj )

(21)

Assuming that the model is correct, in order to get the minimum quantization error, the distribution of prototypes should follow eq. 15. Thus, we use it in the equation above, and get the probability of choosing a data point as prototype in order to minimize the quantization error: −r

p l+r (xj ) Pr [xj ∈ Mi ] = N K ∗

(22)

1

0.5

0.5 2

1

0

x

x

2

Density Preservation and Quantization in Immune-Inspired Algorithms

−0.5 −1

43

0 −0.5

0

1

2 x

1

3

4

−1 −1

0 x

1

3

Fig. 3. 2D projections of the first three dimensions of the dataset N sampled from the three uniform clusters in R20 . The cubes are centered in all but the first dimension.

The constant K does not affect the roulette, and can be ignored. Now, for each exponent r, a different distribution Mi of prototypes is generated. If the model is correct, then the quantization error should be minimum when the value of r in the eq. 22, used to generate the prototype distribution, is the same as the r of eq. 2, used to calculate the resulting quantization error. We will now use this fact in order to test the model. The experiment is run five times for three different Mi , each one with different r in eq. 22. The experimental quantization error from eq. 2, however, is always calculated using r = 2. Then, if the model is correct, the error should be minimum for the set M of prototypes generated using r = 2 in eq. 22. The total number of prototypes is always M = 300. The experimental result is shown in table 1. There is a clear reduction of the resulting mean quantization error when r = 2 in eq. 22. Table 1. Resulting number of prototypes mi in each cluster and mean quantization error for sets Mi of prototypes generated by using different values of r in eq. 22. The expected number of prototypes in each cluster m∗i is given by eq. 20. r m∗1 m∗2 m∗3 m1 m2 m3 Qr=2 0 100.0 100.0 100.0 106 ± 6 97 ± 5 97 ± 9 2.20 ± 0.03 1 86.9 44.9 168.2 93 ± 7 42 ± 3 165 ± 10 2.06 ± 0.02 2 62.4 17.7 219.9 61 ± 5 18 ± 5 221 ± 7 1.987 ± 0.008

These results only show that the quantization error is lower when r = 2 in eq. 22 than when density is preserved (r = 0). But, according to the model, the quantization error should be greater for any value of r different of 2. Hence, further tests have been performed. Various sets of prototypes were generated from eq. 22 with values of r ranging from 0 to 5, and the resulting quantization error was then calculated using r = 2 in eq. 2. Each experiment was repeated five times. The mean quantization error over the five repetitions, for each value of r, are shown in Fig. 4. The figure clearly indicates that the minimum quantization error is attained when the r used in eq. 22 coincides with r = 2 used in eq. 2. On the other hand, the relative entropy is minimal when the density is preserved (r = 0), and increases linearly with r, as predicted by eq. 16. For both criteria, the model fits well to the experiment.

A.G. Azzolini, R.P.V. Violato, and F.J. Von Zuben

2.45

10 Mean Quantization Error (r=2) Min/Max Quantization Error Minimum Mean Relative Entropy (k=8)

Quantization Error (r = 2)

2.4 2.35

9 8

2.3

7

2.25

6

2.2

5

2.15

4

2.1

3

2.05

0

1

2

3

4

5

Relative Entropy (KNN estimate)

44

2

r

Fig. 4. The resulting quantization error for different values of r in eq. 22. The mean, maximum and minimum of the error are calculated over five repetitions of the experiment. The minimum of the mean was found when r = 2, as predicted by the model.

6

Using Obtained Results to Modify ARIA

Based on the model and the results found in the previous sections, it is possible to adapt the ARIA algorithm [1] to respond to different objectives, from minimization of the relative entropy to the minimization of quantization error. In ARIA, each antibody i has a radius σi which regulates their mutual suppression, in the following way: if the center of one antibody is inside the hypersphere of a second antibody, the one with larger radius is suppressed. By this suppression rule, plus the cloning, the distance between two nearest-neighbor antibodies tends to be proportional to its radius. Hence, the nearest-neighbor density estimation can be used backwards to give the radius σi which leads to the desired prototype density pM , in the following way: − 1l

σi = ts (pM )

(23)

where ts is a parameter of the algorithm. Now, if we replace eq. 15 into the equation above, we get l σi = ts (pN ) l+r (24) This way, the behavior of ARIA is controlled by ts and its new parameter r, introduced above: if r = 0, then pM = pN and there is preservation of density. If r = 2, the algorithm will organize the antibodies in a way that tends to reduce the mean squared distance from data to prototypes (the mean quantization error with r = 2), which is what the classical k-means does locally. Preliminary tests of the modified version of ARIA were performed using the dataset of two Gaussians in R26 of section 3, in two cases: first seeking to preserve density (r = 0), and then seeking to minimize the quantization error (r = 2). The number of iterations is set to 50, the decay rate to 0.8, with an initial population of one randomly-placed antibody. The parameter ts of eq. 24 is set to 0.868 for r = 0 and 0.95 for r = 2. This configuration leads to a similar number

Density Preservation and Quantization in Immune-Inspired Algorithms

45

of antibodies in the two cases, even though r is different. Differently from the original ARIA, which uses the histogram method, pN is estimated using the k-nearest neighbor method with k = 8. Results are shown in Table 2. At least in this simple dataset, eq. 24 indeed leads the algorithm to respect the desired density pM of antibodies, given by eq. 15. This is evident by com1 paring the ratio m m2 of antibodies over each Gaussian with the optimum shown in Fig. 2(a). Moreover, the execution with r = 2 results in a lower quantization error for the resulting set of antibodies, even if the total number of antibodies is lower than the one for r = 0. The relative entropy, however, is lower when r = 0. Table 2. Results of running ARIA 10 times with different r in eq. 24. m1 and m2 are the number of antibodies over each Gaussian, M = m1 + m2 , Qr=2 is the mean ˆ is the estimated relative entropy (KNN with k = 8) quantization error and H ˆ r M m1 m2 Qr=2 H 0 163 ± 23 80 ± 9 84 ± 14 3.61 ± 0.08 3.5 ± 0.7 2 159 ± 33 134 ± 25 25 ± 8 3.50 ± 0.09 5.5 ± 0.9

7

Concluding Remarks and Future Work

A model was developed to find the overall distribution of antibodies which minimizes the quantization error, given a distribution of antigens. The model was tested, and verified at least for simple datasets. Finally, an adaptation to the algorithm ARIA was proposed taking into account the results obtained. The analytical results of section 4 show that, in non-parametric compression algorithms such as aiNet and ARIA, preserving the relative density of data (antigens) and prototypes (antibodies) is not a good strategy if the goal is to minimize the quantization error. Even if the model works well for simple highly dimensional datasets and also for some datasets in lower dimensions, questions remain about the limitations of the model for real datasets with estimated distribution. As it is well known, the density estimation in highly dimensional data requires a prohibitive number of samples in order to provide reasonable results [8]. Probably, similar limitations apply to the model proposed here. Nevertheless, this paper clearly shows that there is, indeed, an important difference between density preservation and minimization of the quantization error, which are often taken as criteria for clustering with no further consideration. The analytical results can be applied to immune algorithms in order to control the density of antibodies, as performed in section 6 for ARIA. Several modifications other than the modification of the suppression radius may be proposed to ARIA in order to make the prototypes follow the desired density.

Acknowledgements The authors would like to thank Fapesp and CPqD for the financial support and Professor Paulo Augusto Valente Ferreira for assistance on Variational Calculus.

46

A.G. Azzolini, R.P.V. Violato, and F.J. Von Zuben

References 1. Bezerra, G.B., Barra, T.V., de Castro, L.N., Von Zuben, F.J.: Adaptive Radius Immune Algorithm for Data Clustering. In: Jacob, C., Pilat, M.L., Bentley, P.J., Timmis, J.I. (eds.) ICARIS 2005. LNCS, vol. 3627, pp. 290–303. Springer, Heidelberg (2005) 2. de Castro, L.N., Von Zuben, F.J.: aiNet: An artificial immune network for data analysis. In: Abbass, H.A., Sarker, R.A., Newton, C.S. (eds.) Data Mining: A Heuristic Approach, ch. 12, pp. 231–259. Idea Group Publishing, USA (2001) 3. Chiu, C.Y., Lin, C.H.: Cluster Analysis Based on Artificial Immune System and Ant Algorithm. In: ICNC 2007, vol. 3 (2007) 4. Fukunaga, K., Hayes, R.R.: The reduced Parzen classifier. IEEE Trans. on Pattern Analysis and Machine Intelligence 11(4), 423–425 (1989) 5. Han, J., Kamber, M.: Data Mining: concepts and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2006) 6. MacQueen, J.B.: Some Methods for classification and Analysis of Multivariate Observations. In: Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability (1967) 7. Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: a review. ACM computing surveys (CSUR) 31(3), 264–323 (1999) 8. Silverman, B.W.: Density Estimation for Statistics and Data Analysis. Chapman & Hall, Boca Raton (1986) 9. Stibor, T., Timmis, J.: An Investigation on the Compression Quality of aiNet. In: Proceedings of the 2007 IEEE Symposium on Foundations of Computational Intelligence (FOCI 2007), pp. 495–502 (2007) 10. Timmis, J., Neal, M., Hunt, J.: An artificial immune system for data analysis. Biosystems 55(1-3), 143–150 (2000) 11. Troutman, J.L.: Variational calculus and optimal control: optimization with elementary convexity. Springer, Heidelberg (1996)

Immune Inspired Information Filtering in a High Dimensional Space Nikolaos Nanas1,2 , Stefanos Kodovas1,2, Manolis Vavalis1,2 , and Elias Houstis1,2 1

2

Lab for Information Systems and Services Centre for Research and Technology - Thessaly (CE.RE.TE.TH) Computing and Telecomunications Department, University of Thessaly {n.nanas,s.kodovas,m.vavalis,e.houstis}@cereteth.gr

Abstract. Adaptive Information Filtering is a challenging computational problem that requires a high dimensional feature space. However, theoretical issues arise when vector-based representations are adopted in such a space. In this paper, we use AIF as a test bed to provide experimental evidence indicating that the learning abilities of vector-based Artificial Immune Systems are diminished in a high dimensional space.

1

Introduction

The research domain of Adaptive Information Filtering (AIF), seeks to provide a solution to the problem of information overload, particularly on the Web, through the automatic construction of a representation of a user’s information interests, called “user profile”, and its continuous adaptation to temporal changes in these interests. The user profile is responsible for evaluating the relevance of new incoming information to the user’s interests and this assessment is exploited for providing the user with the appropriate information. More specifically, in content-based AIF, the user profile comprises descriptive features extracted from the content of relevant information items and these features are matched to those in new information items to assess their relevance. For example, in the case of textual information, which has been the main focus of research in AIF in general, and of the current work in particular, keywords extracted from the text of documents are used to abstract their content and to build the user profile. Typically, both the user profile and the documents are represented as binary, or weighted, keyword vectors in a space with as many dimensions as the number of unique keywords in the underlying document vocabulary. This allows the application of trigonometric measures of similarity for calculating how close to the profile’s vector is a document’s vector. Such vector-based representations has been the cornerstone of research in AIF, but have also been fundamental for research in the domain of Artificial Immune Systems (AIS). AIF is a complex and dynamic computational problem with no established solution. As we further discuss in [14], there is a characteristic lack of broadly adopted web applications that are based on content-based AIF, and this is in part E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 47–60, 2010. c Springer-Verlag Berlin Heidelberg 2010 

48

N. Nanas et al.

due to its distinguishing and challenging requirements. One of these requirements is the need for increased dimensionality. Even if we restrict ourselves to textual information, we still need a substantial number of keywords to be able to cover the variety of topics that may attract the interest of users. If we wish to further extend the scope of a user profile beyond textual information and allow the additional incorporation of features extracted from the content of other media types, such as audio or image, then the number of necessary profile features increases even more. Overall, as we explain in detail in [14], successful AIF requires profile representations that can incorporate a large number of features. In other words, we need profile representations capable of maintaining their accuracy within a high dimensional feature space. AIS and Genetic Algorithms (GAs) have already been applied to AIF of textual information, or related problems, such as text classification. A comprehensive review of evolutionary and immune-inspired AIF can be found in [12]. In both cases, antibodies, or chromosomes, are typically represented as keyword vectors and trigonometric measures of similarity are deployed for measuring the affinity between antibodies and antigens (documents), or when calculating the fitness of chromosomes, respectively. Recently however, theoretical concerns have been raised regarding the adequacy of such vector representations in the case of high dimensional spaces [7]. In such spaces, the discriminatory power of distance metrics between vectors diminishes significantly and at the same time, the search space increases radically. In this paper, we use AIF as a test bed to conduct experiments that investigate the above concerns. We start with a review of Evolutionary and Immune-Inspired approaches to AIF, based on vector-based representations, in the next section and we then summarise in section 3, the theoretical issues that arise when the dimensionality of the vector space increases. In section 4 we describe the methodology of our simulated experiments and the evaluated algorithms and we present the experimental results. We conclude with a summary and pointers for future work.

2

Evolutionary and Immune Inspired AIF

AIF is not a typical optimisation problem. It is reminiscent of Multimodal Dynamic Optimisation (MDO), but it is better cast as a continuous learning problem. It is natural to assume that a user’s multiple interests develop over time within a continuously changing information environment. A researcher, for instance, is interested in various topics and subtopics of one or more research domains and these interests change over time with the acquisition of new knowledge, or due to the requirements of various research projects. At the same time, the relevant literature is also continuously updated with new publications. An AIF system has to be able to learn an accurate representation of the user’s multiple interests and to continuously adapt to changes in them within a dynamic information environment. The dynamic nature of the problem has attracted the application of GAs for adapting a population of profiles, that collectively represent the user’s interests,

Immune Inspired Information Filtering in a High Dimensional Space

49

to the changes in them. The relevant approaches are reviewed in [12] and so here we only briefly summarise the techniques of interest to the current work. The most common practice has been to represent each document and each profile as a weighted keyword vector in a vector space with as many dimensions as the number of unique words in the documents’ vocabulary [9,24,21]. To assess the relevance of a document, trigonometric measures of similarity, like the inner product [4] and the cosine similarity measure [21,9], have been used to compare the vector of each profile in the population to the document’s vector. The document is then assigned the largest achieved relevance score. The evaluated documents are presented to the user in decreasing relevance score. The user chooses to read and possibly rates some of these documents and this feedback provides the basis for the fitness evaluation of profiles. The fitness of profiles responsible for the evaluation of documents that received positive user feedback gets reinforced and vice versa. Fit chromosomes are selected for reproduction through crossover and mutation. Crossover randomly combines the keywords of two parent profiles to produce two offspring that usually replace less fit individuals in the population and hence the size of the population remains constant. Mutation randomly modifies the keyword vector of a child profile to trigger further exploration of the information space. When Memetic Algorithms are adopted, then the above process is augmented with a learning algorithm that moves the vector of profiles towards the vector of documents that received positive feedback and vice versa. The ability of the immune system to distinguish between the host organism’s molecules (“self”) and external pathogens (“non-self”) serves well as a metaphor for AIF, where the task is to distinguish between relevant and non-relevant information. Various immune inspired algorithms have been applied to similar text classification problems and are exhaustively reviewed in [12]. Here, we concentrate on those adopting vector-based representations. For example, binary keyword vectors have been used to represent documents and immune receptors as part of the coevolutionary AIS algorithm in [25]. Antibodies are also represented as weighted keyword vectors in [2], where the aiNet algorithm [3] is used to perform hierarchical clustering of documents into groups with similar content. Of particular interest to the current work is the Artificial Immune System for Email Classification (AISEC) proposed in [20]. Antibodies and emails (antigens) are represented as binary keyword vectors and a combination of clonal selection and memory is deployed for continuously learning to identify unwanted (spam) emails. Alternative representations have also been adopted. For instance, regular expressions have been used for spam filtering in [17] and a weighted keyword network is proposed in [11] for profile representation in AIF. Despite these exceptions, vector-based representations have been generally the “de facto” choice in the domain of AIS [7]. In [12] and more analytically in [10], we have argued that theoretically GAs are not well suited to AIF due to their tendency to converge to a single optimum (a single topic of interest) and thus progressively loose the necessary diversity for further adaptation. This is mainly due to the fact that selection of chromosomes for reproduction is based on their relative fitness, combined with the fixed

50

N. Nanas et al.

population size. On the contrary, AIS have the inherent ability to boost and maintain the diversity of the immune repertoire. This is achieved with heterostasis, i.e., the preservation of diversity, and heterogenesis, i.e., the introduction of diversity. Heterostasis, is achieved by triggering the cloning of antibodies, not according to their relative concentration (fitness), but rather according to the absolute value of their affinity to invading antigens. So even antibodies that have not been successful in the past can be triggered to clone and proliferate. The introduction of long-lived memory cells that have been successful in recognising antigens in the past and idiotypic effects [6] that suppress similar antibodies, contribute further to heterostasis. Heterogenesis, on the other hand, is achieved in two ways. The hypermutation of antibody clones causes the generation of new antibody types. At the same time, further diversity can be introduced with the recruitment of new antibody types produced in the bone marrow. It is also important to note at this point, that the clones and the recruited antibodies do not necessarily replace existing antibodies in the repertoire. Therefore, the size of the immune repertoire is not fixed but changes dynamically. It increases with the addition of new antibodies and reduces when antibodies are eventually removed from the repertoire, typically due to some decay process. Despite the above theoretical advantages of AIS over traditional GAs, we are not aware of any relevant experimental comparisons.

3

Theoretical Issues Regarding Dimensionality

According to the Vector Space Model (VSM) [19], representing both documents and profiles as binary, or more frequently, weighted keyword vectors and using trigonometric measures of similarity to compare them, has dominated AIF research. Such representations and similarity measures have been naturally adopted by most evolutionary and immune inspired approaches to AIF. They are also very common in AIS research in general [7]. Recently however, theoretical issues have been raised concerning the appropriateness of vector-based representations and trigonometric measures of similarity in high dimensional spaces. According to [7], the shape-space formalism underlies most AIS approaches. Each antibody is represented as a point in an ndimensional parameter space and defines a continuous recognition region. Antigens are also represented as points in the same space and if they fall within an antibody’s recognition region then they get identified. The affinity between cells is typically measured by calculating the distance or the correlation between the cells’ n-dimensional vector representations. This is an abstraction that is only a simplification of the underlying biology and reduces AIS to another kind of instance-based methods, such as the nearest-neighbour classifiers of machine learning [8]. The problem is that these methods, which are based on the notion of locality between points in the shape-space, suffer due to the “curse of dimensionality” [1]. In high dimensional spaces the volume of the space increases exponentially and hence distance based metrics lose their discriminatory power. First, as the number of dimensions increases, the coverage of a recognition region

Immune Inspired Information Filtering in a High Dimensional Space

51

expressed as a fixed-size hypercube [7], or hypersphere [23] diminishes. Furthermore, to increase the volume of the recognition region we need an increasing coverage of each dimension. In addition, in high dimensional spaces, data points become equidistant and scalar metrics defined on this space lose their discriminatory power. For these reasons and due to the shape of high dimensional spaces instance-based methods scale with the number of data points and become impractical for many applications, especially those that involve continuous learning. Similarly, it has been argued that in high dimensional spaces the use of hyperspheres as recognition regions in negative selection algorithms can result in poor classification accuracy [23]. According to the authors, the adverse properties of hyperspheres in high dimensional spaces “could bias all (artificial immune system) algorithms, which employ hyperspheres as recognition units”. AIF is a high dimensional problem. Even if we focus on textual information, a large number of words is necessary to cover the various topics of interest. There are more than 600,000 words in the English language and as we will see, even a moderate collection of 20,000 documents may include more than 30,000 unique words. This number of words (i.e., dimensions) is far larger than the dimensionality of many problem domains. For instance, the dimensionality of hand written digit recognition is just 256 [26]. In practice, dimensionality reduction techniques are deployed to reduce the number of keywords involved, but still, this number is quite high. For instance, 4419 unique keywords are extracted out of 20,000 documents in [2]. Vector-based approaches to AIF are also based on the notion of locality. The profile’s keyword vector defines the “centroid” of user interests and the relevance of documents is measured by how close or how correlated their vector is to this centroid. In the case of evolutionary, or immune-inspired AIF, a population of profiles, rather than a single profile, is used to represent a user’s interests, as with instance-based approaches. So, all of the above theoretical issues are relevant, but complementary issues also arise. As the number of dimensions (keywords) increases, the number of possible keyword combinations increases exponentially. The search space becomes very large and as we further argue and experimentally support in [13], it becomes unlikely that random vector operations, like crossover and mutation, will produce the right combinations of weighted keywords to accurately represent a user’s interests. Furthermore, as the number of possible keyword combinations increases, a weighted keyword vector becomes ambiguous because it can not distinguish between relevant and non-relevant keyword combinations. For example, if a user is interested in “river banks” and “long holidays”, she is not necessarily interested in “long rivers” and “bank holidays”, but a weighted keyword vector containing these four words equally represents all their possible pairs. This term-independence assumption is inherent in the VSM and its orthogonal dimensions and is true not only for textual information, but any type of feature space. Despite the above adverse characteristics of vector-based representations and trigonometric measures of similarity in high dimensional space, they have been the de facto choice in AIF and AIS. The relevant research focuses on problem

52

N. Nanas et al.

instances with manageable dimensions, or relies heavily on dimensionality reduction techniques. AIF though, is a complex problem that can only be successfully tackled in a high dimensional feature space and there are possibly other real world applications with similar requirements. In the rest of this paper, we use AIF as a test bed to experimentally demonstrate that the learning abilities of AIS and GAs may seriously diminish in a high dimensional space. The aim is to prompt practitioners, in accordance with [5], to seek alternative representation and similarity measures.

4

Experimental Evaluation

The current work is part of an ongoing research effort that treats AIF not just as a practical engineering problem, but also as a general computational problem that due to its dynamic and high dimensional nature, challenges existing models and algorithms. Unfortunately, since the removal of the dedicated track from the well established Text Retrieval Conference (TREC), there is no standardised evaluation methodology for AIF experiments and this has caused a decline of the research interest in the domain. One of our goals is to reanimate this interest and if possible to attract the application of biologically inspired approaches to AIF. In [16], we have proposed a novel methodology for performing simulated AIF experiments. In contrast to previous practices, even those TREC adopts, the methodology more accurately simulates users with multiple and changing interests within a dynamic information environment. We wish to use this methodology to evaluate a series of biologically inspired approaches, such as GAs, Neural Networks, AIS and more. Here however we start with a stripped down version of the methodology that focuses mainly on the high dimensional aspects of AIF, rather than its dynamics. The adopted methodology was first introduced in [13], where it was used to compare GAs to a linear learning algorithm. Here we complement these experiments with the evaluation of an AIS and we juxtapose the results. In what follows, we describe this simplified methodology, the evaluated algorithms and the experimental results. 4.1

Methodology

The adopted methodology simulates users with a single topic of interest and requires a profile that can continuously learn this topic. In contrast to the methodology described in [16], it does not simulate users with multiple and changing Table 1. Topics involved in the experiments and their corresponding size (i.e., number of relevant documents in the collection) topic

earn

acq money-fx crude grain trade interest wheat ship

corn

dlr oilseed

size

3987

2448

801

254

217

topic money-supply sugar

gnp

size

163

190

184

634

628

552

513

306

305

coffee veg-oil gold nat-gas soybean bop livestock cpi 145

137

135

130

120

116

114

112

192

Immune Inspired Information Filtering in a High Dimensional Space

53

interests. The experiments are based on the Reuters-21578 corpus1 , a collection of 21578 news stories published by Reuters in 1987, but the methodology could be applied given any preclassified collection of documents. There are 135 topic categories in Reuters-21578 and our experiments are based on only those 23 topics with more than 100 relevant documents. Table 1 presents the code names of these topics and their sizes2 , i.e., number of relevant documents in the collection. The documents in the collection are ordered according to publication date and there are temporal variations in the content of documents relevant to a topic. The documents are pre-processed with the removal of function words and are reduced to their common stem with Porter’s algorithm. The common Term Frequency Inverse Document Frequency (TFIDF) [22] is then used to assign a weight to each word in every document. The average TFIDF of words across all documents is calculated and the words with highest weight are chosen to play the role of keywords for building a vector space with corresponding dimensions. All documents and profiles are then expressed as weighted keyword vectors in this vector space. Each experimental run involves one evaluation cycles per topic. Depending on the algorithm, an evaluation cycle involves a single profile, or a population of profiles, which represents a single topic of interest and processes each of the 21578 documents sequentially. Every processed document is evaluated and assigned a relevance score. If the document is relevant to the topic of interest, then the document’s vector is used as a training instance for learning. So learning is continuous and takes place in parallel to the document evaluation process. The evaluation cycle produces a list of 21578 documents ordered according to decreasing relevance score. This list is used to calculate the Average Uninterpolated Precision (AUP) of the algorithm for the current topic of interest. A topic’s AUP is defined as the sum of the precision–i.e., the percentage of documents relevant to that topic–at each point in the ordered list where a relevant document appears, divided by the topic’s size3 . 4.2

Baseline Learning Algorithm

We used the popular Rocchio’s linear learning algorithm [18] as a baseline. Rocchio’s algorithm linearly moves the vector of a profile towards a relevant document using equation 1. The coefficients in the equation define the pace of this movement. More specifically, coefficient α plays the role of a decay function and allows the profile to “forget” over time. Coefficient β on the other hand, defines how much the user profile is influenced by the relevant document. Based on preliminary experiments, we have chosen α = 0.95 and β = 0.25. Rocchio’s algorithm is discussed in more detail in [16]. Overall, when applied for continuous learning, Rocchio’s algorithm treats the profile’s vector as a centroid that is constantly moved towards the region in the space where recent relevant documents appeared. 1 2 3

http://www.daviddlewis.com/resources/testcollections/reuters21578/ Note that the topics in table 1 are ordered by decreasing size. For a more detailed explanation of AUP see [16].

54

N. Nanas et al.

Pt+1 = α · Pt + β · D,

(1)

where: Pt+1 is the new profile vector, is the previous profile vector and Pt D is the vector of a relevant document.

In [13], we performed a series of experiments using Rocchio’s algorithm, on vector spaces containing an increasing number of dimensions. In particular, we selected according to their average TFIDF weight the best 100, 500, 1000, 5000, 10,000 and 20,000 keywords and built a vector space with corresponding dimensions. We also built a vector space with 31,928 dimensions, as many as the total number of unique words extracted from the document collection. In each of these spaces and for each of the 23 topics a single evaluation cycle was performed. The initial profile for each evaluation cycle was a weighted keyword vector with all weights equal to zero. The results show that the best filtering performance is achieved when all extracted 31,928 keywords are taken into account. It is only in this high dimensional space that the available repertoire of words is enough to cover all 23 topics of interest. When fewer keywords are extracted, then the results are biased towards topics that have a larger number of relevant documents and produce the majority of keywords. These experimental results provide evidence that AIF requires a high dimensional keyword space. The rest of the experiments in this paper were performed on this high dimensional space of 31,928 keywords. We also use the results achieved in this space with Rocchio’s algorithm as a baseline. 4.3

Genetic Algorithm

The evaluated GA is described in detail in [13], so here we only summarise its basic components. The GA maintains a population of 100 profiles, each represented as a weighted keyword vector. The population is initialised by turning into a profile the weighted keyword vector of each of the first 100 relevant documents. A profile’s initial fitness is set equal to zero. In each evaluation cycle, the population processes the documents in the collection in order. For each processed document, we calculate the inner product between the document’s vector and each profile vector and assign to the document a relevance score equal to the highest achieved inner product. If the document is relevant to the topic of interest, the fitness of the winner profile gets reinforced by an amount equal to the document’s relevance score (new fitness = old fitness + relevance score). After each relevant document, genetic operations take place. The 25% of the most fit profiles are selected for reproduction. Two parent profiles mate with single point crossover to produce two offspring and 5% of the keywords in the offspring profiles are mutated by randomly modifying their weights. The offspring replace the worst 25% of the profiles in the population. 4.4

The AISEC Algorithm

The Artificial Immune System for E-mail Classification (AISEC) was proposed in [20]. It targets the problem of spam filtering and uses a combination of clonal

Immune Inspired Information Filtering in a High Dimensional Space

55

selection, memory and idiotypic effects to train in an online fashion a population of receptors (keyword vectors) that identify unwanted emails. Our goal here is not to accurately re-implement and evaluate AISEC. AISEC was chosen because, out of the algorithms reviewed in [12], it is better suited to the AIF problem addressed in this paper, especially due to its online learning capabilities. The details of AISEC and the relevant pseudocode can be found in [20], so here we use the same notation to summarise its main components and highlight the necessary modifications made to fit AISEC to the problem at hand. Many of these modifications were already suggested in [20], by the authors themselves. We should also note that we use the same names and values for the involved system parameters. AISEC uses a population of B-cell receptors divided into a sub-population of naive B-cells (BC) and a sub-population of memory B-cells (M C). In the original algorithm each B-cell comprised two binary keyword vectors, one for the subject and the other for the sender of an email message. Here however, both B-cells and documents are represented as weighted keyword vectors in the high dimensional vector space discussed above. This space comprises the complete set of keywords (31,928) extracted from the content (including title) of the documents in the collection. As in the case of the GA, for each of the 23 topics, the population of B-cells is initialised by turning the weighted keyword vectors of the first 100 relevant documents into naive B-cells. 25 random elements from BC are inserted into M C. The initial stimulation count of naive B-cells and memory B-cells is set to Ksb = 125 and Ksm = 25 respectively. The initialisation process proceeds with the cloning of memory cells that have a strong affinity to initialisation documents. Mutated vectors of the original memory cells are thus introduced to BC (see [20] for more details on the initialisation process). In contrast to the original algorithm, where the affinity between two cells was measured as the proportion of common keywords in their vectors, here we adopt the cosine similarity measure between the two vectors. Both measures return a value between 0 and 1. If the affinity between a B-cell (bc) and a document (antigen, ag), or another B-cell, is greater than Ka = 0.5, then the B-cell is activated and cloned. As it is described in more detail in [20], the number of clones depends on the affinity between bc and ag and a cloning constant (Kl = 7.0). Each clone is mutated by randomly choosing a number of keywords in bc and replacing them with the corresponding keywords in ag. The number of mutated keywords is proportional to the length of the clone’s keyword vector and a mutation factor (Km = 0.7) and is inversely proportional to the affinity between bc and ag. The initial stimulation count of clones is set equal to 125. Note that clones are incorporated in BC without replacing existing B-cells and so the size of the population is not static, but dynamic. In each evaluation cycle, we initialise the population of B-cells and then it sequentially evaluates each document in the collection. A document is assigned a relevance score equal to the highest achieved cosine similarity to the B-cells. Whenever a document is relevant to the current topic of interest then the learning process takes place. This is described in detail by the pseudocode bellow, where

56

N. Nanas et al.

Algorithm 1. Update Population if doc is relevant then for all bc ∈ BC do if cosine(bc, doc) > Ka then increment bc stimulation count by one end if end for select bc best {element of BC with highest cosine similarity to document} BC ← BC ∪ clone mutate(bc best, doc) keep max 250 bc ∈ BC with higher stimulation count, remove the rest {additional element} select mc best {element of MC with highest cosine similarity to document} if cosine(bc best, doc) > cosine(mc best, doc) then BC ← BC −bc best bc best stimulation count ← Ksm MC ← MC ∪ bc best if size of MC > 50 then delete mc best {additional element} end if for all mc ∈ MC do if cosine(bc best, mc) > Ka then decrease mc stimulation count by one {idiotypic effects} end if end for end if for all bc ∈ BC do decrease bc stimulation count by one {decay of naive B-cells} end for for all bc ∈ MC ∪ BC do if bc fitness == 0 then delete bc from system end if end for end if

comments appear within braces and we use these comments to point out the additional algorithmic elements. One significant difference to the original algorithm is the inclusion of upper limits on the number of naive and memory B-cells. This was done because, despite our preliminary experimental efforts, we did not manage to identify appropriate parameter values so that the size of the population reaches an equilibrium irrespective of topic of interest. It is also impractical to allow the population to escalate. Given the above parameter values the number of naive and memory cells initially increases and reaches the upper limit after at most 100 relevant documents. Subsequently, after approximately 150 relevant documents the size of BC progressively declines and given enough relevant documents, it can be depleted. The number of memory cells on the other hand remains relatively static after reaching its upper limit4 . This behaviour is possibly due to the competition between naive and memory cells. As the best naive cells become memory cells, BC is progressively left with less competent vectors with decreasing likelihood to get activated. This is also the reason for ignoring negative feedback in our implementation. When non-relevant documents were treated as negative feedback then the population of naive cells would rapidly decline. Although, the variable population size is a significant advantage of AIS over GAs, which use a constant 4

The relevant data are excluded due to space limitations.

Immune Inspired Information Filtering in a High Dimensional Space

57

AUP

1.00

0.75

0.50

0.25

cpi

bop

livestock

soybean

gold

nat-gas

veg-oil

gnp

coffee

sugar

oilseed

money-supply

dlr

ship

corn

wheat

trade

interest

grain

crude

acq

money-fx

earn

topic 0

Fig. 1. Per topic AUP score for each of the evaluated algorithms

population size, the dynamic control of the population is not straight forward at all. It is an important research issue, but outside the scope of the current work. 4.5

Results

Figure 1 presents the results of the comparative experiments. For each of the 23 topics of interest on the x-axis the graph presents the AUP value (y-axis) of the evaluated algorithms. The topics on the x-axis are ordered by decreasing size (see table 1). On a first glance, the results look positive. AISEC significantly outperforms the GA and is better than the baseline learning algorithm for topics with less than 300 relevant documents in the dataset. However, AISEC’s improved performance is not due to its learning capabilities, but rather its instance-based nature. Like the GA, AISEC starts with one hundred weighted keyword vectors, corresponding to the weighted keyword vectors of the first 100 relevant documents. So, for topics with a small number of relevant documents (that is, the ones to the right on the graph), a significant proportion of these documents is already accurately expressed by B-cells in the initial population. In other words, the population overspecialises in the case of topics with small size and is, as expected, better than the baseline algorithm, which is based on a single centroid vector. For topics though with a large number of relevant documents (that is, the ones to the left on the graph), AISEC cannot generalise and is worse than the baseline. Like the GA, it lacks a mechanism for learning appropriate keyword weights and relies instead on random mutations for generating new receptors, which are unlikely to be a significant improvement of the original B-cell, due to the high dimensionality of the vector space.

58

N. Nanas et al.

To further support the above argument we have conducted a complementary experiment. For each topic, we initialised a population of B-cells and used this population to conduct an experiment, where the population is not updated with the algorithm described in the previous section. So in this case, AISEC’s performance is only due to its initial population. The relevant results are also incorporated in figure 1 (denoted “AISEC init”). They show that AISEC performs worse than its initial population alone. It can not produce better receptors than the ones already there from the start. Furthermore, as the population of naive B-cells eventually declines, valuable diversity is progressively lost. It is the loss of diversity that also explains the GA’s poor performance. Unlike AISEC, the GA has a fixed population size of 100. So after just a few generations, valuable existing keyword vectors in the initial population are replaced by randomly generated re-combinations and mutations of the vectors that have been successful in the beginning of each evaluation cycle. The GA quickly overspecialises to the first few relevant documents in the collection and can not further generalise. In contrast, AISEC’s population is dynamic, and, at least initially, increases in size, because clones do not replace existing B-cells. Furthermore, the memory cells play the role of an “elite” of successful vectors, which are difficult to remove from the population. It is especially through this second type of heterostasis, rather than heterogenesis, that AISEC maintains enough diversity in the population to outperform the GA.

5

Summary and Future Work

In this paper, we used AIF as an experimental test bed, because it is inherently a high dimensional problem. It is in high dimensional spaces that theoretical issues arise regarding the adequacy of vector-based representations, which have been fundamental in both AIF and AIS research. We implement a variation of the AISEC algorithm and compared it to a baseline learning algorithm and a GA. The results provide initial evidence that the learning abilities of both the AIS and the GA are seriously affected in a high dimensional space. These algorithms do not generalise, because they lack mechanisms for learning appropriate keyword weights. Instead, they rely on random re-combinations and modifications that given the inflated search space, are very unlikely to produce improved vector representations. Their performance depends more on instance-based rather than inductive learning and so these algorithms need to scale-up with the number of data points leading to serious practical difficulties. The baseline learning algorithm does better than both the AIS and the GA for topics with more than 300 relevant documents, although it uses a single keyword vector and thus has significantly lower computational requirements. We believe that these first findings are not specific to AISEC or our basic GA, but generalise to any AIS, or evolutionary approach, that uses vector-based representations in high dimensional spaces. Further experimental evidence are surely required to substantiate this argument and we are already working in this direction, based on the complete experimental methodology proposed in [16].

Immune Inspired Information Filtering in a High Dimensional Space

59

Nevertheless, we would like to prompt the investigation of alternative representations that are resistant to the observed dimensionality problems. We have already developed and experiment with such a representation, which is based on a weighted feature network, rather than vector. Comparative experiments show that this network-based representation can achieve performance improvements of up to 55% percent over a vector-based representation as the number of dimensions of the underlying vector-space increases [16,15]. Finally, we would like to point out once more, that AIF is not just a practical application, but a distinct and challenging computational problem that can boost innovation in biologically-inspired computing.

References 1. Bellman, R.: Adaptive Control Processes: A Guided Tour. Princeton University Press, Princeton (1961) 2. Ciesielski, K., Wierzcho´ n, S.R., Klopotek, M.A.: An immune network for contextual text data clustering. In: Bersini, H., Carneiro, J. (eds.) ICARIS 2006. LNCS, vol. 4163, pp. 432–445. Springer, Heidelberg (2006) 3. de Castro, L.N., Zuben, F.J.V.: An evolutionary immune network for data clustering. In: Proc. of the 6th Brazilian Symposium on Neural Networks, pp. 84–89 (2000) 4. Desjardins, G., Godin, R.: Combining relevance feedback and genetic algorithm in an internet information filtering engine. In: RIAO 2000, pp. 1676–1685 (2000) 5. Freitas, A.A., Timmis, J.: Revisiting the foundations of artificial immune systems for datamining. IEEE Trans. on Evolutionary Computation 11(4), 521–540 (2007) 6. Jerne, N.K.: Towards a network theory of the immune system. Annals of Immunology 125(C), 373–389 (1973) 7. McEwan, C., Hart, E.: Representation in the (artificial) immune system. Journal of Mathematical Modelling and Algorithms 8(2), 125–149 (2009) 8. Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997) 9. Moukas, A., Maes, P.: Amalthaea: An evolving multi-agent information filtering and discovery system for the www. Autonomous Agents and Multi-Agent Systems 1(1), 59–88 (1998) 10. Nanas, N., De Roeck, A.: Multimodal dynamic optimisation: from evolutionary algorithms to artificial immune systems. In: de Castro, L.N., Von Zuben, F.J., Knidel, H. (eds.) ICARIS 2007. LNCS, vol. 4628, pp. 13–24. Springer, Heidelberg (2007) 11. Nanas, N., De Roeck, A.: Autopoiesis, the immune system and adaptive information filtering. Natural Computing 8(2), 387–427 (2009) 12. Nanas, N., De Roeck, A.: A review of evolutionary and immune inspired information filtering. Natural Computing (2009) (online first), http://www.springerlink.com/content/g523m8328856gpn4/ 13. Nanas, N., Kodovas, S., Vavalis, M.: Revisiting evolutionary information filtering. In: Congress on Evolutionary Computation (2010) 14. Nanas, N., Vavalis, M., De Roeck, A.: What happened to content-based information filtering? In: Azzopardi, L., Kazai, G., Robertson, S., R¨ uger, S., Shokouhi, M., Song, D., Yilmaz, E. (eds.) ICTIR 2009. LNCS, vol. 5766, pp. 249–256. Springer, Heidelberg (2009)

60

N. Nanas et al.

15. Nanas, N., Vavalis, M., De Roeck, A.: A network-based model for high-dimensional information filtering. To appear in Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (2010) 16. Nanas, N., Vavalis, M., Kellis, L.: Immune learning in a dynamic information environment. In: Artificial Immune Systems, 8th International Conference (ICARIS 2009), pp. 192–205 (2009) 17. Oda, T., White, T.: Immunity from spam: An analysis of an artificial immune system for junk email detection. In: Jacob, C., Pilat, M.L., Bentley, P.J., Timmis, J.I. (eds.) ICARIS 2005. LNCS, vol. 3627, pp. 276–289. Springer, Heidelberg (2005) 18. Rocchio, J.: Relevance Feedback in Information Retrieval, ch. 14, pp. 313–323. Prentice-Hall, Upper Saddle River (1971) 19. Salton, G., McGill, M.J.: Introduction to Modern Information Retrieval. McGraw-Hill Inc., New York (1983) 20. Secker, A., Freitas, A.A., Timmis, J.: AISEC: an artificial immune system for e-mail classification. In: Congress on Evolutionary Computation, pp. 131–139 (2003) 21. Sheth, B.D.: A Learning Approach to Personalized Information Filtering. Master of Science, Massachusetts Institute of Technology (1994) 22. Sparck Jones, K.: A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation 28(1), 11–20 (1972) 23. Stibor, T., Timmis, J., Eckert, C.: On the use of hyperspheres in artificial immune systems as antibody recognition regions. In: Bersini, H., Carneiro, J. (eds.) ICARIS 2006. LNCS, vol. 4163, pp. 215–228. Springer, Heidelberg (2006) 24. Tjoa, A.M., H¨ offerer, M., Ehrentraut, G., Untersmeyer, P.: Applying evolutionary algorithms to the problem of information filtering. In: 8th International Workshop on Database and Expert Systems Application, Toulouse, France, pp. 450–458. IEEE Computer Press, Los Alamitos (1997) 25. Twycross, J., Cayzer, S.: An immune-based approach to document classification. Technical Report HPL-2002-292, HP Research Bristol, UK (2002) 26. Vapnik, V.: The Nature of Statistical Learning Theory, 2nd edn. Springer, Heidelberg (1999)

On the Benefits of Aging and the Importance of Details Thomas Jansen1, and Christine Zarges2 1 2

University College Cork, Department of Computer Science, Cork, Ireland [email protected] TU Dortmund, Fakult¨ at f¨ ur Informatik, LS 2, 44221 Dortmund, Germany [email protected]

Abstract. Aging is a concept that is used in many artificial immune system implementations. It is an important tool that helps to cope with multi-modal problems by increasing diversity and allowing to restart the search in different parts of the search space. The current theoretical understanding of the details of aging is still very limited. This holds with respect to parameter settings, the relationship of different variants, the specific mechanisms that make aging useful, and implementation details. While implementation details seem to be the least important part they can have a surprisingly huge impact. This is proven by means of theoretical analysis for a carefully constructed example problem as well as thorough experimental investigations of aging for this problem.

1

Introduction

Artificial immune systems (AISs) [4, 6] are a class of biologically inspired randomized search heuristics. They are based on the immune systems of vertebrates and derived from immunological theories. Classically, they are applied in the context of anomaly detection and classification, however, there are AISs tailored towards the task of (function) optimization. In this context most often AISs based on the clonal selection principle are applied [3, 5, 12]. While simple AISs can be very inefficient even for simple test problems [14] they are successful in applications when more complex versions are used [3, 5, 12, 15]. One of the more advanced mechanisms used in AISs is static pure aging [1–3]. AISs for optimization operate on a collection of search points. If static pure aging is employed each search point is equipped with an individual age. Initially, this age is set to 0. The search points grow older together: in each round of the algorithm the age of each search point is increased by 1. In the simplest aging variant a parameter τ , the maximal age, determines the lifespan. Each search point with an age exceeding τ is removed. Free places in the collection of search points are filled with new randomly generated search points with age 0. New search points are also created by means of randomly changing existing search points. The new search points inherit the age of the search points they are derived 

This material is based in part upon works supported by the Science Foundation Ireland under Grant No. 07/SK/I1205.

E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 61–74, 2010. c Springer-Verlag Berlin Heidelberg 2010 

62

T. Jansen and C. Zarges

from if they do not improve on them with respect to function value. In case they are an improvement, however, they are rewarded by being assigned age 0. There is only a limited number of theoretical investigations of the benefits and drawbacks of aging. It is known [8] that AISs with aging are very sensitive with respect to the setting of the maximal age τ . Setting the maximal age too small or too large can make the difference between being very efficient and extremely inefficient. Moreover, there is no general good setting: the appropriate maximal age depends on the optimization problem [8]. Static pure aging is not the only kind of aging known. In the field of evolutionary algorithms, another kind of biologically inspired randomized search heuristics, evolutionary aging is more common. There each new search point is always assigned age 0. It is known [9] that these different aging strategies have different strengths and weaknesses. It is remarkable that until very recently in all cases where an AIS with aging is proved to be very much superior to an AIS without aging the same improvement can be achieved when aging is replaced by an appropriate restart strategy [8, 9]. A restart strategy decides at some point of time to stop a randomized search heuristic and start again with a new randomly generated collection of search points. Such restart strategies are conceptually simpler, easier to implement, and computationally cheaper than aging. Thus, it is highly interesting to see what aging can achieve with respect to efficiency of an AIS that cannot be achieved by restarts. Very recently, one example problem has been presented where aging is proven to facilitate a speed-up that cannot be achieved by restarts [10]. We consider this example problem and static pure aging embedded in a simple AIS framework. We discuss seemingly very similar ways of implementing static pure aging that differ only in the exact way the age of new search points is set. In spite of the small differences the impact on the efficiency of the AIS is profound. This finding contributes to the theoretical foundation of aging in AISs. It helps to understand the role of different aspects of aging.

2

Algorithm

We are interested in the analysis of static pure aging [2] and want to see its effects on the efficiency of the optimization by an AIS. Thus, we need a simple AIS that we can use as algorithmic framework. Since it is known that having a real population (as opposed to only a single search point) can be crucial for AISs [15] we use a collection of search points of size μ. The AIS works in rounds where in each round all search points grow older, one new search point is generated as random variation of existing search points, its age is decided, search points that are too old are removed and new randomly generated search points are introduced to keep the number of search points constant at μ. A more formal description of the algorithmic framework is given in Algorithm 1. We use Algorithm 1 for maximization of an objective function f : {0, 1}n → R. In the initialization (line 1) we select μ search points independently uniformly at random and assign each age 0. In line 2 the age of each search point is increased by 1. The new search point z is generated in line 3 using two different

On the Benefits of Aging and the Importance of Details

63

Algorithm 1. Algorithmic Framework. 1. 2. 3. 4. 5. 6. 7.

Initialization: Initialize collection of search points C of size μ. Aging: Growing older: Increase age of all search points in C. Variation: Generate new search point z. Aging: Age of new search point: Set age of the new search point z. Aging: Removal due to age: Remove search points with age exceeding τ . Selection for Replacement: Insert z in C and adjust C to have |C| = μ. Stopping: If stopping criterion not met continue at line 2.

variation operators. One operator is k-point crossover, known from evolutionary algorithms [11]. It is efficient when the collection of search points is sufficiently diverse. Since aging aims at increasing the diversity it is a good idea and interesting test case to combine crossover with aging. In k-point crossover two search points x, y ∈ {0, 1}n are cut into k + 1 pieces by selecting uniformly at random k different cut positions. A new search point is constructed from the pieces by taking all the odd numbered pieces of x (the first, third, . . . ) and all even numbered pieces of y (the second, fourth, . . . ) and concatenating them in increasing interleaving order. Usually, k-point crossover with very small values for k is employed, most often k = 1 or k = 2. The other operator is called mutation. It takes one search point x ∈ {0, 1}n and performs independently for each of the n bits one random experiment. With probability 1/n the bit is inverted, otherwise it remains unchanged. We apply these two variation operators in line 3 in the following way. With probability pc (a parameter of the algorithm), we select two search points from C uniformly at random and perform k-point crossover. The result is subject to mutation. The final result is the new search point z. If no crossover is performed (with probability 1 − pc ), we select one search point from C uniformly at random and mutate it, the result being the new search point z. The decision about the new age (line 4) is where the different implementations of static pure aging differ. We discuss the details afterwards. In line 5 all search points with age larger than τ are removed from C. In line 6 we take care of the number of search points in the collection of search points. The new point z is inserted. If |C| < μ, randomly generated search points are inserted. If |C| > μ, a worst search point is deleted. We give the precise details of this step in Algorithm 2. We avoid to discuss stopping criteria (line 7) by concentrating on the optimization time. Formally, we let Algorithm 1 run forever and consider the first point of time when a global optimum of f is found. As usual we make use of the number of times the objective function f is evaluated as measure of time. Thus, TA,f is the number of function evaluations Algorithm 1 has made when max{f (x) | x ∈ P } = max{f (x) | {0, 1}n} holds for the first time. We are mostly interested in the mean value E (TA,f ) of this random variable. Algorithm 1 has four parameters: the population size μ ∈ N \ {1}, the crossover probability pc ∈ [0, 1], the number of crossover points in k-point crossover k ∈ {1, 2, . . . , n + 1}, and the maximal age τ ∈ N. We need to detail the way we set the age of a new search point. The idea is to assign age 0 if the new search point is an improvement. Otherwise it inherits

64

T. Jansen and C. Zarges

Algorithm 2. Selection for Replacement. 1. If |C| < μ Then 2. If z.age ≤ τ Then 3. Set C := C ∪ {z}. 4. While |C| < μ do 5. Select x ∈ {0, 1}n uniformly at random. 6. Set x.age := 0. Set C := C ∪ {x}. 7. Else If (f (z) ≥ min f (x)) and (z.age ≤ τ ) Then x∈C

8. 9. 10. 11. 12. 13. 14.

If f (z) > min f (x) Then x∈C

Set D := {x ∈ C | f (x) = min f (x )}.  x ∈C

Else f (x )}. Set D := {x ∈ C | f (x) = min  x ∈C

(|x .age − z.age|)} Set D = {x ∈ D | |x.age − z.age| = min  

Select y ∈ D uniformly at random. Set C := (C ∪ {z})\{y}.

x ∈D

its age from the search points it is derived from. Crossover makes things less obvious. Search points created by crossover have two search points as origin. It is unclear how the comparison with respect to the function value is to be made and what age is to be inherited if no improvement was made. One may believe that these are unimportant details. After all, they only matter in the case of crossover and if the new search point is not good anyway. However, as our analytical and experimental results show, these details do matter. We consider three variants that all follow the general scheme that is given in Algorithm 3 and that are defined in Definition 1. Definition 1. A new search point z that was either created by crossover of x and y or by mutation of x (where we have x = y for notational simplicity) is assigned its age as outlined in Algorithm 3. Line 4 of this algorithm is detailed in three variants as follows. In age-based static pure aging the age is set to the age of the older search point: z.age := max{x.age, y.age}. In optimistic value-based static pure aging the age is set to the age of the search point with larger function value, in case of equal function values to the larger age: If f (x)  = f (y) then z.age := argmax{f (x), f (y)}.age, else z.age := max{x.age, y.age}. In pessimistic value-based static pure aging the age is set to the age of the search point with smaller function value, in case of equal function values to the larger age: If f (x)  = f (y) then z.age := argmin{f (x), f (y)}.age, else z.age := max{x.age, y.age}. The idea of static pure aging is to punish a new search point that fails to be an improvement by having it inherit its age. Improvements are rewarded by assigning age 0 and thus a longer lifespan. In the case of crossover the worst punishment possible is to assign the new search point z the larger age of the two

On the Benefits of Aging and the Importance of Details

65

Algorithm 3. Outline of static pure aging. 1. If f (z) > max{f (x), f (y)} Then 2. Set z.age := 0. 3. Else 4. Set z.age := age of either x or y.

other involved search points, x and y. This is what we call age-based static pure aging. It is the variant that has been analyzed before [10]. While being simple it does not appear to be entirely fair. The reason the new search point fails to be an improvement could be that a good search point was combined with a bad search point. It therefore makes sense to compare the function values of x and y. If these function values are equal we set the new search point’s age to the older age. If, however, the two search points have different function values we have a choice. We can react in an optimistic way to this difference and assign the new search point the age of the better search point. This is what we call optimistic valuebased static pure aging. Alternatively, we could be pessimistic and assign the new search point the age of the worse search point. We call this pessimistic valuebased static pure aging. We compare these variants of static pure aging in the following two sections. For comparison of the three aging variants we consider an example problem where aging provably is essential for being efficient. One such example problem where aging even cannot be replaced by restarts is known [10]. We consider the same problem and compare the different variants of static pure aging on it. We start with a formal definition (Definition 2) and add an informal explanation of the example problem’s properties. Definition 2. The function f : {0, 1}n → R is defined for n ∈ N, x ∈ {0, 1}n. ⎧ ⎪ if x = 1n/4 0n/4 q, q ∈ {0, 1}n/2, |q|1 ≥ n/12, ⎨2n f (x) = n + i if x = 1i 0n−i , i ≤ n/4, ⎪ ⎩ n − |x|1 otherwise, where |x|1 = OneMax(x) =

n  i=1

x[i].

In the following, proof methods due to Witt [13] are applied where an algorithm without crossover is considered. An inspection of the proofs in [13] reveals that the upper bounds carry over to our algorithm. For the vast majority of points x in the search space the function value is defined as n − OneMax(x). It is well known [13] that it is easy to follow the direction of increasing function values for such functions. The last point of this type is the all zero bit string 0n . This is the beginning of a path of Hamming neighbors 1i 0n−i with function values n + i. Since it is easy to create the next better point 1i+1 0n−i−1 from 1i 0n−i via mutation we see that the local optimum 1n/4 03n/4 is easy to find. Points of the form 1n/4 0n/4 q with q ∈ {0, 1}n/2 and OneMax(q) ≥ n/12 are special. Theset of all these points OPT := 1n/4 0n/4 q | q ∈ {0, 1}n/2, OneMax(q) ≥ n/12 equals the set of all

66

T. Jansen and C. Zarges

global optima. These points are easy to locate by means of a k-point crossover of the local optimum 1n/4 03n/4 and some y ∈ {0, 1}n that is chosen uniformly at random but very difficult otherwise. For all points x ∈ OPT we have n/3 ≤ OneMax(x) ≤ (3/4)n and thus there are always exponentially many points with the same number of 1-bits. For each number i of 1-bits let OPTi denote the set of bit strings from OPT with this number of 1-bits, i. e., OPTi = {x ∈ OPT | OneMax(x) = i}. Let OPTi = {x ∈ {0, 1}n | OneMax(x) = i} \ OPT i denote the other strings with the same number of 1-bits. We have |OPTi | / OPTi = 2−Ω(n) and we conclude that it is highly unlikely to find OPT by pure random sampling. This implies that restarts do not help. Also randomized search heuristics that are efficient on OneMax are unlikely to encounter OPT since they quickly leave the part of the search space with these numbers of 1-bits. Thus, they sample only a polynomial number of such bit strings and encounter OPT only with probability nO(1) · 2−Ω(n) = 2−Ω(n) . With crossover, however, things are entirely different as the following lemma (from [10]) states. Lemma 1. Let x = 1n/4 03n/4 and y ∈ {0, 1}n selected uniformly at random. Let OPT be the set of global optima of f . ∀k = O(1) : Prob (k-Point-Crossover(x, y) ∈ OPT) = Ω(1).

3

Theoretical Analysis

In this section we prove upper and lower bounds on the optimization time of Algorithm 1 with aging in the pessimistic and optimistic value-based variant on our example problem f and compare these with the age-based variant [10]. The main result is that the pessimistic value-based variant is very similar to the age-based variant but quite different from the optimistic age-based variant. Moreover, all three variants are very similar in their robustness with respect to parameter settings. The most critical parameter is the maximal age τ . We respect the known lower bound τ = ω(μn log μ) [8]. Given this restriction the algorithm (and our proofs) work for most settings of the other parameters. In particular, any polynomially bounded population size μ ≥ 2, any constant crossover probability that is different from 1 and 0, and any number of crossover points that is bounded by a constant lead to efficient optimization in the sense of an expected optimization time that is bounded above by some polynomial. Note that the bound depends on the parameter setting and thus there are better and worse settings. We start with the results for the pessimistic value-based variant. Theorem 1. Consider f : {0, 1}n → R from Definition 2. Let Af-pessimistic denote Algorithm 1 with aging in the pessimistic value-based variant with population size μ ∈ N with μ ≥ 2 and μ = nO(1) , crossover probability pc with 0 < ε ≤ pc ≤ 1 − ε < 1 for any positive constant ε, k = O(1) crossover points in k-point crossover, and maximal



age

τ = ω(μn log μ). Then E TAf-pessimistic ,f = O μ · τ + n2 + μn log n .

On the Benefits of Aging and the Importance of Details

67

Proof. There are three regions of the search space that correspond to phases of a run of Algorithm 1 on f . In the vast majority of the search space the function value is given as n − OneMax(x). Due to our lower bound on the maximal age τ this part can be optimized as Algorithm 1 without aging optimizes OneMax. It is known [13] that its expected optimization time is O(μn + n log n). Second, there are the bit strings of the form 1i 0n−i with i ≤ n/4. Again, due to our lower bound on the maximal age τ this part can be optimized as Algorithm 1 without aging optimizes LeadingOnes, a well-known example function n i  that is given by LeadingOnes(x) = x[j]. This yields on upper bound of i=1 j=1

2  O n + μn log μ [13]. Finally, we are interested in constructing a global optimum by crossover of a local optimum and a randomly chosen search point. This can happen when we have at least two search points in the local optimum and some but not all of those locally optimal search points are removed due to their age. We call such an event a partial restart. The time we need to wait until a search point is removed due to its age is at most τ . Assume that we knew that on average

r such partial restarts  are sufficient to generate an optimum. Then we had O r · τ + n2 + μn log n as upper bound on the expected optimization time. We see that it suffices to prove r = O(μ). Consider a point of time when a first locally optimal search point x is produced. This search point enters the population and is assigned age 0. With probability very close to 1 at this point of time all other search points in C also have form 1i 0n−i for different values of i all with i < n/4. We consider the following τ = ω(μn log μ) rounds. We prove that within these τ steps with probability Ω(1) another locally optimal search point with age different from x.age enters C. To this end, we consider crossover. Consider x = 1n/4 03n/4 and y = 1i 0n−i with 0 ≤ i < n/4. We have Prob (k-Point-Crossover(x, y) = x) = Ω(1) in analogy to Lemma 1. When x entered C it had minimal age 0. Thus, all y = 1i 0n−i with i < n/4 have age different from x. In a crossover of x = 1n/4 03n/4 and y = 1i 0n−i with i < n/4 that yields z = 1n/4 03n/4 the new search point’s age is set to the age of y as we use aging in the pessimistic value-based variant. Thus, such a crossover operation achieves our goal of having two search points with different age in the local optimum. Note that once we have at least two locally optimal search points with different age in C this will always be the case until some restart happens. This is due to the selection for replacement where in case of equal function value a search point with minimal age difference is selected for replacement (Algorithm 2, line 12) . Now consider the point of time when the age of x exceeds τ . In this round x and all its copies with identical age are removed and replaced by purely random search points. Since the expected time for x to takeover the complete collection of search points is O(μ log μ) and τ = ω(μn log n) holds we have with probability close to 1 that all other search points are also locally optimal. Thus, after removing b copies of x we have a collection of search points with μ − b local optima and b random search points. Remember that 0 < b < μ holds since we

68

T. Jansen and C. Zarges





1 = Ω the global have a partial restart. Thus, with probability Ω μb · μ−b μ μ optimum is produced as next new search point. This establishes that on average r = O(μ) partial restarts suffice.  Theorem 2. Consider f : {0, 1}n → R from Definition 2. Let Af-pessimistic denote Algorithm 1 with aging in the pessimistic value-based variant with population size μ ∈ N with μ ≥ 2 and μ = nO(1) , crossover probability pc with 0 < ε ≤ pc ≤ 1 − ε < 1 for any positive constant ε, k = O(1) crossover points in k-point crossover, and any age τ (with τ = 2O(n) ).

 maximal

2 Then E TAf-pessimistic ,f = Ω τ + n + μn log n . Proof. As in the proof of Theorem 1 there are three regions in the search space that correspond to phases of a run of Algorithm 1 with aging on f : the part where the function value is given by n − OneMax(x), the LeadingOnes path to the local optimum and the region of the global optimum. First assume that the maximal age τ is sufficiently large, say ω(μn log μ). As already discussed in Section 2 the probability that the algorithm initializes in some x ∈ OPT is 2−Ω(n) . Moreover, the probability to encounter OPT by optimizing the n − OneMax(x) part in the first phase is 2−Ω(n) . Analogously we can show that Algorithm 1 first hits the path to the local optimum at a linear distance to the local optimum with probability 1 − 2−Ω(n) . Let Li denote the set of bit strings with i 1-bits. Then, the local optimum belongs to Ln/4 and the probability that Algorithm 1 reaches the path at some point with at least n/8 1-bits n/8 is (n/8)/ i=1 |Li | = 2−Ω(n) . We can easily see that the path cannot be reached earlier by means of crossover as after initialization all search points have at least n/16 0-bits within the first n/4 of the bit string with probability 1 − 2−Ω(n) and the number of 0-bits is increasing during the n − OneMax(x) phase. Assume, there are b search points on the path. Then, the probability to create a new best search point on the path by means of mutation is O(b/(μn)) as at least a mutation of a single bit is needed. Considering a crossover operation of a search point x on the path and a search point y that has not yet reached the path, we easily get the same upper bound since the search point on the path, say x = 1i 0n−i , has to be selected first (probability at most b/μ), yi+1 = 1 second (probability at most 1/2), and the first crossover point needs to be i (probability at most 1/n). Clearly, it is not possible to create a new best search point with crossover of two search points on the path. In particular, this holds for the local optimum. Remember that at the point of time where the first search point in the local optimum is created all other search points in C are on the path with probability close to 1. Thus, the first search point in the local optimum is created by mutation. We still need to consider if crossover can asymptotically decrease the time we need to increase the number of best search points on the path from 1 to b. Remember that the probability to increase this number from b to b + 1 by means of mutation is Θ(b/μ). For crossover it is necessary to select a currently best search point first and thus, the probability to create a copy by means of crossover is also O(b/μ). We see that the lower bound for the LeadingOnes part

On the Benefits of Aging and the Importance of Details

69

 carries over from Algorithm 1 without aging [13] and we get Ω n2 + μn log n for the second phase. Once the complete population is of the form 1i 0n−i for possibly different values of i with 1 ≤ i ≤ n/4, the global optimum can only be reached via a direct mutation to the OPT region. Such a mutation has probability at most 1/((n/12)!) as at least n/12 bits in the second half of the bit string have to flip. Thus, the probability to create a global optimum by means of mutation is n−Ω(n) . Clearly, it is not possible to create a global optimum with crossover of two search points on the path. As the waiting time for a partial or complete  restart in the local optimum is at least τ , we need time Ω τ + n2 + μn log n to get into a situation where the first search point in the local optimum is removed due to the maximal age. We need at least one successful partial restart to obtain the global optimum which proves the claimed lower bound in this case. If the maximal age τ is smaller the search process is slowed down as it becomes harder to reach the local optimum. If τ is very small almost constantly new search points are created uniformly at random. In t time steps, at most t · (μ/τ ) new search points are created in this way. Each of these new search points is equal to an optimal search point with probability 2−n as discussed above. Such a process finds the global optimum on average in 2Ω(n) steps.  For f , the only difference between aging in the optimistic and pessimistic valuebased variant is the way partial restarts can be achieved. We therefore re-use the proof of Theorem 1 and 2 , respectively, and only consider this novel aspect here. The main difference is that crossover no longer helps in creating another locally optimal search point with age different from the first search point entering the local optimum. If we perform crossover with of x = 1n/4 03n/4 and y = 1i 0n−i with i < n/4 the age of the new search point is given by the age of the better search point, i. e., by x.age. This is no different from a copy of x. We become more precise in the two following theorems. Theorem 3. Consider f : {0, 1}n → R from Definition 2. Let Af-optimistic denote Algorithm 1 with aging in the optimistic value-based variant with population size μ ∈ N with μ ≥ 2 and μ = nO(1) , crossover probability pc with 0 < ε ≤ pc ≤ 1 − ε < 1 for any positive constant ε, k = O(1) crossover points in k-point crossover, and maximal log μ).

age τ =ω(μn



 n Then E TAf-optimistic ,f = O μ + log μ · τ + n2 + μn log n . Proof. Consider the first x ∈ C that enters the local optimum. It is not difficult to see that by the time half of the population is taken over by copies of x the rest of the population all are of the form 1(n/4)−1 03(n/4)+1 with probability close to 1. The expected time for copies to be created is bounded below by Ω(μ log n): If there are b copies the probability to create a better search point is O(b/(μn)) since at least a mutation of a single bit is needed. On the other hand, the expected time to increase the number of copies of the best to j is bounded above O(μ log j). Since initially we have b = 1 we obtain Ω(μ log n) as lower bound. On average in these steps already copies of the second best have been produced.

70

T. Jansen and C. Zarges

Since the selection for reproduction is uniform these copies are selected with higher probability than the first single best. This yields that all worse search points will be removed. The probability to create another locally optimal search point with age different from x is Ω(1/n) for Θ(μ log μ) steps. After this number of steps this probability can decrease even to 0 since after that time the whole population may be in the local optimum. However, the probability that another locally optimal search point with age different from x is created within these Θ(μ log μ) steps is

c μ log μ  1 − 1 − nc ≥ 1 − e−cc μ log(μ)/n for positive constants c and c . This is Θ(μ log(μ)/n) for μ n/ log(n) and Θ(1) otherwise. Thus, in comparison to the proof of Theorem 1 the factor μ for the expected number of partial restarts changes to O(μ · (1 + n/(μ log μ))) = O(μ + n/ log μ); yielding the theorem.  Theorem 4. Consider f : {0, 1}n → R from Definition 2. Let Af-optimistic denote the Algorithm 1 with aging in the optimistic value-based variant with population size μ ∈ N with μ ≥ 2 and μ = nO(1) , crossover probability pc with 0 < ε ≤ pc ≤ 1 − ε < 1 for any positive constant ε, k = O(1) crossover points in k-point crossover, and any maximal age τ (with τ = 2O(n) ).



  n 2 . Then E TAf-optimistic ,f = Ω 1 + μ log μ · τ + n + μn log n Proof. We can mostly reuse the proof of Theorem 2. Let x be the first search point that reaches the local optimum. Remember that the only difference between the optimistic and pessimistic value-based variant is the way partial restarts can be achieved as crossover no longer helps in creating another locally optimal search point with an age different to the age of x. We have already discussed in the proof of Theorem 3 that by the time half C is taken over by copies of x the rest of the search points are all of the form 1(n/4) − 0(3n/4)+1 with probability close to 1. The probability to create another locally optimal search points is at most 1/n. First assume that μ < n/ log n. Then, μ log μ/(n − 1) ≤ 1 holds. Analogously to Theorem 3 we see that another locally optimal search point with different age is created in μ log μ steps with probability  μ log μ log μ 2 · μn−1 µ log µ 2μ log μ 1 2μ log μ ≤ 1 − e− n−1 ≤ = ≤ 1− 1− μ log μ n n − 1 + 2μ log μ n 1 + 2 · n−1

μ log μ 2x as 1 − e−x ≤ 1+2x holds for x ≤ 1. For μ ≥ n/ log n we get 1 − 1 − n1 ≥ µ log µ

1 − e− n = 1 − e−Ω(1) = Ω(1). As n/(2μ log μ) = Θ(1) for μ ≥ n/ log n the claimed lower bound follows. 

Algorithm 1 is efficient on our example function f for both value-based variants. The gap between the lower and the upper bound on the expected optimization time is Θ(μ) in both cases as the probability p for creating a globally optimal search points by means of crossover after a partial restart can be bounded by p = Ω(1/μ) and p = O(1) respectively. The bounds derived for the pessimistic value-based variant are equal to the bounds for the age-based variant from [10]. We summarize our results and compare them to the results for the age-based variant [10] in Table 1 considering concrete example population sizes μ.

On the Benefits of Aging and the Importance of Details

71

Table 1. Bounds on the expected runtime for example population sizes age-[10], value-based

 pessimistic

 Ω n2 , O n2 

 Ω n2 , O n5/2

  μ = n/ log n Ω n2 log n , O n3  μ=n Ω n2 log2 n, O n3 log 2 n μ = n log n Ω n2 log3 n , O n3 log 4 n μ=2 √ μ =  n

4

optimistic

 value-based  Ω n3 , O n3 

 Ω n5/2 / log n , O n3 / log n

2  3 Ω n log n , O n  Ω n2 log2 n, O n3 log2 n Ω n2 log3 n , O n3 log4 n

Experimental Analysis

We supplement our theorems with experimental results since the theorems do not answer all questions. This is not only due to the gap Θ(μ) between the upper and lower bounds. Generally, asymptotic results may be misleading for typical problem sizes. As the size of the gap between upper and lower bound depends on the population size μ we further investigate the influence of this parameter. It is not obvious what good values for μ are. However, the theoretical results give hints. The bounds for the pessimistic value-based variant indicate that a smaller population size leads to a smaller optimization time. For the optimistic value-based variant, the bounds suggest μ = Θ(n/ log n) as a good choice. However, as our theoretical bounds are not tight, these speculations may be wrong. √ We do all experiments with population sizes μ ∈ {2, n , n/ log n

, n, n log n }. Clearly, μ = 2 is interesting as it is the smallest possible population size and possibly a good choice for the pessimistic value-based variant. For the same reason we √ pick μ = n/ log n

for the optimistic value-based variant. The choice μ ≈ n has often turned out to be a good choice [7]. Moreover we are interested in the effects of population sizes that are not sub-linear, and the effects further small increases have. Thus, we also pick μ = n and μ = n log n . We require τ = ω(μn log μ) and choose τ = 6μn log μ log n for our experiments. All bounds work for arbitrary constant crossover probabilities pc . We use pc = 0.5, a medium sized value. All proofs work for any constant number k of crossover points. We consider the commonly used 1-point crossover. Table 1 shows the resulting bounds on the expected optimization times for this setting. For each experiment we perform 100 independent runs and plot the results using box-and-whisker plots for n ∈ {10, 20, . . . , 1000}. Due to the excessive computation time, we consider only n ∈ {10, 20, . . . , 500} for μ = n (both variants) and μ = 2 (optimistic value-based) and n ∈ {10, 20, . . . , 300} for μ = n log n (both variants). The results are shown in Figure 1 and Figure 2 where the number of function evaluations are drawn in logarithmic scale. The variance decreases with increasing population size in both variants. This is caused by the probability of a partial restart that grows with μ. Consider the situation when the first search point reached the local optimum. In the extreme case μ = 2 there is only one other search point left that needs to enter the local optimum with a different age for a partial restart. For larger μ more trials are possible. If this fails a complete and costly restart is required.

T. Jansen and C. Zarges 1e+08 1e+07

iteration

iteration

1e+06 1e+05 1e+04

1e+07

1e+07

1e+06

1e+06

1e+05

iteration

72

1e+04 1e+03

1e+03

1e+03

1e+02

1e+02 mu=2 median:mu=2

1e+01

1e+02 mu=SqrtN median:mu=SqrtN

1e+01

100 200 300 400 500 600 700 800 900 1000 n

1e+05 1e+04

1e+01

100 200 300 400 500 600 700 800 900 1000 n 9e+07 1e+08

8e+07

1e+07 1e+07

7e+07

1e+06

1e+03

6e+07 iteration

iteration

iteration

1e+06 1e+05 1e+04

1e+05 1e+04

100

200

300

400

1e+07

mu=NLogN median:mu=NLogN

1e+01 500

4e+07

2e+07

1e+02 mu=N median:mu=N

1e+01

median:mu=2 median:mu=sqrt(n) median:mu=n/log(n) median:mu=n median:mu=nlog(n) 72.91n22 9.35n 2 3.76n log(n)

3e+07

1e+03

1e+02

5e+07

mu=NByLogN median:mu=NByLogN 100 200 300 400 500 600 700 800 900 1000 n

50

100

n

150 n

200

250

300

100 200 300 400 500 600 700 800 900 1000 n

√ Fig. 1. Results for the pessimistic value-based variant: μ = 2, μ =  n, μ = n/ log n (top, left to right) and μ = n, μ = n log n, medians (bottom) 1e+08

1e+08

1e+07

1e+07

1e+08 1e+07

1e+06

1e+04

1e+05 1e+04

1e+05 1e+04

1e+03

1e+03

1e+03

1e+02

1e+02

1e+02

mu=2 median:mu=2

1e+01 100

200

300

400

mu=SqrtN median:mu=SqrtN

1e+01 500

1e+01

100 200 300 400 500 600 700 800 900 1000 n

n 1e+08

8e+07

1e+07

1e+06

7e+07

1e+06

6e+07 iteration

iteration

1e+05 1e+04

1e+05 1e+04

5e+07

1e+03

3e+07

1e+02

2e+07

mu=N median:mu=N 100

200

300 n

400

1e+07

mu=NLogN median:mu=NLogN

1e+01 500

50

100

150 n

200

250

median:mu=2 median:mu=sqrt(n) median:mu=n/log(n) median:mu=n median:mu=nlog(n) 12.44n5/2/log(n) 2 4.50n log(n)

4e+07

1e+03 1e+02 1e+01

mu=NByLogN median:mu=NByLogN 100 200 300 400 500 600 700 800 900 1000 n

9e+07

1e+08 1e+07

iteration

1e+06 iteration

iteration

iteration

1e+06 1e+05

300

100 200 300 400 500 600 700 800 900 1000 n

√ Fig. 2. Results for the optimistic value-based variant: μ = 2, μ =  n, μ = n/ log n (top, left to right) and μ = n, μ = n log n, medians (bottom)

The variance is smaller for the pessimistic variant for μ ≥ n/ log n

. For μ ≥ n we see that generally no complete restarts occur. This is different for the optimistic value-based variant. For μ = n/ log n

similar observations can be made with respect to the upper quartiles. For smaller population sizes the effect is less visible. In the pessimistic value-based variant crossover can help in creating a search points with a different age in the local optimum whereas this is not true for the optimistic value-based variant. We compare the effects of the population size (bottom right in Figure 1 and 2). First, note that not only the theoretical bounds for μ ≥ n/ log n

are identical but in fact there is hardly any difference visible in the experimental results. For the pessimistic value-based variant, surprisingly, the algorithm with population√ size μ = 2 is clearly outperformed by its counterparts with population sizes μ =  n and μ = n/ log n

. This shows that (at least for not too large values of n) the asymptotic theoretical results are misleading. We speculate that our upper bounds are not tight. We support this hypothesis by plotting the fitted

On the Benefits of Aging and the Importance of Details 5

quotient:mu2/muSqrtN

quotient:mu2/muNByLogN

4 3.5 3 2.5 2 1.5 1

0

10 8 6 4 2

quotient:mu2/muNByLogN -0.000632x+2.66 100 200 300 400 500 600 700 800 900 1000 n

0

quotient:mu2/muSqrtN 0.008165x+1.47 100 200 300 400 500 600 700 800 900 1000 n

quotient:muSqrtN/muNByLogN

12

4.5

0.5

73

2

1.5

1

0.5

0

quotient:muSqrtN/muNByLogN -0.000389x+1.18 100 200 300 400 500 600 700 800 900 1000 n

Fig. 3. Quotient√of the observed medians of the pessimistic value-based variant with μ = 2 and μ =  n (left), μ = 2 and μ = n/ log n (middle) and of the optimistic √ value-based variant with μ =  n and μ = n/ log n (right)

lower bounds together with the empirical mean in Figure 1 (bottom right) and find a good fit in all cases. That larger population sizes outperform the choice μ = 2 at least partially contradicts the theoretical results. Thus, we take a closer look by comparing the quotients of the observed means. The theoretical bounds predict the quotient for μ = 2 over μ = n/ log n

√ to converge to 0. For μ = 2 over μ =  n it should be bounded above by a positive constant. Note that the theoretical bounds are asymptotic and predict this behavior for n → ∞. For μ = 2 over μ = n/ log n

(Figure 3, left) we see that after an increase for small values of n the quotient does indeed decrease. We fit the graph of the linear function a · x + b to the data and see that already for n ≤ 1000 the results match the asymptotic bounds. Things are different for √ μ = 2 over μ =  n (Figure 3, middle). Instead of being obviously bounded the quotient increases. This impression is confirmed when fitting a · x + b to the data. It is impossible to say from these experiments if the values of n considered are still too small or if the high variance is to blame. √ For the optimistic variant it is hard to tell if μ =  n or μ = n/ log n

performs better and if either our lower or upper bounds are tight. We again plot the derived lower bounds together with the empirical means (Figure 2, bottom right) and find a√good fit in both cases. Considering the quotient of the observed means of μ =  n over μ = n/ log n

(Figure 3, right) and fitting a · x + b to the data confirms that the quotient is slightly decreasing, contradicting the speculation that the derived lower bounds might be tight.

5

Conclusion

Static pure aging is an important concept in AISs for function optimization. We contributed to the understanding of this mechanism by presenting theoretical and experimental analyses. Building on a known example problem where aging cannot easily be replaced by other mechanisms like restarts we considered three different yet similar variants. Aging aims at increasing the diversity of the collection of search points and crossover works particularly well for diverse collections of search points. Thus, we analyze the performance of a simple AIS with static pure aging and crossover. The three variants considered, age-based, pessimistic and optimistic value-based static pure aging, differ only in the way an age is assigned to a new search if it is no improvement. Moreover, the difference is only present when the search point was created by crossover of two search points

74

T. Jansen and C. Zarges

with different function value. This tiny difference has a clear impact on the performance as was shown by our theoretical analysis and in the experiments. The role of static pure aging is far from being completely understood. While example problems help to see effects in a paradigmatic way, a more complete picture needs to be obtained by analyzing meaningful classes of problems. Moreover, it is desirable to develop techniques for a more detailed analysis that allows to close the gaps between our upper and lower bounds.

References 1. Castrogiovanni, M., Nicosia, G., Rascun` a, R.: Experimental analysis of the aging operator for static and dynamic optimisation problems. In: Apolloni, B., Howlett, R.J., Jain, L. (eds.) KES 2007, Part III. LNCS (LNAI), vol. 4694, pp. 804–811. Springer, Heidelberg (2007) 2. Cutello, V., Morelli, G., Nicosia, G., Pavone, M.: Immune algorithms with aging operators for the string folding problem and the protein folding problem. In: Raidl, G.R., Gottlieb, J. (eds.) EvoCOP 2005. LNCS, vol. 3448, pp. 80–90. Springer, Heidelberg (2005) 3. Cutello, V., Nicosia, G., Pavone, M.: Exploring the capability of immune algorithms: A characterization of hypermutation operators. In: Nicosia, G., Cutello, V., Bentley, P.J., Timmis, J. (eds.) ICARIS 2004. LNCS, vol. 3239, pp. 263–276. Springer, Heidelberg (2004) 4. Dasgupta, D., Ni˜ no, L.F.: Immunological Computation: Theory and Applications, Auerbach (2008) 5. de Castro, L., Zuben, F.: Learning and optimization using the clonal selection principle. IEEE Trans. on Evol. Comp. 6(3), 239–251 (2002) 6. de Castrop, L., Timmis, J.: Artificial Immune Systems: A New Computational Intelligence Approach. Springer, Heidelberg (2002) 7. Harik, G., Cant´ u-Paz, E., Goldberg, D., Miller, B.: The gambler’s ruin problem, genetic algorithms, and the sizing of populations. Evol. Comp. 7(3), 231–253 (1999) 8. Horoba, C., Jansen, T., Zarges, C.: Maximal age in randomized search heuristics with aging. In: Proc. of GECCO, pp. 803–810. ACM Press, New York (2009) 9. Jansen, T., Zarges, C.: Comparing different aging operators. In: Proc. of the 8th ICARIS, pp. 95–108. Springer, Heidelberg (2009) 10. Jansen, T., Zarges, C.: Aging beyond restarts. In: Proc. of GECCO. ACM, New York (to appear, 2010) 11. Jong, K.A.D.: Evolutionary Computation. A Unified Approach. MIT Press, Cambridge (2006) 12. Kelsey, J., Timmis, J.: Immune inspired somatic contiguous hypermutation for function optimisation. In: Cant´ u-Paz, E., Foster, J.A., Deb, K., Davis, L., Roy, R., O’Reilly, U.-M., Beyer, H.-G., Kendall, G., Wilson, S.W., Harman, M., Wegener, J., Dasgupta, D., Potter, M.A., Schultz, A., Dowsland, K.A., Jonoska, N., Miller, J., Standish, R.K. (eds.) GECCO 2003. LNCS, vol. 2723, pp. 207–218. Springer, Heidelberg (2003) 13. Witt, C.: Runtime analysis of the (μ+1) EA on simple pseudo-Boolean functions. Evol. Comp. 14(1), 65–86 (2006) 14. Zarges, C.: Rigorous runtime analysis of inversely fitness proportional mutation rates. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 112–122. Springer, Heidelberg (2008) 15. Zarges, C.: On the utility of the population size for inversely fitness proportional mutation rates. In: Proc. of the 10th FOGA, pp. 39–46. ACM Press, New York (2009)

Classifying in the Presence of Uncertainty: A DCA Perspective Robert Oates, Graham Kendall, and Jonathan M. Garibaldi School of Computer Science, The University of Nottingham {rxo,gxk,jmg}@cs.nott.ac.uk http://www.asap.cs.nott.ac.uk/

Abstract. The dendritic cell algorithm is often presented as an immuneinspired one class classifier. Recently the dendritic cell algorithm has been criticised as its current decision making stage has many serious mathematical flaws which bring into question its applicability in other areas. However, previous work has demonstrated that the algorithm has properties which make it robust to a certain source of uncertainty, specifically measurement noise. This paper presents a discussion about the role of uncertainty within classification tasks and goes on to identify the strengths and weaknesses of the dendritic cell algorithm from this perspective. By examining other techniques for protecting against uncertainty, future directions for the dendritic cell algorithm are identified and discussed. Keywords: Machine Learning, Dendritic Cell Algorithm.

1

Introduction

The dendritic cell algorithm (DCA) is an immune-inspired algorithm based on the function of biological dendritic cells. While placing its functionality into a traditional algorithmic category has been a matter of debate within the literature, it is primarily used as a one class classifier [1,2]. In [3] the relationship between the decision making phase of the DCA and traditional linear classifiers was highlighted. Being linked to linear classifiers implies that the DCA has the well-documented limitations that linear classifiers suffer from, primarily stemming from the limited set of achievable classification boundaries. However, the DCA also has beneficial properties lacking in most other machine learning algorithms. Of particular interest, from the perspective of tolerating uncertainty, is the DCA’s filter characteristic. This property of the DCA has been shown to allow the algorithm to ignore the high frequency components of its input signals [4]. For this discussion “uncertainty” is defined as inaccuracies in data or models that cause a drift between the hypothesis constructed by a system and what is happening in reality. Note that this in itself is a difficult thing to measure as the “reality” or “base-line truth” of many problems can only be estimated. Traditional machine learning techniques, such as neural networks and support vector machines assume that the input signals presented to them are accurate measurements. However, the DCA makes aggregated decisions over multiple time-scales, E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 75–87, 2010. c Springer-Verlag Berlin Heidelberg 2010 

76

R. Oates, G. Kendall, and J.M. Garibaldi

adding robustness to transient errors in the input signals. In this paper we aim to create a taxonomy of different sources of uncertainty for the classification problem and use it to identify those areas where the DCA is adequate and those areas that require improvement. The paper is structured as follows: in section 2 the dendritic cell algorithm is explained and expressed in terms of linear classifiers; in section 3 the sources of uncertainty are explored and techniques for managing them are introduced; in section 4 the DCA is discussed with reference to the sources of uncertainty and its benefits and short-comings are described accordingly and section 5 is a discussion about the DCA’s future as a classifier for data sets that suffer from uncertainty.

2

The DCA as a Classifier

The dendritic cell algorithm is an immune-inspired algorithm based on the operation of biological dendritic cells. It was first proposed as part of the ‘Danger Project’ [5]. As the DCA is a relatively new and still developing algorithm, several different versions of it exist in the literature. At the time of writing, the latest version is a variant of the deterministic DCA, (first published in [6]) introduced by Gu et al. [7]. Unless explicitly stated otherwise, when referring to “The DCA” it is this, latest version, of the DCA which is being referred to. The biological model which inspired the DCA is based on the principal that dendritic cells classify antigen as being either anomalous or normal. The DCA uses this metaphor to determine if items within a presented set are a member of a class of interest or not. The DCA is an expert system with four input heuristics. Three of these inputs are time-varying signals and one is a stream of integers representing an enumerated type. The three time-varying signals are: PAMP, danger and safe. PAMP is termed after its biological namesake, “pathogen associated molecular pattern”. It is constrained to operate in the range 0 to 100 and is a heuristic selected to rise proportionally to the presence of signatures that are indicative of the class to be identified. PAMPs in biology are signatures of known pathogens to be reacted to by the immune system [8]. The danger signal is also a time-varying heuristic, which also rises proportionally to the presence of the class to be identified, but is based on markers that are less reliable than specific signatures. In the biological model on which the DCA is based, the danger theory, the danger signal is evidence of cell damage and therefore secondary evidence that there is an invasive pathogen. The safe signal is a third time-varying heuristic, selected to rise proportionally to situations where the other two heuristics might cause a false positive. As a result, it is treated as an inhibitory signal. The final heuristic generates a stream of an enumerated type, termed antigen, also after its biological counterpart. This stream acts as a state description of the items within the problem domain to be classified. For example, in a computer security setting, the antigen heuristic simply passes the process identification numbers of the currently active processes, which are to be classified as anomalous or normal [1]. These inputs are all processed by the DCA to determine a classification for each antigen. The algorithm is based on a

Classifying in the Presence of Uncertainty: A DCA Perspective

77

population of cells, each being exposed to the same input signals, but different antigen. Before the input heuristics are presented to the cells two intermediate signals are generated, both using weighted sums. The first intermediate signal, CSM (named after the biological term “co-stimulatory molecule”) is a measure of the magnitude of the input signals. Weightings vary from implementation to implementation, but PAMP and safe are always given a greater weighting than danger. All signals affect CSM positively, thus constraining CSM to be greater than or equal to 0. The second intermediate signal is known as the K value. This is an abstract variable added to remove some of the biological terms from the algorithm and to simplify its computation [4]. The K value represents the instantaneous difference between indicators that the items detected are a member of the class of interest or not. As a result PAMP and danger affect this value positively and safe affects it negatively. The intermediate signals are presented to the cell population, along with the corresponding antigen stream. All cells are exposed to the intermediate signals equally, but are given different samples of the antigen stream. Over the lifetime of a cell the total amount of CSM and K sampled is recorded. When the CSM total exceeds a cell-specific threshold (termed the “migration threshold”) it performs a classification of its sampled antigen based on the total value of the K variable. By aggregating the classifications from all cells for the sampled antigen a population-wide consensus is reached and this is the final classification. Pseudocode for this implementation of the algorithm can be found in [7]. In [3] the equivalence of a single dendritic cell’s decision making to a linear classifier was demonstrated. It was shown that the migration threshold and decision boundaries for the cells formed hyperplanes in the input signal space. The position of these hyperplanes is given, using the standard dot-product representation for a linear classifier, in equations 1 and 2. yD (x) = WD , x − Mi

(1)

Where yD is the decision boundary, WD is a vector representing the weightings of the safe and danger signals with respect to the CSM signal and Mi is the migration threshold of the cell being considered. yC (x) = WC , x

(2)

Where yC is the classification boundary and WC is a vector representing the weightings of the safe and danger signals with respect to the K signal. This discovery brings into question the number of applications that the DCA can be successfully applied to. However, in [4] it was demonstrated that the DCA has a frequency dependent property which linear classifiers do not. The interaction between the migration threshold and the decision boundary means that the DCA places different amounts of emphasis on to different frequencies of input signal. This characteristic is given in equation 3. WL −1 WL −1 −jb((ω+(2gπ))) e g=0 b=0 jω G(e ) = (3) 2 WL

78

R. Oates, G. Kendall, and J.M. Garibaldi

Where G() is the frequency response of a dendritic cell, j is the imaginary number constant, ω is the frequency of the input signal and WL is the ratio between the CSM signal and the migration threshold of the cell. The typical shape of this response is given in Fig. 1.

Fig. 1. A typical frequency response from a dendritic cell. Changing the CSM, cell migration threshold and sample rate of the algorithm will scale this shape, but not alter its other characteristics. The vertical axis is the gain and the horizontal axis is the frequency as a percentage of the sampling frequency fs . Taken from [4].

As can be observed from Fig. 1, lower frequencies carry more weight in the decision making process than high frequencies, a typical response from a system filtering out transient measurement errors.

3 3.1

Uncertainty and Classification Sources of Uncertainty

Uncertainty is introduced by a wide range of sources, but for this discussion we shall examine noise-related sources, as they are the most quantifiable and algorithmically interesting. In the field of signal processing the distinction between different sources of uncertainty is made. Here, three sources of uncertainty, pertinent to this discussion, are defined.

Classifying in the Presence of Uncertainty: A DCA Perspective

79

Measurement Noise. A great deal of literature exists about frequency based techniques to remove measurement noise. An up to date reference on such techniques can be found in [9]. For this discussion we shall define ‘measurement noise’ as any interference which causes the perceived value of a measured variable to be different from the actual value of the variable. Possible sources include electro-magnetic interference and sensor calibration errors. For the classification problem measurement noise can result in the input data presented to the classifier being incorrect, or potentially more seriously, input data used in the training phase being inaccurate. Process Noise. Recursive estimation techniques such as Kalman filters [10] and Bayesian techniques such as particle filters [11] typically make the distinction between measurement noise and ‘process noise’. For this discussion the term ‘process noise’ is defined as any interference which causes a difference between the actual action from an operation or transformation and the requested action. Possible sources of process noise are mechanical slippage or timing errors in control systems. This source of uncertainty is not directly applicable to the classification problem. However, it is illustrative as it shows precedent for distinguishing between sources of uncertainty as will be discussed in section 3.2. Functional Noise. In [12] a learning algorithm is expressed as a search within the space H of all possible hypotheses, to identify the hypothesis which most accurately estimates f the function which defines perfect classification for a given problem. For this discussion the term ‘functional noise’ is applied to inaccuracies in classification brought about by an inadequate estimation of the true separating hyperplane for a given class. This term is not to be confused with the term ‘classification error’ which refers to the numerical value used to measure a classifier’s performance against a given data set and can be affected by functional noise introduced by the classifier and measurement noise introduced as the items being classified are observed. The term ‘functional noise’ has been selected to place this phenomena in the same terms as the other sources of uncertainty. Dietterich et al. describe three possible sources of this uncertainty [12]. Firstly, for a given set of training data, there may be multiple hypotheses that provide identical results, making identifying the true position of f impossible. Secondly, if f is surrounded by local optima, then a learning algorithm may be unable to reach that region of the search space. Finally, the true value of f may not be in the space of representable hypotheses, given restrictions on dimensionality of the separating hyperplane and the constraints imposed by the learning algorithm itself. 3.2

Compensating for Uncertainty

Three sources of uncertainty have been identified, two of which are directly applicable to the classification problem and a third, (process noise) which is included to allow discussion of the algorithms which take advantage of discriminating between different sources of uncertainty. Here a discussion is presented of the techniques used to compensate for these sources of uncertainty.

80

R. Oates, G. Kendall, and J.M. Garibaldi

Measurement Noise. Measurement noise is most commonly compensated for using frequency based techniques. The underlying assumption of these techniques is that the component of the signal which is incorrect operates within a different band of frequencies than the true signal. For sensor noise this is often a valid assumption, as it is the higher frequency components of the sampled data stream which introduce the uncertainty. From the perspective of computational classifiers, the input streams are sampled signals which are easily manipulated using digital signal processing techniques to achieve the separation or removal of frequency bands. These techniques can be undermined by applications where the frequency of the noise changes and applications where the noise is in an overlapping frequency band to the ‘true’ signal. The former problem can be overcome with adaptive filters, which change their transfer function (the mathematical formula that relates the input to the output) dynamically as the input changes [13]. Process Noise. While not directly applicable to the classification problem, it is possible that techniques to deal with process noise could provide the basis for dealing with other sources of uncertainty. Kalman filters are an example of a technique which places an emphasis on identifying different sources of uncertainty [10]. The Kalman filter is an estimation technique which attempts to identify the ‘true’ state of a system from a series of measurements of that system’s output and access to the inputs of that system. This is done by combining data from a pre-defined model of the system which not only takes into account the possibility that that measurements could be subject to noise, but the process itself could also provide an inconsistent output to the same input. A Markov chain is used to provide an estimate of the true state of the system using separate probability distributions to estimate the uncertainty introduced by the measurement noise and the uncertainty introduced by the process noise. By acknowledging the different sources of uncertainty, it is both possible to take full advantage of all of the data that is available to the system and to create a model which has high fidelity to the reality of the system. However, the Kalman filter does assume that the sources of uncertainty have Gaussian probability distributions. For many applications this assumption acts to add data to the model, giving the Kalman filter an edge over other estimation techniques. In situations where this assumption does not hold, non-parametric estimators using Bayesian mathematics, such as particle filters, out-perform the Kalman filter significantly [14]. Functional Noise. Techniques for reducing functional noise exist within the literature. Reducing the error rate of a classifier will obviously have an effect on the functional noise within the classification process. However, functional noise is considered to be a source of uncertainty for the end classification of the system and therefore, is also effected by issues such as training data bias, training data accuracy (both label and measurement accuracy), and over/under fitting. Using this definition of functional noise, it can be estimated from the classification error, so long as the dataset is a truly representative sample of the entire set of application data and the effects of measurement noise are negligible. As a result,

Classifying in the Presence of Uncertainty: A DCA Perspective

81

any attempt to reduce the classification error, can also be perceived as an attempt to reduce the uncertainty caused by functional noise. Learning algorithms, as applied to the classification problem, attempt to produce an accurate estimate of the separating hyperplane through an optimisation process, minimising the classification error. However, this does not take into account errors or bias in the training data and neither does it typically consider that the true separating hyperplane cannot be represented by the classifier being optimised. In addition, issues common to optimisation, such as local minima and plateaus in the search space, can harm the performance of a classifier and increase the functional noise. A family of methods which attempt to compensate for the uncertainty introduced by the representation and optimisation issues are ‘ensemble classifiers’ [12]. By aggregating the outputs from a set of diverse classifiers, ensemble classifiers can produce equivalent hyperplanes that are a more accurate representation of the true hyperplane and can describe complex hyperplanes that exist outside the set of hypotheses representable by the individuals within the ensemble. This adds complexity to the training of the algorithm as it introduces two new problems, namely how to enforce diversity amongst the population of classifiers and how to aggregate the results in a meaningful manner. Less work has been done on correcting for uncertainty introduced by errors in the training data itself. In [15] multiple classifiers are trained on different subsets of the training data. The performance of these classifiers is analysed statistically and compared. By changing the boundaries of the subsets, it is theorised that erroneously labeled training data will be exposed by the effect they have on the overall classification error. This assumes that the error is only in the label and that a sufficient amount of accurately labeled training data exists.

4

Uncertainty and the DCA

With an understanding of the sources of uncertainty and the techniques used to compensate for them, we now discuss the dendritic cell algorithm and identify its strengths and weaknesses in terms of how it manages uncertainty. 4.1

Measurement Noise in the DCA

In [4] it was demonstrated that the relationship between the magnitude of the signal and the number of samples processed introduced a frequency dependent property into a single dendritic cell. The general shape of the frequency response of the cell means that in applications where the measurement noise exists at a higher frequency than the underlying signal, an appropriately tuned cell places less emphasis on the noise component of the signal. Not only is a dendritic cell a filter, it is also an adaptive filter, as the number of samples over which a classification is performed is proportional to the cell’s migration threshold and inversely proportional to the magnitude of its input signals. No empirical evidence exists to support if this is an appropriate heuristic to use to adjust the frequency response of the cell. However, intuitively, a high magnitude signal

82

R. Oates, G. Kendall, and J.M. Garibaldi

is indicative of a high signal to noise ratio (SNR), where as a low magnitude signal is indicative of a low SNR. As a result, reducing the number of samples taken in situations with a high SNR makes logical sense, as it reduces the effect of filtering. Conversely, increasing the number of samples taken in situations with a low SNR also makes logical sense as it increases the effect of filtering. As yet no methodology exists for tuning the migration thresholds of a population of dendritic cells as it has been demonstrated that traditional frequency based techniques cannot analyse populations of cells acting as asynchronous filters [16]. In addition, no work has been carried out exploring the effects that aggregating the classifications across the entire population of cells has on the frequency domain. 4.2

Functional Noise in the DCA

The DCA has no training phase, so the functional noise comes from poor construction of the input heuristics and the limitations placed on the classification phase by the linear nature of the classifying hyperplanes. The DCA has the problems that all expert systems suffer from, in terms of the transformation of expert domain knowledge into meaningful heuristics. However, the DCA has an additional issue, as its weighted sums for the intermediate signals are hard coded, so the scaling and shape of the produced heuristics have to be hand-tuned to match the weights. The final classification performed by the algorithm is dependent on the aggregation technique used to identify the final classification over the entire population. In the latest version of the DCA, this is performed by calculating the Kα value, first proposed in [6]. The Kα value for a given antigen is the average value of the K signal for every DC which has migrated after sampling that antigen. This is expressed in equation 4. N Km (4) Kα = m=0 N m=0 αm Where N is the number of cells in the population, Km is the K value for DC m and αm is the number of antigen of type α presented by cell m. Like the K signal, negative values of Kα indicate that the antigen is not a member of the set to be identified and positive values indicate membership. As a result, the magnitude of the value is indicative of the certainty of the classification in terms of its distance from the separating hyperplane. However, as an uncertainty measure, it is only valid if the classification hyperplane used is an appropriate estimation of the true classification hyperplane. In addition, the maximum K value magnitude varies from application to application, so the size of Kα is relative to other classifications from the same application rather than an absolute measure of certainty. 4.3

The Boundaries between Decision and Action

Before discussing the DCA as a classifier it is important to address a subtle difference between the classification methodology of the DCA and that of typical

Classifying in the Presence of Uncertainty: A DCA Perspective

83

classifiers. The DCA is based on the operation of biological dendritic cells. Under the biological model on which the DCA is based, DCs are a decision making cell, but are unable to act on the final classification. This is a result of their interaction with T Cells. T Cells are the actors for the decisions made by dendritic cells (i.e. the cells which actually act on the decision), but T Cells undergo a process called negative selection, which removes T Cells with an affinity to antigen that represents the host organism, (known as ‘self’) [5,8]. This means that even if a DC inaccurately identifies a self antigen as a threat, there is no T Cell to respond. This adds an extra layer to the classification process, which prevents false positives from occurring for items that have been previously labeled as not being part of the set to be identified. This is akin to the use of ensemble classifiers, but in this case the diversity of the classification hyperplane is enforced by only training the T Cell phase with counter-examples. This separation of decision and action allows a more complex classification process to emerge as the result of the interaction between the two cell types, where only one is actively making a decision, but the communication of that decision to the rest of the system is controlled by the presence/absence of the corresponding actor. Such a separation between decision and action is not typically recognised in traditional classification techniques, where the output from the classifier is rarely postprocessed before being communicated to the next stage in the system. It is of note, that this interaction means that the biological system on which the DCA is based has very little cause to be concerned by false positives in the classification. Not only are serious errors not communicated to the rest of the system as a result of the process described, but also, false positives which wrongly identify innocuous non-self antigen as potential threats are unlikely to have serious ramifications for the body. Conversely, false negatives, the incorrect classification of pathogens as innocuous, have the potential to be fatal to the host. This suggests that the underlying mechanism for dendritic cells may be tuned in favour of reaction, rather than inhibition.

5

Discussion

By placing the DCA in the context of classifying with uncertainty, it is hoped to provide a new direction for the algorithm’s development. What follows is a discussion of the DCA’s various stages within this context. 5.1

Input Signal Identification

The first phase of implementing the DCA is the construction of input heuristics. If one explores the original biological framework on which the DCA is based, it becomes clear these input heuristics are not as functionally equivalent as they have been presented in the past. In terms of the information content PAMPs are far less ambiguous compared to the other signals and should be considered in terms of known signatures of the set of interest. Conversely, danger and safe are merely secondary indicators which can act to provide additional information

84

R. Oates, G. Kendall, and J.M. Garibaldi

in the absence of clear data. In terms of classifier systems, these relationships are often left to the system to learn via the weights associated with the input parameters. However, in terms of uncertainty, the characteristics of the measurement noise associated with the signals are application specific. Depending on the distance metrics used to assess the signatures and the sensors/data sources used to provide the secondary indicators, the noise characteristics could be very different. Taking inspiration from the field of Kalman filters, a domain expert could provide some measure of certainty, (for example a probability density function to adequately model the noise characteristic). This would allow the algorithm to make more informed decisions based on the true state of the data rather than the potentially noisy measurements. To use a Kalman style estimation would require that the data being analysed can be modeled by a Markov chain, (i.e. that only knowledge of the current state is required to estimate the next state) and that there is some relationship between the parameters in the input data that can be modeled by a domain expert or modeling process. 5.2

Training and Local Classification

The DCA currently has no training phase. This means that it is left to the implementor to construct the input heuristics to match the decision hyperplanes. This is a non-trivial task and it is argued that this adds complexity to the implementation and functional noise to the classification. The use of a training phase is supported by the biological model on which the DCA is based. While the biological DCs are part of the innate immune system and therefore, non-adaptive over the life-span of the host organism, the behviour of the DCs must have been subject to evolutionary pressure and therefore, adaptive over the life-span of the species. On a practical level, for the DCA to be useful to computer scientists and engineers, it must be applicable to multiple data sets, with a minimum of implementation effort. This means removing the current cumbersome trial and error tuning methodology and replacing it with a more structured adaptation. However, as discussed in [16], automatically tuning the frequency response of a population of cells is a non-trivial task. Adding the training of some form of nonlinear classifier to that process would create a significantly harder problem. One possible solution would be to create a repertoire of classifiers, each physically constructed to accept a different number of inputs and trained using training data smoothed over different time scales. Rather than aggregating the K signal, the DCs should individually aggregate the PAMP, danger and safe signals sampled by the cell and present these averages to the classifier trained over the nearest timescale to that of the DC’s life span. This would add a non-linear classification phase to the algorithm and retain the multi-timescale property of the algorithm, thus compensating for both functional and measurement noise. 5.3

Aggregated Classification

For the aggregation phase of the DCA, where the individual cell decisions are processed into a single decision, it may be possible to use techniques from the

Classifying in the Presence of Uncertainty: A DCA Perspective

85

ensemble classifier community. Statistical techniques for combining the outputs of several classifiers exist within the ensemble classifier community and could improve the output of the DCA both in terms of its functional noise and in terms of producing a cross-application measure of certainty, which would be preferable to the current Kα technique. In addition to the method of aggregation, the means of segmenting the system in terms of how often to check the population for migrating cells must also be addressed. Work in this area has already been carried out by Gu et al. [7]. The aggregated classification from the population of cells could be postprocessed to reduce the functional noise. Taking the biological model, the individual classifiers could be trained to “err on the side of caution”. By subsequently post processing the final aggregated result with a second classifier, trained using counter examples of the class, a similar effect to the interaction of DCs with T Cells is achieved. In terms of ensemble classifiers, so long as both classifier equivalents have greater than 50% accuracy, the resultant classifier should be an improvement on the individuals, [12]. This set up suggests a duality between the software implementations of PAMPs and the negative selection process, where both represent signature based examples and counter examples of the class of interest. In regions of the input space where signatures are not available, this self/non-self model could be supported by the danger theory inspired signals of danger and safe. 5.4

Future Work

Fig. 2 is a graphical representation of the proposed system. In this model a sample of the input data is used to train a repertoire of classifiers over several timescales. The antigen heuristic feeds the abstracted state information into the cell population in parallel to the input data presentation. Note that the antigen heuristic is linked to the input data with dashed lines to indicate that it can either be the transformation of features of the input data set or sourced by an expert. When cells migrate they present the sampled data to the classifier which most closely matches their time frame. The resulting classification is recorded by the cell and reported along with the sampled antigen. Population aggregation attempts to form a population wide consensus for the classification of individual antigen. This phase would also be the opportunity for signature based false positive correction. The output from this aggregation would be a probabilistic antigen array, correlating antigen with the probability that they have set membership. The obvious next step for this discussion is an empirical exploration of this system contrasted with traditional machine learning techniques. The structure presented in Fig. 2 attempts to solve many of the noise reduction and classification problems simultaneously, where as traditional machine learning techniques will be better suited to a more modular approach, using filters to remove measurement noise from the input data and probabilistic methods from ensemble classifiers to aggregate the output. Once the algorithm has been created, it is then important to assess how much effect the proposed changes have and if they

86

R. Oates, G. Kendall, and J.M. Garibaldi

Fig. 2. The proposed structure of the robust classification algorithm

warrant the increased computational cost that they add to the existing DCA. This is a significant undertaking, but could potentially yield an immune inspired algorithm that competes with traditional classification techniques, (such as neural networks) in data sets which contain high levels of uncertainty, in terms of classification accuracy.

6

Conclusions

Definitions of various sources of uncertainty have been provided and these definitions have been used to explore how the DCA functions in terms of classifying uncertainty. A possible modification of the algorithm has been explained, though there is much work to do before it can be fully implemented. Many open questions about how such a system can be implemented and on which types of problems this system would be a useful classifier need to be addressed before this undertaking can begin. By framing the DCA in the context of classifying with uncertainty it is the feeling of the authors that it is possible to correct for many of the short-comings of the DCA identified within the literature, while retaining the identified benefits that it provides.

Acknowledgments Many thanks to the staff of both the Intelligent Modelling and Analysis (IMA) and the Automated Scheduling Optimisation and Planning (ASAP) research groups for their discussions about the topics within this paper. Special thanks to Jan Feyereisl for his insights into the direction for this paper.

Classifying in the Presence of Uncertainty: A DCA Perspective

87

References 1. Greensmith, J., Aickelin, U., Twycross, J.: Articulation and clarification of the dendritic cell algorithm. In: Bersini, H., Carneiro, J. (eds.) ICARIS 2006. LNCS, vol. 4163, pp. 404–417. Springer, Heidelberg (2006) 2. Oates, R., Greensmith, J., Aickelin, U., Garibaldi, J.M., Kendall, G.: The application of a dendritic cell algorithm to a robotic classifier. In: de Castro, L.N., Von Zuben, F.J., Knidel, H. (eds.) ICARIS 2007. LNCS, vol. 4628, pp. 204–215. Springer, Heidelberg (2007) 3. Stibor, T., Oates, R., Kendall, G., Garibaldi, J.M.: Geometrical insights into the dendritic cell algorithm. In: [17], pp. 1275–1282 4. Oates, R., Kendall, G., Garibaldi, J.M.: Frequency analysis for dendritic cell population tuning. Evolutionary Intelligence 1(2), 145–157 (2008) 5. Greensmith, J., Aickelin, U., Cayzer, S.: Introducing dendritic cells as a novel immune-inspired algorithm for anomaly detection. In: Jacob, C., Pilat, M.L., Bentley, P.J., Timmis, J.I. (eds.) ICARIS 2005. LNCS, vol. 3627, pp. 153–167. Springer, Heidelberg (2005) 6. Greensmith, J., Aickelin, U.: The deterministic dendritic cell algorithm. In: [18], pp. 291–302 7. Gu, F., Greensmith, J., Aickelin, U.: Integrating real-time analysis with the dendritic cell algorithm through segmentation. In: [17], pp. 1203–1210 8. Greensmith, J.: The Dendritic Cell Algorithm. PhD thesis, The University of Nottingham, Computer Science, Jubilee Campus, Wollaton Road, Nottingham, NG8 1BB (2007) 9. Ifeachor, E., Jervis, P.B.: Digital Signal Processing: A Practical Approach, 2nd edn. Prentice-Hall, Englewood Cliffs (2001) 10. Kalman, R.E.: A new approach to linear filtering and prediction problems. Transactions of the ASME–Journal of Basic Engineering 82(Series D), 35–45 (1960) 11. Gordon, N.J., Salmond, D.J., Smith, A.F.M.: Novel approach to nonlinear/nongaussian bayesian state estimation. IEE Proceedings of Radar and Signal Processing 140(2), 107–113 (2002) 12. Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000) 13. Haykin, S.: Adaptive Filter Theory, 4th edn. Pearson Education, London (January 2004) 14. Choset, H., Lynch, K.M., Hutchinson, S., Kantor, G.A., Burgard, W., Kavraki, L.E., Thrun, S.: Principles of Robot Motion: Theory, Algorithms, and Implementations. MIT Press, Cambridge (June 2005) 15. Venkataraman, S., Metaxas, D., Fradkin, D., Kulikowski, C., Muchnik, I.: Distinguishing mislabeled data from correctly labeled data in classifier design. In: 16th IEEE International Conference on Tools With Artificial Intelligence (ICTAI 2004), Boca Raton, FL, pp. 668–672 (2004) 16. Oates, R., Kendall, G., Garibaldi, J.M.: The limitations of frequency analysis for dendritic cell population modelling. In: [18], pp. 328–339 17. Rothlauf, F. (ed.): Proceedings of Genetic and Evolutionary Computation Conference, GECCO 2009, Montreal, Qu´ebec, Canada, July 8-12. ACM, New York (2009) 18. Bentley, P.J., Lee, D., Jung, S. (eds.): ICARIS 2008. LNCS, vol. 5132. Springer, Heidelberg (2008)

Insights into the Antigen Sampling Component of the Dendritic Cell Algorithm Chris. J. Musselle Dept. of Computer Science, University of Bristol, UK [email protected]

Abstract. The aim of this paper is to investigate the antigen sampling component of the deterministic version of the dendritic cell algorithm (dDCA). To achieve this, a model is presented, and used to produce synthetic data for two temporal correlation problems. The model itself is designed to simulate a system stochastically switching between a normal and an anomalous state over time. By investigating five parameter values for the dDCA’s maximum migration threshold, and benchmarking alongside a minimised version of the dDCA, the effect of sampling using a multi-agent population is explored. Potential sources of error in the dDCA outputs are identified, and related to the duration of the anomalous state in the input data.

1

Introduction

The Dendritic Cell Algorithm (DCA) is an immune inspired algorithm that was developed by Greensmith [1] as part of an interdisciplinary research project between computer scientists and immunologists. The algorithm is an abstract model of dendritic cell behaviour according to the paradigm of danger theory [2], and aims to perform anomaly detection by correlating a series of informative signals (termed either ‘danger’ or ‘safe’), with a sequence of repeating abstract identifiers (termed ‘antigens’) in the dataset. The DCA is based on complex processes in the human immune system, however this has resulted in an algorithm with many interacting components and parameters, which has made identification of the key underlying mechanisms a difficult process. This lack of understanding in the factors responsible for its performance, has meant much trial and error in finding suitable parameters when implementing the DCA [3]. A major drawback of the DCA is its reliance on user expert knowledge to define both the parameter values, and also the mapping from the raw data into appropriate inputs for the DCA (antigens and signals). This means that although many applications of the DCA are built on the same underlying framework, the definition of inputs and parameters can be somewhat arbitrary. This is undesirable as user-chosen heuristics are unlikely to give the best results, especially as there is dependency not only between the parameters themselves but also between the parameters and the input data. E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 88–101, 2010. c Springer-Verlag Berlin Heidelberg 2010 

Insights into the Antigen Sampling Component of the DCA

89

A solution to the above shortcomings would be to have an automated process for determining the mapping of the inputs to the DCA, as well as the appropriate parameters. However the relationship between input data and DCA performance is not fully understood, and previous work attempting to address the problem of parameter tuning has proved unsuccessful [4]. Previous work by Stibor et al. [5] has investigated the signal processing side of the algorithm. As a complement to the work of Stibor et al., this current paper aims to explore the antigen sampling side of the algorithm, by designing a model to generate controllable synthetic data for two temporal correlation problems. It is hoped that by examining how the DCA fairs against a selection of different problem scenarios, insight will be gained on how best to frame the problem for the DCA to solve. For the purpose of this paper the deterministic version of the DCA (dDCA) will be used for ease of analysis. This paper is organised into the following sections. Section 2 covers the background of the DCA, the dDCA and related work. Section 3 outlines the problem statement and proposed solution. Section 4 covers the implementation, with details of the model used to generate the synthetic data, and the experimental design. Section 5 is the evaluation and discussion, with Section 6 stating the conclusions and further work.

2 2.1

Background Versions of the DCA

The DCA has undergone many revisions since its original inception, resulting in multiple versions of the algorithm in the literature. The two main versions are the original or classic version of the DCA [3], detailed in Greensmiths PhD thesis [1], and the deterministic version of the DCA (dDCA), presented in [6]. The original DCA has in excess of 10 parameters, and employs many stochastic elements, making it exceptionally difficult to formally analyse. Therefore, the deterministic version was formulated to help gain better insight into the underlying mechanisms of the algorithm. The main differences between the two, are that all stochastic components were removed or replaced, and the signal processing calculation was simplified, originally proposed in [4]. Other modified versions have also been presented; which have tended to either add additional components to the algorithm [7] [8], or are based on the underlying framework but differ in their method of implementation [9]. In this study, only the dDCA will be considered, as it represents the simplest version of the algorithm for analytical purposes, and has also shown to give comparable results to the classic DCA [3] [6]. For a full account of the original DCA’s inner workings, and the abstractions made from its biological roots, the reader is referred to [10]. 2.2

The Deterministic DCA

What follows in this Section, is a detailed description of the components to the dDCA, which can be seen to have three related parts, the signal processing

90

C.J. Musselle

calculation, the antigen sampling mechanism, and calculation of the output metric. Pseudo code for the dDCA can be found in [6]. The inputs to the dDCA take two forms, antigens and signals. Individual antigen are elements of a finite set of integers, of size L, that act as an abstract identifier for some component or event occurring within the monitored system. Examples of such are the process IDs in a computer, the IP addresses in a network, or a node in a sensor network. The number that occur at any one time step is variable, with repeated entries possible. Each input signal is a time series of length T containing real values, normalised on the interval [0, 100]. Two are used in the dDCA, termed the ‘safe’ and ‘danger’ signal, and are set to monitor some informative feature of the data determined a priori by expert knowledge of the system in question. Therefore at each time step t = {1, . . . , T }, the inputs to the dDCA consist of the values of the safe and danger signals St and Dt , as well as a variable number of antigens At . The algorithm itself consists of N virtual dendritic cell (DC) agents, which carry out the signal processing and antigen sampling components. Signal Processing and Antigen Sampling. All N DCs in the population, sample the same signal values at each time step t, and calculate the following values of csm = St + Dt

and k = Dt − 2St .

The variables csm (from the term ‘co-stimulatory molecules’ [10]) and k are stored internally by each agent DCi = {1, . . . , N }, as two separate cumulative sums, call them CSMi and Ki . The number of antigens present at each time step At (termed ‘pool’), is assigned to the DC population in a round-robin fashion. That is, each DC agent is assigned one antigen in turn, until all antigens in the pool have been assigned. This process is then repeated with the next pool of antigens in subsequent time steps, however you only give a DC agent a second antigen, when all DCs have received one, and a third after every DC agent has received two, etc. It is of note that at each time step t, all DCs will process the signals and update their values of CSMi and Ki , but if At < N only a fraction of the DCs will sample additional antigens. Each DCi is assigned a different threshold Mi , termed the ‘migration’ threshold, and once CSMi ≥ Mi , DCi outputs the value of Ki , now termed Kout . The time steps up to and including when CSMi ≥ Mi are termed the DCs ‘lifetime’. For all antigens sampled by DCi during its lifetime, they are tagged as normal if Kout < 0 and anomalous if Kout > 0. The results of the tagging are logged, and the values of CSMi and Ki reset to zero. All sampled antigens are also cleared. DCi then continues to sample signals and collect antigens as before. Final Classification - the M CAV and Kα Metric. In the original DCA, a final metric, termed the Mature Context Antigen Value or M CAV is calculated for each unique antigen type l = {1, . . . , L} according to M CAVl =

Anomalousl , T otall

Insights into the Antigen Sampling Component of the DCA

91

where Anomalousl is the number of times antigen l was tagged as anomalous, and T otall the amount of times antigen l was tagged in total. The M CAV therefore can be thought of as the probability that a given antigen type is anomalous. This value of the M CAV is then thresholded to achieve the final binary classification of normal or anomalous to each antigen type. The Kα metric, an alternative metric to the M CAV , was proposed with the dDCA in [6]. The Kα uses the average of all output values Kout as the metric for each antigen type, instead of first thresholding them at zero into binary tags. The rational behind this is that with Kα the magnitude of Kout is taken into account, not just its sign, and therefore may offer more polar separation between antigen types. However, the use of Kα requires the prior knowledge of all signal values in order to calculate the appropriate threshold of classification, and is also dependent on the weights used to calculate the individual k values. In the work presented, only the former M CAV metric is used, as it generates a more intuitive output score, capable of being interpreted without the definition of an additional threshold. 2.3

Related Work

In [4] the classic DCA was analysed using frequency analysis, which equated a single DC agent to a combination of filters, specifically those of a moving average filter followed by a downsampler. An attempt was made to automatically tune the Mi distribution using a model representing the transfer function of the combined filters. The results were however unsuccessful, and in an attempt to extend the model to represent multiple DC agents, limitations of the frequency analysis approach, linked to the heterogeneous nature of the DC population were identified [11]. In [5] the authors showed that the signal processing component of the DCA functions as a collection of linear classifiers. Further more, these were also shown to be parallel across the population of DCs, which suggests severe limitations in the regions of signal space the DCA can distinguish between. Across a population of DCs, the positions of the migration decision boundaries were observed to be dependent on past input signal values. This makes the number of DC migrations at any one point in the algorithm very hard to predict.

3

Problem Statement and Proposed Solution

Some of the main barriers to understanding the DCA are the high number of user specified parameters, and the inherent interactions among the algorithm’s many components. In addition, the mapping to the inputs for the algorithm must also be user defined, making it difficult to compare the results of DCA applications, as the nature of the inputs will tend to vary in each implementation. The work of Stibor et al [5] focused solely on the signal processing component of the deterministic DCA, and suggested that more work be done to explore

92

C.J. Musselle

the antigen sampling elements of the algorithm, and its interplay with the signal processing side. This paper sets out to investigate this, through the use of controllable synthetic data, to generate different problem scenarios. Two experiments will present different types of temporal correlation problems. One will vary the frequency of anomalies in the data set, while the second will investigate how offsetting the antigens and signals affects the outputs of the dDCA. The aim is to highlight which problem scenarios the dDCA performs best at, and what parameter values give these results relative to the input data. The antigen sampling of the dDCA is controlled by the minimum and maximum values of Mi , which dictate the size of the window (in terms of signal time steps) in which the antigen are correlated with the signals [4]. These will be the primary parameters of interest. Also, to investigate the effectiveness of population sampling in general, which is employed by the dDCA and all versions of the DCA, a simplified version of the dDCA, termed the minimised dDCA (min-dDCA), will also be implemented as a benchmark. In the min-dDCA, the usual population sampling strategy is replaced by a direct 1-to-1 correlation between signals and antigens.

4

Implementation

This section describes the methodology of the experimental process providing a detailed account of the model used to generate the data and the experimental design for the temporal correlation problems. Section 4.1 describes in detail the stochastic model used to generate synthetic data for use with the dDCA. The model is designed to simulate a simple system that transitions between a normal and an anomalous state repeatedly over a fixed number of time steps. Data are then generated, taking the form of antigens and signals to be used as inputs for the dDCA. The values of the signals and antigens at each time step are dependent on the underlying state (i.e. normal or anomalous) in a stochastic fashion. The model has three main components which correspond to the generation of the underlying states, the signals, and the antigen pool at each time step in the simulation. In Section 4.2, the two versions of the dDCA used in this paper are explained, along with the two experiments E1 and E2, that were designed to evaluate their performance. 4.1

Synthetic Data Model

The first component of the model generates a state string q of length n, whose terms qt = {q1 , . . . , qn } ∈ {N, A} and determine the state of the system at time step t in the simulation (‘N’ indicates a normal state, and ‘A’ an anomalous state). To generate q, a simple two state Markov Chain is used, shown in Figure 1a , whose states correspond to normal and anomalous, with transitions between them determined by the probabilities PNA and PAN . The automaton is started

Insights into the Antigen Sampling Component of the DCA

93

Fig. 1. Overview of the components of the model used to generate synthetic data. Part a) shows the two state Markov Chain used to generate the state string q, part b) the distributions for safe and danger signals in the normal and anomalous state, and part c) the distributions over the antigen alphabet L for each state.

in the normal state and run for n consecutive iterations to generate the output string of states q. The model then uses q to generate two signal time series st = {s1 , . . . , sn } and dt = {d1 , . . . , dn }, whose elements are all real valued on the interval [0, 100], and correspond to the levels of danger and safe signals at time t. These values are generated by drawing from a set of two distinct but overlapping beta distributions, the densities of which are dependant on q (t). For simplicity, the distributions for st and dt for each state are skewed mirror images of each other, and their positions are exactly opposite for the two separate states of normal and anomalous, as shown in Figure 1b. This results in predominantly low values of danger and high values of safe signal on time steps where the state sequence is the normal state, and vice versa for the anomalous state. The choice of beta distributions was made as they have the useful properties of being bounded and easily tunable to give skewed distributions. The sequence of antigens A is coded for in a m-by-t matrix with m the number of antigens in the pool per time step t in the simulation. Each pool of antigens

94

C.J. Musselle

A (∗, t), was obtained by drawing values from discreet beta distributions over the finite antigen alphabet L, which dictates all possible antigen types. As with the signal time series, the distribution drawn from is determined by the state at qt . These distributions, shown in Figure 1c, differ in their range and density over the alphabet L, resulting in two classes of antigen, LN and LA , corresponding to the normal and anomalous state. If these two distributions overlap, then a fraction of the alphabet LOverlap appears under both states. In summary, four sequences are generated using this model for synthetic data, whereby at time step t in the simulation qt determines the true state of the system as either normal or anomalous, st and dt are the levels of safe and danger signal respectively, and A (∗, t) are the pool of antigens that occur. All parameters used for this model, along with their default values are shown in Table 1. Table 1. Default Model Parameter Values – Those varied in experiment E1 are annotated with * Parameter

Value

Description

Parameters that define state sequence q n PNA PAN

500 0.1* 0.5*

Number of time steps in simulation. Probability of transition from normal to anomalous state. Probability of transition from anomalous to normal state.

Parameters that define signal time series s and d B (αSN , βSN ) B (αSA , βSA ) B (αDN , βDN ) B (αDA , βDA )

(10, 4) (4, 10) (4, 10) (10, 4)

Beta Beta Beta Beta

parameters parameters parameters parameters

for for for for

s distribution in normal state. s distribution in anomalous state. d distribution in normal state. d distribution in anomalous state.

Parameters that define antigen sequence A L LN LA LOverlap m B (αA , βA )

4.2

10 0.5 0.5 0 100 (3, 3)

Antigen alphabet size. Fraction of L that occurs in the normal state. Fraction of L that occurs in the anomalous state. Fraction of L that can appear in both states. Events per time step. Beta parameters for the antigen distributions, same for both normal and anomalous state.

Experimental Design

The research reported here investigates how changing the threshold range across the DC population affects the dDCA output for two temporal correlation problems. Two versions of the dDCA are used in these experiments. The first version is the standard dDCA, with parameters set to the literature values [6], unless stated otherwise. The second is a further simplified version of the dDCA, the min-dDCA.

Insights into the Antigen Sampling Component of the DCA

95

The min-dDCA is implemented here as a benchmark, to investigate what happens when heterogeneous population sampling, a strategy common to all versions of the DCA, is replaced by a simple 1-to-1 correlation between signals and antigens. In this way it is hoped that insights into how the addition of a population of DCs with variable thresholds impacts on performance, and how best to set the parameters that dictate this variability. The min-dDCA is a version of the dDCA with only a single DC agent instead of a population of DCs. This one DC performs all the sampling and signal processing throughout the simulation. No migration threshold M is considered, instead the DC will have a lifetime of one, meaning that at every time step in the simulation, the DC will tag all antigens at that time step. The threshold minimum and maximum for the standard dDCA are set to be related to the average total signal value per time step, AveT otal, which can be calculated, as the transition matrix of the states and the signal distributions are known (refer to Table 1 for descriptions of the terms used below). In order to do so, first the stationary distribution, π , is calculated from the transition matrix   1 − PNA PNA Tr = , PAN 1 − PAN according to π Tr = π [12, p.38]. The stationary distribution gives the ratio A:N which the two state Markov Chain tends to over time. From this, probabilities P (N ) and P (A) can be calculated, and used along with the expected values for the Beta distributions to calculate AveT otal. E [B(α, β)] = AveSaf e = P (N ) × AveDanger = P (N ) ×

α α+β

αSN αSA + P (A) × αSN + βSN αSA + βSA αDN αDA + P (A) × αDN + βDN αDA + βDA

AveT otal = AveSaf e + AveDanger . In the experiments that follow, the migration threshold minimum, MMIN is set to 0.5 × AveT otal and the migration threshold maximum, MMAX , is varied at 1, 1.5, 2, 2.5 and 3 × AveT otal. Therefore six versions of the dDCA were used in the experiments below, the min-dDCA and five versions of the standard dDCA with differing values for the threshold maximum. All experiments and generation of synthetic data sets were implemented in Matlab version 7.8.0.347 (R2009a) on a 3.16GHz Intel Core2 Duo machine running CentOS Release 5.2. Due to the stochastic nature of the data sets generated in these experiments, the results from 20 separate initial conditions are reported for each data set.

96

C.J. Musselle

4.3

The Experiments

Significance of the results between the six different versions of the dDCA were tested for with the null hypothesis H0 ; that the results for each version of the dDCA come from the same underlying distribution. E1: Varying the frequency of the anomalous state. In this series of experiments, the transition probabilities PNA and PAN will be varied from 0.1 to 0.5 to create 25 different datasets on which to test all versions of the dDCA. These probabilities dictate the duration and frequency of each state in the data set generated. E2: Varying the time delay between signals and antigens. Though an investigation has already been performed into offsetting the signals and antigens in [6], it is reproduced here with varying threshold maxima for the DC population. This experiment therefore investigates which values of the threshold maximum give the best results when a delay of 1 to 5 time steps is introduced between antigens and signals (antigen occurring first). Similarly the experiment also explores a delay of 1 to 5 time steps between the signals and antigens (signals occurring first).

5

Evaluation and Discussion

In assessing the performance of the DCA, the output values of M CAV for these experiments, are not thresholded in the usual way. Instead a different measure of performance is derived from first averaging the raw values of the M CAV over each antigen class LN and LA . The absolute  Formally difference between these averages is then used as a distance metric, D. this is defined as,     l∈LN M CAVl  l∈LA M CAVl   D= −   , #LN #LA where #LN and #LA are the number of unique antigens in class LN and LA  ∈ respectively. As with the M CAV values themselves, the distance metric D [0, 1].  metric is a summary statistic which best serves the purposes of this The D paper, offering a more fine grained assessment of algorithm performance, as the raw output values are used directly, and not thresholded to a binary classification. It also represents a common way of assessing all experimental results equally, without the need to assign an additional threshold for the algorithm. 5.1

E1: Varying the Frequency of the Anomalous State

Varying the value of PNA between 0.1 and 0.5 showed no significant difference  metrics, for all six versions of the dDCA. This was tested for between the D using the Kruskal-Wallis statistical test (results not shown). However, varying  metric for all versions PAN between 0.1 and 0.5 causes a notable decrease in the D

Insights into the Antigen Sampling Component of the DCA

97

Fig. 2. Change in the distance metric D with transition probability PAN for all six  values are shown at each data point, versions of the dDCA. The spread of all 20 D with the point plotted at the median, and the error bars corresponding to the 25th 75th percentile. Null hypothesis H0 was rejected with p < 0.01 when PAN ≥ 0.2.

of the standard dDCA. The larger the maximum threshold MMAX , the greater  as shown in Figure 2. The min-dDCA however was observed the decrease in D,  as PAN was increased. Only at PAN = 0.1, was the to have a steady value of D null hypothesis H0 not rejected, for all other data sets it was successfully rejected with p < 0.01.  and MMAX is explained by the fact that larger The relationship between D values of MMAX will mean more DCs in the population having longer sampling lifetimes. This equates to more DCs in the population sampling wider time windows of signals, along with the associated antigens. Some of these time windows will fall across the boundaries between the normal and anomalous state in the signal time series, and as antigens of both classes will be collected during this time window, misclassification of some antigens is inevitable. In fact, due to the heavy bias for the safe signal when calculating k (see Section 2.2), sampling across such boundaries is highly likely to tag all antigens collected as normal. The frequency of such misclassification increases as MMAX is increased, lowering the M CAV for the anomalous antigen class and therefore the observed difference  between the classes, D.  with PAN is related to the above. Higher values of PAN will The decrease in D give rise to shorter consecutive anomalous states in the data set, and lead to more transition boundaries between the normal and anomalous state in the time series. This will cause a higher proportion of the anomalous antigens to occur at the boundaries, and as the antigens occurring at these points are prone to misclassification by DCs with longer life times (larger sampling time windows),  decreases with increasing PAN . the metric D  as it dictated the level of consecutive norIncreasing PNA has little impact on D, mal states in the data set. Due to the high bias in calculating k, the normal antigens

98

C.J. Musselle

at the boundaries are almost always tagged correctly, meaning that changing the proportion of antigens occurring at these boundaries has little effect. The min-dDCA performs direct 1-to-1 correlation between signals and antigens throughout the experiment. While this approach was observed to be the best for this problem scenario, it is noted that the temporal correlation required is trivial in nature, and so the capabilities of the dDCA may not have been realised in such a simple task. The experiment does however highlight for the standard dDCA, a relationship between the duration of the anomalous state in the data set, and the average difference between the M CAV values for each  antigen class, as measured with D. 5.2

E2: Time-Offset between Antigen and Signals

In E2, the time-offset between antigens and signals was varied between -5, (antigens appearing before signals) to +5 time steps (antigens appearing after signals). The experiment was repeated with values of PAN at 0.1, 0.3 and 0.5, which are shown in Figures 3a, 3b and 3c respectively. This was done in light of the results of E1, as the duration of the anomalous state was seen to affect the output  considerably. values of D  when the time-offset is The results show a general trend of decreasing D increased in either direction form the origin. The rate of this drop off is increased when PAN is increased, and also appears symmetric about the origin.  with increasing time-offset is expected as the corThe overall decrease in D responding antigen pool becomes increasingly out of phase with the signals set to indicate its state, leading to greater misclassification errors for antigens of both classes. However this effect is amplified when PAN is increased, as when the number of consecutive anomalous states is reduced, there will be less overlap between the out of phase anomalous antigens and the underlying states.  values for the various dDCA versions Significant differences between the D were tested for using the Kruskal-Wallis statistical test. In Figure 3, those annotated with * indicate successful rejection of the null hypothesis H0 with p < 0.05 and ** with p < 0.01. For PAN = 0.1 in Figure 3a these differences were most notable between the min-dDCA and versions with a higher value of MMAX . At time-offsets of 1/1 and 2/-2 the standard dDCA with MMAX in the range of 200-300 showed  when compared to the min-dDCA. For PAN = 0.3 in Figure greater values of D 3b however, no such trend is noted, and in Figure 3c, this trend is reversed,  when compared to the standard with the min-dDCA having greater values of D dDCA with larger values of MMAX . It seems that when the average duration of the anomalous state is sufficiently high, achieved by a lower value of PAN for the data set, sampling over longer time windows is preferable over a direct 1-to-1 correlation, as this has the effect of mitigating errors introduced by the signals and antigens being out of phase by 1-2 time steps. This potentially beneficial effect exhibited by the standard dDCA is however reversed when the number of consecutive anomalous states is reduced (higher values of PAN ).

Insights into the Antigen Sampling Component of the DCA

99

 when varying the time-offset between -5, Fig. 3. Change in the distance metric D (antigens appearing before signals) to +5 time steps (antigens appearing after signals). The experiment was repeated with values of PAN at 0.1, 0.3 and 0.5, which are shown in Figures a), b) and c) respectively for all six versions of the dDCA. The spread of all  values are shown at each data point, with the bar plotted at the median, and the 20 D error bars corresponding to the 25th and 75th percentile. A Kruskal-Wallis statistical  across the various dDCA test was performed to test for a difference in the values of D versions, those annotated with * indicate p < 0.05 and ** p < 0.01.

100

6

C.J. Musselle

Summary and Conclusions

This work introduces a novel model to artificially generate input data for use with the dDCA. Data sets for two experiments were generated using this model, one investigating the effects of altering the frequency and duration of anomalies in the input data, and the second investigating the effect of introducing a delay between signals and antigens (and vice versa). The performances of the six different  versions of the dDCA tested, were defined in terms of a distance metric D detailed in Section 5, measuring the average difference between M CAV values for the two antigen classes. Form the first experiment, it can be concluded that an important factor to consider for dDCA performance, is the likely duration of the anomalous states in the input data. This is because, due to the high bias of the safe signal, anomalous antigens are often misclassified at the transition boundaries between normal and anomalous states. A shorter duration will mean a greater proportion of the anomalous antigen occurring at these boundaries, lowering the output value for the anomalous M CAV ’s, and decreasing the difference between the two antigen classes. This situation may be improved in one of two ways. First the weights used could be changed, so there is no longer such a strong bias for the safe signal. Secondly, segmentation of the input data, originally investigated in [8] to provide a more real-time implementation of the algorithm, could be used to reduce the number of transition boundaries processed at once. Further research would be needed to verify whether this is indeed the case. The second experiment demonstrated that, for a delay of 1-2 time steps between signals and antigen, the standard dDCAs with higher values of MMAX were able to mitigate the errors introduced by the signals and antigen being out of phase. Further work will be needed to investigate why this is the case, though it is speculated that DCs with longer sampling windows make fewer errors overall in this scenario, as long as the duration of the anomalous state is not too short lived. The results of the min-dDCA and standard dDCAs are very comparable for the data sets investigated in this work. However, it is noted that the noise introduced to the signal values by the stochastic nature of the model, was not sufficient to cause misclassification errors with any great frequency. Future work will address this issue by incorporating more noise into the generated data sets so they are closer to real world data. This does suggest however that multi-agent sampling is not always necessary to achieve adequate separation of M CAV values between antigen classes, so long as noise in the signal values is kept to a minimum. One way of achieving this would be through appropriate pre-processing of the signal values. A natural extension of this work, would be to investigate other temporal correlation problems in the same way presented here, by tuning the synthetic model to generate the appropriate input data. In doing so, it may also be appropriate to add extra components to the model, so as to represent more complex data sets. One such extension would be to allow the size of the antigen pool to vary at

Insights into the Antigen Sampling Component of the DCA

101

each time step, which was kept constant in these experiments, to focus primarily on the effect of changing the Mi distribution. Acknowledgements. The author would like to thank Professor Dave Cliff and Dr. Ayalvadi Ganesh for their helpful feedback and advice on the work presented, and the Bristol Centre for Complexity Science (BCCS) for providing funding under the EPSRC grant EP/5011214.

References 1. Greensmith, J.: The Dendritic Cell Algorithm, PhD Thesis, The University of Nottingham (2007) 2. Aickelin, U., Bentley, P., Cayzer, S., et al.: Danger Theory: The Link between AIS and IDS? In: Timmis, J., Bentley, P.J., Hart, E. (eds.) ICARIS 2003. LNCS, vol. 2787, pp. 147–155. Springer, Heidelberg (2003) 3. Manzoor, S., Shafiq, M.Z., Tabish, S.M., et al.: A Sense of Danger for Windows Processes. In: Andrews, P.S., Timmis, J., Owens, N.D.L., Aickelin, U., Hart, E., Hone, A., Tyrrell, A. (eds.) ICARIS 2009. LNCS, vol. 5666, pp. 220–233. Springer, Heidelberg (2009) 4. Oates, R., Kendall, G., Garibaldi, J.: Frequency Analysis for Dendritic Cell Population Tuning: Decimating the Dendritic Cell. Evolutionary Intelligence 1, 145–157 (2008) 5. Stibor, T., Oates, R., Kendall, G., et al.: Geometrical insights into the dendritic cell algorithm. In: Proceedings of the 11th Annual conference on Genetic and Evolutionary Computation, pp. 1275–1282 (2009) 6. Greensmith, J., Aickelin, U.: The Deterministic Dendritic Cell Algorithm. In: Bentley, P.J., Lee, D., Jung, S. (eds.) ICARIS 2008. LNCS, vol. 5132, pp. 291–302. Springer, Heidelberg (2008) 7. Gu, F., Greensmith, J., Aickelin, U.: Further Exploration of the Dendritic Cell Algorithm: Antigen Multiplier and Time Windows. In: Bentley, P.J., Lee, D., Jung, S. (eds.) ICARIS 2008. LNCS, vol. 5132, pp. 142–153. Springer, Heidelberg (2008) 8. Gu, F., Greensmith, J., Aickelin, U.: Integrating real-time analysis with the dendritic cell algorithm through segmentation. In: Proceedings of the 11th Annual conference on Genetic and evolutionary computation, pp. 1203–1210 (2009) 9. Mokhtar, M., Timmis, J., Tyrrell, A.: A modified dendritic cell algorithm for online error detection in robotic systems. In: Proceedings of the Eleventh conference on Congress on Evolutionary Computation, pp. 2055–2062 (2009) 10. Greensmith, J., Aickelin, U., Cayzer, S.: Detecting Danger: The Dendritic Cell Algorithm. In: Robust Intelligent Systems, pp. 89–112 (2009) 11. Oates, R., Kendall, G., Garibaldi, J.: The Limitations of Frequency Analysis for Dendritic Cell Population Modelling. In: Bentley, P.J., Lee, D., Jung, S. (eds.) ICARIS 2008. LNCS, vol. 5132, pp. 328–339. Springer, Heidelberg (2008) 12. Stroock, D.W.: An Introduction to Markov Processes, 1st edn. Springer, Heidelberg (2005)

FDCM: A Fuzzy Dendritic Cell Method Zeineb Chelly and Zied Elouedi LARODEC, Institut Superieur de Gestion de Tunis, 41 Avenue de la liberte, cite Bouchoucha, 2000 Le Bardo, Tunisia [email protected], [email protected]

Abstract. An immune-inspired danger theory model based on dendritic cells (DCs) within the framework of fuzzy set theory is proposed in this paper. Our objective is to smooth the abrupt separation between normality (semi-mature) and abnormality (mature) using fuzzy set theory since we can neither identify a clear boundary between the two contexts nor quantify exactly what is meant by “semi-mature” or “mature”. In this model, the context of each object (DC) is described using linguistic variables. Fuzzy subsets and the corresponding membership functions describe these variables. A knowledge base, comprising rules, is built to support the fuzzy inference. The induction of the context of each object is diagnosed using a compositional rule of fuzzy inference. Experiments on real data sets show that by alleviating the crisp separation between the two contexts, our new approach which focuses on binary classification problems produces more accurate results. Keywords: artificial immune systems, danger theory, dendritic cells, fuzzy set theory.

1

Introduction

The main task of the immune system is to discriminate between self (defined early in life) and nonself (anything that comes later) by a process called “selfnonself discrimination”. This process is performed by several artificial immune system applications, such as the negative selection algorithm (NSA). However, in [12], criticisms of the NSA have been mentioned. In fact, the NSA could not function appropriately since it suffers from issues such as scalability, false positives, the need for an initial learning phase, etc. Hence, in [1], a relatively newer immunological discovery was proposed as a possible alternative to the NSA known as the Danger Theory (DT). The DT overlaps now the way aiming at designing more efficiently a new foundation of artificial immune systems. The most prominent players of the DT are the “dendritic cells” (DCs). An apprehension of DCs behavior led to the development of an inspired immune system algorithm termed the “dendritic cell algorithm” (DCA). E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 102–115, 2010. c Springer-Verlag Berlin Heidelberg 2010 

FDCM: A Fuzzy Dendritic Cell Method

103

The DCA is applied to a wide range of applications. However, it suffers from some limitations [20]. The DCA is also used as a classifier for a static machine learning data set [2], where it was shown that the algorithm can process data classification, but is sensitive to the data order. This is due to an environment characterized by a crisp separation between normality (semi-mature) and abnormality (mature). A possible explanation is the following: if the difference between these two values is small, then the context of the DC will be hard to be defined. Such a small difference could change the decision of the context affectation. Not considering this case, has a negative effect on classification accuracy when the class of data instances changes over time. The objective of this work is to smooth this abrupt separation, since we can neither identify a clear boundary between the two contexts nor quantify exactly what is meant by “semi-mature” or “mature”. This will be handled by the use of fuzzy set theory. The contribution of our work is to improve the classification accuracy as well as to prove that our method does not depend on the class transitions. While other works based on the hybridization of DCA with fuzzy set theory focused on various aspects such as in [2], where a fuzzy migration threshold was defined, our work sheds more light on the context assessment phase of DCA. In this paper, we propose to develop a fuzzy dendritic cell method (FDCM), a new classification technique based on dendritic cells within the framework of fuzzy set theory. In fact, fuzzy set theory has proven its ability in mimicking human decisions [3]. It also permits with linguistic terms, a simple and natural description of problems which have to be settled rather than in terms of relationships between definite numerical values. This paper is structured as follows: in Section 2, we introduce in brief the fuzzy set theory framework; basics of the danger theory will be presented in Section 3, where we also detail the behavior of the dendritic cells. Section 4 describes our fuzzy dendritic cell method in detail; the experimental setup is given in Section 5 and finally, the result analysis is reported in Section 6.

2

Fuzzy Set Theory

Fuzzy set theory was introduced in 1965 by Zadeh [5]. It is a mathematical theory where the fuzziness is the ambiguity that can be found in the definition of a concept or the meaning of a word [6]. Vagueness in expressions like “semimature” or “mature” can be called fuzziness. The major advantage of fuzzy set theory is that it offers the possibility to describe real word problems in a simple and natural way, using linguistic terms. It is more efficient to solve problems in such a way rather than in terms of relationships between definite numerical values. This advantage, dealing with the complicated systems in smooth way, is the major sense why fuzzy set theory is widely applied to various domains. Fuzzy sets were introduced as an extension of the classical notion of a set. In the classical (crisp) set theory, a very precise and clear boundary exists to indicate if an element either belongs or does not belong to the set. In contrast,

104

Z. Chelly and Z. Elouedi

a fuzzy set is a set without a clearly defined boundary. It permits the gradual evaluation of the membership of elements in a set; this is supported by the use of “membership functions”. Fuzzy sets are based on linguistic variables which are defined as means (linguistic expressions) to represent a concept or a variable of a problem in inexact way like “low”, “medium”, “high”, etc. Linguistic variables are those variables whose values are words or sentences in natural or artificial languages [7]. Hence, it is the fact of classifying the variables of a problem in terms of qualitative concepts instead of quantitative ones [6]. The set of values that a linguistic variable can take is called “term set”. Each term set constitutes a fuzzy set in the “universe of discourse” which contains all elements that can come into consideration. A fuzzy set A is defined as follows: A = {x, µA (x)|x ∈ X} .

(1)

where X denotes the universe of discourse and its elements are denoted by x. µA (x) is called the membership function of x in A. The membership function maps each element of X to a membership value between 0 and 1. The membership function µA - by which a fuzzy set A is defined - has the form: µA : X → [0, 1] .

(2)

where [0, 1] is the interval of real numbers from 0 to 1, inclusive. The membership function µA (x) quantifies the grade of membership of the elements x to the fundamental set X. An element mapping to the value 0 means that the member is not included in the given set, 1 describes a fully included member. Values strictly between 0 and 1 characterize the fuzzy members. Membership functions can take different shapes, from the simplest one (triangular functions) to the most complex functions (parameterized by the user). Fuzzy sets are also based on a set of fuzzy rules with C hypotheses (classes) and n attributes. These rules can be written as [8]: Rj : if x1 is Aj1 . . . and xn is Ajn than Class Cj ,

j = 1, . . . , N .

(3)

where x = (x1 ; . . . ; xn ) is an n-dimensional pattern vector (observations); Ai1 is an antecedent linguistic value such as “low” or “medium” (i=1, . . . ,n); Cj is a consequent class and N is the number of fuzzy IF..THEN rules. The antecedent part of each rule is specified by a combination of linguistic values, produced by the partitions of the universe of discourse into a set of linguistic terms [10]. In order to draw conclusions from a rule base, we need a mechanism that can produce an output from a collection of IF-THEN rules. This is done using the “compositional rule of inference”. This process evaluates all the rules and determines their truth values. There are many methods dealing with the inference process such as “Mamdani” method [14], “Max-Prod” [15] and “Sum-Prod” method [16]. Since different fuzzy rules might have different conclusions, all rules should be considered. Thus, fuzzy set theory allows the fact of combining all fuzzy conclusions obtained by the inference process into a single conclusion by a process called “composition” given by (4):

FDCM: A Fuzzy Dendritic Cell Method

µC = max[min(µX1 ), min(µX2 ), . . . , min(µXN )] .

105

(4)

where Xi are the observations and µC is the single conclusion. From the composition process, a fuzzy value is obtained. This value has to be converted into a crisp value. This mechanism is called the “defuzzification”. There are many defuzzification methods such as the maximum method (the maximum truth value) [18] and the centroid method (the center of gravity of the membership function) [17] given by (5): N N   (µ(i) ∗ output(i))/ (µ(i) ) . (5) i=1

i=1

where µ(i) is the truth value of the result membership function for rule i, output(i) is the value (for rule i) where the result membership function is maximum over the output variable fuzzy set range and N is the number of rules.

3

Danger Theory

The “Danger Theory” is proposed by Matzinger [9]. The most prominent players of the DT are the dendritic cells. An inspiration from the DCs behavior lead to the development of an immune system algorithm termed the dendritic cell algorithm (DCA). A brief overview of the biological principles used in the DCA is introduced. 3.1

Introducing DCs

DCs are antigen presenting cells. They are responsible for capturing, processing and displaying antigens to T-cells. Furthermore, DCs express receptors on their surfaces to receive signals from their neighborhood. DCs’ behavior depends on the concentration of the signals received. Thus, they differentiate into three states termed “immature”, “semi-mature” and “mature”, which determine their exact function. 3.2

Immature DCs

The DC immature state is the initial maturation state of a DC. Immature DCs (iDCs) reside in the tissue where they collect signals and antigen which could be a “safe” molecule or something foreign. Differentiation of iDCs depends on the combination of the various signals received leading to a full or partial maturation state. 3.3

Semi-mature DCs

In the presence of cell death (apoptosis), iDCs migrate to a terminate semimature state known as “semi-mature DCs” (smDCs). They also migrate from the tissue to the lymph node. By the receipt of safe signals (SS), smDCs produce a cytokine in response known as “interleukin-10” (IL-10). This cytokine suppresses

106

Z. Chelly and Z. Elouedi

T-cells which match the presented antigen. Hence, causing T-cell tolerance. As a consequence, antigens collected with SS are presented in a tolerant context [11].

3.4

Mature DCs

iDCs migrate to the mature state if they are more exposed to danger signals (DS) and to pathogenic associated molecular patterns (PAMPs) than safe signals. Thus, they are termed the “mature DCs” (mDCs) drifting from the tissue to the lymph node. mDCs produce an inflammatory cytokine termed “interleukin-12” (IL- 12) which stimulates T-cells activation in order to be reactive to antigen presentation. Moreover, mDCs produce costimulatory molecules (CSMs) which are known for facilitating the antigen presenting process [11]. 3.5

DCs under Exposure to Signals

Swapping from one state to another is dependent upon the receipt of different signals throughout the initial state. The signals in question are categorized based on their origin: – PAMPs: are proteins expressed exclusively by bacteria, which can be detected by DCs and result in immune activation. The presence of PAMPs indicates definitely an anomalous situation. – Danger signals: Signals released as a result of necrosis. They are indicators of abnormality, but with lower value of confidence than PAMPs signals. – Safe signals: Signals produced via the process of normal cell death, namely apoptosis. They are indicators of normality, which means that the antigen collected by the DC was found in a normal context. Hence, tolerance is generated to that antigen. – Inflammation: Signals which are evidences that a great number of cells are collected in the tissue area under distress. Inflammation signals have the effect of amplifying the other three categories of input signals, but they have no efficiency when they are present alone in the system. 3.6

The Dendritic Cell Algorithm

The DCA introduced in [2] is capable of joining several signals and antigen to assess the context of each object. Signals pre-categorized as “PAMP”, “danger” and “safe” which reflect the input signals of the system are processed by the algorithm, in order to get three output signals: costimulation signal (Csm), semi-mature signal (Semi) and mature signal (Mat). A migration threshold is incorporated into the DCA in order to determine the lifespan of a DC. As soon as the Csm exceeds the migration threshold; the DC ceases to sample signals and antigens. The DCs differentiation direction is determined by the comparison between cumulative Semi and cumulative Mat. If the cumulative Semi is greater than the cumulative Mat, then the DC goes to semi-mature (context=0, DC “thinks” the antigen is normal), otherwise it goes to mature (contex=1, DC

FDCM: A Fuzzy Dendritic Cell Method

107

“thinks” the antigen is anomalous). At the end, the mature context antigen value (MCAV) - which reflects the probability of an antigen in being anomalous - is calculated. An anomalous threshold is also introduced. Those antigens whose MCAV are greater than the anomalous threshold are classified into the anomalous category, while the others are classified into normal category. program Dendritic Cell Algorithm input : signals from all categories and antigen output: antigen plus context values initialiseDC; while CSM output signal < migration Threshold do get antigen; store antigen; get signals; calculate interim output signals; update cumulative output signals; end cell location update to lymph node; if semi-mature output > mature output then cell context is assigned as 0 ; else cell context is assigned as 1; end print collected antigen plus cell context; for all antigen in total list do increment antigen count for this antigen type; if antigen context equals 1 then increment antigen type mature count; end end for all antigen types do MCAV of antigen type = mature count / antigen count; end

4

The Fuzzy Dendritic Cell Method

The standard DCA does not perform well its classification task in the case of disordered contexts (data randomized between class one and class two). This is due to an environment characterized by the crisp evaluation in the context assessment (normal context (semi-mature) or anomalous (mature)). Hence, it shows serious limitations. The context of each object should then be linguistic and assigned appropriate fuzzy numbers instead of crisp numbers. The fuzzy process consists in the definition of a new model of the standard DCA taking into account the fact of alleviating and mitigating the crisp assessment task. The fuzzy procedure is composed of four main steps. This is shown in Fig.1.

108

Z. Chelly and Z. Elouedi

Fig. 1. Four steps of the fuzzy process

4.1

Fuzzy System Inputs-Output Variables

As stated above, our objective is to smooth the abrupt separation between normality (semi-mature) and abnormality (mature) using fuzzy concepts since there is no clear boundary between the two contexts. We describe each context of each object using linguistic variables. Two inputs (one for each context) and one output are defined. The semi-mature context and the mature context denoted respectively CSemiMature and CMature are considered as the input variables to the fuzzy system. The final state “maturity” of a DC (object), SMaturity , is chosen as the output variable. All the system’s inputs and output are defined using fuzzy set theory. CSemiMature = {µCSemiM ature (cSemiMaturej )/cSemiMaturej ∈ XCSemiM ature } . (6) CMature = {µCM ature (cMaturej )/cMaturej ∈ XCM ature } . (7) SMaturity = {SMaturity (sMaturityj )/sMaturityj ∈ XSM aturity } .

(8)

where cSemiMaturej , cMaturej and sMaturityj are, respectively, the elements of the discrete universe of discourse XCSemiM ature , XCM ature and XSM aturity . µCSemiM ature , µCM ature and µSM aturity are, respectively, the corresponding membership functions. 4.2

Linguistic Variables

As mentioned previously, the basic tools of fuzzy set theory are linguistic variables. The term set T (SMaturity ) interpreting SMaturity which is a linguistic variable that constitutes the final state of maturity of a DC, could be: T (SMaturity ) = {Semi − mature, M ature} .

(9)

FDCM: A Fuzzy Dendritic Cell Method

109

Each term in T (SMaturity ) is characterized by a fuzzy subset in a universe of discourse XSM aturity . Semi-mature might be interpreted as an object (data instance) collected under safe circumstances, reflecting a normal behavior and Mature as an object collected under dangerous circumstances, reflecting an anomalous behavior. Figure 2 gives an illustration of SMaturity as a linguistic variable.

Fig. 2. Fuzzy membership functions for the Maturity output

Similarly, the input variables CSemiMature and CMature are interpreted as linguistic variables with: T (Q) = {Low, M edium, High} .

(10)

where Q = CSemiMature and CMature respectively. 4.3

Fuzzy and Membership Functions Construction

In order to specify the range of each linguistic variable, we have run the DCA and we have recorded both semi-mature and mature values which reflect the (Semi) and (Mat) outputs generated by the DCA. Then, we pick up the minimum and maximum values of each of the two generated values to fix the borders of the range. We assume that the extents and midpoints of the membership functions were determined a priori by the user. The range of the output variable is determined as follows: min(range(SM aturity )) = min(min(rangeCM ature), min(rangeCSemiM ature)) . (11) max(range(SM aturity)) = max(max(rangeCM ature), max(rangeCSemiM ature)) . (12)

A knowledge base, comprising rules, is built to support the fuzzy inference. The different rules of the fuzzy system are extracted from the information reflecting the effect of each input signal on the state of a dendritic cell which is the following:

110

Z. Chelly and Z. Elouedi

– Safe signals: in increase in value is a probable indicator of normality. High values of the safe signal can cancel out the effects of both PAMPs and DS. – Danger signals: in increase in value is a probable indicator of damage, but there is less certainty than with a PAMP signal. – PAMPs: in increase in value is a definite indicator of anomaly. – Inflammation: has the effect of amplifying the other three categories of input signals, but is not sufficient to cause any effect on DCs when used in isolation. From this information, we can generate the following set of rules where all the mentioned signals are taken into account implicitly in the fuzzy system. Rule(1): If (CMature is Low) and (CSemiMature is Low) then (SMaturity is Mature) Rule(2): If (CMature is Low) and (CSemiMature is Medium) then (SMaturity is Semi-mature) Rule(3): If (CMature is Low) and (CSemiMature is High) then (SMaturity is Semi-mature) Rule(4): If (CMature is Medium) and (CSemiMature is Low) then (SMaturity is Mature) Rule(5): If (CMature is Medium) and (CSemiMature is Medium) then (SMaturity is Semi-mature) Rule(6): If (CMature is Medium) and (CSemiMature is High) then (SMaturity is Semi-mature) Rule(7): If (CMature is High) and (CSemiMature is Low) then (SMaturity is Mature) Rule(8): If (CMature is High) and (CSemiMature is Medium) then (SMaturity is Mature) Rule(9): If (CMature is High) and (CSemiMature is High) then (SMaturity is Mature) Let us consider Rule (2) as an example: if the CMature input is set to its first membership function “Low” and the second input CSemiMature to its second membership function “Medium”, then the “Semi-mature” context of the output SMaturity is assigned. This could be explained by the effect of the safe signals (which lead to the semi-mature context) on the judgement of the state of the output, since the high values of SS can cancel out the effects of both PAMPs and DS (which lead to the mature context). The same reasoning is affected to the rest of the rules. 4.4

The Fuzzy Context Assessment

As mentioned before, our FDCM is based on the “Max-Min” composition method and the “centroid” defuzzifacation mechanism. Once the inputs are fuzzified and the output (centroid value) is generated, the cell context has to be fixed. In fact, the output value is compared to the middle of the SMaturity range. This could be explained as follows:

FDCM: A Fuzzy Dendritic Cell Method

111

If the centroid value generated is greater than the middle of the output range then the surface of the SMaturity can be described as skewed to the right (positive asymmetry). In other words, the area of the “Mature” membership function is greater than the “Semi-mature” one. Therefore, the final context of the object is “Mature”. Thus, it indicates that the collected antigen may be anomalous; else the antigen collected is likely to be normal. Example 1. Figure 3 shows an example of the application phase of three rules of two input values [0.3 0.6], so the generation of the centroid value results in (0.6). As we remark, the middle of the output range (0.5) is lower than the centroid value (0.5 < 0.6). As shown in the figure, the surface of the “Maturity” output is skewed to the right. Hence, the final context of the object is “Mature”.

Fig. 3. An illustrative schema of the context assessment

5

Experimental Setup

In this Section, we try to show the effectiveness of our FCDM as well as its performance. The aim of our method is to improve the classification accuracy in the case of contexts’ change. We have developed our program in Matlab V7.1 for the evaluation of our FDCM. Different experiments are performed using six two-class data sets from [4]. These data sets are described in Table 1. Data items with the largest standard deviation form the DS. To generate concentrations for SS and PAMPs, the attribute with the next greatest standard deviation is chosen. In spite of incorporating inflammation signals into the model, they are not used in our experiments, as no obvious mapping is available. Antigen is represented in its simplest form as the identification number of a data item within the database. All featured parameters are derived from empirical immunological data.

112

Z. Chelly and Z. Elouedi Table 1. Description of databases Database

Ref

Mammographic Mass MM Pima Indians Diabetes PID Blood Transfusion Service Center BTSC Wisconsin Breast Cancer WBC Haberman’s Survival HS SPECTF Heart SPECTF

 instances  attributes 961 768 748 700 306 267

6 8 5 9 4 44

The order of the data items varies according to experiments. Experiment 1 uses all class 1 items followed by all class 2 items. The rest of the experiments uses data from class 1 and class 2 that is randomized once, then 20 times, then 60, 70, 90, 120, 140, 160, 200, and finally 300 times successively. Each experiment is performed 20 times. The fact of increasing the number of randomization (R) from one experiment to another lead to a database randomized more between the two classes, so to successive transitions (class of data instances changes multiple times). Our aim is to prove that our FDCM does not depend of such transitions unlike the DCA. Each run samples each data item 10 times, giving 9610, 7680, 7480, 7000, 3060 and 2670 antigen presentations per run for all the mentioned data sets. The threshold for classification is set to 0.3, 0.9, 0.6, 0.65, 0.2647 and 0.6. Items exceeding the threshold are classed as class 2, with lower valued antigen labeled as class 1. These classifications are compared with the labels presented in the original data sets so accuracy can be measured. Table 2 presents a comparison between the DCA and our FDCM in terms of Percent of Correct Classification (PCC). The PCC is determined by measuring the number of instances correctly classified over the total number of instances in the data set. P CC =

6

number of well classified instances ∗ 100 . total number of classified instances

(13)

Results

Previous examinations with DCA, in [2], show that the misclassifications occur exclusively at the transition boundaries. Hence, DCA makes more mistakes when the context changes multiple times in a quick succession. However, this problem is solved with the new fuzzy context assessment phase of our FDCM, which is confirmed by the results presented in Table 2. From Table 2, we can conclude that our fuzzy dendritic cell method has given good results. In fact, by randomizing the values of R, the PCC of our FDCM is better than the one given by DCA. Hence, such randomization of the values of R generally affects badly the classification accuracy of the DCA, which is not the case with our FDCM.

FDCM: A Fuzzy Dendritic Cell Method

113

Table 2. Experimental measures PCC (%) (E1=Experiment 1, R=Randomized) Database Method DCA MM FDCM DCA PID FDCM DCA BTSC FDCM DCA WBC FDCM DCA HS FDCM DCA SPECTF FDCM

E1 98,43 62,85 96,35 98,05 91,71 48,66 99,42 98 83 29 93,63 79,4

R=1 62,85 97,19 98,82 96,87 93,31 48,52 91 97,57 17,32 82,35 91,76 79,4

R=20 62,85 95,63 96,48 97 91,44 98,53 89,71 98,57 17,32 92,15 91,01 92,5

R=60 62,85 96,67 94,14 96,48 93,98 96,8 91,14 99,57 17,64 90,85 91,76 94,38

R=70 62,85 94,48 98,56 97,79 90,1 95,45 90,42 96,57 16,99 82,02 91,38 95,13

R=90 62,85 97,5 95,31 95,83 91,57 96,25 90,71 97,57 17,32 93,8 92,13 94,75

R=120 62,85 96,15 96,35 96,74 93,18 96,25 89,71 99,28 16,99 92,48 91,38 93,63

R=140 62,85 95,42 96,61 98,31 92,24 98,26 90,71 97,57 16,99 92,15 90,63 94,75

R=160 62,85 97,09 96,35 96,22 91,71 98,4 91 97,14 17,64 89,54 90,63 94,75

R=200 62,85 96,77 96,22 95,57 91,57 94,92 90,85 97,14 16,66 92,15 89,13 94,75

R=300 62,85 96,36 96,09 97 91,44 97,33 90,57 99 16,66 91,83 88,01 94,38

Fig. 4. Comparing FDCM PCC and DCA’s one on HS database

For instance, by applying the DCA to the Habermans Survival database and with the variation of the different values of R, the PCC varies from 16,66% to 17,64%. Whereas, with our FDCM, the PCC varies from 82,02% to 93,8%. This is explained by the appropriate use of the fuzzy set theory in the case of a randomized context. Note that in the case of an ordered database (E1), the PCC of the DCA is generally better than the PCC of our FDCM (except for the PID database) since fuzzy set theory is more appropriate to handle cases of randomization than ordered data sets. Figure 4 shows a comparison between our FDCM and the DCA in terms of PCC in the HS database. It shows that when the context changes multiple times in a quick succession (Experiments 2 where R=1, 3 where R=20, . . . , 11 where R=300), it is more appropriate to apply FDCM than the standard DCA since the former produces more accurate results. However, in the case of an ordered data set (Experiment 1), the PCC of our FDCM decreases regarding to the PCC of DCA.

114

Z. Chelly and Z. Elouedi

Furthermore, regarding to the computational complexity, our FDCM achieves the same computational complexity as the DCA even by adding the fuzzy technique and getting the min-max boundaries generated by DCA which is another important characteristic of our method. To summarize, with the variation of the parameter R, our fuzzy dendritic cell method gives better results than the standard DCA in terms of classification accuracy without increasing complexity.

7

Conclusion and Future Works

In this paper, we have developed a fuzzy dendritic cell method (FDCM). Our method is based on both the dendritic cell algorithm and the fuzzy set theory and it aims at coping with the crisp separation between two contexts. Using the fuzzy set theory allows smoothing such a case as well as replacing and describing each DC context with linguistic variables. Our experimentations show significant improvements since there is an important reduction in false alarms in case of contexts’ change. As future work, we intend to further explore this new instantiation of the DCA. This investigation will involve an automatical generation of the extents of the different membership functions.

References 1. Aickelin, U., Bentley, P., Cayzer, S., Kim, J., McLeod, J.: Danger Theory: The Link between AIS and IDS. In: Timmis, J., Bentley, P.J., Hart, E. (eds.) ICARIS 2003. LNCS, vol. 2787, pp. 147–155. Springer, Heidelberg (2003) 2. Greensmith, J., Aickelin, U., Cayzer, S.: Introducing Dendritic Cells as a Novel Immune-Inspired Algorithm for Anomaly Detection. In: Jacob, C., Pilat, M.L., Bentley, P.J., Timmis, J.I. (eds.) ICARIS 2005. LNCS, vol. 3627, pp. 153–167. Springer, Heidelberg (2005) 3. Ross, T.: Fuzzy Logic for Engineering Applications. McGraw-Hill Book Company, New York (1995) 4. UCI machine learning repository, http://archive.ics.uci.edu 5. Zadeh, L.: Fuzzy Sets. Information and Control 8, 338–353 (1965) 6. Zimmermann, J.: Fuzzy Set Theory and Its Applications. European Journal of Operational Research 1, 227–228 (1996) 7. Pedrycz, W., Zadeh, L.: Fuzzy Sets Engineering 1. IEEE Micro, 19–30 (1995) 8. Ishibuchi, H., Nakashima, T.: Effect of rule weights in fuzzy rule-based classification systems. IEEE Transactions on Fuzzy Systems 9(4), 506–515 (2001) 9. Matzinger, P.: The Danger Model in its historical context. Scandinavian Journal of Immunology, 4–9 (2001) 10. Roberto, A., Myriam, D., Heitor, S., Alex, A.: An artificial immune system for fuzzy-rule induction in data mining. In: 8th Brazilian Symposium on Naturel Network, pp. 1011–1020 (2004) 11. Greensmith, J.: The Dendritic Cell Algorithm. PhD Thesis, University of Nottingham (2007)

FDCM: A Fuzzy Dendritic Cell Method

115

12. Stibor, T.: On the appropriateness of negative selection for anomaly detection and network intrusion detection. PhD Thesis, Darmstadt University of Technology (2006) 13. Nauman, M., Muddassar, F.: A Sense of Danger: Dendritic Cells Inspired Artificial Immune System for MANET Security. In: Genetic and Evolutionary Computation Conference, pp. 102–109 (2007) 14. Mamdani, E.H., Assilian, S.: An experiment in linguistic synthesis with a fuzzy logic controller. International Journal of Man-Machine Studies, 1–13 (1975) 15. Kyosev, Y., Peeva, K., Reinbach, I.: Max-Product Fuzzy Relational Equations as Inference Engine for Prediction of Textile Yarn Properties. In: 9th Fuzzy Days, pp. 18–20 (2006) 16. Mizumoto, M.: Fuzzy controls by product-sum gravity-method. Fuzzy Sets and Systems, c1.1–c1.4 (1990) 17. Van Broekhoven, E., De Baets, B.: Fast and accurate center of gravity defuzzification of fuzzy system outputs defined on trapezoidal fuzzy partitions. Fuzzy sets and systems, 904–918 (2006) 18. Lee, C.: Fuzzy logic in control systems: Fuzzy logic controller - Parts 1 and 2. IEEE Transactions on Systems, Man and Cybernetics, 404–435 (1990) 19. Gu, F., Greensmith, J., Aickelin, U.: Further Exploration of the Dendritic Cell Algorithm: Antigen Multiplier and Time Windows. In: Bentley, P.J., Lee, D., Jung, S. (eds.) ICARIS 2008. LNCS, vol. 5132, pp. 142–153. Springer, Heidelberg (2008) 20. Aickelin, U., Greensmith, J.: The Deterministic Dendritic Cell Algorithm. In: Bentley, P.J., Lee, D., Jung, S. (eds.) ICARIS 2008. LNCS, vol. 5132, pp. 291–302. Springer, Heidelberg (2008)

Modular RADAR: An Immune System Inspired Search and Response Strategy for Distributed Systems Soumya Banerjee and Melanie Moses Department of Computer Science, University of New Mexico, USA {soumya,melaniem}@cs.unm.edu Abstract. The Natural Immune System (NIS) is a distributed system that solves challenging search and response problems while operating under constraints imposed by physical space and resource availability. Remarkably, NIS search and response times do not scale appreciably with the physical size of the animal in which its search is conducted. Many distributed systems are engineered to solve analogous problems, and the NIS demonstrates how such engineered systems can achieve desirable scalability. We hypothesize that the architecture of the NIS, composed of a hierarchical decentralized detection network of lymph nodes (LN) facilitates efficient search and response. A sub-modular architecture in which LN numbers and size both scale with organism size is shown to efficiently balance tradeoffs between local antigen detection and global antibody production, leading to nearly scale-invariant detection and response. We characterize the tradeoffs as balancing local and global communication and show that similar tradeoffs exist in distributed systems like LN inspired artificial immune system (AIS) applications and peer-to-peer (P2P) systems. Taking inspiration from the architecture of the NIS, we propose a modular RADAR (Robust Adaptive Decentralized search with Automated Response) strategy for distributed systems. We demonstrate how two existing distributed systems (a LN inspired multi-robot control application and a P2P system) can be improved by a modular RADAR strategy. Such a sub-modular architecture is shown to balance the tradeoffs between local communication (within artificial LNs and P2P clusters) and global communication (between artificial LNs and P2P clusters), leading to efficient search and response. Keywords: scale invariant detection and response, distributed systems, scale invariant response, scale invariant detection, immune system scaling, modular search, modular architecture, sub-modular architecture, peer-to-peer systems, artificial immune systems, immune system modelling, intrusion detection systems, malware detection systems, mobile ad-hoc networks, disruption tolerant networks, wireless sensor networks, multi robot control.

1

Introduction

Distributed systems are becoming increasingly important, for example in environmental monitoring, disaster relief, military operations, multi-robot control, E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 116–129, 2010. c Springer-Verlag Berlin Heidelberg 2010 

Modular RADAR

117

wireless sensor networks and peer-to-peer systems [1]. Such systems typically are distributed in physical space with limited resources like power and bandwidth, and performance scalability is desirable. The natural immune system (NIS) demonstrates that nearly scale invariant performance is possible, even under constraints imposed by physical space and resource availability [2, 3]. The NIS searches trillions of host cells to find small amounts of spatially localized antigen. The search for antigen and response (neutralizing the pathogen) is nearly invariant with respect to the size of the host [2, 3]. Search in the NIS occurs in a hierarchical modular network of lymph nodes (LN) distributed throughout the body. LNs are the primary locations in which populations of adaptive and innate immune cells interact to recognize and respond to pathogens. We hypothesize that this NIS architecture promotes modular RADAR (Robust Adaptive Decentralized search with Automated Response). Previous work [2, 3] shows how the architecture of the lymphatic network enables the NIS to detect antigen and respond by producing antibodies in time that is nearly invariant with animal size. Our analysis of data on experimental infection by West Nile Virus (WNV) of a range of animals (from 30g sparrows to 300 kg horses) indicates that the NIS neutralizes WNV in approximately 3 days regardless of body size [2, 3]. This is surprising in the light of two facts: First, the NIS of a horse has to search for a needle (rare antigen) in a much larger haystack (body) than the NIS of a sparrow. Second, experimental data indicates that a critical concentration of WNV-specific antibody needs to be present in the bloodstream to neutralize WNV. Since the volume of blood in an organism is proportional to body mass [4], a horse will have to produce 10000 times more antibody, in the same time, as a sparrow. Hence the search and response problem is more challenging in larger organisms. We show that a semi-modular NIS architecture is capable of nearly scale-invariant detection and response. We then use this as inspiration for search and response strategies in other distributed systems. We show how the modular RADAR architecture improves search and response in a LN inspired multi-robot control application and a peer-to-peer content search system. The remainder of the paper is organized as follows: we review relevant features of the NIS, define attributes of the modular RADAR process, explain our experimental data source, and then explain how a sub-modular architecture makes scale-invariant search and response possible. We then demonstrate how modular RADAR architectures improve search in two distributed systems, and end with concluding remarks.

2

A Review of the Relevant Immunology

LNs reduce the physical space in which lymphocytes search for the particular pathogens that they can recognize and neutralize, and hence they vastly speed up the distributed search and response to novel pathogens. Without concentrating the search into a small volume of tissue, the distributed search for novel pathogens by rare pathogen specific lymphocytes would be prohibitively slow.

118

S. Banerjee and M. Moses

We hypothesize that evolutionary pressures have shaped the number and size of LN to minimize the time to unite rare antigen-specific lymphocytes (B-cells and T-cells) with pathogens. LNs provide a small volume of tissue in which lymphocytes, antigen-presenting cells and antigen encounter each other. The area of tissue that drains into a LN is called its draining region (DR). Dendritic cells (DCs) sample the tissue in DRs for pathogens, and upon encountering them, migrate to the nearest LN to present antigen to T helper cells. Upon recognizing cognate antigen on DCs, T-cells proliferate and build up a clonal population in a process called clonal expansion. While proliferating, T helper cells also migrate to the LN B-cell area to activate B-cells. B-cells are activated by antigen-bearing DC and T helper cells. After activation, B-cells undergo clonal expansion and differentiate into antibody-secreting plasma cells. Cognate T-cells and B-cells specific to a particular pathogen are very rare (1 in 106 cells will bind to a particular antigen [5]), thus the draining LN also recruits B-cells from other LNs. The rate at which B-cells flow into a particular LN is determined by the cross sectional area of the High Endothelial Venule (HEV) which is proportional to LN volume [5]. Rapid detection requires that pathogens are quickly picked up by DC and carried to the LN for presentation to T- and B-cells. Smaller DR reduce the time for DC to migrate to the LN. Rapid response (in this case we consider antibody production as the response), requires that sufficient numbers of B-cells are activated, ultimately to produce sufficient antibody to neutralize the pathogen. Since both the number of B-cells resident in a LN and the rate at which B-cells are recruited to a LN is proportional to LN volume, larger LN reduce the time to activate a B-cell response.

3

Modular RADAR

LNs form a hierarchical distributed detection network that is modular (by dividing the search space into semi-independent modules or LNs), efficient (since search occurs in the small physical space of a LN as opposed to throughout the body) and parallel (since there are many such small searches being conducted in parallel). The size of a module (LN) is set by tradeoffs between local and global communication. We mathematically determine the extent of modularity required to conduct efficient and nearly scale-invariant search and response. We call this a modular RADAR (Robust Adaptive Decentralized search with Automated Response) strategy. Search and response in the NIS is characterized by the following attributes: 1. Robust: There is redundancy and diversity among components, but the exact role of a component is flexible. Large numbers of individuals respond probabilistically to information signals. While sometimes this can lead to responses that grow out of control (i.e. cytokine storms in response to influenza infection [6]), generally the wisdom of the crowd [7] is effective even when individuals make mistakes.

Modular RADAR

119

2. Adaptive: Adaptation occurs by individuals or populations of individuals changing in response to environmental signals. When a B-cell binds to a pathogen and is activated by other immune system cells, that B-cell produces a large and variable population of daughter B-cells. Those that bind to the pathogen most effectively reproduce faster, so the population of cells gets better at neutralizing the pathogen. 3. Decentralized search: Control is completely decentralized and communication between NIS cells is aggregated spatially, e.g. LNs concentrate interactions and chemical signalling between NIS cells. 4. Automated Response: The response is as distributed as search. Individuals act (e.g., kill an infected cell or move along a chemical gradient) by integrating local signals from their environment. Much of the Artificial Immune System (AIS) literature focuses on abstracting algorithms to confer adaptation, learning, memory and robustness from NIS processes (e.g. negative selection, clonal selection algorithms, etc [8]). We do not suggest new algorithms; rather, we propose that the physical division of NIS search into LN and DR can inspire architectures that other distributed algorithms can exploit to balance local and global communication.

4

Data Characterizing Immune Response Times

In previous work [2, 3], we analyzed data from a study of WNV infection. Animals ranging from 0.03 kg sparrows to 300 kg horses, were experimentally infected with WNV and the viral concentration in blood was measured on a daily basis. A critical amount of WNV-specific antibody is required for neutralizing WNV [9]. Previous studies have shown that the time to attain peak viral concentration (tpv ) coincides with the time taken to secrete a critical amount of WNV-specific antibody [9]. For the experimental infection studies described above, tpv ranged from 2 to 4 days post infection, and was uncorrelated with animal body size (tpv vs. M , p-value = 0.35, where M is organism mass) [2, 3]. Since tpv is the time taken to detect pathogens and secrete a critical amount of antibody, the absence of a relationship of tpv with mass indicates that the NIS has nearly scale-invariant detection and response.

5 5.1

Sub-modular Architecture Balances Tradeoff between Local and Global Communication Data on Lymph Nodes

Published empirical data suggest that the mammalian NIS has a sub-modular architecture. LNs increase in both size and in numbers as animal size increases, e.g. 20g mice have 24 LN averaging 0.004g each, and humans are 3000 times bigger and have 20 times more LN, each 200 times bigger [10, 11]. Data from elephants (with LN approaching the size of an entire mouse) and horses (with

120

S. Banerjee and M. Moses

8000 LN) also support the hypotheses that LN size and number both increase with body size [11, 12]; however data for more species are required in order to determine scaling exponents that quantitatively describe how LN size and number change with body size. We hypothesize that total LN volume (the number of LN multiplied by average LN volume) is linearly proportional to animal mass because most organ and fluid volumes (i.e heart, blood and liver volume) are linearly proportional to mass [4]. The data are consistent with this hypothesis. This suggests a tradeoff between LN size and number. Since DR size equals animal volume divided by LN number, there is also a tradeoff between DR size and LN size. 5.2

Mathematical Analysis

In previous work [2, 3] we hypothesized that the NIS is sub-modular because it is selected not just to minimize time to detect pathogens, but also to minimize the time to produce a sufficient concentration of antibody in the blood (Abcrit ). We hypothesize that the NIS has evolved to minimize two quantities: the time to detect antigen and the time to recruit B-cells from other LNs and produce absolute quantities of antibody (Ab), where Ab is proportional to M (organism mass). The smallest LN (e.g. in mice or sparrows) contain on the order of a single B-cell that recognizes any particular pathogen [5]. If LNs in all organisms were of a fixed size (and therefore contained a fixed number of B-cells), each LN would serve a fixed-size DR of tissue, and a parallel search for antigen in a fixed space would be repeated more times in a larger organism. This would achieve scale-invariant search. However, the NIS of larger organisms has to activate a number of B-cells (Bcrit ) ∝ M , in order to build up the critical density of antibodies in a fixed period of time [9]. Activating Bcrit (which is proportional to M ) to fight an infection like WNV that is initially localized in a single DR, requires recruiting on the order of M B-cells from distant LN. We consider this activation of B-cells from remote LN as global communication. The number of LNs that a single infected site LN has to communicate with (Ncomm ) in order to recruit more B-cells is proportional to the critical number of B-cells required to neutralize the pathogen (Bcrit ) divided by the number of antigen-specific B-cells resident in a LN (N umBcell ): Ncomm ∝ Bcrit /N umBcell . Noting that Bcrit ∝ M and N umBcell ∝ VLN (volume of a LN) we have Ncomm ∝ M/VLN . The rate at which new B-cells from other LN enter into infected LN through the high endothelial vessels (ratecomm ) is proportional to the volume of the LN, ratecomm ∝ VLN [5]. The time spent in communicating with other LNs and recruiting and activating other B-cells (tcomm ) is then given by tcomm = 2 Ncomm /ratecomm and tcomm ∝ M/VLN . Hence if LNs in all organisms are of the same size (if VLN ∝ M 0 ), and the number of LNs scaled linearly with organism size, there are increasing costs to communicating with other LN as the organism gets bigger (tcomm ∝ M ); the NIS would conduct efficient search but not efficient antibody production.

Modular RADAR

121

An alternative is to have a fixed number (N ) of LN whose volumes increase linearly with M : VLN ∝ M 1 and N ∝ M 0 . This increases the rate of influx of Bcells (ratecomm ∝ VLN ∝ M ) and also situates more NIS cells inside the infected site LN. Since all the necessary NIS cells which need to be activated are within the LN, this architecture has no communication cost. However, if the number of LN is invariant, each LN in a larger organism services a larger volume of tissue. Such an architecture would lead to DC migration times that are prohibitively 1/3 long for large animals (tDC ) since antigen-loaded DCs would have migrate ∝ M to migrate longer distances to reach the draining LN [3]. The architecture that strikes a balance between the two opposing goals of antigen detection (local communication) and antibody production (global communication) is found by minimizing tcomm and tmigrate which gives VLN ∝ M 3/7 and N ∝ M 4/7 where VLN is the size (volume) of a LN and N is the number of LNs in an organism of mass M [3]. This is a sub-modular architecture in which LN size and numbers both scale sub-linearly with organism size (Fig. 1). These equations indicate that time for DC to carry antigen to LN scales as M 1/7 (since tmigrate is proportional to the radius of the DR, whose volume is M/N or M 3/7 ). Similarly, the time to recruit cognate B-cells to the infected site is N divided by the rate of B-cell migration into the LN (which is proportional to VLN ), giving, again, M 1/7 . In previous work [3] we use an agent based model parameterized with empirical values to show that this scaling results in a total time for the entire search and response process that differs by less than 1 day between horses and sparrows (which is in agreement with empirical data). In summary, due to the requirement of activating increasing numbers of NIS cells for antibody production in larger organisms, there are increasing costs to global communication as organisms grow bigger. The sub-modular architecture balances the opposing goals of detecting antigen using local communication and producing antibody using global communication. Such a modular RADAR search strategy leads to optimal antigen detection and antibody production time, and makes the NIS robust and adaptive.

6

Applications to Artificial Immune Systems and Other Distributed Systems

Spatially constrained networks are being increasingly used in environmental monitoring, disaster relief and military operations [1]. These networks operate under constraints similar to an NIS and hence their design can be informed by architectural strategies employed by their biological counterpart. We incorporate modular RADAR strategies for rapid search and response to two distributed systems. 6.1

Application to Mobile Devices

Original System. Our work has implications for the architecture of AIS algorithms for multi-robot control [13–15]. The robots and their communication are constrained by physical space, and scaling of performance with system size is an important design criterion. These systems use computer servers (analogous

122

S. Banerjee and M. Moses

Fig. 1. A sub-modular detection network of lymph nodes. The shaded regions are LNs and the unshaded regions are the DR. The hypothetical organism on the right is four times as big as the one on the left. The number of LNs and their size both increase with the size of the organism. The local communication (antigen loaded DC migration to LN) is shown by a dotted arrow and the global communication (recruitment of NIS cells from other LNs) is shown by solid arrows. The size of the incoming arrow into a LN represents the size of the HEV (which is proportional to the size of a LN).

to LNs) to coordinate information exchange between mobile devices (analogous to dendritic cells) in a physical area (analogous to the DR of a LN). When the robots encounter obstacles (analogous to antigen), they communicate with software agents (B-cells) residing in the local computer server (LN). The software agents transmit actions to robots to help overcome their obstacles, and agents also share information globally by migrating to other computer servers (analogous to B-cells migrating between LN). Hence computer servers share efficient solutions locally with robots in their DR and globally with other computer servers. The system is diagrammed in Fig. 2 (modified from [13]). Modifying the Original System Using the Modular RADAR Architecture. By analogy with the NIS, a modular architecture can be used to minimize the time taken by a robot to transmit information about an obstacle (local detection), the time taken by a computer server to transmit back an initial rule-set of actions (local response) and the time taken by a computer server to communicate good rule-sets to other agents (global response). There are two potential communication bottlenecks: communication between robots and computer servers, and communication between computer servers. A bottleneck in (local) communication between robot and server demands many small DRs. A bottleneck in (global) server communication requires a few large servers. If both local and global communication are constrained, the architecture which balances these opposing requirements is sub-modular, i.e. the number of servers increases sublinearly with system size and the capacity of

Modular RADAR

123

Robot

Robot Agent

Agent

Agent Agent

Robot

Robot Robot

Robot

Agent

Agent

Agent

Agent

Robot

Robot

Fig. 2. (A) Left Panel: a scaled down version of the multi-robot AIS system. The shaded regions are artificial LNs (computer servers) and the unshaded regions are the artificial DR. Light arrows denote communication between robots and servers (local communication) and bold arrows denote communication between servers (global communication). (B) Right Panel: a scaled up multi-robot AIS system with sub-modular architecture. Note that the number of artificial LNs and their size (the number of robots they service and the number of software agents they have in memory) both increase with the size of the system.

each server (bandwidth, memory and number of robots serviced by each server) increases sublinearly with system size (shown in Fig. 2). The local communication time within an artificial DR is a function of the number of robots (d) serviced by the artificial LN tlocal = f (d)

(1)

The function f will depend on constraints on communication between robots and servers, influenced, for example, by how robot requests are queued on the server and the distance over which low power robots can send and receive messages. The global communication time between artificial LNs is also a function of the number of LNs in the system (n/d) where n is the total number of robots in the entire system tglobal = g(n/d)

(2)

The function g depends on communication constraints between servers. For low latency and high bandwidth connections among servers, tglobal may not scale appreciably. However, low power servers distributed in remote environments, may preclude broadcast communication such that tglobal increases with n/d. An increase in the size of an artificial LN (and hence the number of robots serviced, d) would reduce tglobal at the cost of tlocal . The size and number of artificial LNs to balance local and global communication depends on the precise functions f and g mediating local and global communication. We turn to another application in peer-to-peer systems where local and global communication are specified.

124

S. Banerjee and M. Moses

In summary, the incorporation of a modular RADAR strategy will lead to faster search and response times (T = tlocal + tglobal ) in such LN inspired AIS applications. It would also improve robustness since there are more components (robots in this case) in a single DR as the system grows larger, and hence failure can be compensated by the presence of other redundant components. 6.2

Peer-to-Peer Systems

Peer-to-peer (P2P) systems are emerging as a significant vehicle for providing distributed services like search, content integration and administration. In such systems, computer nodes store data or a particular service and no single node has complete global information about the whole system. Hence these decentralized systems present fundamental challenges during location of resources (data, services, etc) distributed over a multitude of locations. Original System. Search for resources (data, services, etc) distributed over multiple nodes in a P2P network, is similar to the search for pathogens by DCs within a LN and the search for cognate B-cells between LN. Here we focus on a specific P2P overlay network called Semantic Small World (SSW) that supports efficient context-semantic based search [16]. SSW represents objects by a collection of attribute values derived from object content or metadata. The SSW P2P overlay network aggregates data objects with similar semantics close to each other in clusters in order to facilitate efficient search. Real-world applications require a large number of attributes to identify data objects, so search is through a high-dimensional search space. SSW follows Kleinberg [17] to understand how to conduct efficient decentralized search when each node only has information about its neighbours, and no nodes have global knowledge about the location of resources. For such search to efficiently scale up to large numbers of nodes, each node is required to maintain some long-distance connections drawn from a particular probability distribution. Each node maintains l long-distance links each of which are drawn with probability proportional to 1/d (d is the distance between two nodes) in addition to s short-distance links. With such a small-world distribution of links, the network structure itself provides latent structural cues, such that each node with only local information can guide a message to a distant target [17]. Search across clusters (global search) proceeds via the short and long distance links by comparing coordinates of the destination and subspaces of the traversed nodes. Search within a cluster (local search) is by neighbour flooding whereby each node sends out a search request to all its nearest neighbours. The system is diagrammed in Fig 3. The average time for search across clusters (global search) 2 is given by tglobal = O( log (n/c) ) where n is the number of nodes, c is the size l of a cluster (number of nodes in a cluster) and l is the number of long distance connections per node [16]. Modifying the Original System Using the Modular RADAR Architecture. SSW keeps the size of a cluster (c) and the number of long-distance links per node (c) constant, even as the number of nodes, n, increases. Our

Modular RADAR

125

contribution is to modify the system by varying the cluster size and the number of long-distance links per node, analogous to the way that the NIS varies LN size and communication between LN, respectively. We show that there exists a tradeoff between local search time (within a cluster) and global search time (across clusters), and the modular RADAR architecture efficiently balances them.

Fig. 3. (A) Left Panel: a P2P content search system showing clusters (big circles) and nodes within them (small ovals). Short-distance links are shown by light arrows and a long-distance connection is shown in bold arrows (shown only for a single node). (B) Right Panel: a scaled up version of the P2P content search system with a sub-modular architecture. Both the number of clusters and their size (number of nodes within them) now increase with system size (total number of nodes). Note that there are also more long-distance connections per node (shown only for a single node).

First we vary the number of long-distance connections per node as l = O(log(n/c)). This sets the number of long-distance links per node to the logarithm of the number of clusters (n/c). This is an example of densification [18] in which the number of connections increases slowly with system size in order to minimize the time to send a message to a distant node and also increases robustness. Densification is found empirically in technological networks [18]. The resulting average time for search across clusters is now given by tglobal = O(log(n/c)). The worst case time taken to propagate a message within a cluster (local search) using neighbour flooding is given by tlocal = O(c1/2 ). This reflects the longest message propagation distance (the diagonal) in a lattice of c elements. Thus, the total time to search within and across clusters is given by T = tlocal + tglobal = α1 c1/2 + α2 log(n/c)

(3)

where α1 and α2 are constants. Simplifying we have T = α1 c1/2 −α2 logc+α2 logn. Since we desire a relation for c (cluster size) in terms of n (number of nodes), intuitively we see that the expression containing c (α1 c1/2 −α2 logc) must scale at most as the term containing n (α2 logn). Otherwise, the expression containing c asymptotically dominates the expression containing n, and T would scale faster than logn. To achieve better than logarithmic scaling with n, we set α1 c1/2 and

126

S. Banerjee and M. Moses

α2 logn equal to each other giving c = O(log 2 n). Making these notions concrete we minimize T by differentiating with respect to c and have dT α1 α2 dn α2 = 1/2 − − =0 dc n dc c 2c

(4)

Simplifying and integrating we have α2 logc − α1 c1/2 = α2 logn + F where the constant F subsumes all the constants of integration. Asymptotically the optimal cluster size c = O(log 2 n) (5) The resulting total time for search in the SSW overlay network is now T = O(logn − loglogn)

(6)

Intuitively a larger cluster size reduces the number of clusters leading to a reduction in global search time across cluster. However a larger cluster size also increases local search time since there are more nodes. Our architecture balances these tradeoffs and improves the scaling of search time (T ) over prior P2P systems [19]. In addition to achieving rapid search, the modified P2P system is also robust and adaptive. The system is robust to node failures since there is a cluster of size O(log 2 n) leading to increased robustness by redundancy of similar data. Additionally, if search for data residing on a particular node does not succeed due to node failure, data similar to the one being searched for can be supplied by neighbouring semantically clustered nodes. Each node has connections with multiple clusters which leads to more redundant search pathways. The system adapts by having progressively more long-distant connections (l = O(logn)) in order to improve its search characteristics. Such densification properties are also observed in other technological networks (like the World Wide Web, citation networks, and autonomous systems) [18] and arise as an emergent property of these systems. The cluster size also adaptively changes to efficiently balance local and global search performance. In summary, we see that similar to an NIS, P2P systems exhibit tradeoffs between local communication (search within a cluster of nodes) and global communication (search across clusters). A sub-modular architecture in which the size of a cluster increases with system size is shown to efficiently balance these opposing goals. We modified an existing P2P system by incorporating a modular RADAR search strategy inspired by the natural immune system and demonstrated that it can conduct efficient search while being robust and adaptable.

7

Future Work

Future work will focus on augmenting two intrusion detection applications LISYS [20] and process Homeostasis (pH) [21], with a modular RADAR strategy. In this context, a subnet of computers would be an artificial DR and a security node in charge of the subnet would be analogous to a LN. A modular RADAR architecture can balance local communication (intrusion detection) and global

Modular RADAR

127

communication (alert or patch propagation). This would further advance work on the architecture of intrusion detection systems [22]. Such an approach may be particularly useful for security of low power distributed systems. For example, mobile phone viruses propagate by small hops to neighbouring within-range devices, and hence physical proximity of devices is much more important than in traditional computational domains [1]. Mobile phone viruses also consume low power and bandwidth in order to evade conventional detection mechanisms. Since mobile phones are constrained to communicate to nearby neighbors (e.g. through Bluetooth connections and local cell towers), the NIS analogy can be extended to view mobile phone transmission towers as LNs and the area of mobile phone users serviced by it as the DR. Finally, mobile ad-hoc networks (MANETs) and disruption tolerant networks (DTNs) [1] are of increasing interest. Such systems could also have a masterslave relationship in which individual components report aggregated data to a processing center (local communication) and processing centers distribute data globally among all components (global communication) [23]. A modular RADAR strategy can enhance message propagation times and increase robustness in such systems. In all of these systems, the balance between LN size and number depends on the constraints on local and global communication. In the NIS, we considered one scenario in which the pathogen is initially localized in the tissue of a single DR, and the draining LN must recruit additional B-cells from other LN through HEV that deliver blood. However this is one of several scenarios that the IS architecture has evolved. For example, some pathogens propagate immediately through blood and are exposed to multiple LN at once. In such a scenario there is no need for recruitment to a single LN because the pathogen is distributed to multiple LNs. In other scenarios, an infection may stay local, and again there is no need for recruitment. Understanding how the semi-modular architecture of the immune system balances these different scenarios is another area for future work.

8

Conclusions

The physical architecture of the immune system has co-evolved with the software, i.e. dynamic interactions between immune system cells and signals to achieve scalable Robust Adaptive Decentralized search and Automated Response (RADAR). In this paper we have focused on how and why search is distributed across LN, whose size and number increase sublinearly with animal size. We have shown that such a design balances local antigen detection with global antibody response to minimize total immune response time. As a result, immune response times are nearly scale invariant, in sharp contrast to the vast majority of biological rates that slow systematically as body size increases [24]. We have suggested that modular design principles from the immune system can complement the AIS software approaches to distributed computation, particularly to achieve scalable mobile device control, peer-to-peer networks, and potentially computer security applications.

128

S. Banerjee and M. Moses

Acknowledgements We would like to acknowledge fruitful discussions with Dr. Alan Perelson, Dr. Stephanie Forrest and Dr. Jedidiah Crandall. This work supported by a grant from the National Institute of Health (NIH RR018754). S.B. would like to acknowledge travel grants from RPT, SCAP and PIBBS at the University of New Mexico.

References 1. Kleinberg, J.: The Wireless Epidemic. Nature 449, 287–288 (2007) 2. Banerjee, S., Moses, M.: A Hybrid Agent Based and Differential Equation Model of Body Size Effects on Pathogen Replication and Immune System Response. In: Andrews, P.S. (ed.) ICARIS 2009. LNCS, vol. 5666, pp. 14–18. Springer, Heidelberg (2009) 3. Banerjee, S., Moses, M.: Scale Invariance of Immune System Response Rates and Times: Perspectives on Immune System Architecture. Swarm Intelligence (2010) (under review) 4. Peters, R.H.: The Ecological Implications of Body Size. Cambridge University Press, Cambridge (1983) 5. Soderberg, A.K., et al.: Innate Control of Adaptive Immunity via Remodeling of Lymph Node Feed Arteriole. PNAS 102, 16315–16320 (2005) 6. Huang, K.J., et al.: An interferon-gamma-related cytokine storm in SARS patients. Journal of Medical Virology 75(2), 185–194 (2005) 7. Surowiecki, J.: The wisdom of crowds. Little, Brown, London (2004) 8. Forrest, S., Beauchemin, C.: Computer immunology. Immunological Reviews 216, 176–197 (2007) 9. Diamond, M.S., et al.: A Critical Role for Induced IgM in the Protection against West Nile Virus Infection. Journal of Experimental Medicine (2003), doi:10.1084/jem20031223 10. Halin, C., et al.: In vivo imaging of lymphocyte trafficking. Ann. Rev. Cell Devel. Biol. 21, 581–603 (2005) 11. Altman, P.L., Dittmer, D.S.: Biology Data Book, 2nd edn., vol. 3. Federation of American Societies for Experimental Biology, Bethesda (1974) 12. Hildebrandt, T.B., et al.: Ultrasonographic assessment and ultra-sound guided biopsy of the retropharyngeal lymph nodes in Asian elephants (Elephas maximus). Vet. Rec. 157, 544–548 (2005) 13. Nair, S.B., et al.: An Immune System based Multi-Robot Mobile Agent Network. In: Bentley, P.J., Lee, D., Jung, S. (eds.) ICARIS 2008. LNCS, vol. 5132, pp. 424–433. Springer, Heidelberg (2008) 14. Mokhtar, M., et al.: An Artificial Lymph Node Architecture for Homeostasis in Collective Robotic Systems. In: Workshop on Pervasive Adaptive Systems (2008) 15. Hart, E., Davoudani, D.: Dendritic Cell Trafficking, From Immunology to Engineering. In: Andrews, P.S. (ed.) ICARIS 2009. LNCS, vol. 5666, pp. 11–13. Springer, Heidelberg (2009) 16. Li, M., et al.: Semantic Small World: An Overlay Network for Peer-to-Peer Search. In: IEEE International Conference on Network Protocols (ICNP 2004), pp. 228–238 (2004)

Modular RADAR

129

17. Kleinberg, J.: The small-world phenomenon: An algorithmic perspective. In: Proc. 32nd ACM Symposium on Theory of Computing (2000) 18. Leskovec, J., et al.: Graphs over Time: Densification Laws, Shrinking Diameters and Possible Explanations. In: Proc. 11th ACM SIGKDD Intl. Conf. on Knowledge Discovery and Data Mining (2005) 19. Lua, E.K., et al.: A Survey and Comparison of Peer-to-Peer Overlay Network Schemes. IEEE Communications Survey and Tutorial (2004) 20. Hofmeyr, S.A., Forrest, S.: Architecture for an artificial immune system. Evol. Comput. J. 8, 443–473 (2000) 21. Somayaji, A., Forrest, S.: Automated response using system-call delays. In: Usenix Security Symposium (2000) 22. Dasgupta, D.: Immunity-based intrusion detection system: A general framework. In: Proceedings of the 22nd National Information Systems Security Conference, NISSC (1999) 23. Delin, K.A.: The Sensor Web: a macro-instrument for coordinated sensing. Sensors 2, 270–285 (2002) 24. Brown, J.H.: Toward a Metabolic Theory of Ecology. Ecology 85, 1771–1789 (2004)

A Faster Clonal Selection Algorithm for Expensive Optimization Problems Heder S. Bernardino1 , Helio J.C. Barbosa1 , and Leonardo G. Fonseca2 1 2

Laborat´ orio Nacional de Computa¸ca ˜o Cient´ıfica – LNCC [email protected], [email protected] Universidade Federal do Esp´ırito Santo – CEUNES/UFES [email protected]

Abstract. Artificial Immune Systems (AISs) are computational methods, inspired by the biological immune system, that can be applied to solve optimization problems. In this paper we propose the use of a similarity-based surrogate model in conjunction with a clonal selection algorithm in order to improve its performance when solving optimization problems involving computationally expensive objective functions. Computational experiments to assess the performance of the proposed procedure using 23 test-problems from the literature are presented. Keywords: Clonal Selection, Artificial Immune System, Optimization, Surogate Model.

1

Introduction

When exposed to antigens an organism develops an efficient immune response where specific antibodies are produced to atack the antigens. The best immunological cells multiply (cloning) and are improved (hypermutation and replacement) while new cells, produced by the bone marrow, are generated. Thus, if the organism is again attacked by the same antigen a quicker immune response takes place. This scheme of adaptation is known as clonal selection and affinity maturation by hypermutation or, more simply, clonal selection [13]. Computational methods inspired by the biological immune system are called Artificial Immune Systems (AISs) and have found applications in many domains. One them, optimization, is a key ingredient to design and operational problems in all types of engineering, as well as a tool for formulating and solving inverse problems such as system identification in scientific and engineering situations. From the optimization perspective, the AISs are stochastic population-based search methods which do not require a continuous, differentiable, or explicit objective function, and do not get easily trapped in local optima. However, the AISs, as well as other nature-inspired techniques, usually require a large number of objective function evaluations in order to reach a satisfactory solution. As modern problems have lead to the development of increasingly complex and computationally expensive simulation models, this becomes a serious drawback to their application in areas such as Structural Mechanics, Reservoir Simulation, Fluid E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 130–143, 2010. c Springer-Verlag Berlin Heidelberg 2010 

A Faster Clonal Selection Algorithm for Expensive Optimization Problems

131

Dynamics, and Molecular Dynamics. Thus, the user has a given computational budget that places a strong limit to the number of calls to the expensive simulation model. It is then necessary to modify the search process in order to obtain either a reduction on the total computational cost or an increase in the convergence speed of the optimization procedure. The solution considered here is the use of a surrogate model (or metamodel), which provides a relatively inexpensive approximation of the objective function, replacing the computationally intensive original simulator evaluation. The idea of using less computationally expensive function evaluations appeared early in the evolutionary computation literature [15]. We may also mention additional reasons for using surrogate models in evolutionary algorithms: (a) to reduce complexity [18], (b) to smooth the fitness landscape [30], (c) to alleviate user fatigue in interactive design systems[28], (d) fast evaluation of candidate solutions [24,27], (e) noisy environments [16], and (f) efficiency enhancement [26,11,12]. Many surrogate models are available in the literature, such as polynomial models, artificial neural networks, Kriging or Gaussian processes, and support vector machines. Alternatively, several surrogates may be derived from physical or numerical simplifications of the original simulation model, in which case they are more strongly problem-dependent. In this paper we propose an artificial immune system (AIS) assisted by a similarity-based surrogate model (SBSM) in which the objective is, although still using a fixed number of expensive simulations, to allow the AIS to perform additional (approximate) objective function evaluations, in order to (hopefully) obtain a final solution which is better than the one the AIS would find using only that fixed amount of expensive simulations. This paper is organized as follows. Section 2 states the optimization problems considered. AISs for optimization are discussed in Section 3. Sections 4 and 5 present the surrogate models and the surrogate-assisted AIS, respectively. A discussion of the computational results is given in Section 6, and the paper ends with some concluding remarks and suggestions for future work.

2

The Optimization Problem

The optimization problems considered here correspond to finding x ∈ Rn that minimizes the objective function f (x), subject to the bounds xl ≤ x ≤ xu . We are particularly interested in the situation where obtaining the value of f (x) requires a computationally expensive simulation. In the proposed algorithm, the individuals evaluated by the original function (i.e., solutions evaluated exactly) are stored in a database (memory cells). The population of memory cells is used to construct a surrogate, based on similarity, which is used along the optimization procedure to perform extra (surrogate) evaluations, resulting in a larger number of total (surrogate plus exact) evaluations. Those extra surrogate evaluations involve a very simple procedure, with relatively negligible computational cost.

132

3

H.S. Bernardino, H.J.C. Barbosa, and L.G. Fonseca

Artificial Immune Systems for Optimization

In optimization problems [5], the AIS algorithms evolve improved solutions by means of natural immune mechanisms, such as clonal selection, immune network theory, vaccination, or other immune system concepts. In general, an immune optimization algorithm will have a population of antibodies (candidate solutions) and another set composed by the antigens (objectives) that the antibodies attempt to reach or match (optimize). The main differences among the AIS techniques applied to optimization reside in which natural immune mechanism is considered to evolve the antibodies, i.e., how the candidate solutions evolve. According to the clonal selection theory –the immune mechanism used by the algorithm considered here– there is a selection process which leads to the evolution of the immune system repertoire during the lifetime of the individual. Furthermore, on binding with a suitable antigen, activation of lymphocytes occurs and clones of the activated lymphocyte are produced expressing receptors identical to the original one that first encountered the antigen. The clonal selection culminates in the increase in the average affinity between the antibodies and antigens due to the somatic hypermutation and selection mechanisms of clonal expansion. That is responsible for the fact that upon a subsequent exposure to the antigen, a stronger immune response is produced [3]. The affinity maturation, as it is also known, is a mutation of the individuals applied with a high rate, which is inversely proportional to the fitness of the antibody (affinity antibodyantigen), unlike the standard mutation of Evolutionary Algorithms (EAs). Thus, inferior individuals are more strongly modified than the better ones, which need a finer tuning. To avoid a random search, a selection method is necessary to keep the good solutions, eliminate the worst ones, and maintain diversity. 3.1

Clonal Selection Algorithm

Based on the clonal selection theory, de Castro and Von Zuben proposed an AIS algorithm (CLONALG [8]) that performs computational optimization and pattern recognition tasks. In this method, each antibody is cloned, hypermutated, and those with higher affinity are selected. The main features of this technique are (i) the mutation rate, normally inversely proportional to the affinity of the antibody with respect to the antigens and (ii) the absence of recombination operators (such as crossover in GAs). Figure 1 shows CLONALG’s pseudo-code which is inspired by the algorithm presented in [5]. In Algorithm 1, af f inities are the values of the objective function to be optimized (fitness) and antibodies is the population of candidate solutions. Also, β is the number of clones each antibody will generate, ρ is a parameter used to compute the mutation rate, normalizedAf f inities (f¯) contains the affinities of the antibodies normalized (by function “normalize”) in the interval [0, 1], pRandom is the percentage of new cells that were randomly generated (representing the mesenchymal stem cells generated by the bone marrow), clones is the population of clones generated by the function “clone”, and cloneAf f inities are the corresponding affinities. The function “initializePopulation” initializes,

A Faster Clonal Selection Algorithm for Expensive Optimization Problems

133

Algorithm 1. A CLONALG pseudo-code for optimization problems.

1 2 3 4 5 6 7 8 9 10 11 12

Data: β, ρ, pRandom, populationSize Result: bestSolution begin antibodies ←− initializePopulation(populationSize); af f inities ←− evaluateAffinity(antibodies); while stopping criteria is not met do normalizedAf f inities ←− normalize(af f inities); clones ←− clone(antibodies, af f inities, β); hypermutate(clones, normalizedF itness, ρ); cloneAf f inities ←− evaluateAffinity(clones); select(antibodies, af f inities, clones, cloneF itness); generateAndEvaluateNewCells(antibodies, af f inities, pRandom); bestSolution ←− getBest(antibodies); end

usually randomly, a population of cells, “evaluateAffinity” calculates the affinity antibody-antigen, “hypermutate” performs the somatic hypermutation, “select” selects the best candidate solution between each antibody and its clones to compose the antibodies population [7], and finally, “generateAndEvaluateNewCells” randomly generates new antibody cells (it also evaluates and includes them in the candidate solutions population).

4

Surrogate Models

Surrogate modeling, or metamodeling, can be viewed as the process of replacing the original evaluation function (a complex computer simulation) by a substantially less expensive approximation. The surrogate model should be simple, general, and keep the number of control parameters as small as possible. Similarity-Based Surrogate Models (SBSMs), an example of such surrogates, will be described in the following sections. In contrast to “eager” learning algorithms, such as neural networks and polynomial response surfaces, which generate a model and then discard the inputs, the Similarity-Based Surrogate Models store their inputs and defer processing until a prediction of the affinity value of a new candidate solution is requested. Thus, SBSMs can be classified as “lazy” learners or memory-based learners [2] because they generate the output value by combining their stored data using a similarity measure. Any intermediate structure or result is then discarded. The following section describes in detail the k-nearest neighbor method, which is the SBSM used here. 4.1

The k-Nearest Neighbor Technique (k-NN)

The k-NN technique will be used here as a surrogate model. The idea of using k-NN to assist an evolutionary algorithm was explored in [23,22,10] to reduce the

134

H.S. Bernardino, H.J.C. Barbosa, and L.G. Fonseca

number of exact function evaluations needed during the search, and in [11,12] to extend the generations and to guide the search towards improved solutions. Given a candidate solution xh and the archive D = {(xi , f (xi )), i = 1, . . . , η} containing η exactly evaluated solutions xi , the corresponding value f(xh ) ≈ f (xh ), to be assigned to xh is ⎧ f (xj ) if xh = xj , for some j = 1, . . . , η ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ k Ijh u Ijh h h f(x ) = (1) j=1 s(x , x ) f (x ) otherwise ⎪  h ⎪ k I h ⎪ j u ⎪ j=1 s(x , x ) ⎪ ⎩ where s(xh , xi ) is a similarity measure between xh and xi , Ijh is the j-th element of the list that stores the individuals in the set D most similar to xh , k is the number of neighbors used to build the surrogate, and u is set to 2. In the real-coded CLONALG adopted, the similarity measure is given by s(xh , xi ) = 1 −

dE (xh , xi ) , dE (xU , xL )

where dE (xh , xi ) is the Euclidean distance between xh and xi . This technique does not require a predefined functional form nor a probability distribution. The variables can be either continuous or discrete, and the database is updated whenever it is necessary to add or remove candidate solutions. The computational cost for evaluating an individual is due to the search for the k nearest neighbors, as no training procedure is required.

5

Surrogate-Assisted Artificial Immune System

The next crucial step is to define how the surrogate model will be incorporated into the search process. Some possibilities discussed in the literature include incorporating the surrogate models as (i)local approximators [21], (ii)surrogateguided evolutionary operators [20], (iii)surrogate-assisted local search [17,29], and (iv)a pre-selection technique [14,19], as well as the use of multiple surrogates [1,17,25]. To the best of our knowledge, no surrogate-assisted immune system has been presented in the literature except [6] where the authors introduce the surrogate models into the immune inspired algorithm cycle by means of a stochastic management procedure which, in each iteration, uses in a cooperative way both surrogate and exact models. For that first proposal, called Random Selection, a candidate solution is evaluated by the exact function according to a user specified probability. The use of a surrogate model allows for increasing the total number of iterations of the algorithm and, for almost all problems considered, leads to solutions which are better than those provided by the baseline clonal selection algorithm

A Faster Clonal Selection Algorithm for Expensive Optimization Problems

135

(CLONALG). However, the authors suggested that, for some objective functions, the surrogate model increases the exploitation of the baseline CLONALG, inducing a faster convergence to local optima. The model management adopted in this work can be referred to as Deterministic Selection (DS). All candidate solutions from the initial population are evaluated exactly and compose the archive which defines the surrogate model. When performing clonal expansion, all generated clones are evaluated by the surrogate model. Then the best clone of each antibody is evaluated by the exact function. If the exact value is better than the affinity of the original antibody then the parent is replaced by this clone. One should notice that, while the model management used in [6] leads to a population containing both exactly and approximately evaluated individuals, here the population contains only exactly evaluated solutions. The use of the surrogate model is controlled by the number β of clones defined by the user. As more clones are generated by each antibody, more approximate evaluations are performed. It is easy to see that CLONALG is recovered when the antibodies generate only one clone (β = 1). Also, if the number of exact objective function evaluations is used as the stop criterion, the amount of final generations is equal for all β values. Every individual evaluated by the exact function is imediately stored in the archive used by the surrogate model. In the immunological paradigm, this database of antibodies corresponds to the memory cells: a set of representative cells stored with the objective of improving the immune system response to subsequent attacks.

6

Computational Experiments

In order to assess the performance of the surrogate-assisted clonal selection algorithm proposed here for expensive optimization problems, a set F of testfunctions fj , j ∈ {1, . . . , nf } often used in the literature [31] was chosen. Functions f1 to f13 correspond to high-dimensional problems (n = 30). Functions f1 to f5 are unimodal. Function f6 is the step function, which has one minimum. Function f7 is a noisy quartic function. Functions f8 to f13 are multimodal functions where the number of local minima increases exponentially with the problem dimension. They appear to be the most difficult class of problems for many optimization algorithms [31]. Functions f14 to f23 are low-dimensional functions which have only a few local minima. These problems correspond to explicitly defined functions and do not require expensive simulations. The idea is to assess the relative performance of the proposed technique when a fixed number of (potentially expensive) objective function evaluations is allowed. The parameters used by the algorithm presented in Section 3.1 are: ρ = 5, pRandom = 0, and populationSize = 30. Hypermutation is accomplished by adding or subtracting 30% of the current design variable value. This mutation procedure is inspired by that used in [13] and the mutation rate is equal to ¯ e−ρf , as proposed in [8] in which f¯ is the normalized affinity of the original antibody. These parameters and the hypermutation process are the same for all CLONALG variants in our experiments.

136

H.S. Bernardino, H.J.C. Barbosa, and L.G. Fonseca

A set of variants of the basic procedure corresponding to particular sets of parameter values (the number of clones β, and the number of neighbors k) was considered, namely: (i) {k = 2, β = 5}, (ii) {k = 4, β = 5}, (iii) {k = 2, β = 10}, and (iv) {k = 4, β = 10}, which were compared to the baseline CLONALG (β = 1), plotted at the left hand side. The boxplots corresponding to a set of 30 independent runs are displayed in Figures 1-4. From Figure 1 it is clear that all variants of the surrogate assisted CLONALG outperform the baseline CLONALG on test-problems one to six. However, for f05 , only the {k = 4, β = 10} variant clearly outperforms CLONALG. In Figure 2 it is seen that all variants of the surrogate assisted CLONALG outperform the baseline CLONALG on test-functions 7, 11, and 12, but are outperformed in test-functions 8 and 10, while comparable performances are observed in test-function 9. From Figure 3 it is clear that all variants of the surrogate assisted CLONALG outperform the baseline CLONALG except in test-problems 14, 15, and 18, where no relevant improvement was observed. It is clear from Figure 4 that all variants of the surrogate assisted CLONALG outperform the baseline CLONALG in test-problems 20 to 23. The most common way of assessing the relative performance of a set V of variants vi , i ∈ {1, . . . , nv } is to define a set F of representative test-functions fj , j ∈ {1, . . . , nf } and then test all variants against all problems measuring the performance tf,v of variant v ∈ V when applied to function f ∈ F . The performance indicator to be maximized here is the inverse of the minimum objective function value found by variant v in test-function f averaged over 30 runs. Now a performance ratio can be defined as rf,v =

tf,v min{tf,v : v ∈ V }

(2)

Although each tf,v is worth considering by itself, and can be observed in the Figures 1-4, it is interesting to be able to assess the performance of the variants in V on a potentially large set of test-functions F in a compact graphical form. This can be attained following Dolan & More [9] and defining ρv (τ ) =

1 |{f ∈ F : rf,v ≤ τ }| nf

where |.| denotes the cardinality of a set. Then ρv (τ ) is the probability that the performance ratio rf,v of variant v ∈ S is within a factor τ ≥ 1 of the best possible ratio. If the set F is large and representative of problems yet to be tackled then variants with larger ρs (τ ) are to be preferred. The performance profiles thus defined have a number of useful properties [9,4] such as (i) ρv (1) is the probability that variant v will provide the best performance in F among all variants in V . If ρV 1 (1) > ρV 2 (1) then variant V 1 was the winner in a larger number of problems in F than variant V 2, (ii) the area under the ρv curve (AU Cv = ρv (t) dt) is an overall performance indicator/measure for variant

A Faster Clonal Selection Algorithm for Expensive Optimization Problems f01

f02 CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

0

0.1

1

2

20

3

4

5

6

Objective Function 40 60

7

Objective Function 0.2 0.3

8

0.4

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

80

137

1

5 10 Number of Clones

1

5 10 Number of Clones

40

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

Objective Function 30 35

3.0e−05

25

1.5e−05

0

20

0e+00

20

Objective Function 40 60

80

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

f04 45

f03

1

5 10 Number of Clones

1

5 10 Number of Clones

f06

8

10

12

14

Objective Function 40 60

16

80

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4) 1

5 10 Number of Clones

20

Objective Function 25.5 26.0 26.5 27.0 27.5 28.0 28.5

f05

1

5 10 Number of Clones

Fig. 1. Boxplots for functions f01 to f06 . In some plots an extra box is presented in order to zoom in the results for better viewing.

138

H.S. Bernardino, H.J.C. Barbosa, and L.G. Fonseca f08 CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

−7000

0.10

Objective Function −6600 −6200

Objective Function 0.15 0.20 0.25

0.30

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

−5800

f07

1

1

5 10 Number of Clones

5 10 Number of Clones

f10

Objective Function 10 12 14 CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4) 1

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

8

60

Objective Function 80 100 120

16

18

140

f09

5 10 Number of Clones

1

f12 CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

30 25 20 15 0

5

10

Objective Function 100 150 200 0

1.00

50

1.01

1.001

1.002

1.003

Objective Function 1.02 1.03 1.04

1.004

1.05

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

250

1.06

f11

5 10 Number of Clones

1

5 10 Number of Clones

1

5 10 Number of Clones

Fig. 2. Boxplots for functions f07 to f12 . In some plots an extra box is presented in order to zoom in the results for better viewing.

A Faster Clonal Selection Algorithm for Expensive Optimization Problems f13 30

f14 25

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

20

40

60

80

Objective Function 10 15 20

100

2500

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

0

0

5

500

0

Objective Function 1000 1500 2000

139

5 10 Number of Clones

1

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

Objective Function 1.5 2.0

3

1.0

Objective Function 1 2 −1

0.5

0

0.01

5 10 Number of Clones

1

5 10 Number of Clones

1

5 10 Number of Clones

f19 CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

0

−3.8

100

0

20

40

60

80

Objective Function −3.6 −3.4 −3.2

100

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

−3.0

600

f18 500

2.5

f17

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

4

f16

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

0.00

1

Objective Function 200 300 400

5 10 Number of Clones

f15

Objective Function 0.02 0.03 0.04

0.05

1

1

5 10 Number of Clones

1

5 10 Number of Clones

Fig. 3. Boxplots for functions f13 to f19 . In some plots an extra box is presented in order to zoom in the results for better viewing.

140

H.S. Bernardino, H.J.C. Barbosa, and L.G. Fonseca f21 CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

−8

Objective Function −6 −4

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

−2

Objective Function −3.2 −3.1 −3.0 −2.9 −2.8 −2.7 −2.6

f20

1

5 10 Number of Clones

1

f23 −2

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

−10

−7

−6

−8

Objective Function −6 −4

Objective Function −5 −4 −3

−2

CLONALG Fast CLONALG (k=2) Fast CLONALG (k=4)

−1

f22

5 10 Number of Clones

1

1

5 10 Number of Clones

5 10 Number of Clones

Fig. 4. Boxplots for functions f20 to f23

v in the problem set F : the larger the AUC the higher the variant efficiency, and (iii) a measure of the reliability of variant v is its performance ratio in the problem where it performed worst: Rv = sup{τ : ρv (τ ) < 1}. As a result, the most reliable variant is the one that minimizes Rv ; that is, it presents the best worst performance in the set F : v ∗ = arg min Rv = arg min sup{τ : ρv (τ ) < 1} v∈V

v∈V

It can be seen that (i)the variant {k = 2, β = 10} is the most efficient, being the winner in about 65% of the problems, (ii)the variant {k = 4, β = 5} is the most reliable, since it is able to solve all problems within 1.7 times the value obtained by the best solver, and (iii)CLONALG is the worst overall performer since it presents a significantly smaller area under the curve, reaching results up to 25 times worst than the best possible among the variants considered.

0.6

0.8

1.0

141

1.0

1.2

1.4

1.6

1.8

2.0

0.2

0.4

0.0

CLONALG Fast CLONALG (k=2, β=5) Fast CLONALG (k=4, β=5) Fast CLONALG (k=2, β=10) Fast CLONALG (k=4, β=10)

CLONALG Fast CLONALG (k=2, β=5) Fast CLONALG (k=4, β=5) Fast CLONALG (k=2, β=10) Fast CLONALG (k=4, β=10)

0.0

0.2

0.4

0.6

0.8

1.0

A Faster Clonal Selection Algorithm for Expensive Optimization Problems

0

5

10

15

20

25

Fig. 5. Performance profiles comparing the algorithms

7

Concluding Remarks

In this paper a real-coded clonal selection algorithm is enhanced by means of the use of a similarity-based surrogate model, namely the k-nearest neighbors technique, in order to improve its performance when solving optimization problems involving computationally expensive objective functions. Computational experiments are performed to assess the performance of the proposed procedure using a benchmark with 23 test-problems from the literature where a maximum number of objective function evaluations is prescribed. The results show that the use of a similarity-based surrogate model improves the results obtained by the baseline CLONALG for most test-problems, specially for the unimodal functions. It seems that the k-NN technique tends to compensate for CLONALG’s more exploratory behavior and enhances its convergence speed. Also, in the few cases where no improvement is observed, CLONALG’s performance is only slightly degraded by the use of the k-NN technique. As a future work, the insertion of the meta-model in other clonal selection and evolutionary algorithms, as well as the use of other problems and benchmarks, will be considered for further evaluations. Another idea that could be pursued in order to improve efficiency in the scenario of expensive affinity computation is the immune network theory, aiming at removing similar antibodies by means of a suppression technique. The resulting search algorithm would combine the memory and self-adaptation features of the immune system. Acknowledgments. The authors thank the support from CNPq (140551/20085 and 308317/2009-2) and FAPERJ (E-26/ 102.825/2008).

142

H.S. Bernardino, H.J.C. Barbosa, and L.G. Fonseca

References 1. Acar, E., Rais-Rohani, M.: Ensemble of metamodels with optimized weight factors. Struct. Multidisc Optim. 37(3), 279–294 (2009) 2. Aha, D.W.: Editorial. Artif. Intell. Rev. 11(1-5), 1–6 (1997); Special issue on lazy learning 3. AISWeb. The online home of artificial immune systems, http://www.artificial-immune-systems.org (accessed 11/09/2008) 4. Barbosa, H.J.C., Bernardino, H.S., Barreto, A.M.S.: Using performance profiles to analyze the results of the 2006 CEC constrained optimization competition. In: IEEE World Congress on Computational Intelligence, Barcelona, Spain (July 2010) 5. Bernardino, H.S., Barbosa, H.J.C.: Artificial immune systems for optimization. In: Chiong, R. (ed.) Nature-Inspired Algorithms for Optimisation, pp. 389–411. Springer, Heidelberg (2009) 6. Bernardino, H.S., Fonseca, L.G., Barbosa, H.J.C.: Surrogate-assisted artificial immune systems for expensive optimization problems. In: dos Santos, W.P. (ed.) Evolutionary Computation, pp. 179–198. IntechWeb (2009) 7. Cutello, V., Narzisi, G., Nicosia, G., Pavone, M.: Clonal selection algorithms: A comparative case study using effective mutation potentials. In: Jacob, C., Pilat, M.L., Bentley, P.J., Timmis, J.I. (eds.) ICARIS 2005. LNCS, vol. 3627, pp. 13–28. Springer, Heidelberg (2005) 8. de Castro, L.N., Zuben, F.J.V.: Learning and optimization using the clonal selection principle. IEEE Trans. Evolutionary Computation 6(3), 239–251 (2002) 9. Dolan, E., Mor´e, J.J.: Benchmarcking optimization software with performance profiles. Math. Programming 91(2), 201–213 (2002) 10. Fonseca, L.G., Barbosa, H.J.C., Lemonge, A.C.C.: Metamodel assisted genetic algorithm for truss weight minimization. In: ICMOSPS 2007, Durban, South Africa (2007) (CD-ROM) 11. Fonseca, L.G., Barbosa, H.J.C., Lemonge, A.C.C.: A similarity-based surrogate model for enhanced performance in genetic algorithms. Opsearch 46, 89–107 (2009) 12. Fonseca, L.G., Barbosa, H.J.C., Lemonge, A.C.C.: On similarity-based surrogate models for expensive single- and multi-objective evolutionary optimization. In: Tenne, Y., Goh, C.-K. (eds.) Computational Intelligence in Expensive Optimization Problems, pp. 219–248. Springer, Heidelberg (to appear, 2010) 13. Garrett, S.M.: Parameter-free, adaptive clonal selection. In: IEEE Congress on Evolutionary Computation, vol. 1, pp. 1052–1058 (2004) 14. Giannakoglou, K.C.: Design of optimal aerodynamic shapes using stochastic optimization methods and computational intelligence. Progress in Aerospace Sciences 38(1), 43–76 (2002) 15. Grefenstette, J., Fitzpatrick, J.: Genetic search with approximate fitness evaluations. In: Proc. of the Intl. Conf. on Genetic Algorithms and Their Applications, pp. 112–120 (1985) 16. Jin, Y., Branke, J.: Evolutionary optimization in uncertain environments-a survey. IEEE Trans. Evolutionary Computation 9(3), 303–317 (2005) 17. Lim, D., Jin, Y., Ong, Y.-S., Sendhoff, B.: Generalizing surrogate-assisted evolutionary computation. IEEE Trans. on Evolutionary Computation (in press) 18. Ong, Y., Nair, P., Keane, A.: Evolutionary optimization of computationally expensive problems via surrogate modeling. AIAA Journal 41(4), 687–696 (2003) 19. Praveen, C., Duvigneau, R.: Low cost PSO using metamodels and inexact preevaluation: Application to aerodynamic shape design. Computer Methods in Applied Mechanics and Engineering 198(9-12), 1087–1096 (2009)

A Faster Clonal Selection Algorithm for Expensive Optimization Problems

143

20. Rasheed, K., Vattam, S., Ni, X.: Comparison of methods for using reduced models to speed up design optimization. In: Proc. of Genetic and Evolutionary Computation Conference, New York, pp. 1180–1187. Morgan Kaufmann, San Francisco (2002) 21. Regis, R.G., Shoemaker, C.A.: Local function approximation in evolutionary algorithms for the optimization of costly functions. IEEE Trans. Evolutionary Computation 8(5), 490–505 (2004) 22. Runarsson, T.: Approximate evolution strategy using stochastic ranking. In: Yen, G.G., et al. (eds.) IEEE World Congress on Computational Intelligence, Vancouver, Canada, pp. 745–752 (2006) 23. Runarsson, T.P.: Constrained Evolutionary Optimization by Approximate Ranking and Surrogate Models. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., MereloGuerv´ os, J.J., Bullinaria, J.A., Rowe, J.E., Tiˇ no, P., Kab´ an, A., Schwefel, H.-P., et al. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 401–410. Springer, Heidelberg (2004) 24. Salami, M., Hendtlass, T.: A fast evaluation strategy for evolutionary algorithms. Applied Soft Computing 2, 156–173 (2003) 25. Sanchez, E., Pintos, S., Queipo, N.: Toward an optimal ensemble of kernel-based approximations with engineering applications. In: Structural and Multidisciplinary Optimization, pp. 1–15 (2007) 26. Sastry, K., Lima, C.F., Goldberg, D.E.: Evaluation relaxation using substructural information and linear estimation. In: Proc. of the 8th Annual Conference on Genetic and Evolutionary Computation, pp. 419–426. ACM Press, New York (2006) 27. Smith, R.E., Dike, B.A., Stegmann, S.A.: Fitness inheritance in genetic algorithms. In: Proc. of the ACM Symposium on Applied Computing, pp. 345–350 (1995) 28. Sun, X.Y., Gong, D., Li, S.: Classification and regression-based surrogate modelassisted interactive genetic algorithm with individual’s fuzzy fitness. In: Proc. of the 11th Annual Conference on Genetic and Evolutionary Computation, pp. 907–914. ACM Press, New York (2009) 29. Wanner, E.F., Guimaraes, F.G., Takahashi, R.H.C., Lowther, D.A., Ramirez, J.A.: Multiobjective memetic algorithms with quadratic approximation-based local search for expensive optimization in electromagnetics. IEEE Trans. on Magnetics 44(6), 1126–1129 (2008) 30. Yang, D., Flockton, S.J.: Evolutionary algorithms with a coarse-to-fine function smoothing. In: IEEE Intl. Conf. on Evolutionary Computation, vol. 2, pp. 657–662 (1995) 31. Yao, X., Liu, Y., Lin, G.: Evolutionary programming made faster. IEEE Trans. on Evolutionary Computation 3, 82–102 (1999)

An Information-Theoretic Approach for Clonal Selection Algorithms Vincenzo Cutello, Giuseppe Nicosia, Mario Pavone, and Giovanni Stracquadanio Department of Mathematics and Computer Science University of Catania V.le A. Doria 6, I-95125 Catania, Italy {cutello,nicosia,mpavone,stracquadanio}@dmi.unict.it

Abstract. In this research work a large set of the classical numerical functions were taken into account in order to understand both the search capability and the ability to escape from a local optimal of a clonal selection algorithm, called i-CSA. The algorithm was extensively compared against several variants of Differential Evolution (DE) algorithm, and with some typical swarm intelligence algorithms. The obtained results show as i-CSA is effective in terms of accuracy, and it is able to solve large-scale instances of well-known benchmarks. Experimental results also indicate that the algorithm is comparable, and often outperforms, the compared nature-inspired approaches. From the experimental results, it is possible to note that a longer maturation of a B cell, inside the population, assures the achievement of better solutions; the maturation period affects the diversity and the effectiveness of the immune search process on a specific problem instance. To assess the learning capability during the evolution of the algorithm three different relative entropies were used: Kullback-Leibler, R´enyi generalized and Von Neumann divergences. The adopted entropic divergences show a strong correlation between optima discovering, and high relative entropy values. Keywords: Clonal selection algorithms, population-based algorithms, information theory, relative entropy, global numerical optimization.

1 Introduction Global optimization is the task of finding the set of values that assures the achievement of a global optimum for a given objective function; these problems are typically difficult to solve due to the presence of many local optimal solutions. Since in many realworld applications analytical solutions are not available or cannot be approximated, derivative-free methods are often the only viable alternative. Without loss of generality, the global optimization requires finding a setting x = (x1 , x2 , . . . , xn ) ∈ S, where S ⊆ Rn is a bounded set on Rn , such that the value of n-dimensional objective function f : S → R is minimal. In particular, the goal for global minimization problem is to find a point xmin ∈ S such that f (xmin ) is a global minimum on S, i.e. ∀x ∈ S : f (xmin ) ≤ f (x). The problem of continuous optimization is a difficult task, both because it is difficult to decide when a global (or local) optimum has been reached, and because there could be many local optima that trap the search process. As the problem dimension increases, the number of local optima can grow dramatically. E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 144–157, 2010. c Springer-Verlag Berlin Heidelberg 2010 

An Information-Theoretic Approach for Clonal Selection Algorithms

145

In this paper we present a clonal selection algorithm (CSA), labelled as i-CSA, to tackle global optimization problems as already proposed in [8,9]. The following numerical minimization problem was taken into account: min(f (x)), Bl ≤ x ≤ Bu , where x = (x1 , x2 , . . . , xn ) ∈ Rn is the variable vector, i.e. the candidate solution; f (x) is the objective function to minimize; Bl = (Bl1 , Bl2 , . . . , Bln ), and Bu = (Bu1 , Bu2 , . . . , Bun ) represent, respectively, the lower and the upper bounds of the variables such that xi ∈ [Bli , Bui ] with i = (1, . . . , n). Together with clonal and hypermutation operators, our CSA incorporates also the aging operator that eliminates the old B cells into the current population, with the aim to generate diversity inside the population, and to avoid getting trapped in a local optima. It is well known that producing a good diversity is a central task on any population-based algorithm. Therefore, increasing or decreasing the allowed time to stay in the population (δ) to any B cell influences the performances and convergence process. In this work, we show that increasing δ in CSA, as already proposed in [8,9], we are able to improve its performances. Two different function optimization classes were used for our experiments: unimodal and multimodal with many local optima. These classes of functions were used also to understand and analyze the learning capability of the algorithm during the evolution. Such analysis was made studying the learning gained both with respect to initial distribution (i.e. initial population), and the ones based on the information obtained in previous step. Three relative entropies were used in our study to assess the learning capability: Kullback-Leibler, R´enyi generalized and Von Neumann divergences [16,18,19]. Looking the learning curves produced by the three entropies is possible to observe a strong correlation between the achievement of optimal solutions and high values of relative entropies.

2 An Optimization Clonal Selection Algorithm Clonal Selection Algorithm (CSA) is a class of AIS [1,2,3,4] inspired by the clonal selection theory, which has been successfully applied in computational optimization and pattern recognition tasks. There are many different clonal selection algorithms in literature, some of them can be found in [20]. In this section, we introduce a variant of the clonal selection algorithm already proposed in [8,9], including its main features, as cloning, inversely proportional hypermutation and aging operators. i-CSA presents a population of size d, where any candidate solution, i.e. the B cell receptors, is a point of the search space. At the initial step, that is when t = 0, any B cell receptor is randomly generated in the relative domain of the given function: each variable in any B cell receptor is obtained by xi = Bli + β · (Bui − Bli ), where β ∈ [0, 1] is a real random value generated uniformly at random, and Bli and Bui are, respectively, the lower and the upper bounds of the i − th variable. Once it has been (t=0) created the initial population Pd , and has been evaluated the fitness function f (x) for each x receptor, i.e. computed the given function, the immunological operators are applied. The procedure to compute the fitness function has been labelled in the pseudo-code (table 1) as comp fit(∗). Through the cloning operator, a generic immunological algorithm produces individuals with higher affinity (i.e. lower fitness function values for

146

V. Cutello et al.

minimization problems), by introducing blind perturbations (by means of a hypermutation operator) and selecting their improved mature progenies. The Cloning opera(clo) tor clones each B cell dup times producing an intermediate population PN c , where N c = d× dup. Since any B cell receptor has a limited life span into the population, like in nature, become an important task, and sometime crucial, to set both the maximum number of generations allowed, and the age of each receptor. As we will see below, the Aging operator is based on the concept of age associated to each element of the population. This information is used in some decisions regarding the individuals. Therefore one question is what age to assign to each clone. In [5], the authors proposed a CSA to tackle the protein structure prediction problem where the same age of the parent has been assigned to the cloned B cell, except when the fitness function of the mutated clone is improved; in this case, the age was fixed to zero. In the CSA proposed for numerical optimization [8,9], instead, the authors assigned as age of each clone a random value chosen in the range [0, τB ], where τB indicates the maximum number of generations allowed. A mixing of the both approaches is proposed in [11] to tackle static and dynamic optimization tasks. The strategy to set the age of a clone may affect the quality of the search inside the landscape of a given problem. In this work we present a simple variant of CSA proposed in [8,9] obtained by randomly choosing the age in the range [0, 23 τB ]. According to this strategy, each B cell receptor is guaranteed to live more time into the population than in the previous version. This simple change produces better solutions on unconstrained global optimization problems, as it is possible to note in section 4. Table 1. Pseudo-code of the i-CSA i-CSA (d, dup, ρ, τB , Tmax ) f en := 0; N c := d × dup; (t=0) Pd := init pop(d); (t=0) comp fit(Pd ); f en := f en + d; while (f en < Tmax )do (clo) (t) PNc := Cloning (Pd , dup); (hyp) (clo) PNc := Hypermutation(PNc , ρ); (hyp) comp fit(PNc ); f en := f en + N c; (t) (hyp) (t) (hyp) (P ad , P aNc ) := aging(Pd , PNc , τB ); (t+1) (t) (hyp) Pd := (μ + λ)-selection(P ad , P aNc ); t := t + 1; end while (clo)

The hypermutation operator acts on the B cell receptor of PN c , and tries to mutate each B cell receptor M times without an explicit usage of a mutation probability. Its feature is that the number of mutations M is determined in inversely proportional way respect fitness values; as the fitness function value increases, the number of mutations performed on it decreases. There are nowadays several different kinds of hypermutation

An Information-Theoretic Approach for Clonal Selection Algorithms

147

operators [12] and [13]. The number of mutations M is given by the mutation rate: ˆ α = e−ρf(x) , where fˆ(x) is the fitness function value normalized in the range [0, 1]. As described in [8,9], the perturbation operator for any receptor x randomly chooses a variable xti , with i ∈ {1, . . . , } ( is the length of B cell receptor, i.e. the problem dimension), and replace it with     (t+1) (t) (t) = (1 − β) × xi + β × xrandom , xi (t)

(t)

where xrandom  = xi is a randomly chosen variable and β ∈ [0, 1] is a real random number. All the mutation operators apply a toroidal approach to shrink the variable value inside the relative valid region. To normalize the fitness into the range [0, 1], instead to use the optimal value of the given function, we have taken into account the best current fitness value in Pdt , decreased of an user-defined threshold Θ, as proposed in [8,9]. This strategy is due to making the algorithm as blind as possible, since is not known a priori any additional information concerning the problem. The third immunological operator, aging operator, has the main goal to design an high diversity into the current population, and hence avoid premature convergence. It acts on the two popu(t) (hyp) lations Pd , and PN c , eliminating all old B cells; when a B cell is τB + 1 old it is erased from the current population, independently from its fitness value. The parameter τB indicates the maximum number of generations allowed to each B cell receptor to remain into the population. At each generation, the age of each individual is increased. Only one exception is made for the best receptor; when generating a new population the selection mechanism does not allow the elimination of the best B cell. After the ag(t) (hyp) ing operator is applied, the best survivors from the populations P ad and P aN c are (t+1) selected for the new population Pd , of d B cells. If only d1 < d B cells survived, then (μ + λ)-Selection operator (with μ = d and λ = N c) pick at random d − d1 B cells among those “died” from the set   (t) (t) (hyp) (hyp) (Pd \ P ad ) (PN c \ P aN c ) . Finally, the evolution cycle ends when the fitness evaluation number (f en) is greater or equal to the allowed maximum number of fitness function evaluations, labelled with Tmax ; f en is a counter that is increased whenever the procedure comp fit(∗) is called. Table 1 summarizes the proposed CSA described above.

3 Kullback-Leibler, R´enyi and von Neumann Entropic Divergences A study on the learning process of i-CSA during the evolution was made, using three different divergence metrics, as Kullback-Leibler, R´enyi generalized, and Von Neumann divergences. These information were obtained both with respect the initial distribution, and the ones in previous step. Shannon’s entropy [15] is a commonly used measure in information theory and it represents a good measure of randomness or uncertainty, where the entropy of a random

148

V. Cutello et al.

variable is defined in terms of its probability distribution. Shannon’s theory, information is represented by a numerically measurable quantity, using a probabilistic model. In this way, the solutions of a given problem can be formulated in terms of the obtained amount of information. Kullback-Leibler divergence (KLd ) [16], also known as Information gain, is one of the most frequently used information-theoretic distance measure, based on two probability distributions of discrete random variable, called relative information, which found many applications in setting important theorems in information theory and statistics. This divergence measures the quantity of information the system discovers during the learning phase with respect to the initial population [6]. (t) t We define the B cells distribution function fm as the ratio between the number, Bm , of B cells at time step t with fitness function value m, and the total number of B cells: Bt (t) fm = h m m=0

t Bm

=

t Bm . d

The KLd is formally defined as: (t) (t0 ) KLd(fm , fm )

=



 (t) fm

log

(t)

fm

(t )

fm 0

m

 .

The gain is the amount of information the system has already learned during its search process, with respect to the randomly generated initial population P (t=0) (initial distribution). Once the learning process begins the information gain increases monotonically until it reaches a final steady state (see figure 1). This is consistent with the idea of a d maximum Kullback-Leibler principle [17] of the form: dKL ≥ 0. Since the learning dt dKLd process will end when dt = 0, then such maximum principle may be used as termination condition [6,14]. We are aware that the same metric can be adopted to study Information Gain 25

f5 f7

f10 20

15

10

5

0 1

2

4

8 Generations

16

32

Fig. 1. Learning of the problem. Kullback-Leibler curves of i-CSA algorithm on the functions f5 , f7 , and f10 . Each curve was obtained over 50 independent runs, with the following parameters: d = 100, dup = 2, τB = 15, ρ = 3.5 and Tmax = 5 × 105 .

An Information-Theoretic Approach for Clonal Selection Algorithms

149

the gain of information regarding the spatial arrangement of the solution in the search landscape; in particular, in these terms, when the gain goes to zero it states that the individuals represent the same solution. Figure 1 shows the KLd curves obtained by i-CSA, on the functions f5 , f7 , and f10 of the classical benchmark used for our experiments, and proposed in [7]. In this figure one can see as the algorithm gains quickly amounts of information on the functions f7 and f10 , rather than on f5 , whose KLd is slower; it starts to gain information after generation number 20. This is due because the search space on f5 seems to be more complex than in f7 and f10 , and this behavior is consistent with the obtained experimental results, where i-CSA, and optimization algorithms in general, need a greater number of fitness function evaluations to achieve a good solution. The curves plotted in such figure were obtained with the following experimental protocol: d = 100, dup = 2, τB = 15, ρ = 3.5 and Tmax = 5 × 105 . This experiment was performed over 50 independent runs. Clonal Selection Algorithm: i-CSA 25

20

300

15

250 200 10

150 100 50

5

0 16

32

64

0 16

32 Generations

64

Fig. 2. Kullback-Leibler curves of i-CSA algorithm on the function f5 . The inset plot shows the standard deviation. Both curves are a zoom into the range [16, 64].

Together with the amount of information learned during the search process, we want to know the uncertainty over the population of i-CSA during all performed runs. Therefore, the standard deviation was also used, which measures the dispersion of a set of data from its mean: more the data are spread apart, higher is the standard deviation. The plot 2 shows the monotonous behavior of the information gain only on function f5 (being one of the most complex), whilst in the inset plot is plotted the standard deviation measure. Both curves are a zoom into the range [16, 64]. The standard deviation increases quickly (the spike in the inset plot) when the algorithm begins to learn information; once the algorithm begins to gain more information, i.e. after 2 − 3 generations from beginning the learning process, the curve of the standard deviation decreases towards zero. The algorithm converges to the best solution in this temporal window. Thus, the highest point of information learned corresponds to the lowest value of uncertainty.

150

V. Cutello et al.

Clonal Selection Algorithm: i-CSA 4e+09

avg fitness best fitness

3.5e+09 3e+09 25

gain entropy

Fitness

2.5e+09 20 2e+09

15 10

1.5e+09

5 1e+09

0 64

32

16 5e+08 0 0

2

4

6

8

10

Generations

Fig. 3. Average fitness versus best fitness of i-CSA on the function f5 , for the first 10 generations. The inset plot shows the entropy and information gain curves. Each curve was obtained over 50 independent runs, with the following parameters: d = 100, dup = 2, τB = 15, ρ = 3.5 and Tmax = 5 × 105 .

In figure 3 we show the average fitness and the best fitness for i-CSA in the first 10 generations, where the algorithm quickly decreases from solutions of the order 109 , until to reach solutions of the order (101 − 1). Also in this figure, the inset plot shows the curves of the information gain, and the entropies. In the last two plots, the best solution found during the experiments is 0.0, whilst the mean of the best solutions is 15.6, with 14.07 as standard deviation. The experiments were obtained using the following parameters: d = 100, dup = 2, τB = 15, ρ = 3.5 and Tmax = 5 × 105 . Since the validity of the metrics is independent from the specific form adopted, we have evaluated the learning capability of i-CSA also with respect two well-known entropic divergences: (1) R´enyi generalized divergence (Rd ) [18], formally defined as   (t) α  fm 1 (t) (t0 ) Rd (fm , fm , α) = log , (t0 ) α−1 α−1 m f m

with α > 0 and α  = 1, and (2) Von Neumann divergence (V Nd ) [19], defined as: (t) (t0 ) , fm )=− V Nd (fm

1  (t) 1  (t) (t0 ) (t) (fm log fm )− (f log fm ) n m n m m

In figure 4 is showed the comparison among these different learning measures; KLd and Rd have the same monotonous increasing behavior, although Rd starts to learn information some generations before than KLd, which instead reaches in faster way the first peak. As already saw, Rd loses information in the first generations (around 20 generations), whilst in the same generations KLd neither loses information, but nor even is able to learn information. Since the behavior of the V Nd curve is not easy to understand, the inset plot shows only this curve with a zoom in y axis. We have performed the same experiments with the goal to analyze also the learning process with

An Information-Theoretic Approach for Clonal Selection Algorithms

151

Learning with respect to initial distribution (t0) 30

Von Neumann distribution (VNd) Renyi generalized Kullback-Leibler entropy

25

1.2

20

VNd

1 15

0.8 0.6

10

0.4 0.2

5

0 16

32

64

0

-5 16

32

64

Generations

Fig. 4. Kullback-Leibler entropy, R´enyi generalized divergence, and Von Neumann entropy curves with respect the initial distribution. The inset plot shows the only Von Neumann distribution, zooming in the y-axis. Learning with respect (t-1) time step 30

Von Neumann divergence (VNd) Renyi generalized Kullback-Leibler entropy

25

0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

20

15

10

5

VNd

16

32

64

0

-5 16

32 Generations

64

Fig. 5. Kullback-Leibler entropy, R´enyi generalized divergence, and Von Neumann entropy curves with respect the previous time step, i.e. at (t − 1) time step. The inset plot shows the only Von Neumann distribution, zooming in the y-axis.

respect the information obtained at the previous time step (t − 1). The figure 5 shows the behaviors of the learning processes of the three entropies used. From this plot, we see as the Rd entropy continues to have a similar behavior to one analyzed respect to the initial distribution, whilst KLd and V Nd once reached the maximum information (the peak), and thus the greater gain, they begin to lose information. Finally, for all figures shown in this section (from figure 2 to figure 5), we have taken into account the only function f5 being one of the most complex function of the test-bed used.

152

V. Cutello et al.

Table 2. Comparison between IA, i-CSA and rand/1/bin variant The mean of the best individuals on all runs and the standard deviation (in a new line) are presented for all algorithms, and we highlight in boldface the best results, except when the results are the same. The showed results are obtained using n = 30 and n = 100 dimensional search space. IA f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13

f1 f2 f3 f4 f5 f6 f7

0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 12 13.22 0.0 0.0

1.521 × 10−5 2.05 × 10−5 −1.256041 × 10+4 25.912 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

6.447 × 10−7 3.338 × 10−6 74.99 38.99 0.0 0.0 1.59 × 10−5 3.61 × 10−5

i-CSA 30 variables 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

7.48 × 10−6 6.46 × 10−6 −9.05 × 10+3 1.91 × 104 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100 variables 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 22.116 39.799 0.0 0.0 1.2 × 10−6 1.53 × 10−6

−4.16 × 10+4 2.06 × 10+2 0.0 0.0

−2.727 × 10+4 7.63 × 10−4 0.0 0.0

f10

0.0 0.0

0.0 0.0

f11

0.0 0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0 0.0

0.0

0.0

f8 f9

f12 f13

DE rand/1/bin [21] 0.0 0.0 0.0 0.0

2.02 × 10−9 8.26 × 10−10 3.85 × 10−8 9.17 × 10−9 0.0 0.0 0.0 0.0 4.939 × 10−3 1.13 × 10−3

−1.256948 × 10+4 2.3 × 10−4 0.0 0.0 −1.19 × 10−15 7.03 × 10−16 0.0 0.0 0.0 0.0 −1.142824 4.45 × 10−8 0.0 0.0 0.0 0.0

5.87 × 10−10 1.83 × 10−10 1.128 × 10−9 1.42 × 10−10 0.0 0.0 0.0 0.0

7.664 × 10−3 6.58 × 10−4

−4.1898 × 10+4 1.06 × 10−3 0.0 0.0 8.023 × 10−15 1.74 × 10−15 5.42 × 10−20 5.42 × 10−20 0.0 0.0 −1.142824 2.74 × 10−8

4 Results In order to understand the search capability of i-CSA inside the landscape, and also its ability to escape from a local optima, we have used the classical benchmark functions proposed in [7]. In particular, making a proper comparisons on the same functions, we have reduced our experiments on the first 13 functions of this benchmark, which belong to two different categories: unimodal and multimodal with many local search. We will indicate all used functions as (f1 , . . . , f13 ), in the same order as proposed in [7].

An Information-Theoretic Approach for Clonal Selection Algorithms

153

Table 3. Comparison between IA, i-CSA, and several DE variants. The mean of the best individuals on all runs are presented for all algorithms, and the best results are highlighted in boldface. Results have been averaged over 100 independent runs, using Tmax = 1.2 × 105 , and n = 30 dimensional search space. For i-CSA was fixed d = 100. f1 i-CSA 0.0 IA 0.0 DE rand/1/bin 0.0 DE rand/1/exp 0.0 DE best/1/bin 0.0 DE best/1/exp 407.972 DE current-to-best/1 0.54148 DE current-to-rand/1 0.69966 DE current-to-rand/1/bin 0.0 DE rand/2/dir 0.0

i-CSA IA DE rand/1/bin DE rand/1/exp DE best/1/bin DE best/1/exp DE current-to-best/1 DE current-to-rand/1 DE current-to-rand/1/bin DE rand/2/dir

f5 16.2 11.69 19.578 6.696 30.39087 132621.5 30.984666 31.702063 24.260535 30.654916

Unimodal Functions f2 f3 0.0 0.0 0.0 0.0 0.0 0.02 0.0 0.0 0.0 0.0 3.291 10.6078 4.842 0.471730 3.503 0.903563 0.0 0.000232 0.0 30.112881 Multimodal Functions f9 f10 0.0 0.0 0.0 0.0 0.0 0.0 97.753938 0.080037 0.0 0.0 40.003971 9.3961 98.205432 0.270788 92.263070 0.164786 0.0 0.0 0.0 0.0

f4 f6 f7 0.0 0.0 2.79 × 10−5 0.0 0.0 4.89 × 10−5 1.9521 0.0 0.0 3.7584 0.84 0.0 0.0017 0.0 0.0 1.701872 2737.8458 0.070545 4.2337 1.394 0.0 3.298563 1.767 0.0 0.149514 0.0 0.0 0.044199 0.0 0.0 f11 0.0 0.0 0.001117 0.000075 0.000722 5.9278 0.219391 0.184920 0.0 0.0

f12 0.0 0.0 0.0 0.0 0.0 1293.0262 0.891301 0.464829 0.001007 0.0

f13 0.0 0.0 0.0 0.0 0.000226 2584.85 0.038622 5.169196 0.000114 0.0

Nowadays, in literature there are many evolutionary methodologies able to tackle effectively the global numerical function optimization. Among such algorithms, differential evolution (DE) has shown better performances than other evolutionary algorithms on complex and continuous search space [23,24]. Therefore, in this section we present the comparisons of i-CSA, not only with its original version, but also with several variants of the DE published in [25,21]. For simplicity we will use IA to indicate the algorithm proposed in [8,9]. As first experiment, IA and i-CSA have been compared each other and with one of the best variants of DE, called rand/1/bin. The results are showed in table 2, and for this comparison was used the same experimental protocol adopted in [21,8]. We pick as reference results for DE, the experiments carried out in [21]. Two different dimensions were used by fixing n ∈ {30, 100}, with a maximum number of fitness function evaluations Tmax = 5 × 105 , and Tmax = 5 × 106 , respectively. For each function we present the mean of the best B cells on all runs, and, in a new line, the standard deviation. As made in [21], all results below 10−25 obtained by IA and rand/1/bin were reported as 0.0; this instead was not considered for i-CSA. From this comparison, it is possible to note that the new variant improves effectively the results, primarily in the function f5 with n = 30, where i-CSA is able to reach the best solution (0.0), where instead the previous variant IA failed. Further, it is clear that the overall performances of i-CSA are comparable on 30 dimension, instead the algorithm outperforms the ones produced by rand/1/bin DE variant on 100 dimension variables ((f3 , f4 , f7 , f10 , f11 ) versus (f5 , f8 , f13 )), which are among the most complex functions. In table 3, IA and

154

V. Cutello et al.

Table 4. Comparison between i-CSA and several Swarm Intelligence algorithms. For each function is showed the mean and standard deviation (in a new line), obtained on 30 independent runs. The best results have been highlighted in boldface.

GA PSO PS-EA i-CSA ABC1 ABC2

g1

g2

0.050228 0.029523 0.079393 0.033451 0.222366 0.0781 0.0 0.0 0.00087 0.002535

1.3928 0.76319 2.6559 1.3896 0.43404 0.2551 0.0 0.0 0.0 0.0

0.000329 0.00185

0.0 0.0

1.0139 0.026966 PSO 0.030565 0.025419 PS-EA 0.59036 0.2030 i-CSA 0.0 0.0 ABC1 2.01 × 10−8 6.76 × 10−8 ABC2 0.0 0.0 GA

1.2342 0.11045 PSO 0.011151 0.014209 PS-EA 0.8211 0.1394 i-CSA 0.0 0.0 ABC1 2.87 × 10−9 8.45 × 10−10 ABC2 0.0 0.0 GA

g3 g4 g5 10 variables 46.3184 0.59267 1.9519 33.8217 0.22482 1.3044 4.3713 9.8499 × 10−13 161.87 2.3811 9.6202 × 10−13 144.16 25.303 0.19209 0.32037 29.7964 0.1951 1.6185 0.0 0.0 1.27 × 10−4 0.0 0.0 1.268 × 10−14 0.034072 7.8 × 10−11 1.27 × 10−9 0.045553 1.16 × 10−9 4 × 10−12

0.012522 0.01263 20 variables 6.0309 103.93 1.4537 29.505 12.059 77.382 3.3216 94.901 1.8135 72.452 0.2551 27.3441 0.0 0.0 0.0 0.0 −8 1.45 × 10 0.13614 5.06 × 10−8 0.132013 0.0 0.014458 0.0 0.010933 30 variables 10.4388 166.283 2.6386 59.5102 32.476 402.54 6.9521 633.65 3.0527 98.407 0.9985 35.5791 0.0 0.0 0.0 0.0 0.033874 0.219626 0.181557 0.152742 0.0 0.020121 0.0 0.021846

4.6 × 10−11 5.4 × 10−11

1.27 × 10−9 4 × 10−12

0.92413 0.22599 1.1778 × 10−6 1.5842 × 10−6 0.32321 0.097353 0.0 0.0 1.6 × 10−11 1.9 × 10−11 0.0 1 × 10−12

7.285 2.9971 543.07 360.22 1.4984 0.84612 237.5652 710.4036 19.83971 45.12342 0.000255 0

1.0989 0.24956 1.4917 × 10−6 1.8612 × 10−6 0.3771 0.098762 0.0 0.0 3 × 10−12 5 × 10−12 0.0 0.0

13.5346 4.9534 990.77 581.14 3.272 1.6185 2766.804 2176.288 146.8568 82.3144 0.000382 1 × 10−12

i-CSA are compared with 8 DE variants, proposed in [25], where Tmax was fixed to 1.2 × 105 [25,9]. For each function 100 independent runs were performed, and the variables dimension was fixed to 30. Because in [25], the authors have modified the function f8 to have its minimum at zero (rather than −12569.5), it is not included into the table. Looking the comparison in such table, we can see that the variant of the proposed IA outperform the several DE variants in all functions, except only for the functions f5 and f7 . Recently, immune systems have been related to swarm systems, since many immune algorithms operate in a very similar manner. Thus, we have compared i-CSA with some swarm intelligence algorithms, proposed in [10]: particle swam optimization (PSO); particle swarm inspired evolutionary algorithm (PS-EA), and artificial bee colony.

An Information-Theoretic Approach for Clonal Selection Algorithms

155

For this kind of experiments we have used the same experimental protocol used in [10]; we consider problems of various dimension ranging in n = {10, 20, 30}; the termination criterion was fixed to 500, 750, and 1000 generations, respectively. Only five functions were taken into account for these experiments, labelled (g1 , . . . , g5 ), where the first four are included in the same benchmark cited above which corresponding exactly: g1 = f11 , g2 = f9 , g3 = f5 , and g4 = f10 1 . From table 4, it is possible to affirm that i-CSA all swarm system algorithms on all functions, except on the function g5 . The best performances of i-CSA are confirmed also for an increasing problem dimension. Conversely, for the function g5 , PS-EA reaches best solutions respect increasing problem dimension. Finally, the results reported for ABC2 were obtained using a different experimental protocol: the termination criterion was increased to 1000, 1500 and 2000, respectively. Although i-CSA has been tested with a smaller number of generations is possible to notice that the results are comparable, and often outperform ABC2 . This experimental evidence show as i-CSA reaches competitive solutions, near to global optima, in less time than artificial bee colony algorithm.

5 Conclusions Global numerical optimization has been taken into account to prove the effectiveness of a derivative-free clonal selection algorithm, called i-CSA. The main features of i-CSA can be summarized as: (1) the cloning operator, which explores the neighbourhood of a given solution, (2) the inversely proportional hypermutation operator, that perturbs each candidate solution as a function of its fitness function value (inversely proportionally), and (3) the aging operator, that eliminates the oldest candidate solutions from the current population in order to introduce diversity and thus avoiding local minima during the search process. Since any B cell receptor has a limited life span into the population becomes crucial the decision of the age to assign to each B cell receptor. Therefore, the strategy adopted to assign such value to the clones may affect the quality of the search inside the landscape of the given problem and, hence, the capability to escape from a local optima. In this work we present a simple variant of the CSA proposed in [8,9], which is able to effectively improve its performances allowing a longer maturation of any B cell. A large set of classical numerical functions was taken into account from [7], which is divided into two different categories: unimodal and multimodal (with many local optima) functions. i-CSA was compared with several variants of the DE algorithm, since it has been shown to be effective on many optimization problems. Afterwards, because nowadays immunological systems are considered similar to swarm systems, we have compared i-CSA with state-of-the-art swarm algorithms. The analysis of the results shows that i-CSA is comparable, and often outperforms, all nature-inspired algorithms in terms of accuracy to the accuracy, and effectiveness in solving large-scale instances. By analyzing one of the most difficult function of the benchmark, (f5 ), we characterize the learning capability of i-CSA. The obtained gain has been analyzed both with respect the initial distribution, and the ones obtained in the previous step. For this study, three different entropic metrics were used, as Kullback-Leibler, R´enyi and von Neumann 1

For details on these five functions see [10].

156

V. Cutello et al.

divergences [16,18,19]. By the relative curves, is possible to observe a strong correlation between optima (peaks in the search landscape) discovering, and high relative entropy values.

References 1. Timmis, J., Hone, A., Stibor, T., Clark, E.: Theoretical advances in artificial immune systems. Theoretical Computer Science 403(1), 11–32 (2008) 2. Smith, S., Timmis, J.: An Immune Network Inspired Evolutionary Algorithm for the Diagnosis of Parkinsons Disease. Biosystems 94(1-2), 34–46 (2008) 3. Timmis, J., Hart, E., Hone, A., Neal, M., Robins, A., Stepney, S., Tyrrell, A.: ImmunoEngineering. In: Proc. of the international conference on Biologically Inspired Collaborative Computing (IFIP 2009), vol. 268, pp. 3–17. IEEE Press, Los Alamitos (2008) 4. Dasgupta, D., Ni˜no, F.: Immunological Computation: Theory and Applications. CRC Press, Boca Raton (in press) 5. Cutello, V., Nicosia, G., Pavone, M., Timmis, J.: An Immune Algorithm for Protein Structure Prediction on Lattice Models. IEEE Trans. on Evolutionary Computation 11(1), 101–117 (2007) 6. Cutello, V., Nicosia, G., Pavone, M.: An immune algorithm with stochastic aging and kullback entropy for the chromatic number problem. Journal of Combinatorial Optimization 14(1), 9–33 (2007) 7. Yao, X., Liu, Y., Lin, G.M.: Evolutionary programming made faster. IEEE Trans. on Evolutionary Computation 3(2), 82–102 (1999) 8. Cutello, V., Nicosia, G., Pavone, M., Narzisi, G.: Real Coded Clonal Selection Algorithm for Unconstrained Global Numerical Optimization using a Hybrid Inversely Proportional Hypermutation Operator. In: Proc. of the 21st Annual ACM Symposium on Applied Computing (SAC 2006), vol. 2, pp. 950–954 (2006) 9. Cutello, V., Krasnogor, N., Nicosia, G., Pavone, M.: Immune Algorithm versus Differential Evolution: A Comparative Case Study Using High Dimensional Function Optimization. In: Beliczynski, B., Dzielinski, A., Iwanowski, M., Ribeiro, B. (eds.) ICANNGA 2007. LNCS, vol. 4431, pp. 93–101. Springer, Heidelberg (2007) 10. Karaboga, D., Baturk, B.: A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. Journal of Global Optimization 39, 459–471 (2007) 11. Castrogiovanni, M., Nicosia, G., Rascun´a, R.: Experimental Analysis of the Aging Operator for Static and Dynamic Optimisation Problems. In: Apolloni, B., Howlett, R.J., Jain, L. (eds.) KES 2007, Part III. LNCS (LNAI), vol. 4694, pp. 804–811. Springer, Heidelberg (2007) 12. Cutello, V., Nicosia, G., Pavone, M.: Exploring the capability of immune algorithms: a characterization of hypermutation operators. In: Nicosia, G., Cutello, V., Bentley, P.J., Timmis, J. (eds.) ICARIS 2004. LNCS, vol. 3239, pp. 263–276. Springer, Heidelberg (2004) 13. Cutello, V., Nicosia, G., Pavone, M.: An Immune Algorithm with Hyper-Macromutations for the Dill’s 2D Hydrophobic-Hydrophilic Model. In: Proc. of Congress on Evolutionary Computation (CEC 2004), vol. 1, pp. 1074–1080. IEEE Press, Los Alamitos (2004) 14. Cutello, V., Nicosia, G., Pavone, M.: A Hybrid Immune Algorithm with Information Gain for the Graph Coloring Problem. In: Cant´u-Paz, E., Foster, J.A., Deb, K., Davis, L., Roy, R., O’Reilly, U.-M., Beyer, H.-G., Kendall, G., Wilson, S.W., Harman, M., Wegener, J., Dasgupta, D., Potter, M.A., Schultz, A., Dowsland, K.A., Jonoska, N., Miller, J., Standish, R.K. (eds.) GECCO 2003. LNCS, vol. 2723, pp. 171–182. Springer, Heidelberg (2003)

An Information-Theoretic Approach for Clonal Selection Algorithms

157

15. Shannon, C.E.: A Mathematical Theory of Communication. In: Congress on Evolutionary Computation, vol. 1, pp. 1074–1080. IEEE Press, Los Alamitos (2004); Bell System Technical Journal 27, 379–423, 623–656 (1948) 16. Kullback, S.: Statistics and Information Theory. J. Wiley and Sons, New York (1959) 17. Jaynes, E.: Probability Theory: The Logic of Science. Cambridge University Press, Cambridge (2003) 18. R´enyi, A.: On measures of information and entropy. In: Proc. of the 4th Berkeley Symposium on Mathematics, Statistics and Probability, pp. 547–561 (1961) 19. Kopp, A., Jia, X., Chakravarty, S.: Replacing energy by von Neumann entropy in quantum phase transitions. Annals of Physics 322(6), 1466–1476 (2007) 20. Cutello, V., Narzisi, G., Nicosia, G., Pavone, M.: Clonal Selection Algorithms: A Comparative Case Study using Effective Mutation Potentials. In: Jacob, C., Pilat, M.L., Bentley, P.J., Timmis, J.I. (eds.) ICARIS 2005. LNCS, vol. 3627, pp. 13–28. Springer, Heidelberg (2005) 21. Versterstrøom, J., Thomsen, R.: A Comparative Study of Differential Evolution, Particle Swarm Optimization, and Evolutionary Algorithms on Numerical Benchmark Problems. In: Congress on Evolutionary Computing (CEC 2004), vol. 1, pp. 1980–1987 (2004) 22. Noman, N., Iba, H.: Enhancing Differential Evolution Performance with Local Search for High Dimensional Function Optimization. In: Genetic and Evolutionary Computation Conference (GECCO 2005), pp. 967–974 (2005) 23. Storn, R., Price, K.V.: Differential Evolution a Simple and Efficient Heuristic for Global Optimization over Continuous Spaces. Journal of Global Optimization 11(4), 341–359 (1997) 24. Price, K.V., Storn, M., Lampien, J.A.: Differential Evolution: A Practical Approach to Global Optimization. Springer, Heidelberg (2005) 25. Mezura–Montes, E., Vel´azquez–Reyes, J., Coello Coello, C.: A Comparative Study of Differential Evolution Variants for Global Optimization. In: Genetic and Evolutionary Computation Conference (GECCO 2006), vol. 1, pp. 485–492 (2006)

Antibodies with Adaptive Radius as Prototypes of High-Dimensional Datasets Ricardo P.V. Violato1,2 , Alisson G. Azzolini1 , and Fernando J. Von Zuben1 1

School of Electrical and Computer Engineering - University of Campinas (Unicamp) P.O. Box 6101, CEP: 13083-970, Campinas, Brazil {azzolini,violato,vonzuben}@dca.fee.unicamp.br http://www.fee.unicamp.br/ 2 CPqD, Telecommunications Research Center CEP: 13086-902, Campinas, Brazil [email protected] http://www.cpqd.com.br/

Abstract. An adaptive radius immune algorithm proposed in the literature, denoted as ARIA, is claimed to preserve the density distribution of the original dataset when generating prototypes. Density-preserving prototypes may correspond to high-quality compact representations for clustering applications. The original samples in the dataset are interpreted as antigens, and the prototypes are interpreted as antibodies. In this paper, some theoretical results are provided to demonstrate that the original version of ARIA is not capable of generating density-preserving prototypes when high-dimensional datasets are considered. Further, the same theoretical results are explored to conceive a new version of ARIA, now capable of exhibiting the announced density-preserving attribute. The main innovation is in the way the algorithm estimates local densities. Keywords: Immune-inspired algorithm, adaptive radius prototypes, density estimation.

1

Introduction

The recognition capability of immune-inspired algorithms can be directly explored in the stage of preprocessing datasets for clustering purposes [8,11]. The antibodies will serve as prototypes, which are expected to populate accordingly the regions of the data space occupied by available samples, interpreted as antigens. Thus, the artificial immune system simply reacts to the presence of antigens, generating antibodies with high affinity to the antigens. The resulting number of antibodies depends on the distribution of antigens and on the radius of recognition associated with each antibody. When the radius of each antibody is self-adjusted by the immune-inspired algorithm, then the algorithm will have better condition to deal with clusters characterized by distinct densities. More concentrated antigens will require antibodies with smaller radius, E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 158–170, 2010. c Springer-Verlag Berlin Heidelberg 2010 

Adaptive Radius Algorithm for High-Dimensional Datasets

159

and vice-versa. However, the degree of concentration depends on the dimensionality of the data space, and it is precisely this aspect that will be deeply explored in this paper. After convergence, the antibodies obtained are called prototypes, given that they represent a summary of the original data distribution. This summary tends to be more interpretable and more amenable to inspection for the presence of clusters [9,10]. A frequently applied immune-inspired algorithm for data clustering is aiNet (artificial immune network) [1], characterized by antibodies with identical and fixed radius. The radius is an input parameter to be defined by the user. In [3], the authors showed that the data reduction process implemented in aiNet leads to an inadequate representation of data, when the distribution of the clusters is not approximately uniform. Previously to these evidences concerning the limitations of fixed-radius antibodies, an adaptive-radius immune-inspired algorithm had already been proposed in the literature and denoted ARIA [2]. ARIA attempts to maximally preserve the density information during the compression step, supposedly allowing a suitable identification of data clusters. Density-preserving prototypes may correspond to high-quality compact representations for clustering applications. In a few words, both aiNet and ARIA algorithms try to find a reduced set of antibodies, called prototypes, that closely represents the input dataset (viewed as antigens). After that, a partitioning or visualization technique can be applied to the resulting arrangement, separating the prototypes into clusters. One issue that arises from these techniques is how to measure the quality of the compressed representation. An approach aiming at measuring the similarity between the input dataset and the reduced set (prototypes) was proposed by Fugunaka and Hayes [4]. They suggested to estimate the density of data and prototypes using a non-parametric model (Parzen Window), and then calculate the relative entropy (Kullback-Leibler divergence) between the input dataset and the reduced set of prototypes. However, Parzen window is not the only non-parametric way to estimate probability densities. The k-nearest neighbors (KNN) method can be an attractive alternative. In this work we verify that the capability of ARIA to preserve density does not hold for high dimensional data, and therefore we propose a theoretically supported modification in its density estimation procedure. Practical experiments using controlled datasets indicate that the new version of ARIA is now capable of preserving in the prototypes the original distribution of the dataset. The paper is organized as follows: section 2 introduces the compression quality evaluation method adopted here; section 3 addresses the density estimation problem in the original version of ARIA, supported by a simple case study; section 4 presents a modification in the ARIA density estimation procedure, founded on conceptual results; section 5 outlines some experimental results, and the final remarks and steps for further research are in section 6.

160

2

R.P.V. Violato, A.G. Azzolini, and F.J. Von Zuben

Evaluation of the Compression Quality

Many algorithms that perform data clustering work firstly trying to find a reduced set of prototypes that closely represents the input dataset, or in other words, find a compressed representation that will serve as input to the next stages of the clustering analysis. This is the approach adopted, for example, by aiNet [1], ARIA [2], and other immune-inspired algorithms. In [3], the authors evaluated the compression quality of the aiNet. They applied the measure proposed in [4], based on an approximated form of the relative entropy (Kullback-Leibler divergence) to measure the similarity between two probability density functions, which are non-parametrically estimated from the original (input) data distribution and its reduced version, obtained with aiNet. Suppose that the probability density function pN (x) (which generates the input dataset) and pM (x) (which generates the reference vectors) are known. Then it is possible to measure the dissimilarity between these two data distributions using the relative entropy, as suggested in [4] and given by eq. 1.    pM (x) H = ln · pN (x)dx ≤ 0 (1) pN (x) The functions pN (x) and pM (x) are usually unknown and hence they must be estimated. There are two classical non-parametric approaches to density estimation [5]: the k-nearest neighbour (KNN) method and the kernel estimator. The KNN estimator is defined as: k pˆ(x) = (2) cl [dk (x)]l where dk (x) is the distance from x to its k-nearest neighbour in a dataset sampled from the distribution p, and cl is the volume of the unit-sphere in Rl . Since in this work we only compare densities, cl is taken to be 1. The advantage of the KNN method is that k is easier to determine than the kernel’s bandwidth (see below), and the smoothness and integrability of the estimator are not crucial in the approximation of entropy shown in eq. 6 below. The kernel estimator (also called Parzen window) is defined as: pˆ(x) =

N 1  K(x − xi ) N i=1

A kernel function K must satisfy the condition: ∞ K(x)dx = 1

(3)

(4)

−∞

In this work the kernel function employed is, for comparison reasons (see section 5), the multivariate Gaussian function:   2 1 x exp − 2 K(x) = (5) 2h (2π)l/2 hl

Adaptive Radius Algorithm for High-Dimensional Datasets

161

The window width (also called bandwidth) h controls the smoothness, i.e. the influence of the surrounding points (xi or mi ), whereas the kernel function K(·) determines the shape [5]. In order to discretize and simplify eq. 1, one can take its expectation with respect to pˆN (x) (replacing the integral by a summation) and substitute pˆN (xi ) by 1/N , thus obtaining an approximation of the relative entropy (6). In [4], they ˆ ≤ 0 is no longer guaranteed, suggested this approximation and noticed that H but reported good experimental results. N  pˆM (xi ) ˆ = 1 H ln N i=1 pˆN (xi )

(6)

In their work, Stibor and Timmis [3] reported that the aiNet gives poor results when the data distribution is not nearly uniform. The ARIA algorithm was proposed to overcome this problem. Here, the same metodology will be used to evaluate the performance of ARIA, whose main purpose is to place the prototypes in a way that preserves the relative density of the data distribution, i.e., it tries to place more prototypes where there is a higher concentration of data points and fewer prototypes where distribution is sparser. Finally, note that, since we are using a non-parametric density estimation, every prototype is considered to have the same weight. Hence, in order to preserve the density, each prototype has to represent, in average, the same number of data points. If a semi-parametric model were used instead, this requirement would no longer apply, because each prototype could be assigned a different weight and shape. Nevertheless, we restrict ourselves to the non-parametric case, since it is used in [3] and it is the way ARIA works.

3

Density Estimation Problem - An Illustrative Example

Consider an l-dimensional dataset sampled from two Gaussian distributions, with l = 26. The center of the first Gaussian is μ1 = (0, 0 · · · 0) and the center of the second is μ2 = (10, 10 · · · 10). One has twice the standard deviation of the other, σ1 = 1 and σ2 = 0.5. The same number of points N = 1000 is sampled from each one, forming clusters 1 and 2, respectively. Figure 1 shows this data points projected in the first two dimensions. As the distributions are simetrical in all dimensions, any projection would give a similar result. Note that the clusters are well separated, so it is easy to discriminate the prototypes belonging to each one of them. The volume of a hypersphere of radius r in the l-dimensional space is proportional to the l − th power of r [6]: V (r) = cl · rl

(7)

The estimated density ρ around a certain point x is given by: ρ(x) =

k V (r)

(8)

162

R.P.V. Violato, A.G. Azzolini, and F.J. Von Zuben 15

x2

10

5

0

−5 −5

0

5 x1

10

15

Fig. 1. Two well defined data clusters, generated from two Gaussian distributions sampled 1000 times each

where k is the number of samples inside a hypersphere of radius r centered in x and V (r) is the volume of this hypersphere. In order to find the ratio of density at two different points x1 and x2 , we can proceed in two ways: (i) vary r in order to fix the number of points k inside each hypersphere (the KNN method) or (ii) keep r constant and count the number of points k1 and k2 inside each hypersphere (the histogram method). 1) Let us estimate the ratio of density ρρ12 (μ (μ2 ) in the center of the two clusters defined above, using method (i). We place a hypersphere of radius r1 in the center of the cluster 1, and a hypersphere of radius r2 in the center of cluster 2.  If we choose r1 and r2 proportional to their respective standard deviations r1 σ1 r2 ≡ σ2 = 2 , we will have the same expected number of points k1 = k2 = k inside each hypersphere. Then, using eqs. 7 and 8, we get: ρ1 (μ1 ) k V (r2 ) k cl r2l = = = ρ2 (μ2 ) V (r1 ) k cl r1l k



r2 r1

l

= 2−26

(9)

However, when we try to use method (ii), using the same radius r in the two hyperspheres and counting k1 and k2 , we get: k1 V (r) ρ1 (μ1 ) k1 = = ρ2 (μ2 ) V (r) k2 k2

(10)

1) Replacing the value of ρρ12 (μ (μ2 ) obtained in eq. 9 in the equation above, we should have k2 = 226 k1 in order to correctly estimate the density ratio. Thus, a problem arises: an extremely large dataset (more then 60 million points) would be needed in order to obtain a reasonable result. In the ARIA algorithm, the radius of each antibody (prototype) is used as a suppression threshold. It is defined in a way that the volume of the hypersphere of such radius is inversely proportional to the local density. The density

Adaptive Radius Algorithm for High-Dimensional Datasets

163

is estimated by method (ii), counting the number of antigen (data points) in its neighborhood, as described by eq. 8. The neighborhood is defined as a hypersphere of radius E, where E is the average value of all antibody’s radii at that iteration. The radius of an antibody i is given by eq. 11, where ts is a parameter that determines the smallest radius an antibody can have (a parameter of the algorithm), ρi is its estimated density, ρmax is the highest density of an antibody at that iteration and l is the data dimension.

1 ρmax l Ri = ts (11) ρi Since ARIA uses method (ii), it suffers from the limitation shown above. In order to illustrate this, we suppose that there are two antibodies representing the dataset in the example above, each one located at the center of one cluster. We want to find out the data density in their vicinity, according to eq. 8, counting points inside hyperspheres of radius E around each prototype. Table 1 show the results for different values of E. Table 1. Number of antigens k1 and k2 inside a hypersphere of radius E, centered in clusters 1 and 2, respectively E 1.0 1.6 1.7 3.0 3.2 3.7 4.9 7.3

k1 0 0 0 0 2 29 417 1000

k2 0 3 7 897 965 1000 1000 1000

Suppose now that we use E = 3.2. From the Table, we have: ρ2 n2 965 ∼ 9 = = =2 ρ1 n1 2

(12)

Replacing this result in eq. 11, we get the ratio of radii between the antibodies: R1 = R2



ρ2 ρ1

 261

9 1 ∼ = 2 26 ∼ = 23

(13)

1 = 2. That is why ARIA indicates usually when the correct value should be R R2 only one antibody representing the denser distribution and many antibodies (the number depends on the parameters set used) representing the sparser cluster (see section 5). To obtain a prototype distribution that preserves the density information the result should be the same number of prototypes representing each cluster.

164

R.P.V. Violato, A.G. Azzolini, and F.J. Von Zuben

Note also from Table 1 that there are some values of ki that are 0. This fact would lead to a division by zero, when trying to calculate the antibody radius. Anyone that implements ARIA faces this problem and the simplest solution is to replace 0 by 1. This is an explicit evidence of the incapacity of the algorithm to estimate density when the ratio of densities is much greater than the number of samples available. In applications involving low dimensional data spaces, this problem is generally absent. But in high dimensional cases, even a small difference in variance leads to a great change in density. And the original version of ARIA cannot deal with this practical aspect.

4

Proposed Modification

This limitation of the algorithm when trying to preserve density was already identified in [7], where ARIA was applied to a vector quantization task for speech processing. In that work, the authors introduce a constant κ in the original formula to calculate de radius Ri of an antibody i:

Ri = ts

ρmax ρi

 κl

The ratio between two radius becomes:

 κl ρ2 R1 = R2 ρ1

(14)

(15)

In particular, for the example presented in section 3, if we use κ = 3 and eq. 12, we have: 3 27 R1 = 29 26 = 2 26 ∼ (16) =2, R2 which is the correct value in order to preserve density. Experimental results using this formula with κ = 3 do indeed produce nearly the same number of prototypes representing the two clusters. Though this scheme has worked well in this example, it introduces a new parameter that depends strongly on the data distribution. For more complex datasets, the constant κ cannot be analitically determined. One can argue that this procedure also improved ARIA performance for a much more complex dataset, as reported in [7]. But that is only because a certain value of κ produced an average better estimation of the radii. Actually, the constant κ should vary to correctly estimate the radii, because this strategy is only a way to mask the inappropriate density estimation method used (the counting of data points or histogram method). A real solution to the problem requires a different method to estimate local densities. Two well known methods are the Parzen window [4] and k-nearest neighbour [5], which were already described in section 2. Both have approximately the same execution cost of the histogram method and also involve a parameter that depends on the data distribution, as the histogram method has (radius E of the hypersphere), but they do not suffer from the data scarcity problem.

Adaptive Radius Algorithm for High-Dimensional Datasets

165

The k-nearest neighbour method (KNN) has the parameter k, and the density at a given point (prototype) is estimated using the volume of the hypersphere with radius equal to the distance from that point to the k -th nearest neighbour (data point). That is, eq. 8 is still used, but with a given k instead of a given V . Because of this straightfoward adaptation of the algorithm and because of the direct interpretation of the procedure, we decided to use de KNN method for estimating local densities in ARIA. Calling di the distance from a given antibody i to its k -th nearest antigen, the local density of this antibody i is obtained from eqs. 7 and 8, and is defined as: ρi =

k k = l Vi di

(17)

The constant k is the same for every antibody and remains the same along all iterations. The remaining of the algorithm is kept unmodified, including the formula to calculate the antibody’s radius (see eq. 11).

5 5.1

Tests Datasets

First we test the proposed modification with a dataset similar to the one described in section 3. We generate 1200 points from two Gaussian distributions, varying the number of points sampled from each one and their variances. The first Gaussian distribution, named cluster 1, is centered in μi = 0, i = 1, . . . , 26, and the second, named cluster 2, is centered in μi = 10, i = 1, . . . , 26. Table 2 describes the test sets and three of them are ploted in Figure 2. Since the clusters are well separeted, it is easy to count the number of prototypes representing each one of them. Note that, since we are using a non-parametric model for density estimation, the number of prototypes generated over each cluster should be proportional to the number of data points in the respective cluster, in order to preserve density. On the other hand, if we had adopted a semi-parametric model with Gaussian kernel, two prototypes with different weights and deviations would be sufficient to represent all information. Although these datasets are quite trivial, they illustrate the deficiency of the original algorithm and they attest the density preservation capability of the new proposal. For comparison purpose, we tested the original ARIA against the modified version suggested here, considering the datasets used in [3] to evaluate aiNet. It consists of four different sets, each one containing 400 points, sampled from different probability distributions, all of them in two dimensions. Briefly, three sets are generated from a mixture of Gaussian distributions and one of them is calculated from a sine/cosine function plus an additive Gaussian noise. In [3] the reader can find a more detailed description, the R source code [12] used to generate the datasets and also figures with the data points ploted. In the results to be outlined in the next section, these datasets will be referenced as datasets 10 to 13.

166

R.P.V. Violato, A.G. Azzolini, and F.J. Von Zuben Table 2. Description of the datasets used

set set set set set set set set set

1 2 3 4 5 6 7 8 9

total 1200 1200 1200 1200 1200 1200 1200 1200 1200

N. of samples Standard deviation cluster 1 cluster 2 cluster 1 cluster 2 600 (50%) 600 (50%) 0.6 1.2 600 (50%) 600 (50%) 0.4 1.2 600 (50%) 600 (50%) 0.3 1.2 400 (33.33%) 800 (66.67%) 0.6 1.2 400 (33.33%) 800 (66.67%) 0.4 1.2 400 (33.33%) 800 (66.67%) 0.3 1.2 300 (25%) 900 (75%) 1.2 0.6 300 (25%) 900 (75%) 1.2 0.4 300 (25%) 900 (75%) 1.2 0.3

Data Set 1

15 10 x2

Data Set 2

15 10

10

5

5

5

0

0

0

−5 −5

0

5

10

15

−5 −5

Data Set 3

15

0

5 x1

10

15

−5 −5

0

5

10

15

Fig. 2. Some datasets projected in two dimensions

Besides those artificial datasets, the proposed modification was also tested for a real data distribution. These dataset was introduced in [7] and was generated by a parametrization process over speech signals. Briefly, a recorded speech database consisting of 50 sentences (in Portuguese), was first divided into 28146 frames and windowed. Then, a set of 26 attributes was calculated for each one of them. 5.2

Results

The results obtained for the datasets with two Gaussian clusters are summarized in Tables 3 and 4. A total number of 10 independent executions of the algorithms were performed and the average values are presented. The algorithm was configured as follows: the number of iterations was set to 50, the initial mutation rate to 1, the mutation rate decay to 0.9, starting at the second generation. The initial population was formed by a single antibody (prototype), corresponding to a data point chosen randomly from the antigens population (dataset). The radius multiplier (ts) was set in a way that both original and modified version of the algorithm produced approximately the same number of prototypes. Their values are presented in the corresponding tables. For the modified version (see section 4), the KNN method was used to estimate local densities with k = 10 (see eq. 17). Table 3 summarizes the resulting prototypes alocation and their radii: M is the average population size at the end of the algorithm execution, m1 and m2

Adaptive Radius Algorithm for High-Dimensional Datasets

167

are the average number of prototypes representing clusters 1 and 2 respectively, and R1 and R2 are the average radius of prototypes representing clusters 1 and 2, respectively. As we can see, the modification introduced in the algorithm was capable of increasing its performance significantly. The prototype’s distribution is the same of the data and their radii are also proportional to the standard deviations of the data, and thus proportional to local density. In the original formulation, the radii were able to indicate that a region is denser than the other (after all, prototypes on that region present smaller radii), but they are not proportional to local density. Moreover the prototypes distributions are completely distorted in all tested cases. The results for all datasets were qualitatively the same. Thus, aiming at a clear presentation, we report here only three of them. Table 3. Obtained results with the datasets 1, 5 and 9 (well separated clusters) Dataset 1 5 9 Density Original Proposed Original Proposed Original Proposed ts 2.1 1.5 2,0 1,0 1,15 0,7 M 163.2 174.5 193,5 182,6 191 185,2 m1 (%) 1 (0,61) 88.7 (50,82) 1 (0,52) 61,2 (33,60) 190 (99,47) 45,9 (24,79) m2 (%) 162.2 (99,39) 85.8 (49,18) 192,5 (99,48) 121,4 (66,40) 1 (0,53) 139,3 (75,21) R1 2.100 1.701 2,000 1,137 1,439 3,575 R2 2.432 3.357 2,514 3,236 1,150 0,832

Given these results, it is expected that the relative entropy (eq. 6) will also present better results for the proposed modification. Table 4 shows the mean absolute values of the relative entropy, since it could be positive or negative (because of the approximations assumed in eq. 6). The closer to zero, the better the result. The estimated densities pˆ(x) were calculated with the KNN method described in section 2 for different values of k. For all cases, the distribution obtained with the proposed modification presented a relative entropy value smaller than the one of the original algorithm. Table 4. Relative entropy results with the datasets 1, 5 and 9, using the KNN method with various values of k Dataset k 2 5 10 15 20

1 Original Proposed 36.18 ± 0.05 0.63 ± 0.04 35.83 ± 0.05 0.19 ± 0.03 35.45 ± 0.04 0.02 ± 0.02 35.22 ± 0.02 0.10 ± 0.04 35.02 ± 0.02 0.19 ± 0.04

5 Original Proposed 27.28 ± 0.05 0.62 ± 0.06 27.21 ± 0.03 0.16 ± 0.05 26.95 ± 0.03 0.05 ± 0.03 26.79 ± 0.04 0.06 ± 0.05 26.69 ± 0.04 0.18 ± 0.05

9 Original Proposed 68.7 ± 0.1 0.62 ± 0.05 68.01 ± 0.05 0.16 ± 0.04 67.37 ± 0.04 0.03 ± 0.02 66.97 ± 0.03 0.10 ± 0.04 66.68 ± 0.03 0.20 ± 0.05

The results obtained for datasets 10 to 13 are presented in Table 6. For these datasets, the relative entropy was calculated using the Parzen window method (section 2) and the bandwidth h was set to 0.96 (that is the value used in [3]). We

168

R.P.V. Violato, A.G. Azzolini, and F.J. Von Zuben

Table 5. Relative entropy results with the speech dataset, using the KNN method with various values of k M k=5 k = 10 k = 20 k = 50 k = 100

Original 125.4 ± 5.1 26.1 ± 0.4 27.2 ± 0.2 29.9 ± 0.1 36.5 ± 0.5 44.4 ± 1.0

Proposed 125.4 ± 6.3 21.9 ± 0.2 22.9 ± 0.4 25.5 ± 0.4 29.1 ± 0.5 39.9 ± 1.8

Original 246.4 ± 7.5 23.5 ± 0.5 24.2 ± 0.5 26.0 ± 0.4 31.1 ± 1.0 35.9 ± 0.8

Proposed 247.2 ± 4.7 18.8 ± 0.2 18.8 ± 0.2 20.2 ± 0.2 23.6 ± 0.2 25.9 ± 0.2

Original 513.2 ± 11.1 21.3 ± 0.5 21.8 ± 0.6 23.4 ± 0.5 27.0 ± 0.8 30.0 ± 0.4

Proposed 493.6 ± 13.4 16.09 ± 0.07 15.7 ± 0.1 16.5 ± 0.1 19.1 ± 0.1 20.5 ± 0.2

reproduce also the results taken from [3] to facilitate comparison. Analyzing the results, ARIA performance dominates aiNet’s. On the other hand, the modified version overcomes the original formulation of ARIA, confirming the previous results. The results for the speech dataset are also better using the proposed version. Five executions of the algorithms were performed and Table 5 presents the average relative entropy results, estimated with the KNN method for 5 distincts values of k. And also three different values of ts were used, aiming at producing circa 125, 250 and 500 clusters. The algorithm was configured similarly as for the first experiments, except that the mutation rate decay was set to 0.95 and the initial population was set to 20. Although the proposed algorithm was superior to the original formulation for these real dataset, the difference between their performances was not so expressive if one compares it with the results for the artificial datasets. It is clear that this dataset represents a greater challenge for the algorithms, mainly because of its size, dimensionality, and because the clusters are not as well defined as those of the artificial datasets.

6

Final Remarks and Future Work

The adaptive radius immune algorithm, named ARIA, is claimed to preserve the density distribution of the original dataset when generating prototypes. In this paper, some theoretical results are provided to demonstrate that the original version of ARIA is not capable of performing density-preserving clustering when high-dimensional datasets are considered. Furthermore, the same theoretical results are explored to conceive a new version of ARIA, which has proved through experimental results to be capable of exhibiting the announced densitypreserving attribute, while the original algorithm has failed. The closeness measure used to evaluate the proximity of the original distribution and the compressed prototypes distribution was the relative entropy and it was described in this paper too. Another innovation introduced in this work is the way the algorithm estimates local densities, substituting the histogram method with the k-nearest neighbor method (KNN). The remaining of the algorithm was kept unchanged.

Adaptive Radius Algorithm for High-Dimensional Datasets

169

Table 6. Relative entropy results with the datasets 10 to 13, using the Parzen window method with h = 0.96 aiNet ts 0.2000 ˆ −0.118 ± 0.006 H M 130 ± 5 ts 0.1000 ˆ −0.032 ± 0.003 H M 284 ± 5 ts 0.0500 ˆ −0.017 ± 0.003 H M 380 ± 5 ts 0.2000 ˆ −0.072 ± 0.008 H M 145 ± 4 ts 0.1000 ˆ −0.020 ± 0.002 H M 293 ± 4 ts 0.0500 ˆ −0.010 ± 0.002 H M 388 ± 7 ts 0.2000 ˆ −0.236 ± 0.009 H M 100 ± 3 ts 0.1000 ˆ −0.134 ± 0.004 H M 198 ± 4 ts 0.0500 ˆ −0.076 ± 0.005 H M 288 ± 5 ts 0.2000 ˆ −0.014 ± 0.009 H M 50 ± 4 ts 0.1000 ˆ −0.003 ± 0.003 H M 169 ± 5 ts 0.0500 ˆ −0.003 ± 0.002 H M 305 ± 7

Original ARIA Dataset 10 0.0760 −0.087 ± 0.009 127 ± 5 0.0230 −0.050 ± 0.004 289 ± 7 0.0100 −0.010 ± 0.003 377 ± 4 Dataset 11 0.0610 −0.040 ± 0.009 141 ± 5 0.0220 −0.024 ± 0.003 290 ± 6 0.0070 −0.0018 ± 0.0006 380 ± 4 Dataset 12 0.0420 −0.14 ± 0.02 100 ± 3 0.0210 −0.135 ± 0.007 197 ± 4 0.0100 −0.070 ± 0.004 293 ± 4 Dataset 13 0.1000 0.007 ± 0.003 50 ± 3 0.0300 0.003 ± 0.002 172 ± 7 0.0150 0.002 ± 0.002 308 ± 4

Proposed ARIA 0.0250 0.008 ± 0.004 128 ± 5 0.0020 0.005 ± 0.006 285 ± 6 0.0005 0.002 ± 0.002 376 ± 5 0.0160 0.006 ± 0.002 146 ± 2 0.0018 0.003 ± 0.002 291 ± 7 0.0003 0.0013 ± 0.0009 387 ± 2 0.0150 0.02 ± 0.01 96 ± 3 0.0020 0.007 ± 0.004 203 ± 8 0.0008 0.008 ± 0.006 275 ± 8 0.0600 0.004 ± 0.003 44 ± 2 0.0070 0.002 ± 0.002 165 ± 3 0.0010 0.003 ± 0.001 312 ± 5

170

R.P.V. Violato, A.G. Azzolini, and F.J. Von Zuben

The algorithm was also compared with another well known immune-inspired algorithm for data clustering, called aiNet, and the original version of ARIA presented a better performance than aiNet, as well as the modified version of ARIA introduced in this paper overcame the original one. We are now planning to perform more advanced and challenging tests with the new version of ARIA. Beyond that, other changes may be suggested to improve the algorithm performance, mainly in its mutation and cloning procedures, given the particularities of practical applications.

Acknowledgements The authors would like to thank Fapesp and CPqD for the financial support.

References 1. de Castro, L.N., Von Zuben, F.J.: aiNet: An artificial immune network for data analysis. In: Abbass, H.A., Sarker, R.A., Newton, C.S. (eds.) Data Mining: A Heuristic Approach, ch. 12, pp. 231–259. Idea Group Publishing, USA (2001) 2. Bezerra, G.B., Barra, T.V., de Castro, L.N., Von Zuben, F.J.: Adaptive Radius Immune Algorithm for Data Clustering. In: Jacob, C., Pilat, M.L., Bentley, P.J., Timmis, J.I. (eds.) ICARIS 2005. LNCS, vol. 3627, pp. 290–303. Springer, Heidelberg (2005) 3. Stibor, T., Timmis, J.: An Investigation on the Compression Quality of aiNet. In: Proceedings of the 2007 IEEE Symposium on Foundations of Computational Intelligence (FOCI 2007), pp. 495–502 (2007) 4. Fukunaga, K., Hayes, R.R.: The reduced Parzen classifier. IEEE Transaction on Pattern Analysis and Machine Intelligence 11(4), 423–425 (1989) 5. Silverman, B.W.: Density Estimation for Statistics and Data Analysis. Monographs on statistics and applied probability, vol. 26. Chapman & Hall, London (1986) 6. Stibor, T., Timmis, J., Eckert, C.: On the use of hyperspheres in artificial immune systems as antibody recognition regions. In: Bersini, H., Carneiro, J. (eds.) ICARIS 2006. LNCS, vol. 4163, p. 215. Springer, Heidelberg (2006) 7. Violato, R.P.V., Von Zuben, F.J., Sim˜ oes, F.O., Uliani Neto, M., Nagle, E.J., Runstein, F.O., Gomes, L.C.T.: Density-preserving clustering applied to the quantization of speech signals (in Portuguese). In: 30o. Iberian-Latin-American Congress on Computational Methods in Engineering, CILAMCE 2009 (2009) 8. Chiu, C.Y., Lin, C.H.: Cluster Analysis Based on Artificial Immune System and Ant Algorithm. In: ICNC 2007, vol. 3 (2007) 9. Han, J., Kamber, M.: Data Mining: concepts and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2006) 10. Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: a review. ACM computing surveys (CSUR) 31(3), 264–323 (1999) 11. Timmis, J., Neal, M., Hunt, J.: An artificial immune system for data analysis. Biosystems 55(1-3), 143–150 (2000) 12. R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Austria (2006)

GAIS: A Gaussian Artificial Immune System for Continuous Optimization Pablo A.D. Castro and Fernando J. Von Zuben Laboratory of Bioinformatics and Bioinpired Computing (LBiC), Department of Computer Engineering and Industrial Automation (DCA), School of Electrical and Computer Engineering (FEEC), University of Campinas (Unicamp), P.O. Box 6101, 13083-852 Campinas-SP, Brazil {pablo,vonzuben}@dca.fee.unicamp.br

Abstract. This paper proposes a Gaussian Artificial Immune System (GAIS) to deal effectively with building blocks (high-quality partial solutions coded in the solution vector) in continuous optimization problems. By replacing the mutation and cloning operators with a probabilistic model, more specifically a Gaussian network representing the joint distribution of promising solutions, GAIS takes into account the relationships among the variables of the problem, avoiding the disruption of already obtained high-quality partial solutions. Two versions of the algorithm were developed. In the first one, the estimation of the joint probability distribution is achieved by means of a single multivariate Gaussian distribution. In the second version, the estimation is carried out using a Gaussian mixture model. The algorithms were applied to eight benchmarks and the results compared with those produced by an immune-inspired algorithm and an estimation of distribution algorithm. Keywords: Artificial immune system, Gaussian network, Gaussian mixture model, continuous optimization.

1

Introduction

Over the last decades, a variety of bio-inspired algorithms have been proposed for solving optimization problems. Among the appealing approaches, artificial immune systems (AISs) have received special attention due to their interesting features: (i) dynamic control of population size in response to the particularities of the problem; (ii) efficient mechanism of exploration/exploitation of the search space, which allows to find and preserve the local optima as well as to insert and maintain diversity in the population [1] [2] [3]. Despite their high performance as general problem solving tool, there are some shortcomings associated with these immune-inspired algorithms. Firstly, as the complexity and scale of the problem increase, the performance of the algorithms becomes more and more associated with a proper choice of the design parameters, such as mutation rate. Otherwise, very poor solutions can be generated [4]. In E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 171–184, 2010. c Springer-Verlag Berlin Heidelberg 2010 

172

P.A.D. Castro and F.J. Von Zuben

addition, it is noticeable that, when the solution is represented by a vector of attributes, the population of candidate solutions may contain partial highquality solutions to the problem, called building blocks [5]. The existing AISs suffer from the lack of ability to identify and effectively manipulate building blocks of the problem [6]. As affinity maturation requires cloning followed by the mutation of the newly-generated cells, and assuming that the mutation operator cannot discover by itself crucial relationships among the variables of the problem, building blocks are not supposed to survive, being disrupted by mutation. Recently, we have proposed an immune-inspired algorithm for solving combinatorial optimization problems, denoted Bayesian Artificial Immune System (BAIS) [6] which replaces the traditional mutation operator with a probabilistic model representing the probability distribution of the promising solutions found so far. Then the obtained probabilistic model is used to generate new individuals. A Bayesian network was adopted as the probabilistic model, due to its capability to properly capture the most expressive interactions among the variables of the problem. Besides the capability to deal with building blocks, BAIS still preserves the aforementioned advantages of AISs. The proposed algorithm was successfully applied to many optimization problems and the results are reported in the literature [7] [8] [9] [10]. Now, we extend the proposal in [6] aiming at investigating its usefulness in continuous optimization problems. Since the variables of the problem are continuous, the probabilistic model utilized is a Gaussian network, guiding to Gaussian Artificial Immune Systems (GAISs). Two versions of the algorithm were developed. The first one utilizes a single multivariate Gaussian probability distribution and the other one utilizes a Gaussian mixture model. In order to alleviate the computational cost required to learn the structure of the Gaussian network, this task is performed at a certain number of iteration while the probabilities associated with the structure are updated at each iteration. The main objective of this study is to design a competent algorithm with qualitative advantages over the contenders, as will be outlined in Section 4. Generally, the corresponding quantitative advantages arise as a natural consequence. Experiments on eight well-known functions have been carried out to evaluate the effectiveness of the proposed methodology when compared to other algorithms. This paper is organized as follows. In Section 2, we provide a background to artificial immune system and its limitation regarding the handling of building blocks. Section 3 describes the GAIS in details. The experimental results are outlined and analyzed in Section 4. Finally, in Section 5 we draw some concluding remarks and present the further steps of the research.

2

Artificial Immune Systems and Building Blocks

Artificial Immune System (AIS) is a relative new computational paradigm inspired by the immunological system of vertebrates and designed for solving a wide range of problems, such as optimization, clustering, pattern classification and computational security [2] [11].

GAIS: A Gaussian Artificial Immune System for Continuous Optimization

173

Several immune-inspired algorithms have been proposed in the literature, and they differ mainly on the immunological metaphor used as inspiration. Two important principles from immunology are the Clonal Selection Theory [12] and the Immune Network Theory [13]. Every standard AIS based on these two principles evolves the population of solutions according to their fitness by increasing the concentration of particular antibodies. The newly-generated antibodies are mutated, and antibodies with the lowest fitness among similar antibodies are suppressed. As the complexity and scale of the problem handled by the algorithm increase, the performance becomes more and more associated with a proper choice of the design parameters [4]. Moreover, since many problems can be decomposed into sub-problems, it is noticeable that antibodies in the population may contain partial solutions to the global problem. These partial solutions are called building blocks [5]. The existing artificial immune systems suffer from the lack of ability to identify and manipulate these interactions among the variables of the problem. Traditional mutation operators generally adopted in the literature can easily guide to the disruption of these partial solutions. In the bio-inspired computation area, various attempts to prevent the disruption of important partial solutions have been made by changing the representation of solutions in the algorithm, or by designing specific operators [5] [14] [15] [16]. However, these approaches are highly problem-dependent, leading to the design of very specific strategies. Furthermore, they require prior domain knowledge of the problem so that the variables in the antibodies can be properly arranged with respect to the mutation operator. Actually, in most of the real-world problems, this knowledge is not available a priori. One way to allow AISs to handle building blocks effectively is to make the algorithm learn the relationships among the variables by using statistical information extracted from the set of promising solutions. From a conceptual point of view, the selected set of promising solutions can be viewed as a sample drawn from an unknown probability distribution. An estimation of this probability distribution would allow the immune algorithm to generate new solutions that are somehow similar to the ones contained in the original selected solutions. Additionally, a probability distribution can effectively capture a building block structure of a problem, even without any prior information. Under these assumptions, challenging optimization problems with complex interactions among the variables may then become more manageable. This interesting approach to deal with building blocks is inspired by a class of evolutionary algorithms denoted Estimation of Distribution Algorithms (EDAs) [17] [18]. These evolutionary algorithms replace the traditional crossover and mutation operators with a probabilistic model. This probabilistic model represents the distribution of probabilities of the promising solutions and can be used to generate new individuals. There are several EDAs in the literature with alternative probabilistic models, depending on the intended degree of relationship among the variables [19] [20]. Despite the appeal of EDAs and the reported success, there are some difficulties

174

P.A.D. Castro and F.J. Von Zuben

in applying them effectively. One difficulty is related to the lack of diversity in the population [21]. Since EDAs generate new solutions based on the distribution of probability of previous selected solutions, they tend to explore the search space in a biased way. It is more likely to visit a region of the search space already visited in the past, leading the algorithm to get trapped into local minima easily. As a result, any basic EDA is not capable of dealing with multimodal optimization problems in an effective way. Although additional mechanisms for diversity maintenance can be incorporated into EDAs, they still tend to present a poorer performance when compared with artificial immune systems, which are algorithms inherently designed to promote diversity [22]. This aspect constitutes one of the motivations to develop our algorithms, and one of the arguments to support the advantage of our proposals over EDAs.

3

Gaussian Artificial Immune System

We propose a novel immune-inspired algorithm for solving continuous optimization problems which has the mutation and cloning operators replaced by a probabilistic model in order to generate new antibodies. The probabilistic model used in our proposal is a Gaussian network due to its capability to properly represent the most expressive interactions among the variables. The pseudo-code of the proposed algorithm, called Gaussian Artificial Immune System (GAIS), is presented in Algorithm 1. Notice that the cloning and mutation steps were replaced with the building of the Gaussian network and the subsequent sampling of new individuals according to the generated model. Algorithm 1. Gaussian Artificial Immune System Begin Initialize the population of antibodies; while stopping condition is not met do Evaluate the population; Select the best antibodies; Build the Gaussian network at every p iterations; Sample new antibodies; Suppress antibodies with fitness lower than a threshold; Eliminate similar antibodies; Insert new antibodies randomly; end while End

In GAIS, the initial population is generated at random. From the current population, the best solutions are selected. A Gaussian network that properly fits the selected antibodies is constructed. A number of new antibodies sampled from the network are then inserted into the population and those similar ones and with lower fitness are eliminated. We computed similarity based on the

GAIS: A Gaussian Artificial Immune System for Continuous Optimization

175

vector of solutions. Next, few individuals are generated randomly and inserted into the population in order to favour diversity, yielding a better exploration of the search space. Aiming at decreasing the computational cost of the algorithm and looking for a speed up in the execution time, the structure of the network is rebuilt at every p iterations (p > 1), whereas the redefinition of the probabilities remains at every iteration. Besides the capability to maintain diversity in the population, GAIS also performs multimodal optimization, adjusts dynamically the size of the population according to the problem, and, most importantly, identifies and manipulates building blocks. As a direct consequence of these characteristics, the search process becomes more robust in the sense that there is no much variation in performance when the algorithm is run several time for the same problem. In addition, a considerable reduction in the number of evaluations of candidate solutions occurs until the algorithm reaches a certain level of performance. Two aspects should receive special attention. The first is related to the way of building the Gaussian network from the selected individuals; and the other is how to use the network to generate new solutions. In the next section we explain in detail how to perform these two tasks. 3.1

Gaussian Network - Learning and Sampling

These two tasks, learning and sampling of Gaussian network, are fundamental to the GAIS algorithm. Learning the Gaussian network for a given set of promising solutions corresponds to estimating their joint distribution. Sampling new instances according to the network guides to new candidate solutions to the problem. Gaussian networks are probabilistic graphical models often used to estimate probabilistic dependencies among variables belonging to continuous domains [23]. Formally, a Gaussian network for a set of variables X = {X1 , X2 , ..., Xn } is a directed acyclic graph whose nodes are variables of the problem and the edges indicate relationships of dependence among the connected variables. The Gaussian network receives this name because the joint probability for X is represented by an n-dimensional Gaussian distribution with mean vector μ and covariance matrix Σ: f (x) = f (x1 x2 . . . xn ) = N (μ, Σ) = (2π)−n/2 |Σ|−1/2 e−1/2(x−μ)

t

Σ −1 (x−μ)

, (1)

where Σ is the determinant of the covariance matrix and Σ −1 is the inverse of the covariance matrix, also called precision matrix and denoted by W . For instance, if there is an edge from node X1 to node X2 , we say that variable X1 is parent of variable X2 and, therefore, the value of X2 is in a conditional dependence on the value of X1 . If a variable Xi has a set of parents, denoted here by pai , its probability distribution is characterized by a conditional probability density function (pdf) and it is expressed by f (Xi |pai ). On the other hand, the probability distribution of a variable Xj which has no parents is expressed by

176

P.A.D. Castro and F.J. Von Zuben

its unconditional pdf, f (Xj ). In this sense, the joint probability distribution for X can be re-written as a product of conditional pdf, each of which belongs to an independent Gaussian distribution. That is: f (x) =

n 

f (xi |pai ),

(2)

i=1

where f (xi |pai ) = N (μi +

 xk ∈pai

bki (xk − μk ), σi2 ),

(3)

where μi is the mean of Xi , σi2 is the variance of Xi conditioned to the parents of Xi and bki is a linear coefficient reflecting the strength of the relationship between Xk and Xi . If bki = 0, then there is no relationship between the variables Xk and Xi . The transformation of bki and σi2 of the Gaussian network into the precision matrix W of the equivalent Gaussian distribution is achieved by the following recursive formula [25]: ⎞ ⎛ b bti+1 −bi+1 W (i) + i+1 2 2 σi+1 σi+1 ⎠ (4) W (i + 1) = ⎝ −bti+1 1 , σ2 σ2 i+1

i+1

where W (i) is the i × i upper left submatrix of W , W (1) = vector (b1i , ..., b(i−1)i )t and bti is the transposed vector.

1 , σi2

bi is the column

– Learning The Gaussian network learning from a dataset can be stated as follows. Given a collection of observed data, find the network model to explain these data with maximum likelihood. By finding the network we mean to provide the structure of the graph, as well as the probability distribution of each variable that best fits the data. One usual approach to this task is to adopt a procedure for searching the space of all possible candidate network structures, given a metric that can provide a relative score to each point in the space. The heuristic search utilized by GAIS begins with an empty network, i.e. with no edges. Next, the probability distribution of each variable is estimated using the dataset and the score of the network is computed. The search process proposes small changes to the structure in order to obtain a network with higher score than the previous one. These small changes can be accomplished by adding or deleting an edge, or reversing the edge direction. Every time a change is made it is necessary to compute the probability distribution of the variables for the modified network. Regarding the scoring metrics, GAIS utilizes the Bayesian Information Criterion (BIC) for Gaussian networks [24]: N n

 1 −1/2vi (xli −mi − x ∈pa bki (xlk −mk ))2 k i √ p(D|G) = e − f (N ) ∗ dim(G), 2πv i l=1 i=1 (5)

GAIS: A Gaussian Artificial Immune System for Continuous Optimization

177

where D represents the dataset, G is the network under evaluation, N is the number of instances, n is the number of variables, bki is the linear coefficient of relationship between Xk and Xi , pai is the set of parents of Xi and vi is the ) = 12 ln N conditional variance of Xi given X1 , ..., Xi−1 ∀i, k. The function f (N n is responsible for penalizing complex models and dim(G) = 2n + i=1 pai is the number of parameters to be estimated. – Sampling Once the Gaussian network is built, we can generate new instances using the joint probability distribution encoded by the network (Eq. 1). To accomplish this task, we utilize a method that finds an ancestral ordering of the nodes in the Gaussian network and instantiates one variable at a time in a forward manner, that is, a variable is not sampled until all its parents have already been sampled [26]. In GAIS, the number of sampled instances varies with the problem. 3.2

GAIS with a Gaussian Mixture Model

GAIS supposes that the available dataset for building the probabilistic model was generated by a single multivariate Gaussian distribution. To solve unimodal functions, the algorithm achieves a good estimation of the probability distribution. However, to solve multimodal functions, where the local optima are not concentrated in only one region, but scattered, a single multivariate Gaussian distribution can not model the data satisfactorily. Therefore, we have also developed a version of GAIS that utilizes a Gaussian mixture model, namely GAISM . The components of the mixture are defined by means of clustering. The algorithm groups the selected solutions and process each group separately. This means that, for each group, a Gaussian network is generated using the same procedure as before. GAIS applies k-means to cluster the solutions, due to its simplicity and effectiveness. The number of clusters, k is specified empirically. With this modification, the density function of joint probability of the best solutions is said to be a mixture of Gaussian probability density functions, expressed by: f (x) =

k 

αi fi (x),

i=1

k 

αi = 1, αi ≥ 0,

(6)

i=1

where fi (x) is a single multivariate Gaussian probability density function, k is the number of clusters and, consequently, the number of components of the mixture, αi is the coefficient of the i-th component of the mixture. The value of each αi is proportional to the number of elements in the i-th cluster: ci αi = k

j=1 cj

, i = 1, ..., k,

where ci is the number of elements in cluster i.

(7)

178

4

P.A.D. Castro and F.J. Von Zuben

Experiments

This section describes the experiments carried out to evaluate the proposed algorithm. We have applied GAIS to eight well-known problems and compared the performance with other optimization tools reported in the literature. 4.1

Test Functions

Eight functions often utilized in the literature were tested during the experiments. In what follows, we provide a description of each function for a generalized dimensionality d. – Sphere: This function is probably the most standard unimodal benchmark problem for optimization. It involves the minimization of a single hyperparabola and the minimum value for any dimensionality is 0 which is obtained if all xi take the value of 0. F1 (x) =

d 

x2i ,

xi ∈ [−100; 100]d.

(8)

i=1

– Summation Cancellation (SumCan): This problem has multivariate linear interactions between the variables. So, algorithms that are capable of modelling these dependencies are supposed to outperform algorithms that are not capable of doing so. The optimum is located at a very sharp peak, which implies that the optimization algorithm needs to be able to prevent premature convergence in order to reach the global optimum. d F2 (x) = 100/(10−5 + i=1 |yi |), xi ∈ [−3; 3], (9) where y1 = x1 , yi = xi + yi−1 , i ≥ 2. – Rosenbrock: It is a highly nonlinear function. It has a curved valley along which the quality of the solutions is much better than in its neighborhood. This valley has a unique minimum for any dimensionality, which is obtained when all xi are set to the value of 1. Rosenbrock’s function has proven to be a real challenge for any algorithm. F3 (x) =

d−1 

100(xi+1 − x2i )2 + (xi − 1)2 ,

xi ∈ [−5.12; 5.12]d.

(10)

i=1

– Griewank: It is a function with many local optima. Basically, it is a parabola superimposed with a sine function to obtain these local optima. The minimum value for this function for any dimensionality is 0, which is obtained if all xi are set to the value of 100. F4 (x) = 1 +

d d   x2i xi − cos( √ ), 4000 i i=1 i=1

xi ∈ [−600; 600]d.

(11)

GAIS: A Gaussian Artificial Immune System for Continuous Optimization

179

– Ackley: The value of the global optimum is 0 for any dimensionality, which is obtained when all xi take the value of 0. √1 d d 1 F5 (x) = −20e−0.2 d i=1 xi −e d i=1 cos(2πxi ) +20+e, xi ∈ [−32.768; 32.768]d. (12) – Michalewicz: It is a function with many local optima. Michalewicz’s function has many long channels along which the minimum value throughout the channel is the same. The minimum value for this function depends on its dimensionality.  d  (i + 1)x2i F6 (x) = − sin(xi ) sin20 , xi ∈ [0; π]d . (13) π i=1 – Rastrigin: Rastrigin’s function has many local minima, but just one global optimum. The value of global optimum is 0 for any dimensionality, which is obtained when all xi are set to 0. F7 (x) = 10d

d 

(x2i − 10 cos(2πxi )),

xi ∈ [−5.12; 5.12]d.

(14)

i=1

– Schwefel: It is a function with many peaks and valleys. The global minimum is located far from local minima, leading the algorithms to get stuck at these local minima. The value of the global optimum is -418.9829d, which is obtained when all variables take the value 420.9687. F8 (x) =

d 

−xi sin( |xi |),

xi ∈ [−500; 500]d.

(15)

i=1

4.2

Experimental Setup

All functions were tested using 30 dimensions, except for Michalewicz’s function where we adopted 10 dimensions. The reason to use a 10-dimensions Michalewicz’s function is that in the literature this function has been considered in this way. This is the same reason why we adopted 30 dimensions for the remaining cases. The parameters of GAIS and GAISM are the same for all cases. The initial population contains 100 antibodies. The number of components of the mixture for GAISM was established empirically as 3. We have compared the proposed algorithms with two alternative approaches. The first one is the Iterated Density Estimation Algorithm (IDEA) [27], that also utilizes a Gaussian mixture model. IDEA applies the Leader clustering algorithm [28] to generate the components of the mixture. The number of clusters was also defined as 3 and the population size was equal to 500. The other algorithm considered for comparative analysis is the Artificial Immune Network for Optimization (opt-aiNet) [29]. The initial population contains 100 antibodies

180

P.A.D. Castro and F.J. Von Zuben

and each one generates 2 clones. Parameters of both, opt-aiNet and IDEA, were adjusted following their authors’ guidelines. Notice that opt-aiNEt is an immune-inspired algorithm not endowed with probabilistic models, and IDEA uses probabilistic models but are not endowed with the search capabilities of immune-inspired algorithms. The authors believe that using these two algorithms as contenders is justifiable in an inaugural paper proposing a Gaussian immune-inspired algorithm, because it allows two reasonable contrasts: (i) immune-inspired against non-immune-inspired Gaussian algorithms; (ii) Gaussian against non-Gaussian immune-inspired algorithms. For GAIS, GAISM and IDEA, the Gaussian network is built at every 10 iterations, but the probabilities of the network are updated at every iteration. In order to penalize the complexity of the model, we have imposed a constraint on the number of parents a node can have. It corresponds to a maximal order of interactions that can be covered and it directly influences the complexity of the model. By our previous experience on Gaussian network learning, we know that when the complexity of the network is too high, it is more likely to detect spurious correlations on the data. Thus, each variable can have only two parents. For GAIS and GAISM , 80% of the best solutions are utilized to build the probabilistic model the number of samples generated is half of the size of the current population. The number of random individuals inserted into the population in order to create diversity is around 3% of the current population size. The stopping condition for all algorithms is the maximum number of function evaluations, defined as 105 for all benchmarks. 4.3

Results

The average results obtained by the four algorithms over 30 executions are presented in Table 1. To evaluate the statistical significance of the difference among the performance of the algorithms, the t-test was employed with a significance level of 95% (α=0.05). The symbol  denotes there is a statistical difference between the results obtained by GAIS and the compared algorithm. On the other hand, the symbol † denotes a statistical difference related to GAISM . Observing Table 1, we can see that all algorithms have found the global optimum for Sphere function. In the case of Summation Cancellation, which is a function with strong interdependencies among the variables, GAIS has found the global optimum whereas GAISM and IDEA have achieved a value very close to the global optimum. This indicates that these algorithms really have captured the relationship between the variables. On the other hand, opt-aiNet performs very badly in this function because it is not provided with a similar mechanism. For the remaining functions, our algorithms achieved better results, with a slight advantage to GAISM . Once GAISM utilizes a mixture model, generally it can fit the data (promising solutions) more satisfactorily than the other algorithms. Although IDEA also utilizes a mixture model, its mechanism to exploit/explore the search space is not so good as in GAISM , leading the algorithm to get stuck at local minima easily.

GAIS: A Gaussian Artificial Immune System for Continuous Optimization

181

Table 1. Average results over 30 executions obtained by GAIS, GAISM , IDEA and opt-aiNet on the eight benchmark functions Function GAIS GAISM IDEA opt-aiNet F1 0 ± 0.000 0 ± 0.000 0 ± 0.000 0 ± 0.000 7 6 6 F2 10 ± 0.0 9.87×10 ± 4169 () 7.44×10 ± 13996 (†) 6.82 ×106 ± 9502 (†) F3 0.8417 ± 0.007 0.9025 ± 0.019 1.091 ± 0.056 (†) 7.67 ± 1.245 (†) F4 0.0058 ± 0.002 0 ± 0.000 () 0.037 ± 0.022 (†) 0.403 ± 0.066 (†) −6 F5 4.4 × 10 ± 0.003 0 ± 0.000 () 0.0008 ± 0.007 (†) 0.096 ± 0.002 (†) F6 -9.6539 ± 0.016 -9.6604 ± 0.027 -9.6257 ± 0.019 (†) -9.0841 ± 0.086 (†) F7 0.0028 ± 0.001 0.0007 ± 0.004 0.0032 ± 0.003 (†) 1.96 ± 0.907 (†) F8 -12569.87 ± 0.048 -12569.49 ± 0.001 -12569.32 ± 0.460 -8903.24 ± 0.734 (†)

Table 2. Average number of fitness evaluations until the best value for the eight functions is reached Function F1 F2 F3 F4 F5 F6 F7 F8

4.4

GAIS 6719.2 ± 291.7 41658.8 ± 1326.4 81037.3 ± 2188.3 7213.5 ± 1721.6 7508.1 ± 1840.7 3052.8 ± 1389.5 4782.2 ± 1762.9 5368.4 ± 1065.2

GAISM 6485.9 ± 314.2 37014.0 ± 1244.9 75368.2 ± 2046.3 8644.2 ± 1645.8 5542.5 ± 1112.3 4907.4 ± 1503.3 3381.6 ± 1040.8 6194.8 ± 1249.2

IDEA 11038.6 ± 392.5 80255.3 ± 2072.1 142604.8 ± 4192.7 19016.6 ± 2031.5 23864.2 ± 2970.4 17046.1 ± 2192.7 11406.9 ± 2826.4 27324.3 ± 2240.6

opt-aiNet 9076.8 ± 427.3 143892.4 ± 1705.4 92831.2 ± 3064.2 27513.7 ± 2482.6 15376.9 ± 2062.4 13594.1 ± 2485.8 36033.3 ± 3064.6 19740.5 ± 1740.6

Discussion

As stated before, GAIS offers conceptual advantages over the contenders and they are described in what follows. The first advantage is related to the maintenance of building blocks. With this mechanism, GAIS avoids disrupting the partial solutions found so far, leading to a great improvement in the quality of the candidate solutions even in the first iterations. We have observed during the experiments that while GAIS and GAISM have found high-quality solutions with few fitness evaluations, the other methodologies needed more fitness evaluations to achieve a satisfactory performance. Table 2 shows the average number of fitness evaluations until the algorithms reach the best solution. Observing Table 2, we note that our proposals are able to find the best solution much faster than the other algorithms. Other aspect to be highlighted is the effective mechanism of GAIS and GAISM to perform a multimodal search, finding diverse high-quality local optima quickly, as depicted in Table 3. In addition, the initial population size is not crucial to GAIS due to its capability to control the population size along the search process in response to the particularities of the problem. Regarding the implementation of GAIS and GAISM , we notice that the algorithms do not require a large amount of computational resources [30]. Although a Gaussian network has to be produced, the proposed methodology still preserves

182

P.A.D. Castro and F.J. Von Zuben Table 3. Average number of local optima found by the algorithms Algorithm GAIS GAISM IDEA opt-aiNet

F3 20.2 ± 4.1 22.7 ± 3.8 9.8 ± 3.3 20.6 ± 3.7

F4 40.3 ± 42.1 ± 16.8 ± 36.4 ±

5.7 6.3 7.0 9.3

F5 29.2 ± 33.7 ± 14.5 ± 26.2 ±

5.3 5.2 4.6 3.4

F6 21.7 ± 20.4 ± 11.7 ± 18.5 ±

6.2 4.7 5.4 4.6

F7 16.0 ± 3.4 18.6 ± 6.1 8.6 ± 2.7 20.1 ± 5.8

F8 19.5 ± 20.9 ± 10.8 ± 17.3 ±

5.2 3.2 4.1 3.8

the computational tractability due to the restriction of at most two parents for each node in the network. Additionally, we avoid the synthesis of the network at each iteration.

5

Concluding Remarks and Future Work

In this paper we have proposed a novel immune-inspired algorithm for solving continuous optimization problems, taking into account the interdependencies between the variables of the problem. Our proposal, called Gaussian Artificial Immune System (GAIS), replaces the traditional mutation and cloning operators with a probabilistic model representing the joint distribution of promising solutions and, subsequently, utilizes this model for sampling new solutions. The probabilistic model used is a Gaussian network due to its capability to properly capture the most relevant interactions among the variables of the problem. Two versions of the algorithm were developed. The first one utilizes a single multivariate Gaussian probability distribution and the other one utilizes a Gaussian mixture model. In order to alleviate the computational cost required to learn the structure of the Gaussian network, this task is performed at a certain number of iteration while the probabilities associated with the structure are updated at each iteration. To evaluate the algorithm, we have considered eight functions and compared the obtained results with those produced by other approaches. The experiments pointed favorably to our proposal for all tested functions. We are currently investigating some aspects that can be further improved, such as alternative clustering algorithms to generate the components of the mixture and even other methods, such as Expectation Maximization (EM). We are also analyzing the performance of the algorithms in more challenging problems. Furthermore, comparisons with other evolutionary algorithms will be carried out. Another aspect to be considered is the extension of the proposals to handle multiobjective optimization problems in continuous domains.

References 1. Timmis, J., Hone, A., Stibor, T., Clark, E.: Theoretical advances in artificial immune systems. Theoretical Computer Science 403(1), 11–32 (2008) 2. de Castro, L.N., Timmis, J.: An Introduction to Artificial Immune Systems: A New Computational Intelligence Paradigm. Springer, Heidelberg (2002) 3. Dasgupta, D. (ed.): Artificial Immune Systems and Their Applications. Springer, Heidelberg (1999)

GAIS: A Gaussian Artificial Immune System for Continuous Optimization

183

4. de Castro, L.N., Von Zuben, F.J.: Learning and optimization using the clonal selection principle. IEEE Trans. Evolutionary Computation 6(3), 239–251 (2002) 5. Holland, J.H.: Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. MIT Press, Cambridge (1992) 6. Castro, P.A.D., Von Zuben, F.J.: BAIS: A Bayesian Artificial Immune System for the effective handling of building blocks. Information Sciences 179(10), 1426–1440 (2009) 7. Castro, P.A.D., Von Zuben, F.J.: Feature subset selection by means of a Bayesian artificial immune system. In: Proc. Eighth International Conference on Hybrid Intelligent Systems, pp. 561–566 (2008) 8. Castro, P.A.D., Von Zuben, F.J.: MOBAIS: A Bayesian artificial immune system for multi-objective optimization. In: Bentley, P.J., Lee, D., Jung, S. (eds.) ICARIS 2008. LNCS, vol. 5132, pp. 48–59. Springer, Heidelberg (2008) 9. Castro, P.A.D., Von Zuben, F.J.: Multi-objective Bayesian artificial immune system: Empirical evaluation and comparative analyses. Journal of Mathematical Modelling and Algorithms 1, 151–173 (2009) 10. Castro, P.A.D., Von Zuben, F.J.: Multi-objective feature selection using a Bayesian artificial immune system. Journal of Intelligent Computing and Cybernetics (2010) (in press) 11. Dasgupta, D.: Advances in Artificial Immune Systems. IEEE Computational Intelligence Magazine, 40–49 (2006) 12. Ada, G.L., Nossal, G.J.V.: The clonal selection theory. Scientific American 257(2), 50–57 (1987) 13. Jerne, N.K.: Towards a network theory of the immune system. Ann. Immunol. (Inst. Pasteur) 125C, 373–389 (1974) 14. Goldberg, D.E., Deb, K., Kargupta, H., Harik, G.: Rapid accurate optimization of difficult problems using fast messy genetic algorithms. In: Proc. of the Fifth Int. Conf. on Genetic Algorithms, pp. 56–64 (1993) 15. Goldberg, D.E., Korb, G., Deb, K.: Messy genetic algorithms: Motivation, analysis, and first results. Complex Systems, 493–530 (1989) 16. Goldberg, D.E.: Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Reading (1989) 17. M¨ uhlenbein, H., Paass, G.: From recombination of genes to the estimation of distributions I. Binary parameters. In: Ebeling, W., Rechenberg, I., Voigt, H.-M., Schwefel, H.-P. (eds.) PPSN 1996. LNCS, vol. 1141, pp. 178–187. Springer, Heidelberg (1996) 18. Baluja, S., Davies, S.: Using optimal dependency-trees for combinational optimization. In: Proc. of the 14th Int. Conf. on Machine Learning, pp. 30–38 (1997) 19. Pelikan, M., Goldberg, D.E., Lobo, F.: A survey of optimization by building and using probabilistic models. Comput. Optim. Appl. 21(1), 5–20 (2002) 20. Pelikan, M.: Probabilistic Model-Building Genetic Algorithms. Springer, Heidelberg (2005) 21. DelaOssa, L., G´ amez, J.A., Mateo, J.L., Puerta, J.: Avoiding premature convergence in estimation of distribution algorithms. In: Proc. of the 11th Congress on Evolutionary Computation, pp. 455–462 (2009) 22. de Franca, F.O., Coelho, G.P., Von Zuben, F.J.: On the diversity mechanism of opt-aiNet: a comparative study with fitness sharing. In: Proc. of the IEEE Congress on Evolutionary Computation (2010) 23. Geiger, D., Heckerman, D.: Learning Gaussian Networks. Technical Report MSRTR-94-10, Microsoft Research (1994)

184

P.A.D. Castro and F.J. Von Zuben

24. Lu, Q., Yao, X.: Clustering and learning Gaussian distribution for continuous optimization. IEEE Transactions on Systems, Man, and Cybernetics, Part C 35(2), 195–204 (2005) 25. Shachter, R.D., Kenley, C.R.: Gaussian influence diagrams. Management Science 35(5), 527–550 (1989) 26. Ripley, B.D.: Stochastic simulation. John Wiley & Sons, Chichester (1987) 27. Bosman, P., Thierens, D.: Expanding from discrete to continuous estimation of distribution algorithms: The IDEA. In: Deb, K., Rudolph, G., Lutton, E., Merelo, J.J., Schoenauer, M., Schwefel, H.-P., Yao, X. (eds.) PPSN 2000. LNCS, vol. 1917, pp. 767–776. Springer, Heidelberg (2000) 28. Hartigan, J.A.: Clustering algorithms. Wiley, New York (1975) 29. de Castro, L.N., Timmis, J.: An artificial immune network for multimodal function optimization. In: Proc. of the 2002 Congress on Evolutionary Computation, pp. 699–704 (2002) 30. Broom, B., Subramanian, D.: Computational methods for learning Bayesian networks from high-throughput biological data. In: Bayesian inference for gene expression and proteomics. Cambridge University Press, Cambridge (2006)

An Immune Algorithm for Minimum Interference Channel Assignment in Multi-radio Wireless Mesh Networks Su-Wei Tan Faculty of Engineering, Multimedia University, Malaysia [email protected]

Abstract. Wireless mesh networks typically employ mesh routers that are equipped with multiple radio interfaces to improve network capacity. The key is to cleverly assign different channels (i.e. frequency bands) to each radio interface to form a mesh network with minimum interference. The channel assignment must obey the constraints that the number of different channels assigned to a router is at most the number of interfaces on the router, and the resultant mesh network is connected. This problem is known to be NP-hard. We propose an immune algorithm which performs greedy channel assignment to obtain feasible solutions, and use the clonal selection principle to improve the solutions. Through extensive simulations, we show that our algorithm out-perform a genetic algorithm and a graph-theoretic algorithm proposed for the same problem.

1

Introduction

A wireless mesh network is a form of multi-hop wireless network. It consists of a number of stationary wireless mesh routers which are connected wirelessly forming a mesh-like backbone structure. Some of the routers also serve as wireless access point for clients (e.g. computer/mobile devices with wireless access) to associate themselves with the network. The data to and from the clients are delivered by the routers over the backbone. Typically, one or more routers are connected to wired network serving as gateway to the Internet. Fig. 1 (a) illustrates a sample wireless mesh network consisting of 5 mesh routers, one of which also functions as gateway. Wireless mesh networking has seen an increasing growth in deployment especially in providing low-cost Internet access for low-income neighborhoods and scarcely populated areas. This can be attributed to the reduced dependency on wired infrastructure of the technology, which substantially lower the deployment cost. By leveraging on commodity IEEE 802.11 (or more commonly known as Wi-Fi) hardware and self-configuring ad-hoc networking techniques, wireless mesh network can be deployed easily. In addition, the technology is resilient to failure of nodes due to the inherent redundancy of the mesh topology. Interested reader is referred to [6] for other application areas of wireless mesh networks. One main consideration in deploying the technology is its capacity or effective throughput offered to the clients. In a general wireless network, the broadcast E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 185–198, 2010. c Springer-Verlag Berlin Heidelberg 2010 

186

S.-W. Tan

Wired network Operating Channel 3 1

Gateway Mesh router

2 2 3

x

2

Wireless link Radio interface Mobile device

Fig. 1. A multi-radio wireless mesh network with channel assignment

nature of the wireless medium dictates that transmissions from different devices over a same channel will lead to signal collision, which in turn causes data loss. This mandates the use of multiple access techniques (e.g. random access in IEEE 802.11 networks) to regulate the transmissions over the channel. However, the effectiveness of such a technique degrades as the number of devices increases. This can be improved if the devices transmit over different channels as the interference between the devices is reduced. The IEEE 802.11 networks for example provide a number of non-overlapping channels to facilitate the above. Similarly, the capacity of a wireless mesh network can be enhanced by equipping the routers with multiple radio interfaces, each of which is tuned to a different channel. The effectiveness of a multi-radio wireless mesh network largely depends on which channels to use on a particular router. This is referred to as channel assignment problem in which the channel assignment must obey the constraints that the number of different channels assigned to a router is at most the number of interfaces on the router, and the resultant mesh network is connected. This problem is known to be NP-hard [7]. A simple channel assignment is shown in Fig. 1. The approaches to channel assignment in wireless mesh networks can broadly be classified into two categories: (i) Dynamic; and (ii) Quasi-static [7]. The dynamic approach requires every router to switch their interfaces dynamically from one channel to another between successive data transmission. This necessitates the use of hardware with fast channel-switching capability, which cannot be provided by commodity hardware. In this paper, we follow the quasi-static approach in which channels are assigned to routers statically. The channel assignment can be changed upon significant changes to traffic load or network topology. The channel assignment problem can be solved centrally or in a distributed fashion. This paper focuses on centralized algorithms for the problem. We investigate the use of immune algorithm as an optimization tool for the channel assignment problem. As to be discussed later, the problem can be viewed as a variant of the graph coloring problem. In [9], immune algorithm has been successfully applied to the graph coloring problem with competitive results to those obtained by the best evolutionary algorithms. Furthermore, immune algorithm achieves this without the need for specialized crossover operators. Motivated by

An Immune Algorithm for Minimum Interference Channel Assignment

187

this, we proposed an immune algorithm based on CLONALG [1] as the strategy to evolve and improve solutions obtained using a simple greedy channel assignment procedure. Through extensive simulations, we show that the algorithm performs better than a genetic algorithm [3] and a graph-theoretic algorithm proposed for the channel assignment problem. The rest of this paper is structured as follows. In next section, we present the system model, channel assignment problem formulation, and discuss some related proposals. In Section 3, we describe our proposed algorithm in details. Section 4 presents the simulation experiments and results. Section 5 concludes this paper.

2 2.1

Problem Formulation System Model

We study the problem of assigning channels to backbone mesh routers in a wireless mesh network. The channel assignment problem is solved centrally at a channel assignment server (CAS) using information (i.e. connectivity and interference between the routers) collected from the network. The location of the CAS has no direct influence on the channel assignment performance, but has impact on the amount of control overhead introduced to the network. We assume that the CAS is co-located with the gateway node, which is typically strategically placed in the network. As the gateway node needs to perform routine per-packet processing, the processing and memory requirement of a channel assignment algorithm needs to be as small as possible. Following typical deployment scenarios, we assume that backbone structure and client access use two different radio standards which operate on two distant central frequencies. For example, IEEE 802.11-a (5.0 GHz) is used for the backbone structure and IEEE 802.11-b/g (2.4 GHz) is used for the client access. Since the frequencies are far apart, their transmissions are unlikely to interfere with each other. Hence, our study focus on the channel assignment for the backbone routers. We consider a wireless mesh network with stationary mesh routers arbitrarily distributed on a plane. Each router is equipped with one or multiple radio interfaces based on IEEE 802.11 standards. As in [2], we assume that all radio interfaces use omni-directional antennas and have identical transmission ranges (denoted by R). The connectivity between the routers (nodes) is modeled using an undirected graph, G(V, E) where V denotes the set of nodes and E denotes the set of connectivity links in the network. A connectivity link (i, j) marks that the nodes i and j are within each other’s communication range. Henceforth, we refer to G as connectivity graph. In order for two nodes of a link (i, j) ∈ E to actually communicate, both nodes must have one of their respective radio interfaces tuned to a common channel. Due to the broadcast nature of the wireless medium, transmissions by nodes that are within each other’s interference range may interfere and this results

188

S.-W. Tan

a

b

c

e

d

R R’

(a)

a

b

c

(b)

d

e

l ab

l bc

l cd

l de

(c)

Fig. 2. (a) 5 nodes spaced equally on a plane; (b) Corresponding connectivity graph; (b) Corresponding conflict graph

in data loss. The interference between the transmissions is typically represented using an interference model, which defines how transmission over a wireless link can interfere with other links in the network. Various interference models exist in the literature, most notably the physical and the protocol interference models [4]. The discussion in this paper is independent of any specific interference model as long as the interference model is defined on pairs of links. For ease of exposition, we assume a binary interference model [7] in which two links either interfere or do not interference. Given an interference model, a conflict graph [5] can be used to represent the interference between the links. A conflict graph is defined by first creating a set of vertices Vc corresponding to the links in the connectivity graph, i.e. Vc = {lij |(i, j) ∈ E}. Next, an edge is placed between two vertices (say, lij and lpq ) in the conflict graph if the corresponding links ((i, j) and (p, q)) interfere with each other. The weight of the edge indicates the extent of interference between those links, which depends on the amount of traffic transmission over the links. For simplicity, we assume all nodes have equal traffic load, hence all links have unity weight. The conflict graph is represented as Gc (Vc , Ec ) where Ec denotes the set of edges as defined above. The conflict graph can be used to represent any interference model. As in [2], we associate the terms “node” and “link” with the connectivity graph, and use the terms “vertex” and “edge” for conflict graph. We illustrate the concept of connectivity graph and interference graph in Fig. 2. Fig. 2 (a) depicts 5 wireless nodes spaced equally on a plane. Each node has a communication range and interference range of R and R , respectively. The resultant connectivity graph and interference graph are shown in Fig. 2 (b) and (c) respectively. In Fig. 2 (c), the conflict graph has four vertices each representing a link in the network. In this paper, we assume that the nodes use IEEE 802.11 reliable unicast where nodes access to link is control by the Request-To-Send/Clear-To-Send (RTS/CTS) control messages. Due to this, a transmission on link (a, b) interferes with the link (b, c) and (c, d) but not with (d, e), thus gives the conflict graph in Fig. 2 (c).

An Immune Algorithm for Minimum Interference Channel Assignment

2.2

189

Channel Assignment Problem

We consider the channel assignment problem as defined in [7]. The goal of the problem is to bind a unique channel to each link in the connectivity graph such that the number of different channels assigned to the links incident on any node is at most the number of interfaces on the node. The optimization objective is to minimize the total network interference, which is defined as the number of pairs of links that are interfering. This can be calculated from the conflict graph as the number edges connecting two vertices (representing the links) which are assigned a same channel. Consider Fig. 2 (b) and (c), suppose that links (a, b), (b, c) and (c, d) are assigned channel 1 and link (d, e) is on channel 2, the total network interference can be calculated as 3. Consider a wireless mesh network represented by connectivity graph G(V, E) and conflict graph Gc (Vc , Ec ). Let K = {1, 2, ..., K} be the set of K channels in the system and Ri be the number of radio interfaces on node i ∈ V . Formally, the channel assignment problem is to compute a function f : Vc → K to minimize the overall network interference I(f ) defined below while satisfying the below interface constraint. Interface constraint. ∀i ∈ V, |{k | f (e) = k for some e ∈ E(i)}| ≤ Ri .

(1)

Network interference I(f ). I(f ) = |{(u, v) ∈ Ec | f (u) = f (v)}|

(2)

If we view the assignment of channels to vertices as coloring of vertices and recall that vertices (Vc ) represent links (E) in the connectivity graph G, the problem can be viewed as coloring of links in G. Unfortunately, standard edge-coloring formulation fails to capture the interface constraint. As the number of edges incidents on a node may exceed the number of interfaces at the node, existing solutions for the problem are not directly applicable here. In [7], the channel assignment is shown to be NP-hard as it is reducible to an NP-hard Max Kcut problem. In the following, we sometimes use the term “color” in place of “channel” for ease of exposition. 2.3

Related Channel Assignment Algorithms

In this section, we review three closely related works. We note that there are other formulations of channel assignment problem for wireless mesh networks. For examples, the joint channel assignment and routing problems addressed in [12] and [13]. Interested reader is referred to [8] for a more comprehensive survey of work in this area. In [2], Marina and Das proposed a polynomial-time heuristic called Connected Low Interference Channel Assignment (CLICA). The central idea of CLICA is based on a notion of degree of flexibility, which marks the flexibility for a node to make coloring decisions. A priority value is associated with each of the nodes

190

S.-W. Tan

as their respective degree of flexibility, and coloring decisions are made on a node-by-node basis in the order of the priority. Each coloring decision is made in a greedy fashion: a node picks a color that is locally optimal (i.e. with minimum interference) for each of the links incident from itself. The interface constraint is enforced on each coloring decision. In [7], the authors proposed a Tabu-based heuristic for the problem. Unlike CLICA, the algorithm operates on the conflict graph. It consists of two phases. In the first phase, the Tabu search based technique [10] developed for graph coloring problem is applied to find a good solution without subject to the interface constraints. Consequently, the solution obtained may violate the interface constraints. This is remedied in the second phase in which a “merge” operation is applied on nodes wherein the interface constraints are violated. Our algorithm has a similar structure to this algorithm: a first phase in which immune algorithm is applied to find good solution which may contain interface constraints violation, and a second phase that repairs the solution. We briefly discuss the differences that distinguish our algorithm from theirs. First, we devise a greedy decision in coloring the vertices. Second, our first phase attempts to enforce the interface constraints and each violation is given a penalty. We believe this reduces the number of repair operations in second phase which generally increase the interference of the solution. Our repair mechanism is similar to theirs. In a different approach, Chen et al. [3] formulated the channel assignment problem as a two-objective optimization problem: minimizing the network interference and maximizing the network connectivity. The authors developed a genetic algorithm based on the well-known Non-dominated Sorting Genetic Algorithm II (NSGA-II) [11]. Following their notations, an individual is represented as a vector a ¯ = (a1 , a2 , ..., aN ) of dimension N × C, where  aij =

1 if channel j is in used in node i 0 otherwise

(3)

In the above, N represents the number of nodes and C is the number of available channels. In the proposed algorithm, a simple two-point crossover operator is used to generate some new individuals, which are subsequently subject to mutation. The mutation is done by randomly selects part of an individual and perform inversion of bits to the chosen locations. Next, non-dominated sorting with elitism is used to select the next generation of individuals. With a two-objective problem formulation, the best individuals are those that are non-dominated by other individuals in the objective functions. In our problem formulation, we assign channels to conflict vertices (i.e. connectivity links) thus connectivity is ensured. The genetic algorithm’s solution for our problem can be obtained by first choosing those individuals with maximum connectivity and follow by selecting the individual with minimum interference out of these individuals. In [3], the genetic algorithm has been shown to out-perform the Tabu-based algorithm in minimizing the network interference. In Section 4, we compare our algorithm with CLICA and the two-object genetic algorithm.

An Immune Algorithm for Minimum Interference Channel Assignment

3

191

Channel Assignment Algorithm

As discussed earlier, our algorithm consists of two phases. In the first phase, immune algorithm is applied to find good solution which may contain interface constraints violation. The solution is then repaired in the second phase. 3.1

Phase One

Our immune algorithm is based on CLONALG [1], a well-known clonal selection based algorithm. It consists of the following basic steps of CLONALG: 1. Initialization: create an initial random population of individuals 2. Antigenic presentation: for each antigenic patten, do: 2.1 Affinity evaluation 2.2 Clonal selection and expansion 2.3 Affinity maturation 2.4 Metadynamics 3. Cycle: repeat Step 2 until a stopping criterion is met. The algorithm is stopped after a predefined number of generations. The best solution obtained is presented to phase two. We next discuss the above Step 1 and Step 2 in details. Initialization. As in the optimization version of CLONALG, we consider only the existence of antibodies (individuals) to represent the set of candidate solutions. Each individual is defined as strings of integers of finite length L = |Vc |, representing the permutations of vertices in the conflict graph Gc . The initial population is randomly created. Let P represent the individuals in current population and N represent the size of the population. Antigenic Presentation. In this step, the individuals undergo the process of affinity evaluation, clonal selection and expansion, affinity maturation and metadynamics. Affinity evaluation. First, we define the affinity for a given solution as the fractional network interference, If rac : If rac =

I(f ) |Ec |

(4)

where I(f ) is the network interference (2) and |Ec | is the total number of edges in the conflict graph. This represents the number of conflicts that remain even after channel assignment relative to the number of conflicts in the single-channel network. To calculate the affinity for a given individual, we first need to translate the vertex ordering into channel assignment. This is done by a simple greedy assignment strategy. Basically, the vertices are visited and assigned channel oneby-one in the order presented in the individual. For a given vertex v, we assign it

192

S.-W. Tan

1: if i.spareInterfaces ≥ 1 and j.spareInterfaces ≥ 1 then 2: c ← v.pickChannelMinInterferenceFrom(K) 3: else 4: if i.spareInterfaces ≥ 1 and j.spareInterfaces ≡ 0 then 5: C ← v.getChannelsAssignedTo(j) 6: c ← v.pickChannelMinInterferenceFrom(C) 7: else if i.spareInterfaces ≡ 0 and j.spareInterfaces ≥ 1 then 8: C ← v.getChannelsAssignedTo(i) 9: c ← v.pickChannelMinInterferenceFrom(C) 10: else 11: C ← v.getCommonChannels(i, j) 12: c ← v.pickChannelMinInterferenceFrom(C) 13: end if 14: end if Fig. 3. Greedy channel selection procedure

with a channel c that is found to introduce the least increase in the interference to v and neighbors of v. The channel selection process ensures the chosen channel does not result in interface violations. Fig. 3 presents the pseudocode of the greedy channel selection procedure. It takes inputs the set of available channels K, the conflict vertex currently being visited v, and the two end nodes (i, j) of the link represented by v. The procedure relies on the pickMinInterferenceChannel(L) procedure which selects a channel from the given list, L. The list L is determined by the number of spare interfaces available at nodes i and j. First, if both i and j have spare interfaces, this means we are free to select any channel from K, thus L is set to K. Otherwise, the choice of channels has to be limited to those that had been assigned to i and/or j. In particular, if i (or j) has no spare interfaces, L is formed by channels that had already been assigned to the node. Otherwise, L is set to channels that are common to both nodes. In summary, the conditions make sure the chosen channel does not violate the interface constraints of both i and j. Fig. 4 depicts the working of the pickMinInterferenceChannel(L) procedure. In the procedure, each channel l ∈ L is tested against the channels assigned to each of the neighbors of v (line 5), and sum up the interference due to the channel in the neighborhood (line 6). The neighbors of v is represented by adjacency list Adj[v]. It is interesting to point out that as the vertices are being visited in a random order, the greedy channel selection procedure may not able to obtain any valid channel. Specifically, this occurs at line 11 of Fig. 3 if no common channel exists between i and j. A sample visiting order that leads to this is shown in Fig. 5. For ease of exposition, the example shows the visiting order of links in the connectivity graph (indicated by the numbers over the links). Suppose that K = 4 and all nodes have two radio interfaces. It can be seen that a different channel can be assignd to each of the links (a, b), (b, c), (d, e) and (d, f ) as the corresponding nodes have spare interfaces during the visit. By the time we visit link (b, d), both

An Immune Algorithm for Minimum Interference Channel Assignment

193

1: min ← ∞ 2: foreach l ∈ L do 3: sum ← 0 4: foreach e ∈ Adj[v] do 5: if l ≡ e.head.channel then 6: sum ← sum + 1 7: end if 8: end for 9: if sum < min then 10: min ← sum 11: c←l 12: end if 13: end for Fig. 4. pickChannelMinInterference procedure

b and d have used up all their respective interface. Since there is no common channel between the nodes, the link will not be assigned any channel. This is because assigning any channel will violate the interface constraint of either one of the nodes. Due to the above, some of the vertices may not be assigned any channel. This affects the calculation of affinity (see (2) and (4)) for a given solution. Thus, for each unassigned vertex v, a penalty value which is equal to the number of neighbors of v is added as a measure of interference. The penalty value represents that the vertex interferes with all of its neighbors, i.e. the worst case of channel assignment. Clonal selection and expansion. In this step, all individuals in the current population P is subject to a cloning process. Following [1], each individual is cloned Nc times: N  round(β · N ) (5) Nc = i=1

where β is a configurable parameter and N is the total number of individuals. Affinity maturation. In this step, each of the clones undergoes a hypermutation process in which the mutation rate α is given by α = e(−ρ∗f )

(6)

where ρ is a configurable parameter and f is the fitness function value normalized in [0,1]. The equation indicates that α is inversely proportional to the affinity, i.e. the higher the affinity, the smaller the value of α, and vice-versa. The mutation is done by a simple random mutation on the vertex visiting order. In particular, two positions of an individual are randomly chosen and their respective elements are swapped. After the mutation process, the mutated individuals are added to population P and undergone elitist selection in which the best N individuals are retained.

194

S.-W. Tan

a

e 3

1 b c

2

5

d 4 f

Fig. 5. Sample scenario where a valid channel cannot be found for link (b, d)

Metadynamics. This step models the ability of immune system in continuous production and recruitment of novel structures. This is done by replacing d lowest affinity individuals by some newly created random solutions. 3.2

Phase Two

We recall that the solution returned by the first phase (immune algorithm) may violate interface constraints. In phase two, we eliminate the interface violations by applying a “merge” procedure adopted from [7] on each unassigned conflict vertex. Let i and j denote the underlying nodes for link represented by a conflict vertex in question. The merge operation randomly picks one of the two nodes to work on. Suppose that node i is chosen. The objective is to reduce the number of channels assigned to i by one so that a new channel can be assigned to link (i, j). To begin with, two channels c1 and c2 incident on i are picked (to be described later) and all links with c1 will be changed to c2 . To prevent such a change from creating interface constraint violations at other nodes, the merging of c1 to c2 is recursively applied to all links that are “connected” to the links whose channel has been just changed from c1 to c2 . Here, two links are said to be connected if they are incident on a common node. Essentially, the propagation of the merging process ensures that for any node j, either all or none of the links incident on j with c1 are changed to c2 . It is clear that at the completion of one merge procedure, the number of distinct channels incident on i and other nodes is reduced by at most one, and no new channel is introduced. Thus, repeated application of the procedure is guaranteed to resolve all interface constraints. At the end of the procedure, a channel c3 from j is selected for use by link (i, j). Since a merge operation may result in increase in network interference, channels c1 , c2 and c3 are selected for the merge procedure such that they give the least increase in the network interference due to a complete merge operation. Fig. 6 shows the channels assignment of a sample topology before and after the merge operation. In the example, c1 = 4, c2 = 3 and c3 = 1.

4

Performance Evaluation

We compare our algorithm (refer to as IA) against CLICA [2] and the genetic based channel assignment algorithm described in [3] (GA). The performance of the algorithms is judged in terms of fractional network interference (4). Besides, we study the sensitivity of our algorithm with different choices of parameters.

An Immune Algorithm for Minimum Interference Channel Assignment

d

j

i

4

a

1 4

j

b 2 c

d

3

1

?

2

d

d

3

1

2

2

195

1

i

3

1

a

3

b c

Fig. 6. Sample topology before and after a merge operation

In the simulations, we subject the algorithms to varying number of available channels (3 to 12) and radio interfaces per node (from 2 to 4). For each pair of radio interface and channel setting, we run each algorithm over three sets of topologies of 30, 40 and 50 nodes. Each set consists of 100 random topologies (representing the connectivity graphs) with average degrees range from 3.68 to 4.44, and minimum and maximum degrees of 1 and 7 respectively. The interference to communication range ratio of the nodes is set to 2. For the results to be presented, each data point represents average calculated over results for each topology. For IA and GA, the simulation with a given topology is repeated 5 times using different random seeds. The results are plotted with 95% confidence interval. Unless specified otherwise, the algorithms use the following configurations. IA: maximum generation, maxgen = 20; population size N = 5 or N = 10; β = 1.0; ρ = 1.0 and d = 2. For GA: maxgen = 100 (refer to as GA-100) or maxgen = 500 (GA-500); N = 40; mutation rate is 0.01 and crossover rate is 0.9, following [3]. For CLICA, the inputs to the algorithms are the connectivity graph, conflict graph and the set of available channels. It has no configurable parameters. 4.1

Results and Discussions

We have observed similar performance trend in topologies of different sizes. Thus, only the results for 30-node topologies are shown in this paper. Fig. 7 depicts the performance of the algorithms for cases in which the number of radio interfaces per node are (a) equal to 3; and (b) uniformly assigned from 2 to 4, inclusively. First, it is clear that both versions of IA (N = 5 and N = 10) give the best performance for both cases. It is worth pointing out that while a large N (i.e. 10) results in better performance as expected, IA with N = 5 gives very competitive results. For example, the average performance advantage for version with N = 10 over N = 5 is merely 3.78% for case (a). The same measure over GA-100, GA-500 and CLICA is 28.27%, 21.74% and 16.81% respectively. Comparing Fig. 7 (a) and (b), we can see that the performance advantage of IA is more prominent in the case where the nodes have different number of radio interfaces. In practice, the nodes are expected to have different capabilities and have different number of radio interfaces. From both plots, we can see that as the number of channels increases from 3 to 6, the network interference for all algorithms decrease rapidly. However,

S.-W. Tan

Fractional network interference

0.18 0.16

0.06

0.14 0.12

0.16

CLICA GA-100 GA-500 IA, Ν=5 IA, Ν=10

Fractional network interference

196

0.04

0.1

9

10

11

12

0.08 0.06 0.04 0.02

0.14

CLICA GA-100 GA-500 IA, Ν=5 IA, Ν=10

0.06

0.12 0.04

0.1

9

10

11

12

5

6

7

8

0.08 0.06 0.04 0.02

3

4

5

6

7

8

9

10

Number of available channels

(a)

11

12

3

4

9

10

11

12

Number of available channels

(b)

Fig. 7. Fractional network interference performance; Graph size = 30; Number of radio interfaces per node: (a) fixed to 3; (b) uniformly assigned from 2 to 4

subsequent increase in channels results in little improvement for CLICA and IA. Increasing the channels basically increases the flexibility for the algorithms to assign new channels for improvement of the network interference. However, new channels cannot be blindly added as they may break the connectivity of the topology [2]. This explains the performance trend observed. It is interesting to point out that GA actually performs worse for larger number of channels. We believe that this is because the GA is formulated without much domain specific knowledge, hence requires more iterations to improve the solutions. This is can be seen by comparing the curves for both versions of GA (GA-100 and GA-500). Next we investigate the sensitivity of CLONALG parameters (β, ρ, d) to IA. Recall that β is the cloning factor that determines the number of clones to be generated (5), ρ controls the shape of the mutation rate (6), and d is the amount of low-affinity individuals to be replaced. We discuss the results obtained from topologies with 30 nodes, number of available channels is 6 and number of radio interfaces per node is 3. Figure 8 (a) shows the results with different values of β and ρ. It can be seen that larger values of β and ρ yield slight performance advantage. The small difference in the results suggest that our IA is rather tolerant on the choices of parameters. Equations (5) and (6) indicate that a larger value of β and ρ increases the number of clones generated and the number of mutations, respectively. As this results in higher computation cost and memory usage, a small value for both parameters (e.g. β = 1 and ρ = 1) is recommended. Figure 8 (b) shows the impacts of d. As suggested in [1], d should range from 5% to 20% of N , as very high values of d may result in a random search through the affinity landscape. The results show for IA with N = 10 in the figure indicates that a value of d from 0 to 4 yields acceptable performance. In Fig. 9, we show the convergence property of IA with N = 5 and d = 0. The curves MAX, AVG and MIN represent the maximum, average and minimum achieved affinity in each population. It is clear that IA can rapidly improve the solutions within the first few iterations. In the same figure, we plot also the maximum, average and minimum affinity for 5000 randomly generated

An Immune Algorithm for Minimum Interference Channel Assignment 0.0334 ρ=1.0 ρ=2.0 ρ=3.0

0.0348 0.0346

Fractional network interference

Fractional network interference

0.035

197

0.0344 0.0342 0.034 0.0338 0.0336 0.0334 0.0332 0.033 0.0328

0.0333 0.0332 0.0331 0.033 0.0329 0.0328 0.0327

1

1.5

2

0

Cloning factor, β

1

2

3

4

Number of random replacements, d

(a)

(b)

Fig. 8. Impacts of (a) β and ρ; and (b) number of random replacements, d on algorithm performance Fractional network interference

0.18 0.16 MAX-random AVG-random MIN-random MAX AVG MIN

0.14 0.12 0.1 0.08 0.06 0.04 0

5 10 15 Number of generations

20

Fig. 9. Fractional network interference vs generations

solutions (denote by MAX-, AVG- and MIN-random in the figure). Comparing this with IA which has a initial population of 5 individuals, it is clear that IA manages to obtain reasonably good performance after 3 generations, achieves similar performance at the 10th generation, and obtained better solutions at the 19th generation. To give a better picture on this, we count the total number of individuals generated by IA throughout the evolution process. It starts with 5 individuals. At each iteration, 25 clones are created (5 for each original individual). Thus, at the 10th generation, a total of 255 has been generated. In other words, IA is able to evolve from a small set of random solutions to reasonably good ones within small number of iterations.

5

Conclusion

We investigate an artificial immune system based algorithm for channel assignment in multi-radio wireless mesh networks. The clonal selection principle is used to evolve and improve solutions created using greedy channel assignment strategy. To limit the computational complexity and memory usage, we consider small population size (5 and 10) and small number of iterations (20). Simulation

198

S.-W. Tan

results show the algorithm performs better than a genetic-based and a graphtheoretic based algorithms. Futhermore, our algorithm is rather resilient on the choice of parameters settings, an attractive feature for practical deployment.

References 1. de Castro, L., Von Zuben, F.: Learning and Optimization using the Clonal Selection Principle. IEEE Trans. on Evolutionary Computation 6(3), 239–251 (2002) 2. Marina, M.K., Das, S.R.: A Topology Control Approach for Utilizing Multiple Channels in Multi-radio Wireless Mesh Networks. In: Proc. of IEEE International Conference on Broadband Networks (BroadNets), Boston (2005) 3. Chen, J., Jia, J., Wen, Y., Zhao, D., Liu, J.: A Genetic Approach to Channel Assignment for Multi-radio Multi-channel Wireless Mesh Networks. In: 1st ACM/SIGEVO Summit on Genetic and Evolutionary Computation, Shanghai, China, pp. 39–46 (2009) 4. Gupta, P., Kumar, P.R.: The Capacity of Wireless Networks. IEEE Trans. on Information Theory 46(2), 388–404 (2000) 5. Jain, K., Padhye, J., Padmanabhan, V.N., Qiu, L.: Impact of Interference on Multihop Wireless Network Performance. In: Proc. of IEEE/ACM MobiCom, California (2003) 6. Akyildiz, I., Wang, X., Wang., W.: Wireless Mesh Networks: A Survey. Computer Networks 47(4), 445–487 (2005) 7. Subramanian, A.P., Gupta, H., Das, S.R., Cao, J.: Minimum Interference Channel Assignment in Multiradio Wireless Mesh Networks. IEEE Trans. on Mobile Computing 7(12), 1459–1473 (2008) 8. Crichigno, J., Wu, M.Y., Shu, W.: Protocols and Architectures for Channel Assignment in Wireless Mesh Networks. Elsevier Ad Hoc Networks 6, 1051–1077 (2008) 9. Cutello, V., Nicosia, G., Pavone, M.: A Hybrid Immune Algorithm with Information Gain for the Graph Coloring Problem. In: Cant´ u-Paz, E., Foster, J.A., Deb, K., Davis, L., Roy, R., O’Reilly, U.-M., Beyer, H.-G., Kendall, G., Wilson, S.W., Harman, M., Wegener, J., Dasgupta, D., Potter, M.A., Schultz, A., Dowsland, K.A., Jonoska, N., Miller, J., Standish, R.K. (eds.) GECCO 2003. LNCS, vol. 2723, pp. 171–182. Springer, Heidelberg (2003) 10. Hertz, A., de Werra, D.: Using Tabu Search Techniques for Graph Coloring. Computing 39(4) (1987) 11. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II. IEEE Trans. on Evolutionary Computation 6, 182–197 (2002) 12. Mohsenian-Rad, A.H., Wong, V.W.S.: Joint Logical Topology Design, Interface Assignment, Channel Allocation, and Routing for Multi-channel Wireless Mesh Networks. IEEE Trans. on Wireless Communications 6(12), 4432–4440 (2007) 13. Raniwala, A., Gopalan, K., Chiueh, T.C.: Centralized Channel Assignment and Routing Algorithms for Multi-channel Wireless Mesh Networks. Mobile Computing and Communications Review 8(2), 50–65 (2004) 14. Dasgupta, D., Nino, L.F.: Immunological Computation: Theory and Applications. CRC Press, Boca Raton (2009)

A Developmental and Immune-Inspired Dynamic Task Allocation Algorithm for Microprocessor Array Systems Yang Liu1 , Jon Timmis1,2 , Omer Qadir1 , Gianluca Tempesti1 , and Andy Tyrrell1 2

1 Department of Electronics, University of York, UK Department of Computer Science, University of York, UK {yl520,jt517,oq500,gt512,amt}@ohm.york.ac.uk

Abstract. This paper presents a high level dynamic task allocation algorithm that is inspired by the biological development process and the immune system. For a microprocessor (µP) array, a program is partitioned into a number of workload oriented tasks with data dependencies and a number of internal status-oriented tasks. Each µP in the array is capable of processing one of these tasks. The algorithm assigns tasks to the µP array that satisfies the requirements of the problem, and it dynamically recovers the system from faults at runtime.

1

Introduction

Very large scale integration (VLSI) techniques are dramatically improved with state of the art manufacturing technologies with higher integration and frequency being well-known pursuits of the electronic industry. Reliability and fault tolerance are of great importance in safety-sensitive systems with on-line selfdiagnosis and self-repair capabilities being two desired properties. Fault tolerant systems are able to continue operation, despite the presence of errors in the system. By taking inspiration from the biological development process and the immune system, various artificial systems have been developed with fault tolerant capabilities. For example, work in [1] developed an on-line error detection system that afforded automated teller machines (ATM’s) a level of fault tolerance by detecting errors in sequences of states of an ATM, and achieved a mean-time-to-failure (the time between detecting an error and the error manifesting itself as a fault) of around 11 hours. However, this was a software solution, and not developed on hardware. Moving to VLSI systems, the interest of this paper, the Embryonics system [2] and its derivative, the POEtic [3] system, provided a developmental-inspired on-line error recovery mechanism. However, the overhead is too heavy to be used in real world applications. Early explorations, such as [4] and [5], developed the artificial immune systems (AIS) to “protect” the Embryonics system, but the AIS are, of course, assumed to be fault-free and there are no other mechanisms to “protect” the AIS. Notably, the total overheads in those systems are very large. E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 199–212, 2010. Springer-Verlag Berlin Heidelberg 2010

200

Y. Liu et al.

In this paper, we introduce a fault tolerance mechanism for parallel processing, specifically, a high level dynamic task allocation algorithm in microprocessor (μP) array systems which is inspired by the biological development process and the immune system. There are three major differences from the aforementioned systems. Firstly, the granularity of each node in the system is raised from logic gates up to a μP to reduce the effective overhead. Secondly, the assumption that the AIS subsystem is fault-free can be relaxed as every AIS entity is inspected by peers and also reconfigurable. Finally, in the array, fixed spare-cell columns or rows are not required, and fault tolerance is realised by means of dynamic task allocation. The three major characteristics are detailed in the following sections. In this paper, Section 2 introduces the research background and motivation of the algorithm. Section 3 presents the μP array system architecture and the algorithm, and Section 4 provides a simple demonstration. Finally, Section 5 concludes the paper and proposes the future work.

2

Background

2.1

Biological Context

In a biological context, an embryo grows to a multicellular organism by cell division and differentiation. When adult stem cells are induced into a diverse range of specialised cells, they possess the ability to renew the body tissues and replace the damaged cells. This intrinsic ability is encoded in the genome and expressed by gene-protein regulation [6]. During the development process, the immune system, is in part, responsibly for protecting the host body. However, it has been commented that the immune system is not simply a self-protection system, but a self-maintenance system [7]. Agents in the immune system not only detect invasions and generate immune responses to remove malicious agents, but also contribute to the growth and repair of the organism by the inflammation response. Cytokines, as the products of gene-protein regulation, are generated by a variety of immune cells. They affect the process of organism growth, protection, healing and apoptosis. The immune agents and activities are encoded in the genes and developed during the lifelong development process. The two biological phenomena are highly integrated through various protein and cytokine interactions [8]. 2.2

Engineering Context

Fig.1 shows the outline of a system being developed to afford a high-reliability for electronic systems. As the next generation of the Embryonics [2] and the POEtic [3] systems, such a SABRE1 system will be approached from three levels: cell, organ and organism. Cells are composed of a number of basic building blocks, called molecules. Each cell has a particular functionality, such as processing, 1

Self-healing cellular Architectures for Biologically-inspired highly Reliable Electronic systems, funded by the EPSRC under grant No. FP/F06219211.

A Developmental and Immune-Inspired Dynamic Task Allocation Algorithm

201

Fig. 1. An overview of the SABRE architecture developed as part of this work

memory and I/O. A μP is composed of a number of cells and thus considered as an organ. The whole system is a μP array and considered as an organism. Allocating tasks to the μP array is taken as an artificial development process. At each level, there are varying fault tolerance techniques. In this paper, we mainly focus on the the organism level fault tolerance. We propose that it would be advantageous to have a development-inspired growth mechanism and an immuneinspired protection mechanism to dynamically assign functionalities to the μP array. The on-line fault tolerance mechanism is maintained by an integrated framework as outlined in Fig.2. The artificial development substrate (ADS) provides intrinsic capability of self-repair, whilst the AIS detects errors and handles exceptional situations. From the whole SABRE system perspective, the development of the AIS is also a part of the ADS. The functionality of the AIS in this algorithm is self-maintenance rather than merely self-protection. The two subsystems communicate using the same information exchange pathways, which are analogous to the biological signaling materials, such as proteins and cytokines. In the next section, we describe the implementation details.

Fig. 2. Integrated self-repair mechanism

202

3 3.1

Y. Liu et al.

Proposed Self-repairing System System Architecture

A SABRE system consists of a number of μPs. There are two types of communication channels between μPs. The first is a workload channel and the second is a configuration status channel. Channels are implemented by physical buses connecting μPs. Configuration channels can have exclusive buses, or share buses with the workload channels. In the latter case, the data package of tasks are distinguished by different tags. The dynamic task allocation mechanism is inspired by two biological networks: a protein diffusion network amongst the whole organism, and a gene regulatory network within each single cell [9]. Correspondingly, in the μP array, there is an inter-μP status diffusion network and an array of configuration regulatory networks. Similar developmental-inspired architectures can be found in [10] and [11]. In figure 3 the lattice on the left shows a μP array, with each node representing a μP. There are different types of networks in the array with each μP containing an independent regulatory network, and also a node of the diffusion network. The shading of blocks in the diffusion network represents the amplitude of the influence from a μP: the brighter the shade, the stronger.

Fig. 3. Two types of networks in the SABRE array

The status diffusion network is built on the configuration status channels. Each configuration arbitrator exchanges the current status with its close neighbours through these channels. The arbitrator mechanism is implemented by a configuration regulatory network, which calculates the input information from neighbouring μPs and chooses a particular task. For each single μP, the relationship between its configuration and the status of neighbouring μPs is illustrated in Fig.4. The same table of all tasks is stored in every μP. A μP chooses to run one particular task (active), or maintains a non-configured state (dormant), according to the status of neighbouring μPs, its current internal status, and the commands from the immune-inspired managing units (details in Section 3.3). μPs are affected by each other, and thus form a self-organising network.

A Developmental and Immune-Inspired Dynamic Task Allocation Algorithm

203

Fig. 4. The task selection mechanism

3.2

Allocation Criteria

In a SABRE system, this self-organising network must reflect the design requirements of real applications. Generally, a program can be partitioned and represented in a graph G= (V, E). V represents the vertices (processor nodes) and E represents the edges (data dependencies). Given a task graph and a set of criteria, an algorithm is required to optimally allocate each task to a particular μP. This procedure can be operated off-line or on-line. An on-line dynamic task allocation mechanism is very important to system level fault tolerance, because when an error occurs in a μP, we can isolate the malfunctioning μP and dynamically assign the same task to a healthy μP. Typical criteria include, for example, size, traffic, temperature and energy. In the SABRE system, we need to place tasks with direct data dependencies as close as possible to reduce the overall traffic cost, and also, reserve an adequate amount of dormant μPs (DU) around an assigned μP (AU) as on-line back-ups. An illustration of the placement map is shown in Fig.5. Brighter blocks are the AUs and darker blocks are the DUs. Tasks should be placed according to the partitioned task graph. Generally, the distance between two AUs should be inversely proportional to the strength of the data reliance between two nodes in the task graph. The stronger the data dependency between the two nodes, the closer they should be placed and vice versa. Eq. (1) defines the traffic cost function, which calculates the overall traffic overhead.  1 Cost = · Dij (1) Wij where i and j are the indices of two nodes, W is their connection weight in the task graph, and Dij is their physical distance on the hardware plane. Minimising this cost implies that the overall traffic overhead is optimised. However, tasks should not be conjunctive as there should be a number of redundant DUs as on-line back-ups. If a part of an IC chip has a higher fault probability than elsewhere, it would be preferable to place more DUs around an AU in that area. A neighbourhood of an AU, where there are only DUs, is called the AU’s exclusive zone, which is also represented by thin dashed circles

204

Y. Liu et al.

Fig. 5. A task allocation example

in Fig.5. The approximate size (number of μPs) of the ith AU’s exclusive zone can be calculated as Ei Size ≈ M · N (2) n En where M is the total number of μPs in the array; E is a fault index of an AU, higher the index value, higher fault probability it represents. There are different methods to calculate the index value from the fault probability according to applications. Usually, all μPs have equal fault indices. By calculating the size in the area, we can estimate its radius, assuming the exclusive zone is symmetrical. We can see that the two criteria actually conflict with each other. Therefore, a practical solution is usually a compromise between Eq. (1) and (2). 3.3

Configuration Mechanism

As shown in Fig.2, the dynamic task allocation algorithm is composed of two integrated subsystems: an ADS and an AIS. The development substrate is organised by all arbitrators in the μP array, and the AIS is formed by a few μPs which are assigned immune-inspired tasks. The development of the AIS is part of the ADS, because assigning an immune-inspired task for a μP is also decided by the arbitrator. In addition, the AIS contributes to the artificial development process, when the built-in configuration network in each μP is not able to handle exceptional situations. In short, the ADS focuses on creating an optimised system and contributes to the dynamic task allocation, whilst the AIS is primarily designed for the dynamics.

A Developmental and Immune-Inspired Dynamic Task Allocation Algorithm

205

Development Substrate. Metaphorically, assigning a particular task in a prepartitioned task graph to a DU is taken as specialising stem cells during the development process, and a gene regulatory network inspired task allocation algorithm is created. In a SABRE system, each single task has a unique ID. Once a μP is configured to perform a particular task, it will broadcast the corresponding task ID to its neighbourhood, through the configuration status channels. DUs do not broadcast any ID information. A “diffusion amplitude” is used to indicate the relationship of a μP and the AUs which broadcast their task IDs. At a particular position in the array (x, y), the total amplitude C of a task j is the summation from all the neighbouring broadcasting μPs (diffusers), described in Eq. (3). Dk > 0 is the Manhattan distance from the k th diffuser. Gejk ∈ [0, 1] is the status of the k th diffuser, which is defined by later Eq. (9). βk is a scaling parameter.  Gejk · βk (3) Cj (x, y) = Djk k

A closer look of the configuration regulatory network is illustrated in Fig.6, where solid lines represent excitatory impacts and dashed lines represent inhibitory impacts. We see that, on the left, there are N receivers to capture their matching IDs, and N transmitters on the right to broadcast IDs, where N denotes the total number of pre-partitioned tasks. Each receiver or transmitter only associates to a particular task ID. For each μP, the diffusion amplitudes of tasks must satisfy two conditions and then trigger a series of actions. Conditions Gpi and Gni , as shown in Fig.6, are defined by Eq. (4) and (5). ⎧ ⎪ Ci  0 ⎨0 Ci Gpi = T pi , 0 < Ci < T pi (4) ⎪ ⎩ 1, Ci  T pi

Fig. 6. A sketch of the configuration regulatory network

206

Y. Liu et al.

Gni =

⎧ ⎪ ⎨0,

Ci −T pi , ⎪ T ni

⎩

1,

Ci  T pi T pi < Ci < T ni Ci  T ni

(5)

T pi and T ni are input activation thresholds. Higher T p value causes the AUs to be assigned closer and higher T n value pushes the AUs away from each other. T p and T n together decide the minimal radius of an AU’s exclusive zone. For each input Ci , we apply the amplitudes of a set of intermediate values, denoted as ej , j = 1, 2, ..., N . Eq. (6) and (7) describe the individual contributions Δeij by Ci to the changes of the intermediate values Δej .  Gni · αij , αij  0 Δeij = (6) (Gpi − Gni ) · αij , αij > 0  Δej =

0, N i

∀Gej=i > 0 Δeij , else

(7)

where αij is a scalar parameter. Gej is the output status, which is described by Eq. (9). If any Gej=i > 0, then Δej = 0, which means ej can not increase. This is an inhibition back-propagation mechanism to prevent more than one tasks status from being activated. The amplitude of ej is accumulated by time, as: ej (t) = max{ ej (t − 1) + Δej − kj , 0 }

(8)

where t is the time step and kj is a decrement rate. The status Gej , shown in Fig.6, is defined by ⎧ ej  T ej ⎪ ⎨0, ei −T ej Gej = (9) T c j , T e j < e i < T cj ⎪ ⎩ 1, e j  T cj where T ej and T cj are two output activation thresholds. When ej > T ej is satisfied, this μP will be configured with a task and then broadcast its ID. Therefore the corresponding diffusion amplitude of the μP array must be updated by Eq. (3). Additionally, this feeds back to the ej=i value and thus guarantees that in each μP no more than one task status Ge can be activated. Therefore, a μP can be configured as only one of the predefined tasks, or as being dormant. Note that every μP receives and processes the ID information, but only the AUs are able to broadcast the ID information. The input of the configuration regulatory network is from the status diffusion network, and the input of the whole status diffusion network is from the pre-settings of the system. Specifically, for instance, the I/O tasks have to be initialised at particular μPs, because usually the I/O positions of an integrated circuit (IC) are fixed by the layout of the printed circuit board (PCB). After the system is launched, these μPs start to affect other μPs, and more and more AUs emerge, like a “growth” of the SABRE system.

A Developmental and Immune-Inspired Dynamic Task Allocation Algorithm

207

Immune Maintenance. According to the criteria introduced in Section 3.2, ADS parameters are optimised. A SABRE system is expected to have an optimised layout of the task allocation. Usually, when an AU is faulty at runtime, the ADS will reallocate the same task to a back-up DU, according to the current local diffusion amplitudes on a μP. The same is true for the launching process. However, this cannot be guaranteed, because our simple optimisation procedure is not prepared for the complete set of fault-recovery sequences. For instance, suppose there are N μPs and M AUs. AUs become faulty one after another, and at each time, a new AU will emerge to replace the failed AU. Maximally, there are N M−N possible fault-recovery sequences (with length of M − N ). Therefore, training parameters for a complete set of the dynamic responses is practically impossible. Inevitably, we will face a situation that the local diffusion amplitudes on a μP are unable to satisfy the rules of the regulatory network, and the ADS based dynamic task allocation mechanism fails. In order to improve the system stability, the AIS in this algorithm provides the ADS with an extra on-line assistance.This mechanism is inspired by the inflammatory response of the innate immune system [8]. The innate immune system senses the anomaly in the organism, removes the injurious stimuli, and initiate the healing process, which is part of the lifelong development process as being encoded in the genes. During inflammation, various specific cytokines and chemokines are generated to induce adult stem cells to be differentiated and replace the injured tissues. Analogously, we expect that the AIS not only actively monitor the health status of the AUs using conventional on-line diagnosis technique, but more importantly, the AIS will also be able to passively recognise anomalous events when an AU fails and contribute to the recovery. Namely, the AIS should help reallocate the missing task to a DU. Based on these considerations, we divide the AUs into two types. One is workload-oriented and the other is status-oriented. The workload-oriented AUs are called function units (FU), and the status-oriented ones are called immune units (IU). As the name suggests, the IUs are inspired by the immune system and designed to deal with exceptional situations of the artificial development process. In performing different applications, tasks running on the FUs may be different, because they are required to process different work-load using different methods; but tasks running on the IUs are all the same, because they are only concerned about the internal status of the μP array system. The operation of IUs is memory based, which is primarily inspired by the innate branch of the immune system. A memory table records the anomalous events and their corresponding responses. The format of an entry is shown in Table 1, where the reporter is an affected AU. After an AU is faulty, if there is no replacement AU emerging and thus compensating the change of diffusion Table 1. The format of the anomaly-response table

Anomaly report Reporter position Its current status

Corresponding response Target position Assigned task ID

208

Y. Liu et al.

amplitudes, other AU(s), which have data dependencies with the faulty AU, will be adversely affected. An affected AU reports its current status to the IUs. The IUs receive this report and look up the anomaly-response table for a solution. If there is a corresponding entry, the IUs will cooperatively allocate the required task to a target DU and drive the system back to a new stable state. Notably, only when a DU receives more than one IUs command, it will be initiated to perform the required task. This table is predefined and deterministic, and thus the response is very fast. One of the simplest ways to create such a table is to inject faults to every AU sequentially and check the diffusion amplitudes every time, until an anomalous event is triggered. As discussed, when the scale of the system increases, the number of possible sequences will exponentially increase. So, a statistical sampling method, with respect to the fault probability, would be applied to reduce the computational overhead. The higher fault probability, the higher priority that an AU has to be tested in the sequence. It is worth mentioning that IUs not only monitor FUs, but also themselves. This mechanism has two advantages over the conventional centralised inspection mechanism. Firstly, μPs are monitored by more than one IU, and thus error detections are performed using a voting system. IUs work in a decentralised and collective way, so that they can avoid false positive diagnoses, which have lower chance to happen simultaneously to all IUs than any single one. Moreover, this mechanism prevents hierarchical inspections. There is no need to presume the monitor on top of the inspection hierarchy being fault free. In other words, when an IU fails, the system will re-create a new one, by the cooperation of the ADS and the AIS. Moreover, the ADS regulatory network in each μP is independent from the AIS anomaly-response table. Therefore, the AIS is designed separately after the ADS is optimised. This extra protection can effectively simplify the parameter tuning processes, because even sub-optimised parameters are potentially able to maintain the system stability to an extent.

4

Demonstration Experiments

The basic idea of this test is to demonstrate the self-maintenance capability of the AIS in a SABRE system by comparing the number of recovery times of the ADS alone and the ADS with the help of the AIS. The hypothesis of the test is that the system with the AIS has a higher number of recoveries than the other. 4.1

Experimental Design

In this test, we allocate an example task graph, as shown in Fig.7, onto a 5×5 μP array. There are five workload-oriented tasks, which are denoted as I, O, A, B and C, and one status-oriented task, denoted as M. Among them, tasks I and O, as an input and an output of the system, are pre-allocated to two μPs on the edges. By optimising the parameters in the ADS for the given pre-partitioned task graph, the remaining four tasks, A, B, C and M were expected to emerge

A Developmental and Immune-Inspired Dynamic Task Allocation Algorithm

209

Fig. 7. A partitioned task graph

from this initial setup. We then created an anomaly-response table in the AIS by randomly injecting a set of fault sequences. Both of them are operated off-line at design time. Training the ADS. The parameters in Eq. (3) to (9) are tuned according to the criteria defined by Eq. (1) and (2). In practice, T p and T n values are predefined according to Eq. (2), and assumes the fault probability for every μP is the same. Also, we can use the connection weights Wij to replace the αij directly, which reflects the dependencies of two connected nodes, so that the number of tuned parameters is reduced to O(N ). All parameters are defined to be integer from -128 to 127 to suit the minimum hardware specifications of an 8-bit μP. In this application we used a simple genetic algorithm without crossover to tune the parameters. The population is 10 and the maximum generation is 10,000. From 100 independent runs, 3 table layouts are found and shown in Fig.8. It is possible that, if increasing the precision of those tuned parameters, more stable layouts may be found, but the hardware and computational overhead will also be increased.

MA C I C MA

BM O BM

(a) Layout 1

M AC I AC M

B B

M

M

O

I

M

M

(b) Layout 2

CA CA

M B O B M

(c) Layout 3

Fig. 8. Three stable solutions

We can see that, in Fig.8, tasks are all assigned symmetrically. Except for the input and output tasks, all other tasks have at least two copies in the array. Specifically, task M is assigned to four μPs, and so there are four IUs in the system. Each copy of a task can be seen as a hot back-up. Hot back-ups are created as a side-effect of the algorithm, and provide further fault tolerance to the system, because even if the task on a faulty AU is not able to be reallocated, there is at least one hot back-up still working.

210

Y. Liu et al.

Training the AIS. The anomaly-response table was then created by artificially injecting a fault to an AU one at a time. After a given period of recovery time, if there exists at least one copy for each task, we consider the system is still working. In theory, for this 12-AU and 25-μP system, there are 1213 independent fault injection sequences, with length of 13, and the system should have 1213 ×13 entries in the table. This theoretical number is based on an assumption that the system has no other self-repair mechanisms. However, we have the ADS as an intrinsic self-repair mechanism, which reduces the AIS memory usage. For simplicity, we randomly selected 50 fault injection sequences with length 5 and recorded the anomaly-response table. Thus, the maximum length of the table is 250, which suits the requirement of an 8-bit μP. It should be mentioned that in this table, contents sometimes conflict with each other. However, this is not explored in this paper. 4.2

Results and Analysis

After training, another 100 random fault injection sequences with length 5 are applied on the 3 optimised (or sub-optimised) layouts to simulate the runtime situations. Based on the same 100 random sequences, the self-maintenance capability, by the ADS+AIS and the ADS alone (as a contrast), were separately tested. In order to intuitively display this fault injection method, a series of snapshots of the two systems, based on layout 1, are shown in Appendix A. The number of times that each layout of the system survives the fault injections are listed in Table 2. The AIS memory usages are also attached, in terms of the number of entries in the anomaly-response table. Table 2. Performance comparison

ADS (times) ADS+AIS (times) Memory (entries)

Layout 1 62 81 45

Layout 2 25 64 226

Layout 3 38 73 171

We can see that, in vertical comparisons, the ADS+AIS does provide the system higher fault tolerance than the ADS alone, which confirms our hypothesis. Whilst in horizontal comparisons, the system in layout 1 was recovered more times with less memory. This is probably because in this layout, the ADS had already provided the system with higher fault tolerance, so that the AIS did not need to store too many records of exceptional situations. We propose that the higher fault recovery capability of an ADS on its own, the less memory the AIS requires, and vice versa. We also noticed that in terms of the overall performance of the ADS+AIS, the three layouts have rather similar recovery capability, compared to only using the ADS. Therefore, we have good reason to believe that the AIS could reduce the burden of parameter optimisation. Sub-optimised solutions could have similar performance to the optimised ones, if the system provides a large enough memory

A Developmental and Immune-Inspired Dynamic Task Allocation Algorithm

211

space. This again turns back to the tradeoff between the computation-intensive and the memory-intensive implementations. This should be taken into serious consideration in different applications.

5

Conclusion and Future Work

In this paper, we presented a dynamical task allocation algorithm for the SABRE systems. This algorithm is under a self-maintenance framework, inspired by the biological development process and the innate immune system, specifically the inflammatory response. Generally, the ADS and the AIS cooperatively maintain the system stable state. Differing from previous work, no fault-free units are assumed. In the future, an adaptive immune system inspired mechanism will be developed. Generally, these adaptive immune-inspired operations will be computationintensive. Our adaptive AIS, will focus on the runtime situations which are hard to predict at design time, and will resolve conflict contents in the anomalyresponse table. Thereafter, a full comparison with the performance of a standard allocation system will be another essential work.

References 1. De Lemos, R., Timmis, J., Forrest, S., Ayara, M.: Immune-inspired adaptable error detection for automated teller machines. IEEE Transactions on Systems, Man and Cybernetics - Part C 37, 873–886 (2007) 2. Ortega, C., Mange, D., Smith, S., Tyrrell, A.M.: Embryonics: A bio-inspired cellular architecture with fault-tolerat properties. Genetic Programming and Evolvable Machines 1(3), 187–215 (2000) 3. Tempesti, G., Roggen, D., Sanchez, E., Thoma, Y., Canham, R., Tyrrell, A.: Ontogenetic development and fault tolerance in the POEtic tissue. In: Tyrrell, A.M., Haddow, P.C., Torresen, J. (eds.) ICES 2003. LNCS, vol. 2606, pp. 141–152. Springer, Heidelberg (2003) 4. Canham, R.O., Tyrrell, A.M.: A hardware artificial immune system and embryonics array for fault tolerant systems. Genetic Programming and Evolvable Machines 4, 359–382 (2003) 5. Zhang, X., Dragffy, G., Pipe, A., Zhu, Q.: Artificial innate immune system: An instant defence layer of embryonics. In: Nicosia, G., Cutello, V., Bentley, P.J., Timmis, J. (eds.) ICARIS 2004. LNCS, vol. 3239, pp. 302–315. Springer, Heidelberg (2004) 6. Gerhart, J., Kirschner, M.: Cells, embryos, and evolution: toward a cellular and developmental understanding of phenotypic variation and evolutionary adaptability. Blackwell Science, Malden (1997) 7. Cohen, I.R.: Tending Adam’s garden: evolving the cognitive immune self. Elsevier Academic Press, Amsterdam (2000) 8. Abbas, A., Lichtman, A., Pillai, S.: Cellular and Molecular Immunology, 6th edn. Saunders Elsevier, Philadelphia (2007) 9. Haddow, P.C.: Evolvable hardware: A tool for reverse engineering of biological systems. In: Hornby, G.S., Sekanina, L., Haddow, P.C. (eds.) ICES 2008. LNCS, vol. 5216, pp. 342–351. Springer, Heidelberg (2008)

212

Y. Liu et al.

10. Gordon, T.G.W., Bentley, P.J.: Development brings scalability to hardware evolution. In: NASA/DoD Conference on Evolvable Hardware, pp. 272–279. IEEE Computer Society, Los Alamitos (2005) 11. Trefzer, M.A., Kuyucu, T., Miller, J.F., Tyrrell, A.M.: A model for intrinsic artificial development featuring structural feedback and emergent growth. In: IEEE Congress on Evolutionary Computation, pp. 301–308 (2009)

A

Visual Illustration

The fault injection sequence is: A → B → M → O → C → C (marked with a cross). Fig.9 shows the recovery capability of the ADS alone. In Fig.9 (b), (d), (f) and (h), after the injected fault, a new AU emerged from the neighbourhood of the faulty FU or IU. However, from Fig.9 (j) to (l), task C was not reallocated after a next fault injection. Consequently, the FUs on task A were not able to maintain their corresponding status. The whole system finally failed. MA C I C MA

BM

M BM C A I O C MA BM

M BM C A I O C MA BM

M M C A B I O C MA BM

M M C A B I O C MA BM

M C AMB I O C MA BM

(a) t=0

(b) t=10

(c) t=20

(d) t=30

(e) t=40

(f) t=50

M C AMB I O C MA BM

M C AMB I C O MA BM

M C AMB I C O MA BM

(g) t=60

(h) t=70

(i) t=80

O BM

M I C MA

AMB O BM

(j) t=90

M I MA

AMB O BM

(k) t=100

M I M

MB O BM

(l) t=110

Fig. 9. Intrinsic fault tolerance capability of the ADS

In contrast to the ADS-only system, the recovery capability of the ADS+AIS system is shown in Fig.10. Given the same scenario, in Fig.10 (c), highlighted IUs on task M received the anomaly report from FUs on tasks A. The latter are adversely affected by the missing task C. IUs cooperatively decide to assign task C to a particular DU, as shown in Fig.10 (d). The top-right IU did not receive the report because it was isolated by two faulty neighbours which were unable to route the information. M C AMB I C O MA BM

(a) t=80

M I C MA

AMB O BM

(b) t=90

M I MA

AMB O BM

(c) t=95

M I C MA

AMB O BM

(d) t=100

Fig. 10. System recovery on the ADS+AIS

An Immunological Algorithm for Doping Profile Optimization in Semiconductors Design Giovanni Stracquadanio, Concetta Drago, Vittorio Romano, and Giuseppe Nicosia Department of Mathematics and Computer Science University of Catania Viale A. Doria 6, 95125, Catania, Italy {stracquadanio,drago,romano,nicosia}@dmi.unict.it

Abstract. The doping profile optimization in semiconductor has been tackled as a constrained optimization problem coupled with a driftdiffusion model to simulate the physical phenomenon. In order to design high performance semiconductor devices, a new immunological algorithm, the Constrained Immunological Algorithm (cIA), has been introduced. The experimental results confirm that cIA clearly outperforms previous state-of-the-art algorithms in doping profile optimization.

1

Introduction

Designing micro-electronic devices is a complex process, which takes into account increasing frequency and bandwidth ranges, small size factor, high reliability and low power consumption [1]. By inspecting the field of semiconductor devices, a crucial step is represented by the doping process; it introduces impurities into an intrinsic semiconductor in order to change its electrical property. In general, the doping is performed in order to obtain a current gain at prefixed voltage levels; it is important to note that is not possible to arbitrarily modify the doping, since high doped regions can degenerate the devices and producing a behaviour that is similar to a conductor. In this research work, we focus on finding an optimal doping profile of a semiconductor device in order to obtain a current amplification; a gained current Jg can be obtained is obtained by slightly changing the doping profile. In order to produce feasible devices, we introduce a constraint on the doping, by allowing a prefixed deviation from a reference feasible doping profile. For this reason, the doping profile optimization has been tackled as a constrained optimization problem (COP) [6]; in particular, we minimize the current gain by allowing at most θ deviation from the reference doping profile. Due to the non-linearity of the constraint and difficulty in approximating derivatives information, a new black-box immune optimization algorithm, called Constrained Immunological Algorithm(cIA), has been introduced; the algorithm mimics the clonal selection mechanism of the immune system, by iteratively E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 213–222, 2010. c Springer-Verlag Berlin Heidelberg 2010 

214

G. Stracquadanio et al.

producing high quality solutions using cloning and hyper-mutation operators. Moreover, the algorithm is designed for selecting solution inside or close to the feasible region, in order to assure an asymptotic convergence to feasible solutions. In our experiments, we taken into account a P-N silicon diode; we adopt a drift-diffusion model for device simulation [2–4], which is a model that combines an accurate description of physical phenomenon with a low computational cost [5]. Successively, it assesses the performance of the cIA algorithm with evolutionary and classical optimization algorithms in Electronic Design Automation [7–9]. The results confirm that cIA clearly outperforms the other approaches in terms of quality of the solutions. Moreover, we prove experimentally that the immune algorithm is able to define the feasible region of faced optimization problem, without using any information on the problem itself. The paper is organized as follows; in section we present the cIA algorithm; in section 2 and then we introduce the drift-diffusion model in section 3; finally, in section 4, we show the experimental results and in section 5 we outline conclusions and future works.

2

The Immunological Algorithm

In order to tackle effectively the optimization of doping profiles, we design a new Constrained optimization Immunological Algorithm, called cIA. The proposed approach is a stochastic black-box optimization algorithm inspired on the clonal selection principle of the Immune System (IS) [14]: a problem is an antigen and a candidate solution is a B-cell. The affinity between an antigen and a B-cell is given by the objective function of the optimization problem. Each B-cell is a vector of n real values, where n is the dimension of the problem. Each candidate solution has associated an age τ ; it indicates the number of iterations since the last successful mutation, and, initially, it is set to zero. An initial population P (0) of dimension d is randomly generated, with each variable constrained in the bounds. However, it could be useful to use an ad-hoc population to start the optimization process; cIA can take in input a starting point pst , and it use this point to initialize one B-cell of the population and the remaining d − 1 with perturbation of pst . The algorithm is iterative; each iteration is made of a cloning, mutation and selection phase. The algorithm stops when the maximum number of fitness function evaluations is reached. By inspecting the pseudo-code in Alg.1, the cloning phase is responsible for the production of copies of the B-cells. Each member of the population is cloned dup times producing a population Pclo of size d × dup, where each cloned Bcell takes the same age of its parent; simultaneously, the age of the parent is increased by one. After the Pclo population is created, it undergoes to mutations in order to find better solutions; in this phase, the hyper-mutation and hyper-macromutation are applied to each candidate solution. Firstly, the hyper-mutation operator mutates a randomly chosen variable xi of a given B-cell using a self-adaptive Gaussian

An Immunological Algorithm for Doping Profile Optimization

215

Algorithm 1. Pseudo-code of the Constrained optimization Immunological Algorithm (cIA). 1: procedure cIA(d, dup, τB , ρ, β, sa ) 2: t←0 3: BCarch ← Create Archive(sa ) 4: P (t) ← Initialize(d) 5: Evaluate(P (t)) 6: while ¬Stop Condition() do 7: Pclo ← Cloning(P (t) , dup) 8: Phyp ← Hypermutation(Pclo, ρ) 9: Pmacro ← HyperM acromutation(Phyp, β) 10: Evaluate(Pmacro ) 11: EvaluateConstraints(Pmacro) 12: Aging(P (t), Pmacro , τB ) 13: P (t+1) ← Selection(P (t) , Pmacro , BCarch ) 14: t←t+1 15: end while 16: end procedure

mutation computed as xnew = xi + σN (0, 1), where σi = σi exp((τ N (0, 1)) + i  (τ Ni (0, 1)). Successively, the hyper-macromutation applies a convex perturbation to a given solution by setting xnew = (1 − γ)xi + γxk , where xi is a variable i randomly chosen such that xi = xk , with γ ∈ [0.1] a uniformly distributed random variable. Since variables xi and xk typically have different ranges, the value xk is normalized within the range of xi using the following equation: = Li + xnorm k

(xk − Lk ) × (Li − Ui ) (Uk − Lk )

(1)

where Li , Ui are the lower and upper bounds of xi and Lk , Uk are lower and upper bounds of xk . The value used to mutate the xi variable is xnorm . k These mutation operators are controlled by specific mutation rate α; for the hyper-mutation, we define α = e−ρf , instead for the hyper-macromutation we adopted α = β1 e−f , where f is the fitness function value normalized in [0, 1]. These operators are applied sequentially; the hyper-mutation operator acts on the Pclo producing a new population Phyp . The hyper-macromutation mutates Phyp generating the Pmacro population. After mutations, the population Pmacro is evaluated; if a B-cell achieves a better objective function value, its age is set to zero otherwise it is increased by one. The aging operator is applied on P (t) and Pmacro ; it erases B-cells with an age greater than τb + 1, where τb is a parameter of the algorithm. The deleted B-cells are saved into the archive BCarch ; since the archive contains at most sa solutions, if there is enough space, the B-cell is put into the first available location, otherwise it is put in a random location. Finally, the selection is performed and the new population P (t+1) is created by picking the best individuals from the parents and the mutated B-cells; however,

216

G. Stracquadanio et al.

if |P (t+1) | < d, d − |P (t+1) | B-cells are randomly picked from the archive and added to the new population. In many real world applications, it is common to deal with constraints, which could be imposed on input and output value. In general, a constraint is a function g(x) that certificates if a solution for a given optimization problem is feasible or not. We consider constraints defined as g(x) : Rn → R if g(x) ≤ θ, where θ is a feasibility threshold. A solution x is feasible respect an inequality constraint h(x) : Rn → R if h(x) ≤ θ, where theta is a feasibility threshold. The algorithm considers the constraint values during the selection procedure. Given two individuals p1 , p2 , if both are feasible the one with the lowest objective function value is picked; if p1 is feasible and the p2 is unfeasible, p1 is chosen, otherwise if p1 and p2 are unfeasible the one with the lowest constraints violation is selected.

3

The Drift Diffusion Model

The stationary standard Drift Diffusion model for semiconductor devices enclosed in a bounded domain Ω ⊂ Rd , d ∈ {1, 2, 3} [10] is given by the following balance equations for the electron density n and hole density p, coupled to the Poisson equation for the electrostatic potential V : Jn Jp divJn divJp ΔV

= q(Dn ∇n − μn n∇V ) = −q(Dp ∇p + μp p∇V ) = qR(n, p) = −qR(n, p) = q(n − p − C)

(2)

where C denotes the doping concentrations, Jn and Jp the current densities of electrons and holes, respectively. The parameters Dn , Dp , μn , μp denote the diffusion coefficients and the mobilities of electrons and holes respectively. The physical constants are the elementary charge q and the materials permittivity constant . The total current density is given by J = Jn + Jp . In the model, generation-recombination processes are included via the recombination rate R : R2 → R; in particular, we adopt the Shockley-Read-Hall term [11]: RSRH =

np − n2i τ (n + ni ) + τn (p + ni )

(3)

where the physical constants are the carrier life times τn and τp and the intrinsic density ni . Successively, we consider only regimes in which we can assume the validity of the Einstein relations: Dn = UT μn

(4)

Dp = UT μp

(5)

where UT = KB TL /q is the thermal voltage at the temperature T and KB denotes the Boltzmann constant. In high field applications, the mobilities depend on the electric field E = −∇V , as stated by the Canghey-Thomas relations [10]:

An Immunological Algorithm for Doping Profile Optimization

⎡ μn,p = μ0n,p ⎣1 +



μ0n,p |E| vs

1 2 ⎤− 2 ⎦

217

(6)

where μ0n , μ0p stand for the mobilities of the field-independent scattering models and vs is the saturation velocity. In order to obtain a well posed problem, the system (2) needs appropriate boundary conditions; in particular, we assume that the boundary ∂Ω splits into two disjoint parts ΓN ΓD , where ΓD models the Ohmic contacts of the device and ΓN the insulating parts. Let ν denotes the outward normal vector along the boundary. By assuming charge neutrality and thermal equilibrium at the Ohmic contacts ΓD and zero current flow and vanishing electric field at the insulating part ΓN , we obtain the following boundary conditions: cn = nD , p = pD , V = VD ∇nν = ∇pν = ∇V ν = 0

(7) (8)

where n = nD , p = pD , V = VD are the following H 1 (Ω)-extensions on ΓD :  C + C 2 + 4n2i cnD = (9) 2 −C + C 2 + 4n2i pD = (10) 2

nD VD = −UT log +U (11) ni where U denotes the applied voltage. Successively, we assume that the device is operating near thermal equilibrium, thus we assume that no generationrecombination effects are present and the mobilities μ0n , μ0p are kept constant. Moreover, in order to obtain dimensionless equations, we perform the following scaling: ˜ , p → Cm p˜, x → L˜ x cn → Cm n C μ qU T m 0 ˜ ˜ V → UT V˜ , Jn,p → C → Cm C, Jn,p L

(12) (13)

where L denotes a characteristic device length, Cm the maximal absolute value of the background doping profile and μ0 a characteristic value for the mobilities. Successively, we introduce the Debye length λ2 , and we obtained the following scaled equations [12]: cΔn − div(n∇V ) = 0 Δp + div(p∇V ) = 0

(14) (15)

λ2 ΔV = n − p − C

(16)

218

G. Stracquadanio et al.

The new set of scaled equations leads to the following modified Dirichlet boundary conditions on ΓD [13]: √ C + C 2 + 4δ 4 (17) cnD = √2 −C + C 2 + 4δ 4 pD = (18)  2n D VD = −UT log +U (19) δ2 where δ 2 = ni /Cm denotes the scaled intrinsic density.

4

Experimental Results

Let Γ0 be a portion of the Ohmic contacts ΓD at which we can measure the total current density J. The objective is the attainment at Γ0 of a gained current Jg ¯ The reference doping by allowing deviations from a reference doping profile C. ¯ profile C can be approximated by the following Gaussian distribution:

α x2 ¯ ¯ √ C= exp − 2 − N (20) 2σ 2πσ where α is the amplitude of the curve and σ the relative standard deviation, ¯ is a shifting constant useful to put the junction of the N-P regions at the and N origin of the axis. It is straightforward to note that we have to find the values of α, σ that achieved a gained current value and differs at most θ from the reference doping profile. We define the current gain error Jerr as follows:

2 Jerr = Jds − Jg (21) Γ0

where J is the total current density and Jg is target gained current. Successively, we define the doping profile error Derr as follows:

2 ¯ Derr = C − Cdx (22) Ω

where C is the current doping profile and C¯ is the reference doping profile. The problem can be formulated as a constrained optimization problem which objective is to minimize the current gain error Jerr , with a constraint on doping profile error Derr ≤ θ. In our experiments, we focused on a P − N silicon diode, 1μm long and the a bias voltage Vbias = 0.259V . The reference current J¯ = 7.033 × 10−4 is obtained by setting α = 5 and σ = 0.6; in our experiments, we set the bounds as follows α = [1, . . . , 10] and σ = [0.1, . . . , 1]. The state system is discretized by a variant of the exponentially fitted Scharfetter-Gummel model on a uniform grid of 101 points [11]. We try to identify three doping profiles that

An Immunological Algorithm for Doping Profile Optimization

219

Table 1. Doping profile optimization at current gain J¯ = 50% and Vbias = 0.259V . For each algorithm, we report the expected output current (Jerr ) and the doping error from the reference profile (Derr ); in bold face, the best results. Algorithm cIA cIA

Jerr

Derr −15

3.738 × 10 1.212 × 10−10

RGA [15] 1.245 × 10−10 CRS [15, 16] 1.168 × 10−10 DiRECT [15] 1.066 × 10−10

2.031 × 10−1 1.988 × 10−1 2.175 × 10−1 2.050 × 10−1 2.182 × 10−1

achieves a gain of 50%, 75%, 100% respect the reference current. In order to set a meaningful threshold on doping profile deviation, we compute Derr values of the best solution found in [15] and we fix θ to its double; this setting is justified because we do not want to prevent the algorithm to explore promising regions For the cIA algorithm, we use a population d = 10, dup = 2, β = 1, ρ = 6, τb = 15, sa = 103 and 2 × 103 objective function evaluations; preliminary experiments have been done, on numerical functions, for tuning the algorithm. We compare the new approach with the profiles obtained by Controlled Random Search (CRS,[7, 16]), Divide Rectangle (DiRECT, [8]) and the Real Coded Genetic Algorithm (RGA,[9]). In table 1, we report the results on the doping profile optimization for a gain of 50%; it is possible to note that cIA clearly outperforms the previous state-ofthe-art optimization algorithms. The proposed solutions reports the best Derr value, and it achieves a Jerr value that is five order of magnitude better than the other approaches; it is possible to conclude that only cIA is able to achieve the prefixed gained current (Fig.1). Successively, using the same settings, we apply cIA to the doping profile optimization for a gain of 75% and 100%; in table 2, it is possible to note that the algorithm is able to find a doping profile configuration that reaches the prefixed current gain in both cases. In order to understand the impact of Vbias on doping profile optimization, we try to achieve a gained current of 50% with Vbias ∈ {0.520V, 0.780V, 1.0V }; in table 3, it is possible to note that we obtain good doping profile at different Vbias values. Table 2. Doping profile optimization at current gain J¯ ∈ {75%, 100%} and Vbias = 0.259V . For each algorithm, we report the expected output current (Jerr ) and the doping error from the reference profile (Derr ). Gain 75% 100%

Jerr

Derr −15

7.281 × 10 2.159 × 10−15

2.724 × 10−1 3.220 × 10−1

220

G. Stracquadanio et al.

Table 3. Doping profile optimization at current gain J¯ = 50% and different Vbias Vbias

Jerr

Derr −28

4.353 × 10−1 2.647 × 10−1 1.834 × 10−1

9.910 × 10 5.447 × 10−15 1.602 × 10−13

0.520V 0.780V 1.000V

Doping

p

1

1.6 Optimized Reference

0.5

Optimized Reference

1.4 1.2

0

1

-0.5

0.8 0.6

-1

0.4 -1.5

0.2

-2

0 0

20

40

60

80

100

0

20

40

n 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

16 14 12 10 8 6 4 2 0 -2 -4 -6

Optimized Reference

0

20

40

60

80

100

60

80

100

V

60

80

100

Optimized Reference

0

20

40

Fig. 1. Behaviour of the best doping profile found by cIA for current gain of 50% and Vbias = 0.259V in the dimensionless case Gain = 75%

Gain = 100%

4e-07

4e-07

3.5e-07

3.5e-07

3.5e-07

3e-07

3e-07

3e-07

2.5e-07

2e-07

1.5e-07

2.5e-07

Current Gain Error

Current Gain Error

Current Gain Error

Gain = 50% 4e-07

2e-07

1.5e-07

2.5e-07

2e-07

1.5e-07

1e-07

1e-07

1e-07

5e-08

5e-08

5e-08

0

0 0 0.1 0.2 0.3 0.4 0.5 Doping Error

0 0 0.1 0.2 0.3 0.4 0.5 Doping Error

0 0.1 0.2 0.3 0.4 0.5 Doping Error

Fig. 2. Feasible regions of doping profiles for current gain at {50%, 75%, 100%} and Vbias = 0.259V . In the x-axis, we report the doping profile error Derr , instead in the y-axis the current gain error Jerr .

An Immunological Algorithm for Doping Profile Optimization

221

Since the feasible region of faced optimization problem is not known and it is hard to mathematically define it, we tried to explore it experimentally using the cIA algorithm. In particular, we consider all the feasible solutions founds by cIA and we plot them in figure 2; the plot shows a clear conic-like shape, and if we consider the Jerr and Derr optimal values obtained by cIA we can note that they resides exactly in the bottom of the conic sections. In order to verify the experimentally obtained shape of this region, we perform a classical Monte-Carlo sampling using 104 trials generated according to a normal distribution; also in this case, the feasible region shows a conic-like shape, and it seems to confirm that cIA is able to effectively explore and exploit the feasible region.

5

Conclusion

Determining the doping profile of semiconductor is a complex design problem, both for the modelling and optimization aspects. In this research work, we have tackled this computational design task as a constrained optimization problem, where the devices are simulated using a drift diffusion model and the optimization is demanded to a new algorithm called constrained Immunological Algorithm (cIA). cIA mimics the Clonal Selection mechanism of the immune system with the aim of producing a high quality solution trough cloning and hypermutation; the immunological scheme has been extended in order to take into account black-box constraints, using a selection strategy that forces the algorithm towards the feasible region of the search landscape. The experimental results on a P-N diode show that cIA clearly outperforms the state-of-the-art optimization algorithms in terms of quality of solutions and convergence speed; moreover, the analysis of the candidate solutions produced during the convergence process confirms the ability of cIA of approximating with high accuracy the feasible region of the solution space. Supplementary Material. More numerical results are available at: http://www.dmi.unict.it/~stracquadanio/eda.html

References 1. Anile, A.M., Cutello, V., Nicosia, G., Rascuna, R., Spinella, S.: Comparison among evolutionary algorithms and classical optimization methods for circuit design problems. In: IEEE Congress on Evolutionary Computation, Edinburgh, vol. 1, pp. 765–772. IEEE Press, Los Alamitos (2005) 2. J¨ ungel, A.: Quasi-hydrodynamic semiconductor equations. Birkh¨ auser, Basel (2001) 3. Degond, P., Gallego, S., M´ehats, F.: An entropic quantum drift-diffusion model for electron transport in resonant tunneling diodes. J. of Computational Physics 221(1), 226–249 (2007) 4. El Ayyadi, A., Jungel, A.: Semiconductor simulations using a coupled quantum drift-diffusion Schrodinger-Poisson model. SIAM J. on Applied Mathematics 66(2), 554–572 (2006)

222

G. Stracquadanio et al.

5. Burger, M., Pinnau, R.: Fast optimal design of semiconductor devices. SIAM J. on Applied Mathematics, 108–126 (2003) 6. Biondi, T., Ciccazzo, C., Cutello, V., D’Antona, S., Nicosia, G., Spinella, S.: Multiobjective evolutionary algorithms and pattern search methods for circuit design problems. J. of Universal Computer Science 12(4), 432–449 (2006) 7. Price, W.: A controlled random search procedure for global optimisation. The Computer J. 20(4), 367–370 (1977) 8. Jones, D., Perttunen, C., Stuckman, B.: Lipschitzian optimization without the Lipschitz constant. J. of Optimization Theory and Applications 79(1), 157–181 (1993) 9. Sinha, A., Tiwari, S., Deb, K.: A population-based, steady-state procedure for realparameter optimization. In: Proceedings of the IEEE Congress on Evolutionary Computation, vol. 1, pp. 514–521 (2005) 10. Markowich, P.: The stationary semiconductor device equations. Springer, Heidelberg (1986) 11. Scharfetter, D., Gummel, H.: Large-signal analysis of a silicon read diode oscillator. IEEE Trans. Electron Devices 16(1), 64–77 (1969) 12. Abdallah, N., Degond, P.: On a hierarchy of macroscopic models for semiconductors. J. of Mathematical Physics 37, 3306–3333 (1996) 13. Chen, X., Chen, L., Jian, H.: The Dirichlet problem of the quantum drift-diffusion model. Nonlinear Analysis 69(9), 3084–3092 (2008) 14. Nicosia, G., Rinaudo, S., Sciacca, E.: An evolutionary algorithm-based approach to robust analog circuit design using constrained multi-objective optimization. In: AI 2007, pp. 175–183. Springer, Heidelberg (2007) 15. Di Stefano, V., Drago, C.R., Milazzo, C.: Evolutionary Algorithm for Doping Profile Optimization in Semiconductor Design. In: SIMAI, vol. 2, pp. 367–370 (2006) 16. Rinaudo, S., Moschella, F., Muscato, O., Ahile, A.M.: Controlled random search parallel algorithm for global optimization with distributed processes on multivendor cpus. In: Arkeryd, L., et al. (eds.) Progress in Industrial Mathematics - ECMI (1998) 17. Romano, V.: 2D numerical simulation of the MEP energy-transport model with a finite dierence scheme. J. of Computational Physics 221(2), 439–468 (2007) 18. Martin, J., Simpson, T.: Use of Kriging models to approximate deterministic computer models. AIAA J. 43(4), 853–863 (2005) 19. Wan, Z., Igusa, T.: Statistics of Nadaraya-Watson estimator errors in surrogatebased optimization. Optimization and Engineering 7(3), 385–397 (2006) 20. McDonald, D., Grantham, W., Tabor, W., Murphy, M.: Global and local optimization using radial basis function response surface models. Applied Mathematical Modelling 31(10), 2095–2110 (2007)

QML-AiNet: An Immune-Inspired Network Approach to Qualitative Model Learning Wei Pang1,2 and George M. Coghill1 1

2

Department of Computing Science, University of Aberdeen, Aberdeen, AB24 3UE, Scotland, UK {pang.wei,g.coghill}@abdn.ac.uk College of Computer Science and Technology, Jilin University, Changchun, 130012, P.R. China [email protected]

Abstract. In this paper we continue the research on applying immuneinspired algorithms as search strategies to Qualitative Model Learning (QML). A new search strategy based on opt-AiNet is proposed, and this results in the development of a novel QML system called QML-AiNet. The performance of QML-AiNet is compared with previous work using the CLONALG approach. Experimental results shows that although not as efficient as CLONALG, the opt-AiNet based approach still shows promising results for learning qualitative models. In addition, possible future work to further improve the efficiency of QML-AiNet is also pointed out.

1

Introduction

Qualitative Reasoning (QR), as an area of artificial intelligence, has undergone continuous development over the last three decades [1]. Qualitative Reasoning includes a number of sub-fields, among which are: Qualitative Simulation (QS) -examples of which include QSIM [2] and Morven [3]), and Qualitative Model Learning (QML) [4]- examples of which include GENMODEL [5], MISQ [6], QSI [7], QME [8], and ILP-QSI [9]). Among these two sub-fields, QML has received some attention recently because of its promising application potential and many unsolved problems existing in this field. In this paper, we focus on the most essential part of QML, the search strategies used to search the model space. In particular, in previous work [10], CLONALG [11] was employed as the search strategy to deal with the scalability of QML, and this immune-inspired approach proved to be more efficient than deterministic approaches for learning large-scale qualitative models. In this research we aim to explore an alternative opt-AiNet [12,13] based approach as the search strategy because of its promising performance for dealing with large multi-modal search space. E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 223–236, 2010. c Springer-Verlag Berlin Heidelberg 2010 

224

W. Pang and G.M. Coghill

Table 1. Some qualitative constraints in QSIM and their corresponding mathematical equations

1.1

QSIM Constraints

Mathematical Equations

ADD(X,Y,Z) MULT(X,Y,Z) DERIV(X,Y) MINUS(X,Y) M + (X,Y) M − (X,Y)

Z(t) = X(t) + Y (t) Z(t) = Y (t) ∗ X(t) dX(t)/dt = Y (t) Y (t) = −X(t) Y (t) = f (X(t)), f  > 0 Y (t) = −f (X(t)), f  > 0

Model Representation: Qualitative Differential Equations

Both QS and QML use Qualitative Differential Equations (QDE) to represent qualitative models. A QDE is the conjunction of a set of qualitative constraints, which link the qualitative variables in the model and express the relations among these variables. Qualitative variables can only take values from their associated quantity space. A quantity space is composed of several qualitative values, for example, in QSIM qualitative values are landmark values and intervals between two landmark values. As for the qualitative constraints, there are two kinds: those representing algebraic relations, such as qualitative addition, substraction, multiplication, and division, and those representing incomplete knowledge about the function relations, such as M + and M − constraints in QSIM, and function constraints in Morven. Table 1 lists some commonly used qualitative constraints in QSIM and their corresponding mathematical relations. In this table variables in the right column such as X(t) are continuous functions of time t. f is a function that is continuously differentiable over its domain (the so-called reasonable functions in QSIM), and f  stands for the first derivative of f. Similarly, Table 2 lists some Morven constraints and the corresponding mathematical equations. Morven constraints are more flexible: Each place in a constraint can represent not only the magnitude, but also arbitrary derivative of a variable. So there is no DERIV constraint in Morven. Also the Func constraint in Morven are more general than M + and M − constraints and it can be specialised to represent the same relations as M + and M − . A QDE is an abstraction of a set of Ordinary Differential Equations (ODEs) because the function relations of a QDE correspond to an infinite number of quantitative mathematical functions, and the qualitative values assigned to variables in a QDE represent various quantitative values. Table 2. Some qualitative constraints in Morven and their corresponding mathematical equations Morven Constraints

Mathematical Equations

sub (dt 0 Z, dt 0 X, dt 0 Y) mul (dt 0 X, dt 0 Y, dt 0 Z) Func (dt 0 Y, dt 0 X) sub (dt 1 Z, dt 0 X, dt 0 Y) Func (dt 1 Y, dt 0 X)

Z(t) = X(t) − Y (t) Z(t) = Y (t) ∗ X(t) Y (t) = f (X(t)) dZ(t)/dt = X(t) − Y (t) dY (t)/dt = f (X(t))

QML-AiNet: An Immune-Inspired Network Approach to QML

1.2

225

Details of QML and QML as a Search and Optimisation Problem

In Qualitative Reasoning, a qualitative state is a complete assignment of all qualitative variables in the system, and a qualitative behaviour is a series of qualitative states linked by their legal transitions. Both QS and QML can be considered as a function that describes relations of the QDE models and qualitative behaviours. The difference lies in that QS takes as input a QDE and its output is qualitative behaviours; while QML takes as input behaviours and generates possible QDE models. More specifically, QS aims to predict possible behaviours given a QDE model, while the goal of QML is to construct a QDE model from available data, which could be either quantitative or qualitative. A Q2Q (Quantitative-to-Qualitative) conversion algorithm is used if the given data are quantitative. For a specific problem P, given the background knowledge BK, the variable set VS which includes all variables (may also include unknown variables), and the set RS that contains all possible qualitative relations of these variables (such as M + , M − , and algebraic relations), we can generate a set CS containing all possible constraints by using all combinations of elements in RS and VS, which is defined as follows: CS = {c = (r, a, b, d) | r ∈ RS, a, b ∈ V S, d ∈ V S ∪ {∅}}

(1)

In the above a qualitative constraint is represented by a four-tuple vector c, where r denotes a qualitative relation, and a, b, d are variables. In addition, if r is a function relation, such as M + and M − , d will be empty. For example, ADD(X, Y, Z) in QSIM is represented as c = (ADD, Z, X, Y ), and M + (X, Y ) is represented as c = (M + , Y, X) in Formula (1). For the Morven formalism, constraint Sub(dt 0 Z, dt 0 X, dt 0 Y) is represented as c = (Sub, Z, X, Y ), and Func (dt 1 X, dt 0 Y) is represented as c = (F unc, X  , Y ). In this sense VS in Formula (1) also includes the first derivatives of state variables if the Morven formalism is used. In particular, if we consider the causal ordering [14] of a model, the leftmost variable in constraint c cannot be an exogenous one. If we denote GDS as the given data set and consider the background knowledge BK, we can use GDS and BK to filter out the inconsistent constraints in CS, and generate a filtered constraint set FCS. This is shown as follows: F CS = { c ∈ CS | c s.t. BK , c s.t. GDS}

(2)

The meaning of the above formular is as follows: each constraints in FCS is consistent with BK and covers GDS. The implicit qualitative model space QMS will contain all possible qualitative models generated from FCS, as shown below: QM S = {m | m ∈ ℘(F CS)}

(3)

In the above m is a possible model and the symbol ℘ stands for the powerset operation. The task of QML is to find a candidate set of models, denoted as

226

W. Pang and G.M. Coghill

CM, and each element (a model) m of CM satisfies BK and covers GDS, which is written as follows: QM LP (QM S) = CM = {m ∈ QM S | m s.t. BK ∧ QS(m) ⊇ GDS} (4) where QM LP stands for qualitative model learning for Problem P ; QS stands for the qualitative simulation of Model m, and QS(m) is a set containing all qualitative states obtained from simulation. From Formula (3) and (4) we see that QML is essentially a search and optimisation problem, that is, to search for the best models from QMS that satisfy Formula (4). Note that as the size of the search space QMS could be too large to feasibly enumerate all its elements, in the search process it is often the case that only a portion of the search space is explored. In addition, QMS is often multi-modal, or even highly multi-modal. This is because when GDS and BK are incomplete, there may be many models that satisfies formula (4), which correspond to many global optima in the search space, and there may also exist many local optima. All the above make the search tasks difficult. Consequently the search strategy of QML is the fundamental component of QML and should be improved in many aspects. 1.3

Deterministic vs. Randomised Search Strategies

The search strategy being used is the most important and distinguishable feature of QML. All the search strategies employed by QML can be categorized into two types: deterministic approaches and randomised approaches. QML systems using deterministic search strategies include GENMODEL [15,5], MISQ [6], QSI [7], and the latest ILP-QSI [9]. Among these systems, GENMODEL and MISQ used straightforwardly generate-and-test strategies; QSI utilised a complicated iterative and extension approach; and ILP-QSI made use of inductive logic programming (ILP) [16] to facilitate the search. The limitation of the deterministic approaches is the lack of scalability: With the increase of the size of model space, the deterministic approaches will become more and more inefficient. As for the randomised approaches, there are two existing QML systems using them: QME [8] which used a genetic algorithm, and our previously developed QML system [10,17] using the CLONALG approach. The latter QML system will be named QML-CLONALG in this paper for the ease of description. QME is the first attempt to apply the evolutionary approach to QML, but the limitation of QME is its inability to learn models from only positive data. The positive data are those data that a dynamic system can attain and can be observed from experiments; and on the other hand, the so-called negative data are those data that a dynamic system cannot attain. QME needs negative data to learn the model, which is not realistic because it is not always possible to obtain negative data in the real-world application. QML-CLONALG is a very promising QML system which can deal with positive only data. It is the first QML system that applies an immune-inspired approach to QML, which demonstrates the potential of immune-inspired approaches to QML as search strategies.

QML-AiNet: An Immune-Inspired Network Approach to QML

1.4

227

The Motivation and Organisation of This Paper

The success of QML-CLONALG motivates us to explore alternative immuneinspired approaches to QML. This is because the CLONALG approach used by QML-CLONALG is a classical and well-established immune-inspired algorithm, but it is acknowledged that the immune network approaches to optimisation problems, such as opt-AiNet [12,13] and opt-IA [18], can better deal with the large-scale multi-modal search space, which is the case of QML as mentioned in Section 1.2. This fact inspired us to explore the potential of applying the immune network approaches as search strategies to QML. So in this research we start from adapting opt-AiNet to QML as a search strategy, evaluating its performance and pointing out possible future improvements. The rest of this paper is organised as follows: In Section 2 we describe in detail the developed QML system in this research, QML-AiNet. All the experiments performed to evaluate the newly developed QML system is presented in Section 3. Finally Section 4 concludes the paper and explores possible future work.

2

QML-AiNet

We propose QML-AiNet, a novel QML system which employs an opt-AiNet based search strategy. Apart from the core search strategy, the other components of QML-AiNet are largely based on QML-CLONALG. Like QML-CLONALG, QML-AiNet uses the Morven formalism [3] to represent models, and JMorven [19], a Java implementation of Morven, to verify models. As in QMLCLONALG, QML-AiNet made use of well-posed model constraints [9] proposed in ILP-QSI, which serves as background knowledge (BK ) to narrow the search space. In the rest of this section, we first introduce the pre-processing phase of QMLCLONALG, then describe in detail the modified opt-AiNet algorithm for QML. 2.1

Pre-processing Phase

As in QML-CLONALG, the pre-processing phase of QML-CLONALG includes four sub-components: Constraint Generation, Constraint Filtering, Calculation of Conflict Set and Dependency Set, and Constraint Set Partition. These four components will be briefly introduced in this section, and for more details, the reader is directed to [10]. (1) Constraint Generation: generate all possible constraints CS, as shown in Formula (1). (2) Constraint Filtering: generate the filtered constraint set FCS in Formula (2). (3) Pre-calculation: For each constraint in FCS, calculate its conflict set and dependency set. The conflict set for a constraint c contains all all constraints in FCS that conflict with c. Two constraints are conflicting if they are logically inconsistent or redundant if they appear in the same model.

228

W. Pang and G.M. Coghill

The dependency set for a constraint c contains all constraints in FCS that depend on c. We say that constraint C1 depends on constraint C2 if the leftmost variable of C2 appears in any non-leftmost position of C1. The calculation of the dependency set is used for checking the causal ordering [14] of a model. The pre-calculation is for the ease of future computation, and the results are stored for later use. (4) Constraint Set Partition: A defining constraint for a variable v is the constraint in which v is the leftmost variable. FCS is divided into several subsets Si (i=1 to N, and N is the number of variables including hidden variables), and each of these subsets contains all defining constraints for the same variable. We define a set DS= {Sn } (n=1 to N ). According to the theorem presented in [10], for each Si in DS, if Si contains defining constraints for a non-hidden variable, a well-posed model must include one and only one constraint in this Si ; if Si contains defining constraints for a hidden variable, a well-posed model can include at most one constraint in Si . According to the above description the search space QMS in Formula (3) can be significantly narrowed down: QM Swp = {m = (c1 , c2 , ..., cn ) | ci ∈ Si ∪ {φ}, Si ∈ DS, i = 1, 2, ...n}

(5)

In the above QM Swp stands for the qualitative model space under well-posed model constraints; m is a possible model composed of several constraints ci ; φ stands for an empty constraint; and n is the number of elements in DS. 2.2

Antibody Encoding and Decoding

The original opt-AiNet for function optimisation employs the real number encoding. Each variable in the function is assigned a value within its range. In QML-CLONALG, the integer encoding is used: the antibody is composed of several slots, each of which corresponds to an element Si in DS (DS is described in Section 2.1). The integer assigned to each slot indicates the selection of a constraint in Si . Similar to QML-CLONALG, in QML-AiNet an antibody is also composed of several slots, and each of them corresponds to a constraint subset Si in DS. Unlike QML-CLONALG, in QML-AiNet the value assigned to each slot is a real number, which is the same as in the original opt-AiNet. The range of values assigned to each slot is from 1 to N, and N is the number of constraints in the constraint subset Si to which this slot corresponds. The real number encoding is compatible with the affinity proportion mutation operator in QML-AiNet, which will be described in Section 2.4. As the real number encoding strategy is used, when we decode an antibody, for a real number R in a slot, we will take the integer I which is closest to R. If R is in the middle of two integers, the smaller integer will be taken. Figure 1 shows the antibody encoding and decoding of QML-AiNet. In this figure, the antibody has n slots, which correspond to S1 , S2 , ...Sn . In Slot 1 the current value is 2.2. After decoding we get integer 2, and the second constraint c2 in S1 is selected (indicated in bold font). In Slot n the assigned value is 4.5, which is in the middle of 4 and 5. After decoding we obtained integer 4, and the

QML-AiNet: An Immune-Inspired Network Approach to QML

229

Fig. 1. The Antibody Encoding and Decoding of QML-AiNet

fourth constraint c91 in Sn is selected. It is similar for the other slots. Finally the model decoded from the antibody shown in this figure contains constraints c2, c12, and c91. 2.3

Fitness Evaluation

The fitness evaluation is based on the well-posed model constraints, and takes the same scoring system as in QML-CLONALG. It is noted that in QML-CLONALG this process is called the affinity evaluation, and in QML-AiNet the affinity has a different definition. An antibody is first decoded to a model, then this model is checked against the well-posed model constraints, including completeness, model connection, confliction, causal ordering, and coverage. In the fitness evaluation the most expensive procedure is the coverage test of a model, for which the qualitative simulation has to be used. In this research we utilise JMorven [19] to perform the simulation. 2.4

Mutation

The mutation operator follows the original opt-AiNet. For each slot of the antibody, the current value C will be mutated to a new value C’ according to the following equations: (6) C  = C + α ∗ N (0, 1) ∗ 1 α = ∗ exp−f (7) β In the above equations, f ∗ is the normalized fitness with the range [0,1]. N(0,1) is a Gaussian random variable which has the mean value of 0 and standard devi∗ ation of 1. exp−f is the inverse exponential function. α stands for the amount of

230

W. Pang and G.M. Coghill

mutation. β is a parameter that adjusts this exponential function. As we expect the mutated value is different after decoding, in all experiments the value of β is set to 1, instead of the default value 100 in opt-AiNet. 2.5

Affinity

In opt-AiNet the affinity is defined as the Euclidean distance of two antibodies. In QML-AiNet as we use the integer decoding strategy, and each antibody represents a qualitative model, we define the affinity between two antibodies as the “model distance” between two models which these two antibodies represent. The model distance between two models is defined as the number of different constraints in these two models. 2.6

The Pseudo Code of QML-AiNet

The steps of QML-AiNet is basically the same as opt-AiNet. First we give the parameters used by the algorithm: – – – – –

Ni : Number of initial antibodies in the population Nc : Number of clones for each antibody AvgF itError: Threshold determines the stability of population Supp: The suppression threshold. d: The percentage of new antibodies to be added into the population.

The pseudo code of QML-AiNet is as follows: Step 1 : Randomly generate Ni antibodies. While (stop criteria are not satisfied) iteratively execute Step 2 ∼ Step 4 : Step 2 : Clonal Selection – Step 2-1 : Antibody fitness evaluation: calculate the fitness values of all antibodies according to the description in Section 2.3. – Step 2-2 : Clone: Generate Nc clones for each antibody. – Step 2-3 : Mutation: Each antibody will be mutated according to the description in Section 2.4 – Step 2-4 : Fitness Evaluation: evaluate all the newly cloned antibodies. Calculate the normalised fitness value for each antibody. – Step 2-5 : Selection: Select the antibody which has the biggest fitness value from each parental antibody and its clones. All the selected antibodies construct a new antibody population. – Step 2-6 : Average Fitness Error Calculation: Calculate the average fitness of the new population. If the difference between the old average fitness and new average fitness is less than the given threshold AvgF itError, repeat Step 2; otherwise proceed to Step 3. Step 3 : Network Suppression: Each antibody interacts with others. If the affinity of two antibodies are less than the suppression threshold Supp, the one with the smaller fitness value will be removed. Step 4 : Add d percent of the randomly generated antibodies to the population.

QML-AiNet: An Immune-Inspired Network Approach to QML

3

231

Experiments

In this section we evaluate QML-AiNet through a series of experiments. First, we give a brief introduction about the compartmental models, which are used as the test bed in this research. Second, we present the experiment design. Finally we report the experimental results and analysis of these results. 3.1

Experimental Test Bed

We select the compartmental models as the experimental test bed, which is the same as in QML-CLONALG. The compartmental models are abstractions of many dynamic systems, and their applications have been found in many disciplines, such as biology, physiology, and ecology. Furthermore, compartmental modelling is a more general methodology, many de facto benchmarks in the qualitative reasoning community, such as the single-tank, U-tube, cascaded tanks, and coupled tanks, have their analogous compartmental models. The detailed description of these benchmarks is given in [9]. In [10], four compartmental models have been studied. In this research as we focus on the scalability of QML, we choose the three and four-compartment models to test the performance of both QML-CLONALG and QML-AiNet. The compartmental models used in the experiments are shown in Figure 2. In this figure, CM2 Ex3 and CM2 Ex4 are the names of the two models ( these two names were used in [10]); c1, c2, c3, and c4 stand for the concentrations in the compartments, and they are also used to “label” the compartments; f12, f21, f23, and f34 denote the flows from one compartment to another; u is the input flow; f30 and f40 are the output flows to the environment. The qualitative model of Model CM2 Ex3 using the Morven formalism is given in Figure 3. In this figure, fx1 , fx2 , and fx3 are net flows of compartments c1, c2, and c3, respectively; Inc is a function relation describing the monotonically increasing relation as described in [10]. Note as the Morven formalism uses the concept of differential planes, in this model Differential Plane 0 contains the constraints as applied to the magnitudes of the variables, and constraints in Differential Plane 1 are obtained by differentiating the corresponding constraints in Differential Plane 0. 3.2

Experiment Design

Table 3 shows all the experiments to be performed by QML-AiNet and QMLCLONALG. We use CM2 Ex3 E1 as an example to explain the meaning of this table: In experiment CM2 Ex3 E1 the net flow fx3 is a hidden variable, which means it cannot be observed. After simulating this model by JMorven, we obtained 68 qualitative states, and the size of the search space is 6.95*108. The parameter values used in QML-CLONALG are as follows: the population size is 100 for CM2 Ex3 E1 and 1000 for others; the clone size is 10; the hyper-mutation probability is 0.9; the surviving time for all antibodies is 1000

232

W. Pang and G.M. Coghill

Differential Plane 0 C1: Inc (dt 0 f12, dt 0 c1) C2: Inc (dt 0 f23, dt 0 c2) C3: Inc (dt 0 f30, dt 0 c3) C4: sub (dt 1 fx1 , dt 0 u, dt 0 f12) C5: sub (dt 1 fx2 , dt 0 f12, dt 0 f23) C6: sub (dt 1 fx3 , dt 0 f23, dt 0 f30) Differential Plane 1 C7: Inc (dt 1 f12, dt 1 c1) C8: Inc (dt 1 f23, dt 1 c2) C9: Inc (dt 1 f30, dt 1 c3) C10: sub (dt 2 fx1 , dt 1 u, dt 1 f12) C11: sub (dt 2 fx2 , dt 1 f12, dt 1 f23) C12: sub (dt 2 fx3 , dt 1 f23, dt 1 f30)

Fig. 2. The Compartmental Models

Fig. 3. The JMorven Model for CM2 Ex2

generations. The parameter values used in QML-AiNet are as follows: the number of initial cells Ni is 20; the clone size Nc is 10; AvgF itError is 0.001; the suppression threshold supp is 6; d is 0.4. The values of the these parameters are determined by the complexity and features of the search space, and also based on the performance consideration. In addition, we use the totally random algorithm as a bottom line for these two algorithms. The totally random algorithm assembles a model from QM Swp by randomly picking one constraint from each Si in DS (See Formula (5)). In all experiments, complete qualitative data were used, and the stop criteria is that a well-posed model that can cover all the given data is found. All the experiments are performed on a computer cluster with 43 compute nodes (each node has 16 Intel XEON E5520 (2.26GHz) CPUs and 11.7GB RAM). 3.3

Experimental Results and Analysis

For each experiment listed in Table 3, each of the three algorithms is run for ten trials and the best and average running time is recorded, as shown in Table 4. The detail of the ten trials for each experiment is shown by the Box-andWhisker plots in Figure 4. In this figure, the values on the vertical axis are the running time of the algorithms, and the vertical axis is on a base-10 logarithmic scale and its unit of time is millisecond. The diamonds on the left side of each Box-and-Whisker plot represent the running time of individual trials. From the results presented in Table 4 one can see that both QML-AiNet and QML-CLONALG improve the scalability of QML for learning complex compartmental models when compared to the totally random algorithm. We also note that in experiment CM2 Ex3 E3 QML-AiNet performs better than QMLCLONALG; for the other three experiments the performance of QML-AiNet is not as good as QML-CLONALG, although in experiment CM2 Ex4 E2 the performance of QML-AiNet is very close to that of QML-CLONALG. The results demonstrate the potential of QML-AiNet. For this initial stage of the research we focused on simply adapting the opt-AiNet to perform QML.

QML-AiNet: An Immune-Inspired Network Approach to QML

233

Table 3. Experiment Configuration Experiment ID CM2 Ex3 E1 CM2 Ex3 E2 CM2 Ex3 E3 CM2 Ex4 E2

Hidden Variables fx3 fx2 , fx3 fx1 , fx2 , fx3 fx4

Num. Of States 68 48 48 340

Search Space 6.95*108 4.81*1010 6.31*1011 4.22*1012

Table 4. Experimental Results Best Running Time (millisecond ) Experiment Totally Random QML-CLONALG QML-AiNet ID Algorithm CM2 Ex3 E1 259,003 4,516 3,216 CM2 Ex3 E2 709,127 247,067 434,236 CM2 Ex3 E3 3,898,710 20,211 1,507,146 CM2 Ex4 E2 107,570,008 19,517,666 49,291,177 Average Running Time (millisecond ) Experiment Totally Random QML-CLONALG QML-AiNet ID Algorithm CM2-Ex3-E1 689,662.4 50,082 144,958 CM2-Ex3-E2 48,334,152 1,362,385 4,888,222 CM2-Ex3-E3 106,941,796 14,219,243 12,716,194 CM2-Ex4-E2 1,822,075,689 184,163,947 188,650,143

To achieve this the operators of the original opt-AiNet were modified to make QML-AiNet work operational; no effort was expended at this stage to make it efficient for performing QML. Even with these straightforward modifications of opt-AiNet, QML-AiNet already outperforms QML-CLONALG in one of the four experiments, which leads us to believe that future developments to QML-AiNet may result in significant improvements to the performance that will make it consistently do better than QML-CLONALG. The success of QML-AiNet is ascribable to the natural ability of opt-AiNet to deal with large multi-modal search space. First, it maintains a dynamic population the size of which can be changed according to the multi-modal features of the search space: the more peaks in the landscape of the search space, the more antibodies the population will have. Second, the network suppression operator ensures that the antibodies will not cluster in the same peak. Third, hyper-mutation refines the search performed by the antibodies and makes the antibodies eventually find their neighbouring local optima. Finally we believe, from the results obtained thus far, that the operators of QML-AiNet have great potential to be improved in the future. One possible improvement of QML-AiNet is the modification of the hyper-mutation operator. In all experiments the value of β in Formula (7) is set to 1 because we expect the mutated values are different after decoding. In the future we may make the value

234

W. Pang and G.M. Coghill

1E9

Run Time (log

10

milllisecond)

1E8

1E7

Experiment ID 1. CM2Ex3E1 2. CM2Ex3E1 3. CM2Ex3E2 4. CM2Ex3E2 5. CM2Ex3E3 6. CM2Ex3E3 7. CM2Ex4E2 8. CM2Ex4E2

1000000

100000

10000

1

2

3

4

5

6

7

Algorithm AiNet CLONALG AiNet CLONALG AiNet CLONALG AiNet CLONALG

8

Experiment ID

Fig. 4. Ten Trials of All Experiments

of β change dynamically according to the features of different search spaces and current positions of all antibodies in the search space; we may also modify the structure of Formula (7) to make it adapt to discrete search space.

4

Conclusions and Future Work

In this paper, we have presented an immune network approach to qualitative model learning. The proposed QML system QML-AiNet employs an opt-AiNet based search strategy to search the model space. In QML-AiNet, the operators of optAiNet were straightforwardly modified and adapted to QML; the antibodies used a problem-dependent encoding/decoding strategy; and the values of some parameters in opt-AiNet were adjusted to make QML-AiNet perform effective search. A comparison of the performance of QML-AiNet with QML-CLONALG indicates that the two methods are comparable with respect to the task set. This leads us to believe that improving the efficiency of QML-AiNet would make it outperform QML-CLONALG thus making it very suitable as a tool for QML. In the future it will be of interest to explore other immune-inspired approaches to QML, and compare their performance. Finally, we point out that QML is a discrete optimisation problem because of its discrete model space. So the development of QML-AiNet inspires us to explore the potential of the opt-AiNet approach to general discrete optimisation problems.

Acknowledgements The authors would like to thank Prof. Jon Timmis for his inspiring suggestion about applying opt-AiNet to QML. We would also like to thank Dr.Paul

QML-AiNet: An Immune-Inspired Network Approach to QML

235

Andrews for providing the java source code of opt-AiNet and the helpful discussion about the antibody encoding. WP and GMC are supported by the CRISP project (Combinatorial Responses In Stress Pathways) funded by the BBSRC (BB/F00513X/1) under the Systems Approaches to Biological Research (SABR) Initiative. WP was partially financially supported by the National Natural Science Foundation of China under grant No. 60433020, 60673099, 60773095.

References 1. Price, C., Trave-Massuyes, L., Milne, R., Ironi, L., Forbus, K., Bredeweg, B., Lee, M., Struss, P., Snooke, N., Lucas, P., Cavazza, M., Coghill, G.: Qualitative futures. The Knowledge Engineering Review 21(4), 317–334 (2006) 2. Kuipers, B.: Qualitative Reasoning: Modeling and Simulation with Incomplete Knowledge. MIT Press, Cambridge (1994) 3. Coghill, G.M.: Mycroft: A Framework for Constraint based Fuzzy Qualitative Reasoning. PhD thesis, Heriot-Watt University (September 1996) 4. Pang, W., Coghill, G.M.: Learning qualitative differential equation models: a survey of algorithms and applications. Knowledge Engineering Review 25(1), 69–107 (2010) 5. Hau, D.T., Coiera, E.W.: Learning qualitative models of dynamic systems. Machine Learning 26, 177–211 (1993) 6. Ramachandran, S., Mooney, R.J., Kuipers, B.J.: Learning qualitative models for systems with multiple operating regions. In: The Eighth International Workshop on Qualitative Reasoning about Physical Systems (QR 1994), Nara, Japan (1994) 7. Say, A.C.C., Kuru, S.: Qualitative system identification: deriving structure from behavior. Artificial Intelligence 83, 75–141 (1996) 8. Varsek, A.: Qualitative model evolution. In: Mylopoulos, J., Reiter, R. (eds.) Proceedings of the 12th International Joint Conference on Artificial Intelligence, Sydney, Australia, August 1991, vol. 2, pp. 1311–1316 (1991) 9. Coghill, G.M., Srinivasan, A., King, R.D.: Qualitative system identification from imperfect data. Journal of Artificial Intelligence Research 32, 825–877 (2008) 10. Pang, W., Coghill, G.M.: Modified clonal selection algorithm for learning qualitative compartmental models of metabolic systems. In: Thierens, D. (ed.) Genetic and Evolutionary Computation Conference (GECCO 2007), pp. 2887–2894. ACM Press, New York (2007) 11. de Castro, L.N., Von Zuben, F.J.: The clonal selection algorithm with engineering applications. In: Proceedings of GECCO, Workshop on Artificial Immune Systems and Their Applications, Las Vegas, USA, July 2000, pp. 36–39 (2000) 12. de Castro, L.N., Timmis, J.: An artificial immune network for multimodal function optimization. In: Proceedings of IEEE Congress on Evolutionary Computation (CEC 2002), pp. 674–699. IEEE Press, Los Alamitos (2002) 13. Timmis, J., Edmonds, C.: A comment on opt-ainet: An immune network algorithm for optimisation. In: Kalyanmoy, D. (ed.) GECCO 2004. LNCS, vol. 3102, pp. 308–317. Springer, Heidelberg (2004) 14. Iwasaki, Y., Simon, H.A.: Causality and model abstraction. Artificial Intelligence 67(1), 143–194 (1994) 15. Coiera, E.: Generating qualitative models from example behaviours. Technical Report DCS Report 8901, Department of Computer Science, University of New South Wales, Sydney, Australia (1989)

236

W. Pang and G.M. Coghill

16. Bergadano, F., Gunetti, D.: Inductive Logic Programming From Machine Learning to Software Engineering. MIT Press, Cambridge (1996) 17. Pang, W.: QML-Morven: A Framework for Learning Qualitative Models. PhD thesis, University of Aberdeen (2009) 18. Cutello, V., Narzisi, G., Nicosia, G., Pavone, M.: An immunological algorithm for global numerical optimization. In: Talbi, E.-G., Liardet, P., Collet, P., Lutton, E., Schoenauer, M. (eds.) EA 2005. LNCS, vol. 3871, pp. 284–295. Springer, Heidelberg (2006) 19. Bruce, A.M.: JMorven: A Framework for parallel non-constructive qualitative reasoning and fuzzy interval simulation. PhD thesis, Department of Computing Science, Univeristy of Aberdeen (October 2007)

Biomedical Article Classification Using an Agent-Based Model of T-Cell Cross-Regulation Alaa Abi-Haidar and Luis M. Rocha School of Informatics and Computing, Indiana University, Bloomington IN 47401, USA FLAD Computational Biology Collaboratorium, Instituto Gulbenkian de Ciˆencia, Oeiras, Portugal {aabihaid,rocha}@indiana.edu

Abstract. We propose a novel bio-inspired solution for biomedical article classification. Our method draws from an existing model of T-cell cross-regulation in the vertebrate immune system (IS), which is a complex adaptive system of millions of cells interacting to distinguish between harmless and harmful intruders. Analogously, automatic biomedical article classification assumes that the interaction and co-occurrence of thousands of words in text can be used to identify conceptually-related classes of articles—at a minimum, two classes with relevant and irrelevant articles for a given concept (e.g. articles with protein-protein interaction information). Our agent-based method for document classification expands the existing analytical model of Carneiro et al. [1], by allowing us to deal simultaneously with many distinct T-cell features (epitomes) and their collective dynamics using agent based modeling. We already extended this model to develop a bio-inspired spam-detection system [2, 3]. Here we develop our agent-base model further, and test it on a dataset of publicly available full-text biomedical articles provided by the BioCreative challenge [4]. We study several new parameter configurations leading to encouraging results comparable to state-of-the-art classifiers. These results help us understand both T-cell cross-regulation and its applicability to document classification in general. Therefore, we show that our bio-inspired algorithm is a promising novel method for biomedical article classification and for binary document classification in general. Keywords: Artificial Immune System, Bio-medical Document Classification, T-cell Cross-Regulation, Bio-inspired Computing, Artificial Intelligence, BioCreative.

1

Introduction

With faster genome sequencing [5] and microarray analysis [6], the last decade has witnessed an exponential growth of metabolic, genomic and proteomic documents (articles) being published [7]. Pubmed [8] encompasses a growing collection of more than 18 million biomedical articles. Manually classifying these articles as relevant or irrelevant to a given topic of interest is very time consuming and inefficient for curation of new published articles [9]. A few conferences E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 237–249, 2010. c Springer-Verlag Berlin Heidelberg 2010 

238

A. Abi-Haidar and L.M. Rocha

have been dedicated to literature or text mining offering challenges to address biomedical document classification. BioCreative is a community-wide effort for assessing bio-literature mining [4] . Machine Learning has offered a plethora of solutions to this problem [9, 10], but even the most sophisticated of solutions often overfit to the training data and do not perform as well on real-world data such as that provided by BioCreative—in this case, articles for curation selected from FEBS Letters.[11–13]. The immune system is a complex biological system made of millions of cells all interacting to distinguish between harmless and harmful intruders, to ultimately attack the latter [14]. In analogy, relevant biomedical articles for a given concept need to be distinguished from irrelevant ones which should be discarded in topical queries. To employ computational intelligence to automatically implement this topical classification, we can use the occurrence and co-occurrence of thousands of words in a document describing an approach, an experiment, a result or a conclusion. In this sense, words can be seen as interacting in a text in such a way as to allow us to distinguish between relevant and irrelevant documents. Recent advances in artificial immune systems [15] have offered a few immune-inspired solutions to document classification in general, though none to our knowledge has been applied to biomedical article classification. Our aim is not to explore the applicability of existing immune inspired solutions on biomedical article classification [16], but to propose a new solution and compare it with state-of-art classifiers. We extend an existing model of T-cell cross-regulation [1] to deal with multiple features simultaneously using agent based modeling. We applied a first version of our agent-based model to a similar document classification problem dealing with spam detection. On that task, we obtained encouraging results, which were comparable to state-of-art text classifiers [2, 3]. However, our preliminary implementation did not explore all parameter configurations such as T-cell death rates, different training scenarios, and lacked extensive parameter search for optimized performance [2, 3]. In the work reported here, we test variations of our agent-based model to understand the effect of T-cell death and of training exclusively on relevant articles. We also test our agent-based model on full-text biomedical data from BioCreative and compare it with state-of-art classifiers to understand the model’s applicability to real-World biomedical classification specifically, and to document classification in general. This more extensive study allows us to establish the capability of T-cell cross-regulation dynamics to classify data. It also leads to a competitive, novel bio-inspired text classification algorithm. In section 2, we describe the original T-cell cross-regulation model [1]. In section 3, we describe the expanded agent-based implementation of the crossregulation model and explain its parameters. In section 4, we discuss the biomedical data from BioCreative and the feature selection process. In section 5, we report our results on biomedical document classification and compare them to those obtained by Naive Bayes [17] and SVM [18].

Biomedical Article Classification Using an Agent-Based Model

2

239

The Cross-Regulation Model

The T-cell Cross-Regulation Model (CRM) [1] is a dynamical system that aims to distinguish between harmless and harmful protein fragments (antigens) using only four possible interactions of three cell-types: Effector T-cells (E), Regulatory T-cells (R) and Antigen Presenting Cells (APC). As their name suggests, APC present antigens for the other two cell-types, E and R, to recognize and bind to them. Effector cells (E) proliferate upon binding to APC, unless adjacent to regulatory cells (R), which regulate E by inhibiting their proliferation. For simplicity, proliferation of cells is limited to duplication in quantity in contrast to having a proliferation rate. T-cells that do not bind to APC die off with a certain death rate. The four possible interactions, illustrated in Fig. 1, can be simply expressed by the following equations: E− → {} → {} and R− dR dE A+R→A+R

(2)

A + E → A + 2E A + E + R → A + E + 2R

(3) (4)

(1)

The first equation (1) expresses E and R cell death with the corresponding death rates dE and dR . The last three proliferation equations express (2) the maintenance of R, (3) the duplication of E, and (4) the maintenance of E and duplication of R. Carneiro et al. [1] developed the analytical CRM to study the dynamics of a population of T-cells and APC that recognize a single antigen. In [3, 2], we adapted the original CRM model to deal with multiple populations of textual features using agent-based modeling. Our basic implementation of the model yielded encouraging results when applied to spam detection, a binary document classification problem. More recently, Sepulveda [21, pp 111-113] extended the original CRM to study multiple populations of T-cells that can be recognized by APC, each capable of recognizing at most two distinct T-cell populations. In our preliminary model [3, 2], we have used APC that are capable of recognizing hundreds of T-cells of different populations, simultaneously, using the same four interaction rules of the CRM. In the following section, we explain in more details our agent-based model adapted for document classification.

3

The Agent Based Cross-Regulation Model

In order to adapt CRM to an Agent-Based Cross-Regulation Model (ABCRM) for text classification, one has to think of documents as analogous to the organic substances that upon entering the body are broken into constituent pieces. These pieces, known as epitopes, are presented on the surface of Antigen Presenting Cells (APC) as antigens. In the ABCRM, antigens are textual features (e.g. words, bigrams, titles, numbers) extracted from articles and presented by artificial APC such that they can be recognized by a number of artificial Effector

240

A. Abi-Haidar and L.M. Rocha

Fig. 1. The diagram illustrates the CRM interactions underlying the dynamics of APC, E and R as assumed in the model where APC can only form conjugates with a maximum of two T-cells

T-cells (E) and artificial Regulatory T-cells (R). In other words, individual E and R have receptors for a single, specific textual feature: they are monospecific. E proliferate upon binding to antigens presented by APC unless suppressed by R; R suppress E when binding in adjacent locations on APC. Individual APC present various document features: they are polyspecific. Each APC cell is produced when documents enter the cellular dynamics, by breaking the latter into constituent textual features. Therefore we can say that APC are representative of specific documents whereas E and R are representative of specific features. A document d contains a set of features Fd ; An artificial APC Ad that represents d, presents antigens/features fi ∈ Fd to artificial E and R T-cells. Ei and Ri bind to a specific feature fi on any APC that contains it; if fi ∈ Fd , then either Ei or Ri may bind to Ad as illustrated in figure 2. In biology, antigen recognition is a more complex process than mere polypeptide sequence matching but for simplicity we limit our feature recognition to string matching. Once T-cells bind to an APC Ad , every pair of adjacent T-cells on Ad proliferates according to the last three interaction rules of equations (2-4). APC are organized as a sequence of pairs of “slots” of textual features, where T-Cells, specific for those features, can bind. We use this simplified antigen/feature presentation scheme of pairs of “slots” to simplify our algorithm. In future work we will study alternative feature presentation scenarios. In summary, each T-cell population is specific to and can bind to only one feature presented by APC. Implementing the algorithm as an Agent-based model (ABM) allows us to deal with recognition of many features simultaneously, rather than a single one as the original mathematical model does.

Biomedical Article Classification Using an Agent-Based Model

241

Fig. 2. To illustrate the difference between the CRM and the ABCRM, the top part of the figure represents a single APC of the CRM which can bind to a maximum of two T-Cells. The lower part represents the APC for a document d in the ABCRM, which contains many pairs of antigen/feature “slots” where pairs of T-cells can bind. In this example, the first pair of slots of the APC Ad presents the features fi and fj ; in this case, a regulatory T-cell Ri and an effector T-cell Ej bind to these slots, which will therefore interact according to reaction (4)—Ri inhibits Ej and in turn proliferates by doubling. The next pair of slots leads to the interaction of T-cells Ri ,Rk , etc.

The ABCRM uses incremental learning to first train on N labeled documents (relevant and irrelevant), which are ordered sequentially (typically by time signature) and then test on M unlabeled documents that follow in time order. The sequence of articles is assumed to be of importance to our model [2] but is outside the scope of this study. Carneiro et al. [1] show that both E and R T-cells co-exist in healthy individuals assuming enough APC exists. R T-cells require adequate amounts of E T-cells to proliferate, but not too many that can out-compete R for the specific features presented by APC. “Healthy” T-cell dynamics is identified by observing the co-existence of both E and R features with R ≥ E. “Unhealthy” T-cell dynamics is identified by observing E  R, and should result when encountering many irrelevant features in a document. In other words, features associated with relevant documents should have E and R T-cell representatives in comparable numbers in the artificial cellular dynamics (with slightly more R). In contrast, features associated with irrelevant documents should have many more E than R T-cells. Therefore, when a document d contains features Fd , that bind mostly to E rather than R cells, we can classify it as irrelevant—and relevant in the opposite situation. The ABCRM is controlled by 6 parameters: • E0 is the initial number of Effector T-cells generated for all new features • R0− is the initial number of Regulatory T-cells generated for all new features in irrelevant and unlabeled documents • R0+ is the initial number of Regulatory T-cells generated for all new features in relevant documents • dE is the death rate for Effector T-cells that do not bind to APC

242

A. Abi-Haidar and L.M. Rocha

• dR is the death rate for Regulatory T-cells that do not bind to APC • nA is the number of slots in which each feature fi is presented on an APC When the features of a document d are encountered for the first time, a fixed initial number of E0 and R0 , for every new feature fi , is generated. These initial values of T-cells vary for relevant and irrelevant documents in training and in testing stages. More Regulatory T-cells (R0+ ) than Effector T-cells are generated for features that occur for the first time in documents that are labeled relevant in the training stage (R0+ > E0 ), while fewer Regulatory T-Cells (Ro− ) than Effector T-cells are generated in the case of irrelevant documents (R0− < E0 ). Features appearing in unlabeled documents for the first time during the testing stage are treated as features from irrelevant documents, assuming that new features are foreign until neutralized by co-occurrence with relevant ones. Of course, relevant features might occur in irrelevant documents and vice versa. However, the assumption is that relevant features tend to co-occur more frequently with other relevant features in relevant documents and similarly for irrelevant features thus correcting the erroneous initial bias. The following pseudocode highlights the minor differences between the training and validation/testing stages of the algorithm: TRAINING: ∀d generate Ad presenting each fi at nA slots, where fi ∈ Fd Let Ct be the set of all Ek and Rk for all features fk in the cellular dynamics / Ct and Ri ∈ / Ct then, ∀fi ∈ Fd , if Ei ∈ Ei = E0 (generate E0 Effector T-cells for feature fi ) if d is labeled relevant Ri = R0+ (generate R0+ Regulatory T-Cells for feature fi ) otherwise Ri = R0− (generate R0− Regulatory T-Cells for feature fi ) Let Ei and Ri bind specifically to matching fi presented on Ad : ∀ pair of adjacent (fi , fj ) on Ad apply the last three interaction rules: Ri+Rj+Ad->Ri+Rj Ei+Ej+Ad->2Ei+2Ej Ei+Rj+Ad->Ei+2Rj ∀Ri and Ei that bind to Ad , update total number of Ei and Ri ∀Rk , Ek ∈ Ct that do not bind to Ad , cull Ek and Rk via death rates dE and dR TESTING: ∀d generate Ad presenting each fi at nA slots, where fi ∈ Fd Let Ct be the set of all Ek and Rk for features fk in the cellular dynamics / Ct and Ri ∈ / Ct then, ∀fi ∈ Fd , if Ei ∈ Ei = E0 (generate E0 Effector T-cells for feature fi ) Ri = R0− (generate R0− Regulatory T-Cells for feature fi ) Let all the Ei and Ri bind specifically to matching fi presented on Ad : ∀ pair of adjacent (fi , fj ) on Ad apply the last three interaction rules:

Biomedical Article Classification Using an Agent-Based Model

243

Ri+Rj+Ad->Ri+Rj Ei+Ej+Ad->2Ei+2Ej Ei+Rj+Ad->Ei+2Rj total number of Ei and Ri ∀Ri and Ei that bind to Ad , update  and compute for d: R(d) = ∀fi ∈Fd (Ri ) and E(d) = ∀fi ∈Fd (Ei )  Hence the normalized score for d is S(d) = (R(d) − E(d))/ R2 (d) + E 2 (d) If S(d) > 0 then classify d as relevant, otherwise irrelevant ∀Ek , Rk ∈ Ct that do not bind to Ad , cull Ek and Rk via death rates dE and dR According to the original CRM model [1], T-cells that do not bind to a presented antigen die at a certain death rate determined by dE and dR . Cell death is supposed to help the algorithm forget old features and focus on more recently encountered ones. Cell death was not fully explored in our previous application of this model [2, 3] and therefore in section 5 we test the effect of cell death in the dynamics of the ABCRM. Negative selection in the adaptive immune system is thought to help discrimination between harmless and harmful antigens by eliminating immature Effector T-cells that bind to harmless or self antigens in the thymus—thus helping to prevent auto-immunity. Mature Effector T-cells that did not bind to harmless antigens are released from the thymus to recognize harmful antigens [14]. Therefore, Effector T-cells are trained to discriminate between harmless and harmful antigens avoiding autoimmunity, by preliminary “training” on harmless or self antigens. In the context of machine learning, this is known as positive unlabeled (PU) training, which we test here against training on both relevant (positive) and irrelevant (negative) documents.

4

Data and Feature Selection

The BioCreative (BC) challenge aims to assess state-of-art in bio-literature mining— in particular, biomedical document classification. More recently, the article classification task of BC2.5 [4] was based on a training data set comprised of 61 full-text articles relevant to protein-protein interaction (PT ) and 558 irrelevant ones (NT ). This imbalance between the relevant and irrelevant instances can be very challenging. In order to assess our bio-inspired algorithm as a biomedical text classifier, we first identify optimal parameters on samples of training that are balanced in the numbers of relevant and irrelevant documents since we cannot predict if the validation data will be imbalanced. We assume that the adaptive nature of our algorithm, will adapt well to unpredictable imbalance by adjusting the proportions between the populations of E and R T-cells automatically. For the purpose of identifying optimal parameters, we chose the first 60 relevant and sampled 60 irrelevant articles that were published around the same date (uniform distribution between Jan and Dec 2008) as illustrated in figure 3.

244

A. Abi-Haidar and L.M. Rocha

Fig. 3. Numbers of relevant (P ) and irrelevant (N ) documents in the training (T ) and testing (V ) data sets of the Biocreative 2.5 challenge. In the parameter search stage, we use a balanced set of 60 PT (blue) and 60 NT (red) randomly selected articles from the training data set. In the testing stage we use the unbalanced validation set containing 63 PV (black) and 532 NV (black) documents. Notice that the validation data was provided to the participants in the classification task of Biocreative 2.5 unlabeled, therefore participants had no prior knowledge of class proportions.

Fig. 4. We choose the top 650 ranked features according to the rank product R(f) = 1 and the x-axis represents the index of TF.IDF(f) × S(f). The y-axis represents R(f ) R(f) for the sorted features. Features ranked below the 650th feature have a similar 1 < 0.00001. score R(f )

We compared our fine-tuned algorithm with Naive Bayes classifier (NB) [17] and support vector machine (SVM) [18]. For testing and validation we used the Biocreative 2.5 testing data set consisting of 63 full-text articles relevant to protein- protein interaction (PV ) and 532 irrelevant ones (NV ) as shown in figure 3.

Biomedical Article Classification Using an Agent-Based Model

245

We pre-processed all articles by filtering out stop words1 and porter stemming [22] the remaining words/features. We then ranked features f extracted from BC2.5 training articles in addition to BC22 , according to two scoring methods. The first one is the average TF.IDF3 per feature over all documents [10] and the second one is according to the separation score S(f ) = |pRelevance (f ) − pIrrelevance (f )| where prelevance is the probability of a feature occurring in a relevant article and pirrelevance is the probability of it occurring in an irrelevant one [19, 11, 20, 13]. The final rank for every feature fi is defined by the rank product R(f) = TF.IDF(f) × S(f). We only use the top 650 ranked features as shown in figure 4 to represent each document d as a vector of these top 650 features for optimization purposes.

5

Results

5.1

Parameter Search

We performed an exhaustive parameter search by training the ABCRM on 60 balanced full-text articles (30 PT and 30 NT from BC2.5 training) and testing it on the remaining 60 balanced ones (30 PT and 30 NT from BC2.5 training) as illustrated in figure 3. Each run corresponds to a unique configuration of 6 parameters. The explored parameter ranges are listed in table 1 and they sum up to a total of 192500 unique parameter configurations for each experiment. In the case of the PU learning experiment, we only trained on 30 relevant articles (30 PT from BC2.5 training) and tested on 60 balanced ones (30 PT and 30 NT from BC2.5 training) as illustrated in figure 3. Finally, the parameter configurations were sorted with respect to the resulting F-scores4 and the top 6 results are reported in table 2 for the four possible outcomes of two different experiments. The F-score is a fair measure between precision and recall when applied to balanced data [23]. Therefore, we use it to evaluate the performance of the ABCRM for all parameter configurations of each of the following two experiments: comparing a range of T-cell death rated to no cell death and comparing training on PT and NT with PU learning. 1

2

3

4

The list of stop words includes 33 of the most common English words from which we manually excluded the word “with”, as we know it to be of importance to protein interaction. The BC2 challenge offered Pubmed abstracts for the classification task. We downloaded some of the full-articles and used a balanced data set of 558 relevant and 558 irrelevant only for the feature selection process. We also used these features in [20, 13]. TF.IDF is a common feature weighting measure to evaluate the importance of a feature/word to a document in a certain corpus. TF stands for term frequency and IDF for inverse document frequency. [10] recision.Recall P P F-score = 2.P where Precision = T PT+F and Recall = T PT+F . True P recision+Recall P N Positives (TP) and False Positives (FP) are our positive predictions while True Negatives (TN) and False Negatives (FN) are our negative predictions.

246

A. Abi-Haidar and L.M. Rocha

Table 1. The parameter ranges used for the parameter search for fine-tuning the algorithm Parameter E0 R0− R0+ dE dR nA

Range [1,7] [3,12] [3,12] [0.0,0.4] [0.0,0.4] [2,22]

Step 1 1 1 0.1 0.1 2

In the first experiment we compare the top 50 parameter configurations obtained using cell death to those with no cell death. We choose only the top 50 configurations to study the algorithm at its best performance that is robust to parameter changes. We conclude that cell death, which helps in the forgetting of useless features, improves the classification performance of the algorithm regardless of whether the algorithm is trained on both PT and NT or not. In the second experiment we compare the top 50 parameter configurations according to F-score obtained using training on both positive and negative to those obtained using training on positive only (PU learning). We conclude that training on both classes gives a better overall performance regardless of cell death. Table 2. Top 6 parameter configurations of the ABCRM in terms of F-score. The top 50 parameter configurations of the four possible outcomes are plotted in figure 5. The highlighted parameter configuration has the highest F-score and is selected for the ABCRM to test on a different set of unbalanced articles in the following subsection.

TRAINING on PT and NT

TRAINING on PT

F-score 0.85 0.84 0.84 0.84 0.83 0.83 0.85 0.84 0.82 0.81 0.81 0.80

CELL DEATH NO CELL DEATH [E0 ,R0− ,R0+ ,dR ,dE ,nA ] F-score [E0 ,R0− ,R0+ ,dR ,dE ,nA ] [ 2 11 10 0.3 0.2 18 ] 0.83 [ 1 4 7 0.0 0.0 18 ] [ 1 11 10 0.3 0.1 22 ] 0.81 [ 1 4 6 0.0 0.0 16 ] [ 1 8 6 0.1 0.1 22 ] 0.78 [ 5 7 6 0.0 0.0 10 ] [ 1 12 6 0.4 0.1 22 ] 0.78 [ 2 5 6 0.0 0.0 16 ] [ 1 9 8 0.3 0.2 22 ] 0.77 [ 2 7 5 0.0 0.0 16 ] [ 1 8 7 0.1 0.1 22 ] 0.77 [ 1 3 3 0.0 0.0 8 ] [ 1 12 8 0.1 0.0 8 ] 0.75 [ 2 12 6 0.0 0.0 18 ] [ 1 8 8 0.3 0.2 16 ] 0.75 [ 2 9 6 0.0 0.0 18 ] [ 1 12 9 0.1 0.0 8 ] 0.75 [ 2 8 6 0.0 0.0 18 ] [ 1 7 10 0.2 0.1 16 ] 0.75 [ 2 11 6 0.0 0.0 18 ] [ 1 11 12 0.4 0.1 18 ] 0.74 [ 2 10 6 0.0 0.0 18 ] [ 3 7 10 0.2 0.3 18 ] 0.73 [ 2 6 6 0.0 0.0 18 ]

We confirm our comparisons statistically using the paired student t-test with the null hypothesis being that the two samples were drawn from the same distribution. We reject the null hypothesis for p-values less than 0.01. The top 6 configurations are listed with their corresponding F-score measure in table 2 and the 50 top-ranked configurations of each of the experiments are plotted in figure 5:

Biomedical Article Classification Using an Agent-Based Model

247

Fig. 5. The two experiments resulting in four possible outcomes: 1.1) training on both sets with cell death (red squares), 2.1) PU learning with cell death (green circles), 1.2) training on both sets with no cell death (blue triangles) and PU learning with no cell death (orange pluses) are clearly distinguishable for the top 50 configurations of each experiment. On the right, the horizontal lines represent the mean, the boxes represent 95%CI, and the whiskers represent standard deviation of F-scores from the top 50 parameter configurations.

5.2

Classification Performance

We finally adopt the parameter configuration from the experiment resulting with the best F-score (highlighted in table 2) and test our algorithm on a larger set of imbalanced full-text articles obtained from BC2.5 as illustrated in figure 3. We then compare our algorithm with the multinomial Naive Bayes (NB) with boolean attributes, explained in [17], and the publicly available SVMlight implementation of support vector machine applied to normalized feature counts [18]. All classifiers were tested on the same features obtained from the same data set. The F-score metric is not very reliable for evaluating imbalanced classification [23], therefore we also use the Area Under the interpolated precision and recall Curve (AUC) to evaluate the performance of the algorithms on the imbalanced BC2.5 testing data. The AUC was the preferred performance measure of the Biocreative 2.5 challenge [4]. Table 3 lists the results in contrast to the central tendancy of the results submitted by all Biocreative 2.5 teams participating in the article classification task. However, the ABCRM, NB, and SVM classifiers, used only single-word features in order to establish the feasibility of the method, while most classifiers submitted to the Biocreative 2.5 challenge used more sophisticated features such as n-grams. Therefore, it is not surprising that the

248

A. Abi-Haidar and L.M. Rocha

Table 3. F-Score and AUC performance of various classifiers when training on the balanced training set of articles and testing on the full unbalanced Biocreative 2.5 testing set. Also shown is the mean performance values for all systems submitted to Biocreative 2.5. Precision Recall F-score AUC

ABCRM 0.22 0.65 0.33 0.34

NB 0.14 0.71 0.24 0.19

SVM BC2.5 Mean 0.24 0.38 0.94 0.68 0.36 0.39 0.46 0.43

performance of these methods was below the average. Nevertheless, when we compare the performance of the ABCRM to NB and SVM on the exact same single-words, the results are encouraging. Hence, we establish the ABCRM as a new bio-inspired text classifier to be further improved in the future with more sophisticated features.

6

Conclusion

We adapted a simple and novel mathematical model of T-cell cross-regulation in the adaptive immune system to recognize multiple textual features and classify biomedical articles using agent based modeling. We tested several variations of our algorithm to classify full-text articles according to their relevance to protein interaction. We obtained encouraging results comparable to state-of-art text classifiers. In summary, we have shown that our novel bio-inspired algorithm is promising for biomedical article classification, and for binary document classification in general.

References 1. Carneiro, J., Leon, K., Caramalho, ´I., van den Dool, C., Gardner, R., Oliveira, V., Bergman, M., Sep´ ulveda, N., Paix˜ ao, T., Faro, J., et al.: When three is not a crowd: a Crossregulation Model of the dynamics and repertoire selection of regulatory CD4 T cells. Immunological Reviews 216(1), 48–68 (2007) 2. Abi-Haidar, A., Rocha, L.: Adaptive Spam Detection Inspired by a CrossRegulation Model of Immune Dynamics: A Study of Concept Drift. In: Bentley, P.J., Lee, D., Jung, S. (eds.) ICARIS 2008. LNCS, vol. 5132, p. 36. Springer, Heidelberg (2008) 3. Abi-Haidar, A., Rocha, L.: Adaptive spam detection inspired by the immune system. In: Bullock, S., Noble, J., Watson, R., Bedau, M.A. (eds.) Artificial Life XI: Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems, pp. 1–8. MIT Press, Cambridge (2008) 4. Krallinger, M., et al.: The BioCreative II. 5 challenge overview. In: Proc. the BioCreative II. 5 Workshop 2009 on Digital Annotations, pp. 7–9 (2009) 5. Myers, G.: Whole-genome DNA sequencing. Computing in Science & Engineering [see also IEEE Computational Science and Engineering] 1(3), 33–43 (1999)

Biomedical Article Classification Using an Agent-Based Model

249

6. Schena, M., Shalon, D., Davis, R., Brown, P., et al.: Quantitative monitoring of gene expression patterns with a complementary DNA microarray. Science (Washington) 270(5235), 467–470 (1995) 7. Hunter, L., Cohen, K.: Biomedical Language Processing: What’s Beyond PubMed? Molecular Cell 21(5), 589–594 (2006) 8. Pubmed 9. Jensen, L.J., Saric, J., Bork, P.: Literature mining for the biologist: from information retrieval to biological discovery. Nat. Rev. Genet. 7(2), 119–129 (2006) 10. Feldman, R., Sanger, J.: The Text Mining Handbook: advanced approaches in analyzing unstructured data. Cambridge University Press, Cambridge (2006) 11. Abi-Haidar, A., Kaur, J., Maguitman, A., Radivojac, P., Rechtsteiner, A., Verspoor, K., Wang, Z., Rocha, L.: Uncovering protein interaction in abstracts and text using a novel linear model and word proximity networks. Genome Biology 9(2), S11 (2008) 12. Krallinger, M., Valencia, A.: Evaluating the detection and ranking of protein interaction relevant articles: the BioCreative challenge interaction article sub-task (IAS). In: Proceedings of the Second Biocreative Challenge Evaluation Workshop (2007) 13. Kolchinsky, A., Abi-Haidar, A., Kaur, J., Hamed, A., Rocha, L.: Classication of protein-protein interaction documents using text and citation network features (in press) 14. Hofmeyr, S.: An Interpretative Introduction to the Immune System. In: Design Principles for the Immune System and Other Distributed Autonomous Systems (2001) 15. Timmis, J.: Artificial immune systems today and tomorrow. Natural Computing 6(1), 1–18 (2007) 16. Twycross, J., Cayzer, S.: An immune system approach to document classification. Master’s thesis, COGS, University of Sussex, UK (2002) 17. Metsis, V., Androutsopoulos, I., Paliouras, G.: Spam Filtering with Naive Bayes– Which Naive Bayes? In: Third Conference on Email and Anti-Spam, CEAS (2006) 18. Joachims, T.: Learning to classify text using support vector machines: methods, theory, and algorithms. Kluwer Academic Publishers, Dordrecht (2002) 19. Abi-Haidar, A., Kaur, J., Maguitman, A., Radivojac, P., Retchsteiner, A., Verspoor, K., Wang, Z., Rocha, L.: Uncovering protein-protein interactions in the bibliome. In: Proceedings of the Second BioCreative Challenge Evaluation Workshop, pp. 247–255 (2007) ISBN 84-933255-6-2 20. Kolchinsky, A., Abi-Haidar, A., Kaur, J., Hamed, A., Rocha, L.: Classification of protein-protein interaction documents using text and citation network features. In: BioCreative II.5 Workshop 2009: Special Session on Digital Annotations, Madrid, Spain, October 7-9, p. 34 (2009) 21. de Sepulveda, N.H.S.: How is the t-cell repertoire shaped (2009) 22. Porter, M.: An algorithm for suffix stripping. In: Program 1966-2006: Celebrating 40 Years of ICT in Libraries, Museums and Archives (2006) 23. Sokolova, M., Japkowicz, N., Szpakowicz, S.: Beyond accuracy, f-score and roc: a family of discriminant measures for performance evaluation. In: Sattar, A., Kang, B.-h. (eds.) AI 2006. LNCS (LNAI), vol. 4304, pp. 1015–1021. Springer, Heidelberg (2006)

An Artificial Immune System Approach for Artificial Chemistries Based on Set Rewriting Daniel Schreckling and Tobias Marktscheffel Institute of IT-Security and Security Law University of Passau Passau, Germany {ds,tm}@sec.uni-passau.de

Abstract. An artificial immune system approach for artificial chemistries (ACs) based on set rewriting methods is introduced. We model signals which are generated by the execution of rewriting rules in the artificial chemistry. They induce signal patterns which trigger a system response. This response is also based on the rewriting rules of the AC. The latter inhibit or accelerate self-reproducing rewriting instructions and eliminate or inhibit non-self rewriting rules in the AC. Exemplarily, the developed artificial immune system model is integrated into the computational model of Fraglets, an AC which is based on multiset rewriting. Experimental results show the feasibility of this approach.

1

Introduction and Motivation

To allow for software systems with self-organising and emergent properties, artificial chemical computing models are gaining increasing prominence [1–4]. The nature of chemical computing metaphor is inherently parallel and decentralised. This makes it an attractive alternative to classical programming methods. We explore the potential of chemical programming models in the context of autonomic environments. In particular, we focus on their potential to withstand different types of attacks. This involves the capability to detect and repair their own failures, and to identify, isolate and eliminate potentially harmful programs. Artificial chemistries (ACs) allow for highly dynamic programs which can easily change structure while still maintaining their original functionality. This makes it hard to detect and isolate them with classical security methods, e.g. intrusion detection, anti-viruses, etc. Further, the freedom that the chemical computing paradigm introduces to support new types of applications and to enable emergent behaviour introduces new threats to the systems which execute the AC. Finally, the AC models have reached a maturity which clearly identify their advantages over classical programming approaches [5–8]. Thus, it is only a matter of time until these models are deployed in real systems. In this contribution we focus on a representative of multiset rewriting based ACs: Fraglets [9]. In this model code and data are considered to be virtual molecules that are transformed using a chemical reaction metaphor. Computational fragments are consumed, produced, and transformed during execution. E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 250–263, 2010. c Springer-Verlag Berlin Heidelberg 2010 

An AIS Approach for ACs Based on Set Rewriting

251

Represented as strings, they can be easily dispersed over several nodes in a network. Hence, on the one hand, self-replication and mobile code are easy to realise, on the other hand, the Fraglets model also generates a large variety of attack vectors which exploit the inherent openness of this approach. The long-term goal of our research is to define an artificial immune system for Fraglets which avoids the introduction of strong security constraints on the computational model keeping its expressiveness and ability to model self-organising systems. To achieve this goal, this contribution combines self-replicating code with a new signal-rule system for the Fraglets framework. This setting allows the modelling of important immune system entities and functionalities which can successfully detect and respond to self- and non-self molecules. This paper is structured as follows: Section 2 first introduces related chemical models, their corresponding security issues, and existing solutions for the MSR model of Fraglets. Section 3 briefly outlines the Fraglets model, its instruction set and self-replication capabilities. Afterwards, Section 4 introduces the signalling model developed for Fraglets. Section 5, finally explains how selfreplication capabilities and our signalling system can be combined to model an artificial immune system (AIS) for fraglets. This section also contains first experimental results. Before we conclude our work in Section 7 we briefly discuss our approach and sketch important future steps in Section 6.

2

Background and Related Work

In this section, we identify security issues of existing artificial chemistries (ACs). For this purpose, we partially follow the classification of ACs given in [10]. We focus on representatives from the classes of assembler automata, arithmetic operations, artificial molecular machines, and of rewriting or production systems. An assembler automata is a parallel computation machine which concurrently executes a population of processes. A very popular assembler automaton is the game Core Wars which forms the basis for the AC core world [11]. In core world it is difficult to identify molecules – single processes – which consist of multiple parts. To address this security issue, Tierra [12] introduced weak memory protection. It guarantees weak integrity for molecules and helps to identify the respective parts of an organism. This contribution will show that Fraglets face the same problem. We use some of the insights from core world to design our immune system approach. Arithmetic operations yield a so called artificial number-division chemistry [2]. It operates on the set of natural numbers as its molecules but can not change the type of reactions. While these chemistries not directly vulnerable, similar but more complex chemistries such as the matrix-multiplication chemistries [13] do yield security problems. Their molecules either represent pure information or executable code. Thus, manipulation or exchange of data can directly induce system damage [14]. Main goal of artificial molecular machines is to map single functional units of a living system into different compartments in which artificial chemistries simulate

252

D. Schreckling and T. Marktscheffel

natural procedures [10]. Molecules are also described as binary strings. They group into data and executable machine molecules. The latter encode transformations which operate on data molecules. Despite different molecule types, there are still transformations which convert data into code and thus form a security risk. Similar to the results from Tierra [12] the generation and extinction of parasites was observed by Ikegami and Hashimoto [15]. Rewriting or production systems (RPS) are based on simple match-and-replace principles. Rules describe which symbol or string of symbols is replaced by which other symbol or string of symbols. An application may consist of one single rule such as in [16, 17] or of various rules comparable to the lambda calculus [18]. The Chemical Abstract Machine [17] ensures the isolated development of populations. It exploits the physical encounter of components, communicating via membranes. Membrane and airlock laws control the diffusion of information. A similar approach has been taken by Martinelli and Petrocchi in [19]. By using an antechamber concept the authors introduce an access control mechanism for the multiset rewriting system of Fraglets [20]. These mechanisms employ classical security concepts increasing the number of constraints on the execution pathways of this rewriting system. This disallows for spontaneous modifications or adaptations – a desirable property of the system. Less intrusive is the approach presented in [21]. It translates the instruction set of Fraglets to semantically equivalent formulae in Maude and allows for the verification of security properties of a specific set of Fraglets. While this allows the analysis of a particular execution flow, it is not possible to alter the original fraglets set during runtime. Meyer presents another approach [22] which makes extensive use of the underlying specific characteristics of the computational model. It introduces a reproduction mechanisms for fraglets which is able to survive code deletion attacks [5, 7, 8] and led to the proposition of a new programming paradigm: Programming by Equilibria [6]. By exploiting the characteristics of the computation model Meyer et al. were able to derive a very effective robustness mechanism. Our work continues this branch of research. Instead of distorting Fraglets to obtain specific security characteristics, it extends its reaction model by simple mechanisms which complement and exploit the new robustness characteristics to yield a powerful artificial immune system.

3

The Fraglets Reaction Model

Fraglets originally aimed at the synthesis and evolution of communication protocols [9]. A fraglet, or computation fragment, is a string of symbols [s1 s2 s3 ... sn ] that can be interpreted as a code/data sequence. There is a fixed number of production rules describing substitution patterns. For example, the application of the rule [sum R X Y TAIL] → [R X+Y TAIL] to the string [sum r 10 13 d] result in the string [r 23 d]. Table 1 lists a selection of such production rules. Fraglets is an artificial chemistry [4] defined by the triple (S, R, A) where S denotes the set of all possible fraglets or molecules. The finite set of production

An AIS Approach for ACs Based on Set Rewriting

253

Table 1. Selected production rules of a Fraglets system. S, T and U are placeholders for symbols, TAIL stands for a potentially empty word of symbols. Instruction Educt(s)

Product(s)

split fork copy nop nul sum match

[PART1] + [PART2] [S TAIL] + [T TAIL] [TAIL] + [TAIL] [TAIL] [] [S i1 +i2 TAIL] (do. for mult etc) [TAIL1 TAIL2]

[split PART1 * PART2] [fork S T TAIL] [copy TAIL] [nop TAIL] [nul TAIL] [sum S i1 i2 TAIL] [match S TAIL1] + [S TAIL2]

rules implicitly define the potentially infinite set of reactions R among molecules. The dynamic behaviour of a simulation is characterised by the algorithm A. Fraglets are injected into a virtual reaction vessel, which maintains a multiset of fraglets and simulates their reactions using the Gillespie algorithm [23]. It calculates the collision (reaction) probability of two fraglets in a well-stirred tank reactor. The reaction rate is proportional to the “concentration” of the reaction educts. 3.1

Auto-catalytic Quines for Robustness

We now explain self-replication with fraglets. In general, a quine is the simplest form of a self-replicating program which prints itself. As discussed in [22], a quine consists of two parts, one which contains the executable code, and the other which contains the blueprint of the code. The code in the blueprint is used twice: First it serves as code which constructs a new quine. Secondly, the blueprint is attached to the new offspring, so that it is also able to replicate. Hence, a simple quine can be built by finding a code and a data fraglet that react and, in doing so, regenerate themselves. Thus, a quine in fraglets would generally look like the following two fraglets: [ match bp fork fork fork nop bp] and [bp match bp fork fork fork nop bp]. Two copies of the information are present: the first is the executable (active) copy, and the second is the code storage (blueprint), guarded by tag bp. Their reaction produces a new fraglet starting with [fork fork ...] which replicates the information by generating two copies of the remaining part. The resulting two identical fraglets again start with a fork instruction. They individually translate the information: One copy reduces to the active part, restarting the cycle, and the other re-installs the original blueprint (see also Figure 1). To support quines the fraglets reactor was extended by a non-selective excess dilution flow. It ensures resource bounds in terms of memory and processing power. Hence, auto-catalytic quines are not able to grow forever. Further, the flux creates an artificial selective pressure such that only programs able to selfreplicate can “survive” in the system.

254

D. Schreckling and T. Marktscheffel [fork fork fork nop bp match bp fork fork fork nop bp] 2 [fork nop bp match bp fork fork fork nop bp]

(seed)

[nop match bp fork fork fork nop bp] [match bp fork fork fork nop bp] [bp match bp fork fork fork nop bp] (active) (blueprint)

Fig. 1. Simple auto-catalytic quine

With these modifications we were able to show how auto-catalytic quines are able to perform simple function computation while keeping their concentrations stable in the presence of a dilution flux [22]. Further, we showed the resistance against targeted code deletion attacks.

4

Fraglets Signalling System

This section adopts the idea of self-replication and combines it with a mechanism able to recognise malicious fraglets and to invoke appropriate system responses. If possible, a fraglets reactor will execute any fraglet which enters the reactor. After its injection it is first recognised again when it reacts with other fraglets or when it transforms itself. This offers numerous possibilities for manipulating the fraglets set stored in a reactor. Therefore, we introduce a signalling system. It reports the execution of instructions stored in a fraglets reactor by generating signals. Each transformation instruction generates the signal for the respective transformation. Reactions additionally generate a signal for the involved reactants. As an example, consider the following fraglets: [sum x 13 100] and [match x result]. The first computes the sum 13 + 100, i.e. sum transforms the original fraglet into [x 113]. This transformation generates the signal sum. In contrast, the second fraglet denotes the reaction match. It requires a reactant with the head tag x. Thus, for the reaction [match x result] + [x 113] → [result 113] the signals match and x are generated. Signals are generated before the actual execution takes place (see also Figure 2). More precisely, after the fraglets reactor has selected fraglets for their execution, they first pass through a signalling reactor where the respective signals associated with this execution are generated. This isolation is necessary as the attacker would otherwise be able to directly manipulate signals. The execution is interrupted and the selected fraglet can not react/transform. This also allows the binding of any fraglet, e.g. a system response, to the head of the fraglet. The signalling system checks whether the newly generated signals generate a pattern which cause the activation of a rule (see next section). Patterns are only generated by signals which belong to the same fraglet. Thus, any rule activation allows us to interfere with the execution of a particular fraglet.

An AIS Approach for ACs Based on Set Rewriting

255

Signal Reactor Rules

Patterns

activate

Signals

match to

produce

generates

Rewritten Code

Code Execution

do nothing (positive rule) prevent Execution (negative rule)

Fraglets Reactor Fig. 2. Signal Processing Scheme

4.1

Patterns and Rules

A pattern is a set of concentrations of particular signals which are accumulated during execution. These patterns can be used to identify characteristic instruction sequences which are either expected or undesirable. Consider an extended version of the quine discussed in Section 3.1): [bp match y split match bp fork fork fork nop bp * sum x 13] and [match y split match bp fork fork fork nop bp * sum x 13] It consumes a fraglet with the head y, e.g. [y A], where A ∈ N. The latter might be generated by an outsider or by another quine in the fraglets reactor. The quine then regenerates itself and computes the sum A + 13. This sequence generates the signal pattern as depicted in Table 2. Please note that the quine Table 2. Signal quantities generated during the quine execution Sig. bp split

Quan. 1 1

Sig. Quan. fork sum

3 1

Sig. match y

Quan. 2 1

Sig. Quan. nop bp

1 1

keeps on regenerating itself. Thus, during this process, the signal sequence does not exactly match the pattern above. However, after one blueprint of this quine has generated a valid output, i.e. the fraglet [x ...], the above pattern fits again. Thus, by multiplying every signal concentration with n we can also recognise every n-th execution of a blueprint. As explained above, signal patterns are used to trigger rules. They either prevent the further execution of the fraglet to which the signals are associated to or they simply allow the fraglet to continue to execute. With each pattern match the signals belonging to the particular pattern are consumed, i.e. deleted from the signalling reactor. As an example, consider the rule r [sig match 10 sig y 15 * NEWHEAD]. The r in front of the string above denotes a rule. It enumerates the signals of the required pattern and their quantity. In our example, the pattern matches

256

D. Schreckling and T. Marktscheffel

after a fraglet has executed 10 or more match instructions and reacted with 15 or more fraglets with head tag y. This rule generates fraglet NEWHEAD concatenated with remainder of the fraglet which generated the signal pattern. This composition is injected into the fraglets reactor, i.e. if fraglet [F] generated the above pattern then fraglet [NEWHEAD F] is injected into the reactor. As NEWHEAD may contain any sequence of instructions we can model any type of response. The associated rules group into positive and negative rules. Positive rules do not interfere with the original fraglets execution. They allow the execution of a fraglet if it generates a particular signal pattern and simply consume the associated signals to suppress other, possibly negative, system responses. In contrast, negative rules allow the generation of countermeasures. These rules inject fraglets into the main reactor which try to encapsulate and destroy the fraglets which generated the signal pattern. Main goal is to interrupt the further execution and to remove possibly malicious by-products. 4.2

Rule to Signal Pattern Affinity

The complexity of a program implemented in fraglets directly increases the complexity of the signal patterns which need to be recognised. Hence, also the probability for false positives increases, i.e. a negative rule is triggered by a signal pattern which does not belong to a malicious fraglet. Likewise, this complexity also increases the difficulty to construct appropriate rules for fraglets which generate unwanted signal patterns. Therefore, it is inevitable to introduce an appropriate affinity measure between rules and signal patterns. Our signalling system measures the difference between generated signals in the fraglets reactor and the required signals specified in the rule system. The smaller the difference the higher the probability the appropriate rule is activated. A rule is activated with 100% probability if all or more of the required signals have been generated. Thus, the processing of signals within the signal reactor resembles the processing of fraglets in the fraglets reactor.

5

Artificial Immune System for Fraglets

This section combines the signalling system with the robustness features we introduced for Fraglets by using auto-catalytic quines. 5.1

Terminology

Antigens are malicious or “non-self” Fraglets which attempt to access data in an unauthorised way using fraglets. In our context, data is represented by other fraglets in the fraglets reactor. Further, fraglets which aim at destroying other fraglets originally stored in a reactor are also considered to be malicious. The attacker can only influence the fraglets reactor by sending new fraglets to this store at a finite rate. The attacker is not able to manipulate the set directly.

An AIS Approach for ACs Based on Set Rewriting

257

Antibodies are fraglets which directly bind to antigens or fraglets which bind to fraglets generated by or required by an antigen. The binding is a matching process which adds a destructive instruction to the antibody. B-quines are auto-catalytic quines which generate fraglets also able to bind to fragments of an antigen to prevent its execution. T helper fraglets are generated by the signalling system to increase the replication speed of B-quines, i.e. T helper fraglets activate B-quines. In contrast, suppressor fraglets, inhibit or decrease the replication speed of B-quines to prevent them to produce antibodies which might bind to non-malicious or “self” fraglets. 5.2

General Structure and Functionality

The general structure of our AIS depicted in Figure 3 is based on the signal processing architecture outlined in Figure 2. The fraglets reactor is executing a program which yields the required functionality, e.g. routing of fraglets to other nodes. To maintain robustness we assume that every program in the reactor is implemented using auto-catalytic quines. We call these quines self quines. Consequentially, the fraglets reactor is configured with a dilution flow (see Section 3.1). Each reactor can receive quines from other reactors. They group selfquines which extend or support functionality and non-self quines otherwise. The execution of every quine generates signals in the signal reactor. Patterns of these signals trigger rules which finally generate the system response. We distinguish four different responses. In case signal patterns of self-quines are generated (1), the corresponding signals are simply deleted from the signal reactor and the quine continues its execution as expected. Steps (2) and (3) usually coupled. If the signal reactor matches a malicious pattern the fraglet whose execution was interrupted, i.e. the antigen which generated the malicious pattern, is directly bound to a killer fraglet (2). It destroys the antigen when it is re-injected to the fraglets reactor. At the same time the system produces helper fraglets (3). They activate B-quines which in turn produce antibodies against the antigen. This activation process increases the replication speed of the B-quines so that it can produce antibodies at a high rate. As B-quines also generate signals, any immune system response can be monitored. In case the system recognises a response which threatens the survival of the remaining system, e.g. it prevents the execution of other fraglets, step (3) can also generate suppressor fraglets which inhibit the high reproduction rate of B-quines. Finally, step (4) can produce more specific B-quines which generate an optimised system response. More precisely, if the signalling system recognises a system response with specific antibodies, it can directly generate new B-quines which respond to the same antigen and generate the specific antibodies. Currently, our implementation does not allow B-quines to match antigens directly (see Section 6). Therefore, new B-quines permanently generate new antibodies to improve the system response in a future attack.

258

D. Schreckling and T. Marktscheffel Signal Reactor Rules

Patterns

activates

Signals

match to

produce

generates

Rewritten Fraglets kill 2 (Killer Fraglets)

do nothing

1

Self & Non-Self Quines

produce produce

Suppressor & Helper Fraglets

generates

4 3

B-Quine(s) activate/moderate

receive

reproduce kills

reproduce

Fraglets Reactor

Fig. 3. Functionality of the Fraglets AIS

5.3

Experimental Results

For simplicity, our experimental settings1 are based on the quine from Section 4.1 as the self-quine whose functionality, the production of fraglet [x ...], must be protected by the AIS. Its input fraglets [y ...] are generated by the interpreter at a constant rate of 500 Hz. Of course, this input may be generated by other fraglets but this would unnecessarily increase the complexity for this example. Additionally, a neutral B-quine is also present in the fraglets reactor. It is powered by a heartbeat fraglet, generated at a lower rate of 31.25 Hz. Again, this heartbeat can be generated by other immune system quines but we avoid this additional complexity in this example. With the dilution flow which restricts the capacity of the fraglets reactor to 1000 fraglets the system approaches a steady state after an initialisation phase. This is also reflected by the shared concentrations of the B-quine,self-quine, and the functionality fraglet [x ...] at the beginning of the simulation. The latter two have 16-fold concentration each compared to the B-quine concentration. The average sum of these fraglets and the input fraglet [y ...] is 1000. Our attack injects the following, rapidly reproducing antigen which generates abnormal very long chaotic sequences: [fork match x x match x fork copy]. Without an immune response this antigen can very quickly interrupt the execution of the self-quines, flood the fraglets reactor, and annihilate any self-quine. As depicted in Figure 4(a)) we start at 1.6sec to inject the antigen with a rate of 400 Fraglets per second for a period of two seconds. The system does not survive. Fortunately, this antigen can be recognised by the following rule: [sig fork 1 sig x 1 sig match 1 * split heartbeat x nul * nul] The corresponding immune response is depicted in Figure 4(b). After one fork of the non-self quine has been executed and after one x has been matched, this rule will first bind the killer fraglet nul to the fraglet triggering the rule. When the killer fraglet executes it automatically destroys the antigen. Secondly, 1

For our experiments we extended the C++ implementation of an existing fraglets interpreter which has been developed at the University of Basel.

An AIS Approach for ACs Based on Set Rewriting

259

(a) Deactivated AIS: Antigens populate the reactor (see avg. fraglets length)

(b) AIS increases reproduction speed of B-quines to generate antibodies Fig. 4. Artificial immune system response

the rule also accelerates the B-quine (green graph) by generating helper fraglets [heartbeat x nul] (dark blue graph) which also contain information about the antibodies to be generated. As a consequence, the B-quine can reproduces itself more quickly. With the beginning of this process it also immediately starts generating the respective antibodies. Here, the antibodies are the fraglets [x nul] (steep yellow graph). During execution, the antigen will produce fraglets of the form [match x ...]. Thus, the antibody can easily bind to the antigen and destroy it. It has no chance to reproduce itself which is also depicted by the flat red graph which indirectly indicates the presence of the antigen. Of course, this immune response tremendously increases the concentration of immune system cells in the reactor. Therefore, the original functionality of the good quine is also partially suppressed, i.e. the blue and light blue graphs drop. The injection of new antigens stops at 3.6 seconds. At this point, no new helper fraglets are generated. As a consequence, the replication speed of the B-quine decreases which also induces a dropping antibody concentration. After some relaxation phase the original concentration steady state is obtained again. The system survives the attack. We conducted experiments in which we directly attacked the self-quines, including the most sensitive components of our AIS, the B-quines. In particular,

260

D. Schreckling and T. Marktscheffel

we designed attacker quines which successfully eliminate or exploit the symbols in the quine required for replication (the blueprint). All of these attacks can be mitigated by appropriate rules in the signal reactor even in a rule system which successfully responds to all of these attacks. Further, we investigated the effect of the slowly decreasing activity of the Bquine on the reaction of this AIS to further attacks. As the reproduction rate decreases rather rapidly we introduced new rules which recognise signal patterns that belong to the immune system response. In case, the AIS recognises the non-self quine and triggers a strong response, i.e. the signal quantities exceed a specific threshold, this new rule generates B-quines which permanently produce antibodies even if the attacks is already. With this method the response time to future attacks was increased and the system is more balanced when the attack takes place. Due to space restrictions we are not able to include these results here.

6

Discussion and Future Work

Biological systems possess innate immunity at birth. This immunity is supported by so called toll-like receptors (TLR). They belong to a group of special receptors which are able to recognise particular pathogens and initiate an appropriate immune response. In our AIS we map TLRs to our signal pattern recognition system. One major difference is the fact that the signalling system can only be activated after fraglets become active. Hence, potential damage, a malicious fraglet may have already caused, must be repaired using robustness mechanisms as introduced in Section 3.1. In real immune systems the overall number of TLRs is rather small. This indicates high specialisation. For complex fraglets system we still have to develop this specialisation. For this purpose we are investigating the widely accepted model of clonal selection [24] in combination with genetic programming applied to our rule system and the involved fraglets. Starting from a general B-quine we generate new and more specific B-quines by modifying the instructions which convey the immune response. If these quines damage self-quines they are withdrawn. All other quines represent new B-quines. So far we did not address cell mediated immunity. It generally ensures that infected or tumour cells are killed. The existing fraglets interpreter supports communication between multiple machines and it is integrated into a derivative of the OMNET++ discrete event simulator. Thus, future work may very well investigate the influence of the AIS on sets of fraglet reactors. The simulator will additionally allow us to investigate how AIS rules can be distributed between different signal reactors. This can be exploited to model passive and active immunisation of a set of fraglets reactors. Humoral immunity on the other hand is mediated by antibodies which are contained in body fluids. It is based on the interaction of B-cells with its corresponding antigen and the generation of antibody plasma. We can interpret a fraglets reactor as the body fluid which contains B- and self-quines. If non-self

An AIS Approach for ACs Based on Set Rewriting

261

quines enter the reactor this humoral immunity is triggered by the signalling system. However, the activation of the B-quines should not require an interaction with the signalling system. Currently, the Fraglets model which does not allow the matching of instructions within a fraglet. To also enable real humoral immunity, we are investigating a modified model which also allows the weighted matching on multiple symbols within a fraglet even on instructions. This will allow for active B-quines which do not require additional helper fraglets. We further identified weaknesses of our signalling and rule system. Currently, we can only define symbol patterns which are specific for a given symbol sequence. With this knowledge an attacker may adjust his attack by replacing single symbols. The AIS would not recognise an attack anymore. The introduction of meta-symbols or placeholder will remove this problem but may also increase the probability for false positives. In some cases we also recognised that the signalling system would benefit from signals which are generated by the products of a reaction. Therefore, we are currently experimenting with a interpreter version which also generates signals for reaction products.

7

Conclusion

This contribution introduced a new approach to address security issues in artificial chemistries based on multiset rewriting. It can be used to counter various types of attacks such as, the unauthorised access to resources or malicious code injection in general. Although our system is specifically designed for the artificial chemistry of Fraglets, the approach is sufficiently general to be integrated in similar rewriting and production systems. To recognise malicious code we showed how to generate isolated signals for each operation which induces a set rewriting. These signals produce patterns triggering rules which can influence the further rewriting, i.e. they modify the rewriting rules. Within the rewriting set itself we further define rewriting rules which take over the functionality of the general AIS components. Through a dilution flow a system pressure is generate which on the one hand can extinguish “non-self” rewriting rules and which on the other hand allow for the application of auto-catalytic rewriting rules. As we can see from Section 6 our research is in the fledgling stages which offers numerous opportunities for future work. However, the combination of our signalling system with the robustness mechanisms and the flexibility and adaptability of fraglets yields promising results and an AIS which is already able to withstand tailored attacks.

Acknowledgements The authors wish to thank Thomas Meyer of the University of Basel for his valuable input on implementation issues and the design of the signalling and response system. We would also like to thank the students Vitali Amann, Lukman Iwan, Eugen Betke, and Eduard Angold who contributed to this framework.

262

D. Schreckling and T. Marktscheffel

The support for this work by the EU FET Project BIONETS (FP6-027748) is gratefully acknowledged.

References 1. Banˆ atre, J.P., Radenac, Y., Fradet, P.: Chemical specification of autonomic systems. In: Proc. 13th International Conference on Intelligent and Adaptive Systems and Software Engineering (IASSE 2004), July 2004, pp. 72–79 (2004) 2. Banzhaf, W., Dittrich, P., Rauhe, H.: Emergent computation by catalytic reactions. Nanotechnology 7, 307–314 (1996) 3. Dittrich, P.: Chemical Computing. In: Banˆ atre, J.-P., Fradet, P., Giavitto, J.-L., Michel, O. (eds.) UPP 2004. LNCS, vol. 3566, pp. 19–32. Springer, Heidelberg (2005) 4. Dittrich, P., Ziegler, J., Banzhaf, W.: Artificial Chemistries – A Review. Artificial Life 7(3), 225–275 (2001) 5. Meyer, T., Yamamoto, L., Tschudin, C.: A Self-Healing Multipath Routing Protocol. In: Proceedings of the 3rd International Conference on Bio-Inspired Models of Network, Information, and Computing Systems (BIONETICS 2008), ICST, Brussels, Belgium, November 25-28, pp. 1–8 (2008) 6. Tschudin, C., Meyer, T.: Programming by Equilibria. In: 15th Kolloquium Programmiersprachen und Grundlagen der Programmierung (KPS 2009), October 2009. Technische Universit¨ at Wien, vol. 2009-X-2, pp. 37–46 (2009) 7. Meyer, T., Tschudin, C.: A Self-Healing Load Balancing Protocol and Implementation. Technical Report CS-2009-001, University of Basel (July 2009) 8. Meyer, T., Tschudin, C.: Chemical networking protocols. In: Proceedings of the 8th ACM Workshop on Hot Topics in Networks (HotNets-VIII) (October 2009) 9. Tschudin, C.: Fraglets - a Metabolistic Execution Model for Communication Protocols. In: Proc. 2nd Annual Symposium on Autonomous Intelligent Networks and Systems (AINS), Menlo Park, USA (July 2003) 10. Dittrich, P., Ziegler, J., Banzhaf, W.: Artificial Chemistries - A Review. Artificial Life 7, 225–275 (2001) 11. Rasmussen, S., Knudsen, C., Feldberg, P., Hindsholm, M.: The coreworld: emergence and evolution of cooperative structures in a computational chemistry. Phys. D 42(1-3), 111–134 (1990) 12. Ray, T.S.: An Approach to the Synthesis of Life. In: Langton, C.G., Taylor, C., Farmer, D.J., Rasmussen, S. (eds.) Artificial Life II, Redwood City, CA, pp. 371–408. Addison-Wesley, Reading (1992) 13. Banzhaf, W.: Self-Organizing Algorithms Derived from RNA Interactions. In: Banzhaf, W., Eckman, F.H. (eds.) Evolution and Biocomputation, Computational Models of Evolution. LNCS, vol. 899, pp. 69–102. Springer, Heidelberg (1995) 14. Banzhaf, W.: Self-replicating sequences of binary numbers. Computers and Mathematics 26, 1 (1993) 15. Ikegami, T., Hashimoto, T.: Active mutation in self-reproducing networks of machines and tapes. Artificial Life 2, 305–318 (1995) 16. Kanada, Y., Hirokawa, M.: Stochastic problem solving by local computation based. In: 27th Hawaii International Conference on System Sciences on Self-organization Paradigm, pp. 82–91 (1994) 17. Berry, G., Boudol, G.: The chemical abstract machine. In: POPL 1990: Proceedings of the 17th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, pp. 81–94. ACM Press, New York (1990)

An AIS Approach for ACs Based on Set Rewriting

263

18. Fontana, W.: Algorithmic chemistry. In: Langton, C.G., Taylor, C., Farmer, J.D., Rasmussen, S. (eds.) Artificial Life II, Redwood City, CA, pp. 159–210. AddisonWesley, Reading (1992) 19. Martinelli, F., Petrocchi, M.: Access control mechanisms for fraglets. In: BIONETICS, ICST (2007) 20. Martinelli, F., Petrocchi, M.: Signed and weighted trust credentials for fraglets. In: BIONETICS, ICST (2008) 21. Martinelli, F., Petrocchi, M.: Executable specification of cryptofraglets in maude for security verification. In: BIONETICS, ICST (December 2009) 22. Meyer, T., Schreckling, D., Tschudin, C., Yamamoto, L.: Robustness to Code and Data Deletion in Autocatalytic Quines. In: Priami, C., Dressler, F., Akan, O.B., Ngom, A. (eds.) Transactions on Computational Systems Biology X. LNCS (LNBI), vol. 5410, pp. 20–40. Springer, Heidelberg (2008) 23. Gillespie, D.T.: Exact Stochastic Simulation of Coupled Chemical Reactions. Journal of Physical Chemistry 81(25), 2340–2361 (1977) 24. Forsdyke, D.R.: The origins of the clonal selection theory of immunity as a case study for evaluation in science. The FASEB journal: official publication of the Federation of American Societies for Experimental Biology 9(2), 164–166 (1995)

Further Experimentation with Hybrid Immune Inspired Network Intrusion Detection Robert L. Fanelli United States Military Academy, West Point, New York [email protected]

Abstract. This paper presents continued experimentation on the Network Threat Recognition with Immune Inspired Anomaly Detection, or NetTRIIAD, model. This hybrid model combines established network monitoring methods with artificial immune system methods to achieve improved performance. The paper presets experiments investigating the model’s performance in detecting novel threats and the performance contribution of the individual components.

1 Introduction Detecting and responding to threats on networks in a reliable and efficient manner presents a significant problem. Established network security monitoring methods, such as misuse-based intrusion detection systems, have proven to be effective safeguards against many known threats. However, reducing the incidence of false alarms and extending the capabilities to detect novel threats will help to keep these methods effective in the future. A network intrusion detection capability that effectively recognizes and mitigates threats with little or no human intervention has been a long-time goal in the security community. Some have viewed the robustness, effectiveness and flexibility of biological immune systems as an inspiration for such a capability, proposing the creation of ‘computer immune systems’ [6]. This paper presents results from continued experimentation on a hybrid threat recognition model that combines established network security monitoring methods with methods from the field of artificial immune systems. This model is intended to overcome some of the limitations seen in these approaches when used separately. The results show some performance gains and suggest some advantages from this hybrid approach. The remainder of this paper is organized as follows. Section 2 presents background on the immunological inspiration for the model and discusses related work. Section 3 provides an overview of the model. Section 4 presents experimentation with a prototype implementation of the model. Section 5 discusses conclusions and areas for future work. E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 264–275, 2010. © Springer-Verlag Berlin Heidelberg 2010

Further Experimentation with Hybrid Immune Inspired Network Intrusion Detection

265

2 Background This section provides an overview of the immunological concepts that inspire the network threat recognition model presented in this paper. This overview is derived primarily from immunology texts [13, 22]. The section also highlights work related to that presented here. 2.1 Overview of Immune System Concepts The immune system is the complex collection of structures and processes that protect an organism from pathogens and other external threats. Most multi-cellular organisms possess some form of innate immune system that reacts to pathogens and other threats in non-specific ways, acting as a dependable and efficient first line of defense. This ability to detect and respond to persistent threats is encoded in the genome and does not normally change during an organism’s lifetime. More complex organisms also possess an adaptive immune system with the capability to recognize and resist pathogens that may evade the innate immune system. The adaptive immune system has the ability to carry out highly specific reactions to new or mutated pathogens, but generally takes longer than the innate immune system to recognize a threat and produce its response. A primary mechanism of the immune system is the binding of various molecular structures, called antigens, with complementary receptors on immune cells and antibodies. This binding allows the immune system to distinguish pathogens from the tissues of the host organism and to direct immune reactions to specific targets. An important concept in immunology is self – nonself discrimination. The antigens that are a normal part of the organism comprise ‘self’ while all other antigens comprise ‘nonself’. The underlying imperative of the immune system is to tolerate self antigens while reacting to nonself antigens. The adaptive immune system contains an immense, continually changing population of effector T lymphocytes, or T cells, each of which can recognize a specific antigen and promote immune reactions to that antigen. During the production of new T cells, a process of negative selection kills off any that bind to self antigens. This process works to prevent autoimmune responses in which the immune system incorrectly attacks the tissues of the organism itself. The innate immune system includes dendritic cells located in tissues throughout the organism. A dendritic cell ingests pathogens and cellular debris in its vicinity and then presents the constituent antigens on its surface. A portion of dendritic cells eventually become mature and migrate to the lymph nodes to present their collected antigens for recognition, potentially leading to an immune reaction. In the lymph node, an effector T cell that binds to an antigen presented by a dendritic cell becomes activated, causing the T cell to proliferate, rapidly producing many copies of itself. The activated T cell copies then circulate throughout the organism, stimulating other immune system components, such as B Cells, to attack the antigen in question. Although self – nonself discrimination describes the fundamental imperative of the immune system, this mechanism alone does not explain the full range of immune system behavior. For example, beneficial intestinal bacteria are tolerated despite being nonself. Similarly, the immune system sometimes reacts to self antigens, harming host tissues and causing autoimmune diseases.

266

R.L. Fanelli

Matzinger’s Danger Model provides an explanation for some of the immune system behavior that is inconsistent with self – nonself discrimination [17]. The Danger Model asserts that the primary trigger for the immune response is not the fact that a pathogen is foreign, or nonself, so much as that it does harm to the organization and is therefore dangerous. The Danger Model holds that dendritic cells are sensitive to chemical danger signals, such as compounds released during necrotic cell death or proteins associated with bacterial flagella. These danger signals stimulate a dendritic cell to mature and signal to the adaptive immune system that the antigens being presented were found in the presence of ‘danger’ and represent a threat. The Danger Model also allows for healthy tissue to emit ‘calming’ or ‘safe’ signals that cause dendritic cells become semi-mature and indicate that the antigens presented are not a threat and should be tolerated [18]. These semi-mature, or safe-context, dendritic cells stimulate the activation and proliferation of regulatory T cells that tend to suppress, rather than promote, immune reactions to their matching antigens. 2.2 Related Work Previous work has applied artificial immune system methods to problems in network intrusion detection. Dasgupta defines an artificial immune system as “an intelligent methodology, inspired by the natural immune system, for real-world problem solving” [3]. Artificial immune systems provide powerful computational capabilities and the intuitive appeal of using the mechanisms that protect biological systems as inspiration for methods to protect information systems. Hofmeyr and Forrest presented a Lightweight Intrusion Detection System, or LISYS, based on a self-nonself discrimination approach [6]. Although able to detect threats, this approach received criticism as being unlikely to scale successfully to real network intrusion problems [25]. Kim and Bentley presented DynamiCS, an immune inspired system intended to overcome scaling issues in LISYS [14]. However, approaches based solely on self - nonself discrimination do not appear to be sufficient for general intrusion detection problems. Aickelin et al. discussed the potential of using the Danger Model as inspiration for intrusion detection [1]. This work proposed using correlation of danger signals as a method to allow an AIS intrusion detection system to scale up to real world problem sizes. Kim et al. subsequently presented an approach to host intrusion detection inspired by the Danger Model [15]. This approach examined sequences of process system calls with the goal of identifying malicious processes by their ‘dangerous’ behavior. The Dendritic Cell Algorithm [9], also inspired by the Danger Model, has been applied with favorable results to security problems such as SYN scan detection [10] and ‘bot’ software detection [2].

3 An Immune Inspired Network Threat Recognition Model The Danger Model serves as inspiration for a hybrid model to detect threats on computer networks. The Network Threat Recognition with Immune Inspired Anomaly Detection (NetTRIIAD) model emulates processes in the innate and adaptive immune systems whereby the immune system decides when and how to

Further Experimentation with Hybrid Immune Inspired Network Intrusion Detection

267

react to threats. The NetTRIIAD model combines established network monitoring methods with artificial immune system techniques in an attempt to overcome limitations found in each. NetTRIIAD is intended to extend and augment existing, proven threat detection capabilities rather than replacing them wholesale. This section provides an overview of the NetTRIIAD model. Additional detail on the model can be found in [4, 5]. Figure 1 depicts the NetTRIIAD model. The model is divided into two main portions, the Innate Layer and Adaptive Layer.

Fig. 1. NetTRIIAD model overview

3.1 The Innate Layer The Innate Layer emulates the activity of dendritic cells collecting antigens and chemical danger signals in the peripheral tissues of an organism. The Innate Layer incorporates established network security monitoring methods to examine network traffic, events and system states and make determinations regarding the danger or safety of the traffic. The Innate Layer consists of a misuse-based network intrusion detection system (NIDS), three danger model signal generators and a Peripheral Immune Node (PIN).

268

R.L. Fanelli

The NIDS, in addition to performing its normal function examining traffic, also synthesizes antigens from the observed packets. The NetTRIIAD antigen is a vector of real-valued features representing an IPv4 packet’s source and destination endpoints and its protocol. The NIDS emits an antigen into the PIN for each packet processed. The danger model signal generators examine data about the state of the network and its attached hosts, emitting corresponding danger model signals. The NetTRIIAD danger model signals may represent an observation of danger or of safety. A danger model signal contains a feature value representing the source of the observation, such as a specific IP address. It also contains a strength value describing the level of danger or safety corresponding to the observation. The generators forward their danger model signals to the PIN for processing. The NetTRIIAD model contains three danger model signal generators, the Alert Monitor, Network Monitor and Host Monitor. The Alert Monitor observes alerts emitted by the NIDS and generates danger model signals corresponding to the source and destination of the alerts and the danger they represent. The Network Monitor observes the overall ‘stress’ of network traffic flow, signaling safety for normal states and danger for unusual states, such as traffic spikes or high numbers of ICMP unreachable packets. The Host Monitor evaluates the ‘health’ of the hosts on the network, signaling safety for hosts operating normally and danger for hosts that are impaired or non-operational. The PIN serves as the location in which a population of artificial dendritic cells (DC) interacts with the antigens synthesized by the NIDS and danger model signals sent from the danger model signal generators. A DC acts as a container for a single antigen type and any corresponding danger model signals. As antigens arrive, the PIN compares them with the current DC population. If a DC exists such that its antigen has identical feature values to those of the arriving antigen, the antigen is absorbed by that DC. If no such DC exists, the PIN creates a new DC to take in the new antigen. DCs remain in the PIN in an ‘immature’ state until they receive sufficient stimulation from danger model signals to become mature. The PIN compares arriving danger model signals with each DC in the population to determine if they bind. Binding occurs when a danger model signal’s feature value matches one of the feature values of a DC’s antigen. Upon binding, the danger model signal stimulates the DC and the DC adds the signal’s danger or safety strength value to its corresponding sum of values. A single danger model signal may bind with and stimulate multiple DCs in the PIN. A DC matures if the sum of stimulation it has received exceeds specified thresholds. If the sum of danger signals exceeds the danger maturation threshold, the DC matures in a dangerous context, indicating that the antigen contained in the DC was found to be in the proximity of some danger and may represent a threat. Conversely, if the sum of safe stimulation exceeds the safe maturation threshold, then DC matures in a safe context indicating that the antigen is unlikely to represent a threat. These thresholds may be adjusted to alter the sensitivity of the detection process. A mature DC leaves the PIN and migrates to the Adaptive Layer to present its antigen and the corresponding context of danger or safety. The Innate Layer thus provides output consisting of a stream of DCs containing antigens, representing network traffic, and a corresponding classification of danger or safety.

Further Experimentation with Hybrid Immune Inspired Network Intrusion Detection

269

3.2 The Adaptive Layer The Adaptive Layer emulates the interactions between mature dendritic cells and the adaptive immune system’s T cells in locations such as the paracortex of a lymph node. The Adaptive Layer is built around an Artificial Lymphatic Paracortex (ALP). The ALP contains a set of artificial T cells (TC), each of which represents a population of identical, activated T cells. NetTRIIAD defines a TC as the combination of an integer population value and a T cell receptor (TCR). The TCR specifies the NetTRIIAD antigen for which the represented population has greatest affinity. Thus the TCR is a feature vector defined identically to the NetTRIIAD antigen described in Section 3.1. TCs are divided into two classes: effectors and regulators. An effector TC promotes immune reactions to antigens matching its TCR while a regulator TC suppresses immune reactions to matching antigens. When a mature DC arrives at the ALP, the antigen presented by the DC undergoes a process of self – nonself discrimination (SNSD). The adaptive layer uses SNSD to determine if an antigen, and thus the corresponding network traffic, is anomalous. The ALP contains a self set representing normal, threat-free traffic on the protected network. The self set consists of antigens derived from training traffic and antigens added through the acquired tolerance mechanism, described below. The ALP compares the antigen presented with the self set. If the antigen matches a member of the self set, the ALP classifies it as self. Otherwise the antigen is considered nonself. When a DC presents a nonself antigen, the TC that has TCR matching the antigen becomes activated. If the presenting DC matured in the dangerous context effector TCs become activated. Conversely, a presentation by safe context DC causes regulator TCs to become active. The population value of the TC increases by an amount equivalent to the danger or safe signal level in the presenting DC. The ALP tolerates self antigens presented by DCs and does not process them further. This represents the fact that no T cell has a TCR matching self antigen, thus there are no matching TCs to be activated. T cells have a finite lifespan. The population level of each TC decays over time, allowing a given antigen presentation to affect the immune reaction process for a period of time but have its effect reduced as time passes. A TC with a population level reaching zero becomes inactive and is removed from the ALP until a matching antigen is again presented by a DC to reactivate it. The ALP periodically evaluates the TC population to determine if an immune reaction should occur. If an effector TC becomes sufficiently activated to overcome the suppression of any matching regulator TC and exceed an immune reaction threshold it becomes a candidate for an immune reaction. Multiple TCs may cross the threshold and become candidates in response to a single threat. The ALP uses a clonal selection process to determine which TC is most specific to the detected threat. The clonal selection groups the candidate TCs into disjoint sets according to the address features in their TCRs and then selects only the TC in the set with the highest stimulation level for an immune reaction. This drives the ALP to react only to the antigen most descriptive of the threat traffic and to ignore the related, but lessspecific, antigens than may also have crossed the immune reaction threshold. The ALP then carries out an immune reaction, emitting a threat alert for the appropriate network source and destination endpoints and protocol.

270

R.L. Fanelli

NetTRIIAD also includes an acquired tolerance mechanism that allows it to adapt to normal, safe traffic over time. A sufficient flow of safe context DC presentations of a given antigen may allow a regulator TC to persist in the ALP despite population decay. If a regulator TC persists in the ALP for a sufficiently long of time, a tolerization reaction occurs. The antigen matching the regulator TC is then added to the self set and the corresponding traffic will be classified as normal. The hybrid NetTRIIAD model preserves the usefulness of established network security monitoring techniques upon which it is built. These tools retain their full detection and reporting capabilities regardless of NetTRIIAD’s performance. NetTRIIAD extends and augments these tools rather than simply replacing them. The alerts emitted by the ALP provide an additional source of information for decision making, combining data from multiple sources to identify dangerous events on the network.

4 Experimentation A prototype implementation of NetTRIIAD is used for experiments into the utility of the model and its performance relative to a typical misuse-based NIDS. The NetTRIIAD components are separate processes, connected by socket-based interprocess communication. This enables flexible deployment of the components in support of typical network intrusion detection architectures. Snort version 2.6.1.3 serves as the misuse-based NIDS component of the NetTRIIAD implementation [24]. A custom Snort plug-in synthesizes antigens and gathers network statistics for use by the Network Monitor process. The prototype implementation uses the Nagios host monitoring application to collect host status information for the Host Monitor [7]. The PIN process runs alongside Snort on the IDS sensor platform. The ALP process runs on a separate host, allowing it serve as a central analysis point for multiple sensor platforms hosting pairings of Snort and the PIN. These experiments utilize the DARPA / MIT 1999 Intrusion Detection Evaluation (IDEVAL99) data sets [11]. Despite shortcomings noted by McHugh [19] and Mahoney and Chan [16], this work is well-documented and represents one of the few open source data sets for controlled, repeatable intrusion detection experimentation. The IDEVAL99 data includes two weeks of threat-free traffic for training. This traffic yields the ALP self set for these experiments, comprising 31,215 distinct NetTRIIAD antigens. The IDEVAL99 data sets also contain two weeks of test set traffic containing 187 network threat events. NetTRIIAD and Snort alone achieve true positive rates of 0.30 and 0.33, respectively, on this data. The difference between these rates is not significant (t = 1.01766, df = 16, p > .25). Positive predictive value (PPV) gives a measure of the likelihood that a positive threat detection result represents a true positive (TP) rather than a false positive (FP). PPV is the ratio of true positive results to all positive results. PPV = TP / (TP + FP) .

(1)

PPV is a useful metric for the effectiveness of an intrusion detective system. A higher PPV gives greater confidence that a response to an IDS alert is in fact going towards a

Further Experimentation with Hybrid Immune Inspired Network Intrusion Detection

271

true threat and not an innocuous event. This is useful to make the most efficient use of finite human resources or to ensure that automated responses do not affect legitimate traffic. On the IDEVAL99 data, NetTRIIAD achieves a PPV of 0.65, significantly better than the PPV of 0.38 for Snort alone (t = 4.85328, df=16, p < .001) [4]. Both Snort and NetTRIIAD operate deterministically. Replaying the IDEVAL99 test data under consistent conditions yields identical detection results across multiple runs. Each of the experiments with NetTRIIAD and Snort alone was confirmed with identical detection results on no fewer than three repetitions of each day of the IDEVAL99 test data. 4.1 Experiments with Novel Threats This set of experiments investigates the ability of NetTRIIAD to detect novel threats. The IDEVAL99 data does not contain any threats that are novel to an IDS rule set created years later. Thus, a two-pronged approach was used to facilitate testing NetTRIIAD versus novel threats. The first method involves removing any knowledge of some of the threats contained in the data sets from the misuse-based IDS and other parts of the NetTRIIAD implementation, making these threats effectively novel. The second method is to introduce newly-created threats that exploit vulnerabilities disclosed after implementation of the prototype system. Three threats contained in the IDEVAL99 data serve as effectively novel attacks. The choice of these three threat stems from Snort’s ability to reliably detect them in the IDEVAL99 traffic using specific signatures. The first such threat is a named buffer overflow attack on BIND DNS, detected by Snort signature 3153 DNS TCP inverse query overflow. The second is the back Apache web server denial of service attack, detected by Snort signature 1156 apache directory disclosure attempt. The third is an exploit for phf, a vulnerable Apache CGI script. This exploit matches Snort signatures 886 phf access and 1763 phf arbitrary command execution attempt. After removing the corresponding signatures, Snort no longer detects these threats. Three custom threats, constructed using the Metasploit Framework [20], exploit the ‘DNS RPC management’ vulnerability in some versions of Microsoft Windows [21]. The custom threats employ three different payloads, one returning a reverse shell to the attacker, one creating a new administrator account on the target and one attempting to install and run the VNC service, giving the attacker a remote desktop on the target. The Snort release and rule base used in these experiments predate the disclosure of the ‘DNS RPC management’ vulnerability. When tested alone, this Snort implementation does not emit alerts for these novel threats. The experiments examine NetTRIIAD’s detection of novel threats during multiple replays of the IDEVAL99 data. The three custom threats are added to the Week 5 Day 1 traffic by directing them at a live host configured to emulate part of the IDEVAL99 network. NetTRIIAD is considered to have successfully detected a threat when it reacts to the appropriate antigen, thus identifying the source and destination of the threat traffic. Table 1 depicts the novel threat detection results. NetTRIIAD displays some ability to detect novel threats without explicit prior knowledge, primarily by the danger they represent and the anomalous, or non-self, nature of their network traffic. The somewhat more disruptive and ‘noisy’ attacks are readily detected by NetTRIIAD. For example, the named attack results in generation

272

R.L. Fanelli

of danger signals both from its effect of temporarily disabling the DNS service and its use of a recognizable ‘NOP sled’, a common exploitation technique. Conversely, the ‘phf’ and ‘DNS RPC with add user’ attacks create little traffic and do not disrupt the operation of their targets. At the network level, these attacks to not create events seen as dangerous and thus go undetected by NetTRIIAD. Table 1. Novel threat detection results

Novel Threat

Result

Buffer Overflow, named Denial of Service, back Vulnerable Script, phf Custom Threat 1, DNS RPC with reverse shell Custom Threat 2, DNS RPC with add user Custom Threat 3, DNS RPC with inject VNC server

Detected Detected Missed Detected Missed Detected

4.2 Ablation Experiments A series of ablation experiments provide insight into the contribution of the components to the overall NetTRIIAD performance. Ablation experiments systematically remove system components in an attempt to learn about the contribution of the components to overall system performance. In previous work, Glickman, Balthrop and Forrest used ablation experiments to examine the LISYS AIS [8]. Each ablation experiment is a separate threat detection run against the IDEVAL99 Week 5 Day 1 traffic. This day of traffic contains a large and diverse set of threat events, providing the best input for the ablation experiments. Table 2 relates detection performance for the ablation experiments. The ‘disabled SNSD’ experiment examines NetTRIIAD performance with no self nonself discrimination capability. All antigens are thus treated as nonself. This result demonstrates the importance of the self – nonself discrimination capability in reducing false positive detections and maintaining a favorable positive predictive value. The ‘empty self set’ experiment allows NetTRIIAD to perform self - nonself discrimination, but gives it an initially empty self set. The acquired tolerance mechanism adds antigens to the self set during the run, allowing the system to tolerate normal, safe traffic. This effect is shown by the lower number of false positive detections when compared to the previous experiment with self - nonself discrimination completely disabled. The ‘disabled alert monitor’, ‘disabled host monitor’ and ‘disabled network monitor’ experiments investigate NetTRIIAD performance with each of these danger model signal generators disabled in turn. Depriving the system these information sources has a negative effect on overall performance. However, the Network Monitor appears to provide the least benefit to NetTRIIAD. Disabling this component has no effect on the true positive detections and only a small increase in false positive detections.

Further Experimentation with Hybrid Immune Inspired Network Intrusion Detection

273

Table 2. Ablation experiment results

True Positive Detections

False Positive Detections

False Negative Detections

Positive Predictive Value

True Positive Rate

Disabled SNSD

9

37

17

0.196

0.346

Empty Self Set

9

29

17

0.237

0.346

Disabled Alert Monitor

7

4

19

0.636

0.269

Disabled Host Monitor

3

3

23

0.500

0.115

Disabled Network Monitor

10

7

16

0.588

0.385

Full NetTRIIAD

10

6

16

0.625

0.385

W5D1 26 Threat Events Ablation Experiment:

5 Conclusions and Future Work The NetTRIIAD model demonstrates some benefit from a hybrid, immune-inspired approach for detection of network threats. The positive predictive value is significantly higher than that of a misuse-based NIDS working alone. This can be useful to better prioritize limited security resources to focus on more likely threats. A higher positive predictive value is also beneficial for avoiding the unwanted disruption of legitimate traffic by any automated security response mechanisms relying on the detection method. The Danger Model inspired approach in NetTRIIAD is also successful at detecting threats without specific prior knowledge. This novel threat detection is more successful with disruptive or noisy threats. However, this capability has the potential to identify and contain novel, fast-moving threats such as worms exploiting ‘zeroday’ vulnerabilities, where an automated response could be especially beneficial. Future work may extend NetTRIIAD to include more of the immune system metaphor, adding automated responses derived from the immune alerts emitted. The ablation testing shows the utility of the self-nonself discrimination capability in the Adaptive Layer to reduce false positive detections. The ablation tests also show the disappointing contribution by the Network Monitor to the overall performance of NetTRIIAD. Future work may investigate the use of different metrics and danger computations to improve the contribution by the Network Monitor. Experiments with the IDEVAL99 data sets provide some initial insight into the performance of the NetTRIIAD prototype. Additional experimentation with other network traffic is needed. Ideally, this experimentation would use more recent network

274

R.L. Fanelli

traffic data that is as well-documented and controlled as the IDEVAL99 data. The labeled data capture project of Sangster et al. is one potential source of such data [23]. Future work could also investigate the potential for basing antigen creation and detection at the level of network flows. Network flows summarize network traffic in terms of conversations, such as TCP sessions, rather than as individual packets. This could provide a larger context for deciding if a threat is present but still allow for the identification of IP addresses, ports and protocols needed to implement automated traffic shaping. Adding data collection from the host level would provide additional sources of danger model signals. The original NetTRIIAD concept has been to stay entirely network-based and avoid a requirement to install host level agents. However, host level information, such as system events logs, process behavior or file integrity could be useful in detecting and containing threats that are not detectable with network monitoring alone.

References [1] Aickelin, U., et al.: Danger Theory: The Link between AIS and IDS? In: Timmis, J., Bentley, P.J., Hart, E. (eds.) ICARIS 2003. LNCS, vol. 2787, pp. 147–155. Springer, Heidelberg (2003) [2] Al-Hammadi, Y., Aickelin, U., Greensmith, J.: The DCA for Bot Detection. In: Proceedings of the IEEE World Congress on Evolutionary Computation 2008 (CEC 2008), pp. 1807–1816. IEEE Press, New York (2008) [3] Dasgupta, D. (ed.): An Overview of Artificial Immune Systems and Their Applications. Springer, Heidelberg (1998) [4] Fanelli, R.: A Hybrid Model for Immune Inspired Network Intrusion Detection. In: Bentley, P.J., Lee, D., Jung, S. (eds.) ICARIS 2008. LNCS, vol. 5132, pp. 107–118. Springer, Heidelberg (2008) [5] Fanelli, R.: Network Threat Detection Utilizing Adaptive and Innate Immune System Metaphors. Dissertation, University of Hawaii (2008) [6] Forrest, S., Hofmeyr, S.A., Somayaji, A.: Computer immunology. Communications of the ACM 40(10), 88–96 (1997) [7] Galstad, E.: Nagios Home Page, http://www.nagios.org [8] Glickman, M., Balthrop, J., Forrest, S.: A Machine Learning Evaluation of an Artificial Immune System. Evolutionary Computation Journal 13(2), 179–212 (2005) [9] Greensmith, J., Aickelin, U., Cayzer, S.: Introducing Dendritic Cells as a Novel ImmuneInspired Algorithm for Anomaly Detection. In: Jacob, C., Pilat, M.L., Bentley, P.J., Timmis, J.I. (eds.) ICARIS 2005. LNCS, vol. 3627, pp. 153–167. Springer, Heidelberg (2005) [10] Greensmith, J., Aickelin, U.: Dendritic Cells for SYN Scan Detection. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2007), London, pp. 49–56 (2007) [11] Haines, J.W., et al.: 1999 DARPA Intrusion Detection Evaluation: Design and Procedures, TR-1062, Lincoln Laboratory, Massachusetts Institute of Technology, Lexington, MA (2001) [12] Hofmeyr, S.A., Forrest, S.: Architecture for an Artificial Immune System. IEEE Transactions on Evolutionary Computation 8(4), 443–473 (2000) [13] Janeway, C.A.: Immunobiology, 6th edn. Garland Science, New York (2005)

Further Experimentation with Hybrid Immune Inspired Network Intrusion Detection

275

[14] Kim, J., Bentley, P.: Towards an Artificial Immune System for Network Intrusion Detection: An Investigation of Dynamic Clonal Selection. In: Proceedings of the 2001 Congress on Evolutionary Computation (CEC 2001), Seoul, pp. 1244–1252 (2002) [15] Kim, J.W., et al.: Malicious Code Execution Detection and Response Immune System Inspired by the Danger Theory. In: Proceedings of the Adaptive and Resilient Computing Security Workshop (ARCS 2005), Santa Fe, NM (2005) [16] Mahoney, M.V., Chan, P.K.: An Analysis of the 1999 DARPA/Lincoln Laboratory Evaluation Data for Network Anomaly Detection. In: Vigna, G., Krügel, C., Jonsson, E. (eds.) RAID 2003. LNCS, vol. 2820, pp. 220–237. Springer, Heidelberg (2003) [17] Matzinger, P.: Tolerance, Danger, and the Extended Family. Annual Review of Immunology 12, 991–1045 (1994) [18] Matzinger, P.: Friendly and dangerous signals: is the tissue in control? Nature Immunology 8(1), 11–13 (2007) [19] McHugh, J.: Testing Intrusion Detection Systems: A Critique of the 1998 and 1999 DARPA Intrusion Detection System Evaluations as Performed by Lincoln Laboratory. ACM Transactions on Information and System Security 3(4), 262–294 (2000) [20] Metasploit LLC: The Metasploit Project, http://www.metasploit.com/ [21] MITRE Corp: Common Vulnerabilities and Exposures - CVE-2007-1748, http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-1748 [22] Paul, W.E.: Fundamental Immunology, 4th edn. Lippincott Williams & Wilkins, Philadelphia (1998) [23] Sangster, B., et al.: Toward Instrumenting Network Warfare Competitions to Generate Labeled Datasets. In: USENIX Security’s Workshop on Cyber Security Experimentation and Test, CSET (2009) [24] Sourcefire Inc: Snort - The Open Source Network Intrusion Detection System, http://www.snort.org [25] Stibor, T., Timmis, J., Eckert, C.: On the Appropriateness of Negative Selection Defined Over Hamming Shape-Space as a Network Intrusion Detection System. In: Proceedings of the 2005 IEEE Congress on Evolutionary Computation, pp. 995–1002. IEEE Press, New York (2005)

Danger Theory and Intrusion Detection: Possibilities and Limitations of the Analogy Mark Vella, Marc Roper, and Sotirios Terzis Department of Computer and Information Sciences, University of Strathclyde, Livingstone Tower, 26 Richmond Street, G1 1XH Glasgow, United Kingdom {mark.vella,marc.roper,sotirios.terzis}@cis.strath.ac.uk

Abstract. Metaphors derived from Danger Theory, a hypothesized model of how the human immune system works, have been applied to the intrusion detection domain. The major contribution in this area, is the dendritic cell algorithm (DCA). This paper presents an in-depth analysis of results obtained from two previous experiments, regarding the suitability of the danger theory analogy in constructing intrusion detection systems for web applications. These detectors would be capable of detecting novel attacks while improving on the limitations of anomalybased intrusion detectors. In particular, this analysis investigates which aspects of this analogy are suitable for this purpose, and which aspects of the analogy are counterproductive if utilized in the way originally suggested by danger theory. Several suggestions are given for those aspects of danger theory that are identified to require modification, indicating the possibility of further pursuing this approach. These modifications could be realized in terms of developing a robust signal selection schema and a suitable correlation algorithm. This would allow for an intrusion detection approach that has the potential to overcome those limitations presently associated with existing techniques.

1

Introduction

Intrusion detection systems (IDS) are employed within the domain of computer security with the goal of flagging system behavior caused by malicious activity. Anomaly detection is a type of intrusion detection [17,25] that allows for novel attack detection, in contrast to the more popular misuse detectors [5], that are only able to detect known attacks. Current anomaly detectors designed to detect web-based attacks, are shown to be limited by unacceptable detection effectiveness rates [21] and the need for continuous administration effort [36,27]. The idea of applying Artificial Immune Systems (AIS) to the intrusion detection problem is proposed on the basis of the ability of the Human Immune System (HIS) to efficiently solve a similar problem [32]. The HIS, specifically the adaptive immune system, is capable of detecting previously unseen infections, with a low recorded rate of auto-immune responses (synonymous to false positives within the intrusion detection problem). Moreover, the HIS functions in a lightweight, distributed and self-organized manner, coinciding with all the E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 276–289, 2010. c Springer-Verlag Berlin Heidelberg 2010 

Danger Theory and Intrusion Detection

277

desired characteristics for an IDS [22]. Initial AIS algorithms, called Negative Selection Algorithms (NSA), were based on the central tolerance analogy [10,8]. This is a process that goes on in the thymus and bone marrow allowing the HIS to avoid auto-immune responses. This analogy demonstrated the potential for this kind of algorithm to be able to effectively detect virus infected files, and stimulated further research on network intrusion [18] and process anomaly [19] detection. However, this approach suffers from scalability issues [23,4] and difficulty in detecting malicious behavior that is close to normal behavior [16]. Given the limitations of these initial algorithms, an alternative model of the HIS, the Danger Theory [28], was proposed [2] as being a more promising approach to intrusion detection [1]. Danger Theory-based algorithms have been evolving over the last couple of years [33,13,12], and have been applied to the problem of process anomaly and SYN/Ping scan detection. In this paper, we aim to provide deeper insight into the utility of the danger theory analogy to intrusion detection. More specifically, we examine the potential of this analogy in constructing an IDS capable of effective novel attack detection targeting web applications, with minimal administrative effort. The analysis shows that whilst the overall mechanism suggested by danger theory provides us with an interesting analogy with which to approach the problem of intrusion detection, there exist various aspects of this analogy that would be counterproductive if employed as suggested by this theory. Despite of these limitations, several suggestions are proposed for how these aspects could be modified in order to better support the intrusion detection application domain. These modifications to the danger theory analogy, provide us with the necessary ground work for the development of a danger theory-inspired detector. This type of detection has the potential for overcoming those limitation presently associated with misuse and anomaly detectors. Specifically, the development of a robust signal selection schema and an appropriate correlation algorithm, would allow for the realization of this potential. The work presented in this paper concerns specifically the domain of web applications.

2

Danger Theory as a Model for Novel Attack Detection

The Danger Theory model of the HIS [28,2], was proposed by immunologists in an attempt to address the limitations of the negative selection model, and to explain those “safe” non-self antigen that do not trigger an immune response. Danger theory hypothesizes that, in the case of a previously unseen infection, an immune system response is only triggered when sampled antigen is correlated with the presence of signs of an ongoing infection (or danger signals). In the human body these signs consist of the residue from necrotic cell death. When non-self antigen is not correlated with danger signals (or synonymously correlated with safe signals), an immune response is not triggered, and this antigen is said to be tolerated via peripheral tolerance (figure 1). The most representative AIS of this kind is the Dendritic Cell Algorithm (DCA) [14,12,15]. This algorithm, rather than basing detection on antigen pattern matching as in the negative selection approach, works by correlating environment antigen with

278

M. Vella, M. Roper, and S. Terzis

perceived environment signals. These signals, as defined by a signal selection schema, allow for ranking monitored antigen in terms of suspicion. Signal selection schemas are made up of danger and safe signals, where danger signals are associated with system wide signs of successful attacks, whilst safe signals with their normal behavior counterpart. A danger context consists of a period of system execution associated with a higher concentration of danger signal equivalents. Antigen associated with a danger context, is ranked highest in this suspicion ranking. This approach has the potential to overcome the scalability issues of NSA’s, due to the elimination of the detector generation phase. However, the proper formulation of a signal selection schema becomes crucial. It is not as yet completely understood whether a schema exists, that can correctly characterize danger contexts and properly distinguish between normal and malicious behavior. The DCA has been shown to be sensitive to safe/danger/safe context switches [13], and tolerant to a certain amount of delay between antigen sampling and related signal detection [12]. On the other hand, it has also been criticized to involve too many configuration parameters, self antigen being wrongly regarded as suspicious due to co-occurrence with dangerous non-self antigen (termed innocent bystanders, see figure 1), and incomplete antigen ranking in environments where not enough copies of the same antigen are generated [11].

Fig. 1. Danger theory detection

Based on the potential of this approach, two experiments were carried out. The first experiment [34], investigated the nature of signal schema formulation, since this is the crucial point in achieving correct antigen classification, and thus directly affecting detection effectiveness. In particular, the robustness of the schema for the ping scan experiment was evaluated, i.e. how easy it is for such a schema to misclassify monitored normal sessions that are also associated with danger signals. The second experiment [35], investigated the potential set of candidate danger and safe signals for formulation of a single signal selection schema that serves for the detection of novel attacks against all possible web-based systems. Given the results derived from these two experiments, the next two sections aim to share our insights about the danger theory analogy to intrusion detection.

Danger Theory and Intrusion Detection

3

279

Up Until Which Point Does the Analogy Hold? - The Benefits of the Analogy

The intrusion detection problem could benefit from the following aspects of the danger theory analogy: Detecting novel malicious patterns - Danger theory explains the detection of unknown pathogenic proteins by basing detection on the effects these cause in the infected cells, rather than on their exhibited molecular pattern. Similarly, the analogy would allow an IDS to detect new types of intrusions from the damage that these cause to computer systems, instead of utilizing signature matching for detecting specific (possibly novel) attack techniques. Whichever the employed attack technique and associated misuse signature, some form of damage is expected, and possibly detectable via an increase in the level values of danger signal equivalents. This way, a detection technique based on this analogy has the potential to keep a low false negative (FN) rate. Simply modifying the attack technique would not be enough to evade detection. Safe non-self tolerance - In the danger theory, any antigen that is not associated with danger signals, even though it may not be recognized to form part of the self antigen set, is not responded against. In this same way, conversely to the previous point, in a danger theory-based IDS, any traffic that does not cause damage to the underlying execution environment will not be flagged as malicious behavior. This also applies to unsuccessful attack traffic. The sole purpose of such traffic would be to increase unnecessarily the alarm rate, with the scope of distracting systems security officers from the real alarms, as with snot and stick attacks [33]. This way, an IDS that has the potential to keep a low false positives (FP) rate by design, could be achieved. Autonomy - The danger theory also explains how the HIS responds to novel infections in an autonomous manner. In the HIS there is no external component that provides some self profile or informs it of new types of pathogen. The HIS is completely autonomous and able to learn new types of pathogenic patterns. A danger theory-based detection system would not require a normal behavior profile to be continuously supplied. The only requirement would be access to the monitored antigen, and the environment signal level values, with which to correlate the antigen and assign suspicion rankings. A set of these environment signals is referred to as a signal selection schema. Once a signal selection schema that is formulated, the detection system would require no further maintenance. In our case we are interested in investigating the possibility of identifying a single signal selection schema that represents the notion of distress common to all web applications [35]. Part of a multi-layered defense system - The HIS is made up of multiple layers of defense [32]. The first layer consists of the physiological barrier presented by the skin and the pH value of the intestinal fluids. Pathogenic proteins that make their way through this initial barrier are then met by the innate immune system that recognizes a set of pathogenic material as built-in knowledge. In the danger theory, this layer is modeled by Pathogenic Associated Molecular Patterns (PAMP) signals, that allow the efficient detection of known infectious agents.

280

M. Vella, M. Roper, and S. Terzis

Pathogens that are not detected by the innate immune system are then detected at the point when successful infection is underway. Cell internals released during necrotic cell death that ensues during infection, allow for the creation of a danger context with which the responsible pathogen is correlated via the function of dendritic cells. This eventually triggers an immune system response when the sampled pathogenic antigen is presented by dendritic cells to antibodies residing in lymph nodes. The adaptive immune system is also able to efficiently detect the re-introduction of the same pathogen within the human body via memorized specific antibodies. This way, the adaptive immune system is capable both of an inefficient primary response and a secondary efficient response. In line with this analogy, a danger theory-based IDS could be positioned to provide functionality similar to the primary inefficient response of the adaptive immune system. One possible scenario in line with this analogy could be: the physiological first defensive barrier represented by the various preventive secure mechanisms such as file permissions, firewall rules, secure protocols and end-user input sanitization filters [29] [31]; the innate and efficient secondary response of the adaptive layers, could be represented by a misuse detector that is able to efficiently detect known attacks [5]; the role for the detection of novel attacks is then left to the danger theory-based intrusion detection component, that is able to identify novel attacks based on correlation of monitored system transactions with system-wide signal level values.

4

Beyond Which Point Does the Analogy Break? - The Limits of the Analogy

Though several benefits exist for the application of danger theory metaphors as inspiration for the design of IDS’s, our experimentation also highlighted the limitations of this analogy in certain aspects. Danger signal characterization - Whilst the danger theory of how the human immune system works is based on the fact that a pathogenic infection always causes the same type of distress, that of necrotic cell death, the same does not apply to computer systems. In fact, experimental results [35] show that a multitude of danger signals are necessary in order to capture all possible types of distress symptoms. In this case, when several attacks were launched against a vulnerable web-based system, different attacks were observed to affect different kinds of probes. An increased HTTP rate, an increased system load, and an increased rate of error log entries, are only some of the identified possible danger signals. This poses the problem of completeness of danger signal coverage i.e. running the risk of having attacks that are not represented by the chosen set of danger signals. Moreover, it is not as yet clear whether all possible types of attacks within a computer system cause such observable damage. For example, one possibility could involve attacks that exploit insufficient mechanisms, such as the use of predictable passwords, or even worse, weak password management [20,31]. In such cases, an attack is possible without requiring the attacker to ’break’ secure mechanisms in a noisy manner (e.g. as compared to a fully fledged

Danger Theory and Intrusion Detection

281

password brute-forcing attack), but rather allowing an attack to take place in a manner that is indistinguishable from normal behavior (e.g. a successful login preceded by just a couple of failed attempts, or an apparently normal password changing procedure). Another possibility could involve attacks where attackers exploit powerful runtime features at their disposal. A notorious example is the register globals setting in PHP server scripting, where the simple setting of an HTTP GET variable is enough to modify the behavior of a server process [30]. A typical example would be sending a request of the form GET /auth.php?authorized=1, causing all script sections simply based on the non-initialized variable $authorized, to be set to 1. This would allow non-authorized access to privileged functionality. Though this specific feature has been removed from the latest version of PHP, interpreted run-time environments still allow the execution of run-time fed instructions via some form of eval(script instructions sequence) function. One further type of attacks that may not result in obvious danger signals, are those that exploit trust. A typical example of such an attack is a disgruntled administrator that is entrusted with high system privileges, but these are then employed in a malicious manner for personal benefit. What is observed in this case by a system monitor, is a sequence of system actions carried out by a user account that is sufficiently privileged. In addition to these type of attacks that do not generate obvious signals, there are also those attacks that exploit a vulnerability on one host, but then execute their payload on another. One typical example is that of SQL injection attacks [30,31] occurring in web application scenarios, where once the vulnerable server code (e.g. $query = ”SELECT * from Customers where cust id = $getvarid”) is exploited (e.g. GET /customers.php?id=1OR%201%3D1 ), the actual injected SQL command (”SELECT * from Customers where cust id = 1 OR 1=1”) is transported to the database server and the payload executed over there. In this case, detecting damage is not trivial since at the point where the successful attack takes place on the host running the database service, it is not easy to distinguish whether the database service is executing a legitimate SQL query, or one that was maliciously injected by an attacker within the web application server code. Another related example attack, is that of Cross Site Scripting (XSS), where this time the attack payload is executed on the web client host (e.g. injection of <script type=”text/javascript”> window.location =”http://www.attackerwebsite .com” ). Detecting distress at the point of exploitation is also not trivial, since in both cases the attacker is simply exploiting a run-time feature. In the first case that of allowing dynamic construction of SQL queries, and in the latter that of allowing dynamic web content to be sent to the web client, based on the previously received user input. Conversely, there is also the problem of danger signals that are not exclusively related to attack signals. These danger signals could be seen to overgeneralize, in that rather than just covering over all possible attack techniques, these also cover normal behavior. This poses an obvious FP rate risk. For example, the danger signal of an increased system load, a possible sign of distress caused by

282

M. Vella, M. Roper, and S. Terzis

a denial of service attack, could also be the result of increased system usage. Another example could be of an increase in the HTTP request rate, where this could be the distress sign of some ongoing malicious profiling activity, but also could represent an increase in application usage. Safe signal characterization - The role of the safe signal in the danger theory [28] is that of activating peripheral tolerance in order not to react against nonself that is not dangerous. This may be characterized by contexts that lack the presence of danger signals, or possibly an active signal indicating stability. Experimental results [34], show that a safe signal based on this analogy is not enough to avoid false positives. In this case [34], the goal of the experiment was that of distinguishing between a malicious ping scan and a benign file upload. The danger signal was characterized as the outbound traffic rate and the safe signal as the inverse rate of change of the outbound traffic, representing stability. An increase in the file upload size was shown to increase the danger signal to a level even exceeding the one associated with the pings scan session. The safe signal in this case was not enough to tolerate the file upload session in order to avoid misclassification. Time-based correlation - Correlation of antigen with environment signals within the human body is presumed to happen on a time basis. This is logical since during an ongoing infection, it is expected that pathogens are present within the human tissue cells while these are being infected. On the other hand, this may not always be the case within a computer system. In a process anomaly detection scenario [14], where antigen is represented by a process id, the analogy holds. However this does not hold when concerned with the flagging of network packets, where the execution of the attack payload could be delayed. This would cause signs of distress associated with the attack payload to be observed long after malicious antigen sampling has been carried out. One example could be a command injection attack [30], that injects the command - ; sleep 3600; wget http://www.attacker.com/malware.file. In this case, a whole hour would pass before the observable signs of a suspicious file download can be detected. This gives rise to the unwanted possibility that the attack HTTP request is not correlated with these signs of distress, but instead other benign HTTP requests would be. This example highlights the fact that infection within the human body and a computer system may vary in terms of the simultaneous occurrence of infectious antigen and the associated signs of infection. The DCA [13] conforms to this analogy, but so far has only been evaluated in process anomaly detection scenarios, where malicious antigen is expected to occur in simultaneous with the signs of distress. Whilst the DCA is tolerant to a limited amount of delay between antigen sampling and danger signal observation, extending the delay more than a couple of seconds has been shown to be counterproductive [11]. Furthermore, the ’innocent bystander effect’ also stems from this type of time-based correlation (see figure 1), where the simultaneous presence of malicious and benign antigen, renders them indistinguishable.

Danger Theory and Intrusion Detection

283

Antigen sampling - Linked to the correlation analogy presented above is the notion of the antigen sampling analogy. During a pathogenic infection within the human body, it is expected that a large number of the same type of pathogen get introduced within the body, with this number expected to increase when these pathogens replicate. In this scenario it is logical to hypothesize that during the antigen correlation process, only a sample of the antigen found within tissue fluids is considered, allowing for a lightweight correlation process. On the other hand, within a computer system scenario where each antigen is expected to be different, such as network packets, holding on to the antigen sampling analogy may not be effective due to absence of sufficient antigen [11]. In the danger theory-based AIS development framework libtissue, the antigen multiplier parameter allows for the creation of multiple copies of the same antigen, enabling this sampling process libtissue. However, the question here remains whether it is more effective to do away with the sampling process and simply process all the incoming antigen during the correlation process. Overall, this may prove to be more lightweight rather than utilizing the antigen multiplier work-around. Inefficient detection - Within a human organism, the trade off between efficient detection and novel pathogen detection seems to be a workable solution. In the case of a new type of infection, the human body falls ill for a time period until the HIS picks up the signs of distress and generates the appropriate antibodies. When this happens, the human body is then expected to recover. Within a computer system, it could be the case that by the time the first signs of distress are observed, it could already be too late to prevent irreversible damage from occurring. For example, let’s say that an attacker spawns a reverse spawning shell as a payload [9]. If this type of malicious payload is detected possibly before the attacker starts sending shell commands that would allow stealing sensitive data from the host, the data remains confidential. On the other hand, if sensitive information is immediately disclosed, by say an injected SQL command, then the possibility of recovery could be minimized, and confidentiality is compromised. In this case, it makes a difference whether say the breach of confidentiality concerns stolen credit card numbers that could be de-activated in a timely manner, or whether the stolen information relates to some secret business plan. Table 1 summarizes where the danger theory analogy holds, and where it would be best modified, in order to better reflect the IDS application domain requirements. The next section describes how modifying certain aspects of the danger theory analogy could create a new approach to IDS design. Table 1. Summary of analogy benefits and limitations

284

5

M. Vella, M. Roper, and S. Terzis

Modifying the Analogy - A Proposal for a Danger Theory-Inspired Approach to Intrusion Detection

Although several aspects of the danger theory analogy have shown not to be suitable for the purpose of intrusion detection design, there exits the possibility to modify these in a way to formulate an approach to intrusion detection. This approach would still benefit from those desirable characteristics of the human immune system. We call this a danger theory-inspired approached since various aspects of the analogy are modified in order to make it more suitable to the problem at hand. The potential for danger-theory inspired detection is being argued to be an important one. This potential is further explained in terms of comparison with anomaly detection (AD) and misuse detection (MD), the two current main approaches to intrusion detection. Danger signal characterization - Whilst some successful intrusions in a computer system may not generate obvious signs of distress, by breaking off from the analogy, alternate sources of danger signals could possibly be found. One example concerns the potential signals associated with the pre-exploitation phase of attacks. Several network/cyber attacks, require an initial profiling stage [7,26] in order to allow attackers to get hold of valuable information before launching the attack proper. Most of this profiling usually requires the direct poking of the system in order to force information elicitation, rendering it noisy [35]. Furthermore, another possibility could be the generation of danger signals when preventive measures, such as input sanitization, are evaded. One example could be the malicious employment of character encoding techniques [31] in order to cause a time-of-check-time-of-use (TOCTOU) type of attack, where the string inspected by the input filter is different from the one that actually gets processed by the application. Signs of TOCTOU attempts have been shown to be detectable within HTTP request packets [35] . Safe signal characterization - Within a danger theory-inspired IDS, rather than simply implying the non-presence of danger signals, an important role of the safe signal could be that of controlling overgeneralizing danger signals. Figure 2(a) illustrates the consequence of overgeneralizing danger signals erroneously causing the formation of a danger context. On the other hand, employing safe signals (S) to actively control this type of danger signals (D) overgeneralizing, could prevent this from happening as shown in figure 2(b). Here the computation of a danger context value (k) takes the general form of k = D − S, representing the various specific approaches taken in danger theory-based AIS’s so far [11,12]. For example, the danger signal associated with web application profiling activity [35], that of an increased HTTP request rate, a candidate safe signal in this case would be that of the increase in the number of source IP addresses. In case the HTTP request rate increases due to an increase in legitimate application usage, one would expect that the number of clients increases accordingly and thus the safe signals would prevent the erroneous danger context formation. Specifically, the safe signal allows for discriminating between attack and normal behavior that both cause an increase in some danger signal.

Danger Theory and Intrusion Detection

285

Fig. 2. (a) Overgeneralizing danger signal (top) (b) Overgeneralizing danger signal controlled by safe signal (bottom)

Correlation - Correlation within a computer system could be based on a forensics approach, where antigen is ranked according to the likelihood of having caused a particular type of danger context. This type of correlation could utilize regular expression-based pattern matching, taking advantage of the multiple types of danger signals that could be expressed within a computer system, as opposed to the single possible type of danger signal in the human body. In this case, a ’real-time’ form of digital forensics [6,29], could make the implementation of this type of correlation a real possibility. Intrusion backtracking [24], is another forensics technique that could be employed for conducting correlation. Intrusion backtracking consists of tracing backwards all the steps taken by an attacker, from detection point up till intrusion point. In the case of a danger theory-inspired detector, tracing would start at the point where a danger context forms. Comparing to anomaly detection - Figure 3 illustrates an abstract model of the anomaly detection approach. A sample of normal behavior is utilized to create a normal behavior profile, and a distance function utilized to flag as malicious all those monitored transactions that exceed a preset detection threshold. The detection effectiveness of any detector based on this approach is limited by the ranking accuracy achieved by the statistical/clustering model in terms of producing higher deviations for malicious system transactions as compared to normal behavior ones [17]. In reality, current anomaly models [25,21] produce rankings that are similar to the one shown in 3a, where the detection rate can be increased only at the expense of the false positives rate. This undermines de-

286

M. Vella, M. Roper, and S. Terzis

Fig. 3. (a) Anomaly detection approach - requires normal behavior profiling while incorrect ranking limits detection effectiveness (left) (b) Representative shape for ROC curves obtained by anomaly detection [21] (right)

tection effectiveness with a typical ROC curve shape as shown in figure 3b. This ROC curve shows the continuous undesired increase in the FP rate [3] when the detection threshold is moved in order to detect all malicious transactions. On the other hand, figure 4 shows that a danger theory approach to intrusion detection does not require a normal behavior profile and has the potential to avoid incorrect rankings. The reason for this being that observable attack results could provide a feature space that is much more appropriate for intrusion detection. Contrary to the anomaly models exclusively based on HTTP packet contents [25], this feature space allows for a distance function that enables a ranking that can clearly separate between http requests generated by normal and attack behavior. This feature space would be made possible once a robust signal schema and a suitable correlation algorithm are identified. In the case that these can be devised in a way to apply to the entire class of web applications, then the requirement for normal behavior profiling per individual application is eliminated. Comparing to misuse detection - In comparing the danger theory approach to the misuse detection approach, one could identify the signal selection schema and the misuse signature repository as similar features. In effect these have to be distinct if the danger theory detector is to be able to detect novel attacks. The signal selection schema should be one that focuses on the actual damage associated with attacker objectives, rather than the actual specific technique utilized during exploitations. Whilst the ways to intrude a system may be infinite, the ultimate attacker objectives are expected to be finite and small in number, possibly inline with the six threat categories as defined by the STRIDE taxonomy [35]. These ’generalized signatures’ go against the spirit of writing specific signatures in order to avoid false positives [5]. In the case of the signal selection schema, it could be the role of the safe signal, as previously described, to control the over-generalizing danger signals. Furthermore, a danger theory detector

Danger Theory and Intrusion Detection

287

Fig. 4. Danger theory approach - does not require profiling and potentially maximizes detection effectiveness

does not stop at signalling the presence of an intrusion, but is also responsible to correlate the monitored system transactions with the observed signals in order to flag those transaction responsible for the detected danger context.

6

Conclusions and Future Work

The analysis carried out in this paper proposes how the danger theory analogy could be of benefit to the development of intrusion detection systems capable of effective novel attack detection targeting web-based systems, with minimal administrative effort. Nonetheless, several aspects of the danger theory analogy seem not to be suitable for the purpose of IDS design. The open issues of a robust signal selection schema and suitable correlation algorithms formulation are reflections of these aspects. In order to formulate robust signal selection schemas, it is being proposed to look beyond the most obvious candidate danger signal sources, as well as assigning a very specific role to safe signals. Danger signals could be produced when target victim hosts are being profiled during pre-exploit stages, or when preventive measures are evaded. The set of safe signals should be identified with the specific role of controlling danger signal over-generalization. Whilst time-based correlation has been shown to pose various limitations, the environment of a computer system allows for correlation based on digital forensics techniques, such as regular expression-based pattern matching or intrusion backtracking. Danger theory-inspired detection has the potential of overcoming those limitations presently associated with misuse and anomaly detection. The importance of this potential, and given the fact that the proposed modifications to the danger theory analogy present us with the necessary ground work for realizing this potential, beckon for further research in this direction. Specifically, the development of a robust signal selection schema and a suitable correlation algorithm, would allow for the realization of a danger theory-inspired detector. These tasks present us with the next research work to be undertaken.

288

M. Vella, M. Roper, and S. Terzis

References 1. Aickelin, U., Bentley, P., Cayzer, P., Kim, J., McLeod, J.: Danger theory: The link between AIS and IDS? In: Timmis, J., Bentley, P.J., Hart, E. (eds.) ICARIS 2003. LNCS, vol. 2787, pp. 147–155. Springer, Heidelberg (2003) 2. Aickelin, U., Cayzer, P.: The danger theory and its application to artificial immune systems. In: Proceedings of ICARIS 2002. LNCS, Springer, Heidelberg (2002) 3. Axelsson, S.: The base-rate fallacy and the difficulty of intrusion detection. ACM Transactions on Information and System Security (2000) 4. Ayara, M., Timmis, J., de Lemos, R., Duncan, R.: Negative selection: How to generate detectors. In: Proceedings of 1st ICARIS (2002) 5. Baker, A.R., Esler, J.: Snort IDS and IPS Toolkit. Syngress (2007) 6. Cheswick, W., Bellovin, S., Rubin, A.: Firewalls and Internet Security: Repelling the Wiley Hacker, 2nd edn. Addison-Wesley, Reading (2003) 7. Clarke, J., Dhanjani, N.: Network Security Tools. O’Reilly, Sebastopol (2005) 8. D’haeseleer, P., Forrest, S., Helman, P.: An immunological approach to change detection: Algorithms, analysis, and implications. In: Proceedings of the 1996 IEEE Symposium on Security and Privacy (1996) 9. Erickson, J.: Hacking: The Art of Exploitation, 2nd edn. No Starch (2008) 10. Forrest, S., Perelson, A., Allen, L., Cherukuri, R.: Self-nonself discrimination. In: Proceedings of the 1994 IEEE Symposium on Security and Privacy (1994) 11. Greensmith, J., Aickelin, U.: The Dendritic Cell Algorithm. PhD thesis, University of Nottingham (2007) 12. Greensmith, J., Aickelin, U.: The deterministic dendritic cell algorithm. In: Bentley, P.J., Lee, D., Jung, S. (eds.) ICARIS 2008. LNCS, vol. 5132, pp. 291–302. Springer, Heidelberg (2008) 13. Greensmith, J., Aickelin, U., Cayzer, S.: Introducing dendritic cells as a novel immune-inspired algorithm for anomaly detection. In: Jacob, C., Pilat, M.L., Bentley, P.J., Timmis, J.I. (eds.) ICARIS 2005. LNCS, vol. 3627, pp. 153–167. Springer, Heidelberg (2005) 14. Greensmith, J., Aickelin, U., Twycross, J.: Articulation and clarification of the dendritic cell algorithm. In: Bersini, H., Carneiro, J. (eds.) ICARIS 2006. LNCS, vol. 4163, pp. 404–417. Springer, Heidelberg (2006) 15. Gu, F., Greensmith, J., Ackelin, U.: Further exploration of the dendritic cell algorithm:Antigen multiplier and time windows. In: Bentley, P.J., Lee, D., Jung, S. (eds.) ICARIS 2008. LNCS, vol. 5132, pp. 142–153. Springer, Heidelberg (2008) 16. Helman, P., Forrest, S., Esponda, F.: A formal framework for positive and negative detection schemes. IEEE Transaction on Systems, Man, and Cybernetic (2004) 17. Helman, P., Liepins, G.: Statistical foundations of audit trail analysis for the detection of computer misuse. IEEE Transactions on Software Engineering (1993) 18. Hofmeyr, S., Forrest, S.: Architecture for an artificial immune system. IEEE Transactions on Evolutionary Computation (2000) 19. Hofmeyr, S., Forrest, S., Somayaji, A.: Intrusion detection using sequences of system calls. Journal of Computer Security (1998) 20. Howard, M., Le Blanc, D., Viega, J.: 19 Deadly Sins of Software Security: Programming Flaws and How to Fix Them. McGraw-Hill/Osborne, New York (2005) 21. Ingham, K.L., Inoue, H.: Comparing anomaly detection techniques for http. In: Kruegel, C., Lippmann, R., Clark, A. (eds.) RAID 2007. LNCS, vol. 4637, pp. 42–62. Springer, Heidelberg (2007)

Danger Theory and Intrusion Detection

289

22. Kim, J., Bentley, P.: The human immune system and network intrusion detection. In: EUFIT 1999 Proceedings (1999) 23. Kim, J., Bentley, P.: An evaluation of negative selection in an artificial immune system for network intrusion detection. In: GECCO 2001 Proceedings (2001) 24. King, S.T., Chen, P.M.: Backtracking intrusions. In: Proceedings of the 19th ACM symposium on Operating Systems Principles, SOSP 2003 (2003) 25. Kruegel, C., Vigna, G., Robertson, W.: A multi-model approach to the detection of web-based attacks. Computer Networks 48(5) (2005) 26. Long, J., Bayles, A., Foster, J., Hurley, C., Petruzzi, M., Rathaus, N., Wolfgang, M.: Penetration Tester’s Open Source Toolkit. Syngress (2006) 27. Maggi, F., Robertson, W., Kruegel, C., Vigna, G.: Protecting a moving target: Addressing web application concept drift. In: Balzarotti, D. (ed.) RAID 2009. LNCS, vol. 5758, pp. 21–40. Springer, Heidelberg (2009) 28. Matzinger, P.: The danger model: A renewed sense of self. Science (2002) 29. Northcutt, S., Zeltser, L., Winters, S., Kent, K., Ritchey, R.: Inside Network Perimeter Security. Sams (2005) 30. Riden, J., McGeehan, R., Engert, B., Mueter, M.: Web application threats. Know Your Enemy (2008) 31. Scambray, J., Shema, M., Sima, C.: Hacking Exposed - Web Applications, 2nd edn. McGraw-Hill, New York (2006) 32. Somayaji, A., Hofmeyr, S., Forrest, S.: Principles of a computer immune system. In: Proceedings of the 1997 New Security Paradigms Workshop (1997) 33. Twycross, J., Aickelin, U.: libtissue - a software system for incorporating innate immunity into artificial immune systems (2006), http://www.cpib.ac.uk/~ jpt/papers/libtissue-tecv.pdf 34. Vella, M., Roper, M., Terzis, S.: Achieving anomaly detection effectiveness beyond the symmetric error lower bound, in web-based systems (2009), http://www.cis.strath.ac.uk/~ mv/trep1.pdf 35. Vella, M., Roper, M., Terzis, S.: Characterization of a danger context for detecting novel attacks targetig web-based systems (2010), http://www.cis.strath.ac.uk/~ mv/trep2.pdf 36. Wang, W., Guyet, T., Knapskog, S.J.: Autonomic intrusion detection system. In: Balzarotti, D. (ed.) RAID 2009. LNCS, vol. 5758, pp. 359–361. Springer, Heidelberg (2009)

Electronic Fraud Detection for Video-on-Demand System Using Hybrid Immunology-Inspired Algorithms Rentian Huang, Hissam Tawfik, and Atulya Nagar Faculty of Business and Computer Sciences, Liverpool Hope University, Liverpool, United Kingdom {Huangr,Tawfikh,Nagara}@hope.ac.uk

Abstract. This paper proposes an improved version of current electronic fraud detection system by using logging data sets for Video-on-Demand system. Our approach is focused on applying Artificial Immune System based fraud detection algorithm for logging data information and accounting and billing purposes. Our hybrid approach combines algorithms from innate and adaptive parts of immune system, inspired by the Self non-self theory and the Danger theory. Our research proved the possibility of combining these to perform Efraud detection. The experimental results demonstrated that hybrid approach has higher detection rate, lower false alarm when compared with the performances achieved by traditional classification algorithms such as Decision Tree, Support Vector Machines, and Radial Basis Function Neural Networks. Our approach also outperforms AIS approaches that use Dendritic Cell Algorithm, Conserved Self Pattern Recognition Algorithm, and Clonal Selection Algorithm individually. Keywords: Artificial Immune System; Fraud Detection; Video-on-Demand.

1 Introduction There is money, there is fraud. This has been true for decades. As E-Commerce has become a dynamic force, changing all kinds of business operations world-wide, new forms of fraud based on Internet has been invented. In comparison with traditional fraud, electronic fraud (e-fraud) poses more challenges in terms of prevention and detection. Consequently, fraud detection remains an interesting research issue. Limitations in exchanging ideas and data sets about fraud undisclosed to the public hamper e-fraud research and make it more difficult to develop new fraud detection methods. At present, there are a number of different computational methods applied for fraud detection such as expert systems [1], neural networks [2], pattern recognition [3], and decision tree [4] etc. These algorithms focus only on very specific types of their applications, and do not try to implement an extensible approach to the prevention of different kinds of online fraud. Hence, more efficient techniques for efraud detection need to be developed. This paper proposes an improved approach of current online fraud detection system. This approach is focused on logging data since many systems have logging data for accounting purposes. Artificial immune systems are one of the most rapidly E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 290–303, 2010. © Springer-Verlag Berlin Heidelberg 2010

Electronic Fraud Detection for Video-on-Demand System

291

emerging biologically motivated computing paradigms and proved very powerful in addressing computational problems. In our paper, a novel hybrid artificial immune inspired approach is introduced for detecting online fraud in the Video-on-Demand system. The approach combines the Dendritic Cell Algorithm (DCA), Conserved Self Pattern Recognition Algorithm (CSPRA), and Clonal Selection Algorithm (CSA). Due to the different characteristics of each method, our empirical results of our hybrid approach demonstrate that detection rate can be improved, and the false alarm can be reduced much more than when using the three algorithms separately.

2 Proposed AIS Based Video-on-Demand Fraud Detection System The current risk management pipeline system for online fraud management is well established which consists of Automated Screening, Manual Review, and Accept/Reject components in Fig. 1, which mainly depends on validation service and purchase device tracing. Some merchants also apply decision and rules systems to determine whether the transaction should be accepted, rejected or suspended for manual review [5]. Our approach focuses on improving the current risk management pipeline by adding AIS based e-fraud detection on logging data since many systems have logging for accounting and billing purposes. Those logging sources can then provide signs of suspicious user action, give information that clarifies the nature of the user actions, the identity or location of the user, or any other useful information about the fraud.

Fig. 1. Proposed Fraud Detection System

Since the risk management pipeline system is a well established fraud management tool, to develop a system to use AIS based fraud detection techniques to log and event data is the main task of the paper (Fig. 2).

292

R. Huang, H. Tawfik, and A. Nagar

Fig. 2. AIS based fraud detection system

2.1 Target System Our target system is a Video-on-Demand system. This system consists of a number of components, shown in Fig. 3. Each user has a set-top-box at home, which can be connected to the Internet using a fast xDSL connection. When the set-top-box was turned on, the user will be allowed to contact the Dynamic Host Configuration Protocol server; eventually a dynamic IP address will be allocated to the user. Then, the user can login to the application server, browse the video database and order a movie. When the user provides correct identity and billing information to the server, the application server will generate an authentication ticket for the user. The Video-on-Demand server then starts delivering the chosen movie after verifying of the ticket. The details of data generation can be found in [6].

Fig. 3. Video on Demand System’s Structure

Fig. 4. (a) Data record file format, (b) Router daily logs file format

2.2 Log, Event Management After data collection from our Target System, two more useful and summarized information are extracted from the raw logs produced by the Video-on-Demand system. As shown in Fig. 4(a), data record file will start with Event Index 0 and 1 will indicate the set-top-box states (0 is off, 1 is on), Index larger than 1 is the event activities during this session which can be successful login or not, successful order or

Electronic Fraud Detection for Video-on-Demand System

293

not, billing and delivery. Further information also includes event date, time, and IP address from the user. The second file (Fig. 4(b)) called router daily logs, which contains one line per day for the information of the date, time, user IP address, uploaded bytes and downloaded bytes. By analysis normal and fraud cases, summarized information from raw log, event data, and the sum of events of the user (over 24 hours) will be the attributes for fraud detection as follows: 1. 2. 3. 4. 5. 6. 7.

Sum of failed login attempts Sum of successful login attempts Sum of failed movie order Sum of successful movie orders Sum of movie delivery notifications Sum of billing notifications Ratio between uploaded & downloaded (bytes)

Since the user’s behaviour is changing over time, an exponential trace memory to maintain a moving average of pass input: xi (t ) = (1 − μ i )xi (t ) + μ i xi (t − 1) is

introduced. The configurable μ allows for the representation of averages spanning various intervals of time. We empirically defined μ =0.7 in our detection tests, which proved to provide a sufficient decay rate for our purposes. 2.3 AIS Based Detection Engines

The AIS detection engines implements AIS based algorithms that can classify input data as normal or fraud user. In our case study, the AIS detection engines can be logically separated as 3 layers. • Layer 1 The Layer 1 implements the Dendritic Cells Algorithm. The idea of DCA is to correlate disparate data-streams in the form of antigen and signals and label groups of identical antigens as normal or anomalous, and no training is required [7]. The selected attributes will be categorized into three different signals including Pathogen Associated Molecular Patterns (PAMP), Danger, and Safe signal, and DCA will be able to classify the current data into Low, Medium, and High level of danger. These three signals for each signal is a separate function performed on the inputs to produce the appropriate output cytokines: costimulatory molecules (CSM), mature (M) and semi-mature (SM). The equation (1) which determines a final decision by assigning a different weight to each signal is borrowed from [7]. O[csm, m , sm ] =

(WP ∗ CP ) + (WS ∗ CS ) + (WD ∗ CD )

(1)

WP + WS + WD

Where WP , WS , W D , C P , C S , C D are weights and concentrations of PAMP, Danger, Safe signals respectively. When the output of CSM exceeds its own threshold, immature DC will move to maturation level. The overall context is termed as safe if the output of SM is greater than M, and vice-versa.

294

R. Huang, H. Tawfik, and A. Nagar

Each antigen will be sampled multiple times in order to appear in different contexts. A mature context antigen value (MCAV) is calculated for each antigen by dividing the number of times that antigen has appeared in the danger context by total number of its appearances. Finally, a threshold is applied to MCAV to make the final decision. Algorithm (1) is the pseudo code for DCA. 1: Samples Stage FOR Cycle=1:Max_DC_Cycle FOREACH DC in population Sample Antigens from Pool & store it Calculate Signals for each sampled antigen & store it Calculate output CSM, SM, M using Equation (1) Add CSM, SM, M to total CSM, SM, M respectively IF total CSM > Threshold IF total SM > M, Add DC to SM population ELSE, Add DC to M population ENDFOR ENDFOR 2: Analysis Stage FOREACH Antigen in SM & M population Calculate No. of times appear in SM and M population IF SM/(SM+M) > MCAV(High) Antigen is Fraud User IF SM/(SM+M)> MCAV(Medium) Antigen is Active User ELSE Antigen is None Active User ENDFOR Algorithm 1. Pseudo code for DCA •

Layer 2 The main component of Layer 2 is an improved version of the negative selection called Conserved Self Pattern Recognition Algorithm (CSPRA). CSPRA will select parts of data from normal users to generate detectors in order to distinguish the current data as self or non-self data with the help of the APC detector [8]. In training stage, it learns about normal behaviour in the system. At the end of the stage, the system will be able to select the conserved pattern and generate an APC detector. Also the system runs the negative selection process and creates its detectors. For detection stage, detectors generated by negative selection and APC detector will work together to check if newly collected antigens represent the behavior of good or bad users. The Euclidian distance rule is used for affinity measurement. Algorithm (2) is the pseudo code for CSPRA.

Electronic Fraud Detection for Video-on-Demand System

295

1: Training Stage Generate T detectors using negative selection & APC detector T1: threshold for self data T2: threshold for T detectors T3: threshold for APC detector T4: threshold for suspicious antigen 2: Detection Stage FOREACH Antigen COMPUTE dist(T) with T detectors, dist(S) with Self data IF dist(T) < T2 & dist(S)>T1 IF dist(T) > T4 COMPUTE dist(C) with APC detector IF dist(C) >T3, Antigen is non-self data ELSE Antigen is undetected data ELSE Antigen is non-self data IF dist(T)>T2 & dist(S)
Layer 3 The Layer 3 will be triggered when the decision cannot be made by Layer 1 and 2. It implements Clonal Selection Algorithm (CSA) for classification [9]. In training stage, layer 3 will select one part of training samples to initialize the immune memory. Another part of samples will pass to a dynamic training algorithm. The immune memory cells training and testing go hand in hand to obtain a better memory cell for classification. The Euclidian distance rule is used for affinity measurement. Classification is implemented by a fuzzy k-nearest neighbor (KNN) algorithm [10]. Data was passed through the memory cells as the fuzzy KNN selected k closest memory cells from the immune memory. The average distance between test sample and k closest memory cells will compare with defined threshold in order to classify the test sample as normal user or fraud user. Algorithm (3) is the pseudo code. 1: Training Stage WHILE No. <= size of antigens Selected an antigen & start to train Classify the antigen using the current updated memory cells IF classification strategy recognized antigen, start with next antigen ELSE generate antibodies randomly and calculate the affinity Select n antibodies having the highest affinity and clone them Apply hyper-mutation to the clone set C i to produce mutated offspring C i* Re-calculate affinity between antigen and C i* . Select & update memory cells Stop training if the required number of matured antibodies is generated ENDWHILE 2. Detection Stage

296

R. Huang, H. Tawfik, and A. Nagar

Train the Fuzzy KNN with memory cells generated by CSA FOREACH test antigen Determine the k closest training samples based on the calculated affinity Select the most frequent class label among the k closest training samples. Classify the test antigen according to the most frequent class label ENDFOR Algorithm 3. Pseudo code for CSA classification

3 Hybrid Algorithm The hybrid algorithm is for combining multiple outputs of different models to achieve more reliable decisions and increase predictive performance over single model. The following Fig. 5 shows the implementation of the hybrid algorithm.

Fig. 5. Hybrid approach based on AIS algorithms

The newly collected data of the users will be first passed to the Layer 1 and classify the user as none active user (Low level of danger), active user (Medium level of danger), and fraud user (High level of danger) based on DCA. In Layer 2, a pool of qualified detector candidates is generated for CSPRA. The DCA is further deployed to acquire the optimal detectors with high level of danger are used for CSPRA. In the classification, Layer 2 will classify the new sample as self, non-self, or undetected data based on CSPRA. By combining the results of Layer 1 and 2, the systems will have the ability to classify the user into different categories as shown in Table 1. There will be 9 different combination of output provide by Layer 1 and 2. The rules based decision will determine whether the transaction of the user should be accepted, rejected or suspended for further analysis. Table 1. Decision rules for Layer 1 & 2 Layer 1 Low Medium High Low Medium High Low Medium High

Layer 2 Self Self Self Non-self Non-self Non-self Undetected Undetected Undetected

Decision Accept Accept Suspend Suspend Reject Reject Suspend Suspend Reject

Table 2. Decision rules for suspended cases Suspended Cases Self/High Self/High Non-self/Low Non-self/Low Undetected/Low Undetected/Low Undetected/Medium Undetected/Medium

Layer 3 Normal Fraud Normal Fraud Normal Fraud Normal Fraud

Decision Accept Reject Accept Reject Accept Reject Accept Reject

Electronic Fraud Detection for Video-on-Demand System

297

The Layer 3 is used under the situations when Layer 1 and 2 cannot make a decision such as when the new sample is classified as self data with high level of danger, non-self data with low level of danger, undetected data with low level of danger, or undetected data with medium of danger. In these suspended cases, the final decision then will be made following the rule in Table 2. The suspended cases will be accepted by the system if Layer 3 classifies it as normal user. The suspended cases will be rejected the system if the Layer 3 classifies them as fraud user.

4 Experimental Setup In Video-on-Demand system, 3 fraud cases were simulated by a team of fraudsters, and from their actions, synthetic data containing fraudulent users were generated and used for training and testing. The simulation resulted in synthetic data for 7 months containing 600 normal users, 100 break-in fraudsters, 100 billing fraudster, and 100 illegal redistribution fraudster. After data pre-processing, 3 datasets are generated for each fraud case, each dataset has more than 100,000 examples. The break-in frauds data which mimic a user who has taken over the identity of a legal user by hacking the user’s set-top-box will be used for our experiments. In this section, a data set with 45 users (one third none active user, one third active user, and one third fraud user), each user has 20 samples which include normal and fraud samples that follow the distribution of 2:1 will be used to choose the best parameters for 3 different layers. The factors used to define the parameters for algorithms are the detection rate (DR) and false alarm rate (FA), which can be calculated from true positive (TP), false positive (FP) true negative (TN), and false negative (FN) in equation (2) and (3). DR=TP/(TP+FN)

(2)

FA=FP/(FP+TN)

(3)

4.1 DCA Experimental Setup (Layer 1) DCA did not require training, but has a large number of parameters and stochastic elements. All the training data set is used for setting the parameters. A summary of the experiments performed for parameters are list in Table 3. Unless stated, all other parameters are shown in Table 4. And the weights for signal processing defined in Table 5 were derived from empirical biological data [7]. Table 3. Experiment codes and settings Parameters Number of Antigen Receptor Number of DCs Migration Threshold Range Number of Antigen Multiplier

Parameter Values 1; 2; 3 100; 200; 300; 400; 500; 700 0.3-0.8; 0.4-1.0; 0.6-1.1; 0.8-1.5; 1.1-1.6 5; 50; 70; 100; 120

298

R. Huang, H. Tawfik, and A. Nagar

Table 4. Default parameter setting for DCA Parameters

Values

Number of Antigen Receptor Number of DCs Migration Threshold Range

1 100 0.6-1.1

Number of Antigen Multiplier

35

Table 5. Default weights for function (1) Weight WP WD WS

CSM 2 1 1

SM

M

0 0 1

2 1 0

Table 6. Signal Mapping for DCA Signals PAMP

Definition of the listed signals A PAMP signal is a strong indicator of a pathogenic presence, a strong indication of break-in could be the number of the movies they try to order. The order failures may result from many factors, but definitely the fraudster will fail more time than the authentic user when try to order movies.

Danger

A Danger Signal may or may not indicate an anomalous; however the probability of an anomaly is higher than normal. The number of successful order is been considered as danger signal. In normal circumstances, successful movie order should not excess a certain number, as the break-in fraudster will try to maximize their chance to order the movie successfully, the number of successful order for fraudster will bigger than normal user’s in most of the time.

Safe

A Safe Signal almost certainly indicates that no anomalies are present. The first two are associated with login, number of time fail to login and number of successful login. The third one is the number of billing notification. The number of successful billing notification will be smaller than normal user in normal circumstances.

Data will be normalized within a range of 0 to 1, based on maximum values derived in preliminary experiments and the signal mapping is shown in Table 6. Experiment results on the number of cells showed that once the cell population drops below 300 and above 500, differences are shown in the detection rate and false alarm rate. In our case, number of DCs should be defined between 300 and 500. Each antigen is copied multiple times in the system. The classification decision is average value over the replicated population. According to the experimental results, the antigen multiplier did not enhance the performance of the DCA consequentially. In our case study, multiplier between 50 and 90 gave better accuracy. The median value of migration threshold range is assigned equates to DC sampling for 2 iterations when the signal strengths are half of the expected total input signal maximum according to equation (4). The range of migration threshold slightly below the median value gave a better performance than the one above. The best migration threshold in our experiment is between 0.4 and 1.0.

m _ value = 0.5 ∗ ((max p ∗ wp , c ) + (maxd ∗ wd ,c ) + (max s ∗ ws ,c )) = 1.0844

(4)

The number of DC antigen receptors means that the number of antigens of each DC can be sample per iteration. The default value of antigen receptor which is 1 exhibited better result, as the increase in the number of receptors will decrease detection rate.

Electronic Fraud Detection for Video-on-Demand System

299

The best scenario for the parameters setting is shown in Table 7 which exhibit excellent performance. As a result, we have determined threshold values (MCAV) of 0.43 for active user and 0.68 for fraud user. The moving average of pass input in section 2.2 gave the similar effect of moving time-windows for DCA. Table 7. Best parameters in DCA for break-in fraud Parameters Number of Signals categories Number of Cells Number of DC antigen receptors Max Cycle Antigen Multiplier Migration Threshold Range

Value 3 300 1 100 65 0.5-1.2

4.2 CSPRA Experimental Setup (Layer 2)

A range of experiments of the CSPRA are decided carefully in order to evaluate the performance of the algorithm and analyze the experimental results. Firstly, conserved self pattern needs to be selected correctly. The Pearson Product Moment Correlation Coefficient developed is the most widely measurement for correlation [11]. The computational formula is as below in equation (5): r=

N ∑ XY − (∑ X )(∑ Y )

N ∑ X 2 − (∑ Y )

2

N ∑ Y 2 − (∑ Y )

(5)

2

Table 8. Statistic analysis for attributes Input 1 2 3 4 5 6 7

Attribute Login Fail Login Order Fail Order Billing Delivery Upload/Download

Min 0 0 0 0 0 0 0

Max 11 12 18 33 15 2 12

Mean 0.8639 0.9508 1.0104 1.3238 0.8721 0.4336 7.0090

Correlation -0.0758 -0.0314 +0.7115 +0.2554 +0.0725 +0.1620 -0.0010

In Table 8, input 3 and 4 with higher correlation are selected as the conserved self patterns. The antigen which is being detected as a suspicious antigen by negative selection will be costimulated by the APC detector. The distance between suspicious antigen p and APC detector d is calculated by equation (6): w

Dist ( p, d ) = ∑ i =1

pi − di mi − ni

(6)

300

R. Huang, H. Tawfik, and A. Nagar

Where w is the number of dimensions for the conserved pattern; mi and ni represent the lower and upper bounds of the i attribute; p i is the value of i attribute to be examined; d i is the mean of all the values in the i column. After selected the conserved pattern, data will be normalized within a range of 0 to 1, based on maximum values derived in preliminary experiments. The effect of variation number of the T detector, data dimensionality, threshold of self data, T detector, and APC detector are been examined. Unless stated, all other parameters are kept as default values show in Table 9. CSPRA only require self data for training, 70% of the normal data will be used for training, the remaining normal data and fraud data will be used for testing. Table 9. Experiment codes and settings for CSPRA Parameters Dimensionality of data space Number of Detector Self detector threshold APC detector threshold T-detector threshold Suspicious antigen threshold

Parameter Values 2; 3;4;5 250; 500;750; 1000 0.04 to 0.5 0.1 to 1.5 0.2 to 2 0.1 to 1

Default Value 4 500 0.4 0.8 0.6 0.4

Experimental results showed that, when the space dimensions increase, the number of samples not detected increased as well, but the false negative numbers also decreased. By balancing the dimensions and the detection rates, the input from 1, 2, 5, and 6 in Table 8 gave the better results. The size of the T detector has only a small impact on the detection rate and the false alarm rate when the number is greater than 500. Having more detectors did not guarantee a better detection rate. In our case, the T detector size and the training data set size should be similar. Results also showed that the self data, T detector and APC detector thresholds have a significant effect on the performance of the system. For the T detector threshold, the detection rate and the false alarm rate increased when the threshold increases. When the APC detector threshold is different, the detection rate and the false alarm rate decrease when the threshold increases. Similar for threshold of self data, false alarm rate increased when the threshold inverses. The experimental results show that the parameters in the Table 10 gave the best performance. Table 10. Best parameters setting in the experiments Parameters Self detector threshold T-detector threshold APC detector threshold T-detector size Threshold for suspicious antigen Dimensionality of data space

Value 0.3 0.8 1 700 0.5 4

Electronic Fraud Detection for Video-on-Demand System

301

4.3 CSA Experimental Setup (Layer 3)

For the Layer 3, 7 inputs mentioned in 2.2 are normalized within a range of 0 to 1, based on maximum values derived in preliminary experiments. In the training, the 25% samples (including normal and fraud) of each user are used to generate the initial memory cells. Each antigen produced 5 antibodies before generating them from the remaining 75% data. These remaining antigens also generated 5 antibodies each. The Euclidean distance threshold is 0.2. The initial population for antibodies was 30, 15% of population will go through cloning and mutation, and the mutation probability was 0.05. All parameters were determined by experimentation. Classification results are also depended on the classification strategy. The performance of K=7 giving a better performance when compared with K= 3, 5, 9. In immunology, cloning selects a number of antibodies with the highest affinity and cloning them based on their antigenic affinities. The higher the antigenic affinity is, the higher the number of clones that will be generated. The total number of clones generated N c is defined in equation (7) as follow: n ⎛β ⋅N⎞ N c = ∑ round⎜ ⎟ ⎝ i ⎠ i =1

(7)

where β is a multiplying factor, N is the total number of antibodies, round (.) is the operator that rounds its argument toward the closet integer. In clonal mutation, the clone set C i is used to produce mutated offspring C i* . The higher the affinity an antigen has, the smaller the mutation rate. The algorithm for mutation is described in Algorithm (4). The equation (8) is defined as follows: ⎛ t ⎞ ⎛ ⎜ 1− ⎟ Δ(t , y ) = y⎜1 − r ⎝ T ⎠ ⎜ ⎝

λ

⎞ ⎟ ⎟ ⎠

(8)

where t is the iteration number, T is the maximum iteration number, r is a random value in the range [0, 1], and λ is used to decide the nonconforming degree. Find the Max and Min in C i . FOREACH antibody (Ab), do Generate a random value in the range [0, 1], named mr Generate another random value in the range [0, 1], named t0 IF mr < mutation_rate IF t0 >=0, Ab =Ab + Δ (t, Max-Ab) ELSE Ab =Ab- Δ (t, Ab-Min) ENDFOR Algorithm 4. Mutation rate control algorithm

5 Results and Comparison In this section, the performance of our hybrid algorithmic approach for break-in fraud detection will be compared with 3 other classification methodologies including Radial

302

R. Huang, H. Tawfik, and A. Nagar

Basis Function (RBF) neural networks, Support Vector Machines (SVMs) and Decision Tree (C4.5). Full details of implementation and parameters setting of different algorithms can be found in [12]. The 45 users which produced 900 samples in section 3 will be used in training for SVMs, C4.5, and RBF algorithms and the best parameters setting for 3 different layers can be borrowed for the hybrid approach. Our testing experiment is based on randomly selecting 2000 samples that follow a distribution of 50% normal data and 50% fraud data. The first 1000 samples forms will be the testing data set 1, and the remaining 1000 forms the testing data set 2. Table 11 has concluded different classification methods results for the training, and the average score of 2 testing data sets in 20 runs. The N/A is intended to refer to cases where the algorithms do not need a training phase such as DCA or where the data it uses for training does not allow for a ‘like-for-like’ comparison such as CSPRA and SVMs; as such we use the N/A (not applicable) notation. Table 11. Training and testing results for different approaches Methods RBF SVMs C4.5 CSPRA(8% undetected) DCA CSA Hybrid Algorithm

TP% Train Test 70 55.4 99 66 95.2 70.2 N/A 79 N/A 78 N/A 82 N/A 86.7

TN% Train Test 98.6 89.2 99 67.6 98.8 89.2 N/A 85 N/A 85 N/A 90 N/A 93.5

FP% Train Test 1.4 10.8 1 32.4 1.2 10.8 N/A 12 N/A 15 N/A 10 N/A 6.5

FN% Train Test 30 44.6 1 34 4.8 29.8 N/A 16 N/A 22 N/A 18 N/A 13.3

In the training section (Table 11), SVMs and C4.5 achieved more than 95 % TP and TN; this left only less than 5% for the FP and FN. But in the testing section, the TP and TN for C4.5 and SVMs only produced an average of approximately 68%. Accuracy was reduced from 95% to 68%. Especially for SVMs, false alarm rate increased from training 1% to 32.4%. The results of RBF neural network also produced ordinary results. In the testing section, the results achieved by 3 individual layers already exhibited better performance, with more than 10% higher in TP in testing compared with RBF neural network, SVMs, and C4.5. These results show the potential usage of DCA, CSPRA, and CSA are 3 efficient and reliable techniques to classify normal and fraud users in an online environment. Our hybrid algorithm demonstrates improvements over the DCA, CSPRA, and CSA. It shows a significantly better positive predictive value of fraud detection than what is achieved by three single approaches when applied on the same data. The results calculated by equation (2) and (3) show that the hybrid approach has a higher detection rate at 86.7% with far lower false alarm rate at 6.5%.

Electronic Fraud Detection for Video-on-Demand System

303

6 Conclusions This paper proposes an improved approach of E-fraud detection by using hybrid the AIS techniques of CSA, DCA, and CSPRA. Our hybrid algorithm successfully applied to Video-on-Demand System based on logging data for E-fraud detection. The paper has investigated the potential usage of the CSA, CSPRA and DCA for classifying normal and fraud users in an online environment which provided better results than traditional classification algorithms such as Support Vector Machines, RBF Neural Network, and Decision Tree (C4.5). Our hybrid model for online fraud detection, which combines three artificial immune system algorithms, demonstrate more favorable results in terms of higher detection rate, lower false alarm and handle high dimensional data set better when compared with the results achieved by only using CSPRA, DCA, and CSA as individual algorithms.

References 1. Rozsnyai, S., Schiefer, J., Schatten, A.: Solutionarchitecture for detecting and preventing fraud in real time. In: Proceeding of Digital Informatino Management, ICDIM 2007, pp. 152–158 (2007) 2. Guo, T., Li, G.-Y.: Neural data mining for credit card fraud detection. In: Proceeding of International Conference on Machine Learning and Cybernetics, pp. 3630–3634 (2008) 3. Xu, J., Sung, A.H., Liu, Q.: Tree Based Behaviour Monitoring for Adaptive Fraud Detection. In: Proceeding of International Conference on Pattern Recognition, pp. 1208– 1211 (2006) 4. Kirkos, E., Spathis, C., Manolopoulos, Y.: Data mining techniques for the detection of fraudulent financial statements. Expert Systems with Applications, 23–32 (2007) 5. CyberSource: Online Fraud Report, http://forms.cybersource.com/forms/ FraudReport2009NACYBSwww020309 6. Lundin, E., Kvarnström, H., Jonsson, E.: A synthetic fraud data generation methodology. In: Deng, R.H., Qing, S., Bao, F., Zhou, J. (eds.) ICICS 2002. LNCS, vol. 2513, pp. 265–277. Springer, Heidelberg (2002) 7. Greensmith, J., Twycross, J., Aickelin, U.: Dendritic cells for anomaly detection. In: Proc. of the Congress on Evolutionary Computation (CEC), pp. 664–671 (2006) 8. Yu, S., Dasgupta, D.: Conserved Self Pattern Recognition Algorithm. In: Bentley, P.J., Lee, D., Jung, S. (eds.) ICARIS 2008. LNCS, vol. 5132, pp. 279–290. Springer, Heidelberg (2008) 9. Huang, R., Tawfik, H., Nagar, A.: Licence Plate Character Recognition Using Artificial Immune Technique. In: Bubak, M., van Albada, G.D., Dongarra, J., Sloot, P.M.A. (eds.) ICCS 2008, Part I. LNCS, vol. 5101, pp. 823–832. Springer, Heidelberg (2008) 10. Keller, J.M., Gray, M.R., Givens Jr., J.A.: A Fuzzy K-Nearest Neighbor Algorithm. IEEE Transactions on Systems, Man, and Cybernetics 15(4), 580–585 (1985) 11. Moore, D.: Basic Practice of Statistics. W.H. Freeman, San Francisco (2006) 12. Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, Chichester (2001) ISBN 0-471 05669-3

Converging Bio-inspired Robotics and Socio-inspired Agents for Intelligent Transportation Systems Jeremy Pitt1 , Yiannis Demiris1 , and John Polak2 1 2

Department of Electrical & Electronic Engineering, Centre for Transport Studies, Imperial College London, Exhibition Road, London SW7 2BT, UK {j.pitt,y.demiris,j.polak}@imperial.ac.uk

Abstract. This position statement presents a brief overview of our research programme investigating the convergence of biologically-inspired robotics with sociologically inspired agents, and its potential application in multi-dimensional intelligent transportations systems.

1

Introduction

Despite significant technological developments that have led to substantial improvements in safety, comfort and economy, automobile operation remains hazardous, stressful, and sub-optimal. There is substantial research and development in intelligent transportation (see e.g., the Agents in Traffic and Transportation workshops, the IEEE Transactions on Intelligent Transportation Systems, the DARPA Grand Challenge on the driverless vehicle, and so on) which addresses this issue. In this research we are concerned with contributing and complementing that work by taking a multi-dimensional approach to increasing the ‘intelligence’ of vehicles and the transport infrastructure in which they operate. This intelligence uses traffic data from a multitude of sensors pervading the transport infrastructure, leading to the adaptation of individual behaviour, of local group coordination, and of macro-level metropolitan policy.

2

Bio-inspired Robotics and Socio-inspired Agents

We propose to investigate the manifestation of intelligence in the individual, collective and (application) infrastructure dimensions, to develop a micro-mesomacro constitution of integrated transportation. We are concerned with (i) intelligence ‘at the edge’, where each vehicle is packed with sensors and electronics for location, telemetry, and communications, integrated through a common service bus, and is capable of individual decision-making with respect to its own (drivers/passengers) goals; (ii) ‘social’ intelligence: through the execution of local computations with respect to local policies and the traffic environment, groups of vehicles can form a transient association that adaptively and locally E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 304–306, 2010. c Springer-Verlag Berlin Heidelberg 2010 

Converging Bio-inspired Robotics and Socio-inspired Agents

305

controls junction traffic, overtaking manoeuvres, lane/speed/distance allocation in motorway ‘shock waves’, etc., for ‘the common good’; and (iii) intelligent infrastructure: the transport infrastructure is itself saturated with sensors, and requires an overlay of intelligence to provide a ‘big picture’ of coordinated control, policy management, and interaction with other infrastructure (e.g. other transport systems, energy, business, etc.). Edge intelligence is concerned with individual behaviour, and thus at the micro-level; social intelligence is concerned with collective behaviour, or behaviour in temporary clusters, and so at the meso-level, while intelligent infrastructure is concerned with the system as a whole, and therefore operates at the macro level. The three levels are inter-dependent and working out the influence of micro- and meso-level behaviour on macro-level policy, and vice versa, is essential to understanding and developing the structures and strategies that will lead to improved efficiency and increased safety. To develop this multi-dimensional intelligent transportation system, we propose to integrate three directions of research. First, we build on sociologicallyinspired computational frameworks for intelligent decision support, including (i) a socio-cognitive/socio-economic framework which takes into account direct experience, indirect signals, economic utility and confidence to compute an anytime trust decision, which distinguishes between first encounter risk trust and nth encounter reliance trust [NP04]; (ii) an opinion formation framework for decentralised systems where knowledge may be partial and imperfect, allowing local computation of a mindset based on influence and affinity, and expression of a preference which can be mapped into a collective choice or decision [RP06]; and (iii) an organizational framework, which identifies roles, social constraints, social structures, ontologies and communication language, which can be used to represent and reason with actions to determine legal/valid behaviour [PKSA06]. Secondly, we use a biologically-inspired models of robotic behaviour, including: (i) a decision making architecture, HAMMER [DK06] which uses distributed competition mechanisms among multiple hypotheses to generate plans or recognize plans and intentions of other agents. This architecture has been inspired by neurophysiological data on the ‘mirror neuron system’ of humans and other primates; (ii) algorithms that adapt templates which conform to safety and legal requirements to new contexts when planning trajectories and paths for multiple vehicles; and (iii) lifelong assistive robots, which use the HAMMER-based intention recognition algorithms to help humans in difficult tasks; examples include the assisted operation of robotic wheelchairs, and novel methods for programming devices through human task demonstration [Dem07]. Thirdly, we seek to integrate aspects of stream-, cloud- and pervasive computing for traffic intelligence [Netal09], where stream computing is applied to the many data streams generating traffic data, including telemetry from the cars, roadside detectors (e.g. inductive loop detectors and traffic cameras); cloud computing provides services for anomaly detection, feature extraction, pattern detection, real-time event recognition, etc.; and pervasive computing, which concerns the disaggregation of traffic intelligence to personalized services.

306

3

J. Pitt, Y. Demiris, and J. Polak

Future Directions

We speculate that the micro-meso-macro constitution of intelligent transportation can be the basis of a form of ‘systemic’ intelligence of use to city planners and transport authorities. A transportation system has macro-level function (e.g., minimal journey time with minimal accidents and minimal CO2 emissions). A city planner wants to control this function, but it is not predictable from past behaviour, and it not for certain what effect specific policies will have on the form of this function. However, a transport system has a user population, who form social networks with discrete dynamics and different experiences, beliefs, values, attitudes and cultural norms. These social networks will disseminate information (and attitudes) in certain ways according to the behaviour of the system as provided by the edge, social and infrastructure intelligence. Different policies have different impacts on a micro-level. These contribute to meso-level functions with semi-continuous dynamics, for example population distribution, choice of transportation, traffic density and so on. Our ultimate goal is to determine if the trajectory of the macro-level function can be predicted from the aggregation of meso-level functions, themselves determined from the aggregation of different micro-level social networks (the semantics of which are determined by the (subjective) micro-meso-macro constitution of intelligent transportation). The output of this function is ‘intelligence’ available to the infrastructure, so we also want to determine if the policy stabilises the system. Acknowledgements. This work has been supported by IBM Faculty Award ‘Autognomy’, with thanks to Andreas Tsotsias, Rashik Parmar, and Ian Nussey.

References [Dem07] [DK06]

[NP04]

[PKSA06] [RP06]

[Netal09]

Demiris, Y.: Prediction of intent in robotics and multi-agent systems. Cognitive Processing 8, 151–158 (2007) Demiris, Y., Khadhouri, B.: Hierarchical Attentive Multiple Models for Execution and Recognition (HAMMER). Robotics and Autonomous Systems 54, 361–369 (2006) Neville, B., Pitt, J.: A computational framework for social agents in agent mediated e-commerce. In: Falcone, R. (ed.) AAMAS 2004 Trust Workshop, pp. 83–91 (2004) Pitt, J., Kamara, L., Sergot, M., Artikis, A.: Voting in multi-agent systems. Computer Journal 49(2), 156–170 (2006) Ramirez-Cano, D., Pitt, J.: Follow the Leader: Profiling Agents in an Opinion Formation Model of Dynamic Confidence and Individual MindSets. In: Proceedings IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT), pp. 660–667. IEEE Computer Society, Los Alamitos (2006) North, R., Cohen, J., Wilkins, S., Richards, M., Hoose, N., Polak, J., Bell, M., Blythe, P., Sharif, B., Neasham, A., Visalakshmi, S., Galatioto, F., Hill, G., Mead, I., Jones, R., Beresford, A., Chen, H., Ropkins, K., Goodman, P.: Field deployments of the MESSAGE system for environmental monitoring. Traffic Engineering and Control 50(11), 484–488 (2009)

On Homeostasis in Collective Robotic Systems Jon Timmis1,2 and Andy Tyrrell1 1

2

1

Department of Electronics, University of York, UK {jt517,amt}@ohm.york.ac.uk Department of Computer Science, University of York, UK

Introduction and Position

The biological term Homeostasis refers to an organisms ability to maintain steady states of operation in a massively changing internal and external environment. Pervasive Adaptive (PerAda) systems operate in potentially hostile environments where there is a need to cope with a wide variety of changes, over multiple timescales, deal with interacting agents, cope with degraded data and cope with failing components. In the context of this paper, we consider the case of a swarm, and organism, of robotic units, as those developed as part of the SYMBRION project1 . Such a collective robotic system can be considered a case of a PerAda system. Our position is that for such systems to operate for extended periods of time, without human intervention, they should be homeostatic and that it is possible to develop such homeostatic PerAda systems inspired by the immune system.

2

Rationale for Argument

A popular held purpose for the immune system is defence against pathogens, requiring the discrimination between self and non-self. In physiological terms, the output of the immune system is simply inflammation. The effect of inflammation is to perform maintenance on the body keeping it fit for living, not the discrimination of self from non-self antigen. It has been argued before, by people such as Cohen [1] that the result of inflammation, and hence the role of the immune system, is to repair and maintain the body. In order to achieve body maintenance, the immune system must select and regulate the inflammatory response according to the current condition of the body. Work in [2] outlines the concept of immune inspired homeostasis and propose three desirable properties for homeostatic control systems: Prediction: The homeostatic system should respond to an environmental change by manipulating the internal condition, or environment to avoid a deviation from a set point before it has happened; Innate and Adaptive Response: The homeostatic system should consist of innate and adaptive responses which are used to bring homeostatic variables back to set points. Innate responses are involuntary, unpremeditated and unlearned, and are instigated in response to a particular stimulus, internal or external. 1

http://www.symbrion.eu

E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 307–309, 2010. Springer-Verlag Berlin Heidelberg 2010

308

J. Timmis and A. Tyrrell

Adaptive responses are learned to correct unforeseen deviations from set points and Acclimatisation: A system should be able to semi-permanently change a response to semi-permanent change in the environment. Collective robotic systems in SYMBRION are being developed with deployment for extended periods of time, up to 100 days, in a swarm of 100 robots that can join together to form robotic organisms, capable of undertaking tasks that a single, or small number, of units can not do [3]. In order to achieve this longevity, fault tolerance is a key factor to success. In such a scenario, prediction is important, as identifying a failing robot before it joins a robotic organism will potentially prevent that failing unit having a detrimental effect on the performance of the organism: thus in someways helping to maintain a level of homeostasis at the organism level. Both innate and adaptive responses are needed, as the robots can easily be pre-programmed with specific error detection abilities. However, adaptation is needed, as over time components will fail in a way that may never have been considered before, given the environmental and operational changes that will be experienced. Finally, robotic units will be required to acclimatise not only to the environment, but to other robots, not only in the swarm, but also in the organism. What was normal operation in swarm mode may be very different when in organism mode.

3

One Possible Counter Argument

Could you not afford homeostasis using alternatives to AIS?. Indeed, this would be possible to some degree and we do not exclude that possibility: any solution is more than likely some form of hybrid. We argue that using the immune system as inspiration, may lead to new insights into homeostatic concepts for PerAda systems, such as robotic organisms, and immune algorithms may afford an extended level of operation that is currently not possible.

4

Conclusion

In this position statement, we argue that the immune system is a suitable candidate for the development of homeostatic concepts for collective robotic systems that are required to operate in challenging environments for upto 100 days of continual operation.

Acknowledgement This work was supported in part by the SYMBRION project under FP7: FET Proactive Initiative: PERVASIVE ADAPTATION (PERADA), grant number FP7-ICT- 2007.8.2. The authors thank Maizura Mohktar, Nick Owens, Ran Bi and Lachlan Murray for their work on SYMBRION.

On Homeostasis in Collective Robotic Systems

309

References 1. Cohen, I.R.: Tending Adam’s Garden: Evolving the Cognitive Immune Self. Elsevier Academic Press, Amsterdam (2000) 2. Owens, N., Timmis, J., Greensted, A., Tyrrell, A.: On immune inspired homeostasis for electronic systems. In: de Castro, L.N., Von Zuben, F.J., Knidel, H. (eds.) ICARIS 2007. LNCS, vol. 4628, pp. 216–227. Springer, Heidelberg (2007) 3. Kernbach, S., Hamann, H., Stradner, J., Thenius, R., Schmickl, T., Crailsheim, K., van Rossum, A., Sebag, M., Bredeche, N., Yao, Y., Baele, G., Van de Peer, Y., Timmis, J., Mohktar, M., Tyrrell, A., Eiben, A., McKibbin, S., Liu, W., Winfield, A.: On Adaptive Self-Organization in Artificial Robot Organisms. In: Computation world 2009, pp. 33–43 (2009)

Can a Developmental AIS Provide Immunity to a Multi-cellular Robotics System? Maizura Mokhtar and Yang Liu Department of Electronics, University of York, UK {mm520,yl520}@ohm.york.ac.uk

Abstract. The major challenge to multi-cellular robotics system is how to ensure the system is homeostatically stable. This position paper proposes a developmental artificial immune system (dev-AIS) framework that tries to provide and maintain homeostasis to the multi-cellular robotics system. If immunity is defined as the ability to maintain homeostasis; the dev-AIS framework will be designed based on the understanding and the abstraction of how different organisms attain for this property through evolution and developmental process. Progress in evolution drove the evolution of immunity from this simple relationship to the development of the immune system in multi-cellular organisms.

1

Introduction

This position paper proposes the idea of a developmental AIS (dev-AIS) framework that may provide and maintain homeostasis of a multi-cellular robotics system autonomously. The dev-AIS framework describes how immunity of a complex multi-cellular system emerges from immunity of its smaller components, and how homeostasis of a complex system is modulated, whilst homeostasis of its smaller components remains unaffected. The dev-AIS provides for this ability by abstracting the evolution and developmental processes of immunity from the predator-and-anti prey relationship of single-celled organism to the multi-cellular and multi-organ biological immune system.

2

Objectives of the dev-AIS Framework

If a robot is considered as a cell, the multi-cellular robotic organism may resemble that of a biological multi-cellular organism. Immunity can be achieved (i) in singular robot: by applying predator-and-anti-prey relationship of single-cell organism, for example how flora bacteria survive in the human gut by acting as predator to the food digested in the gut and anti-prey or resistance to condition of the gut via gene mutation and/or lateral gene transfer [1] (ii) in swarm and aggregation mode: by applying so, cells specialisation occur in early multi-cellular organisms such as Volvox and D. discoideum [2] and in (iii) aggregated mode: the regulation of functions for different immune cells in evolved multi-cellular E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 310–311, 2010. Springer-Verlag Berlin Heidelberg 2010

Can a dev-AIS Provide Immunity to a Multi-cellular Robotics System?

311

organism may help regulate the functionality of each robot in the robotic organism. (i) can be described using genetic regulatory network (GRN) and (ii) and (iii) uses the combination of GRN and AIS algorithms. The predator and anti-prey relationship of single celled organism can describe the system management of a robot, responsible for efficient power and task distribution by predator: to solve a task within good time by making full use of available resources and anti-prey: to avoid running out of energy too quickly and/or scheduling error or bottleneck to its processes. A robot must have the correct balance between the two to ensure its survivability. The development and specialisation of cells in early multi-cellular organism can help describe the aggregation process; by defining how a strong and stable structure is produced [3], whilst homeostasis of its individual robots remains unaffected. This is achieve via predator: to produce the structure within minimum time and reject weak robots and anti-prey: to produce with minimum power and provide compensation to weaknesses. Aggregation is instigated, for example, because a task is too demanding and needs additional resources and/or a robot is at fault or with low energy. Such detections can be indicated by the AIS [4]. When the robots are aggregated in a single multicellular organism, it is more efficient that not all (individual) robots do similar action but instead each robot provides for its specialised functionality. How biological multi-cellular organisms modulate the interaction between the innate and adaptive immune cells can help ensure good coordination of the specialised processes. This includes anomaly detection and the definition of simple behaviour of individual robots (inherited) (Innate) and the response and adaptation to the environment (Adaptive).

3

Summary

If, homeostasis is the ability to maintain steady state operation despite any condition; in biology, homeostasis assimilates to immunity and is an emerging property. If we employ similar evolutionary and developmental processes of immunity in biology to a multi-cellular aggregated robotic system, homeostasis (immunity) of the aggregated robotic system may emerge because of the propagation of homeostasis of its individual robots.

References 1. Ley, R.E., Peterson, D.A., Gordon, J.I.: Ecological and evolutionary forces shaping microbial diversity in the human intestine. Cell 124, 837–848 (2006) 2. Bonner, J.T.: Evolutionary strategies and developmental constraints in the cellular slime molds. The American Naturalist 119, 530–552 (1982) 3. Steiner, T., Trommler, J., Brenn, M., Jin, Y., Sendhoff, B.: Global shape with morphogen gradients and motile polarized cells. In: Proc. of Congress on Evolutionary Computation (CEC2009). IEEE Press, Los Alamitos (2009) 4. Humza, R., Scholz, O., Mokhtar, M., Timmis, J., Tyrrell, A.M.: Towards Energy Homeostasis in an Autonomous Self-Reconfigurable Modular Robotic Organism. In: Proc. ADAPTIVE 2009. IEEE Computer Society, Los Alamitos (2009)

Using Virtual Embryogenesis for Structuring Controllers Ronald Thenius, Michael Bodi, Thomas Schmickl, and Karl Crailsheim Artificial Life Laboratory of the Department of Zoology, Karl-Franzens University Graz, Universit¨ atsplatz 2, A-8010 Graz, Austria [email protected]

Abstract. One possibility to increase the efficiency of classical controller paradigms is to implement substructures with different tasks and abilities (heterogeneous controller structure). To automatically develop these structures it is necessary to use a process, that is on the one hand easy to evolve, and on the other hand rich in different solutions. One inspiration for such processes can be found in the biological process of embryogenesis. By using virtual embryogenetical processes in combination with artificial evolution we develop a novel method for controller structuring.

With rising complexity of the tasks standard controllers like “classical” Artificial Neural Networks (ANNs) soon become inefficient due to increasing complexity of learning or adaptation processes. One solution to increase the adaptivity of such controlling structures is to implement substructures with different tasks within the controller. The structuring of the controller can be manually predefined [1], or shaped by artificial evolution [2]. To solve the task to develop evolutionary shaped controllers it is important to find an evolution friendly method to decrease calculation cost and predict evolution from finding local optima. To reach this goal we developed the concept of Virtual Embryiogenesis (VE) by modelling the biological process of embryogenesis. During biological evolution the processes of embryogenesis showed to be an ideal tool for shaping the bodies (including the control structures) of all multicellular lifeforms. The mechanisms shaping an lifeform are known as EvoDevo [3], which are perfectly able to work as a substrate for evolutionary processes. In biology, four phases can be observed in the evolution of multicellular organisms: 1. Optimisation: A morphometrical structure (including controlling structures, such as neural ganglia) are optimised by evolutionary processes. 2. Serialisation: Due to changes on genetic level the optimised morphological structure is built identically several times within an animal. 3. Exploration: Former identical morphological structures start to differ in their shape and function. 4. Specialisation: If a new shape becomes advantageous for a new function, the optimisation process is repeated. Disadvantageous structures are discarded. E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 312–313, 2010. c Springer-Verlag Berlin Heidelberg 2010 

Using Virtual Embryogenesis for Structuring Controllers

313

The structuring process of the biological embryos is controlled by morphogens, which are emitted by the cells of the embryo (see Fig. 1(a)). The cells themselves react to different concentrations of morphogens, e.g., with duplication or emission of morphogens. The reaction of the cells is encoded in the genome of the cells. By modelling these processes (see Fig. 1(b)) we are able to develop controller structures (see Fig. 1(c)) and test them regarding defined abilities (for details see [2]). Although VE was developed to shape ANNs, we plan to adapted to different kinds of controllers, that can be described by graphs.

(a) biological embryo

(b) virtual embryo

(c) VE-ANN

Fig. 1. Biological and virtual embryos. (a): Different morphogen concentrations in a biological embryo (from [4]). (b): Morphogen concentrations in a hand-coded virtual embryo. Dots indicate single cells, colours indicate morphogen gradients. (c): Structured ANN developed inside a virtual embryo. Dark lines indicate short distance connections (local sub-networks), light lines indicate long distance connections (global network).

References 1. Nolfi, S., Parisi, D.: Auto-teaching: Networks that develop their own teaching input. In: Deneubourg, J.L., Bersini, H., Goss, S., Nicolis, G., Dagonnier, R. (eds.) Proceedings of the Second European Conference on Artificial Life (1993) 2. Thenius, R., Schmickl, T., Crailsheim, K.: Novel concept of modelling embryology for structuring an artificial neural network. In: Troch, I., Breitenecker, F. (eds.) Proceedings of the MATHMOD (2009) 3. M¨ uller, G.B.: Evo-devo: Extending the evolutionary synthesis. Nature Reviews Genetics 8, 943–949 (2007) 4. Jaeger, J., Surkova, S., Blagov, M., Janssens, H., Kosman, D., Kozlov, K.N., Manu, Myasnikova, E., Vanario-Alonso, C.E., Samsonova, M., Sharp, D.H., Reinitz, J.: Dynamic control of positional information in the early Drosophila embryo. Nature 430, 368–371 (2004)

Towards Self-aware PerAda Systems Emma Hart and Ben Paechter Edinburgh Napier University {e.hart,b.paechter}@napier.ac.uk

Abstract. Pervasive Adaptation (PerAda) refers to massive-scale pervasive information and communication systems which are capable of autonomously adapting to highly dynamic and open technological and user contexts. PerAda systems are thus a special case of collective adaptive systems which have particular constraints e.g. they are networked and highly distributed; they involve interaction with humans; they are large scale; the boundaries of systems are fluid; their context is dynamic; and they operate using uncertain information. In order to achieve their ultimate goal of adapting seamlessly to their users and to deliver the expected quality of service at all times, we propose that these systems must exhibit self-awareness. This position statement proposes mechanisms by which self-awareness might be achieved.

1

A Homunculus for PerAda Systems

To be self-aware, a system must have an internal self-image: moreover, in the case of PerAda systems, that image must be dynamic as the system itself has a dynamic and fluid composition. Borrowing terminology used in both neurology and immunology we propose that collective systems must develop their own homunculus. Stemming from the Latin term “little man”, the neurological homunculus refers to the functional map of the body represented in the brain which determines how it interacts with its environment; the immunological homunculus refers to the immune system’s internal image of the body. Both the neurological and immunological homunculi can be viewed as a dynamic representation of the body and its various states: the representation is a reduced, virtual image of the system in that it contains far less information than the total amount of information contained in the body itself, but encodes the functional essence of the body’s state [2]. In contrast to traditional perceptions of immunity, the key concept of the homuculus is that auto-immunity, that is, the ability of the immune system to detect and respond to the state of the body’s cells and tissues, is a normal and critical process of the functioning of the immune system; the image is created by body cells that impinge on the immune receptors and crucially by the response of cells to this information [3]. This results in a self-referential system which has obvious advantages for achieving collective behaviour; it is not necessary for every part of the system to observe the state of the overall system but sufficient to observe how local parts of the system are behaving. However, in contrast to the immunological or neurological homunculi, a pervasive system does not have a defined boundary. This fluidity of boundary requires E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 314–316, 2010. c Springer-Verlag Berlin Heidelberg 2010 

Towards Self-aware PerAda Systems

315

the concept of self-awareness to be completely reviewed when self is a fluid entity. Thus, although the notion of internal-image of an engineered system has been previously considered in the literature, we suggest that in order to create a self-image of a PerAda system, we need to go beyond current thinking: we propose a PerAda homunculus might be created if the system can: – Look inside: nodes and systems must be able to inspect and understand the implications of their own internal state and operational conditions – Look outside: nodes and systems need to be aware of the environment in which they exist, including other nodes and systems as well as users of the environment – Look inside from outside: nodes and systems need to be aware of how their actions and responses are perceived by other nodes and systems external to them, including users. – Look backward: nodes and systems need to accumulate and make sense of information collected from the system in the past and draw correlations between data and actions – Look forward: nodes and systems need to look forward in a predictive manner and anticipate potential problems and maybe take corrective action We dub this new awareness look-* and propose that looking to the natural immune system and the mechanisms by which it computes a homunculus can provide some inspiration for building systems which encapsulate these concepts.

2

The Recursive Nature of PerAda Systems

We further note that PerAda systems can be thought of as recursive, i.e. almost fractal in nature, in that a system consists of many subsystems, each of which can be thought of as system in itself such that each part is a reduced size copy of the whole with respect to the general challenges of adapting to the environment and in that the boundaries between the subsystems are fluid: in the horizontal dimension, at any time a system (component, devices, group, network etc.) may be joined by one or more other systems to form a new system incorporating the goals and context of all. Similarly, a system may split into two or more systems each requiring goals and context based on the parent system. In the vertical dimension a system might be considered to be at different levels depending on the viewpoint. Something that is a device from one point of view might be a component from another point of view. What we might view as a complete system (e.g. all the components, devices, groups and networks that exist within our car) might be seen as single component from another point of view (e.g. when considering the traffic flow through a city). Adaptation takes place across multiple scales; the adaptation process repeats itself from the small scale at the innermost level of the system (e.g. a component in a car) to the large scale outermost level (e.g. the traffic in a city). Again, looking to the immune system might inform our thinking: the immune system operates across significant different scales, ranging from interactions between tissues via cytokine release which

316

E. Hart and B. Paechter

occur across time-scales ranging from minutes to hours and distances of up to a metre to intercellular networks which operate across seconds over distances of a few millimetres [1].

3

Conclusions

In summary, collective systems are fluid, fractal-like systems which must develop their own internal self-image, a homunculus, through which they can successfully interact with the environment and collectively adapt. Steps towards defining and maintaining a collective homunculus will be made by considering self-awareness from a multi-faceted perspective, such as look-* self awareness which accounts for the self-referential nature of the homunculus and the fractal-like nature of the system itself.

References 1. Callard, R., Stark, J.: Networks of the Immune System. In: Biological Networks. World Scientific Publishing, Singapore (2007) 2. Cohen, I.: Immune system computation and the immunological homunculus. In: Model Driven Engineering Languages and Systems, pp. 499–512. Springer, Heidelberg (2006) 3. Cohen, I.: Real and artificial immune systems; computing the state of the body. Nature Reviews Immunology (2007)

Is Receptor Degeneracy Suitable for Automatic Response Decisions in Ad Hoc Networks? Sven Schaust, Martin Drozda, and Helena Szczerbicka Simulation and Modeling Group, Faculty of Electrical Engineering and Computer Science, Leibniz University of Hannover, Welfengarten 1, 30167 Hannover, Germany {svs,drozda,hsz}@sim.uni-hannover.de

1

Introduction and Position

Wireless ad hoc networks are an example for pervasive communication systems. Due to their decentralized design, they can be subject to a large variety of communication errors, system failures or even attacks and intrusions. Current research focuses either on specific avoidance and/or protective actions such as secure routing protocols or on general misbehavior detection with human operator based response management [1]. However, due to the complexity of large-scale ad hoc networks, self-organization and self-healing properties are necessary. A concept which allows for automatic responses to mitigate the effects of misbehavior is missing. In this statement we discuss whether an adaptive decision making process based on the concept of receptor degeneracy, being a part of the cognitive immune self theory described by Cohen in 2000, is feasible. 1.1

Position Statement

In order to provide an automatic response to detected misbehavior, a response selection approach is necessary. We assume that such a selection approach cannot be represented by a simple one-to-one mapping strategy. For example a single reaction could be used to mitigate the effects of different types of misbehavior. Therefore a mapping which considers many-to-many relations seems necessary. We propose that this problem can be satisfactorily addressed by the concept of receptor degeneracy.

2

Cognitive Immune Self and Receptor Degeneracy

In 2000 Cohen [2] published a distinct view of the role of the immune system. Instead of being purely a discriminator between self and non-self antigens, he suggests that the immune system has a cognitive component which allows for maintenance and repair of the body. In his view the process of inflammation and the removal of pathogens are simply a way to maintain the health of a body. Cohen suggests that the evolution of the immune system did not produce a perfect system, the evolutionary process rather created a system that is able to find E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 317–319, 2010. c Springer-Verlag Berlin Heidelberg 2010 

318

S. Schaust, M. Drozda, and H. Szczerbicka

“good enough“ solutions. Cohen proposes several properties of the immune system which enable it to act as a maintainer (see P. Andrews [3] for more details). One important issue of the cognitive immune self is the ability of its receptors to be degenerate, meaning that they are able to bind to more than one antigen. Besides the cognitive immune self, homeostasis is another important concept of biological systems. It describes a system’s ability to stay in an equilibrium using maintenance procedures. The cognitive immune self and its maintenance procedures can be further described as a method to achieve homeostasis in the body. In the context of ad hoc networks, homeostasis is achieved if the network is able to perform its mission successfully, even in the presence of misbehavior. We eventually want to achieve homeostasis by applying an automatic response decision process to an existing Artificial Immune System based detection system. We consider the concept of receptor degeneracy as a feasible approach to the correlated classification problem.

3

Conceptual Design of an Approach

As briefly described in the previous section the concept of receptor degeneracy allows the binding of one receptor to more than one antigen. This allows a receptor to enact different immunological responses under different conditions. Translated to the misbehavior response problem, degeneracy allows a system to choose a response from a group of possible responses for a specific misbehavior, under different network conditions, even if none of the responses match exactly. The mitigation and thus the reaction procedures and their impact towards the network need to be evaluated by a feedback control loop. Such a control loop helps to improve the response selection and at the same time creates a response memory for faster and more accurate responses. Another goal of such an approach is to achieve network homeostasis. In this context homeostasis can be defined as the ability of an ad hoc network to mitigate the effects of failures and misbehavior in such a way that the network is still able to fulfill its mission. We also hope to improve the detection performance due to a better understanding of the effects triggered by misbehavior and response. 3.1

A Network Scenario Based Example

Let us now consider an Artificial Immune System based misbehavior detection system which can be performed locally at each node of an ad hoc network. This system is extended by a response repository, offering a selection of possible responses. Once a misbehavior has been detected, it needs to be classified in order to provide a selection of binding possibilities. Each response has several receptors (thus being applicable to several classes of misbehavior) which allow to bind to one or more specific misbehavior antigens. Additionally we assume that each response is weighted by its costs for a node (being an energy constrained participant) to ensure that cheap solutions get used more often. Once a response has been selected and triggered, network quality of service (QoS) parameters

Is Receptor Degeneracy Suitable for Automatic Response Decisions

319

are collected. These QoS parameters are used within a control feedback loop to identify whether a local equilibrium is achieved. Network homeostasis can be achieved on that way if the proposed approach is able to learn which responses are the most suitable ones, dependent on the locally determined network conditions. Note that we do not exclude cooperation between nodes in order to gain a better understanding of the reaction effects. Thus the system should be able to maintain a reasonable network service level.

4

Possible Counter Argument

Is receptor degeneracy really necessary for classification? The question how to define a suitable mapping between possible reactions and detected misbehavior can be approached from different research backgrounds. A possible solution would be a simple one-to-one mapping function which maps a specific reaction to a misbehavior, regardless of the network conditions. This might work for simple scenarios, however such an approach lacks any adaptability towards changing network conditions. We believe that using the concept of receptor degeneracy is a suitable approach providing adaptability and learning capabilities which are necessary for autonomous systems in changing environments.

5

Conclusion

In this position statement we propose a conceptual design of an adaptive response decision mechanism that uses receptor degeneracy as an underlying concept. The main goal of this approach is to enhance an existing Artificial Immune System in such a way that it supports automatic responses to mitigate the effects of misbehavior and thus achieves homeostasis in the network. We expect to gain new insights on the applicability of concepts adapted from the cognitive immune systems theory for pervasive autonomous systems.

Acknowledgments This work is supported by the German Research Foundation (DFG) under the grant no. SZ 51/24-3 (Survivable Ad Hoc Networks – SANE).

References 1. Anantvalee, T., Wu, J.: A survey on intrusion detection in mobile ad hoc networks. Wireless/Mobile Network Security, 159–180 (2007) 2. Cohen, I.: Tending Adam’s Garden: evolving the cognitive immune self. Academic Press, San Diego (2000) 3. Andrews, P.S.: An Investigation of a Methodology for the Development of Artificial Immune Systems: A Case-Study in Immune Receptor Degeneracy. PhD Dissertation, University of York, UK (2008)

Biochemically-Inspired Emergent Computation Lidia Yamamoto1 and Thomas Meyer2 1 Data Mining and Theoretical Bioinformatics Team Image Sciences, Computer Sciences and Remote Sensing Laboratory (LSIIT) University of Strasbourg, France [email protected] 2 Computer Networks Group, Computer Science Department University of Basel, Switzerland [email protected]

1

Artificial Chemistries for Pervasive Adaptation

Pervasive Adaptation software systems are expected to exhibit life-like properties such as robust operation in uncertain environments, adaptive immunity against foreign attackers, self-maintenance, and so on. The traditional software design model based on top-down human engineering fails in this context, where new, bottom-up emergent computation [1,2] techniques seem more appropriate. Since chemistry and biochemistry are the basis of life, Artificial Chemistries [3] and Artificial Biochemistries [4] stand out as natural ways to model such bottomup life-like software. However, understanding and harnessing the power of emergent behavior in such complex systems is difficult. This position statement highlights some potentially fruitful research directions towards this goal. We advocate that an important research goal within such bottom-up approach is to construct systems able to achieve automatic transitions from lower levels of complexity to higher ones.

2

Towards Life-Like Biochemically-Inspired Software

A bottom-up approach to software construction using artificial (bio-)chemistries would necessarily start at the microscopic level of elementary molecules, and then move up in complexity, to cause molecules to organize into autocatalytic reaction networks forming primitive metabolisms [5], then into protocells [6], and later into more complex cellular structures including a genome and a genetic regulatory network (GRN). The cells would then organize into multicellular organisms relying on morphogenetic mechanisms for building complex structures and dividing tasks among its constituent elements. Some of the cells in such a virtual organism could differentiate into an artificial immune system (AIS) to protect the organism from external aggressions. Several bio-inspired techniques covering each of these complexity levels have been proposed. However, little is known about how the transition from a lower level to an upper one can be achieved in an artificial system in an emergent way. The benefit of a system exhibiting such autonomous complexity growth behavior would be open ended evolution, leading to continuous and unlimited adaptation potential. E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 320–322, 2010. c Springer-Verlag Berlin Heidelberg 2010 

Biochemically-Inspired Emergent Computation

321

We now briefly discuss some of these levels, and some research issues related to their transitions to upper levels, relating the discussion with our own research experience. We focus on the lower levels that are the foundations for the upper ones, and are more closely related to our experience. At the bottom level of individual molecules, computation can be performed in vitro by encoding information within molecules as in DNA Computing, via concentrations of molecules, or via the propagation of waves of chemicals. Several in silico algorithms inspired by chemistry can be found in literature [3]. In [7] we proposed a chemically-inspired technique for self-healing software based on autocatalytic quines, self-replicating program fragments that react together to perform some computation and to reproduce. The bimolecular interactions in this model lead to a “survival of the common” outcome, which has two consequences: it makes the population resistant to invasion, conferring an immunity to attacks and harmful mutants; but it also shows poor evolutionary capabilities, any potential improvement being quickly washed out. A transition to an upper level of organization is therefore difficult to achieve with such system. In [8] we proposed an energy model to be used for computations with artificial chemistries. We showed that this model is able to control the otherwise spontaneous elongation of autocatalytic quine strings. Moreover, clusters of cooperating quines exhibiting controlled and localized autocatalytic activity emerged under the right amount of energy inflow. In [9], we use a similar energy model to obtain catalytic search, a new optimization algorithm that makes use of the phenomenon of catalytic focusing described in [5] to steer the flow of chemical reactions towards those reactions leading to better-fit products. This could be a step in the direction of coupling the emergence of self-maintaining reaction networks with the computation of products that are fit for a given task. Moreover it could potentially exhibit a transition from a catalytic to a self-reproducing model that could lead to evolution, although we have not been able to demonstrate this yet. The transition from chemical networks to cells is a difficult one, related to the origins of life and the emergence of protocell structures such as the chemoton [10]. The use of such cellular structures for computation is definitely worth further exploration. Once a basic cellular structure is there, the transition to a multicellular level requires the ability of cells to communicate via signalling mechanisms, to process these signals inside the cell via gene regulation networks, and to use these signals for pattern formation in a morphogenetic process. Recently [11], we have evaluated the robustness of specific morphogenetic algorithms, which can be regarded as another example of chemically-inspired algorithms with self-healing properties. Finally, combining the lowest with the upper levels, the quines from [7] have been applied in [12] to devise an AIS in an artificial chemistry context.

3

Research Agenda and Outlook

The investigation of transition steps from one level of complexity to the next is definitely an important item in the agenda towards open-ended evolutionary

322

L. Yamamoto and T. Meyer

systems capable of unlimited adaptation. In biology the transition mechanisms are also fundamental and not entirely understood [13]. Each transition stage can be studied independently. Currently, we are focusing on the first one: how to design an artificial chemistry that can perform intended computations and at the same time move towards higher levels of complexity when needed.

Acknowledgments This paper was written mostly while Lidia Yamamoto was with the University of Basel, Switzerland. The authors would like to thank Prof. Christian Tschudin and Prof. Emma Hart for their helpful support and encouragement.

References 1. Forrest, S.: Emergent Computation: Self-organizing, Collective, and Cooperative Phenomena in Natural and Artificial Computing Networks. Physica D 42(1-3), 1–11 (1990) 2. Banzhaf, W., Dittrich, P., Rauhe, H.: Emergent Computation by Catalytic Reactions. Nanotechnology 7, 307–314 (1996) 3. Dittrich, P., Ziegler, J., Banzhaf, W.: Artificial Chemistries – A Review. Artificial Life 7(3), 225–275 (2001) 4. Lones, M.A., Tyrrell, A.M., Stepney, S., Caves, L.S.: Controlling Complex Dynamics with Artificial Biochemical Networks. In: Proc. EuroGP (April 2010) 5. Bagley, R.J., Farmer, J.: Spontaneous Emergence of a Metabolism. In: Artificial Life II, pp. 93–140. Addison-Wesley, Reading (1991) 6. Rasmussen, S.: Protocells: Bridging Nonliving and Living Matter. MIT Press, Cambridge (2008) 7. Meyer, T., Schreckling, D., Tschudin, C., Yamamoto, L.: Robustness to Code and Data Deletion in Autocatalytic Quines. In: Priami, C., Dressler, F., Akan, O.B., Ngom, A. (eds.) Transactions on Computational Systems Biology X. LNCS (LNBI), vol. 5410, pp. 20–40. Springer, Heidelberg (2008) 8. Meyer, T., Yamamoto, L., Banzhaf, W., Tschudin, C.: Elongation Control in an Algorithmic Chemistry. In: Proc. ECAL (September 2009) 9. Yamamoto, L.: Evaluation of a Catalytic Search Algorithm. In: Proc. NICSO, Granada, Spain (May 2010) 10. G´ anti, T.: Chemoton Theory: Theoretical Foundations of Fluid Machineries, vol. 1. Kluwer Academic, Dordrecht (2003) 11. Yamamoto, L.: Evaluating the Robustness of Activator-Inhibitor Models for Cluster Head Computation. In: Proc. ANTS, Special Session on Morphogenetic Engineering, Brussel, Belgium (September 2010) (to appear) 12. Schreckling, D., Marktscheffel, T.: An Artificial Immune System Approch for Artificial Chemistries Based on Set Rewriting (2010) (submitted for publication) 13. Maynard Smith, J., Szathm´ ary, E.: The Major Transitions in Evolution. Oxford University Press, Oxford (1995)

Nature-Inspired Adaptivity in Communication and Learning Borbala Katalin Benko and Vilmos Simon Department of Telecommunications, Budapest University of Technology and Economics, Hungary, {bbenko,svilmos}@hit.bme.hu

In this paper we investigate two bottlenecks of today’s networked adaptive systems (communication and learning) from a nontraditional viewpoint.

Adaptive Communication Schemes Paradigms like delay-tolerant networks or opportunistic networking appeared as networked environments called for new approaches in communication. Due to the often decentralized and self-organized character of these systems, the amount of explicit control in these systems must be reduced radically. Our vision moves one step further, towards a system where the communication scheme itself also emerges from the actual conditions. When each element of the network uses a simple mechanism to fine-tune the amount and way of communication to match its local needs and resources, we can easily end up in an emergent system where the communication (amount and content) is globally adaptive, ‘by itself’. As an extremity, there may be no fixed information dissemination algorithm in the system at all, but rather, a set of building blocks that can be assembled into meaningful protocols. Various bio-inspired directions can be utilized to cope with the creation and selection of the best protocols, like the principle of natural selection of the most adequate communication algorithm in changing environments, or an evolutionary programming language for assembling the existing protocol primitives into new, more complex ones. The survival of the competing algorithms shall depend on their efficiency. The decentralized evolution of the communication schemes should be driven by the actual environmental and contextual conditions (changing topology and varying information needs). It should be highlighted that the emergent properties of the communication’s evolution are unpredictable and unprecedented; the complex behavior of the resulted networking protocols can not be predicted easily or deduced from the behavior of the lower-level protocol primitives. Moreover, if we use local fitness functions only (based on locally available performance indicators), it will result in a fast convergence to locally effective new communication schemes, while sidestepping any central control, and keeping the system scalable and robust. What could be the application area for this adaptive communication scheme? Certain properties of such a protocol would be unacceptable in traditional E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 323–325, 2010. c Springer-Verlag Berlin Heidelberg 2010 

324

B.K. Benko and V. Simon

telecommunication approaches due to the strict quality of service regulations; however, a similar concept (policies) already managed to find its place in IP based communication schemes. On the other hand, the further deployment of mobile ad hoc and sensor systems converging to opportunistic networking will most likely reshape the expectations here. Opportunistic networking is not capable of superseding existing cellular mobile networks and IP based systems, but these ad hoc, peer-to-peer based communication schemes will offer a good alternative in scenarios where traditional solutions would perform poorly in sense of scalability and robustness. Just imagine the digital city of tomorrow, where the majority of users will access services via wireless links and mobile terminals, together with myriads of sensors providing a spate of unclassified information, creating a dynamically changing digital environment. By using a fixed and central infrastructure or database, it would be very hard to serve users with fresh local information, not to speak about scalability problems of the underlying infrastructure. A high level of flexibility is needed not only from the data perspective, but also from the viewpoint of communication schemes. In these scenarios, where the terminals of the network let users communicate with each other directly (without any intermediate infrastructural element) the network topology will change in a dynamic and unpredictable way. This problem clearly calls for adaptive communication protocols, which we propose facing with mechanisms of evolution and natural selection. A change in the network topology or environment will actuate a change in the communication protocol; and protocols will evolve, new ones will beat the existing protocol in an everlasting competition, always adapting to the actual user needs.

Adaptive Learning Today’s learning systems - including open systems - often have painful limitations. Even though being communicated otherwise, generally, they’re still not open or adaptive enough to perform well in real environments (the ways of openness are hardcoded). No wonder, because the usage of ontology, self- and worldmodels, deduction and other ‘conscious’ tools - where the system ‘understands’ its input well and ‘knows’ when and how to utilize it - makes it theoretically impossible to be open to drastically new information sources (which are not part of the world or self model) or to develop drastically new directions for utilizing the knowledge (which are not part of the initially hardcoded adaptation mechanisms). In our point of view, besides the ‘conscious’ part - where powerful tools such as deduction and reasoning can be used - tomorrow’s systems also need a ‘subconscious’ part - where the system is able to detect radically new relationships between inputs or outputs, without having information about the meaning or utilization ways of the data. One may ask whether such a ‘subconscious’ way of operation could be of high enough quality. The outcome of recent data mining competitions suggests: yes. For example, in case of the ACM KDDCup 2009, the challenge was to outperform France Telecom’s churn prediction model (developed by the company’s

Nature-Inspired Adaptivity in Communication and Learning

325

professionals who had full access to all resources), but without knowing anything about the meaning of the data (the input was 10.000 unlabeled Z-normed values per person). Within two weeks, a ‘subconscious’ - blind, purely data mining based - solution outperformed the original ‘conscious’ solution by far. The two parts of the adaptive system - conscious and subconscious - should coexist, and strengthen each other. Subconscious findings may be propagated upwards, and may result in new, conscious-level knowledge (which can be later utilized effectively though reasoning, for example). On the other hand, high-level knowledge may help in filtering out nonsense directions on the lower level. The utilization of subconscious findings may also happen directly on the subconscious level: a detected correlation relationship - e.g. between two sensorial inputs - may lead to adaptive sensor usage (use the “cheaper” one, if they basically provide the same information). Predictive relationships may help in estimating the future and optimizing the system’s operation. Reflexes (semantically not explained, but practically working actions or reactions) may also be developed. In terms of implementation for the subconscious pier, simple yet efficient mechanisms are required, that also take into account that the amount of processing power is not unlimited. The actual tools for that should be co-inspired by nature (neurons, brain, swarms) and information science (e.g. cognition or data mining). We think of data mining techniques as a basis (filtering and aggregating the sensed data, and identifying knowledge - patterns, correlations, trends, predictive relationships - in them), combined with a competitive-selective model, for example evolutionary systems (e.g. competing data mining mechanisms) or data driven learning (focusing on what’s available at the moment). Nature often reaches learning and adaptation goals by means of multitudes of simple mechanisms (such as neural networks or the immune system), and often in a collaborative manner. Similarly in our vision, the series of active or passive observations and the collaboration of the learning entities should naturally lead to the evolving adaptation behavior. In such a system, the body of knowledge emerges from and during the system’s normal operation (learning while doing) which also means that adaptation does not stop after a certain time but seamlessly continues and helps the system reacting to changes. An optimal mixture of conscious and subconscious mechanisms is needed here: the conscious part shall take care of optimality and convergence, and the subconscious part shall ensure the openness. The emergence of knowledge may not stop at the level of single nodes. Collaboration between learners may lead to the emergence of a common know-how, or, a ‘common culture’. Locally identified findings may be evaluated collectively, and if valid and useful, may be saved or disseminated. In such systems, an evolved society may inject this culture into newborn/newcoming elements, speeding up the adaptation process and sparing time and resources. Open questions include: validity of knowledge over time, generalization techniques (e.g. conditionally valid truth), scaling questions, and ways of gathering feedback about the results.

Symbiotic Cognitive Networks: A Proposal Tinku Rasheed1 , Emma Hart2 , Jim Bown3 , and Ruth Falconer3 1 Create-Net Edinburgh Napier University 3 University of Abertay [email protected], [email protected], {j.bown,r.falconer}@abertay.ac.uk 2

Abstract. We describe the concept of a cognitive network and propose that ecosystems of co-existing networks which are globally energy efficient while providing the expected quality of service can be realised by exploiting two mechanisms which occur in biological systems; symbiosis and co-existence.

1

Cognitive Radio Networks

Cognitive radio is a paradigm for wireless communications in which either a network or a device can change its transmission or reception parameters (power, frequency etc.) to communicate efficiently avoiding interference with adjacent networks. This alteration of parameters is based on the active monitoring of several factors in the external and internal radio environment, such as radio frequency spectrum, user behaviour and network status. Cognitive networks are characterized by flexible network architectures exploiting the capabilities of cognitive radios[2] and have the potential to drive significant innovations [1]. As the global telecommunication industry is gearing itself towards the implementation of wireless networks which are autonomously managed, we can foresee a plethora of flexible and heterogeneous wireless networks including: wide-area terrestrial infrastructure networks, sensor networks, femto-cellular networks, mesh networks, self-organizing ad hoc networks which will need to coexist and cooperate in a global environment. Figure 1 depicts a typical scenario; a single telecom operator leases spectrum in a certain range - within this spectrum, a number of individual (and heterogenous) networks (CN1, CN2,...) are required to operate without causing interference to each other. In order to minimize interference, each network has two choices: 1. Operate at a unique frequency (depicted by the vertical bars on the right hand graph) 2. Operate at low power which limits interference with other networks (depicted by the horizontal bar). Therefore, for the global system to function, each of the networks must cooperate and/or collaborate by sharing and/or adapting its resources. Cognitive E. Hart et al. (Eds.): ICARIS 2010, LNCS 6209, pp. 326–328, 2010. c Springer-Verlag Berlin Heidelberg 2010 

Symbiotic Cognitive Networks: A Proposal

327

radio offers the potential to endow the networks with the ability to adapt such that a scalable system of networks can be created, optimised in terms of energy efficiency and resource conservation. However, its use to date has been mainly limited to frequency agility, i.e in selecting the best radio access technologies and frequencies at which to operate. In order to fully exploit its potential, we propose to exploit the adaptive capabilities of cognitive radio to create systems of networks. A major challenge is to quantify the degree of interaction between the heterogeneous sub-networks and to enable these sub-networks to operate efficiently without compromising their own resources. Moreover, a unified approach to the design of cognitive networks is necessary to characterize the scale at which the cognitive network interactions can grow and flourish, in a global system of networks. While designing systems of cooperating networks may be challenging for engineers, it is a problem which has been neatly solved in nature by evolution, which enables stable eco-systems to exist in which multiple species either co-exist - that is, live in separate niches which do not overlap, or exist in symbiotic relationships, a term which was first coined in 1879 to describe ’the living together of unlike organisms’. We propose that co-existence within natural ecosystems has an obvious anaology to property (1) listed above where as symbiotic relationships map neatly to property (2). In the remainder of the paper we outline the main principles of these biological relationships and describe a mapping between the concept of an ecosystem of cognitive radios and biological systems. We propose that by looking in detail at two biological systems - the immune system and fungal communities - we can develop novel techniques for implementing the cognitive network paradigm.

Fig. 1. A system of networks composed of heterogeneous physical sub-networks

2

Relationships in Natural Ecosystems

Symbiotic relationships in nature occur in three forms; commensal, mutualistic and parasitic. The first two are of relevance to our argument. In commensal symbiotic relationships one organism reaps the benefit of another, while the other is not significantly harmed. The mammalian immune system provides an excellent example of such a relationship in which some species of bacteria successfully colonize our body and provide us with essential function without which

328

T. Rasheed et al.

we could not survive, such as stimulation of the gut, immune system development, rejection of pathogenic bacteria, and contributions to our nutrient supply. In mutualistic relationships, both species involved in the relationship benefit. Such relationships are observed between fungi and plants; fungi invade or live amongst the secondary roots of the plant - this benefits the plant which achieves a more efficient mineral uptake and also the fungi which benefit from the sugars translocated to the root by the plant. This relationship is also interesting from the perpective of the plant which could maintain its own mineral supply, but chooses to leave this to the fungi in order to optimize its resources elsewhere. Co-existence relationships are also observed in fungal communities; for examples diferent colonies can co-exist by sharing resource yet maintaing exclusive areas. We propose that by modelling and abstracting these biological processes, it may be possible to construct cognitive networks which can collectively exist within an eco-system that is globally optimal in terms of efficiency and usage of resource. A mapping between the two paradigms is shown in table 1. Table 1. Mapping between biological and cognitive networks Cognitive Network Example Biological System Device laptop, phone Organism Network wireless network Species Network of Networks spectrum licenced by single operator ecosystem

3

Future Work

We propose to investigate the possibility of creating scalable, adaptive ecosystems of networks by looking to biology. In summary, the utimate goals of our work are to explore theoretical models and abstract mechanisms of symbiosis to identify and model the interaction patterns of symbiotic relationships in light of the requirements stemming from future generation wireless networks; to quantify the amount of resources required for specific operations in coexisting networks, thereby conceiving an overall eco-footprint model for cognitive networks via adapted metrics derived from the proposed mapping; and to realize a unified framework for symbiotic cognitive networking synthesized from a novel multidisciplinary scientific approach.

References 1. Cognitive radio. MIT Technology Review (March-April) 2. Fortuna, C., Mohorcic, M.: Trends in the development of communication networks: Cognitive networks. Computer Networks (2009)

Author Index

Abi-Haidar, Alaa 237 Aickelin, Uwe 4 Albergante, Luca 1 Andrews, Paul 1 Azzolini, Alisson G. 33, 158 Banerjee, Soumya 116 Barbosa, Helio J.C. 130 Beattie, Lynette 1 Benko, Borbala Katalin 323 Bernardino, Heder S. 130 Bodi, Michael 312 Bown, Jim 326 Castro, Pablo A.D. 171 Chelly, Zeineb 102 Coghill, George M. 223 Crailsheim, Karl 312 Cutello, Vincenzo 144

Falconer, Ruth 326 Fanelli, Robert L. 264 Figueredo, Grazziela P. 4 Fonseca, Leonardo G. 130 75

Hart, Emma 18, 314, 326 Houstis, Elias 47 Huang, Rentian 290 Jansen, Thomas

61

Kaye, Paul M. 1 Kendall, Graham 75 Kodovas, Stefanos 47 Liu, Yang

75

Paechter, Ben 314 Pang, Wei 223 Pavone, Mario 144 Pitt, Jeremy 304 Polak, John 304 199

Schaust, Sven 317 Schmickl, Thomas 312 Schreckling, Daniel 250 Simon, Vilmos 323 Stracquadanio, Giovanni 144, 213 Szczerbicka, Helena 317 Tan, Su-Wei 185 Tawfik, Hissam 290 Tempesti, Gianluca 199 Terzis, Sotirios 276 Thenius, Ronald 312 Timmis, Jon 1, 199, 307 Tyrrell, Andy 199, 307 Vavalis, Manolis 47 Vella, Mark 276 Violato, Ricardo P.V. 33, 158 Von Zuben, Fernando Jos´e 33, 158, 171

199, 310

Marktscheffel, Tobias McEwan, Chris 18

Oates, Robert

Rasheed, Tinku 326 Rocha, Luis M. 237 Romano, Vittorio 213 Roper, Marc 276

102

Garibaldi, Jonathan M.

Nagar, Atulya 290 Nanas, Nikolaos 47 Nicosia, Giuseppe 144, 213

Qadir, Omer

Demiris, Yiannis 304 Drago, Concetta 213 Drozda, Martin 317 Elouedi, Zied

Meyer, Thomas 320 Mokhtar, Maizura 310 Moses, Melanie 116 Musselle, Chris. J. 88

Yamamoto, Lidia

320

Zarges, Christine

61

250