Ying-ping Chen and Meng-Hiot Lim (Eds.) Linkage in Evolutionary Computation
Studies in Computational Intelligence, Volume 157 Editor-in-Chief Prof. Janusz Kacprzyk Systems Research Institute Polish Academy of Sciences ul. Newelska 6 01-447 Warsaw Poland E-mail:
[email protected] Further volumes of this series can be found on our homepage: springer.com Vol. 134. Ngoc Thanh Nguyen and Radoslaw Katarzyniak (Eds.) New Challenges in Applied Intelligence Technologies, 2008 ISBN 978-3-540-79354-0 Vol. 135. Hsinchun Chen and Christopher C. Yang (Eds.) Intelligence and Security Informatics, 2008 ISBN 978-3-540-69207-2 Vol. 136. Carlos Cotta, Marc Sevaux and Kenneth S¨orensen (Eds.) Adaptive and Multilevel Metaheuristics, 2008 ISBN 978-3-540-79437-0 Vol. 137. Lakhmi C. Jain, Mika Sato-Ilic, Maria Virvou, George A. Tsihrintzis, Valentina Emilia Balas and Canicious Abeynayake (Eds.) Computational Intelligence Paradigms, 2008 ISBN 978-3-540-79473-8 Vol. 138. Bruno Apolloni, Witold Pedrycz, Simone Bassis and Dario Malchiodi The Puzzle of Granular Computing, 2008 ISBN 978-3-540-79863-7 Vol. 139. Jan Drugowitsch Design and Analysis of Learning Classifier Systems, 2008 ISBN 978-3-540-79865-1
Vol. 145. Mikhail Ju. Moshkov, Marcin Piliszczuk and Beata Zielosko Partial Covers, Reducts and Decision Rules in Rough Sets, 2008 ISBN 978-3-540-69027-6 Vol. 146. Fatos Xhafa and Ajith Abraham (Eds.) Metaheuristics for Scheduling in Distributed Computing Environments, 2008 ISBN 978-3-540-69260-7 Vol. 147. Oliver Kramer Self-Adaptive Heuristics for Evolutionary Computation, 2008 ISBN 978-3-540-69280-5 Vol. 148. Philipp Limbourg Dependability Modelling under Uncertainty, 2008 ISBN 978-3-540-69286-7 Vol. 149. Roger Lee (Ed.) Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, 2008 ISBN 978-3-540-70559-8 Vol. 150. Roger Lee (Ed.) Software Engineering Research, Management and Applications, 2008 ISBN 978-3-540-70774-5 Vol. 151. Tomasz G. Smolinski, Mariofanna G. Milanova and Aboul-Ella Hassanien (Eds.) Computational Intelligence in Biomedicine and Bioinformatics, 2008 ISBN 978-3-540-70776-9
Vol. 140. Nadia Magnenat-Thalmann, Lakhmi C. Jain and N. Ichalkaranje (Eds.) New Advances in Virtual Humans, 2008 ISBN 978-3-540-79867-5
Vol. 152. Jaroslaw Stepaniuk Rough – Granular Computing in Knowledge Discovery and Data Mining, 2008 ISBN 978-3-540-70800-1
Vol. 141. Christa Sommerer, Lakhmi C. Jain and Laurent Mignonneau (Eds.) The Art and Science of Interface and Interaction Design (Vol. 1), 2008 ISBN 978-3-540-79869-9
Vol. 153. Carlos Cotta and Jano van Hemert (Eds.) Recent Advances in Evolutionary Computation for Combinatorial Optimization, 2008 ISBN 978-3-540-70806-3
Vol. 142. George A. Tsihrintzis, Maria Virvou, Robert J. Howlett and Lakhmi C. Jain (Eds.) New Directions in Intelligent Interactive Multimedia, 2008 ISBN 978-3-540-68126-7 Vol. 143. Uday K. Chakraborty (Ed.) Advances in Differential Evolution, 2008 ISBN 978-3-540-68827-3 Vol. 144. Andreas Fink and Franz Rothlauf (Eds.) Advances in Computational Intelligence in Transport, Logistics, and Supply Chain Management, 2008 ISBN 978-3-540-69024-5
Vol. 154. Oscar Castillo, Patricia Melin, Janusz Kacprzyk and Witold Pedrycz (Eds.) Soft Computing for Hybrid Intelligent Systems, 2008 ISBN 978-3-540-70811-7 Vol. 155. Hamid R. Tizhoosh and M. Ventresca (Eds.) Oppositional Concepts in Computational Intelligence, 2008 ISBN 978-3-540-70826-1 Vol. 156. Dawn E. Holmes and Lakhmi C. Jain (Eds.) Innovations in Bayesian Networks, 2008 ISBN 978-3-540-85065-6 Vol. 157. Ying-ping Chen and Meng-Hiot Lim (Eds.) Linkage in Evolutionary Computation, 2008 ISBN 978-3-540-85067-0
Ying-ping Chen Meng-Hiot Lim (Eds.)
Linkage in Evolutionary Computation
123
Ying-ping Chen Department of Computer Science National Chiao Tung University 1001 Ta Hsueh Road HsinChu City 300 Taiwan Email:
[email protected]
Meng-Hiot Lim School of Electrical & Electronic Engineering, Block S1 Nanyang Technological University Singapore 639798 Singapore Email:
[email protected]
ISBN 978-3-540-85067-0
e-ISBN 978-3-540-85068-7
DOI 10.1007/978-3-540-85068-7 Studies in Computational Intelligence
ISSN 1860949X
Library of Congress Control Number: 2008931425 c 2008 Springer-Verlag Berlin Heidelberg This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typeset & Cover Design: Scientific Publishing Services Pvt. Ltd., Chennai, India. Printed in acid-free paper 987654321 springer.com
Preface
For the past three decades, genetic and evolutionary algorithms (GEAs) have gradually established a strong foothold as powerful search methods and have been widely applied to solve problems in many disciplines. In order to improve the performance and applicability, numerous sophisticated mechanisms have been introduced and integrated into GEAs. One major category of these enhancing mechanisms is the concept of linkage. The motivation for linkage is to model the relationships or interactions between the decision variables, similar to that of genetic linkage observed in biological systems, through linkage learning techniques. Linkage learning connects the computational optimization methodologies and the natural evolution mechanisms. Not only can learning and adapting natural mechanisms enable us to design better computational methodologies, but also the insight gained by observing and analyzing the algorithmic behavior permits us to further understand biological systems, based on which GEAs are developed. In recent years, the issue of linkage in GEAs has garnered greater attention and recognition from researchers. Conventional approaches that rely much on ad hoc tweaking of parameters to control the search by balancing the level of exploitation and exploration are grossly inadequate. As shown in the work reported here, such parameters tweaking based approaches have their limits; they can be easily ”fooled” by cases of triviality or peculiarity of the class of problems that the algorithms are designed to handle. Furthermore, these approaches are usually blind to the interactions between the decision variables, thereby disrupting the partial solutions that are being built up along the way. The whole volume consisting of 19 chapters is divided into 3 parts: (i) Models and Theories; (ii) Operators and Frameworks; (iii) Applications. Part I consists of 7 chapters, dealing primarily with the theoretical foundations associated to linkage. Most of the chapters in this part focus on establishing models for formulating the linkage problem. One clear trend that emerged from these chapters is the general acceptance of estimation of distribution algorithms (EDAs) as a viable technique for linkage learning. However, the issue of computational complexity associated to EDAs needs to be addressed. The next 7 chapters which are
VI
Preface
grouped under Part II discuss issues pertaining to search operators and computational frameworks to facilitate linkage learning. The needs for operators that are designed to minimize the disruptions of good partial solutions are emphasized in these chapters. Along with this, the frameworks for extracting and analyzing solutions need to be in place so that learning can take place. Part III consists of five chapters, presenting applications that incorporate schemes for analyzing intermediate solutions so that linkage based learning can be incorporated into problem-solving. These are all interesting applications; one of the chapters incorporating the aspect of linkage learning in the design synthesis of MEMS structure. We feel that more work on applying linkage to real-world problems should be encouraged and our hope is for this edited volume to be a significant step in that direction. We hope that this edited volume will serve as a useful guide and reference for researchers who are currently working in the area of linkage. For postgraduate research students, this volume will serve as a good source of reference. It is also suitable as a text for a graduate level course focusing on linkage issues. Although it is unlikely to be complete even when confined within the scope of evolutionary linkage, it is reasonably comprehensive; the collection of chapters can quickly expose practitioners to most of the important issues pertaining to linkage. For practitioners who are looking at putting into practice the concept of linkage, the few chapters on applications will serve as a useful guide. We are extremely fortunate and honored to have a list of distinguished contributors who are willing to share their findings and expertise in this edited volume. For this, we are truly grateful. Besides being cooperative and accommodating, the authors have done a marvelous job, judging from the high quality manuscripts submitted.
Taiwan & Singapore, May 2008
Ying-ping Chen Meng-Hiot Lim
Contents
Part I: Models and Theories Parallel Bivariate Marginal Distribution Algorithm with Probability Model Migration Josef Schwarz, Jiri Jaros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
Linkages Detection in Histogram-Based Estimation of Distribution Algorithm Nan Ding, Shude Zhou . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25
Linkage in Island Models Zbigniew Skolicki . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
41
Real-Coded ECGA for Solving Decomposable Real-Valued Optimization Problems Minqiang Li, David E. Goldberg, Kumara Sastry, Tian-Li Yu . . . . . . . . . . .
61
Linkage Learning Accuracy in the Bayesian Optimization Algorithm Claudio F. Lima, Martin Pelikan, David E. Goldberg, Fernando G. Lobo, Kumara Sastry, Mark Hauschild . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
87
The Impact of Exact Probabilistic Learning Algorithms in EDAs Based on Bayesian Networks Carlos Echegoyen, Roberto Santana, Jose A. Lozano, Pedro Larra˜ naga . . . 109 Linkage Learning in Estimation of Distribution Algorithms David Coffin, Robert E. Smith . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Part II: Operators and Frameworks Parallel GEAs with Linkage Analysis over Grid Asim Munawar, Mohamed Wahib, Masaharu Munetomo, Kiyoshi Akama . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
VIII
Contents
Identification and Exploitation of Linkage by Means of Alternative Splicing Philipp Rohlfshagen, John A. Bullinaria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 A Clustering-Based Approach for Linkage Learning Applied to Multimodal Optimization Leonardo Emmendorfer, Aurora Pozo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 Studying the Effects of Dual Coding on the Adaptation of Representation for Linkage in Evolutionary Algorithms Maroun Bercachi, Philippe Collard, Manuel Clergue, Sebastien Verel . . . . . 249 Symbiotic Evolution to Avoid Linkage Problem Ramin Halavati, Saeed Bagheri Shouraki . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 EpiSwarm, a Swarm-Based System for Investigating Genetic Epistasis Thomas Goth, Chia-Ti Tsai, Fu-Tien Chiang, Clare Bates Congdon . . . . . 315 Real-Coded Extended Compact Genetic Algorithm Based on Mixtures of Models Pier Luca Lanzi, Luigi Nichetti, Kumara Sastry, Davide Voltini, David E. Goldberg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
Part III: Applications Genetic Algorithms for the Airport Gate Assignment: Linkage, Representation and Uniform Crossover Xiao-Bing Hu, Ezequiel Di Paolo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 A Decomposed Approach for the Minimum Interference Frequency Assignment Gualtiero Colombo, Stuart M. Allen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389 Set Representation and Multi-parent Learning within an Evolutionary Algorithm for Optimal Design of Trusses Amitay Isaacs, Tapabrata Ray, Warren Smith . . . . . . . . . . . . . . . . . . . . . . . . 419 A Network Design Problem by a GA with Linkage Identification and Recombination for Overlapping Building Blocks Miwako Tsuji, Masaharu Munetomo, Kiyoshi Akama . . . . . . . . . . . . . . . . . . 441 Knowledge-Based Evolutionary Linkage in MEMS Design Synthesis Corie L. Cobb, Ying Zhang, Alice M. Agogino, Jennifer Mangold . . . . . . . . 461 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485 Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487
List of Contributors
Alice M. Agogino University of California, Berkeley United States
[email protected]
Manuel Clergue Universite de Nice-Sophia Antipolis France
[email protected]
Kiyoshi Akama Hokkaido University Japan
[email protected]
Corie L. Cobb University of California, Berkeley United States
[email protected]
Stuart M. Allen Cardiff University United Kingdom
[email protected]
David Coffin University College London United Kingdom
[email protected]
Maroun Bercachi Universite de Nice-Sophia Antipolis France
[email protected]
Philippe Collard Universite de Nice-Sophia Antipolis France
[email protected]
John A. Bullinaria University of Birmingham United Kingdom
[email protected]
Gualtiero Colombo Cardiff University United Kingdom
[email protected]
Fu-Tien Chiang National Taiwan University Hospital Taiwan
[email protected]
Clare Bates Congdon University of Southern Maine United States
[email protected]
X
List of Contributors
Nan Ding Tsinghua University China
[email protected]
Jiri Jaros Brno University of Technology Czech Republic
[email protected]
Carlos Echegoyen University of the Basque Country Spain
[email protected]
Pier Luca Lanzi Politecnico di Milano Italy
[email protected]
Leonardo Emmendorfer University of Algarve Portugal
[email protected]
Pedro Larra˜ naga Technical University of Madrid Spain
[email protected]
David E. Goldberg University of Illinois at UrbanaChampaign United States
[email protected]
Minqiang Li Tianjin University China
[email protected]
Thomas Goth Colby College United States
[email protected]
Claudio F. Lima University of Algarve Portugal
[email protected]
Ramin Halavati Sharif University of Technology Iran
[email protected]
Fernando G. Lobo University of Algarve Portugal
[email protected]
Mark Hauschild University of Missouri at St. Louis United States
[email protected] Xiao-Bing Hu University of Sussex United Kingdom
[email protected] Amitay Isaacs University of New South Wales Australian Defence Force Academy Australia
[email protected]
Jose A. Lozano University of the Basque Country Spain
[email protected] Jennifer Mangold University of California, Berkeley United States
[email protected] Asim Munawar Hokkaido University Japan
[email protected]
List of Contributors
Masaharu Munetomo Hokkaido University Japan
[email protected]
Josef Schwarz Brno University of Technology Czech Republic
[email protected]
Luigi Nichetti Politecnico di Milano Italy
Saeed Bagheri Shouraki Sharif University of Technology Iran
[email protected]
Ezequiel Di Paolo University of Sussex United Kingdom
[email protected] Martin Pelikan University of Missouri at St. Louis United States
[email protected] Aurora Pozo Federal University of Paran´ a Brazil
[email protected] Tapabrata Ray University of New South Wales Australian Defence Force Academy Australia
[email protected] Philipp Rohlfshagen University of Birmingham United Kingdom
[email protected] Roberto Santana University of the Basque Country Spain
[email protected] Kumara Sastry University of Illinois at UrbanaChampaign United States
[email protected]
XI
Zbigniew Skolicki Google, Inc. United States
[email protected] Robert E. Smith University College London United Kingdom
[email protected] Warren Smith University of New South Wales Australian Defence Force Academy Australia
[email protected] Chia-Ti Tsai National Taiwan University Hospital Taiwan
[email protected] Miwako Tsuji Hokkaido University Japan m
[email protected] Sebastien Verel Universite de Nice-Sophia Antipolis France
[email protected] Davide Voltini Politecnico di Milano Italy
XII
List of Contributors
Mohamed Wahib Hokkaido University Japan
[email protected]
Ying Zhang Georgia Institute of Technology United States
[email protected]
Tian-Li Yu University of Illinois at UrbanaChampaign United States
[email protected]
Shude Zhou China Academy of Electronics and Information Technology China
[email protected]
Parallel Bivariate Marginal Distribution Algorithm with Probability Model Migration Josef Schwarz and Jiri Jaros Department of Computer Systems, Faculty of Information Technology, Brno University of Technology, CZ {schwarz,jarosjir}@fit.vutbr.cz
Summary. This chapter presents a new concept of parallel Bivariate Marginal Distribution Algorithm (BMDA) using the stepping stone communication model with the unidirectional ring topology. The traditional migration of individuals is compared with a newly proposed technique of probability model migration. The idea of the new adaptive BMDA (aBMDA) algorithms is to modify the classic learning of the probability model (applied in the sequential BMDA [24]). In the proposed strategy, the adaptive learning of the resident probability model is used. The evaluation of pair dependency, using Pearson’s chi-square statistics is influenced by the relevant immigrant pair dependency according to the quality of resident and immigrant subpopulation. Experimental results show that the proposed aBMDA significantly outperforms the traditional concept of migration of individuals.
1 Introduction The concept of traditional parallel genetic algorithm (PGA) is well known. It stems from the idea that the large problem can be successfully solved using decomposition of the original problem into smaller tasks. Consequently, the tasks can be solved simultaneously using multiple processors. This divide-and-conquer technique can be applied to GA in many distinct ways. Mostly, the population is divided into a few subpopulations or demes, and each of these demes evolves separately on different processors. Exchange of information among subpopulations is possible via a migration operator. In this context, the term island model is commonly used. Island populations are free to converge toward different sub-optima. The migration operator is supposed to mix good features that emerge locally in the different demes. Many topologies can be defined for connecting the demes, like mesh, torus, hypercube or ring. The most common models are the island model and the stepping stones model. In the basic island model, migration can occur between any subpopulations, whereas in the stepping stone model, migration is restricted to neighboring demes. In [7], the theory is published providing rational decisions for the proper setting of control parameters. An interesting survey of PGA is published in [2]. An effective technique for the massive parallelization of compact GA was published in [15]. An extremely prestigious PGA which is capable of solving billion-variable optimization problems was recently published in [10]. Y.-p. Chen, M.-H. Lim (Eds.): Linkage in Evolutionary Computation, SCI 157, pp. 3–23, 2008. © Springer-Verlag Berlin Heidelberg 2008 springerlink.com
4
J. Schwarz and J. Jaros
This chapter concerns the application of the stepping stone model (for simplicity we will use the term island-based model) for bivariate marginal distribution algorithm BMDA. This new approach using probability model migration is conceptually different from the traditional parallel genetic algorithms with migration of individuals/solutions and also from the EDAs using parallel building of pseudo-sequential probabilistic models. The remaining sections are organized as follows: Section 2 introduces the basic concept of EDA algorithm and current techniques used in the parallelization of the EDA algorithms. In Section 3 the sequential BMDA is described including the factorization and graphical representation of the probability model. Section 4 presents the motivation and a new idea of learning the probability model using a concept of probability model migration. Experimental results are shown in Section 5, Section 6 concludes the chapter.
2 Traditional EDAs EDAs belong to the class of advanced evolutionary algorithms based on the estimation and sampling of graphical probabilistic models [4, 5, 6, 11, 13, 22, 23, 26]. They do not suffer from the disruption of building blocks typical in standard genetic algorithms. The canonical sequential EDA is described in Fig. 1. EDAs often surpass classical EAs in the number of required fitness function evaluations. However, the absolute execution time is still a limiting factor which determines the size of practically tractable problems. Referring to Fig. 1 the most time-consuming task is the estimation of probability model for many problems. Most papers on EDAs concentrate on parallel construction and sampling of probabilistic models. The well-known algorithms employing parallel construction of Bayesian network are published in [17, 20, 21]. In [25], the theory of population sizing and timing to convergence is published. A new idea of the multideme parallel estimation of distribution algorithm (PEDAs) based on PBIL algorithm was published in [1]. In [16], mixtures of distribution with
Set t ĸ 0; Generate initial population D(0); While termination criteria is false do begin Select a set of promising solution Ds(t); Construct a new probability model M from Ds(t) using chosen metric; Sample offspring O(t) from M; Evaluate O(t); Create D(t+1) as a subset of O(t) ∪ D(t) with cardinality N; t ĸ t + 1; end Fig. 1. The pseudo code of canonical EDA
Parallel BMDA with Probability Model Migration
5
Bayesian inference are discussed. Parallel learning of belief networks in large domains is investigated in [27]. Using the concept of PBIL algorithm [3, 12, 19], the classical phenomenon of migration in island-based EAs was carried over into probability distribution of EDAs. A new approach of probability vector crossover was implemented with very good performance. 2.1 Linkage Learning in EDA Algorithms In competent genetic algorithms, various sophisticated linkage learning techniques must be implemented to discover Building Blocks (BBs). In EDA algorithms, the linkage learning is automatically incorporated into a graphical probabilistic model. EDAs support an effective detection, mixing and reproduction of BBs, so that they are capable of solving complex optimization problems including deceptive problems. But it is important to select the probabilistic model with a proper complexity reflecting the fitness function complexity. We can recognize three categories of model complexity: without dependency (Univariate Marginal Distribution Algorithm), pairwise dependency (MIMIC, BMDA) and multivariate dependency (BOA, EBNA). 2.2 Migration of Probabilistic Parameters for UMDA The concept of migration of probabilistic parameters instead of individuals was firstly published in [8] where on UMDA platform the convex combination of univariate probability models is investigated for various network topologies (ring, star etc.). Further enhancement of this concept is described in [9] where the local search methods are used to identify which parts of the immigrant model can improve the resident model. In the following sections we describe the proposal of a new concept of islandbased BMDA algorithm with unidirectional ring topology based on the combination of two adjacent bivariate probability models.
3 Sequential BMDA The well known representative of bivariate EDAs is the Bivariate Marginal Distribution Algorithm (BMDA) proposed by Pelikan and Mühlenbein [19, 24]. This algorithm uses a factorization of the joint probability distribution that exhibits second-order dependencies. EDAs are also population based algorithm but unlike GAs the new population is generated by sampling the recognized probability model. Let us denote: D = (X0, X1,..., XN-1) with X ∈ D, is the population of strings /solutions/individuals, X = (X0, X1,..., Xn-1) is a string/solution of length n with Xi as a variable, x = (x0, x1,..., xn-1) is a string/solution with xi as a possible instantiation of variable Xi, xi ∈{0,1}, p(X) = p(X0, X1,..., Xn-1) denotes the n dimensional probability distribution, p(x0, x1,..., xn-1) = p(X0 = x0, X1 = x1,..., Xn-1 = xn-1) denotes a probability of a concrete n dimensional vector.
6
J. Schwarz and J. Jaros
The probabilistic model used in BMDA can be formalized by M = (G, Θ), where G is a dependency graph and Θ = (θ0, θ2,…, θn-1) is a set of parameters which are estimated by local conditional or marginal probability for each node/variable of the dependency graph. A greedy algorithm for building dependency graphs is used. At the beginning, the root node is selected and subsequently the nodes with maximum dependency value are searched among the remaining nodes and joined. These pairwise dependencies in BMDA are discovered by Pearson’s chi-square statistics:
⎞ ⎟ 1 − ∑ ∑ ⎜ ∀x ∈Dom( X ) ∀x ∈Dom( X ) m( xi )m( x j ) ⎟⎟ i j j ⎠ ⎝ i ⎛
m 2 ( xi , x j )
χ i2, j = N ⎜⎜
(1)
where N is the size of parent population and m(xi, xj), m(xi) resp. m(xj) denote the number of individuals in the parent population with concrete values of xi and/or xj. These values are stored in the contingency tables. From the theoretical point of view this metric can be seen as statistical testing of hypothesis – for example binary genes Xi and Xj are considered to be independent at 95 percent confidence level if χ i2, j < 3.84 . Like COMIT, BMDA also uses a variant of minimum spanning tree technique to learn a model. However, during the tree construction, if none of the remaining variables can be “rooted” to existing tree, BMDA starts to form additional tree from remaining variables. The final probability distribution is thus a forest distribution (a set of mutually independent dependency trees):
p( X ) =
∏ p ( X r ) ∏ p( X i | X j ( i ) )
X r ∈R
(2)
X i ∈V \ R
where V is the set of nodes of dependency tree, R is the set of root nodes and Xj(i) denotes the parent node of Xi. Given the tree dependency structure, the univariate marginal probability distributions are estimated from the promising/parent population: p( X i = 1) =
m( X i = 1) N
(3)
and the bivariate conditional probability distributions p( X i | X j (i ) ) are estimated as
p( xi | x j(i ) ) =
m( xi , x j(i ) )
(4)
m( x j(i ) )
For example, the joint probability distribution for the dependency graph in Fig. 2 can be expressed by the factorization: 1. p(X) = p(X4) p(X3 |X4) p(X2 |X3) p(X1 |X2) p(X0|X1) 2. p(X) = p(X2) p(X3 |X2) p(X0 |X4) p(X4) p(X1 |X4) The time complexity of the complete BMDA algorithm can be expressed by the formula: O(n3)+O(4Nn2)+O(Nn), where the first component is a cubical time
Parallel BMDA with Probability Model Migration
3
4
7
4
3
0
0
2
1
a)
2
1
b)
Fig. 2. Example of dependency graph for: a) COMIT, b) BMDA
complexity of the dependency graph construction, the second component is a quadratic time complexity of contingency tables collection and the third component of the formula reflects a linear complexity of new solution sampling.
4 Island-Based BMDA 4.1 Migration of Individuals
In traditional island-based PGA algorithms, an infrequent migration of individuals among subpopulation is incorporated. The migration process is controlled by several parameters. It is necessary to determine the number and size of the subpopulations, the frequency and the intensity of migration and the method used for selection of candidate migrants. By analogy, it is possible to build island-based parallel BMDA, whereas the GA demes are replaced by BMDA ones. In BMDA and generally in EDAs, as it is known, new individuals are generated by the sampling of the probabilistic model. Consequently, a question pops up, whether it is possible to replace the migration of individual just by the probability model transfer. This topic is investigated in the next subsection. 4.2 Migration of the Probabilistic Model
The principal motivation for the proposal of a new concept of BMDA parallelization is to discover the efficiency of the transfer of probabilistic parameters in comparison with the traditional transfer of individuals. The main goal is to find a robust computational tool for hard optimization problems. The present approaches recently published in [1, 8, 9] use a simpler probability model only (PBIL, UMDA). Consistent with the theoretical conclusion shown in [24] and on the basis of experimental works done in [17], we used the island-based communication model with unidirectional ring topology with synchronization, see Fig. 3. We have simulated the island-based system partly on a single processor computer and partly on a real parallel system composed of a cluster of eight Linux-based workstations. It is evident that we can simply decompose the migration process in the ring loop into pairwise interactions of two adjacent islands - one of them is considered to
8
J. Schwarz and J. Jaros
Immigrant island
Migration direction
Subpopulation
Resident island
Fig. 3. Ring topology of island-based BMDA
be a resident island specified by resident probabilistic model and the second one is considered to be an immigrant island which probabilistic model is transferred to participate on building up a new resident model after a predefined migration rate. We focused on the problem of how to compose the resident model with the incoming model belonging to the immigrant island. In general, the modification of the resident model by the immigrant model can be formalized by the adaptation rule [3, 19]:
M´R = β M R ° (1- β) MI ,
(5)
where operator ° can be for example, a summing operator and the coefficient β in the range <0, 1> specifies the influence of the immigrant model. 4.3 Adaptive Learning of Probabilistic Model
We applied the adaptive learning for the both parts of the probabilistic model M R = (GR , ΘR) – the dependence graph GR and the parameter set ΘR. The new dependency graph G´R is not built by the aggregation of the original graph GR and the incoming graph GI but by means of Pearson’s chi-square statistics:
χ i2, j = βχ i2R , j R + (1 − β ) χ i2I , j I
(6)
The new parameters Θ´R are calculated by the simple adaptation rule: Θ´R = β ΘR + (1–β)ΘI
(7)
The adaptation coefficient β is defined by the formula: ⎧ ⎪
FR
β = ⎨ FI + FR ⎪⎩ 0.9
if FI ≥ FR ,
(8)
otherwise
where FR represents the mean fitness value of the resident subpopulation and FI represents the mean fitness value of the immigrant subpopulation.
Parallel BMDA with Probability Model Migration
9
Procedure (Output: M’R, Input:SubPopI, SubPopR) Calculate FR for the resident subpopulation; Calculate FI for the immigrant subpopulation; Calculate ȕ: °
FR
β = ® FI + FR °¯ 0.9
if FI ≥ FR otherwise
For i=0 to n-1 do begin For j=0 to n-1 do begin Calculate χ i2R , j , χ i2I , j ; R I Store in Chisqr_Table[i,j]: χ i2, j = βχ i2R , j + (1 − β ) χ i2I , j R I end end Build the new dependency graphs G´R according Chisqr_Table; Calculate set of the parameters: ĬR(G´R) , ĬI(G´R) ; Learning of the parameters: Ĭ’R = ȕ ĬR + (1-ȕ)ĬI Store the new resident model: M´R = (G´R , Ĭ´R), Sample the adapted model M´R ; Replacement of SubPopR; end Fig. 4. Adaptive learning of the resident model
The major part of all experiments was implemented using this pseudo parallel version of the algorithm aBMDA, see Fig. 4. 4.4 Parallel Implementation on a Cluster of Workstations
In the parallel version of aBMDA, it is necessary to transfer some components of the probability model from the immigrant node to the resident one. In the proposed version, the contingency tables are transferred. The spatial complexity of all transported tables is O(4n2), where n is the cardinality of the solved problem. Seeing that chisquare is symmetric, and dependencies between the same variables irrelevant, the spatial complexity can be reduced to O(2(n2 – n)). In contrast to the probabilistic model migration, the migration of individuals used in iBMDA works with the spatial complexity O(nkN), where kN is the number of migrating individuals. Because the communication overhead in modern interconnection networks depends more strongly on the start-up latency of communication than on a transported message size, we can consider that the communication overhead will be nearly the same for both approaches. Moreover, using an asynchronous or non-blocking type of migration [14], the communication overhead could be simply overlapped. Our parallel implementation of aBMDA derives benefits from overlapping of communication and computation, based on non-blocking MPI [18] communication subroutines MPI_ISend, MPI_IRecv and MPI_Wait. The basic idea is shown in Fig. 5.
10
J. Schwarz and J. Jaros
The information exchange between the resident and the immigrant node begins with the initiation of receiving request. During the receiving procedure, the resident node can compute its contingency tables and the mean fitness value FR of the resident population. Next, all computed data are packed into a simple send buffer using standard C routine memcpy and sent using non-blocking communication to the neighbor node. The resident chisqr-table is computed from the resident contingency tables in the next step. Now, the resident node has to wait until the immigrant data are completely received. After finalization, the data from the immigrant node are unpacked from a receive buffer and the immigrant chisqr-table is computed. Now, the probabilistic model composition can be started. First, the resident and the immigrant chisqrtables are combined together using beta parameter to produce a new chsqri-table. A new dependency graph is created according to the information stored in the learned chisqr-table. Second, a set of parameters Θ’R are calculated using new dependency graph, and the original resident and the immigrant contingency tables. As a result the new probabilistic model M´R = (G´R, Θ´R), is determined.
Procedure MakeExhangeIslandInformation(); MPI_IRecv(Receive buffer); Calculate the mean fitness value FR of the resident island; Calculate resident contingency tables; Pack resident contingency tables and FR into a send buffer; MPI_ISend(Send buffer); Calculate Chisqr_Table_Resident[i,j] = χ i2R , j R MPI_Wait(Waiting for receiving finish); Unpack immigrant contingency tables and FI from a receive buffer; Calculate ȕ; Calculate the Chisqr_Table_Imigrant[i,j]= χ i2I , j ; I
Calculate items of the composed Chisqr_Table[i,j]: χi2, j = βχ i2 , j + (1 − β ) χi2 , j Build the new dependency graphs G´R according to new Chisqr_Table; Calculate set of the parameters: ĬR(G´R) , ĬI(G´R) using contingency tables ; Learning of the parameters: Ĭ´R = ȕ ĬR + (1-ȕ)ĬI Compose new resident model: M´R = (G´R , Ĭ´R) MPI_Wait(Waiting for sending finish); end R
R
I
I
Fig. 5. MPI communication between the resident and the immigrant node
The migration of individuals used in iBMDA can be realized in similar way. The computation of contingency tables and mean fitness value is simply replaced by the selection of individuals intended for the migration. In this case, only selected individuals are packed into a send buffer and transported to the neighbor node. Received solutions are then unpacked in the resident node and incorporated into resident population. Finally, a new population is created in the standard way.
Parallel BMDA with Probability Model Migration
11
Besides the described type of communication, the MPI_Gather [18] operation was employed after each generation. During this operation, all necessary information from all processing nodes are collected to compute global statistics including the global mean fitness value, the best global solution, etc.
5 Experimental Results In our experiments, we compared four different variants of the BMDA algorithm. The first group consists of two versions of parallel BMDA algorithm: 1. aBMDA, with adaptive learning of dependency graph. 2. iBMDA, with the migration of individuals. These two parallel BMDA algorithms work with 8 islands of subpopulations, each consisting of 256 individuals as a portion of the full population with 2048 individuals. The second group used for the comparison includes two classical variants of BMDA: 3. sBMDA, sequential BMDA, with full population of 2048 individuals (as the whole eight-island model). 4. oBMDA - sequential BMDA with reduced population consisting of 256 individuals (as in case of one island). The fixed subpopulation size has been used for the whole range of problem size. We have not wittingly used the possibility of the adaptation of the subpopulation size according to problem size as discussed in [25]. Our goal was to compare namely the parallel adaptive aBMDA version with traditional iBMDA version under limited resources (subpopulation size). The value of the population size for sBMDA is set to 2048 derived partially from our experience and from the experimental results published in [25] for the 3-Deceptive problem. In all BMDA variants, truncation-based selection strategy was used, i.e. all individuals were ordered by their fitness value and the better half was used for model building. The truncation-based replacement strategy was also used for the replacement operator, i.e. the new generated solutions (offspring) replace the worse half of the subpopulation. The probabilistic model is built in each generation. Frequency of the model migration or individual migration was even - once per five generations. In case of the algorithm with migrating individuals, the elitism is used, that is, 13 best individuals of the immigrant subpopulation (i.e. about k=5 percent of the subpopulation) replace the worse individuals of the resident subpopulation. First stop condition was met after 500 generations; the second condition was activated if there was no improvement in the interval of 50 generations. 5.1 Specification of Benchmarks
For our experimental study, four well known benchmarks with various complexity and known global optimum were used. The OneMax and TwoMax problems served as the basic benchmarks for the testing of their performance. The Quadratic problem represents the adequate benchmark that should be solvable just by any BMDA
12
J. Schwarz and J. Jaros
algorithm. The 3-Deceptive task belongs to the hard deceptive benchmark for BMDA and it is often used for the testing of BOA algorithms. n −1
OneMax: f OneMax ( x ) = ∑ x i
(9)
i =0
TwoMax: f TwoMax ( x ) =
n −1
∑x
i
−
i =0
n
n n + 2 2
−1
2 Quadratic: f Quadratic ( x ) = ∑ f 2 ( xπ ( 2i ) , xπ ( 2i +1) )
(10)
(11)
i=0
where f 2 (u, v ) = 0.9 − 0.9(u + v ) + 1.9uv n −1 3
3-Deceptive: f 3− Deceptive ( x) = ∑ f 3 ( xπ ( 3i ) + xπ ( 3i +1) + xπ (3i + 2 ) )
(12)
i =0
where
⎧ ⎪ ⎪ f 3 (u ) = ⎨ ⎪ ⎪⎩
0.9 0.8 0 1
if if if
u=0 u =1 u=2
otherwise
The four mentioned objective functions were used to form fitness functions (FF) without additional modification. We have tested four variants of BMDA using 30 independent runs. To have baseline to island based versions, we first tested the classic sequential BMDA (sBMDA) with ordinary population of 2048 individuals and the classical sequential BMDA with reduced population (oBMDA). The first metric is represented by the often used success rate of the global optimum discovery. The second metric is calculated as the average value of the best fitness function (FF) over 30 runs. The third metric is computed as the mean value of the number of correctly discovered buildings blocks over 30 runs. These metrics/statistics are discussed in the next sections. 5.2 OneMax Problem
The sBMDA and aBMDA algorithms succeeded in the whole range of the problem size, see Fig. 6. Classical iBMDA version produces comparative result only up to 260 variables. Consecutively, the rapid drop follows after this threshold. It is also evident that oBMDA was significantly outperformed by all the other algorithms. 5.3 TwoMax Problem
In the case of TwoMax problem, see Fig. 7, the results of the tested algorithms are similar to the results achieved for OneMax problem. The aBMDA version outperformed all other versions and achieved the same results as sBMDA. The drop in success rate for the iBMDA version with migration of individuals is more significant than in the case of OneMax problem.
Parallel BMDA with Probability Model Migration
13
Success rate for OneMax problem 100 90 80
Success rate [%]
70 60 50 40 30
aBMDA iBMDA
20
sBMDA oBMDA
10 0 20
40
60
80
100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400 420 440 460 480 500
Problem size [variables]
Fig. 6. Success rate for OneMax problem
Success rate for TwoMax problem 100 90 80
Success rate [%]
70 60 50 40 30
aBMDA iBMDA
20
sBMDA oBMDA
10 0 20
40
60
80
100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400 420 440 460 480 500
Problem size [variables]
Fig. 7. Success rate for TwoMax problem
5.4 Quadratic Problem
To achieve global solution for this problem, the probability model with the bivariate dependency is required. This benchmark is thus perfectly suitable for testing and comparing all BMDA variants. In Fig. 8, the success rate for all compared algorithms can be seen. The best results were reached by sBMDA that succeeded in the nearly whole range of the problem sizes. Similar behavior can be observed even for aBMDA version that achieved 100 percent success rate for up to 260 variables.
14
J. Schwarz and J. Jaros
Success rate for Quadratic problem 100 90 80
Success rate [%]
70 60 50 40 30 aBMDA 20
iBMDA sBMDA
10
oBMDA
0 20
40
60
80
100
120
140
160
180
200
220
240
260
280
300
Problem size [variables]
Fig. 8. Success rate for Quadratic problem Table 1. Statistics results (mean±std of FF) for Quadratic problem
Problem size aBMDA 60 30.00±0.00 80 40.00±0.00 100 50.00±0.00 120 60.00±0.00 140 70.00±0.00 260 130.00±0.00 280 139.92±0.24 300 149.75±0.46
iBMDA 30.00±0.00 39.98±0.30 49.87±0.80 59.67±0.13 67.71±3.14 127.10±0.37 136.53±0.30 146.09±0.27
Algorithm sBMDA 30.00±0.00 40.00±0.00 50.00±0.00 60.00±0.00 70.00±0.00 130.00±0.00 139.99±0.02 149.99±0.02
oBMDA 29.96±0.06 39.88±0.10 49.65±0.14 59.37±0.24 69.08±0.25 126.32±0.57 135.79±0.54 145.33±0.64
Optimum 30 40 50 60 70 130 140 150
Table 2. Statistics results (mean±std of BBs) for Quadratic problem
Problem size aBMDA 60 30.00±0.00 80 40.00±0.00 100 50.00±0.00 120 60.00±0.00 140 70.00±0.00 260 130.00±0.00 280 139.20±2.45 300 147.47±4.64
iBMDA 30.00±0.00 39.83±0.37 48.70±0.78 56.73±1.26 64.50±1.36 101.03±3.71 105.10±3.27 110.67±2.77
Algorithm sBMDA 30.00±0.00 40.00±0.00 50.00±0.00 60.00±0.00 70.00±0.00 130.00±0.00 140.00±0.00 149.90±0.25
oBMDA 29.56±0.61 38.87±1.02 46.50±1.41 53.80±2.48 60.80±2.38 93.47±5.86 98.03±5.15 103.60±4.93
Optimum 30 40 50 60 70 130 140 150
Parallel BMDA with Probability Model Migration
15
Besides the success rate metric the second metric represented by mean±std statistics of the fitness function are presented in Table 1. It is evident that aBMDA version and sBMDA version provide the same results up to 260 variables. For higher number of variables sBMDA achieves better results. The best value achieved for each problem size is written in bold. In Table 2, the third metric represented by mean±std statistics for the number of correctly recognized buildings blocks (BBs) are shown. 5.5 3-Deceptive Problem
The problem was investigated for the variable range from 21 to 120, see Fig. 9. For higher number of variables, the drop in success rate is significant for all proposed algorithms. It is caused by rather high complexity of the 3-Deceptive problem that requires a more complex model and also larger population size for efficient performance. The best success rate was achieved by aBMDA version. Very similar values were also achieved by sBMDA. On the other hand, the worst results were obtained by oBMDA and iBMDA.
Success rate for 3-Deceptive problem
100
aBMDA 90
iBMDA sBMDA
80
oBMDA
Success rate [%]
70 60 50 40 30 20 10 0 21
30
39
51
60
72
81
90
99
120
Problem size [variables]
Fig. 9. Success rate for 3-Deceptive problem
In Table 3, the mean±std statistics of the fitness function are presented. The best results were obtained by the aBMDA version and by sBMDA. The worst mean fitness values were achieved by oBMDA algorithm followed by iBMDA. The mean values and standard deviation of the discovered BBs are presented in Table 4. From Tables 3 and 4, a significant correlation between the mean value of fitness and the mean value of BBs in case of aBMDA version can be recognized. For the case of the 99–variable problem the mean number of BBs is 26.3 which is 80 percent of total 33 blocks. Note that iBMDA discovered only 11.9 BBs (36 percent). It is interesting to compare these values with the experimental results published for BOA algorithm in [25], where the achieved number of BBs for 99 variables and for the population size estimated to 250 equals to 25 percent.
16
J. Schwarz and J. Jaros Table 3. Statistics results (mean±std of FF) for 3-Deceptive problem
Problem size aBMDA 21 7.00±0.00 30 10.00±0.00 39 13.00±0.00 51 16.98±0.04 60 19.96±0.06 72 23.91±0.15 81 26.80±0.15 90 29.58±0.20 99 32.36±0.35 120 38.67±0.29
iBMDA 7.00±0.00 9.92±0.06 12.76±0.11 16.36±0.09 19.13±0.13 22.73±0.14 25.43±0.16 28.11±0.15 30.87±0.15 37.10±0.14
Algorithm sBMDA 7.00±0.00 10.00±0.00 13.00±0.00 16.99±0.03 19.96±0.06 23.90±0.09 26.83±0.10 29.78±0.16 32.63±0.18 39.21±0.2
oBMDA 6.90±0.05 9.75±0.12 12.06±1.99 16.12±0.16 18.78±0.22 22.41±0.24 25.07±0.23 27.84±0.57 30.51±0.22 38.58±0.59
Optimum 7 10 13 17 20 24 27 30 33 40
Table 4. Statistics results (mean±std of BBs) for 3-Deceptive problem
Problem size aBMDA 21 7.00±0.00 30 10.00±0.00 39 13.00±0.00 51 16.83±0.45 60 19.7±0.59 72 23.13±1.02 81 25.00±1.51 90 25.87±2.08 99 26.30±2.99 120 26.23±3.72
iBMDA 7.00±0.00 9.23±0.56 10.63±1.11 10.60±0.92 11.33±1.49 11.53±1.56 11.50±1.62 11.27±1.41 11.86±1.83 11.63±1.80
Algorithm sBMDA 7.00±0.00 10.00±0.00 13.00±0.00 16.90±0.39 19.56±0.56 23.00±0.93 25.33±0.98 27.83±1.61 29.36±1.85 32.10±2.09
oBMDA 6.47±0.50 7.60±1.11 7.33±1.72 8.27±1.67 7.23±2.39 8.23±2.29 7.92±2.39 8.50±2.26 8.67±2.95 9.03±1.74
Optimum 7 10 13 17 20 24 27 30 33 40
5.6 Discussion on Pseudo-parallel Version of Algorithms
In our experiments two groups of algorithms are compared: 1. The new proposed island-based aBMDA with probabilistic model learning and the traditional island-based iBMDA with individual migration. 2. Sequential sBMDA version with the full population size and reduced sequential oBMDA version. In the first experiment, the success rate metric was applied. Both aBMDA and sBMDA versions are capable of finding global optima with 100 percent success rate for up to 500 variables in the case of OneMax and TwoMax problems and up to 260 variables in the case of Quadratic problems. For difficult problems, like 3-Deceptive, the algorithms lack the ability to find repeatedly the optimal solution for problem size larger than 39. It is evident that aBMDA is effective optimization tool outperforming iBMDA version based on the traditional migration of individuals. From this point of view the range of solvable problem size is at least two times larger in case of aBMDA version.
Parallel BMDA with Probability Model Migration
17
In the second experiment the statistics including mean±std values of fitness function (FF) was processed for two harder problems – the Quadratic problem in Table 1 and 3-Deceptive problem in Table 3. The best values are written in bold. From Table 1, it is evident that for Quadratic problem, aBMDA and sBMDA have reached the global optima up to 260 variables and outperformed very significantly iBMDA version. In the case of 3-Deceptive problem, aBMDA outperformed all other algorithms besides the sBMDA that is better for the problem size exceeding 90 variables. Note that for the 120-variable problem the mean value of FF in case of aBMDA equals to 38.6 which is close to global optimum represented by value 40. In Table 2 and Table 4, the statistical results for BBs are shown for Quadratic and 3-Deceptive problems. From Table 2 it is evident, that in case of Quadratic problem, aBMDA discovered all BBs up to 260 variables while iBMDA was successful up to 30 variables only. In case of 3-Deceptive problems, see Table 4, aBMDA outperformed iBMDA in the whole range of problem size. Note that aBMDA achieved approximately two time higher mean value of BBs for the problem size exceeding 60 variables. The computational complexity of all algorithms measured by the number of generation is comparable. For example, in the case of Quadratic problem with 60 variables the average computational time is about 20 generations, see Fig. 10. Note that oBMDA was able to find the global optima for this instance of Quadratic problem only in 66 percent of the 30 runs, see Fig. 8. Time complexity of proposed algorithms (Quadratic 60) 20 19,5 19
Generations
18,5 18 17,5 17 16,5 16 15,5 15 aBMDA
iBMDA
sBMDA
oBMDA
Variants of algorithm
Fig. 10. Time complexity of the proposed algorithms for Quadratic problem
5.7 Performance of Parallel Implementation
The parallel implementations of aBMDA and iBMDA were tested on a cluster of 8 Linux-based workstations equipped with Intel E6550 processor and 2GB RAM connected together by 1Gb LAN network. In case of sequential sBMDA only one station was used.
18
J. Schwarz and J. Jaros
First, the comparison of the mean execution time TG related to one generation was performed. For TG calculation, five independent runs, each composed of 20 generation (including 4 migration cycles), were carried out. The algorithms were compared using OneMax benchmark, see Fig. 11. Let us note that the convergence toward global optima was not checked in this case. From Fig. 11, a marked difference between sequential and parallel approaches is evident, as it was expected. The values of TG for the both parallel algorithms are comparable. Finally, OneMax benchmark is relatively simple, as the fitness function evaluation does not influence the execution time very much. For more complex problems, a gap between sequential and parallel approaches will be deeper, because computational complexity will dominate communication complexity.
Execution time of one generation for OneMax problem
70 aBMDA iBMDA
60
sBMDA 50
T G [s]
40
30
20
10
0 20
40
60
80
100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400 420 440 460 480 500
Problem size [variables]
Fig. 11. The mean execution time TG for OneMax
Speed-up of one generation for OneMax problem 10 9 8
Speed-up
7 6 5 4 3 2 aBMDA 1
iBMDA
0 20
40
60
80
100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400 420 440 460 480 500
Problem size [variables]
Fig. 12. Speed-up of iBMDA and aBMDA with regard to sBMDA for OneMax problem
Parallel BMDA with Probability Model Migration
19
The speed-up of parallel implementations for OneMax problem is displayed in Fig. 12. It varies between 5 and 9 for both parallel algorithms. The decrease of the speed-up for larger instances of OneMax problem is caused by the increased parallelization overhead. The overhead consists of the quadratic time complexity of contingency tables transport and the subsequent model composition in the resident node. The performance of aBMDA algorithm is slightly better. For simpler instances (say up to 200 variables) the achieved speed-up was larger than a number of processing nodes (8 in our experiment) – it is known as the phenomenon of superlinear speed-up [3]. The extent of speed-up was also investigated for Quadratic problem, see Fig. 13. Both parallel algorithms achieved superlinear speed-up in the whole range of problem instances as the consequence of the more complex benchmark. Speed-up of one generation for Quadratic problem 11 10 9
Speed-up
8 7 6 5 4 3 aBMDA 2
iBMDA
1 20
40
60
80
100
120
140 160 180 Problem size [variables]
200
220
240
260
280
300
Fig. 13. Speed-up of iBMDA and aBMDA with regard to sBMDA for Quadratic problem
Let us note that the knowledge of the concrete speed-up can be utilized for prediction of the execution time of the optimization process and also for the setting of proper population size and the number of processing nodes. Finally, we investigated the speed-up of the optimization tasks which resulted in 100% success rate. The value of the speed-up was calculated using the following schema: a) first the number of generations required for achievement the global optima for each version of BMDA and for each instance of the problem was measured and averaged during 30 independent runs, b) this value was multiplied by the mean execution time of one generation TG for relevant version of BMDA, c) finally, this value was normalized according the values obtained by sequential sBMDA and plotted in the Fig. 14 and Fig. 15. From Fig. 14, it is evident that iBMDA algorithm was not able to find optimal solutions for the whole range of OneMax instances. It succeeded only in the range from 20 to 220 variables. aBMDA solved the tasks reliably with the speed-up higher than 6 and even with superlinear speed-up up to 280 variables.
20
J. Schwarz and J. Jaros
Speed-up for OneMax with 100% success rate
10 9 8 7
Speed-up
6 5 4 3 2 aBMDA 1
iBMDA
0 20
40
60
80
100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400 420 440 460 480 500
Problem size [variables]
Fig. 14. Speed-up of iBMDA and aBMDA with regard to sBMDA for OneMax problem in case of 100% success rate
Speed-up for Quadratic with 100% success rate 10 9 8
Speed-up
7 6 5 4 3 aBMDA 2
iBMDA
1 20
40
60
80
100
120
140
160
180
200
220
240
260
Problem size [v ariables]
Fig. 15. Speed-up of iBMDA and aBMDA with regard to sBMDA for Quadratic problem in case of 100% success rate
The capability of the speed-up for Quadratic problem (Fig. 15) was investigated for up to 260 variables. aBMDA was still able to produce optimal solutions with 100% success rate. The speed-up varied between 4.4 and 8.7. iBMDA algorithm achieved comparable speed-up as aBMDA but only up to 60 variables. 5.7.1 Discussion on Parallel Implementation The message passing interface MPI used for parallel implementation of BMDA algorithms appears as an efficient software tool. Two variants of parallel BMDA was investigated – aBMDA and iBMDA. They are mutually compared and also compared to the sequential sBMDA.
Parallel BMDA with Probability Model Migration
21
From Fig. 12 and Fig. 13 it is possible to conclude, that the communication overhead is negligible. The values of achieved speed-up of processing were very close to the number of utilized processing nodes, and even higher in some cases. The ability of finding global optima and its speed-up was also examined for both parallel algorithms tested. From Fig. 14 and Fig. 15, it is evident that aBMDA algorithm is much more robust than iBMDA. The speed-up of optimization process with respect to sBMDA (with 100 percent success rate) varies from 4.4 to 8.7, but for most of the problem instances it is higher than 6.
6 Conclusions This chapter has presented a new idea of parallel BMDA algorithms using islandbased model with the unidirectional ring topology. The cooperation of demes was realized via migration of probabilistic models instead of the traditional migration of individuals. Each two-neighbor demes are organized as a pair of resident and immigrant demes. We have introduced an adaptive learning technique, based on the quality of resident and immigrant subpopulation, which consists of the adaptation of the resident probabilistic model by the incoming neighbor immigrant model. We have introduced an efficient tool how to learn graphical probabilistic model and associated probabilistic parameters. The comparative experimental studies demonstrated that the proposed parallel algorithm aBMDA outperforms the traditional iBMDA using migration of individuals, mainly according to the problem size and the level of the success rate. This is true for all the applied benchmarks beginning with OneMax and ending with the Quadratic problem. The 3-Deceptive problem is a hard problem for all compared algorithms but aBMDA and sBMDA provide relatively best solutions. Note that the sequential sBMDA with full population provides competitive results compared with the aBMDA, but indeed the time complexity of aBMDA version can be significantly reduced by parallel processing. Future work will be focused on more sophisticated modifications of learning techniques with limited size of parameters transfer. We also aim to parallelize the Bayesian Optimization Algorithm (BOA) using a modified concept of probabilistic model migration.
Acknowledgement This work was partially supported by the Grant Agency of the Czech Republic under No.~102/07/0850 Design and hardware implementation of a patent-invention machine and the Research Plan No. MSM 0021630528 - Security-Oriented Research in Information Technology.
References 1. Ahn, C.W., Goldberg, D.E., Ramakrishna, R.S.: Multiple-Deme Parallel Estimation of Distribution Algorithms: Basic Framework and Application (Illigal Report No. 2003016) (2003) 2. Alba, E., Troya, J.M.: A Survay of Parallel Distributed Genetic Algorithms. Complexity 4, 303–346 (1999)
22
J. Schwarz and J. Jaros
3. Baluja, S.: Population-Based Incremental Learning: A method for Integrating Genetic Search Based Function Optimization and Competitive Learning, Technical report CMUCS-94-163, Carnegie Mellon University (1994) 4. Bosman, P.A.N., Thierens, D.: An Algorithmic Framework For Density Es-timation Based Evolutionary Algorithms, Technical report UU-CS-1999-46, Utrecht University (1999) 5. Bosman, P.A.N., Thierens, D.: Linkage information processing in distribution estimation algorithms. In: Proceedings of the Genetic and Evolutionary Computation Conference GECCO 1999, Orlando, Florida, vol. I, pp. 60–67. Morgan Kaufmann Publishers, San Fransisco (1999) 6. Bosman, P.A.N., Thierens, D.: Continuous iterated density estimation evolutionary algorithms within the IDEA framework. In: Proceedings of the Optimization by Building and Using Probabilistic Models OBUPM workshop. Genetic and Evolutionary Computation Conference GECCO 2000, pp. 197–200 (2000) 7. Cantu-Paz, E.: Efficient and Accurate Parallel genetic algorithm. Kluwer Academic Publishers, Dordrecht (2000) 8. delaOssa, L., Gámez, J.A., Puerta, J.M.: Migration of probability models instead of individuals: an alternative when applying the island model to edas. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., Merelo-Guervós, J.J., Bullinaria, J.A., Rowe, J.E., Tiňo, P., Kabán, A., Schwefel, H.-P. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 242–252. Springer, Heidelberg (2004) 9. delaOssa, L., Gámez, J.A., Puerta, J.M.: Improving model combination through local search in parallel univariate EDAs. Evolutionary Computation 2, 1426–1433 (2005) 10. Goldberg, D.E., Sastry, K., Llorà, X.: Toward routine billion-variable optimization using genetic algorithms. Complexity 12, 27–29 (2007) 11. Heckerman, D., Geiger, D., Chickering, M.: Learning Bayesian networks: The combination of knowledge and statistical data, Technical Report MSR-TR-94-09, Microsoft Research, Redmond, WA (1994) 12. Höhfeld, M., Rudolph, G.: Towards a theory of population-based incremental learning. In: Proceedings of 4th International Conference on Evolutionary Computation, pp. 1–5. IEEE Press, Los Alamitos (1997) 13. Larrañaga, P., Lozano, J.A.: Estimation of Distribution Algorithms. Kluwer Academic Publishers, London (2002) 14. Lin, S.C., Punch, W.F., Goodman, E.D.: Coarse-grain parallel genetic algorithms: categorization and a new approach. In: Sixth IEEE Conference on Parallel and Distributed Processing, pp. 28–37. IEEE Press, Piscataway (1994) 15. Lobo, F.G., Lima, C.F., Martires, H.: Massive parallelization of the compact genetic algorithm. In: Ribeiro, R. (ed.) Proceedings of the International Conference on Adaptive and Natural Computing Algorithms (ICANNGA 2005), pp. 530–533. Springer, Heidelberg (2005) 16. Marin, J.M., Mengersen, K., Robert, C.P.: Bayesian Modelling and Inference on Mixtures of Distribution. In: Dey, D., Rao, C.R. (eds.) Handbook of Statistics, vol. 25, pp. 459–507 (2005) 17. Mendiburu-Alberro, A.: Parallel implementation of estimation of Distribution Algorithms based on probabilistic graphical models. Application to chemical calibration models, PhD thesis, the University of Bascue Country, Donostia-San Sebastian (2006) 18. MPI-2 Extension to message-Passing Interface, Message Passing Interface Forum (2003), http://www.mpi-forum.org/docs/mpi2-report.pdf 19. Mühlenbein, H.: The equation for response to selection and its use for prediction. Evolutionary Computation 5(3), 303–346 (1997)
Parallel BMDA with Probability Model Migration
23
20. Ocenasek, J.: Parallel Estimation of Distribution Algorithms, PhD. Thesis, Faculty of Information Technology, Brno University of Technology, Brno, Czech Rep. (2002) 21. Ocenasek, J., Schwarz, J., Pelikan, M.: Design of Multithreaded Estimation of Distribution Algorithms. In: Cantú-Paz, E., Foster, J.A., Deb, K., Davis, L., Roy, R., O’Reilly, U.-M., Beyer, H.-G., Kendall, G., Wilson, S.W., Harman, M., Wegener, J., Dasgupta, D., Potter, M.A., Schultz, A., Dowsland, K.A., Jonoska, N., Miller, J., Standish, R.K. (eds.) GECCO 2003. LNCS, vol. 2724, pp. 1247–1258. Springer, Heidelberg (2003) 22. Pelikan, M., Goldberg, D.E., Cantú-Paz, E.: Linkage problem, distribution estimation, and Bayesian networks, IlliGAL Report No. 98013, University of Illinois at UrbanaChampaign, Illinois Genetic Algorithms Laboratory, Urbana, Illinois (1998) 23. Pelikan, M., Goldberg, D.E., Lobo, F.: A survey of optimization by building and using probabilistic models, IlliGAL Report No. 99018, University of Illinois at UrbanaChampaign, Illinois Genetic Algorithms Laboratory, Urbana, Illinois (1999) 24. Pelikan, M., Mühlenbein, H.: The bivariate marginal distribution algorithm. In: Advances in Soft Computing - Engineering Design and Manufacturing, pp. 521–535. Springer, London (1999) 25. Pelikan, M., Sastry, K., Cantú-Paz, E.: Bayesian Optimization Algorithm, Population Sizing and Time to Convergence, IlliGAL Report No. 2000001, Il-linois Genetic Algorithm Laboratory, University of Illinois at Urbana-Champaing, p. 13 (2000) 26. Pelikan, M., Sastry, K., Goldberg, D.E.: Evolutionary Algorithms + Graphical Models = Scalable Black-Box Optimization, IlliGAL Report No. 2001029, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, Illinois (2001) 27. Xiang, Y., Chu, T.: Parallel Learning of Belief Networks in Large and Difficult Domains. In: Data Mining and Knowledge Discovery, vol. 3, pp. 315–339 (1999)
Linkages Detection in Histogram-Based Estimation of Distribution Algorithm Nan Ding1 and Shude Zhou2 1
Department of Electronic Engineering, Tsinghua University, Beijing, 100084 China 2 China Academy of Electronics and Information Technology, Beijing, 100041 China
Abstract. In this chapter, we review two methods that deal with the linkage detection in the Histogram-based Estimation of Distribution Algorithms; one is based on probabilistic graphical models, and the other is based on space transformation. The two methods deal with the linkage in the optimization problem with different accuracy and with different computational complexity. Probabilistic graphical model is generally more accurate but always associated to high-cost, while transformation is the opposite. In the following, we will mainly discuss the way to reduce the complexity of the method based on probabilistic graphical model and the way to obtain the transformation which captures the dominant linkage of the problem.
1 Introduction Estimation of distribution algorithms (EDAs) are a class of evolutionary algorithms that use probabilistic model of promising solutions found so far to obtain new candidate solutions of optimized problems. One of the charming characteristics of this evolutionary paradigm is that the joint probability density can explicitly represent correlation among variables [1,16]. It is verified by several researchers [2] that multivariate-dependency EDAs have the potential ability to optimize hard problems with strong nonlinearity. However, it is also noted that how to efficiently learn the complex probabilistic model is a bottleneck problem. Therefore, obtaining a good balance between the complication of probabilistic models and the efficiency of learning method is a key factor for designing new EDAs. In continuous domain, the predominant probabilistic model applied by EDAs is based on Gaussian probability distribution. Continuous EDAs based on multivariate Gaussian distribution have polynomial computational complexity. However, the inherent shortcoming of Gaussian-based EDA is that the unimodal model is too rough and thus is likely to mislead the search to a local optimum when solving complex optimization problems. Although clustering techniques such as Gaussian mixture model, Gaussian kernel model are considered to conquer this shortcoming in the literature, their complicated probabilistic distributions make it more difficult to estimate the linkage information, and thus the computational complexity increases remarkably. Histogram model is another alternative probabilistic model in the continuous EDA. It is also called a kind of discretization of the continuous problem. In comparison with Y.-p. Chen, M.-H. Lim (Eds.): Linkage in Evolutionary Computation, SCI 157, pp. 25–40, 2008. © Springer-Verlag Berlin Heidelberg 2008 springerlink.com
26
N. Ding and S. Zhou
unimodal Gaussian model, histogram probabilistic model is able to represent multiple local optima by bins of different heights. Histogram model has already been used in previous work [3-11,13,14]. For example, marginal histogram models are applied in the FWH [7] by S. Tsutsui et al. and the histogram-based EDA (HEDA) [8,9] by N. Ding et al. B. Yuan et al. [11] also proposed the HEDA as an extension of the PBIL [12]. Q. Zhang et al. introduce EDA/L [14] in which several local search strategies are employed in a marginal histogram model. In those above algorithms, the complete probability is approximated by the product of the marginal probability of each variable, that is to say, the linkages of the variables are discarded. However, as we know, when optimizing problems with bounded epistasis, the linkage information should be given prior consideration in the process of the evolutionary algorithms. The IDEA [4,5] based on histogram model (IDEA-H) by P.A.N. Bosman et al. used the multivariate histogram model to consider the variable linkage, but they also remarked that the complexity of the IDEA-H grows exponentially when expressing joint probability of multiple random variables. The aim of the chapter is to conquer the linkage problem of HEDA from two aspects: one is based on probabilistic graphical models (PGM), where the multivariate histogram model is considered; and the other is based on space transformation, where the marginal distribution is built in the transformed space. PGM takes the Markov properties into account, where the variable is independent of each other when given its neighbors in the graphical models, thus it avoids estimating the complete probability. Space transformation works under the assumption that a decent transformation from the original space may cancel out some of the dominant linkages among the variables. The two methods deal with the variables linkages of the optimization problem with different accuracy and with different computational complexity. Probabilistic graphical model is generally more accurate but always associated to high-cost, while transformation is the opposite. We also want to acknowledge that P. Pošík in [18] had a general introduction to the real-valued evolutionary algorithm on the use of probabilistic model and coordinate transform, which is really helpful to our work. The chapter here would mostly focus on the Histogram-based EDA and would contain the specific concerns about the histogram model. This chapter is organized as follows. Section 2 briefly reviews the HEDA, especially the marginal HEDA. In Section 3, HEDA based on probabilistic graphical models is introduced and especially we discuss how to reduce its computational complexity. Section 4 is about HEDA based on space transformation.
2 Histogram-Based Estimation of Distribution Algorithm and Its Marginal Case The histogram-based estimation of distribution algorithm (HEDA) has a main framework as follow: 1. Initialize the histogram model 2. Generate population P(t ) by sampling on the histogram model. 3. Evaluate and rank the population P(t ) .
Linkages Detection in Histogram-Based Estimation of Distribution Algorithm
27
4. Update the histogram model according to the selected individuals P '(t ) . 5. Return to step 2 if not terminated. The marginal histogram model has a general form of l −1
P( Z 0 ,..., Z l −1 ) = ∏ P( Zi )
(1)
i=0
In Eq.(1), each P( Z i ) (i = 0,..., l − 1) denotes a 1-variate histogram model. The core of marginal histogram-based estimation of distribution algorithm is to estimate the marginal distributions of P( Z i ) (i = 0,..., l − 1) and then to generate new individuals by sampling on P( Z i ) for each variable. The FWH [7] and the sur-shrHEDA [9] both belong to this class of HEDA. We now have a brief review of these two algorithms. In the FWH, the height of each bin in each variable Zi is proportional to the count of the selected individuals in it. To sample each variable of the new individual, firstly a bin is sampled according to the heights of the bins of the variable, and then the real value of that variable of the new individual is uniformly sampled in the domain of the bin. Since the height of the bin can be normalized to be equal to the probability of the bin, in the later description we will no longer differentiate the two phrases “the height of the bin” and “the probability of the bin”. In the sur-shr-HEDA, two specific strategies for HEDA, the surrounding effect and the shrinking strategy were developed to conquer the two drawbacks of the HEDA. The initial population should be large enough to sample the variables with a lot of bins; otherwise, many bins will never get a chance to be sampled. The solution accuracy is greatly influenced by the width of bins and highly accurate solutions can only be achieved by setting enough number of bins. With surrounding effect, if an individual in a certain bin No.i is selected, not only the No.i bin gets an improvement on its height, but the surrounded bins, i.e. the No.(i+1) and No.(i-1) bins, also get the minor improvements (always the height of its surrounded bin times the surrounding factor) on their heights respectively. Using the surrounding effect, those bins with heights of zero have the opportunity to be sampled. Furthermore, it has been shown in our previous work that the HEDA with surrounding effect can find the best bins near the current sampled bins with hill-climbing during the search process. That ensures the algorithm to find the optimal bin with small population, even when the number of bins is large. For the shrinking strategy, if the height of the highest bin of a variable is over the threshold value, the domain of that variable will shrink to the domain of that highest bin, and the new domain will be divided into bins as the initial step of the algorithm. Since the searching space gradually shrinks, using the shrinking strategy will make the solution accurate enough. Experimental results have already shown that the HEDA combining both the surrounding effect and the shrinking strategy performs excellently in continuous optimization, especially, in those problems with multiple local optima. [9] Marginal histogram-based estimation of distribution algorithm is frequently applied in practice for its simplicity and efficiency. However, the obvious drawback of the marginal probability estimation is that it loses the ability to detect any variable
28
N. Ding and S. Zhou
linkages of the problem. This drawback is serious in the problems that variables are strongly dependent with each other. In the following, we supply two solutions for this problem. The first is by applying probabilistic graphical model; the second is by applying a space transformation.
3 HEDA Based on Probabilistic Graphical Models Applying probabilistic graphical models (PGM) in HEDA was first put forward by P.A.N. Bosman in his IDEA-H [4] (If we regard HEDA as a special case of discrete EDAs, the history of applying probabilistic graphical model is even longer, for example, see [3].) In the analysis of the IEDA-H, Bosman regarded that the computational complexity of the IDEA-H grows exponentially with the maximum number of the variables that a variable conditionally depends on, which becomes the bottleneck of the IDEA-H. Recently, an accelerated algorithm, which is called the dHEDA, with polynomial complexity dictated by the size of the population, was proposed in [10]. Since the computational complexity is the main concern for HEDA based on PGM, we will discuss the dHEDA in details. 3.1 General Framework In general, the HEDA based on PGM share the same framework as any other HEDAs, except that it iteratively updates and samples from P( Z 0 ,..., Z l −1 ) under the assumption that: l −2
P( Z 0 ,..., Z l −1 ) = P( Z jl −1 )i∏ P( Z ji | Zπ ( ji )0 ,..., Zπ ( ji )|π ( j )|−1 )
In
Eq.(2),
{Z 0 ,..., Zl −1} = {Z j0 ,...Z jl −1 } ,
(2)
i
i=0
but
they
are
in
different
order;
while {Zπ ( ji )0 ,..., Zπ ( ji )|π ( ji )|−1 } ⊆ {Z ji +1 ,...Z jl −1 } , | π ( ji ) |≤ k . Z i (i = 1,..., l ) is the discrete
random variable which represents the bin indices that take values from {1,…, nb } .
Fig. 1. P ( x1 ,..., x6 ) = P ( x1 ) P ( x2 | x1 ) P ( x3 | x1 ) P ( x4 | x1 , x2 ) P ( x5 | x2 , x3 ) P ( x6 | x5 )
Linkages Detection in Histogram-Based Estimation of Distribution Algorithm
29
That is to say, the probabilistic graphical model simplifies the complete factorized probability function by assuming the Markov properties of the variables, see Fig.1. as an example. In the expansion of the complete probability, each variable is regarded to be conditionally independent of any other variables given its parent variables (the nodes with an edge pointing to the current node in the graph, for example, x1 and x2 are both parent variables of x4 ). Interested readers may consult to [19] for more knowledge about PGM. Then the core of the HEDA based on PGM is to estimate each item in the right hand side of Eq.(2). In each generation, there are mainly two jobs to estimate the PGM: 1. The probability density structure (PDS) Z j , Zπ ( j ) ,..., Zπ ( j )
{
}
found. 2. The probability density function (PDF) P( Z j | Zπ ( j ) ,..., Zπ ( j )
) and P ( Z jl −1 )
i
{
i
i |π ( ji )|−1
i 0
i |π ( ji )|−1
i 0
have to be
}
have to be calculated. In the following discussion of this section, more details of the above two jobs in the dHEDA are presented. 3.2 PDS Search
{
In each generation, the PDS Z j , Zπ ( j ) ,..., Zπ ( j ) i
i |π ( ji )|−1
i 0
}
(i = 0,..., l − 1) , is learnt from the
selected individuals. To minimize the difference between Eq.(2) and the complete factorized probabilistic function, like the IDEA-H[4], the dHEDA minimize the following expression when using the Kullback-Leibler (K-L) divergence as a metric: l −2
J ( Z 0 ,..., Z l −1 ) = H ( Z jl −1 ) + ∑ H ( Z ji | Zπ ( ji )0 ,..., Zπ ( ji )|π ( j )|−1 ) i=0
i
(3)
where H (•) denotes the entropy. For variable i with k parents, the conditional entropy is calculated by: H ( Z i | Zπ ( i )0 ,..., Zπ ( i )k −1 ) = H ( Z i , Zπ ( i )0 ,..., Zπ ( i )k −1 ) − H ( Zπ ( i )0 ,..., Zπ ( i )k −1 )
If the variables are of
(4)
nb bins, the joint entropy of {Z 0 ,..., Z n −1} is calculated by: nb −1
nb −1
z0 = 0
zn −1 = 0
H ( Z 0 ,..., Z n −1 ) = − ∑ ... ∑ PZ0 ,..., Z n−1 ( z0 ,..., zn −1 ) ln( PZ0 ,..., Zn −1 ( z0 ,..., zn −1 ))
(5)
where P( Z 0 = z0 ,..., Z n −1 = zn −1 ) is equal to the summation of the probability factors of those selected individuals that fall into the super-bin ( Z 0 = z0 ,..., Z n −1 = zn −1 ) in the dHEDA. Note that we introduce the new term super-bin to avoid confusion with the concept of bin in marginal case. The concept of super-bin is no different from that of bin except that super-bin denotes the bin in more than 1 dimension. We also use ( z0 ,..., zn −1 ) as a short hand of ( Z 0 = z0 ,..., Z n −1 = zn −1 ) in the following.
30
N. Ding and S. Zhou
In order to find the promising PDS, greedy methods can be employed to minimize J ( Z 0 ,..., Z l −1 ) . It has been verified that without considering the calculation cost of Eq.(3), we can minimize J ( Z 0 ,..., Z l −1 ) using greedy methods with polynomial computational complexity. For example, Bosman et al. proposed 3 methods for 3 different general structures (chain structure, tree structure and Bayesian Network structure). These graphical models are successful applied in Gaussian-based EDA [4] and exhibit acceptable polynomial computational complexity. 3.3 Computation Issue on the K-L Divergence
The computation issue on the K-L divergence J ( Z 0 ,..., Z l −1 ) is important because it is the metric to lead the PDS search. It is clear from Eq.(3)(4) that the core of calculating J ( Z 0 ,..., Z l −1 ) is to calculate the joint entropy. In other words, if the entropy calculation is efficient, it will be also efficient to obtain J ( Z 0 ,..., Z l −1 ) . However, the obstacle in histogram-based EDA is that the computational complexity of direct calculation of J ( Z 0 ,..., Z l −1 ) using Eq.(5) is unaffordable, because it would take exponential complexity on the maximum number of the parents nodes over all the nodes. However, it is noticed that the probability of the super-bin P( Z 0 = z0 ,..., Z n −1 = zn −1 ) is non-zero if and only if there are selected individuals in the current generation falling into this super-bin. Since there are only the number of selected individuals nbest for PGM estimation, given any Z i , Zπ ( j ) ,..., Zπ ( j ) , there exist only at most nbest super-bins with non-zero probability. According to Eq.(5), the value of the joint entropy is only contributed by those non-zero super-bins. Thus, without altering the result, the expression of joint entropy is rewritten as:
{
i 0
i |π ( ji )|−1
}
N
H ( Z 0 ,..., Z n −1 ) = −∑ PZ 0 ,..., Z n−1 ( z i 0 ,..., z i n −1 ) ln( PZ0 ,..., Zn−1 ( z i 0 ,..., z i n −1 ))
(6)
i =1
In Eq.(6), N≤nbest indicates the number of non-zero super-bins, and PZ0 ,..., Zn−1 ( z0 ,..., zn −1 ) is the probability (height) of super-bin ( z0 ,..., zn −1 ) which at least one of the selected individuals falls in. The reason of N≤nbest is because there might be more than one individual falling into the same super-bin. So, the process to calculate the entropy of certain joint variables is as following. Initially, H ( Z 0 ,..., Z n −1 ) = 0 . We first pick an unpicked individual; gain its super-bin
( z i 0 ,..., z i n −1 ) ; check if ( z i 0 ,..., z i n −1 ) exists in the memory if yes we improve the i i height of super-bin ( z i 0 ,..., z i n −1 ) , if no we create a new super-bin ( z 0 ,..., z n −1 ) and improve its height. After all individuals are picked; we sum up the probability (heights) of the super-bins in the memory times its logarithms and get the entropy. Now, let us analyze the computational complexity of the calculation of H ( Z i , Zπ ( i ) ,..., Zπ ( i ) ) . The step to find those individuals that belong to the same bin, 0
k −1
2 i(k + 1)] . and to sum up their improvements to get the height of that bin takes O[nbest
Linkages Detection in Histogram-Based Estimation of Distribution Algorithm
31
2 Thus, the complexity of O[nbest i(k + 1)] is taken to calculate H ( Z i , Zπ ( i ) ,..., Zπ ( i ) ) . 0
k −1
2 Overall, the complexity of O[nbest il 2 ] is taken to calculate J ( Z 0 ,..., Z l −1 ) . Overall, we can conclude that we are able to find the PDS with polynomial time applying the above method to calculate entropies. This is much tractable than the way that directly applies Eq.(5) to calculate entropies.
3.4 PDF Calculation
It is noticed that all the PDFs are the byproducts of entropy computation. Thus, there is no more extra step for PDF calculation if we save their results during the PDS search. So far, we have introduced the main framework of the HEDA based on probabilistic graphical model, especially the details of entropy computation because of its importance in computational complexity. There is something more that needs to be remarked: it seems tricky that the probabilities of many super-bins are equal to zero. Does that mean the later-generation individuals will never get a chance to be sampled in those zero-probability bins (which is obviously unreasonable for its lack of general knowledge)? This is not the case. In fact, in the first step, the sampling process chooses the bins only according to the height of the bin, which means those zero-height bins cannot be chosen; but in the second step, the real-value of the individuals can be sampled out of the chosen bins if we applies a variant surrounding effect. Interested readers might refer to [10] for details. 3.5 Experimental Results
We design two experiments: the first is to test the computational efficiency of the accelerated algorithm; the second is to test the performance of the HEDA based on PGM on several benchmark continuous optimization problems.
Fig. 2. The computation time to calculate the joint entropy by the dHEDA and the IDEA-H
32
N. Ding and S. Zhou
First, let us examine the computational time of the dHEDA and that of the IDEAH. The experimental results are shown in fig.2. The number of the bins is respectively 20, 30, 40; the number of the variables in the joint entropy is from 1 to 5; and the number of the selected individuals is 100. The reason that we only test at most 5 joint variables is because our PC memory excesses when we try to run the test of 6 joint variables by the IDEA-H with 40 bins. 20 times are run before the average time is calculated. The experiment is based on CPU-Intel Pentium 2.8GHz. In Fig.2, we notice that the time by the IDEA-H grows remarkably with the increase of the number of the joint variables. If we check the time in the case of bin=40 in TABLE-IV, the time of 5 joint variables entropy calculation by the IDEA-H is over 100s. On the other hand, the time by the dHEDA increases slowly and remains as a small amount of time ( ≈ 10−3 s). The other point is the great difference between them in the computational time with the growing number of the bins: the time by the IDEA-H has a distinct grow, but the time by the dHEDA hardly rises. These experimental results are concordant to our earlier analysis. Recall that the computational complexity for calculating H ( Z i , Zπ ( i ) ,..., Z π ( i ) ) is O[(nb )k +1 ] for 0
k −1
2 i(k + 1)] for the dHEDA. IDEA-H, but only O[nbest Next, we examine the performance of HEDA based on PGM with three other continuous EDAs: the UMDAc, the IDEA-G [4], and the sur-shr-HEDA [9] on 4 benchmark continuous problems: Sphere Function, Summation Cancellation Function, Schwefel-1 Function, and Schwefel-2 Function.
Table 1. Parameter Settings of the 4 Algorithms
IDEA-G
UMDAc
Parameter Settings Maximal evaluation number=3×10 Population size=375 Select-rate=20%
5
Maximal evaluation number=3×10 Population size=375 Select-rate=20%
5
sur-shrHEDA
5
Maximal evaluation number=3×10 , Select rate=20% Population size=375, Mutation rate=5% Bins number=99, Surrounding factor=10%
dHEDA
5
Maximal evaluation number=3×10 , Select rate=20% Population size=375, Mutation rate=5% Bins number=99, Surrounding factor=10%
Linkages Detection in Histogram-Based Estimation of Distribution Algorithm
33
Among the 4 algorithms, the UMDAc and the IDEA-G are based on Gaussian distribution, the sur-shr-HEDA and the dHEDA are based on histogram model; the UMDAc and the sur-shr-HEDA are marginal HEDAs, the IDEA-G and the dHEDA are HEDA based on PGM. Note that we use the dHEDA to substitute the IDEA-H because of the computational concern. The settings of the 4 algorithms are listed in Table 1. Among the 4 problems, Sphere Function and Summation Cancellation Function are unimodal, while the other 2 are multimodal. The Sphere Function and Schwefel-1 Function are separable, while the other 2 are non-separable. The problems are listed in Table 2. Table 2. Problem Descriptions
Representation l −1 JG F ( y ) = ∑ yi2
Sphere
i =0
Summation Cancellation
l −1 JG ⎛ ⎞ F ( y ) = 1/ ⎜10−5 + ∑ X i ⎟ i =0 ⎝ ⎠
X 0 = y0 , X i = yi + X i−1 l −1 JG F ( y ) = ∑ − yi isin( yi )
where
Schwefel-1
i =0
l −1 JG F ( y ) = ∑ ⎡⎣ ( yi2 − y0 )2 + ( yi − 1) 2 ⎤⎦
Schwefel-2
i =0
Domain Sphere
Dimension Type Optimum
[-100,100] l=30
Min
0
l=10
Max
105
Schwefel-1
[-500,500] l=30
Min
-12569.5
Schwefel-2
[-5,5]
Min
0
Summation Cancellation [-3,3]
l=20
Each of the algorithms is run for 20 times for each problem, and the mean value, the best case, and the standard deviation for the 20 runs are collected in Table 3. According to the results in Table 3, we can mainly summarize two points as follows: The first is that the histogram-based EDAs outperform the Gaussian-based EDAs on the above multimodal problems. For example, in the Schwefel-1 Function (which is always regarded as a typical multimodal test problem), both of the histogram-based EDAs are able to achieve the optimum of the problem, while the Gaussian-based EDAs both fail. The excellent performance made by the histogram-based EDAs agrees to our earlier remarks: it is straightforward for the histogram model to estimate the multimodal distribution. Meanwhile, since the Gaussian-based EDAs are unimodal, their models are too rough to estimate the multimodal problems efficiently.
34
N. Ding and S. Zhou
Schwefel-2
Schwefel-1
Summation Cancellation
Sphere
Table 3. Experimental Results
Best Case
Mean Value
Standard Dev.
UMDAc IDEA-G
2.816e-84
1.276e-83
1.064e-83
3.423e-161
2.562e-160
2.992e-160
sur-shr-HEDA
5.872e-15
6.753e-15
7.698e-16
dHEDA UMDAc IDEA-G sur-shr-HEDA dHEDA UMDAc IDEA-G sur-shr-HEDA dHEDA UMDAc IDEA-G
5.104e-19 8.635
7.496e-19 5.798
1.304e-19 2.403
66199
12805
20704
99998
93973
22016
105 -11622.8
105 -10637.5
2.031e-2 6.289e2
-5277.4
-4719.6
2.976e2
-12569.5 -12569.5 8.131e-5
-12569.4 -12569.5 2.576e-2
1.458e-1 1.415e-7 1.984e-2
4.324e-5
1.543e-2
1.652e-1
sur-shr-HEDA
3.793e-19
3.145e-4
1.403e-2
dHEDA
1.987e-21
3.161e-8
1.426e-7
The second is the contrast between the algorithms based on PDS without linkage and the ones based on PDS with linkage. It appears in the experiment that the dHEDAs outperforms the sur-shr-HEDA in both of the non-separable problems. The IDEA-G outperforms the UMDAc in the unimodal non-separable problems, but the two algorithms gain the comparable results on those multimodal non-separable problems. In the Summation Cancellation Function, for example, the IDEA-G’s mean value is above 104 while the UMDAc is below 101 ; and the dHEDA succeeds all of the times but the surshr-HEDA performs unstably with large deviations. This fact clearly illustrates the meaning of the learning of the linkage information in the IDEA-G and the dHEDA.
4 HEDA Based on Space Transformation Although it has already been introduced the accelerated method in probabilistic graphical models, the complexity burden is comparably large in the HEDA. One much simpler method is to deal with the linkages in marginal case. The method is based on the space transformation, especially the linear transformation. P. Pošík in [17,18] proposed several successful evolutionary algorithms on the use of space transformation. We will introduce the transformation according to the covariance-matrix of the samples in this section and apply the method in the HEDA. We call that algorithm to be the Marginal Estimation of Distribution Algorithm in
Linkages Detection in Histogram-Based Estimation of Distribution Algorithm
35
Characteristic Space of Covariance-Matrix (CM-MEDA). Intrinsically, it is equal to preprocessing the samples by PCA in EDA. Similar work can also been found in [15]. 4.1 Covariance-Matrix of the Samples
We first review several basic and well-known properties of Covariance-Matrix. The Covariance-Matrix of a set of samples is defined as follows: JG JG Given a set of m samples xi , where xi = ( xi1 , xi 2 ,..., xil )T The Covariance-Matrix is defined as C=
1 m JG JG JG T JG T ∑ ( xi − xi )( xi − xi ) m − 1 i =1
(7)
And it has the following important properties: 1. The Covariance-Matrix is real and symmetric, that is: cij ∈ R (∀i, j = 1,..., l ) and C = C T 2. The Covariance-Matrix is diagonalizable, that is:
∃Q, s.t. C = Q[λ ]Q −1 where [λ ] = diag (λ1 , λ2 ,..., λl ) 3. All the eigenvectors with different eigenvalues are orthogonal with each other.
JJG JJG
JJG
−1 4. There exists a matrix P = ( p1 , p2 ,..., pl ) , which satisfies C = P[λ ]P and all JJG T −1 pi are orthogonal with each other, that is P = P .
Finally, we formulate some important representations. The space defined in P is the characteristic space of the Covariance-Matrix C, and each pi is the basis of the space. Clearly, according to Property 2 and Property 4, we have C = PT [λ ]P , where
[λ ] = diag (λ1 , λ2 ,..., λl ) . 4.2 General Framework
The CM-MEDA is an algorithm which estimates the marginal distribution of the problem on a transformed space: the characteristic space of the Covariance-Matrix of the selected samples. The algorithm begins with an initialized population that is generated randomly. In each generation, it evaluates the fitness of each sample and selects several promising samples. The above two step has no difference from any other evolutionary algorithms. Then the CM-MEDA calculates the Covariance-Matrix C on the selected samples and then calculates the matrix P, which defines the characteristic space of the Covariance- Matrix C. After P is obtained, the selected samples are transformed to the new space by multiplying PT , so we can obtain the value on each variable in the new coordinate space.
JJG
G
Assuming a selected sample of x, this step is finished by x ' = P ⋅ x . T
36
N. Ding and S. Zhou
The marginal distributions are estimated in the new space. After the marginal disJJG tributions of the variables in the new space have been estimated, new xi' are sampled according to the estimated distribution in each variable. At last, we transform the samples back into the initial coordinate space, according
G
JJG
JJG
−1 T to x = ( P ) ⋅ xi' = P ⋅ xi' . Note that when we transform the samples back, some samples might out of constraints. If that happens, we discard those “illegal” samples and resample the new ones. After we have built the new population, we evaluate the fitness and go to the next generation. In short, the framework of CM-MEDA is as followed:
1. Initialize the population 2. Select the samples with high fitness 3. Calculate the Covariance-Matrix C of the selected samples and build the matrix P 4. Transform the selected samples onto the characteristic space of the CovarianceJG G Matrix by x ' = PT i x 5. Estimate the marginal models on the new space according to the distribution of the transformed selected samples JG
6. Make new samples xi' according to the marginal models in the transformed space 7. Transform the new samples from the transformed space back to the original G
JG
space by x = P ⋅ xi' and check if all the new samples are legal. Resample the illegal samples. 8. Return to step 2 if not terminated. 4.3 CM-MEDA in Histogram Case
In the histogram-based CM-MEDA, the marginal distribution in the transformed space is based on histogram model. There is only one specific concern besides the general framework of the CM-MEDA in the histogram case, that is: How to decide the domain (i.e. upper bound and lower bound) of the histogram model in the transformed space. We solve this problem in a simple way: the domain of each variable is decided according to the range of the selected samples in each variable. Note that, under this way the algorithm can naturally shrink the domain of the variables. Besides, to avoid loss of generality, we make the domain of the variable be a little larger than the range of the variables. In our algorithm, there is one more bin to the left of the leftmost sample and one more bin to the rightmost sample. The leftmost sample is in the center of the bin it belongs to, and so is the rightmost sample. See Fig.3 as an illustration. 4.4 Relation to Principal Component Analysis
Principal component analysis (PCA) is a method that seeks for a projection which best represents the samples in a least-squares sense [20]. It intrinsically provides a promising way of finding the projections which mostly capture the variance of the
Linkages Detection in Histogram-Based Estimation of Distribution Algorithm
37
Fig. 3. The position of the selected samples and the domain of the bins. Note that the leftmost and the rightmost selected samples are in the center of the bins that they belong to. And there is one more bin to the left of the leftmost sample and one more bin to the right of the rightmost sample.
samples. For example, the first component (the principal component) is the projection that the samples have the maximum variance; the second component is the projection with the maximum variance in the subspace orthogonal to the first component, and etc. Since the components given in PCA are the same as the eigenvectors of Covariance-Matrix in the CM-MEDA, we can alternatively think that the core of the CMMEDA is to find the projections which maximize the variance in the space iteratively, because each eigenvector is the projection which has the maximum variance of the samples in the subspace orthogonal to the eigenvectors with higher eigenvalues. The projection which has the maximum variance of samples is the one which mostly needs estimation in evolutionary optimization, because its ultimate aim is to find one optimal point not a scatter of the samples. Thus, we infer that the CM-MEDA can improve the convergence of the EDA by making distribution estimation on the set of axes which maximize the variance of the samples from the viewpoint of PCA. Similar work may also refer to [15,16,17]. According to the framework above, we can find that the CM-MEDA has only three more steps than the MEDA: calculation of covariance matrix, calculation of eigenvectors, and transformation. The calculation of covariance matrix takes O(nbest il 2 ) , where nbest is the number of selected samples, and l is the dimension of the samples. The eigenvectors can be efficiently calculated in O(l 3 ) . And the transformation takes O(l 2 ) . Therefore, it is clear that the CM-MEDA is much faster than the HEDA based
on PGM, even the dHEDA. (Recall that it takes O(nbest 2 il 2 ) to calculate K-L divergence in the dHEDA. ) However, we also note that the CM-MEDA can only accurately capture the linear linkages; while the non-linear linkages are always approximated intrinsically. In contrast, the HEDA based on PGM is able to capture more complex relations among variables if less restrictions on its PDS.
38
N. Ding and S. Zhou
4.5 Experimental Results
We provide a brief experiment to compare the CM-MEDA and MEDA in histogram case on 4 continuous optimization problems. The general settings of the two algorithms are same: the population size is 1000; 20% of the individuals are selected from the population to estimate the model; the number of the bins in each bin is arbitrarily set to be 100; 40 iterations are run each time. The MEDA use the mechanism of the sur-shr-HEDA to update the histogram model. The surrounding factor is set to be 10%, and the mutation rate is set to be 5%. The 4 test functions: Fun-1, Fun-2, Schwefel, Rosenbrock are all non-separable and share a similar formulation. The reason that we choose several functions in this Table 4. Problem Descriptions
Representation n−1
Fun-1
Fun _1( x1 ,...xn ) = ∑100i( xi +1 − 1.3i xi )2 + (1 − x1 )2 i =1
n −1
Fun-2
Fun _ 2( x1 ,...xn ) = ∑ [( xi +1 − xi2 )2 + (1 − xi ) 2 ] i =1
Schwefel
n
Schwefel ( x1 ,...xn ) = ∑ [( x1 − xi2 ) 2 + (1 − xi ) 2 ] i =1
n−1
Rosenbrock
Rosenbrock ( x1 ,...xn ) = ∑ [100i( xi +1 − xi2 )2 + (1 − xi ) 2 ] i =1
Fun-1 Fun-2 Schwefel Rosenbrock
Domain [-5,5] [-5,5] [-5,5] [-5,5]
Dimension 10 10 10 10
Type Min Min Min Min
Optimum 0 0 0 0
Table 5. Experimental Results
Fun-1
Fun-2
Schwefel
Rosenbrock
Best Mean Std. Best Mean Std. Best Mean Std. Best Mean Std.
CM-MEDA 0.1775 0.2760 0.0801 3.879e-11 0.0015 0.0057 1.320e-10 0.0152 0.0359 6.5271 7.2160 0.3181
MEDA 1.7321 3.4524 0.7483 0.0319 0.0571 0.0112 0.0378 0.0682 0.0184 4.2003 8.0493 1.4204
Linkages Detection in Histogram-Based Estimation of Distribution Algorithm
39
formulation is because this formulation of functions is usually hard for Evolutionary Algorithms to solve because of their strong linkages between variables. The problem descriptions are listed in Table 4. Each of the algorithms is run for 20 times in each problem, and the mean value, the best case, and the standard deviation for the 20 runs are collected in Table 5. The results in Table 5 verify the superiority of the CM-MEDA in histogram case in those non-separable cases. For example, when solving Schwefel function, the best solution obtained by the CM-MEDA in 20 runs is 1.320e-10, while the best solution of the MEDA is 0.0378; the mean value of the CM-MEDA is 0.0015, while the mean value of the MEDA is 0.0682.
5 Summary and Further Work In this chapter, we have reviewed the two methods in HEDA which are able to detect linkage among variables in the optimization problems. Using probabilistic graphical model generally represents the relations more accurately but also is of high-cost; while using space transformation is quite fast, but it only roughly represents the major possible relationships. There is, however, still much further work in both topics. The first, for the HEDA, the number of bins in certain problems is always decided arbitrarily in most papers. An alternative method might apply the prior on the bins, such as Dirichlet Process, and then use Bayesian analysis in estimating the model. The second, for the HEDA based on PGM, it is still interesting to have deeper research on dealing with the case of small number of samples with great number of bins in a joint probability. The third, for the HEDA based on space transformation, to find a more efficient transformation, or to define the kernel space might allow the algorithm to capture more complex and non-linear linkages. The fourth, there is little research on comparing the two method above directly. Besides, the attempts to combine the above two methods might have even improved performance, since the space transformation might reduce the number of parents of each nodes in the graphical model.
References [1] Larranaga, P., Lozano, J.A.: Estimation of Distribution Algorithms: A NewTool for Evolutionary Computation. Kluwer Academic Publishers, Dordrecht (2002) [2] Pelikan, M.: Hierarchical Bayesian optimization algorithm: Toward a new generation of evolutionary algorithms. Springer, Heidelberg (2005) [3] Pelikan, M., Goldberg, D.E., Tsutsui, S.: Getting the best of both worlds: Discrete and continuous genetic and evolutionary algorithms in concert. Information Science 156, 147–171 (2003) [4] Bosman, P.A.N., Thierens, D.: An algorithmic framework for Density Estimation based Evolutionary Algorithms. Utrecht University technical report UU-CS-1999-46 (1999)
40
N. Ding and S. Zhou
[5] Bosman, P.A.N., Thierens, D.: Numertical optimization with real- valued Estimation of Distribution Algorithm. Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications, 91–120 (2006) [6] Lu, Q., Yao, X.: Clustering and learning Gaussian distribution for continuous optimization. IEEE trans. on Systems, Man and Cybernetics-Part C 35(2), 195–204 (2005) [7] Tsutsui, S., Pelikan, M., Goldberg, D.E.: Evolutionary algorithm using marginal histogram models in continuous domain. In: 2001 Genetic and Evolutionary Computation Conference Workshop, San Francisco, CA, pp. 230–233 (2001) [8] Ding, N., Zhou, S., Sun, Z.: Optimizing continuous problems using Estimation of Distribution Algorithms based on histogram model. In: The 6th International Conference of Simulated Evolution and Learning, Hefei, China, pp. 545–562 (2006) [9] Ding, N., Zhou, S., Sun, Z.: Histogram-based Estimation of Distribution Algorithm: a competent method for continuous optimization. J. Computer Science and Technology 23(1), 35–42 (2008) [10] Ding, N., Xu, J., Zhou, S., Sun, Z.: Reducing Computational Complexity of Estimating Multivariate Histogram-based Probabilistic Model. In: 2007 IEEE Congress on Evolutionary Computation, pp. 111–118 (2007) [11] Yuan, B., Gallagher, M.: Playing in continuous spaces: Some analysis and extension of population-based incremental learning. In: IEEE Congress on Evolutionary Computation, Canberra, Australia, pp. 443–450 (2003) [12] Baluja, S.: Population-based incremental learning. Carnegie Mellon University, Technical Report. CMU-CS-94-163 (1994) [13] Cantu-Paz, E.: Supervised and Unsupervised Discretization Methods for Evolutionary Algorithm. In: The Genetic and Evolutionary Computation Workshop, San Francisco, CA, pp. 213–216 (2001) [14] Zhang, Q., Sun, J., Tsang, E., Ford, J.: Hybrid Estimation of Distribution Algorithm for Global Optimization. Engineering Computations 21(1), 91–107 (2004) [15] Zhang, Q., Allinson, N.M., Yin, H.: Population Optimization Algorithm Based on ICA. In: the First IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (2000) [16] Zhang, Q., Mühlenbein, H.: On the convergence of a class of estimation of distribution algorithms. IEEE Trans. Evolutionary Computation 8(2), 127–136 (2004) [17] Pošík, P.: On the Utility of Linear Transformations for Population-Based Optimization Algorithms. In: the 16th World Congress of the International Federation of Automatic Control (2005) [18] Pošík, P.: On the Use of Probabilistic Models and Coordinate Transforms in Real-Valued Evolutionary Algorithms. PhD Dissertation, Czech Technical University (2007) [19] Lauritzen, S.L.: Graphical Models. Clarendon Press, Oxford (1996) [20] Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley Interscience, Chichester (2000)
Linkage in Island Models Zbigniew Skolicki George Mason University, Fairfax, VA. Current affiliation is Google, Inc., 1600 Amphitheatre Parkway, Mountain View, CA
[email protected]
Abstract. Island models (IMs) have multiple sub-populations which periodically exchange a fraction of individuals. This complex setup results in distinct dynamics of evolution and therefore, IMs are characterized by several interesting properties of IMs with regard to gene linkage, which are presented in this chapter. Traditional single-population evolutionary algorithms (EAs) suffer from a relatively quick and often random gene fixation, due to evolutionary selection, drift and physical linkage of recombination operators. In IMs, additional, slower inter-island level of evolution together with a recurring local evolution inside islands make searching for optimal allele configurations more systematic. In this chapter, it is shown that IM dynamics may counteract hitch-hiking. Further, multiple building blocks may be better identified and combined in IMs, consequently supporting compositional evolution, which is studied on the H-IFF function. Finally, a discussion follows on how the repeated fixation of genes in islands might be treated as a linkage learning process.
1 Introduction Searching for an optimal solution to a given problem requires mapping of the solution space into a representation space. Solutions are encoded using multiple parameters — or genes in evolutionary computation. However, these parameters/genes are rarely independent, with regard to the characteristics of solutions. Moreover, the mutual dependency between them, called linkage or epistasis, is very often inherent to the problem, and exists regardless of the representation used. In fact, linkage between genes is a fundamental reason to use evolutionary algorithms (EAs). If each gene could be optimized independently, the problem would be perfectly separable. Such problem could be easily solved sequentially, without any need for evolutionary algorithms. The strength of evolutionary algorithms (at least according to the building block hypothesis) lies in their ability to simultaneously analyze multiple configurations of genes (building blocks, BB), to focus on the more promising ones and to combine together such blocks. Recombination has a profound impact in evolutionary systems [10]. One of the properties of recombination operators is that they copy certain genes together more often than others. The resulting dependence between genes is called a physical linkage. There is a difference between inherent and true linkage between parameters (epistasis)
Most of the work was done for author’s PhD dissertation.
Y.-p. Chen, M.-H. Lim (Eds.): Linkage in Evolutionary Computation, SCI 157, pp. 41–60, 2008. c Springer-Verlag Berlin Heidelberg 2008 springerlink.com
42
Z. Skolicki
existing in a given problem, and a physical linkage existing due to recombination operators. In fact, good recombination operators are designed so that their physical linkage approximates a given problem epistasis. Among those operators, more conservative ones focus on maintaining certain gene configurations, while more aggressive operators focus on mixing genes, potentially discovering better blocks. A lot of research has been directed at learning linkage, that is adjusting recombination operators so that their physical linkage matches problem epistasis. An example here is a Messy GA [4] or the Linkage Learning GA [6]. Also SEAM [10] supports recombination of building blocks, regardless of their configuration on a genome, by initially operating on subsets of genes. Unfortunately, regardless of simultaneous analysis of multiple configurations, evolutionary algorithms are imperfect in searching for the best solution. It is well known that repetitive stochastic sampling and the selection operator in any evolutionary algorithm causes genes to lose alleles and a population to converge [1]. The smaller the population, the more visible is the drift. Only a small portion of a usually large number of possible gene configurations have a chance to be verified and potentially remain as part of a genome before a quick fixation. Physical linkage and stochastic sampling results in coincidentally copying certain genes together. A process in which an allele of one gene increases its survival rate due to randomly being selected with good alleles of other genes is called hitch-hiking. Although this effect occurs stronger for more conservative recombination operators, we may observe it even for a uniform crossover, which copies random genes together. There are two seemingly opposite goals of an effective evolutionary algorithm. One is preventing the preliminary convergence and side-effects of stochasticity, and the other is being able to identify, converge to and propagate useful gene combinations. The observations suggest that IMs, thanks to their two-level evolution, may be able to facilitate both the identification of useful gene combinations, and the maintenance of multiple search trajectories at a more general level. These properties should make solving problems with a high level of epistasis easier. In this chapter we will see potential benefits from using IMs, in the context of gene linkage. First, the interaction of the two levels of evolution in IMs is discussed in the context of the fixation of genes, highlighting several interesting implications. We will see that IMs reduce random gene fixation (and hitch-hiking). These observations lead to a more profound conclusion, namely that IMs support so-called compositional evolution [10]. Arguments suggesting that IM dynamics may lead to adaptation of fixations to the linkage between genes conclude the chapter.
2 Island Models and Two-Level Evolution Island models (IMs) are models of evolutionary algorithms in which multiple copies of the evolutionary process are run simultaneously in sub-populations called islands, and individuals are exchanged between them from time to time. Since standard operators are restricted to islands, IMs are an example of non-panmictic algorithms, as opposed to single-population panmictic EAs, where all individuals can interact freely. Island models have been initially studied by evolutionary biologists, and later in computer
Linkage in Island Models
43
science, both in theory and practice. A good background introduction to island models was given by Gustafson [5]. A review was also included by Skolicki [7]. Unfortunately there does not exist a good, widely accepted theory of island models. A classic Shifting Balance Theory by Wright [11] explains the benefits of using island models mainly by their ability to search multiple regions and ”shift” focus of all islands to the promising regions. Drift plays a major role in diversifying the search and thus exploring new regions — somewhat similarly to how it happens according to the theory of Neutral Evolution [9]. Let us confront two possible understandings of IM dynamics. The first one is more common, but there exists another one, which offers explanation to certain IM phenomena. The common understanding of IMs is that they are a collection of independent EAs, which exchange solutions after they evolved them. In this framework islands “compete” to evolve the best possible solution, and the best island “wins.” Such an understanding suggests a few big islands, so that diversity and evolvability is maintained independently inside each of them. A migration either serves to bring the new best individual (and so one should always choose the best emigrants), or sometimes as a big mutation to maintain a high level of diversity. Another understanding is achieved by taking a higher view of the islands to see that they may be treated as individuals in a higher-level population [8, 7]. Interaction between the islands creates a higher-level evolution, and therefore, the whole IM can be seen as a much more compact entity, even if migrations seem to loosely connect the islands. Consequently, two levels of evolution can be identified. One is local, intra-island evolution, which has been studied so far on islands. The other is global, inter-island evolution, occurring between islands. It comes as no surprise that those two levels of evolution interact and mutually impact their dynamics. In this view, the global evolution uses the local evolution to carry out actual computations and significantly changes its dynamics. The local evolution creates the global evolution by “selecting”, “mutating” and “mixing” islands. In IMs sub-populations are relatively smaller (compared to a single population), so genes are getting fixed faster due to selection and drift. However, semi-independent evolution causes islands to fixate genes to different alleles. These blocks of different alleles are later shared by migration. IMs keep restoring the local diversity with the help of migrations and thus practically restarting local evolution when needed. Because migrations occur every migration interval, the global interaction occurs at a slower pace than the local evolution. As a result, the whole evolution occurs at a slower pace compared to a single-population EA. The alternative view of IMs requires rethinking the impact on performance of such standard IM parameters like island size and number, migration size, interval, topology and policy. However, the aim of this chapter is to present the relation between the dynamics of IMs and gene linkage. The analysis of the influence of IM parameters on IM dynamics, although related, is beyond the scope of this chapter. Such analysis of IM performance on a wide spectrum of IM parameters was performed in another publication [7]. Nevertheless, A few issues are worth commenting here, based on these experiments.
44
Z. Skolicki
With sub-populations of smaller size getting converged quickly, there is a chance that some important gene linkage (or local optima) may not be discovered in any island. This disadvantage is to some degree alleviated by the larger number of islands, in which such linkage/optimum may be found. The question is how to balance the size and number of islands to maximize the probability of finding good solutions. As will be shown in the next sections, the ability of inter-island evolution to stimulate intraisland evolution after local convergence, is strong. It is fine to converge on a local level. On the other hand, the diversity of inter-island evolution is not easily increased. This suggests that setups with larger number of islands (being “individuals” of inter-island evolution) than the size of islands should be preferred. In fact, experiments on some simple functions suggest that the performance increases with increasing the number of islands, until rather extreme configurations are reached, when it drops again. A second issue is whether choosing random or elite individuals should be preferred. However, with smaller islands, which are converging within migration intervals, the choice of migration policy is less important. In such IMs, all individuals in a given island will be very similar at the moment of migration, and therefore the performance should be similar regardless of the method of migrant choice. A similar behavior was observed in experiments with various migration policies. Finally, choosing migration size and interval are important. It turns out that migration interval cannot be too short, so that the intra-island evolution is not interrupted prematurely. It was also shown that the mixing between islands can be highly sensitive to the migration size and other parameters. A proper choice of these parameters is required for successful exchange of partial solutions between islands.
3 Experimental Setup For experiments in this chapter, the following model of IMs was used. A number of N = 10 identical islands of size M = 10 individuals is used (often described as NxM model). Generations are synchronized and migrations occur every migration interval i = 10. Instead of specifying a constant number of migrants, a migration probability α = 0.1 is applied to each individual. Such an approach allows for experimentation in general with migration ratios different than fractions resulting from a small island size. A dynamic full topology was used, which means that each time a migration is about to occur a single target population for migrants is chosen randomly out of all populations. A similar approach was used in the literature [3]. In general, dynamic topology allows for comparing setups with different number of islands in a fair way and constant α , because the total number of migrants remain constant. A random migration policy was also used, choosing emigrants randomly and replacing random individuals in the target populations. In some experiments mutation is turned off to better focus on recombination and to make the effects of mixing genes more visible. Mutation operates on single genes, and therefore it is less related to linkage, compared with recombination. Two setups for EAs selection pressure in islands is used, EA-1 and EA-2 [2]. EA-1 is weaker and uses binary tournament parent selection, no survival selection and nonoverlapping model (similar to GA setups). EA-2 is stronger and uses stochastic parent
Linkage in Island Models
45
selection, truncation survival selection and overlapping model with a brood ratio of 1.0 (similar to ES setups, in particular (μ + λ ), where μ = λ ). Those setups refer only to the selection pressure, in the way described above, and reproduction operators (recombination and mutation) are used independently (unlike traditional ES setups with no recombination). Either uniform or one-point crossover is used at a rate of 1.0 (for experiments with the OneMax, OneHitcher and ManyHitchers functions) and at a rate of 0.7 (for experiments with the H-IFF function). Mutation, if used, is performed at a 1/L ratio. It is either bit-flip mutation for binary representation, or a non-adaptive Gaussian mutation with σ = 0.01% of a domain range (quite small) for real-valued representation. Elitism is used for experiments with the H-IFF function. For all measures a mean from 60 iterations is reported, which results in rather small confidence intervals.
4 Oscillating Gene Convergence and Avoidance of Random Fixations EAs operate in a stochastic manner on a finite set of individuals. Therefore, some gene values can be carried from a parent to a child by accident, rather than being deterministically selected, and other alleles may be lost through generations. Additionally, an allele of one gene may survive because of being repetitively copied together with a good allele of another gene, due to the physical linkage of a recombination operator. Such a situation is called “hitch-hiking” and makes the survival of other alleles even less likely. In some cases, hitch-hiking may cause a fixation of the first gene to a nearly randomly chosen value. The effect will be stronger when the recombination operator is more conservative because of a higher chance of survival of physically linked alleles (e.g. it is stronger with a one-point than with a uniform crossover).
20
fixed genes
15
10
5 1x100 single population 10x10 IM 0 0
50
100
150 200 generations
250
300
Fig. 1. Average gradual fixation in IMs compared to fast simultaneous fixation in one-population EA
46
Z. Skolicki 0.5 all genes
locus diversity
0.4
0.3
0.2
0.1
0 0
50
100
150
200
250
300
generations
Fig. 2. Allele diversity for all 20 loci, in a single run with a single population (100 ind.)
0.5 all genes
locus diversity
0.4
0.3
0.2
0.1
0 0
50
100
150 200 generations
250
300
Fig. 3. Allele diversity for all 200 loci, in a single run with a 10x10 IM
The two-level interaction has impact on selecting good gene configurations and, as a result, on gene fixation, which proceeds much slower in IMs than in a single-population EA. Fig. 1 illustrates how the average number of fixed genes grows in IMs compared to a standard EAs and how migrations counteract fixation on the OneMax function.1 Whereas for a standard EA most of genes converge relatively fast, for an IM the convergence oscillates due to migrations. This creates a situation in which some genes are already fixed and some still have multiple alleles across islands. Fig. 2 shows the diversity of alleles for each of 20 loci in a run of a single-population EA and Fig. 3 shows 1
α = 0.1, i = 10, policy = random, topology = full, EA-1, recombination = uniform crossover, L = 20, no mutation. The OneMax problem maximizes the number of 1s in a binary genome.
Linkage in Island Models
47
2 1.6 1.2 0.8 0.4
0
0.2
0.4 x
0.6
0.8
1 0
0.2
0.4
0.6
0.8
1
y
Fig. 4. Function TwoGenes, a = 1 1
average x
0.8
0.6
0.4 1x100 panmictic, EA-1 1x100 panmictic, EA-2 10x10 IM, EA-1 10x10 IM, EA-2
0.2
0 0
100
200 300 generations
400
500
Fig. 5. Average x (first gene) value, a = 0, uniform crossover
each of 20*10=200 loci in an IM. It is again visible that for a standard EA, most of the genes converge in more or less the same time; for an IM some converge much earlier than others, due to migrations that partially restore diversity on islands. If some genes influence fitness more than others, it is more likely that their good values will cause a fitness value to be high enough to determine the selection of genomes containing these alleles. As a result, such genes will be optimized by an EA earlier, whereas the other genes will suffer from stochastic sampling and hitch-hiking and will keep losing alleles. If the less-contributing genes converge at the same time as the morecontributing ones, the first ones cannot be effectively optimized. As we saw, in IMs, it is unlikely that all genes will get fixed at the same time. Only those genes that are non-fixed are still being evolved, and the others are not changed by recombination since the alleles are the same for all individuals in an island. Such a situation in general allows weaker genes to evolve after the more influential ones got optimized. In each island, the less important genes can get fixed to a different value, maintaining their diversity at the inter-island level.
48
Z. Skolicki
1 0.9 0.8 average x
0.7 0.6 0.5 0.4 1x100 single population, EA-1 1x100 single population, EA-2 10x10 IM, EA-1 10x10 IM, EA-2
0.3 0.2 0.1 0
100
200 300 generations
400
500
Fig. 6. Average x (first gene) value, a = 0, one-point crossover 1
average x
0.8
0.6
0.4 1x100 panmictic, EA-1 1x100 panmictic, EA-2 10x10 IM, EA-1 10x10 IM, EA-2
0.2
0 0
100
200 300 generations
400
500
Fig. 7. Average x (first gene) value, a = 10, uniform crossover
Let us take a function TwoGenes,2 defined over [0, 1] × [0, 1] by Equation (1) below and shown in Fig. 4: TwoGenes(x, y) = max(1 − x, 10x − 8.9) + ay .
(1)
The x and y genes are independent. Two peaks exist for the first gene and one peak for the second gene. The parameter a regulates if the second gene is stronger or weaker than the first one. If a is big, y will dominate when calculating the fitness, and x will get fixed to quite random values. After y converges, evolution optimizes x, but since at that time its diversity is low, the process is similar to hill-climbing, often converging to the lower peak, which has a bigger basin of attraction. 2
This function’s name in [7] is IM1 .
Linkage in Island Models
49
0.8 0.7
average x
0.6 0.5 0.4 1x100 single population, EA-1 1x100 single population, EA-2 10x10 IM, EA-1 10x10 IM, EA-2
0.3 0.2 0
100
200
300
400
500
generations
Fig. 8. Average x (first gene) value, a = 10, one-point crossover
10 i=1 i=2 i=5 i=10 i=20
fitness contribution
8
6
4
2
0 0
0.2
0.4
0.6
0.8
1
xi
Fig. 9. Function ManyGenes, gene contributions
Since the two possible peaks for x are 0.0 and 1.0, the average x value from multiple runs is equal to a percentage with which the model converges to the higher peak. The results of experiments confirm that the use of island models increases the average x value, compared to the panmictic case. Fig. 5 – 8 compare the average x using two different values of a.3 With weak selection (EA-1), an IM performs better than a singlepopulational EA, even with a = 0. With strong selection (EA-2), single-population EA does not “lose” individuals from the good peak, when a = 0, so it is performing 3
α = 0.1, i = 10, policy = random, topology = full, recombination = uniform crossover, L = 2, mutation rate = 1/L = 0.5.
50
Z. Skolicki 1.8e+06 1.7e+06 1.6e+06
fitness
1.5e+06 1.4e+06 1.3e+06 1.2e+06 1x100 panmictic, EA-1 1x100 panmictic, EA-2 10x10 IM, EA-1 10x10 IM, EA-2
1.1e+06 1e+06 900000 0
20
40
60
80
100
120
140
generations
Fig. 10. Maximum fitness averaged over runs with ManyGenes function, uniform crossover
1.4e+06 1.35e+06 1.3e+06
fitness
1.25e+06 1.2e+06 1.15e+06 1.1e+06 1.05e+06 1x100 single population, EA-1 1x100 single population, EA-2 10x10 IM, EA-1 10x10 IM, EA-2
1e+06 950000 900000 0
20
40
60 80 100 generations
120
140
Fig. 11. Maximum fitness averaged over runs with ManyGenes function, one-point crossover
comparably to an IM. However, when a = 10, the results for a single population are much worse compared to the IM case, which performance decreases only a little. We see that genes are analyzed more independently in IM than they are in traditional EAs. Let us test this hypothesis on one more example with more genes. Imagine a function ManyGenes,4 shown below in Equation (2), with L = 20 and defined over [0, 1] × [0, 1]. L
ManyGenes(x) = ∑ (2xi )i . i=1
4
This function’s name in [7] is IM1 .
(2)
Linkage in Island Models
51
1
20th gene
0.9 0.8 0.7 0.6 0.5
1x100 single population 10x10 IM
0.4 0.4
0.5
0.6
0.7 0.8 1st gene
0.9
1
Fig. 12. Trajectory of the 1st vs. the 20th gene average values, EA-2, uniform crossover
1 0.9
20th gene
0.8 0.7 0.6 0.5 1x100 single population 10x10 IM 0.4 0.4
0.5
0.6
0.7 1st gene
0.8
0.9
1
Fig. 13. Trajectory of the 1st vs. 20th gene average values, EA-2, one-point crossover
Selected components of this function are shown in Fig. 9. The highest-index (i = L) gene influences fitness most, provided that the average value of the corresponding gene (xL ) is already above 0.5, which is quickly reached. The shape of function ManyGenes causes a very fast convergence of this gene. In a single-populational EA, this behavior causes the other genes to have less of a chance of being optimized properly. A 100-individuals standard EA was compared with a 10 × 10 IM, using EA-2 and turning off mutation to stress the effect of losing the alleles. With mutation, both algorithms would clearly have results very close to the optimum, since ManyGenes is
52
Z. Skolicki
1
20th gene
0.9 0.8 0.7 0.6 0.5
1x100 single population 10x10 IM
0.4 0.4
0.5
0.6
0.7
0.8
0.9
1
15th gene
Fig. 14. Trajectories of the 15th vs. the 20th gene average values, EA-2, uniform crossover 1 0.9
20th gene
0.8 0.7 0.6 0.5 1x100 single population 10x10 IM 0.4 0.4
0.5
0.6
0.7 15th gene
0.8
0.9
1
Fig. 15. Trajectories of the 15th vs. the 20th gene average values, EA-2, one-point crossover
a unimodal function. Fig. 10 and 11 show that the IM performs better than a singlepopulational EA, both for EA-1 and EA-2.5 The analysis of the average change of gene values explains why an IM achieves better results. The value of the first gene (least important) compared against the value of the last, 20-th gene (most important) is plotted in Fig. 12. For a single population, the 20-th gene is optimized first (because of its bigger impact on fitness), and only then the 1-st gene changes its average value. It is, however, too late to optimize it well, because many alleles have been lost. This asymmetry occurs between any two genes in the function, but the more distant they are, the difference between them is more visible. On the other 5
α = 0.1, i = 10, policy = random, topology = full, recombination = uniform crossover, L = 20, no mutation.
Linkage in Island Models
53
hand, in the IM we can clearly see, that the convergence of the 20-th gene slows down, allowing stepwise optimization of the 1-st one between migrations. Toward the ends of these intervals, evolution stagnates, and after migrations additional optimization is possible for both genes. For the case of one-point crossover the effect of using IMs is particularly visible (see Fig. 13). This is because of a linkage preserving property of a one-point crossover. As a result, the hitch-hiking effect is stronger in a single-populational EA. Migration intervals enforce some balance between genes. Stronger ones have to “wait” for the weaker ones. In Fig. 14 and 15, we see analogical plots for the 15-th and 20-th genes, where this effect is visible better.
5 Compositional Evolution In this section IMs are shown to support compositional evolution, which was defined by Watson as “evolutionary processes involving the combination of systems or subsystems of semi-indpendently preadapted genetic material” [10]. In fact, the existence of two levels of evolution, and of the oscillating convergence of genes, suggest that local evolution might be able to construct building blocks, and the global level evolution may be efficient at mixing them. Generally, functions, which have solutions built from multiple sub-solutions, should benefit from using IMs. The examples in this section are based on the H-IFF function, which is the original function used by Watson to test compositional evolution. The domain of this function is binary. The evaluation procedure first splits genome conceptually into pairs of genes and gives credit whenever such two genes are the same (either 00, or 11). Then, fourtuples consisting of two pairs each are checked for having the same allele (either 0000, or 1111). This checking is performed with longer and longer modules, until the whole genome (presumably of length being a power of 2) is tested. At each next level there is half the number of available blocks, but their weight is doubled, so they have the same contribution to the general fitness. This is done on purpose to balance the importance of intra-module and inter-module relations. As shown by Watson, the H-IFF can only be effectively solved by EAs using composition (recombination). In Fig. 16, performance results of IM, single-population and isolated setups are compared on the H-IFF function 6 . One-point recombination was chosen because in the H-IFF function close bits may belong to the same module. The average fitness obtained for an IM is significantly higher than the fitness values obtained for both panmictic and isolated models. Because of a special construction of the H-IFF function, these results confirm that IMs support compositional evolution. In order for inter-island evolution to work, locals and migrants must be effectively mixed. Assumptions about the survivability of blocks consisting of both locals and migrants can be confirmed directly by measuring this survivability, as described below. In each run, towards the end of migration intervals islands converge either back to locals, to migrants, or to some hybrids. For each island and each migration, the change in 6
α = 0.1, i = 10, policy = random, topology = full, EA1, elitism, , L = 64, two-point crossover rate 0.7, mutation rate =0.015 ≈ 1/L).
54
Z. Skolicki 400 350
fitness
300 250 200 1x100 single population 10x10 isolated 10x10 island model
150 100 0
100
200 300 generations
400
500
Fig. 16. Island model obtains much better result than both single-population and isolated multipopulation models, on the H-IFF function
50
migrants Δf locals Δf hybrids Δf
between-migration Δf
40 30 20 10 0 -10 -20 -30 0
100
200 300 generations
400
500
Fig. 17. Rejecting migrants with uniform crossover
fitness in the whole interval was credited to the final individual type. Results were averaged over multiple runs. In each migration interval, the change in fitness is measured with regard to its value at the beginning of the interval, so even when fitness grows continually, this is reflected as repeated periods of fitness increase from zero (or a value immediately following migration). The above procedure tells what type of internal behavior is responsible for fitness increase. If we see high fitness change for hybrids, then we know that mixing locals and migrants (or inter-island “recombination”) is the source of fitness increase. A high value of fitness change for migrants means that they are dominating and increasing average fitness. If we see a high value of fitness change for locals, then it means that rejecting migrants is the best behavior (fitness may increase as long as locals are able to maintain
Linkage in Island Models 50
migrants Δf locals Δf hybrids Δf
40 between-migration Δf
55
30 20 10 0 -10 -20 0
100
200 300 generations
400
500
Fig. 18. Accepting migrants with one-point crossover
evolvability without the help of migration). We see these measures in Fig. 17 for a uniform crossover, and in Fig. 18 for one-point crossover. There is a subtle difference between the two cases. If we look at the curves corresponding to hybrids, we see that for the one-point crossover this curve has the highest values most of the time, which means that this recombination operator successfully supports mixing locals and migrants. On the other hand, with uniform crossover, the curve stays closer to zero, and the cases where locals “regain” the island are characterized by a drop in fitness apparently due to unsuccessful mixing and the loss of good alleles.
6 Potential Adaptation to Existing Linkage A gradual repeated fixation of genes in IMs may adapt fixations to good building blocks, and potentially serve as a linkage learning process. Repeated migrations and exchanging blocks of linked genes are very important for this process. If islands converge totally before migrations, migrant groups will be homogeneous and in practice each island can be treated as a single individual in an inter-island evolution. However, if migrations occur earlier, a variable speed of gene fixation, as seen in section 4, creates a situation when some genes are already fixed, and some still have multiple alleles in a population. Only those genes that are non-fixed are still being evolved, and the others are not changed by recombination since they are the same for all individuals in a population. We can treat the fixed genes as linked, forming a “building block”. Such building blocks would not be specific to any single genome, but would rather result from comparing all individuals in a given island. Migrants represent the fixed genes, and migration should correspond to an exchange of such building blocks, their recombination and re-fixing. Repeated fixations of such BBs should improve their average fitness, as described in the next paragraph. In a single population, once genes get fixed, they remain so, and we have no guarantee that the newly formed building block is indeed a good one. In fact, unless selection
56
Z. Skolicki
is very weak and recombination characterized by a high mixing level, some genes will get fixed to suboptimal alleles. On the other hand, in IMs new individuals are inserted into islands with each migration. If the previously fixed building block was on average producing mediocre solutions, there is a chance that genes of better migrants will fill the island, and a new fixation will replace the previous fixation. The difference comparing this to standard EAs is that in IMs an already pre-evolved block of genes may replace other pre-evolved blocks of genes, which may be more probable than a new good building block emerging in one generation with the help of mutation and recombination. Different fixations are formed inside different islands, and they do not compete with each other until migrants are sent between islands. If the fixation in an island was good, migrants have a smaller chance to change the dominating individuals. As a result, with time, better fixations should survive. Such process could be treated as an example of linkage learning. Even if migrations introduce new alleles into the target island, a fixation occurs systemwide too. After some time, more and more of migrants’ genes will be identical to the genes of local individuals. This will also cause an average number of fixed genes in each island to increase. The new larger “building blocks” should be composed of good smaller building blocks fixed earlier. This behavior is similar to the SEAM algorithm [10]. When the number of islands, N, is larger at the cost of a smaller island size, M, the inter-island BBs competition might play a bigger role. In a special case when M = 1, fixations inside islands are created immediately and may be quite random. Such individuals would correspond to fixations of length L and instead of a gradual linkage learning we would just have a case of choosing the best one from a set of N random ones. Single runs of panmictic and IM setups on the H-IFF function were compared using previously mentioned parameters. In the next three figures, we see exemplary runs for setups with a uniform crossover. Fig. 19 shows average values of each locus in a
Fig. 19. An exemplary run with a panmictic setup, uniform crossover
Linkage in Island Models
57
Fig. 20. An exemplary run with an IM setup (all loci shown), uniform crossover
Fig. 21. An exemplary run with an IM setup (corresponding loci from all islands averaged), uniform crossover
panmictic population, throughout generations. Because genomes were of length 64, there are 64 such loci on the x-axis and 500 generations on the y-axis. The shade of each cell graphically represents the average value. We see that after initial generations, fixation quickly stabilizes the genomes and practically no change occurs afterwards. In Fig. 20 we see an analogical exemplary plot for an IM setup. Note, that each locus in each of the 10 islands is shown separately, and therefore there are 640 loci on the x axis, and the picture is squeezed horizontally. One can immmediately see that fixation
58
Z. Skolicki
Fig. 22. An exemplary run with a panmictic setup, one-point crossover
Fig. 23. An exemplary run with an IM setup (all loci shown), one-point crossover
of genes, even if occurring, is repeatedly disturbed by migrations. In this way, different configurations are tested — and, as we know from the fitness charts (e.g., in Fig. 16), such strategy leads to a better performance. In Fig. 21 corresponding loci from all islands are averaged, resulting again in 64 loci on the x axis. Under this visualization, the IM is treated as one big population, in an analogical way to the panmictic case, to which it can be very easily compared. We can see again that in the IM case the fixation of genes takes much longer and (usually) leads to longer building blocks. Fig. 22 shows a panmictic case, with the one-point crossover. It is difficult to see any significant qualitative difference in behavior compared to the uniform crossover. In
Linkage in Island Models
59
Fig. 24. An exemplary run with an IM setup (corresponding loci from all islands averaged), onepoint crossover
Fig. 23 the IM case with the one-point crossover is shown. This time, one can notice that the change of the recombination operator caused more abrupt changes after migrations. As we know, this is because the one-point crossover better matches the linkage in the H-IFF function. As a result, it copies more meaningful blocks of genes and creates better hybrids after migrations. Again, changes to the fixations occur throughout the whole run. In Fig. 24 we see each locus values averaged over islands, as before. At this level, not only we see that the changes to fixations can occur much longer in IMs than in a standard EA, but also that the gene values very gradually converge to them, thanks to the exchange of genetic material between islands. This gives times for the proper selection of gene configurations.
7 Conclusions Due to a slower pace of the global-level evolution in IMs, allele diversity is maintained much longer among sub-populations. This diversity helps later in optimizing genes that got locally fixed, often to random values. Therefore, using IMs reverts some effects of hitch-hiking. Sub-populations converge locally and evolution stalls until the next migration. Therefore, migration intervals enforce some balance between genes, because they synchronize optimization of different genes. We may say that stronger genes have to ”wait” for the weaker ones. Another property of island models is their two-level evolution. We saw that thanks to the interaction between islands, island models can better exchange building blocks. Still, for this to happen, a good recombination operator must be used. However, in contrast to a standard single-population EA, in IMs recombination operator has much more time and chance to effectively mix building blocks.
60
Z. Skolicki
Finally, a gradual and repeated fixation of genes in island models might be treated as a linkage learning process. In this process fixations in islands are treated as building blocks, because recombination preserves fixed genes in a population. With time, new fixations, with better fitness contribution would be ”learnt” but also the fixations must ultimately grow in size. Although a connection between linkage problems and IMs has been shown, more questions are posed than answered. Some of these questions might be answered based on the research in genetics. Although the role of gene fixation was partially explained by the Shifting Balance Theory [11], and by the two-level view of IMs, they do not explain completely the interaction between islands. A more thorough analysis of the relation between single migrations and linkage, possibly by extending the models used in [7], could help understanding linkage in IMs better.
References 1. De Jong, K.A.: An analysis of the behavior of a class of genetic adaptive systems. Ph.D. thesis, University of Michigan, Ann Arbor (1975) 2. De Jong, K.A.: Evolutionary Computation: A Unified Approach. MIT Press, Cambridge (2006) 3. Fern´andez, F., Tomassini, M., Vanneschi, L.: An empirical study of multipopulation genetic programming. Genetic Programming and Evolvable Machines 4(1), 21–51 (2003) 4. Goldberg, D., Korb, B., Deb, K.: Messy genetic algorithms: Motivation, analysis, and first results. Complex Systems 3(5), 493–530 (1989) 5. Gustafson, S.M.: An analysis of diversity in genetic programming. Ph.D. thesis, The University of Nottingham (2004) 6. Harik, G.R.: Learning gene linkage to efficiently solve problems of bounded difficulty using genetic algorithms. Ph.D. thesis, University of Michigan, Ann Arbor, MI, USA (1997) 7. Skolicki, Z.: An analysis of island models in evolutionary computation. Ph.D. thesis, George Mason University (2007) 8. Skolicki, Z., De Jong, K.: The importance of a two-level perspective for island model design. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2007), pp. 4623– 4630 (2007) 9. Toussaint, M., Igel, C.: Neutrality: A necessity for self-adaptation. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2002), pp. 1354–1359 (2002) 10. Watson, R.A.: Compositional Evolution: The impact of Sex, Symbiosis and Modularity on the Gradualist Framework of Evolution. Vienna Series in Theoretical Biology. MIT Press, Cambridge (2006) 11. Wright, S.: The roles of mutation, inbreeding, crossbreeding and selection in evolution. In: Jones, D.F. (ed.) Proceedings of the Sixth International Conference of Genetics, Brooklyn Botanic Garden, pp. 356–366 (1932)
Real-Coded ECGA for Solving Decomposable Real-Valued Optimization Problems Minqiang Li1,2, David E. Goldberg2, Kumara Sastry2, and Tian-Li Yu2 1
School of Management, Tianjin University, 92 Weijin Road, Nankai District, Tianjin 300072, People's Republic of China 2 Illinois Genetic Algorithms Laboratory, University of Illinois at Urbana-Champaign, 117 Transportation Building, 104 S. Mathews Avenue Urbana, IL 61801, USA
Abstract. We present the real-coded extended compact genetic algorithm (rECGA) for solving decomposable real-valued optimization problems. Mutual information between real-valued variables is employed to measure variable interaction or dependency, and the variable clustering and aggregation algorithms are proposed to identify the substructures of a problem through partitioning variables. Then, mixture Gaussian probability density function is estimated to model the promising individuals for each substructure, and the sampling of multivariate Gaussian probability density function is carried out by adopting Cholesky decomposition. Facet analyses are made about the population sizing and sampling of the factorization based on the Gaussian probability density function. Finally, experiments on decomposable test functions are conducted. The results illustrate that the rECGA is able to correctly identify the substructure of decomposable problems with linear or nonlinear correlations, and achieves a good scalability. Keywords: real-coded ECGA, probabilistic model building, real-valued decomposable optimization problem, population sizing and sampling, genetic and evolutionary computation.
1 Introduction In the field of genetic and evolutionary computation (GEC), there have been growing interests in using probabilistic distribution methods to explore the search space. The common feature of these kinds of evolutionary algorithms, called probabilistic modelbuilding genetic algorithms (PMBGAs), or estimation of distribution algorithms (EDAs), is to build probabilistic models of promising solutions found so far in the population, and generate new individuals by sampling the estimated distribution functions instead of conducting recombination and mutation operations (Pelikan et al. 2002; Goldberg 2002; Larranaga and Lozano 2002). Generally, the PMBGAs consist of three major steps: • Promising individuals are selected from the initial or current population. • A probability distribution factorization model is estimated based on the selected solutions. • Offspring are generated by sampling the estimated model, and a new population is created by using some kind of replacement policy. Most of the PMBGAs were developed to solve binary optimization problems and have achieved successful results. Only a few algorithms were proposed to deal with Y.-p. Chen, M.-H. Lim (Eds.): Linkage in Evolutionary Computation, SCI 157, pp. 61–86, 2008. © Springer-Verlag Berlin Heidelberg 2008 springerlink.com
62
M. Li et al.
real-valued optimization problems with continuous variables, such as the estimation of Gaussian network algorithms (EGNA) (Larranaga and Lozano 2002), the mixed iterated density estimation algorithms (mIDEAs) (Bosman 2003), the real-coded EDA (RECEDA) (Paul and Iba 2003), the real-coded Bayesian optimization algorithms (rBOA) (Ahn et al. 2004). There were also continuous optimization versions of the univariate marginal distribution algorithms (UMDA) (Mühlenbein 1997), the population based incremental learning (PBIL) (Baluja 1994), the mutual information maximization for input clustering (MIMIC) (De Bonet et al 1997), the bivariate marginal distribution algorithms (BMDA) (Pelikan and Mühlenbein 1999), etc. When the PMBGAs are adopted to solve decomposable real-valued optimization problems, there will appear one issue: the structure and parameters of the probability distribution model have to be estimated simultaneously when mixture Gaussian probability density functions are employed so as to minimize the scoring metric, e.g. the mostly adopted one is Bayesian information criterion (BIC). This task is a NPcomplete problem (Pelikan et al. 1998; Pelikan 2002). This issue that doesn’t exist in the binary ECGA is unique to the real-valued optimization tasks when the ECGA is employed. What we propose is to utilize a two-step procedure of building the factorization models to address this issue by employing mutual information criterion and heuristic decomposing methods. The proposed algorithm for solving decomposable optimization problems works in similar logic as the ECGA for decomposable binary optimization problems, and is called real-coded ECGA (rECGA). The remainder of this chapter is organized as follows. In section 2, we make a review of the major algorithms of the real-coded PMBGAs, and point out the critical issues associated with their application. In section 3, we introduce the two-step procedure of factorization to decompose the optimization function by using mutual information metric. Clustering methods are used to group the sample data into multiple subsets for each subproblem, and specific procedures are designed to estimate the parameters of mixture Gaussian density functions in section 4. Then experiments on test functions are conducted and the results are reported in section 5. Section 6 summarizes the paper, and topics for further research are discussed. In appendix, we make facet analyses about the population sizing and sampling of the factorization based on the Gaussian probability density function with specific functions.
2 Review of Related Work opt ( X ) , X ∈ Ω ⊆ R n , X = ( X 1 , X 2 ,… , X n ) ( X i ⊂ [ai , bi ] , ai < bi , i = 1,2, … , n ) denotes the multi-
For a global real-valued optimization function
dimensional continuous random variables, and
x = ( x1 , x2 ,…, xn ) represents one
of its possible instances or a solution. A joint probability density function is estimated over the selected individuals by a probabilistic model M (ς , θ ) , where ς is the structure of the model that describes the factorization of the probability density
Real-Coded ECGA for Solving Decomposable Real-Valued Optimization Problems
63
function (pdf), θ is the parameter vector for all components of the joint pdf. In general, a joint pdf f ( X ) is factorized as (Larranaga and Lozano 2002): n
f (ς ,θ ) ( X ) = ∏ f ( X i | Π i , θi ) ,
(2-1)
i =1
Π i is a set of variables that X i is dependent on, θi is the parameter vector for the conditional pdf of X i . Given a set of data samples S = { x1 , x2 , … , xq } , the
where
Bayesian information metric (BIC) is used to score the fitness of the joint pdf model: q
BIC ( f (ς ,θ ) ( X ), S ) = − ∑ ln( f (ς ,θ ) ( x j )) + λ × ln( q)× | θ | ,
(2-2)
j =1
where the first part is the likelihood metric of the model, λ is the penalty of the model complexity. It is assumed that the variable vector X = ( X1, X2,…, Xn ) of a problem is able to be partitioned into m sub-vectors: X , X ,…, X , then we say that the problem or total structure can be decomposed into a set of subproblems or substructures. Suppose that set(X) = {X1, X2,…, Xn} indicates the set of all variables, a partition is repre1
2
m
m
sented
as
set(Xk ), set(X2),…, set(Xm) ,
set(X) = ∪set(Xk ) ,
and
k =1
set(X ) ∩ set(X ) =∅ ( k1 ≠ k2 ); k1
k2
{X1k , X 2k ,…, X mk k } ; tion of
m
∑m k =1
k
X = (X , X ,…, X ) , k
k 1
k 2
k mk
set(X k ) =
= n , k = 1,2,…, m . Since the real distribution func-
f ( X i , Π i | θi ) may be multimodal or nonlinear, a single Gaussian pdf is not
able to capture the data feature for a substructure. So, mixture Gaussian pdf should be adopted, and the factorization will be a joint pdf as: m
mk
ck , i
f (ς ,θ ) ( X ) = ∏∏∑ β k ,i , j f ( X ik | Π k ,i , θk ,i , j ) ,
(2-3)
k =1 i =1 j =1
where
β k ,i , j
ck , i
∑β j =1
k ,i , j
is the weight for the mixture Gaussian pdf of the variable
X ik , and
= 1 ; ck ,i is the cluster number of data samples regarding X ik and its pa-
rental variables, and is also the number of Gaussian pdfs for estimating
f ( X ik | Π k ,i , θk ,i , j ) . The rBOA (Ahn et al. 2004) used a factorization with mixture Gaussian pdf. It adopted an identical number of mixture Gaussian pdf for a subproblem. An incremental
64
M. Li et al.
greedy algorithm was used to find the optimal acyclic Bayesian network by starting from an empty one based on the BIC metric. Then maximally connected sub-graphs, or building-blocks with tight interaction between variables, were extracted by graph partition algorithms. The mIDEAs tried a different way to do factorization (Bosman 2003). It divided the problem in solution space instead of decomposing the objective function. The sample data of the selected individuals were clustered into multiple groups by leader algorithm, and the joint pdf for each group was modeled with a factorized Gaussian pdf. The mIDEAs resorted to the clustering procedure to break the nonlinearity or interaction of variables. Since it did not consider the decomposability of a high dimensional objective function, a huge population and numerous clusters are often required to do effective factorization. Moreover, it could not realize the probabilistic crossover of building-blocks (Ahn et al. 2004) because there was no cross breeding between different clusters. The RECEDA by Paul and Iba (2003) employed a multivariate Gaussian probability density function to model the selected individuals, which was quite similar to the estimation of the multivariate normal algorithms (EMNAs) (Larranaga and Lozano 2002). The RECEDA and EMNAs were efficient for unimodal or macroly unimodal optimization functions of low dimension with only linear dependency between variables. For complicated high dimensional optimization functions with multimodality and nonlinear interaction, they usually failed to find the global optimum. Therefore, the correct factorization of a real-valued optimization function is critical to the design of an effective algorithm. But for complicated and large scale realvalued optimization problems, we need an efficient way to do the factorization. Although the Bayesian network based on mixture Gaussian pdf is able to model various types of dependences and interactions between real-valued variables, it is pretty difficult to find the optimal structure and parameters. We have to minimize the BIC over m , ck ,i , β k ,i , j concurrently in formula (2-3) if we want to get the optimal factorization of the joint pdf, and the computation would be overwhelming on high dimensional optimization problems. Actually, what we find by heuristic methods is an approximate solution when we use the Bayesian network to model the problems structure in reality. So, we propose to divide the whole factorization procedure into two steps, and make use of proper metric measures instead of the BIC. The first step attempts to find the substructures of the problem through learning the linkage information or interaction between variables by using regular measures as mutual information, and the global structure of the problem is then decomposed into multiple substructures by partitioning the real-valued variables into multiple groups with regard to an objective function. The second step clusters the selected individuals into multiple exclusive groups regarding specific variable partitions or substructures, and then builds a mixture multivariate Gaussian pdf for each substructure. Thus, we can build a joint pdf with mixture Gaussian pdf which is a good factorization of the joint pdf for a decomposable real-valued optimization problem as: m
ck
f (ς ,θ ) ( X ) = ∏∑ β k , j f ( X k , θk , j ) , k =1 j =1
(2-4)
Real-Coded ECGA for Solving Decomposable Real-Valued Optimization Problems
65
which is a simplified form of the (2-3) for decomposable real-valued optimization problems. θk , j is the parameter for the j-th Gaussian pdf of the variable partition
X k in the mixture model. 3 Structure Decomposition of Real-Valued Optimization Function The structure decomposition of a real-valued optimization function is similar to the binary ECGA which uses the product of marginal distribution to model the joint pdf, where binary variables were partitioned into multiple clusters by a greedy search algorithm (Harik 1999). As to the global real-valued optimization problems, there are a lot of approaches to measure the dependency of real-valued variables in statistics and information theory (MacKay 2003). Tsutsui et al. (2001) adopted the piecewise interval correlation coefficient to measure the nonlinear correlation of variables, but it required the correct splitting of the solution space. We use the mutual information to measure the dependency among real-valued variables. The basic procedure on the partition of real-valued variables based on mutual information is presented as below. 3.1 Mutual Information of Real-Valued Variables In order to use mutual information to measure the dependency of continuous variables based on the selected individuals, we need to discretize the real-valued variables in advance. Suppose that the definition domain of a real-valued variable X∈[a, b] is diL
vided into
L partitions:
∪[a , b ] = [a, b] ( a l
l
l
< bl ), i = 1,2,…, n . The mutual in-
l =1
formation between two random variables X i , X j is defined as (MacKay 2003):
I ( Xi; X j ) = H ( Xi ) − H ( Xi | X j ) , where
(3-1)
H ( X i ) is the entropy of X i , H ( X i | X j ) is the conditional entropy of X i
given X j , and I ( X i ; X j ) = I ( X j ; X i ) . It measures the average reduction in uncertainty about
X i that results from learning the value of X j , that is the average
amount of information that
X i conveys about X j ; or vice versa (MacKay 2003).
By the definition of entropy, we get:
I ( Xi; X j ) =
P( xi , x j )
∑ P( x , x ) log P( x ) P( x ) , i
xi , x j
j
i
j
(3-2)
66
M. Li et al.
where I ( X i ; X j ) ≥ 0 , and I ( X i ; X j ) = 0 only when X i , X j are independent. Usually, we scale the mutual information by I ( X i ; X j ) / log( L ) , so that
I ( X i ; X i ) / log( L) ≤ 1 . Thus, we get the mutual information matrix for the continuous vector
X = ( X 1 , X 2 , …, X n )T of a real-valued optimization function: I ( X 1 , X 2 , … , X n ) = [I ( X i ; X j ) ] n ×n ,
(3-3)
which is a symmetrical matrix. 3.2 Partition of Real-Valued Variables The partition of variables is a typical unsupervised learning task in machine learning, and there are lots of algorithms to complete this task. For binary optimization problems, the minimum description length (MDL) was computed as the clustering metric to measure the partition of genes, and evolutionary algorithms or heuristic methods were adopted to search for the best partition by minimizing the metric (Yu et al. 2003). Theoretically, unsupervised machine learning tasks require fixing specific parameters a priori for a problem to yield rational and acceptable results. So, we design a heuristic greedy search algorithm to cluster variables based on their mutual information, which can easily incorporate the expertise of human experts interactively. Initially, we take the interaction between all variables as insignificant or all variables as independent if max I ( X i ; X j ) < α , where α is the threshold i , j =1, 2 ,..., n ;i ≠ j
{
}
parameter for the lowest bound of mutual information, and α = 0 .1 as default. Otherwise, a clustering procedure is implemented to partition the variables into multiple subsets. The major steps of the clustering algorithm is described as follows. Step 1: Sort the upper triangular entries of the mutual information matrix I ( X 1 , X 2 , … , X n ) into an entry list of mutual information of pair-wise
{
}
variables: I sort = I ( X i ; X j )
n ( n −1)/ 2
, where we do not consider diagonal
entries. Step 2: All entries are fetched in descending order. Pick the biggest one I ( X i ; X j ) ,
{ }
and place the two variables into cluster 1: C1 = X i1 2 . Step 3: Pick the next entry I ( X i ; X j ) in
{ }
K clusters: C1 = X i1
c1
I sort ; and suppose that currently we have
{ }
, C 2 = X i2
c2
{ }
, … , C K = X iK
ck
, where
ck ( k = 1,2,…, K ) is the number of variables in cluster k , then we proceed by:
Real-Coded ECGA for Solving Decomposable Real-Valued Optimization Problems
67
(1) if one of the pair-wise variables in I ( X i ; X j ) has been clustered:
X i or X j ∈ C k
,
the
other
is
placed
in
the
Ck ← Ck ∪ { X i , X j } ;
same
cluster:
{ }.
(2) otherwise, a new cluster is created: K ← K + 1 , C K = X iK
2
Step 4: Go to step 3, and repeat until all entries in
I sort are tested, and all variables
are clustered. Step 5: Output clusters: C1 = X i1
, … , C K = X iK
{ } , C = {X }
2 i c2
2
c1
{ }
cK
.
This algorithm usually outputs a lot of small clusters that have only a few variables. The partition is efficient for evidently decomposable optimization problems if the dependency between some variables is much stronger than that of others. But it would break the dependency between some variables when there is not a big difference among the mutual information of highly correlated ones. So, we suggest an aggregation algorithm to merge some clusters, which is implemented together with the clustering algorithm as described above. The major steps of the aggregation algorithm is described as follows.
{ } , C = {X }
Step 1: For clusters C1 = X i1
2
c1
2 i c2
{ }
, … , C K = X iK
cK
, the averaged
mutual information is calculated: ck 2 I (C k ) = I ( X ik ; X kj ) . ∑ ck ( ck − 1) i =1, j >i
Step 2: Find the mutual correlation between any pair of two clusters by:
I (C k1 , C k 2 ) = min {I ( X ik1 ; X kj 2 ) | X ik1 ∈ C k1 , X ik 2 ∈ C k2 }.
Step 3: Sort the mutual correlation in descending order into a list:
I clusters, sort = {I (C k1 , C k2 )}
K ( K − 1)/ 2
.
Step 4: Select the first mutual correlation in I clusters ,sort :
{
}
If I (Ck1 , Ck2 ) > βmerge × max I (Ck1 ), I (Ck2 ) , then merge clusters Ck1 ,Ck2 ,
β merge ( 0 < β merge < 1 ) is a threshold parameter, and β merge = 0.80 for default. The smaller the β merge , the fewer the clusters yielded. where
Step 5: Go to step 4, and repeat until all of the entries in I clusters ,sort are tested, or all clusters pairs are checked.
{ }
{ }
Step 6: Output clusters: C1 = Xi1 c , C2 = X i2 1
c2
{ }
, … , CK ' = XiK′
cK′
,
K′ ≤ K .
In real-world applications, the aggregation procedure can be repeated multiple times until there is no chance of merging any pair of clusters.
68
M. Li et al.
With the clustering algorithm and aggregation algorithm (called the partition algorithm of real-valued variables), the total real-valued variables are partitioned into exclusive clusters, so that the structure of the optimization problem is decomposed into multiple substructures. The final output of the partition algorithm is similar to the results of the rBOA (Ahn et al. 2004), but the procedure is much more efficient. It is fitted particularly to the case where the optimization function is decomposable, especially when there is significant difference between the mutual information values of closely correlated variables and loosely correlated variables. Although it employs a heuristic method to do partition, the approximate factorization is sufficiently good in practice.
4 Estimation and Sampling of Mixture Gaussian Pdf After the structure of an optimization function is decomposed, we proceed to build the mixture Gaussian pdf for each substructure. The identical sample data of the selected individuals are shared, only different coordinate values are used in estimating parameters for a substructure. Let’s consider the substructure k
X k ( k =1,2,…,m), | set(X k ) |= mk . Suppose that k
entries of X are nonlinearly correlated, and the function regarding X is multimodal. A single Gaussian pdf, a unimodal function to model linear correlation, is not able to model the samples. There are a lot of methods for complicated data clustering tasks (Han and Kamber 2006). The mIDEAs used the leader algorithm (Bosman 2003) which was fast but preferred yielding fewer clusters or nonlinear clusters. We propose to incorporate the density-based clustering method (DBSCAN) (Ester et al. 1996) and k-means clustering method to partition the sample data into multiple groups. The DBSCAN is used firstly to divide the data samples into multiple clusters that may probably be nonlinearly separable, and get the initial cluster number which is represented as ck ,min . Then we choose a cluster number for linearly separable samples, and fit a Gaussian pdf to each cluster of data samples. Empirically, we take ck ∈[ck ,min, ck ,max] ,
ck ,max = ck ,min × βcluster(1 + ⎣ mk ⎦) , where βcluster ≥ 2 is a regulating parameter that
controls the number of clusters to be created. Finally, the k-means algorithm divides the data samples into ck groups. Given a set of samples (l
S = { x1 , x2 , … , xq } , xl = ( xl1 , xl 2 ,…, xl n )
= 1,2,…, q ), we project it on X k , and thus get: S k = {x1k , x2k ,…, xqk } ,
xlk = ( xl1 , xl 2 ,…, xlmk ) . In the rECGA, q = τN , N denotes the population size, and τ ( τ = 0.5 ) indicates the proportion of promising individuals selected for building probabilistic models.
Real-Coded ECGA for Solving Decomposable Real-Valued Optimization Problems
Suppose that we have clustered samples into partition:
ck groups regarding the k-th variable
S1k = {x1k , x2k ,…, xqkk } , S2k = {x1k , x2k ,…, xqkk } , …, Sckk = {x1k , x2k ,…, xqkk } , 1
ck
∑q
k j
69
2
ck
=q.
j =1
For a sample cluster
S kj = {x1k , x2k ,…, xqkk } ( j = 1,2,…, ck ), the mean vector j
and
covariance
matrix
μkj = ( μkj,1, μkj,2 ,…, μkj,mk )T 1 μ = k qj k j ,i
q kj
∑x
k li
for
multivariate
[
Gaussian
Σ kj = cov j ( X ik1 , X ik2 )
,
]
pdf
are
estimated
,
mk ×mk
as:
where
i = 1,2,…, mk , covj ( Xik1 , Xik2 ) is the covariance of variables
,
l =1
X , X in partition k over sample cluster S kj ( j = 1,2,…, ck ). k i1
k i2
Thus, we get the factorization with mixture Gaussian pdf for a decomposable realvalued optimization problem: ck
m
f (ς ,θ ) ( X ) = ∏ ∑ β k , j × f ( X k | μkj , Σ kj ) ,
(4-1)
k =1 j =1
where
β k , j = qkj q .
The multivariate Gaussian pdf in a substructure can be sampled by the Cholesky decomposition and algebra transformation (Anderson 2003). Since the covariance matrix
Σ = [cov( X i, X j)] r×r ( r = q kj ) of a multivariate Gaussian pdf is symmetric
and positive definite, it can be decomposed into a lower and an upper triangular matrix:
⎡ a11 ⎢ a21 T ∑ = L ×L , ∑ = ⎢ ⎢ ⎢ ar1 ⎣ where lii = aii −
i −1
∑l
2 ik
a12 a22 ar 2
, l ji = (a ji −
i −1
∑l
…
⎡ l11 ⎤ ⎢ l21 … a2 r ⎥ ⎥ ,L = ⎢ 0⎥ ⎢ ⎢l ⎥ … arr ⎦ ⎣ r1 a1r
0
…
l22
…
0⎤ 0⎥
⎥,
0⎥
lr 2
…
lrr ⎥
⎦
l ) lii ; i = 1, 2, …, r; j = 1, 2, …, i .
jk ik
k =1
k =1
The sample data of a random vector variable Z = ( z1, z2 , … , zn ) that obeys standard
multivariate
Gaussian
distribution,
Z ~ N (0 , 1) ,
S = {z1 , z2 ,…, zs } , zi = ( zi1 , zi 2 ,… , zir ) , i = 1,2,…, s . Z
are
produced:
70
M. Li et al.
Then we get the samples S
X
= { x1 , x2 ,…, xs } of the random vector variable
X that obeys the linearly correlated multivariate Gaussian distribution X ~ N ( μ, Σ ) by transformation: xi = μ + Lzi′ , i = 1,2,…, s , where
(4-2)
xi is an instance of the random multidimensional variable X . For mixture
Gaussian pdf in the k-th variable partition, the prior probability for choosing a Gaussian pdf of the j-th cluster is decided by βk , j . Thus, the offspring can be generated from the joint mixture Gaussian pdf based on formula (4-1), and the dependencies between variables that have been modeled are inherited to the newly produced individuals. The most important is that multivariate building-blocks are preserved and mixed in the sampling of offspring, so there is a good probability to create higher-order building-blocks and to approach asymptotically the global optimum of a real-valued optimization problem.
5 Experiments 5.1 Test Functions
We mainly consider the functions without dependency or with dependency between only two variables, and test functions are designed as follows. (1) Multivariate real-valued deceptive function (MRDFi) This function is defined with no dependency between variables: n
max f MRDFi ( x ) = ∑ fURDF ( xi ) , i =1
f URDF
⎧ a (c − x ) ⎪⎪ c (x) = ⎨ − c) ( b x ⎪ ⎪⎩ 1 − c
xi ∈ [0,1] , i = 1,2,…, n , 0≤ x < c
,
(5-1)
x ∈ [0,1] ,
c≤ x ≤1
where {a , b, c} controls the shape of the univariate real-valued deceptive function (URDF), and a< b , 0.5
xattractor=1 ( f ( xattractor) = 1 ). For the MRDFi, X* = (1,1,…,1) , f ( X* ) = nb ,
Xattractor = (0,0,…,0) , f ( X attractor ) = na . There are totally 2n −1 local optima, and 2n − 2 are partially deceptive attractors.
Real-Coded ECGA for Solving Decomposable Real-Valued Optimization Problems
71
(2) Multivariate real-valued deceptive function with dependency (MRDFd) m
max f MRDFd ( X ) = ∑ f trap −2 d ( xi ,1 , xi , 2 ) , xi , j ∈ [0,1] , j = 1,2 ,
(5-2)
i =1
2 ⎧ b × ( ∑ xi2, j − 1 ) ⎪ j =1 ⎪ 2 −1 f trap − 2 d ( xi ,1, xi , 2 ) = ⎨ 2 ⎪ a × (1 − xi2, j ) ∑ ⎪ j =1 ⎩
2
if
∑x j =1
2 i, j
≥ 1 .0
,
otherwise
where n = 2m , and {a, b} are the parameters to control the shape of 2-d trap function and also its deceptiveness, and {a = 0.8, b = 1.0} is used in experiment.
X* = (1,1,…,1) , f ( X *) = m . (3) Concatenated Rosenbrock function (cRosenbrock): m
min f ( X ) = ∑ f R ( xi ,1 , xi , 2 ) , xi , j ∈ [−5.12,5.12] , j = 1,2 ,
(5-3)
i =1
f R ( xi ,1 , xi ,2 ) = 100 × ( xi , 2 − xi2,1 ) 2 + (1 − xi ,1 )2 , where is n = 2m . X * = (1,1,… ,1) , f ( X *) = 0 . This function was used to test the rBOA by Ahn, et al (2004). 5.2 Experimental Results and Discussions
Considering the stochastic nature of the rECGA, we compute the average performance of one hundred independent executions of the algorithm on tested functions. The truncation selection takes τ = 50% in the rECGA. (1) MRDFi The mutual information based variable partition algorithm finds that all variables are independent and outputs substructures X1 , X2 ,…, Xn . In order to testify that the
correct partition of the variables is very important to find the substructures or building-blocks of a problem, experiments are firstly conducted to illustrate the impact of different partitions of variables on the solution quality yielded by the rECGA with a population size of 1000 on the MRDFi (Fig. 1(a)). The rECGA with global independency which can be detected by the proposed partition algorithm is able to discover correctly the substructure of the MRDFi, and can find quite robustly the global optimum with a 2-clusters mixture Gaussian pdf. When we do not carry out problem decomposition, all variables are treated as totally correlated. For this global dependency, a 2-cluster mixture Gaussian pdf model is built on the selected individuals, and the rECGA definitely fails to find the global optimum. When pair-wise dependency is assumed, neighborhood variables are grouped into substructures. The final solutions output by the rECGA with pair-wise dependency
72
M. Li et al.
are somewhat better than those of the rECGA with global dependency. Since there are 4 ( 2 = 4 ) local optima for each substructure, the rECGA could not find the global optimum in all runs if clusters are less than the number of local optima. In order to estimate the scalability of the rECGA on the MRDFi in terms of the mean number of fitness (function) evaluations (MNFEs) with problem size, we select the smallest population sizes for different dimensions based on mutual information between variables. The 100% success rate is assumed for all rECGA specifications. The smallest population sizes are determined when the difference of mutual information between any pair of variables is less than 10% of the averaged. The smallest population sizes are calculated as N = 20n + 300 for the rECGA with global independency, and N = (20n + 300) × 2 for the rECGA with pair-wise dependency. Since 2
n
the multimodality ( 2 ) increases exponentially along with the dimension of the MRDFi, the number of Gaussian pdf needed to construct the factorization model of joint pdf over selected individuals also increases quickly. We test the 12 clusters Gaussian pdf for the rECGA with the global dependency. The experimental results are shown in Fig. 1(b). 1.10 1.00 yt 0.90 lia uq 0.80 no it ul 0.70 oS
rECGA with global independency rECGA with global dependency rECGA with pair-wise dependency
0.60 0.50
5
6
7 8 9 10 11 Problem size (dimension)
12
Fig. 1. (a) Solution quality of rECGA on MRDFi with three schemes of variable partition (compared with the global optimum) 3000000
rECGA with global dependency rECGA with global independency rECGA with pair-wise dependency
2500000
180000 160000 140000
2000000
120000 s E 100000 F N M 80000
EsF 1500000 MN 1000000
60000 40000
500000 0
200000
20000 5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 Problem size (dimension)
0
Fig. 1. (b) Scalability of rECGA with different variable partition schemes on MRDFi (rECGA with global dependency is measured on left vertical axis)
Real-Coded ECGA for Solving Decomposable Real-Valued Optimization Problems
73
The scalability of the rECGA with global dependency, pair-wise dependency, and no dependency are estimated as 5-order, 2.1-order, and 1.45-order polynomials respectively. If the factorization is built on incorrect problem decomposition as in the rECGA with global dependency or pair-wise dependency, the rECGA experiences poor performance. The mean number of fitness evaluations required for the rECGA with correct variable partition to solve the MRDFi with global optimality grows subquadratically as the problem size increases, which illustrates the good scalability of the rECGA on the MRDFi. The rECGA achieves good performance on the MRDFi because the probabilistic building-blocks recombination works efficiently based on the estimated joint distribution of the selected individuals when the dependency between variables is correctly captured. In addition, compared with the simple GAs, the premature convergence of population is controlled very effectively due to the preservation of real-valued building-blocks, and the genetic drift towards local optima is curbed by the mechanism of the probabilistic modeling and sampling. (2) MRDFd When the MRDFd is used to test the rECGA, it is required that all of the variable dependencies are identified correctly in the first generation with a statistical confidence of 97%. A 2-cluster mixture Gaussian pdf is used to estimate the joint probabilistic distribution model based on the correct identification of substructures. The minimum population sizes, mean generations of convergence (MG), and mean number of fitness evaluations (MNFEs) are displayed in Fig. 2. 9000 8000 zei 7000 s 6000 no 5000 it al 4000 up 3000 oP 2000 1000 0
s E F N M
2
3
4
5 6 7 8 Number of BBs
9
10
160000 140000 120000 100000 80000 60000 40000 20000 0
2
3
4 5 6 7 8 Number of BBs
9
10
Fig. 2. (a) Minimum population sizes of Fig. 2. (b) Mean number of fitness evaluarECGA required on MRDFd tions of rECGA on MRDFd
The population size grows in quadratic polynomial with the number of buildingblocks (or problem size) in order to capture all pair-wise variable dependency based on mutual information. After substructures of the MRDFd are discovered correctly by the variable partition algorithms, the factorization of the joint distribution of selected individuals is modeled. The rECGA captures building-blocks information of the MRDFd very accurately by employing a 2-clusters mixture Gaussian pdf. The computation of the rECGA in terms of MNFEs to find the global optimum increases in 2.4 nearly 3-order polynomial with problem size ( O(n ) ). If the variables are not correctly partitioned, the rECGA could not surely find the global optimum when a 2-clusters mixture Gaussian pdf is employed. When a big
74
M. Li et al.
number of clusters mixture Gaussian pdf is adopted, the scalability of the rECGA without exact variable partition is of high order polynomial or nearly exponential. (3) cRosenbrock function The cRosenbrock function has some particular characteristics. Although it is a unimodal function that has only one optimum, the landscape around the global optimum is quite different from that on the total domain. Empirically, a huge population and a quite lower percentage of truncation selection is required if we want to find the correct interaction between variables based on the mutual information measure in the first generation. However, the rECGA can identify the variable dependency after some number of generations when the local structure around the global optimum is discovered gradually. The experimental results are shown in Fig. 3(a), which shows the mean generations to discover the local structure. 35 Population size: 1000 Population size: 2000
30 s n o 25 i t a 20 r e n e g 15 n a 10 e M 5 0
2
3
4
5 6 7 Number of BBs
8
9
10
Fig. 3. (a) Mean generations required to correctly identify the variable dependency on cRosenbrook function
)g lo ni ( yt il au q no it ul oS
2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0
RCEDA: single RCEDA: mixture
2
3
4
5 6 7 Number of BBs
8
9
10
Fig. 3. (b) Mean objective function values of rECGA on cRosenbrook
In order to test the performance of the rECGA on the cRosenbrock function, we prefer a smaller population size and a longer generation of evolution. Here, the population size is set as N = 1000 , and maximum generations is taken as Tmax = 30 × number of BBs . We also test the performance of the rECGA with a single Gaussian pdf (no clustering) and a 2-clusters mixture Gaussian pdf for each substructure of the problem. The mean function values (MFV) of the best individual found by the rECGA with regard to different problem size are illustrated in Fig. 3(b).
Real-Coded ECGA for Solving Decomposable Real-Valued Optimization Problems
75
Comparatively, the rECGA with mixture Gaussian pdf performs much better than the rECGA with single Gaussian pdf although the subproblem is a unimodal function. The rECGA achieves a linear scalability in terms of solution quality when maximum fitness evaluations grow linearly.
6 Conclusions This chapter presents the real-coded ECGA for solving decomposable real-valued optimization problems. The mutual information between real-valued variables is employed to measure variable correlation or dependency, and the heuristic clustering and aggregation algorithms are developed to partition variables. It is much more efficient to implement, and can easily incorporate the expertise of human experts interactively in real-world applications. Experiments on decomposable test functions are conducted. The rECGA achieves a good scalability on decomposable real-valued optimization problems. There is still much work to be done in developing new methods for identifying the correlation or dependency of continuous variables, which will surely benefit the probabilistic modeling of the rECGA. Hybridization of the rECGA with local searching techniques seems always to be a promising and feasible direction to enhance the efficiency of the algorithm, and general purpose or particular designed techniques should be explored. Acknowledgments. The work was supported by the National Science Foundation of China (Grant No. 70571057) and by the Program for New Century Excellent Talents in Universities of China (NCET-05-0253). The authors are very grateful to two reviewers whose invaluable comments and suggestions substantially helped in improving the quality of the paper.
References Ahn, C.W., Ramakrishna, R.S., Goldberg, D.E.: Real-coded Bayesian optimization algorithm: Bringing the strength of BOA into the continuous world. In: Deb, K., et al. (eds.) GECCO 2004. LNCS, vol. 3102, pp. 840–851. Springer, Heidelberg (2004) Anderson, T.W.: An introduction to multivariate statistics, 3rd edn. Wiley, New York (2003) Baluja, S.: Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning. Technique Report, No. CMU-CS-94163, Carnegie Mellon University, PA: Pittsburgh (1994) Bosman, P.A.N.: Design and application of iterated density-estimation evolutionary algorithms. Ph.D. thesis, Universiteit Utrecht, Utrecht, The Netherlands (2003) De Bonet, J.S., Isbell, C.L., Viola, P.: MIMIC: Finding optima by estimating probability densities. In: Mozer, M.C., Jordan, M.I., Petsche, T. (eds.) Advances in Neural Information Processing Systems, vol. 9, pp. 424–430. MIT Press, Cambridge (1997) Ester, M., Kriegel, H.-P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: Simoudis, E., Han, J.W., Fayyad, U.M. (eds.) Proceedings of KDD 1996, pp. 226–231. AAAI Press, Menlo Park (1996) Goldberg, D.E.: Design of innovation: lessons from and for competent genetic algorithms. Kluwer Academic Publishers, Boston (2002)
76
M. Li et al.
Han, J., Kamber, M.: Data mining: concepts and techniques, 2nd edn. Morgan Kaufmann Publishers, Los Altos (2006) Harik, G.: Linkage learning via probabilistic modeling in the ECGA. University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL, IlliGAL Report No. 99010 (1999) Larranaga, P., Lozano, J.A.: Estimation of distribution algorithms, a new tool for evolutionary computation. Kluwer Academic Publishers, Boston (2002) MacKay, D.J.C.: Information theory, inference, and learning algorithms. Cambridge University Press, Cambridge (2003) Mühlenbein, H.: The equation for response to selection and its use for prediction. Evolutionary Computation 5(3), 303–346 (1997) Paul, T.K., Iba, H.: Real-coded estimation of distribution algorithm. In: Ibaraki, T., Yoshitomi, Y. (eds.) Proceedings of the Fifth Metaheuristics International Conference (MIC 2003), Kyoto, Japan, August 25–28 (2003) Pelikan, M.: Bayesian optimization algorithm: From single level to hierarchy. A dissertation from the University of Illinois at Urbana-Champaign, Urbana, IL, IlliGAL Report No 2002023 (2002) Pelikan, M., Mühlenbein, H.: The bivariate marginal distribution algorithm. In: Roy, R., Furuhashi, T., Chawdhry, P.K. (eds.) Advances in Soft Computing: Engineering Design and Manufacturing, pp. 521–535. Springer, Heidelberg (1999) Pelikan, M., Goldberg, D.E., Lobo, F.: A survey of optimization by building and using probabilistic models. Computational Optimization and Applications, 21(1), pp. 5–20 (Also IlliGAL Report No. 99018) (2002) Pelikan, M., Goldberg, D.E., Cantú-Paz, E.: Linkage problem, distribution estimation, and Bayesian networks. Evolutionary Computation, 8(3), pp. 311–340 (Also IlliGAL Report No. 98013) (1998) Tsutsui, S., Goldberg, D.E., Sastry, K.: Linkage learning in real-coded GAs with simplex crossover. In: Collet, P., et al. (eds.) Proceedings of the 5th International Conference on Artificial Evolution, pp. 73–84. Springer, Heidelberg (2001) Yu, T.-L., Goldberg, D.E., Yassine, A., Chen, Y.-P.: A genetic algorithm design inspired by organizational theory: a pilot study of a dependency structure matrix driven genetic algorithm. In: Dagli, C.H., et al. (eds) Proceedings of ANNIE 2003, vol. 13, pp. 327–332. ASME Press, New York (Also IlliGAL Report No. 2003007) (2003)
Appendix Up to now, there has been no systematic investigation on the performance of the Gaussian pdf based factorization for decomposable real-valued optimization problems. As to the rECGA, the new searching mechanism makes the theoretical analyses of its behavior more complicated. Recently, Ahn (2005) studied the population complexity of the rBOA, and proposed the population size bounds for deciding correctly the adding or deleting of edges between variable nodes with regard to the BIC metric in building the Bayesian network model of promising solutions. He found that the population size for discovering the real-valued variable dependency of a problem depends on the product of the problems size, the order of the sub-problems, and the number of mixture components. Since the rECGA replaces the conventional crossover and mutation operations with the building and sampling of probabilistic models, the functionalities of the population size and the sampling are coupled together. When mixture Gaussian pdf is used in modeling the selected individual at the substructure level, the correct selection of
Real-Coded ECGA for Solving Decomposable Real-Valued Optimization Problems
77
the number of mixture components is critical to the success of the probabilistic building-blocks recombination although the problem substructures can be identified correctly. Therefore, we intend to study the relation of population sizing and sampling to the performance of the rECGA on specific problems. This appendix includes three subsections. The population sizing and sampling of rECGA with a single Gaussian pdf on URDF and MRDFi are investigated in the first and second subsections. In the third subsection, the population sizing and sampling of rECGA with mixture Gaussian pdf on MRDFi is characterized. A1 Schematic Analyses on URDF
We make the following assumptions when we use the rECGA to solve the URDF: (a) the initial population is randomly generalized over [0,1]; (b) 50% individuals are selected in the truncation selection; (c) an univariate Gaussian density function is estimated to learn the probability density function of selected individuals, and the sampling is done based on the univariate Gaussian models. The truncation selection in the rECGA will produce individuals located in subsets
[0, x1* ) or ( x2* ,1] in the first generation (see Fig.A1.1). On the contrary, the indi* * viduals in subset [ x1 , x2 ] will be discarded definitely. There are following facts about the distribution of the selected individuals on the URDF. f(x) b a
0
x1*
c
x2*, x3*
1 x
Fig. A1.1. Schematic illustration of the URDF
x2* − x1* = 0.5 and
fURDF ( x1* ) = fURDF ( x2* ) . Then we 1 A b c 1 1 * get x1 = c − , and x2* = c + , where A = × , and A = 5 , a 1− c 2 A +1 2 A +1 x1* ≈ 0.3833 , x2* ≈ 0.8833 when {a , b, c} takes default values. We can derive There
exists
another important point
x3* = (a/b)(1− c) + c (see Fig.A1.1) from fURDF( x3* ) = a ,
where x3 = 0.96 with default parameters. *
78
M. Li et al.
A2 Population Sizing and Sampling of rECGA with a Single Gaussian Pdf on URDF
Now, we start to do some facet analyses about the performance of the rECGA in association with the population initialization, probabilistic model building, and sampling on the URDF. A population is denoted as P = {x1 , x2 ,…, xN } . The rECGA fails to obtain the
[0, x2* ) , * and the sampling could hardly produce an individual that belongs to ( x3 ,1] with a
global optimum with a finite population size if all individuals fall in subset
pretty small variance based on the estimated Gaussian pdf model. First, it is assumed that the population finds the global optimum when there are individuals locating in the subset ( x3 ,1] , e.g. ∃x ∈ P , x ∈ ( x3 ,1] . The probability of *
*
*
producing one individual belonging to ( x3 ,1] in initialization of the population is calculated as:
p( x ∈ ( x3* ,1]) = 1 − x3* .
(A2-1) *
The probability that there is no individual locating in ( x3 ,1] after the completion of the population initialization is described as:
p(∀x ∈ P, x ∉ ( x3* ,1]) = ( x3* ) N . Suppose that the global optimum is successfully found in initialization in terms of there exist statistical confidence level 1− α1 (0 < α1 < 1) ,
1 − p (∀x ∈ P, x ∉ ( x3* ,1]) ≥ 1 − α1 . We get the inequality: N (α1 ) ≥
log α1 , log x3*
(A2-2)
N (α1 ) ≥ 74 is the lower bound for population sizing with x3* = 0.96 , α1 = 0.05 . With the population size N, the estimated failure rate to produce the
where
global optimum in initialization is:
α1* = ( x3* ) N .
(A2-3) *
Second, we consider the case that there is no individual locating in subset ( x3 ,1] *
*
in initialization, but there are some individuals locating in ( x2 , x3 ] which will survive the truncation selection. Suppose the intermediate population yielded by the truncation selection is denoted as Pselection = { x '1 , x '2 ,…, x ' N / 2 } , the survived individuals that are used to estimate the single Gaussian pdf are separated into two parts:
x '∈ [0, x1* ) , or x '∈ ( x2* , x3* ] . The averaged numbers of individuals are
Real-Coded ECGA for Solving Decomposable Real-Valued Optimization Problems
calculated as
79
o1 = x1* × N / 2 , and o2 = ( x3* − x2* ) × N / 2 . Now, the mean and
variance of the selected individuals are estimated as:
μˆ =
where
o1 x* o2 x * + x 2* 1 ( x1* ) 2 + ( x 3* ) 2 − ( x 2* ) 2 , (A2-4a) × 1 + × 3 = o1 + o 2 2 o1 + o 2 2 2 x1* + x3* − x 2*
s2 =
o1 o1 + o 2
=
* 1 * 3
x x + x − x 2* * 1
μˆ = 0.3134
σˆ = 0.2634 ) pdf,
2
⎛ x1* ⎞ o2 ⎜⎜ − μˆ ⎟⎟ + o 2 1 + o2 ⎝ ⎠
the
,
⎛ x 3* + x 2* ⎞ ⎜⎜ − μˆ ⎟⎟ 2 ⎝ ⎠
2
⎞ ⎛x x −x ⎜⎜ − μˆ ⎟⎟ + * x1 + x 3* − x 2* ⎠ ⎝ 2 * 1
* 3
* 2
2
,
⎞ ⎛x +x ⎜⎜ − μˆ ⎟⎟ 2 ⎠ ⎝ * 3
* 2
(A2-4b)
2
s 2 = 0.0694 (the estimated standard deviation is
with default function parameters. Based on this estimated Gaussian
probability
to
sample
an
offspring
that
satisfies
x > x3* is:
p( x > x3* | μˆ , σˆ ) ≥ 0.007 . In N / 2 times of sampling offspring in one generation, the probability that more * than one offspring locate in subset ( x3 ,1] in terms of statistical confidence level
1 − α 2 ( 0 < α 2 < 1 ) is computed as: 1- [1 −
N 2
p( x > x | μˆ , σˆ )] ≥ 1 − α 2 . * 3
(A2-5)
By considering the sampling in T generations, the probability that more than one offspring satisfy x > x3 is approximated as: *
1- [1 −
p( x > x | μˆ ,σˆ )] * 3
NT 2
≥ 1 − α2 .
(A2-6)
Since the estimated parameters of the Gaussian pdf change with population evolution, this inequality would only hold to some extent of validity in a few generations in the initial evolution phase. Thus, we derive the lower bound for population sizing based on sampling in T generations:
N (α 2 ) ≥ where
2 log α 2 , T log[1 − p( x > x3* | μˆ , σˆ )]
(A2-7)
N (α 2 ) ≥ 852 with α 2 = 0.05 , T = 1 .
The sampling failure rate can be estimated as: NT
α 2* = [1 − p( x > x3* | μˆ , σˆ )] 2
.
(A2-8)
80
M. Li et al.
Thus, the total failure rate of an algorithm is computed as:
α (α1* , α 2* ) = α1*α 2* .
(A2-9)
The experimental results (averaged over 2000 runs) and theoretical values for failure rate of the rECGA with a single Gaussian pdf are illustrated in Fig. A2.1, Fig. A2.2. If the population initialization fails to produce individuals: x ∈ ( x3 ,1] , the algorithms would converge to the deceptive attractor with a pretty high probability. When a bigger population size is employed according to N (α1 ) , the population can find the global optimum in initialization with a statistical confidence bound greater than (1 − α1 ) . *
N (α 2 ) is much bigger that N (α1 ) . By taking the above two * cases into consideration, we can see that the point x3 is the dominating factor to It is evident that
determine the population sizing and sampling based on the single Gaussian pdf model, and more deceptive functions require definitely a much bigger population size for the rECGA. 35
Theoretical alpha 1 Experimental alpha 1
30 ) % ( 25 e t 20 a r e 15 r u l i 10 a F 5 0
30
40
50 60 70 Population size
80
90
*
Fig. A2.1. Experimental and theoretical failure rate ( α1 ) of rECGA in initialization on URDF 25 Theoretical alpha Experimental alpha
)% 20 ( et ar 15 esr 10 ul ia F 5 0
30
40
50 60 70 Population size *
80
*
90
Fig. A2.2. Theoretical and experimental failure rate ( α(α1 , α2 ) ) of rECGA on URDF (maximum generations: 3)
Real-Coded ECGA for Solving Decomposable Real-Valued Optimization Problems
81
A3 Population Sizing and Sampling of rECGA with a Single Gaussian Pdf on MRDFi
As to the multivariate real-valued deceptive function, the population sizing and sampling of the rECGA are much more complicated. Without loss of generality, the rECGA decomposes a n-d MRDFi into n sub-functions which are all univariate deceptive functions with independency and non-overlapping, and then a single Gaussian model is built for each one. Suppose that the variable domain is divided into A = ( x3 ,1] , B = ( x2 , x3 ] , *
*
*
C = ( x1* , x2* ] , and D = [0, x1* ] for all of the 1-d sub-functions of the MRDFi. For intuitiveness, we consider a 2-d MRDFi, and there are three cases when we associate
x1 with
x2
in
different
values:
(a)
{x1 ∈ ( x3* ,1], x2 ∈ ( x3* ,1]} ; (b)
{x1 ∈ [0, x3* ], x2 ∈ ( x2* ,1] ∪ [0, x1* ]} or { x2 ∈ [0, x3* ], x1 ∈ ( x2*, 1] ∪ [0, x1* ]} ; (c)
{x1 ∈ ( x1* , x2* ], x2 ∈ ( x1* , x2* ]} . The neighborhood of global optimum or the global optimum itself is obtained in the case (a). In the most optimistic situation, the individuals that satisfy case (b) would survive in probability to the next generation with the truncation selection. But individuals in case (c) are discarded definitely. The probability of individuals that have a component taking values in A = ( x3 ,1] *
is p( x ∈ ( x3 ,1]) = 1 − x3 in the initialization. Thus, the probability in case (a) that the global optimum is generated more than one copies in the initialization is calculated as: *
*
p( ∧
i =1, 2 ,…,n
where p( n ) =
xi ∈ ( x3* ,1]) = 1 − [1 − p(n )]N ,
(A3-1)
n
∏ p( x ∈ ( x ,1])) = [ p( x ∈ ( x ,1])] i =1
i
* 3
* 3
n
.
The probability of successfully obtaining the global optimum in the initialization in terms of statistical confidence level 1 − α 3 ( 0 < α 3 < 1 ) is computed as:
p(
∧
i =1, 2 ,…, n '
xi ∈ ( x3* ,1]) ≥ 1 − α 3 , then we get the inequality: N (α3 , n ) ≥
log α 3 , log(1 − p( n))
(A3-2)
where N (α 3 , n ) ≥ 2.87 × 10 with α 3 = 0.05 , n = 10 and default function parameters. It is almost impossible to generate the global optimum as shown in case (a) for a 10-d MRDFi by using regular population size. The estimated failure rate to produce the global optimum in initialization for the nd MRDFi is: 14
α 3* ( n ) = (1 − p ( n )) N .
(A3-3)
82
M. Li et al.
If the initialization fails to obtain the global optimum, let’s consider the sampling process based on the estimated Gaussian pdf model. Empirically, the 1-d distribution of selected individuals is very similar to the function shape of the URDF. The sam-
psampling= p(x > x3* | μˆ,σˆ ) for a univariate function of the MRDFi is approximated with 0.33< μˆ < 0.50, 0.30<σˆ < 0.50 in the first generation and a few
pling probability
following generations. We compute empirically a set of values for the MRDFi functions with different dimensions of the MRDFi (see Table A3.1). Table A3.1. Estimated parameters for MRDFi function
Dimension ( n )
{μˆ , σˆ }
psampling
2 3 5 10
{0.40, 0.3200} {0.41, 0.3200} {0.45, 0.3100} {0.47, 0.3100}
0.0401 0.0428 0.0500 0.0570
Thus, the probability for having
xi ∈( x3*,1] ( i = 1,2,…, n ) in sampling is
psampling(n) = p( ∧ xi ∈( x3*,1]) = ( psampling)n , and psampling(n) = 3.62×10-13 with i=1,2,…,n
psampling = 0.057 and n = 10 . So, we get the probability for sampling the global optimum in one generation:
p( ∧
i =1,2,…,n
N 2
xi ∈ ( x ,1]) = 1 − [1 − psampling (n )] . * 3
The population size required to yield the global optimum with statistical confidence level 1 − α4 ( 0<α4 <1) by sampling in T generations is:
N (α 4 , n ) ≥
2 logα 4 , T log(1 − psampling ( n ))
(A3-4)
where N(α4, n) ≥1.65×10 with α4 = 0.05 , n = 10 in one generation for a 10-d MRDFi. There is little possibility to sample the global optimum based on the single Gaussian pdf model estimated on selected individuals. The sampling failure rate after T generations can be estimated as: 13
NT
α 4* ( n ) = [1 − psampling ( n )] 2
.
(A3-5)
The total failure rate of the algorithm on the n-d MRDFi is:
α (α 3* , α 4* , n ) = α 3* ( n ) × α 4* (n ) .
(A3-6)
Real-Coded ECGA for Solving Decomposable Real-Valued Optimization Problems
83
In order to get a clear picture about the performance of the rECGA on the MRDFi, experiments are carried out with n ∈ {2,3,5} . The averaged results of 2000 runs for
N ≤ 10000 , and 200 runs for N > 10000 are drawn in Figures A3.1, A3.2, A3.3 and A3.4. There is one point that should be clarified about Fig.A3.1. Fig.A3.1 shows that the experimental failure data are somewhat bigger than those of the theoretical failure rates. This is due to the fact that we use the best individual to calculate the failure rate in the initial population, and occasionally the best individual may not belong to the 100 90 ) % ( 80 70 e t a r 60 50 s e r 40 u l 30 i a F 20 10 0
Theoretical alpha3 (n=2) Experimental alpha3 (n=2) Theoretical alpha3 (n=3) Experimental alpha3 (n=3)
00 00 00 00 00 00 00 00 00 00 00 00 10 20 30 40 50 60 70 80 90 100 500 000 1 Population size *
Fig. A3.1. Theoretical and experimental failure rate ( α3 (n) ) in initialization for 2-d MRDFi and 3-d MRDFi 90 80 )% 70 ( et 60 ra 50 reu 40 li 30 Fa 20 10 0
Theoretical alpha (n=2) Experimental alpha (n=2) Theoretical alpha (n=3) Experimental alpha (n=3)
00 00 00 00 00 00 00 00 00 00 00 00 10 20 30 40 50 60 70 80 90 100 500 000 1 Population size *
*
Fig. A3.2. Theoretical and experimental failure rate ( α(α3 , α4 , n) ) of rECGA on 2-d and 3-d MRDFi (maximum generations: 3)
90 80 )% 70 ( et 60 ar 50 er 40 ul 30 ia F 20 10 0
Theoretical alpha (max gen:3) Experimental alpha (max gen:3) Theoretical alpha (max gen:5) Experimental alpha (max gen:5) Theoretical alpha (max gen:10) Experimental alpha (max gen:10) Theoretical alpha (max gen:20) Experimental alpha (max gen:20)
00 000 000 000 000 000 000 000 000 000 000 0000 10 2 3 4 5 Population 6 7 8size 9 10 50 10 *
*
Fig. A3.3. Theoretical and experimental failure rate ( α(α3 , α4 , n) ) of rECGA on 3-d MRDFi with different maximum generations
84
M. Li et al. 105 %)( 100 et ar 95 er 90 ul ia F 85 80
Theoretical alpha (max gen:5) Experimental alpha (max gen:5) Theoretical alpha (max gen:10) Experimental alpha (max gen:10) Theoretical alpha (max gen:20) Experimental alpha (max gen:10)
00 000 000 000 000 000 000 000 000 000 000 000 10 2 3 4 5 6 7 8 9 10 50 100 Population size
*
*
Fig. A3.4. Theoretical and experimental failure rate ( α(α3 , α4 , n) ) of rECGA on 5-d MRDFi with different maximum generations
attraction basin of the global optimum. For instance, there are two individuals x1 = (1,1,0) , x2 = (0.98,0.98,0.98) for the 3-d MRDFi, their fitness are
f ( x1) = 2.8 , f ( x2 ) = 2.7 . So, x1 is better than x2 , and the algorithm outputs x1 which is not in the attraction basin of the global optimum, so it is counted as a failure in the initialization. Fig.A3.2 and Fig.A3.3 show that the failure rate will drop down if the algorithm evolves more generations, particularly when a quite big population size is employed as N ≥ 5000 . Although it is theoretically sound, too many generations of population evolution do not empirically increase significantly the probability of sampling the global optimum due to the decreasing of the diversity of populations. The sampling based on the estimated Gaussian model can produce offspring in the attraction basin of global optimum when the population is diverse in the initial generations, but it will surely fail when the individuals selected are totally located in the attraction basins of the deceptive attractors for sub-functions. When a huge population is used as N ≥ 50000 for the 3-d MRDFi, the algorithm can attain the global optimum while the failure rate is quite low in initialization, and the sampling could yield the global optimum in a few generations when the initialization operation fails. Fig. A3.4 shows that the experimental failure rates are close to the theoretical ones in terms of different maximum generations on the 5-d MRDFi, particularly when the maximum generation is set as 5 or 10. In comparison with the 2-d MRDFi and 3-d MRDFi, the failure rates are pretty high even when quite a big population is employed, which is sound statistically. The rECGA algorithm meets great difficulties when it is applied on the 5-d MRDFi, which means that the single Gaussian pdf model for a sub-function usually fails to produce the global optimum for higher than 5-d MRDFi although it works for low dimensional MRDFi functions. A4 Population Sizing and Sampling of rECGA with Mixture Gaussian Pdf on MRDFi
First, let’s observe the evolution dynamics of rECGA with either single or mixture Gaussian pdf. The histograms of the initial population on one coordinate of the n-d MRDFi and those of the selected individuals in different generations of one run are displayed in Fig.A4.1.
Real-Coded ECGA for Solving Decomposable Real-Valued Optimization Problems Initial population
Generation: 2
Generation: 1
140
250
90 80
120
200
80 60 40
distribution of individuals
distribution of individuals
distribution of individuals
70
100
60 50 40 30 20
20 0
100
50
0
0.2
0.4
0.6
0.8
0
1
0
0.2
0.4
0.6
0.8
0
1
700
1400
600 500 400 300
1200 1000 800 600
200
400
100
200
0.6
0.8
1
distribution of individuals
1200
distribution of individuals
1400
1600
0.4
0.4
0.6
0
0.8
1
Generation: 64
Generation: 24 1800
800
0.2
0.2
x
900
0
0
x
Generation: 8
distribution of individuals
150
10
x
0
85
1000 800 600 400 200
0
0.2
0.4
0.6
0.8
0
1
0
0.005
0.01
0.015
x
x
0.02
0.025
0.03
0.035
x
Fig. A4.1(a). Histograms of initial population and selected individuals of rECGA with a single Gaussian pdf on 10-d MRDFi (population size: 10,000, maximum generation: 100) Generation: 1
Initial population 140
Generation: 5
100
450
90
400
120
80 60 40
350 distribution of individuals
distribution of individuals
distribution of individuals
80 100
70 60 50 40 30
0.2
0.4
0.6
0.8
0
1
0
0.2
0.4
700
700
600 500 400 300
0
1
0
0.2
0.4
0
0
0.6
Generation: 20
300
100
0.8
1
1
3000
400
100
0.8
3500
500
200
0.6 x
600
200
x
0.8
distribution of individuals
800
distribution of individuals
distribution of individuals
900
800
0.4
0.6
Generation: 10
Generation: 10 900
0.2
150
x
x
0
200
50
10 0
250
100
20 20 0
300
2500 2000 1500 1000 500
0
0.2
0.4
0.6 x
0.8
1
0
0
0.2
0.4
0.6
0.8
1
x
Fig. A4.1(b). Histograms of initial population and selected individuals of rECGA with mixture Gaussian pdf on 10-d MRDFi (population size: 10,000, maximum generation: 100, k-mean clustering: 2 )
Fig.A4.1 illustrates that the rECGA with a single Gaussian model fails to explore the attraction basin of the global optimum. In contrast, the rECGA with mixture Gaussian model is able to capture the feature of the distribution of selected individuals and make a correct factorization. First, let’s consider the URDF. Suppose that the attraction basins of the deceptive attractor and the global optimum of the URDF are labeled as cluster A and cluster B separately, and proportions of individuals belonging to cluster A and B are denoted as
86
M. Li et al.
pA, pB ( pA + pB =1). For simplicity, we only compute the successful sampling with p( x > x3* | μˆ ,σˆ ) in cluster B. We have in the first generation of the rECGA:
psampling (1) = pB (1) × p( x > x3* | μˆ B (1), σˆ B (1)) ,
(A4-1)
where pB (1) represents the prior probability of sampling offspring that belong to the cluster B based on the mixture Gaussian model in the first generation. The sampling of the rECGA with mixture Gaussian pdf model on the URDF is a discrete dynamic process that is modeled by:
psampling (t ) = pB (t ) × p( x > x3* | μˆ B (t ), σˆ B (t )) ,
(A4-2)
t denotes the t-th generation; pB (t ) , μˆ B (t ) , σˆ B (t ) are estimated on selected individuals of P ( t − 1) .
where
Second, we turn to the MRDFi. The sampling probability for all sub-functions MRDFi to have xi ∈ ( x3 ,1] ( i = 1, 2, … , n ) in the sampling of the t-th generation is estimated as: *
psampling ( n, t ) = p( ∧
i =1, 2 ,…, n
xi ∈ ( x3* ,1]) = ( psampling (t )) n .
(A4-3)
The probability to sample the global optimum in t-th generation: N
p(
∧
i =1, 2 ,…, n
xi ∈ ( x3* ,1], t ) = 1 − [1 − psampling ( n, t )] 2 .
(A4-4)
Thus, the cumulative probability distribution function for successful sampling the global optimum of the MRDFi in total evolution process is computed approximately as: t
N
Pcdf ( ∧ xi ∈ ( x3* ,1], t ) = 1 − (1 − pinitial )∏[1 − ( psampling(τ ))n ] 2 , (A4-5) i =1, 2,…,n
where
τ =1
pinitial = p( ∧
i =1, 2 ,…,n
xi ∈ ( x3* ,1]) = 1 − (1 − p(n )) N .
The formula (A4-5) is only descriptive, and it is not an easy task to estimate the exact model about the cumulative probability distribution function for a specific problem. Empirically, the rECGA with mixture Gaussian pdf achieves good performance on the MRDFi (see Fig.1(b)) because the probabilistic building-blocks recombination based on the estimated joint distribution of the selected individuals works efficiently.
Linkage Learning Accuracy in the Bayesian Optimization Algorithm Claudio F. Lima1 , Martin Pelikan2 , David E. Goldberg3 , Fernando G. Lobo1 , Kumara Sastry3 , and Mark Hauschild2 1
2
3
University of Algarve, Portugal
[email protected],
[email protected] University of Missouri at St. Louis, USA
[email protected],
[email protected] University of Illinois at Urbana-Champaign, USA
[email protected],
[email protected]
Summary. The Bayesian optimization algorithm (BOA) uses Bayesian networks to learn linkages between the decision variables of an optimization problem. This chapter studies the influence of different selection and replacement methods on the accuracy of linkage learning in BOA. Results on concatenated m-k deceptive trap functions show that the model accuracy depends on a large extent on the choice of selection method and to a lesser extent on the replacement strategy used. Specifically, it is shown that linkage learning in BOA is more accurate with truncation selection than with tournament selection. The choice of replacement strategy is important when tournament selection is used, but it is not relevant when using truncation selection. On the other hand, if performance is our main concern, tournament selection and restricted tournament replacement should be preferred. Additionally, the learning procedure of Bayesian networks in BOA is investigated to clarify the difference observed between tournament and truncation selection in terms of model quality. It is shown that if the metric that scores candidate networks is changed to take into account the nature of tournament selection, the linkage learning accuracy with tournament selection improves dramatically.
1 Introduction Unlike traditional evolutionary algorithms (EAs), the Bayesian optimization algorithm (BOA) (23, 21) replaces the standard crossover and mutation operators by building a probabilistic model of promising solutions and sampling from the corresponding probability distribution. This feature allows BOA and other advanced estimation of distribution algorithms (EDAs) (15, 24) to automatically identify the problem decomposition and important problem substructures, leading to superior performance for many problems when compared with EAs with fixed, problem-independent variation operators. Although the main feature of BOA and other EDAs is to perform efficient mixing of key substructures or building blocks (BBs), they also provide additional information about the problem being solved. The probabilistic model of the Y.-p. Chen, M.-H. Lim (Eds.): Linkage in Evolutionary Computation, SCI 157, pp. 87–107, 2008. c Springer-Verlag Berlin Heidelberg 2008 springerlink.com
88
C.F. Lima et al.
population, that represents (in)dependencies among decision variables, is an important source of information that can be exploited to enhance the performance of EDAs even more, or to assist the user in a better interpretation and understanding of the underlying structure of the problem. Examples of using structural information from the probabilistic model for another purpose besides mixing are fitness estimation (31, 25, 30), induction of global neighborhoods for mutation operators (29, 17), hybridization and adaptive time continuation (18, 17), substructural niching (28), offline (34) and online (35) population size adaptation. In this chapter we analyze the structural accuracy of the probabilistic models in BOA and their ability to represent underlying problem substructures. In particular, we use concatenated deceptive trap functions—where the optimal model is known, and accurate linkage learning is critical—to investigate the influence of different selection and replacement strategies on model structural accuracy. The initial results show that as far as model quality is concerned, truncation selection should be preferred over tournament selection, and the choice of replacement strategy matters when tournament selection is used. These results also show that if the objective is to obtain near-optimal solutions with high reliability using minimal number of function evaluations, then tournament selection with restricted tournament replacement is the best strategy for BOA. Additionally, we investigate in detail the Bayesian network learning in BOA to understand the difference observed between tournament and truncation selection in terms of model quality. Selection is analyzed as the mating-pool distribution generator, which turns out to have a great impact on Bayesian network learning. In fact, if the metric that scores networks takes into account the natural distribution of tournament selection, the model quality can be highly improved and comparable to that of truncation selection. This reasoning is confirmed through additional experiments. The chapter is structured as follows. The next section gives an outline of BOA. Section 3 motivates the importance of the research topic addressed in this chapter and makes a short survey on related work. Section 4 introduces the experimental setup used for measuring the structural accuracy of the probabilistic models in BOA. Section 5 analyzes the influence of selection in structural linkage learning, while the next section deals with the influence of the replacement method. In Section 7 Bayesian network learning is investigated at some detail, and in the subsequent section a new scoring metric is employed to improve the model quality obtained by tournament selection. The chapter ends with a summary and major conclusions.
2 Bayesian Optimization Algorithm Estimation of distribution algorithms (15, 24) replace traditional variation operators of EAs by building and sampling a probabilistic model of promising solutions to generate the offspring population. The Bayesian optimization algorithm (23, 21) uses Bayesian networks as the probabilistic model to capture the (in)dependencies between the decision variables of the optimization problem.
Linkage Learning Accuracy in the Bayesian Optimization Algorithm
89
BOA starts with an initial population that is usually generated at random. In each iteration, selection is performed to obtain a population of promising solutions. This population is then used to build the probabilistic model for the current generation. After the model structure is learned and its parameters estimated, the offspring population is generated by sampling from the distribution of modeled individuals. The new solutions are then incorporated into the original population by using any standard replacement method. The next iteration proceeds again from the selection phase until some stopping criteria is satisfied. Bayesian networks (BNs) (20) are powerful graphical models that combine probability theory with graph theory to encode probabilistic relationships between variables of interest. A BN is defined by its structure and corresponding parameters. The structure is represented by a directed acyclic graph where the nodes correspond to the variables of the data to be modeled and the edges correspond to conditional dependencies. The parameters are represented by the conditional probabilities for each variable given any instance of the variables that this variable depends on. More formally, a Bayesian network encodes the following joint probability distribution, p(X) =
p(Xi |Πi ),
(1)
i=1
where X = (X1 , X2 , . . . , X ) is a vector with the variables of the problem, Πi is the set of parents of Xi (nodes from which there exists an edge to Xi ), and p(Xi |Πi ) is the conditional probability of Xi given its parents Πi . The parameters of a Bayesian network can be represented by a set of conditional probability tables (CPTs) specifying the conditional probabilities for each variable given all possible instances of the parent variables Πi . Alternatively, these conditional probabilities can be stored in the form of local structures such as decision trees or decision graphs, allowing a more efficient and flexible representation of local conditional distributions. Therefore, we use BNs with decision trees. The scoring metric used to quantify the quality of a given network is the Bayesian-Dirichlet metric (5, 14, 4), which is given by BD(B) = p(B)
i=1 l∈Li
Γ (mi (xi , l) + m (xi , l)) Γ (mi (l)) i , Γ (mi (l) + mi (l)) x Γ (mi (xi , l))
(2)
i
where p(B) is the prior probability of the network structure B, Li is the set of leaves in the decision tree Ti (corresponding to Xi ), mi (l) is the number of instances in the population that contain the traversal path in Ti ending in leaf l, mi (xi , l) is the number of instances in the population that have Xi = xi and contain the traversal path in Ti ending in leaf l, mi (l) and mi (xi , l) represent prior knowledge about the values of mi (l) and mi (xi , l). Here, we consider the K2 variant of the BD metric, which uses an uninformative prior that assigns mi (xi , l) = 1.
90
C.F. Lima et al.
To favor simpler networks over more complex ones, the prior probability of each network p(B) can be adjusted according to its complexity, that is given by the description length of the parameters required by the network (4, 9). Based on this principle, Pelikan (21) proposed the following penalty term for BOA, p(B) = 2−0.5 log2 (n)
i=1
|Li |
,
(3)
where n is the population size. To learn the most adequate structure for the BN a greedy algorithm is usually used for a good compromise between search efficiency and model quality. We consider a simple learning algorithm that starts with an empty network and at each step performs the operation that improves the metric the most, until no further improvement is possible. The operator considered is the split, which splits a leaf on some variable and creates two new children on the leaf. Each time a split on Xj takes place at tree Ti , an edge from Xj to Xi is added to the network. For more details on BNs with local structures the reader is referred elsewhere (4, 9, 21). The hierarchical BOA (hBOA) was later proposed by Pelikan and Goldberg (22, 21) and results from the combination of BNs with local structures with a simple yet powerful niching method to maintain diversity in the population, known as restricted tournament replacement (RTR) (12). hBOA is able to solve hierarchical decomposable problems, in which the variable interactions are present at more than a single level.
3 Motivation and Related Work While BOA is able to solve a broad class of nearly decomposable and hierarchical problems in a reliable and scalable manner, their probabilistic models oftentimes do not exactly reflect the problem structure. Because the probabilistic models are learned from a sample of limited size (population of individuals), particular features of the specific sample are also encoded, which act as noise when seeking for generalization. This is a well-known problem in machine learning, known as overfitting. Analyzing the dependency groups captured by the Bayesian network with decision trees, it can be observed that while all important linkages are detected, spurious linkages are also incorporated in the model. By spurious linkage we mean additional variables that are considered together with a correct linkage group. While the structure of the BN captures such excessive complexity, the corresponding conditional probabilities nearly express independency between the spurious variables and the correct linkage, therefore not affecting the capability of sampling such variables as if they were almost independent. Although the performance of BOA is not greatly affected by this kind of overfitting, several model-based efficiency enhancement techniques for EDAs (31, 25, 30, 29, 17, 18, 28, 34, 35) crucially rely on the structural accuracy of the probabilistic models. One such example is the exploration of substructural neighborhoods for local search in BOA (17). While significant speedups were obtained by
Linkage Learning Accuracy in the Bayesian Optimization Algorithm
91
incorporating model-based local search, the scalability of this speedup decreased for larger problem sizes due to overly complex model structures learned in BOA. Therefore, it is important to understand in which conditions the structural accuracy of the probabilistic models in BOA and other multivariate EDAs can be maximized. So far, only a small number of studies have been done in this direction (26, 33, 6, 8, 13). In the remainder of this section we take a brief look at these works. Santana, Larra˜ naga, and Lozano (26) analyzed the effect of selection on the arousal of certain dependencies in EDAs for random functions and showed that for these functions, independence relationships not represented by the function structure are likely to appear in the probabilistic model. Additionally, they proposed an EDA that only used a subset of the dependencies that exist in the data, which they called malign interactions. Some preliminary experiments showed that these approximations of the probabilistic model can in certain cases be applied to EDAs. Wu and Shapiro (33) investigated the presence of overfitting when learning the probabilistic models in BOA and its consequences in terms of overall performance when solving random 3-SAT problems. CPTs (to encode the conditional probabilities) and the corresponding BIC metric were used. The authors concluded that overfitting does take place and that there is some correlation between this phenomenon and performance. The reduction in overfitting was proposed by using an early stopping criteria during the learning process of BNs, which gave some improvement in performance. The trade-off between model complexity and performance in BOA was also studied recently (6). Correa and Shapiro looked at the performance achieved by BOA as a function of a parameter that determines the maximum number of incoming edges for each node. This parameter puts a limit on the number of parents for each variable, simplifying the search procedure for a model structure. This parameter was found to have a strong effect on the performance of the algorithm, for which there is a limited set of values where the performance is maximized. These results were obtained using CPTs and the corresponding K2 metric. We should note that in fact this parameter is crucial if CPTs are used with the K2 metric, however this is not the case for more sophisticated metrics that efficiently incorporate a complexity term to introduce pressure toward simpler models. This can be done better with the BIC metric for CPTs, or with the K2 metric for the case of decision trees (21). More recently, Echegoyen et al. (8) applied new developments in exact BN learning into the EDA framework to analyze the consequent gains in optimization. While in terms of convergence time the gain in using exact BN learning was marginal, the models learned by EBNA were more closely related to the underlying structure of the problem. However, the computational cost of learning exact BN is only manageable for relatively small problem sizes (experiments were made for a maximum problem size of 20). Equally recent, Hauschild et al. (13) analyzed the probabilistic models built by hBOA for two common test problems: concatenated trap functions and 2D
92
C.F. Lima et al.
Ising spin glasses with periodic boundary conditions. The authors verified that the models learned closely correspond to the structure of the underlying problem. In their analysis, Hauschild et al. used truncation selection and restricted tournament replacement. In this chapter, we will show the results from (13) do not carry over to other combinations of selection and replacement methods. Before presenting these results, we discuss the details of our empirical analysis.
4 Experimental Setup for Measuring Structural Accuracy of Probabilistic Models This section details the experimental setup and measurements used to investigate the structural accuracy of the probabilistic models in BOA. 4.1
Test Problem and Experimental Setup
To investigate the structural accuracy of linkage learning in BOA, we focus on solving a problem of known structure, where it is clear which dependencies must be discovered (for successful tractability) and which dependencies are unnecessary (reducing the interpretability of the models). In this way the evaluation of the model structure quality to correctly detect both dependencies and independencies is performed. The test problem considered is the m-k trap function where m is the number of concatenated k-bit trap functions. Trap functions (1, 7) are relevant to test problem design because they bound an important class of nearly decomposable problems (10). The Trap function used (7) is defined as follows k, if u = k (4) ftrap (u) = k − 1 − u, otherwise where u is the number of ones in the string, k is the size of the trap function. Note that for k ≥ 3 the trap function is fully deceptive (7) which means that any lower than k-order statistics will mislead the search away from the optimum. In this problem the accurate identification and exchange of the building-blocks (BBs) is critical to achieve success, because processing substructures of lower order will lead to exponential scalability (32). Thus, all variables corresponding to each trap function form a linkage group or BB partition and should be treated together by the probabilistic model. Note that no information about the problem is given to the algorithm, therefore it is equally difficult for BOA if the variables correlated are closely or randomly distributed. A trap function with size k = 5 is used in our experiments. A bisection method is used to determine the minimal population size required to solve the problem (27). For each experiment, 10 independent bisection runs are performed. Each bisection run searches for the minimal population size required to find the optimum in 10 out of 10 independent runs. Therefore, the results for the minimal sufficient population size are averaged over 10 bisection runs, while the results for the number of function evaluations used are averaged over 100 (10 × 10) independent runs.
Linkage Learning Accuracy in the Bayesian Optimization Algorithm
4.2
93
Measuring Structural Accuracy of Probabilistic Models
For accurate linkage learning in BOA at least one of the variables of each trap subfunction should depend on all remaining k − 1 variables, so that all k corresponding variables can be processed together by the probabilistic model. For example, the following dependency relation, X1 ← {X2 , X3 , X4 , X5 }, encodes a linkage group between all variables for the first 5-bit trap subfunction. In addition, the remaining − k variables should not be part of that same dependency relation. If this is the case, the extra variables act as spurious linkage. For example, for the dependency relation, X1 ← {X2 , X3 , X4 , X5 , X6 , X11 }, X6 and X11 are spuriously linked variables. In essence, the dependencies between the groups of k bits corresponding to each subfunction must be discovered, while the remaining dependencies should be avoided to maximize mixing and minimize model complexity. At each generation four different measures are analyzed taking into account only dependency groups of order k or higher: Proportion of BBs with correct linkage group is the proportion of BB partitions or subfunctions (out of m) that have a dependency group in the model that only contains the corresponding k variables. Proportion of BBs with spurious linkage group is the proportion of BB partitions or subfunctions (out of m) that have a dependency group in the model that contains the corresponding k variables plus some additional spuriously linked variables. Proportion of BBs with a linkage group is simply the sum of the two previous statistics. This measure is useful to confirm if every BB partition or subfunction is represented in the model, whether with only correct or additional spurious dependencies. Average size of spurious linkage is the average maximal number of spurious variables in the dependency relations that have spurious linkage (only those relations greater than k are considered).
5 Influence of the Selection Method In this section, the influence of the selection method used in BOA, as well as the corresponding selection pressure, is investigated from the standpoint of structural accuracy of the learned linkage by the probabilistic models. Specifically, we consider two widely used ordinal selection schemes: Tournament and truncation selection. The particular choice of these two selection operators is related to the fact that these two schemes are the most frequently used in EDAs. Also, a previous study (2) on the comparison of several selection schemes has shown that these two schemes differ significantly in relevant features such as selection variance and loss of diversity.
94
C.F. Lima et al.
Table 1. Equivalent tournament size (s) and truncation threshold (τ ) for the same selection intensity (I) (2) I 0.56 0.84 1.03
s 2 3 4
τ 66% 47% 36%
In tournament selection (11, 3), s individuals are randomly picked from the population and the best one is selected for the mating pool. This process is repeated n times, where n is the population size. There are two popular variations with the method, with and without replacement. With replacement, the individuals are drawn from the population following a discrete uniform distribution. Without replacement, individuals are also drawn randomly from the population but there’s the guarantee that every individual from the population participates in exactly s tournaments. While the expected outcome for both alternatives is the same, the latter is a less noisy process. Therefore, in this study we use tournament selection without replacement. In truncation selection (19) the best τ % individuals in the population are selected for the mating pool. This method is equivalent to the standard (μ, λ)selection procedure used in evolution strategies (ESs), where τ = μλ × 100. Note that when increasing the size of the tournament s, or decreasing the threshold τ , the selection pressure (ratio of maximum to average fitness in the population) is increased, which means an increase in the selection strength. For the purpose of studying the influence of different selection strategies, the replacement strategy is kept as simple as possible: the offspring fully replace the parent population. We now turn to our head-to-head comparison between tournament and truncation selection having in mind the structural accuracy of the probabilistic models. In order to compare these two methods on a fair basis, different configurations for both methods with equivalent selection intensity are tested. The relation between selection intensity I, tournament size s, and truncation threshold τ is taken from (2) and is shown in Table 1. Figures 1 and 2 show the linkage information captured by BOA with tournament and truncation selection, respectively. The test problem considered is a concatenated trap function with k = 5 and m = 24, giving a total string length of = 120. For tournament selection, the proportion of BBs that have a correct linkage group (with exactly all corresponding k variables) represented in the model is quite low. Although for s = 2, nearly half of the BBs are still covered, for s = 3 and s = 4 this value approaches zero. Nevertheless, the BBs that are not covered with correct linkage groups are covered in spurious linkage groups, that have additional spuriously linked variables (other than the corresponding k variables). This can be observed in Figure 1 (c), where after the initial generation and until the end of the run, all BBs are represented (whether with only correct or
Proportion of BBs w/ spurious linkage group
Proportion of BBs w/ correct linkage group
Linkage Learning Accuracy in the Bayesian Optimization Algorithm
1 s=2 s=3 s=4
0.8
0.6
0.4
0.2
0
0
5
10
15 20 Generation, t
25
30
95
1 s=2 s=3 s=4
0.8
0.6
0.4
0.2
0
0
(a) Correct linkage groups
5
10
15 20 Generation, t
25
30
(b) Spurious linkage groups
Average size of spurious linkage
Proportion of BBs w/ a linkage group
60 s=2 s=3 s=4
1 0.8 0.6 0.4 0.2 0
0
5
10
15 20 Generation, t
(c) All linkage groups
25
30
s=2 s=3 s=4
50 40 30 20 10 0
0
5
10
15 20 Generation, t
25
30
(d) Average size of spurious linkage
Fig. 1. Linkage group information captured by the probabilistic model of BOA along the run for different tournament sizes, s = {2, 3, 4}, when solving m = 24 concatenated traps of order k = 5 ( = 120). Tournament selection and full replacement is used.
additional spurious dependencies) in the probabilistic model, a necessary condition to be able to solve the problem. In Figure 1 (d), a drastic difference between binary tournament and higher tournament sizes can be noted for the average size of spurious linkages. In fact, the number of spurious variables added to the dependency groups is so high that one might wonder how BOA can sample new solutions efficiently. Analyzing the models in more detail, it can be seen that while the structure learned is much more complex than the underlying structure of the trap subfunctions, the parameters of the model nearly express independence between spurious and correlated variables. The detection of these weak dependencies is in part due to random fluctuations in the population (sample of limited size), that act as noise on the process of learning the real dependencies. Note that selection is performed at the individual-level rather than at the substructural or BB-level. Therefore, a top individual does not necessarily has mostly good substructures, which induces the learning process of BNs into some uncertainty, that is more pronounced in the initial generations. As the run proceeds and this source of noise is reduced
Proportion of BBs w/ spurious linkage group
C.F. Lima et al.
Proportion of BBs w/ correct linkage group
96
1 τ=66% τ=47% τ=36%
0.8
0.6
0.4
0.2
0
0
5
10
15 20 Generation, t
25
30
1 τ=66% τ=47% τ=36%
0.8
0.6
0.4
0.2
0
(a) Correct linkage groups
0
5
10
15 20 Generation, t
25
30
(b) Spurious linkage groups
0.8
Average size of spurious linkage
Proportion of BBs w/ a linkage group
2 τ=66% τ=47% τ=36%
1
0.6 0.4 0.2 0
0
5
10
15 20 Generation, t
(c) All linkage groups
25
30
τ=66% τ=47% τ=36%
1.5
1
0.5
0
0
5
10
15 20 Generation, t
25
30
(d) Average size of spurious linkage
Fig. 2. Linkage group information captured by the probabilistic model of BOA along the run for different truncation thresholds, τ = {66%, 47%, 36%}, when solving m = 24 concatenated traps of order k = 5 ( = 120). Truncation selection and full replacement is used.
by the iterative process of select-model-sample, the spurious linkage size reduces significantly. For truncation selection the results are significantly better. With this selection method, BOA is able to represent almost 100% of the BBs with accurate linkage groups of order k = 5, while the size of spurious linkage is practically insignificant. Also, note that increasing the selection pressure (reducing the truncation threshold) hardly affect the accuracy of the linkage information. This completely different behavior between these two selection schemes lead us to take a look at their scalability behavior for the average size of spurious linkage, population size, and number of function evaluations. That is what is shown in Figures 3 and 4. The computational requirements for truncation are higher than for tournament selection by a significant however constant factor. Nevertheless, if we compare tournament selection with s = 2 and truncation selection with τ = 36%, the requirements for truncation are now smaller while in terms of structural accuracy truncation is still much better.
Average size of spurious linkage
Linkage Learning Accuracy in the Bayesian Optimization Algorithm
97
s=2 s=3 s=4 τ=0.66 τ=0.47 τ=0.36
50 40 30 20 10 0
20
40
60 80 Problem size, l
100
120
Fig. 3. Average size of the spurious linkage for different selection strategies when solving concatenated 5-bit trap functions of varying total string length . Full replacement is used.
Population size, n
Num. of function evaluations, nfe
s=2 s=3 s=4 τ=0.66 τ=0.47 τ=0.36
5
10
4
10
s=2 s=3 s=4 τ=0.66 τ=0.47 τ=0.36
6
10
5
10
4
10
3
10
20
40 Problem size, l
(a) Population size
80
120
20
40 Problem size, l
80
120
(b) Num. of evaluations
Fig. 4. (a) Population size and (b) number of function evaluations required for different selection strategies when solving concatenated 5-bit trap functions of varying total string length . Full replacement is used. Although truncation selection requires larger population sizes, using tournament selection with equivalent population sizes to those required by truncation does not significatively improve the linkage information.
Further experiments (not plotted), where tournament selection was tested with the same population size used for truncation with the same selection pressure, showed that tournament selection only improves by a small factor the linkage information and is still much worse than truncation. This suggests that the difference observed isn’t simply a matter of having enough population size, but more about the different way these selection operators work. In a detailed study (2) about the comparison of several selection schemes, it was shown that
98
C.F. Lima et al.
truncation and tournament selection are in fact quite different in terms of selection variance and loss of diversity. Truncation selection has a higher loss of diversity and lower selection variance (for the same selection intensity) than tournament selection. Although we might expect that a lower loss of diversity and a higher selection variance would be desirable to avoid premature convergence, from the standpoint of EDAs where the probabilistic models are learned at every generation, a faster and clear distinction between good individuals and just above average individuals reduces the noise faced by the learning process of BNs. There is also another important difference between these two operators. While in tournament selection the number of copies of an individual is proportional to its rank1 , in truncation no particular relevance is given to very good individuals, because all individuals selected get exactly one copy into the mating pool. This presents an interesting characteristic from the standpoint of BN learning, as will be discussed further in Section 7.
6 Influence of the Replacement Method In this section, we analyze the influence of the replacement method used in BOA with respect to the accuracy of the probabilistic models versus overall performance. Three different replacement strategies are considered: full replacement (FR), elitist replacement (ER), and restricted tournament replacement (RTR). In full replacement, the offspring population completely replaces the parent population at the end of each generation, therefore there is no overlap between these populations. For elitist replacement a given proportion of the worst individuals of the parent population is replaced by new individuals. A typical strategy is to replace the worst 50% individuals of the parent population by offspring, keeping the best 50% individuals for the next generation. Finally, a niching method called restricted tournament replacement (RTR) (12, 21) is also tested. With RTR, each new solution X is incorporated into the original population using the following procedure: 1. Select a random subset of individuals W with size w from the original population. 2. Let Y be the solution from W that is most similar to X (in terms of genotypic distance). 3. Replace Y with X if the latter is better, otherwise discard X. The window size w is set to w = min{, n/20} (21), where is the problem size and n is the population size. Note that RTR is the replacement method used in hBOA. Figure 5 shows the results obtained for the different replacement strategies, using binary tournament selection. It can be seen that for all replacement methods a significant proportion of the BBs are not well represented in the models 1
The best individual gets exactly s copies, while other top individuals get on average a value close to s.
1
Proportion of BBs w/ spurious linkage group
Proportion of BBs w/ correct linkage group
Linkage Learning Accuracy in the Bayesian Optimization Algorithm
ER 50% FR RTR
0.8
0.6
0.4
0.2
0
0
5
10
15 20 Generation, t
25
30
99
1 ER 50% FR RTR
0.8
0.6
0.4
0.2
0
0
(a) Correct linkage groups
5
10
15 20 Generation, t
25
30
(b) Spurious linkage groups
Average size of spurious linkage
Proportion of BBs w/ a linkage group
10 ER 50% FR RTR
1
0.8
0.6
0.4
0.2
0
0
5
10
15 20 Generation, t
(c) All linkage groups
25
30
ER 50% FR RTR
8
6
4
2
0
0
5
10
15 20 Generation, t
25
30
(d) Average size of spurious linkage
Fig. 5. Linkage group information captured by the probabilistic model of BOA along the run for different replacement methods when solving m = 24 concatenated traps of order k = 5 ( = 120). ER 50% stands for the replacement of the worst 50% parents, FR for full replacement, and RTR for restricted tournament replacement. Binary tournament selection is used.
learned by BOA. Although all BBs have a linkage group that relates all k variables of interest, most of them have spurious linkage. This scenario is equally true for RTR and ER 50%, while for the FR method the linkage information captured is slightly more accurate. Figure 5 (d) shows the average size of spurious linkage, where RTR is clearly the worst option with respect to structural accuracy of the probabilistic models, while ER 50% performs better than RTR but still worse than FR, for which the average size is relatively constant and never goes beyond two. Note that the replacement strategy does not have the same impact as tournament size in the spurious linkage size. In Figures 6 and 7, the scalability of the replacement methods is depicted. Additional values for ER were tested to investigate the progressive influence of the proportion of elitism on the structural accuracy of the learned models. RTR is clearly the strategy that requires smaller population size and fewer evaluations, however at the cost of higher spurious linkage sizes than the remaining replacement methods. This is due to the niching capabilities of RTR. By
100
C.F. Lima et al. 10 Average size of spurious linkage
ER 50% ER 75% ER 90%
8
ER 95% ER 99% FR
6
RTR
4
2
0
20
40
60 80 Problem size, l
100
120
Fig. 6. Average size of the spurious linkage for different replacement strategies when solving concatenated 5-bit trap functions with varying total string length . Binary tournament selection is used. 6
4
10
ER 50% ER 75% ER 90%
3
10
ER 95% ER 99%
Num. of function evaluations, nfe
Population size, n
10
5
10
ER 50% ER 75% ER 90% 4
ER 95%
10
ER 99% FR
FR
RTR
RTR
20
40 Problem size, l
80
(a) Population size
120
20
40 Problem size, l
80
120
(b) Number of function evaluations
Fig. 7. (a) Population size and (b) number of function evaluations required for different replacement strategies when solving concatenated 5-bit trap functions with varying total string length . Binary tournament selection is used.
preserving diversity in the population, BOA can solve the given problem with smaller population sizes and consequently with fewer evaluations. However, the quality of the model is not the best because the drawback of using tournament selection is aggravated with smaller population sizes and increased diversity due to niching. For ER, as the elitist proportion is reduced the structure captured by the models gradually improves, until we get to the case of FR where the best result is obtained in terms of model accuracy. Note that for ER, where the best proportion of individuals is always kept in the population, the model is not required to be as accurate as in FR, where the sampled individuals fully replace
Linkage Learning Accuracy in the Bayesian Optimization Algorithm
101
the original population, in which case the quality of sampled solutions have a stronger influence on the probability of success to find the optimum. While these results were obtained for binary tournament selection, additional experiments (not plotted) were performed with truncation selection. In this case, the replacement strategies were found not to have a significative impact on model accuracy and all methods performed very similar to truncation selection with FR (see Figures 2 and 3).
7 A Closer Look at Model Building in BOA This section pays a closer look at Bayesian network learning in BOA to understand the difference in model quality observed between tournament and truncation selection. First, we observe the learning procedure in detail to investigate when correct and spurious dependencies are inserted in the network. Second, we demonstrate how the selection operator can lead to model overfitting and consequently inaccurate linkage learning. Figure 8 shows the metric gain obtained in model building for a single run of BOA. For each learning step (edge addition) the corresponding gain in the scoring metric is presented, as well if the edge inserted is a correct (upper dots) gen. 1 81 40 0 0
30
60
90
120
90
120
90
120
gen. 6 1996 998 0 0
30
60 gen. 11
8913 4456 0 0
30
60
Fig. 8. Metric gain vs. learning step in Bayesian network learning with BOA. The problem is the 5-bit trap with = 50. The first, middle, and last generations of the run are plotted. Upper dots represent correct edges and lower ones represent spurious dependencies. Binary tournament selection and full replacement are used.
102
C.F. Lima et al.
Expected number of copies, ci
4 s=2 s=3 s=4
3
2
1
0
0
0.2
0.4 0.6 Rank, i (in percentile)
0.8
1
Fig. 9. Distribution of the expected number of copies in the mating pool after tournament selection with s = 2, 3, 4. The expected number of copies varies with rank, which is expressed in percentile.
or a spurious (lower dots) one. The first, middle, and last generations of the run are plotted. One can easily see that the metric gain is higher at the beginning of the learning process and decreases towards zero—which is the threshold for accepting a modification in the Bayesian network. Additionally, the metric gain magnitude increases towards the end of the run. This is due to the fact that as the search focus on specific regions of the search space (loss in diversity) the marginal likelihood increases also. In fact, for the last generation the metric gain has a clear shape when compared to the first and middle generations. With respect to the correctness of the edges added to the network, it is clear that the majority of the spurious edges are inserted in the network rather at the end of the learning procedure, when the very few last correct edges are being inserted. This suggests that an early stop or a higher acceptance threshold (note that spurious edges have a metric gain quite small) in the learning procedure could avoid the acceptance of such spurious linkage. In BOA, the selection operator can also be viewed as the generator of the data set used to learn the Bayesian network at each generation. Since in EDAs we are interested in modeling the set of promising solutions, the selection operator indicates which individuals have relevant features to be modeled and propagated in the solution set (population of individuals). The selection operator can explicitly assign several copies of the same individual to the mating pool, where the number of copies is somewhat proportional to their fitness rank. This is the case for tournament, ranking, and proportional selection. While for truncation selection the expected number of copies for the selected individuals is one, which follows a uniform distribution; in tournament selection, the distribution of the expected number of copies ci can be approximated by a power distribution with p.d.f., f (x) = α xα−1 , 0 < x < 1, α = s,
(5)
Linkage Learning Accuracy in the Bayesian Optimization Algorithm
103
where x is the relative rank of each individual, expressed in percentile. The best solution has rank 1, while the worst has rank 0. For example, the best 123rd individual in a population of n = 1000 have a percentile rank of 0.877. Figure 9 shows the distribution of the expected number of copies in the mating pool after tournament selection. The difference between the two selection methods is significant. While tournament selection assigns increasing relevance to top-ranked individuals according to a power distribution, truncation selection gives no particular preference to any of the selected individuals, all having the same frequency in the learning data set. In tournament selection, the best solutions get approximately s copies in the mating pool, which induces the learned models to focus on particular features of these individuals, which contain good substructures, but also misleading components due to stochastic noise.
8 Adaptive Scoring Metric Another way to look at tournament selection in comparison with truncation selection as the mating-pool generator is recognizing that this selection procedure acts as a biased data resampling on an uniform data set. The uniform data set is the set of unique selected solutions (solutions that win at least one tournament), similar to what happens in truncation, while the resampling is performed when individuals win more than one tournament. This sort of resampling is clearly biased by fitness. In order to counterbalance this resampling bias towards top-ranked individuals, a modification of the scoring metric has been proposed (16). More precisely, the complexity penalty component of the K2 metric (Equation 3) has been changed to (6) p(B) = 2−0.5s log2 (n) i=1 |Li | , where the factor s (tournament size) has been incorporated. In this way, the greater the bias from resampling top-ranked individuals, the more demanding the score metric is in accepting new dependencies. Note that this is equivalent to change the acceptance threshold from zero to a higher value that depends on s. Figure 10 shows the linkage group information captured by the Bayesian network in BOA when using the adaptive complexity penalty. The improvement is significative with respect to the previous results obtained for tournament selection. In fact, the results are slightly better than those obtained for truncation selection. This is somewhat expected since resampling top-ranked individuals can lead to a clear detection of important substructures, as long as the learning procedure is conservative enough to avoid changes in the network guided by minor improvements in the metric (which is achieved by the new scoring metric). The number of function evaluations required by tournament selection with the s−penalty is now closer to that required by truncation selection, but like it was observed for the latter (Figure 4) the scalability behavior is not affected.
Proportion of BBs w/ spurious linkage group
C.F. Lima et al.
Proportion of BBs w/ correct linkage group
104
1 s=2 s=3 s=4
0.8
0.6
0.4
0.2
0
0
5
10
15 20 Generation, t
25
30
1 s=2 s=3 s=4
0.8
0.6
0.4
0.2
0
0
(a) Correct linkage groups
5
10
15 20 Generation, t
25
30
(b) Spurious linkage groups
Average size of spurious linkage
Proportion of BBs w/ a linkage group
1 s=2 s=3 s=4
1 0.8 0.6 0.4 0.2 0
0
5
10
15 20 Generation, t
25
30
(c) All linkage groups
s=2 s=3 s=4
0.8
0.6
0.4
0.2
0
0
5
10
15 20 Generation, t
25
30
(d) Average size of spurious linkage
Fig. 10. Linkage group information captured by the probabilistic model of BOA with adaptive scoring metric for different tournament sizes, s = {2, 3, 4}, when solving m = 24 concatenated traps of order k = 5 ( = 120). Tournament selection and full replacement are used.
9 Summary and Conclusions In this chapter we have analyzed the influence of selection and replacement strategies on the structural accuracy of linkage learning in BOA, for concatenated m-k deceptive trap functions. The empirical results obtained led to a better understanding of the model building process in BOA, and consequently to an improvement with respect to model quality. Empirically, we have found that using truncation instead of tournament selection is much better for the purpose of having accurate structural linkage information. Although truncation selection requires larger population sizes, using tournament selection with equivalent population sizes to those required by truncation does not significatively improve the linkage information. For the same purpose, the replacement strategy was found to be relevant only if tournament selection is used, in which case the full replacement of the parents by their offspring is the most appropriate strategy. On the other hand, if overall performance
Linkage Learning Accuracy in the Bayesian Optimization Algorithm
105
(number of function evaluations) is our main concern, tournament selection and restricted tournament replacement are clearly the best options. Intrigued by the empirical observations, we have investigated the learning process of Bayesian networks in BOA to better understand the difference between tournament and truncation selection. Looking at the distribution of the learning data set (mating pool), it has been shown that the nature of the selection operator can have a great impact on the accuracy of Bayesian network learning. Specifically, if we take into account the power distribution in the mating pool generated by tournament selection, the scoring metric that guides the search for an adequate model structure can be modified accordingly. In this case, the model structural accuracy obtained for tournament selection can be highly improved and compared to that of truncation selection, which generates a more suitable uniform distribution for learning. Overall, this chapter provides important information to practitioners about the trade-off between the parameters used in BOA (and consequent computational cost) and the accuracy of the learned linkage information.
Acknowledgments This work was sponsored by the Portuguese Foundation for Science and Technology (FCT/MCTES) under grants SFRH-BD-16980-2004 and PTDC-EIA-677762006, the Air Force Office of Scientific Research, Air Force Materiel Command, USAF, under grant FA9550-06-1-0096, the National Science Foundation under NSF CAREER grant ECS-0547013, ITR grant DMR-03-25939 at Material Computation Center, UIUC. The work was also supported by the High Performance Computing Collaboratory sponsored by Information Technology Services, the Research Award and the Research Board at the University of Missouri in St. Louis. The U.S. Government is authorized to reproduce and distribute reprints for government purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the Air Force Office of Scientific Research, the National Science Foundation, or the U.S. Government.
References [1] Ackley, D.H.: A connectionist machine for genetic hill climbing. Kluwer Academic, Boston (1987) [2] Blickle, T., Thiele, L.: A comparison of selection schemes used in genetic algorithms. Evolutionary Computation 4(4), 311–347 (1997) [3] Brindle, A.: Genetic Algorithms for Function Optimization. PhD thesis, University of Alberta, Edmonton, Canada. Unpublished doctoral dissertation (1981) [4] Chickering, D.M., Heckerman, D., Meek, C.: A Bayesian approach to learning Bayesian networks with local structure. Technical Report MSR-TR-97-07, Microsoft Research, Redmond, WA (1997)
106
C.F. Lima et al.
[5] Cooper, G.F., Herskovits, E.H.: A Bayesian method for the induction of probabilistic networks from data. Machine Learning 9, 309–347 (1992) [6] Correa, E.S., Shapiro, J.L.: Model complexity vs. performance in the bayesian optimization algorithm. In: Runarsson, T.P., Beyer, H.-G., Burke, E.K., MereloGuerv´ os, J.J., Whitley, L.D., Yao, X. (eds.) PPSN 2006. LNCS, vol. 4193, pp. 998–1007. Springer, Heidelberg (2006) [7] Deb, K., Goldberg, D.E.: Analyzing deception in trap functions. Foundations of Genetic Algorithms 2, 93–108 (1993) [8] Echegoyen, C., Lozano, J.A., Santana, R., Larra˜ naga, P.: Exact bayesian network learning in estimation of distribution algorithms. In: Proceedings of the IEEE Congress on Evolutionary Computation, pp. 1051–1058. IEEE Press, Los Alamitos (2007) [9] Friedman, N., Goldszmidt, M.: Learning bayesian networks with local structure. Graphical Models, 421–459 (1999) [10] Goldberg, D.E.: The Design of Innovation - Lessons from and for Competent Genetic Algorithms. Kluwer Academic Publishers, Norwell (2002) [11] Goldberg, D.E., Korb, B., Deb, K.: Messy genetic algorithms: Motivation, analysis, and first results. Complex Systems 3(5), 493–530 (1989) [12] Harik, G.R.: Finding multimodal solutions using restricted tournament selection. In: Proceedings of the Sixth International Conference on Genetic Algorithms, pp. 24–31 (1995) [13] Hauschild, M., Pelikan, M., Lima, C.F., Sastry, K.: Analyzing probabilistic models in hierarchical BOA on traps and spin glasses. MEDAL Report No. 2007001, University of Missouri at St. Louis, St. Louis, MO (2007) [14] Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: The combination of knowledge and statistical data. Technical Report MSR-TR-94-09, Microsoft Research, Redmond, WA (1994) [15] Larra˜ naga, P., Lozano, J.A.: Estimation of distribution algorithms: a new tool for Evolutionary Computation. Kluwer Academic Publishers, Boston (2002) [16] Lima, C.F., Lobo, F.G., Pelikan, M.: From mating-pool distributions to model overfitting. In: Accepted for the ACM SIGEVO Genetic and Evolutionary Computation Conference (GECCO 2008) (2008) [17] Lima, C.F., Pelikan, M., Sastry, K., Butz, M., Goldberg, D.E., Lobo, F.G.: Substructural neighborhoods for local search in the Bayesian optimization algorithm. In: Runarsson, T.P., Beyer, H.-G., Burke, E.K., Merelo-Guerv´ os, J.J., Whitley, L.D., Yao, X. (eds.) PPSN 2006. LNCS, vol. 4193, pp. 232–241. Springer, Heidelberg (2006) [18] Lima, C.F., Sastry, K., Goldberg, D.E., Lobo, F.G.: Combining competent crossover and mutation operators: a probabilistic model building approach. In: Beyer, H., et al. (eds.) Proceedings of the ACM SIGEVO Genetic and Evolutionary Computation Conference (GECCO 2005), pp. 735–742. ACM Press, New York (2005) [19] M¨ uhlenbein, H., Schlierkamp-Voosen, D.: Predictive models for the breeder genetic algorithm: I. Continuous parameter optimization. Evolutionary Computation 1(1), 25–49 (1993) [20] Pearl, J.: Probabilistic reasoning in intelligent systems: Networks of plausible inference. Morgan Kaufmann, San Mateo (1988) [21] Pelikan, M.: Hierarchical Bayesian Optimization Algorithm: Toward a New Generation of Evolutionary Algorithms. Springer, Heidelberg (2005)
Linkage Learning Accuracy in the Bayesian Optimization Algorithm
107
[22] Pelikan, M., Goldberg, D.E.: Escaping hierarchical traps with competent genetic algorithms. In: Spector, L., et al. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2001), pp. 511–518. Morgan Kaufmann, San Francisco (2001) [23] Pelikan, M., Goldberg, D.E., Cantu-Paz, E.: BOA: The Bayesian Optimization Algorithm. In: Banzhaf, W., et al. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference GECCO 1999, pp. 525–532. Morgan Kaufmann, San Francisco (1999) [24] Pelikan, M., Goldberg, D.E., Lobo, F.: A survey of optimization by building and using probabilistic models. Computational Optimization and Applications 21(1), 5–20 (2002) [25] Pelikan, M., Sastry, K.: Fitness inheritance in the bayesian optimization algorithm. In: Deb, K., et al. (eds.) GECCO 2004. LNCS, vol. 3103, pp. 48–59. Springer, Heidelberg (2004) [26] Santana, R., Larra˜ naga, P., Lozano, J.A.: Interactions and dependencies in estimation of distribution algorithms. In: Proceedings of the IEEE Congress on Evolutionary Computation, pp. 1418–1425. IEEE Press, Los Alamitos (2005) [27] Sastry, K.: Evaluation-relaxation schemes for genetic and evolutionary algorithms. Master’s thesis, University of Illinois at Urbana-Champaign, Urbana, IL (2001) [28] Sastry, K., Abbass, H.A., Goldberg, D.E., Johnson, D.D.: Sub-structural niching in estimation distribution algorithms. In: Beyer, H., et al. (eds.) Proceedings of the ACM SIGEVO Genetic and Evolutionary Computation Conference (GECCO 2005). ACM Press, New York (2005) [29] Sastry, K., Goldberg, D.E.: Designing competent mutation operators via probabilistic model building of neighborhoods. In: Deb, K., et al. (eds.) GECCO 2004. LNCS, vol. 3103, pp. 114–125. Springer, Heidelberg (2004) [30] Sastry, K., Lima, C.F., Goldberg, D.E.: Evaluation relaxation using substructural information and linear estimation. In: Keijzer, M., et al. (eds.) Proceedings of the ACM SIGEVO Genetic and Evolutionary Computation Conference (GECCO 2006), pp. 419–426. ACM Press, New York (2006) [31] Sastry, K., Pelikan, M., Goldberg, D.E.: Efficiency enhancement of genetic algorithms via building-block-wise fitness estimation. In: Proceedings of the IEEE International Conference on Evolutionary Computation, pp. 720–727 (2004) [32] Thierens, D., Goldberg, D.E.: Mixing in genetic algorithms. In: Forrest, S. (ed.) Proceedings of the Fifth International Conference on Genetic Algorithms, pp. 38–45. Morgan Kaufmann, San Mateo (1993) [33] Wu, H., Shapiro, J.L.: Does overfitting affect performance in estimation of distribution algorithms. In: Keijzer, M., others (eds.) Proceedings of the ACM SIGEVO Genetic and Evolutionary Computation Conference (GECCO 2006), pp. 433–434. ACM Press, New York (2006) [34] Yu, T.-L., Goldberg, D.E.: Dependency structure matrix analysis: Offline utility of the dependency structure matrix genetic algorithm. In: Deb, K.,et al. (eds.) GECCO 2004. LNCS, vol. 3103, pp. 355–366. Springer, Heidelberg (2004) [35] Yu, T.-L., Sastry, K., Goldberg, D.E.: Population size to go: Online adaptation using noise and substructural measurements. In: Lobo, F.G., et al. (eds.) Parameter Setting in Evolutionary Algorithms, pp. 205–224. Springer, Heidelberg (2007)
The Impact of Exact Probabilistic Learning Algorithms in EDAs Based on Bayesian Networks Carlos Echegoyen1, Roberto Santana1 , Jose A. Lozano1 , and Pedro Larra˜ naga2 1
2
Intelligent Systems Group Department of Computer Science and Artificial Intelligence University of the Basque Country Paseo Manuel de Lardizabal 1, 20018 Donostia - San Sebastian, Spain {carlos.echegoyen,roberto.santana,ja.lozano}@ehu.es Department of Artificial Intelligence, Technical University of Madrid, 28660 Boadilla del Monte, Madrid, Spain
[email protected]
Summary. This paper discusses exact learning of Bayesian networks in estimation of distribution algorithms. The estimation of Bayesian network algorithm (EBNA) is used to analyze the impact of learning the optimal (exact) structure in the search. By applying recently introduced methods that allow learning optimal Bayesian networks, we investigate two important issues in EDAs. First, we analyze the question of whether learning more accurate (exact) models of the dependencies implies a better performance of EDAs. Secondly, we are able to study the way in which the problem structure is translated into the probabilistic model when exact learning is accomplished. The results obtained reveal that the quality of the problem information captured by the probability model can improve when the accuracy of the learning algorithm employed is increased. However, improvements in model accuracy do not always imply a more efficient search.
1 Introduction In estimation of distribution algorithms (EDAs) [21, 30] linkage learning, understood as the ability to capture the relationships between the variables of the optimization problem, is accomplished by detecting and representing probabilistic dependencies using probability models. EDAs are evolutionary algorithms that do not employ classical genetic operators such as mutation or crossover. Instead, machine learning methods are used to extract relevant features of the search space. The collected information is represented using a probabilistic model which is later employed to generate new points. During the sampling or generation step, the statistical dependencies between the variables are used for the construction of the new solutions. In EDAs, the ability of learning an accurate representation of the relationships between the variables is related to the class of probabilistic models used Y.-p. Chen, M.-H. Lim (Eds.): Linkage in Evolutionary Computation, SCI 157, pp. 109–139, 2008. c Springer-Verlag Berlin Heidelberg 2008 springerlink.com
110
C. Echegoyen et al.
and the methods employed to learn them. One class of model that has been extensively applied in EDAs is Bayesian networks [18]. Among the benefits of EDAs that use this type of models [14, 28, 33, 45] is that the complexity of the learned structure depends on the characteristics of the data (selected individuals). Additionally, the Bayesian networks learned during the search are suitable for human interpretation, aiding the discovery of unknown information about the problem structure. Although, in the case of EDAs that use Bayesian networks, the role of the parameters that penalize the complexity of the networks has been studied [28, 32], a detailed analysis of the accuracy of the methods used for finding the best network and its influence in the behavior of EDAs has not been conducted. An initial attempt to investigate this problem was presented in [13], where exact Bayesian learning was introduced to EDAs. Methods that do exact Bayesian structure learning [12, 20, 42, 43] compute, given a set of data and a prespecified score (in our case, the BIC score [41]), the network structure that optimizes the score. Since the problem of learning the optimal Bayesian network is NP-hard [6], these methods set constraints on the maximum number of variables and/or cases they can deal with. Usually, dynamic programming algorithms are used to learn the structure. In this chapter, we extend the preliminary results presented in [13] and provide evidence that the methods for learning optimal (exact) Bayesian networks can be very useful to analyze the relationship between the search space and the structure of the learned probabilistic models. The advantage of using methods that learn exact models is because alternative approximate methods commonly applied to learn the models in EDAs are very often able to find only suboptimal solutions. Therefore, using exact learning makes it easier to investigate to what extent approximate learning algorithms are responsible for the loss in accuracy in the mapping between the problem structure and the model structure. In general, an exact learning algorithm can serve as a different framework to investigate the influence of the EDA components in the ability of the probability models to capture the problem structure. The chapter is organized as follows. In the next section, Bayesian networks are presented, the general procedures to learn these networks from data are also discussed. In Section 3, we focus on the type of search strategies used to find the Bayesian network structure. Approximate and exact learning methods are analyzed. Section 4 introduces the EBNA algorithm. In Section 5, the experimental framework and functions used to evaluate the exact and local learning methods used by EBNA are introduced. Sections 6 and 7 respectively present experimental results on the time complexity analysis and convergence reliability of the two EBNA variants. Section 8 analyzes ways for using the Bayesian networks learned by EBNA as a source of problem knowledge and presents experimental results for several functions. Work related to our proposal is analyzed in Section 9. The conclusions of our paper are presented in Section 10.
The Impact of Exact Probabilistic Learning Algorithms
111
2 Bayesian Networks 2.1
Notation
Let X be a random variable. A value of X is denoted x. X = (X1 , . . . , Xn ) will denote a vector of random variables. We will use x = (x1 , . . . , xn ) to denote an assignment to the variables. We will work with discrete variables. The joint probability mass function of x is represented as p(X = x) or p(x). p(xS ) will denote the marginal probability distribution for XS . We use p(Xi = xi | Xj = xj ) or, in a simplified form, p(xi | xj ), to denote the conditional probability distribution of Xi given Xj = xj . Formally, a Bayesian network [5] is a pair (S, θ) representing a graphical factorization of a probability distribution. The structure S is a directed acyclic graph which reflects the set of conditional (in)dependencies among the variables. The factorization of the probability distribution is codified by S: p(x) =
n
p(xi |pai )
i=1
where pai denotes a value of variable Pai , the parent set of Xi (variables from which there exists an arc to Xi in the graph S), while θ is a set of parameters for the local probability distributions associated with each variable. If the variable Xi has ri possible values, x1i , . . . , xri i , the local distribution p(xi |paji , θ i ) is an unrestricted discrete distribution: p(xki |paji , θi ) ≡ θijk where pa1i , . . . , paqi i denote the values of Pai and the term qi denotes the number of possible different instances of the parent variables of Xi . In other words, the parameter θijk represents the probability of variable Xi being in its k-th value, knowing that the set of its parent variables is in its j-th value. Therefore, the i i )qj=1 ). local parameters are given by θ i = (((θijk )rk=1 2.2
Learning Bayesian Networks from Data
There are different strategies to learn the structure of a Bayesian network. We focus on a method called “score + search” which is the one used in the experiments presented in this paper. In this strategy, given a set of data D and a Bayesian network whose structure is denoted by S, a value (score) which evaluates how well the Bayesian network represents the probability distribution of the database D is assigned. Different scores can be used. In this work we have used the Bayesian Information Criterion score (BIC) [41] (based on penalized maximum likelihood).
112
C. Echegoyen et al.
A general formula for a penalized maximum likelihood score can be written as follows: ˆ − f (N )dim(S) log p(D|S, θ) where dim(S) is the dimension –number of parameters needed to specify the model– of the Bayesian network with a structure given by S. Thus: dim(S) =
n
qi (ri − 1)
i=1
and f (N ) is a non negative penalization function. The Jeffreys-Schwarz criterion, sometimes called BIC [41], takes into account f (N ) = 12 log N . Thus the BIC score can be written as follows: BIC(S, D) = log
N n
1 log N qi (ri − 1) 2 i=1 n
ˆi) − p(xw,i |paSi , θ
w=1 i=1
(1)
To find the Bayesian network that optimizes the score implies solving an optimization problem. This can be done with exhaustive or heuristic search algorithms. In Section 3, we analyze two variants for finding the Bayesian network structures. Each structure is evaluated using the maximum likelihood parameters. 2.3
Learning of the Parameters
Once the structure has been learned, the parameters of the Bayesian network are calculated using the Laplace correction: Nijk + 1 θˆijk = Nij + ri
(2)
where Nijk denotes the number of cases in D in the variable Xi has the which ri value xki and P ai has its j th value, and Nij = k=1 Nijk .
3 Methods for Learning Bayesian Networks Once we have defined a score to evaluate Bayesian networks, we have to set a search process to find the Bayesian network that maximizes the score given the data. Approximate and exact methods can be used. 3.1
Learning an Approximate Model
In practical applications, we need to find an adequate model structure as quickly as possible. Therefore, a simple algorithm which returns a good structure, even if not optimal, is preferred. An algorithm that fulfills these criteria is Algorithm B [4] which is typically used by most of Bayesian network based EDAs. Algorithm B is a greedy search which starts with an arcless structure and, at each step, adds the arc with the maximum improvement in the score. The algorithm finishes when there is no arc whose addition improves the score.
The Impact of Exact Probabilistic Learning Algorithms
3.2
113
Learning the Exact Model
Since learning the Bayesian network structure is an NP-hard problem, for a long time the goal of learning exact Bayesian networks was constrained to problems with a very reduced number of variables. In [20], an algorithm for learning the exact structure in less than super-exponential complexity with respect to n is introduced for the first time. In [42], the most efficient method so far is presented and it is used in our work. This algorithm is feasible for n < 33 and it was shown to learn a best network for a data set of 29 variables. In that work, the Bayesian network structure S is defined as a vector S = (S1 , .., Sn ) of parent sets, where Si is the subset of X from which there are arcs to Xi , for example S1 is the parent set of X1 . Another necessary concept is the variable ordering. This is simply the variables of X in a determined order. In this order, the ith element is denoted by ordi . Therefore, the structure S = (S1 , .., Sn ) is said to be consistent with an ordering ord when all the parents of the node precede the node in the ordering. Another important concept in the algorithm is the sink node. Every DAG has at least one node with no outgoing arcs, so at least one node is not a parent of any other node. These nodes are called sinks of the network. In this algorithm, the data set D is processed in a particular way and it uses two kinds of data tables. Given W ⊆ X, first it is defined the contingency table CT (W) to be a list of the frequencies of different data-vectors dW in DW , where DW is the data set for W variables. However, the main task is to calculate conditional frequency tables CF T (Xi , W) that record how many times different values of the variable Xi occur together with different vectors dW−{Xi } in the data. On the other hand, many popular scores such as BIC, AIC and BDe can be decomposed to local scores: score(S) =
n i=1
scorei (Si ) =
n
score(CF T (Xi , Si )),
i=1
Thus, the score of the network is the sum of the local scores that only depend on the conditional frequency table for one variable and its parents. Algorithm 1 presents the main steps of the method: The first step is the main procedure and the only one for which data is needed. It starts by calculating the contingency table for all the variables X and continues calculating contingency tables for all smaller variable subsets, marginalizing variables out of the contingency table. After that, for each contingency table, the conditional frequency table is calculated for each variable appearing in the contingency table. These conditional frequency tables can then be used to calculate the local scores for any parent set given a variable. All the n2n−1 local scores are stored in a table which will be the basis of the algorithm. Having calculated the local scores, the best parents for Xi given a candidate set C are either the whole candidate set C itself or one of the smaller candidate
114
C. Echegoyen et al.
Algorithm 1. Exact learning algorithm 1
Calculate the local scores for all n2n−1 different (variable, variable set)-pairs
Using the local scores, find best parents for all n2n−1 (variable, variable set)-pairs 3 Find the best sink for all 2n variable sets 4 Using the results from Step 3, find a best ordering of the variables 5 Find a best network using results computed in Steps 2 and 4 2
sets {C\{c}|c ∈ C}. It must be computed for all 2n−1 variable sets (parent candidate sets) related with Xi . Step 3 of the algorithm is based on the following observation: The best network G∗ for a variable set W must have a sink s. As G∗ is a network with the highest score, sink s must have incoming arcs from its best possible set of parents. In this way, the rest of the nodes and the arcs must form the best possible network for variables W \{s}. Therefore, the best sink for W, sink ∗ (W), is the node that maximizes the sum between the local score for s and the score for the network S without node s. When we have the best sinks for all 2n variable sets, it is possible to yield the best ordering ord∗ in reverse order. Then, for each position from |X| to 1, |X| in ord∗i we have to store the best sink for the set j=i+1 {ord∗j (X)}. Having a best ordering and a table with the best parents for any candidate set, it is possible to obtain a best network consistent with the given ordering. For the ith variable in the optimal ordering, the best parents from its predecessors are picked. More details about the algorithm can be found in [42]. We use an implementation of Algorithm 1 given by the authors1 . The computational complexity of the algorithm is o(n2 2n−2 ). The memory requirement of the method is 2n+2 bytes and the disk-space requirement is 12n2n−1 bytes.
4 Estimation of Distribution Algorithms Based on Bayesian Networks The estimation of Bayesian networks algorithm (EBNA) allows statistics of unrestricted order in the factorization of the joint probability distribution. This distribution is encoded by a Bayesian network that is learned from the database containing the selected individuals at each generation. It has been applied with good results to a variety of problems [3, 19, 23, 24, 25]. Other algorithms based 1
The C++ code of this implementation is available from http://www.cs.helsinki.fi/u/tsilande/sw/bene/download/
The Impact of Exact Probabilistic Learning Algorithms
115
Algorithm 2. EBNABIC 1 2 3 4 5 6 7 8 9 10 11
BN0 ← (S0 , θ0 ) where S0 is an arc-less DAG, and θ0 is uniform n n p0 (x) = i=1 p(xi ) = i=1 r1i D0 ← Sample M individuals from p0 (x) and evaluate them t←1 do { Se Dt−1 ← Select N individuals from Dt−1 ∗ St ← Using a search method find one network structure according to the BIC score t Se θt ← Calculate θijk using Dt−1 as the data set BNt ← (St∗ , θt ) Dt ← Sample M individuals from BNt and evaluate them } until Stopping criterion is met
on the use of Bayesian networks have been proposed in [28, 33, 45]. A pseudocode of EBNA is shown in Algorithm 2. In the experiments presented in this paper, EBNA uses truncation selection and the number of selected individuals equals half of the population. The best solution at each generation is passed to the next population, therefore, at each generation N − 1 new solutions are sampled. The stopping criterion is changed according to the type of experiments conducted.
5 Experimental Framework and Function Benchmark To investigate the impact of exact learning in the behavior of Bayesian network based EDAs, we compare the EBNA versions that use the two different Bayesian network learning schemes described in Section 3. We call them EBNA-Exact and EBNA-Local. We used three different criteria to compare the algorithms. The time complexity, the convergence reliability and the way in which probabilistic dependencies are represented in the structure of the Bayesian network. In this section, we introduce a set of functions that represent different classes of problems and which are used in the following sections to test the behavior of EDAs. 5.1
Function Benchmark n n Let u(x) = i=1 xi , f (x) be a unitation function if ∀x, y ∈ {0, 1} , u(x) = u(y) ⇒ f (x) = f (y). A unitation function is defined in terms of its unitation value u(x), or in a simpler way u.
116
C. Echegoyen et al.
Function OneM ax: OneM ax(x) =
n
xi = u(x)
(3)
i=1
Unitation functions are also useful for the definition of a class of functions where the difficulty is given by the interactions that arise among subsets of variables. One example of this class of deceptive functions is f3deceptive [15]: i= n 3
f3deceptive (x) =
3 fdec (x3i−2 , x3i−1 , x3i )
(4)
i=1 3 is defined as: where fdec
⎧ 0.9 f or ⎪ ⎪ ⎨ 0.8 f or 3 fdec (u) = ⎪ 0.0 f or ⎪ ⎩ 1.0 f or
u=0 u=1 u=2 u=3
Function SixP eaks is a modification of the F ourP eaks problem [1] and it can be defined mathematically as: SixP eaks(x, t) = max{tail(0, x), head(1, x), tail(1, x), head(0, x)} + R(x, t) (5) where tail(b, x) = number of trailing b’s in x head(b, x) = number of leading b’s in x
R(x, t) =
⎧ ⎨n ⎩
0
if tail(0, x) > t and head(1, x) > t or tail(1, x) > t and head(0, x) > t otherwise
The goal is to maximize the function. For an even number of variables this function has 4 global optima, located at the points: t
t
t
t
(0, . . . , 0, 1, . . . , 1) (0, . . . , 0, 1, . . . , 1) (1, . . . , 1, 0, . . . , 0) (1, . . . , 1, 0, . . . , 0) These points are very difficult to reach because they are isolated. On the other hand, two local optima (0, 0, . . . , 0), (1, 1, . . . , 1) are very easily reachable. The value of t was set to n2 − 1. The P arity function [8] is a simple k-bounded additively separable function that has been used to investigate the limitations of linkage learning by probabilistic modeling. It can be seen as a generalization of the XOR function and the Walsh transform. In this case we will work with the concatenated parity function
The Impact of Exact Probabilistic Learning Algorithms
117
(CPF) [8]. It is said that this problem is hard for EDAs in general. The P arity function can be defined mathematically as: Ceven if u(x) is even parity(x) = Codd otherwise where Ceven and Codd are parameters of the function. The CPF is defined as m concatenated parity sub-functions, CP F (x) =
m−1
parity(xik+1 , ..., xik+k )
(6)
i=0
Where k is the size for each sub-function. As in [8], we use k = 5, Codd = 5 and Ceven = 0. Notice that there are 2n−m solutions where the function reaches the global optima. Function Cuban5 [29] is a non-separable additive function. The second best value of this function is very close to the global optimum. Cuban5(x) = 5 (s0 ) Fcuban1
+
m
5 5 (Fcuban2 (s2j+1 ) + Fcuban1 (s2j+2 ))
(7)
j=0
where si = x4i x4i+1 x4i+2 x4i+3 x4i+4 and n = 4(2m + 1) + 1 ⎧ 0.595 f or x = 000 ⎪ ⎪ ⎪ ⎪ 0.200 f or x = 001 ⎪ ⎪ ⎪ ⎪ 0.595 f or x = 010 ⎪ ⎪ ⎨ 0.100 f or x = 011 3 (x) = Fcuban1 1.000 f or x = 100 ⎪ ⎪ ⎪ ⎪ 0.050 f or x = 101 ⎪ ⎪ ⎪ ⎪ 0.090 f or x = 110 ⎪ ⎪ ⎩ 0.150 f or x = 111 5 Fcuban1 (x) = 3 4Fcuban1 (x1 , x2 , x3 ) if x2 = x4 and x3 = x5 0 otherwise
5 (x) = Fcuban2
(8)
⎧ ⎨
u(x) f or x5 = 0 0 f or x1 = 0, x5 = 1 ⎩ u(x) − 2 f or x1 = 1, x5 = 1
The HP protein model In our experiments we also use a class of coarse-grained protein folding model called the hydrophobic-polar (HP) model [11].
118
C. Echegoyen et al.
Fig. 1. An optimal solution of the HP model for sequence HP HP P HHP HP P HP HHP P HP H. The optimal energy corresponding to this sequence is −9.
Under specific conditions, a protein sequence folds into a native 3-d structure. The problem of determining the protein native structure from its sequence is known as the protein structure prediction problem. To solve this problem, a protein model is chosen and an energy is associated to each possible protein fold. The search for the protein structure is transformed into the search for the optimal protein configuration given the energy function. The HP model considers two types of residues: hydrophobic (H) residues and hydrophilic or polar (P) residues. In the model, a protein is considered as a sequence of these two types of residues, which are located in regular lattice models forming self-avoided paths. Given a pair of residues, they are considered neighbors if they are adjacent either in the chain (connected neighbors) or in the lattice but not connected in the chain (topological neighbors). The total number of topological neighboring positions in the lattice (z) is called the lattice coordination number. Figure 1 shows one possible configuration of sequence HP HP P HHP HP P HP HHP P HP H in the HP model. A solution x can be interpreted as a walk in the lattice, representing one possible folding of the protein. We use a discrete representation of the solutions. For a given sequence and lattice, Xi will represent the relative move of residue i in relation to the previous two residues. Taking as a reference the location of the previous two residues in the lattice, Xi takes values in {0, 1, . . . , z − 2}, where z −1 is the number of movements allowed in the given lattice. These values respectively mean that the new residue will be located in one of the z−1 numbers of possible directions with respect to the previous two locations. If the encoded solution is self-intersecting, it can be repaired or penalized during the evaluation step using a recursive repairing procedure introduced in [9]. Therefore, values for X1 and X2 are meaningless. The locations of these two residues are fixed. For the HP model, an energy function that measures the interaction between topological neighbor residues is defined as HH = −1 and HP = P P = 0. The HP problem consists of finding the solution that minimizes the total energy. More details about the representation and function can be found in [40].
The Impact of Exact Probabilistic Learning Algorithms
119
6 Time Complexity Analysis In our case, the time complexity analysis will refer to the study of the average number of generations needed by EBNA-Local and EBNA-Exact to find the optimum. Experiments were conducted for three functions, OneM ax, f3deceptive and SixP eaks. In the first function, there are no interactions between the variables. In the rest, interactions arise between variables that belong to the same definition set of the function. To determine the average number of generations to find the optimum needed by EBNA-Local and EBNA-Exact, we start with a population of 10 individuals and the population size is increased by 10 until a maximum population size of 150 is reached. For each possible combination of function, number of variables n, and population size N , 50 experiments are conducted. For each execution of the algorithm, a maximum of 105 evaluations are allowed. For the OneM ax function we conducted experiments for n ∈ {15, 20}. In order to increase the accuracy of the curves shown in the first figure, for n = 15 we exceptionally conducted 100 experiments. The original idea was to evaluate, under the dimension constraints imposed by the exact learning algorithm, the scalability of both EBNA versions with n ∈ {10, 12, 15, 20}. Nevertheless, in this work we only present the results achieved for the last two sizes. The results of the experiments for n = 15 are shown in Figures 2 (a) and the average results for n = 20 are shown in Figure 2 (b). The analysis of Figure 2 reveals that both algorithms exhibit the same time complexity pattern. However, EBNA-Exact needs, in general, a higher number of evaluations than EBNA-Local to find the optimal solution for the first time. The difference in the number of generations is less evident when the population size approaches 150. For this simple function, it seems that the error in the learning of the model, introduced by the approximate learning algorithm, is beneficial for the search.
18
25 EBNA Exact EBNA Local
16
EBNA Exact EBNA Local
14
Generation number
Generation number
20 12 10 8 6
15
10
5 4 2 0
50
100
Population size
a)
150
0 0
50
100
150
Population size
b)
Fig. 2. Time complexity analysis for function OneM ax,(a) n = 15 and (b) n = 20
120
C. Echegoyen et al.
600
700 EBNA Exact EBNA Local
400 300 200 100 0 0
EBNA Exact EBNA Local
600
Generation number
Generation number
500
500 400 300 200 100
50
100
0 0
150
Population size
50
100
150
Population size
a)
b)
Fig. 3. Time complexity analysis for function f3deceptive ,(a) n = 15 and (b) n = 18
9
11 EBNA Exact EBNA Local
7 6 5 4 3 2 0
EBNA Exact EBNA Local
10
Generation number
Generation number
8
9 8 7 6 5 4
50
100
Population size
a)
150
3 0
50
100
150
Population size
b)
Fig. 4. Time complexity analysis for function SixP eaks, (a) n = 14 and (b) n = 16
In Figure 3 we can observe that both algorithms have an identical curve. For this function, and for the values of n investigated, the influence of the exact learning is not relevant. We can anticipate that the structures learned by both algorithms are similar. For this function, a small population size determines many generations are necessary to reach the optimum. For the SixP eaks function, the optimal value is reached in a significantly lower number of generations as can be appreciated in Figure 4. It can also be observed that EBNA-Local is able to reach the optimum earlier than EBNAExact. From this analysis we deduce that for SixP eaks function, the structures learned by both algorithms could be different. We will later further analyze this behavior of the algorithms when discussing the structures of the models they learn for the f3deceptive and SixP eaks functions. To illustrate the complexity of the function and to study in more detail the algorithms, we introduce Figures 5 and 6 which show the total number of executions needed by each algorithm in order to succeed, i.e. to find the optimum. It should be noticed that in every run, the maximum number of evaluations is bounded by 105 . This constraint strongly influences the behavior of the algorithms.
The Impact of Exact Probabilistic Learning Algorithms
12
30 EBNA Exact EBNA Local
8 6
20 15
4
10
2
5
50
100
EBNA Exact EBNA Local
25
Executions
Executions
10
0 0
121
0 0
150
Population size
50
100
150
Population size
a)
b)
Fig. 5. Number of executions, for each population size, in order to obtain one optimal value for f3deceptive function, (a) n = 15, (b) n = 18
14
18 EBNA Exact EBNA Local
12
EBNA Exact EBNA Local
16 14
Executions
Executions
10 8 6
12 10 8 6
4
4 2 0 0
2 50
100
150
Population size
a)
0 0
50
100
150
Population size
b)
Fig. 6. Number of executions, for each population size, in order to obtain one optimal value for SixP eaks function, (a) n = 14, (b) n = 16
7 Convergence Reliability In the analysis of the convergence reliability, we focus on the critical population size needed by the EDAs to achieve a predefined convergence rate. In the experiments conducted, the goal was to determine the minimum population size needed by the two different variants of EBNA to find the optimum in 20 consecutive experiments. We investigated the behavior of the algorithms for functions Cuban5 (n = 13), SixP eaks (n ∈ {10, 12, 14}) and f3deceptive (n ∈ {9, 12, 15}). The algorithm begins with a population size N = 16 which is doubled until the optimal solution has been found in 20 consecutive experiments. The maximum number of evaluations allowed is 104 . For each function and value of n, 25 experiments are carried out. Table 1 shows the mean and standard deviation of the critical population size found. Table 1 shows that for function Cuban5, EBNA-Exact requires a slightly higher population size than EBNA-Local. The picture is drastically changed for functions SixP eaks and f3deceptive , for which EBNA-Exact needs a much smaller
122
C. Echegoyen et al.
Table 1. Mean, standard deviation and p-value of the critical population size for different functions and number of variables f unction n EBN A − Exact mean std Cuban5 13 118.40 53.07 SixP eaks 10 153.60 52.26 SixP eaks 12 209.92 110.11 SixP eaks 14 312.32 133.64 f3deceptive 9 135.68 38.40 60.94 f3deceptive 12 168.96 58.66 f3deceptive 15 220.16
EBN A − Local T − T est mean std p − value 109.44 57.26 0.57 215.04 109.11 0.014 389.12 249.19 0.019 604.16 318.97 0.001 168.96 60.94 0.025 261.12 86.50 0.001 296.96 95.79 0.0013
population size. This difference is particularly evident for function SixP eaks. Another observation is that the standard deviation of EBNA-Local is always higher than that of EBNA-Exact. Since the only difference between EBNAExact and EBNA-Local is in the class of algorithm used to learn the models, the difference of behaviors is due to the ability of EBNA-Exact to learn a more accurate model of the dependencies. Therefore, at least for functions SixP eaks and f3deceptive , learning a more accurate model determines a better performance of EBNA. To determine if the population sizes obtained for each algorithm are significantly different, we have carried out a Student’s t-test over the two sets of 25 population sizes for each function and value of n. In the last column of Table 1, the probability values of the test are reported. If we consider a significance level of 0.05, we would have to reject the null hypotesis for all cases except for Cuban5 where there are not significant differences. An explanation of the similar behavior achieved with both algorithms for Cuban5 will be presented in the next section, where the structures of the probabilistic models learned by the algorithms are studied. Moreover, for SixP eaks, using the highest value of n, and for f3deceptive , using the two highest problem sizes, the difference between the algorithms is statistically significant at the 1% level.
8 Problem-Knowledge Extraction from Bayesian Networks The objective of this section is to show a number of ways in which knowledge about the problem structure can be extracted from the analysis of the Bayesian networks learned by EBNA. In particular, we investigate the difference between the structures learned using exact and approximate learning algorithms. We also analyze the changes in the pattern (number and type) of the dependencies captured by the algorithms during their evolution.
The Impact of Exact Probabilistic Learning Algorithms
8.1
123
Probabilistic Models as a Source of Knowledge About the Problem
Although the main objective in EDAs is to obtain a set of optimal solutions, the analysis of the models learned by the algorithms during the evolution can reveal previously unknown characteristics of the problem. There is a variety of information that can be obtained from the analysis of the models. Just to cite a few examples, it could be possible to extract: • A description of sets of dependent or interacting variables. • Probabilistic information about most likely configurations for subsets of the problem variables which can be translated into most-probable partial solutions of the problem. • Evidence on the existence of different types of problem symmetry. • Identification of conflicting partial solutions in problems with frustration. • In addition, by considering the change of the models during the evolution (a dynamical perspective), it is also possible to identify patterns in the formation of optimal structures. A central problem in EDAs is the design of methods for extracting and interpreting this information from the models. There are a number of approaches that have been proposed to treat this issue for different classes of probabilistic models used in EDAs. We postpone a review of some of these approaches for the next section and focus now on the extraction of information from Bayesian networks. We identify three main sources of information: 1. The structure of the Bayesian network: By inspecting the topological characteristics of the graphs (e.g. most frequent arcs), we identify structural relationships between the variables. 2. The probabilistic tables of the Bayesian networks: By analyzing the probability associated to variables linked in the network, it is possible to identify promising and also poor configurations of the partial solutions. 3. Most probable configurations given the network: These are the solutions with the highest probability given the model. Thus, they condense the structural and parametrical information stored by the Bayesian network and have not necessarily been generated during the evolution of the EDA. In this paper we focus on the analysis of network structures. 8.2
Analysis of the Bayesian Structures Learned by EBNA
To investigate the type of dependencies learned by EBNA-Exact and EBNALocal, we saved the structures of the Bayesian networks learned during the evolutionary process by both variants of the algorithm for functions f3deceptive , SixP eaks, Cuban5, CP F and P rotein. In all the following experiments, we start by running EBNA-Local and EBNAExact and choose 30 executions in which the optimum was found and the algorithms did not converge in the first generation. The stopping criterion is a
124
C. Echegoyen et al.
maximum number of 105 evaluations. In each of these