Computational Intelligence in Optimization: Applications and Implementations (Adaptation, Learning, and Optimization)

Yoel Tenne and Chi-Keong Goh (Eds.) Computational Intelligence in Optimization

Adaptation, Learning, and Optimization, Volume 7 Series Editor-in-Chief Meng-Hiot Lim Nanyang Technological University, Singapore E-mail: [email protected] Yew-Soon Ong Nanyang Technological University, Singapore E-mail: [email protected] Further volumes of this series can be found on our homepage: springer.com Vol. 1. Jingqiao Zhang and Arthur C. Sanderson Adaptive Differential Evolution, 2009 ISBN 978-3-642-01526-7 Vol. 2. Yoel Tenne and Chi-Keong Goh (Eds.) Computational Intelligence in Expensive Optimization Problems, 2010 ISBN 978-3-642-10700-9 Vol. 3. Ying-ping Chen (Ed.) Exploitation of Linkage Learning in Evolutionary Algorithms, 2010 ISBN 978-3-642-12833-2 Vol. 4. Anyong Qing and Ching Kwang Lee Differential Evolution in Electromagnetics, 2010 ISBN 978-3-642-12868-4 Vol. 5. Ruhul A. Sarker and Tapabrata Ray (Eds.) Agent-Based Evolutionary Search, 2010 ISBN 978-3-642-13424-1 Vol. 6. John Seiffertt and Donald C. Wunsch Unified Computational Intelligence for Complex Systems, 2010 ISBN 978-3-642-03179-3 Vol. 7. Yoel Tenne and Chi-Keong Goh (Eds.) Computational Intelligence in Optimization, 2010 ISBN 978-3-642-12774-8

Yoel Tenne and Chi-Keong Goh (Eds.)

Computational Intelligence in Optimization Applications and Implementations

123

Dr. Yoel Tenne Department of Mechanical Engineering and Science-Faculty of Engineering, Kyoto University, Yoshida-honmachi, Sakyo-Ku, Kyoto 606-8501, Japan E-mail: [email protected] Formerly: School of Aerospace Mechanical and Mechatronic Engineering, Sydney University, NSW 2006, Australia

Dr. Chi-Keong Goh Advanced Technology Centre, Rolls-Royce Singapore Pte Ltd 50 Nanyang Avenue, Block N2, Level B3C, Unit 05-08, Singapore 639798 E-mail: [email protected]

ISBN 978-3-642-12774-8

e-ISBN 978-3-642-12775-5

DOI 10.1007/978-3-642-12775-5 Adaptation, Learning, and Optimization

ISSN 1867-4534

Library of Congress Control Number: 2010926028 c 2010 Springer-Verlag Berlin Heidelberg 

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typeset & Cover Design: Scientific Publishing Services Pvt. Ltd., Chennai, India. Printed on acid-free paper 987654321 springer.com

To our families for their love and support.

Preface

Optimization is an integral part to science and engineering. Most real-world applications involve complex optimization processes, which are difficult to solve without advanced computational tools. With the increasing challenges of fulfilling optimization goals of current applications there is a strong drive to advance the development of efficient optimizers. The challenges introduced by emerging problems include: • objective functions which are prohibitively expensive to evaluate, so typically so only a small number of objective function evaluations can be made during the entire search, • objective functions which are highly multimodal or discontinuous, and • non-stationary problems which may change in time (dynamic). Classical optimizers may perform poorly or even may fail to produce any improvement over the starting vector in the face of such challenges. This has motivated researchers to explore the use computational intelligence (CI) to augment classical methods in tackling such challenging problems. Such methods include population-based search methods such as: a) evolutionary algorithms and particle swarm optimization and b) non-linear mapping and knowledge embedding approaches such as artificial neural networks and fuzzy logic, to name a few. Such approaches have been shown to perform well in challenging settings. Specifically, CI are powerful tools which offer several potential benefits such as: a) robustness (impose little or no requirements on the objective function) b) versatility (handle highly non-linear mappings) c) self-adaption to improve performance and d) operation in parallel (making it easy to decompose complex tasks). However, the successful application of CI methods to real-world problems is not straightforward and requires both expert knowledge and trial-and-error experiments. As such the goal of this volume is to survey a wide range of studies where CI has been successfully applied to challenging real-world optimization problems, while highlighting the

VIII

Preface

insights researchers have obtained. Broadly, the studies in this volume focus on four main disciplines: continuous optimization, classification, scheduling and hardware implementations. For continuous optimization, Neto et al. study the use of artificial neural networks (ANNs) and Heuristic Rules for solving large scale optimization problems. They focus on a recurrent ANN to solve a quadratic programming problem and propose several techniques to accelerate convergence of the algorithm. Their method is more efficient than one using an ANN only. Starzyk et al. propose a direct-search optimization algorithm which uses reinforcement learning, resulting in an algorithm which ‘learns’ the best path during the search. The algorithm weights past steps based on their success to yield a new candidate search step. They benchmark their algorithm with several mathematical test functions and apply to training of a multi-layer perceptron neural network for image recognition. Ventresca et al. use the opposition sampling approach to decrease the number of function evaluations. The approach attempts to sample the function in a subspace generated by the ‘opposites’ of an existing population of candidates. They apply their method to differential evolution and incremental learning and show that the opposition method improves performance over baseline variants. Bazan studies an optimization algorithm for problems where the objective function requires large computational resources. His proposed algorithm uses locally regularized approximations of the objective function using radial basis functions. He provides convergence proofs and formulates a framework which can be applied to other algorithms such as Gauss-Seidel or the Conjugate Directions. Ruiz-Torrubiano et al. study hybrid methods for solving large scale optimization problems with cardinality constraints, a class of problems arising in diverse areas such as finance, machine learning and statistical data analysis. While existing methods can provide exact solutions (such as branch-andbound) they require large resources. As such the study focuses on methods which can efficiently identify approximate solutions but require far less computer resources. For problems where it is expensive to evaluate the objective function, Jayadeva et al. propose using a support-vector machine to predict the location of yet undiscovered optima. Their framework can be applied to problems where little or no a-priori information is available on the objective function, as the algorithm ‘learns’ during the search process. Benchmarks show their method can outperform existing methods such as particle swarm optimization or genetic algorithms. Vouchkov and Keane study multi-objective optimization problems using surrogate-models. They investigate how to efficiently update the surrogates under a small optimization ‘budget’ and compare different updating strategies. They also shows that using a number of surrogate models can improve the optimization search and that the size of the ensemble should increase with the problem dimension. Others study agents-based algorithms, that is, where the optimization is done by agents which co-operate during the search. Dre˙zewski and Siwik review agent-based co-evolutionary algorithms for multi-objective problems.

Preface

IX

Such algorithms combine co-evolution (multiple species) with the agent approach (interaction). They review and compare existing methods and benchmark them over a range of test problems. Results show the agent-based coevolutionary algorithms can perform equally well and even surpass some of the best existing multi-objective evolutionary algorithms. Salhi and T¨ oreyen proposes a multi-agent algorithm based on game theory. Their framework uses multiple solvers (agents) which compete over available resources and their algorithm identifies the most successful solver. In the spirit of game theory, successful solvers are rewarded by increasing their computing resources and vice versa. Test results show the framework provides a better final solution when compared to using a single solver. For applications in classification, Arana-Daniel et al. use Clifford algebra to generalize support vector machines (SVMs) for classification (with an extension to regression). They represent input data as a multivector and use a single Clifford kernel for multi-class problems. This approach significantly reduces the computational complexity involved in training the SVM. Tests using real-world applications of signal processing and computer vision show the merit of their approach. Luukka and Lampinen propose a classification method which combines principal component analysis to pre-process the data followed by optimization of the classifier parameters using a differential evolution algorithm. Specifically, they optimize the class vectors used by the classifier and the power of the distance metric. Test results using real-world data sets show the proposed approach performs equally or better to some of the best existing classifiers. Lastly in this category, Zhang et al. study the problem of feature selection in high-dimensional problems. They focus on the GA-SVM approach, where a genetic algorithm (GA) optimizes the parameters of the SVM (the GA uses the SVM output as the objective values). The problem requires large computational resources which make it difficult to apply to large or high-dimensional sets. As such they propose several measures such as parallelization, neighbour search and caching to accelerate the search. Test results show their approach can reduce the computational cost of training an SVM classifier. Two studies focus on difficult scheduling problems. First, Pieters studies the problem of railway timetable design scheduling, which is an NP-hard problem with additional challenging features as such being reactive and dynamic. He studies solving the problem with Symbiotic Networks, a class of neural networks inspired by the symbiosis phenomenon is nature, and so the network uses ‘agents’ to adapt itself to the problem. Test results show the Symbiotic network can successfully handle the complex scheduling problem. Next, Srivastava et al. propose an approach combining evolutionary algorithms, neural network and fuzzy logic to solve problems of multiobjective time-cost trade-off. They consider a range of such problems including nonlinear time-cost relationships, constrained resources and project uncertainties. They show the merit of their approach by testing it on a real-world test case.

X

Preface

Lastly, for applications of CI to hardware implementations, Meher studies the use of systolic arrays for implementing artificial neural networks in VLSI and FPGA platforms. The chapter studies the use of systolic arrays for efficient hardware implementations of neural networks for real-time applications. The chapter surveys various approaches, current achievements as well as future directions such as mixed analog-digital neural networks. This is followed by Thangavelautham et al. who propose using coarse-coding techniques to evolve multi-robot controllers and where they aim to evolve simultaneously both the controller and sensor configurations. To make the problem tractable they use an Artificial Neural Tissue to exploit regularity in the sensor data. Test results show their approach outperforms a reference one. Overall, the chapters in this volume address a spectrum of issues arising in the application of computational intelligence to real-world difficult optimization problems. The chapters discuss both the current accomplishments and the remaining open issues as well as point to future research directions in the field. September 2009

Yoel TENNE Chi-Keong GOH

Acknowledgement to Reviewers

We express our thanks to the expertise provided by our fellow researchers who have kindly reviewed for the edited book. Their assistance have been invaluable to our endeavors. B.V. Babu Will Browne Pedro M. S. Carvalho Jia Chen Sheng Chen Tsung-Che Chiang Siang-Yew Chong Antonio Della Ciopa Carlos A. Coello Coello Marco Cococcioni Claudio De Stefano Antonio Gaspar-Cunha Kyriakos C. Giannakoglou David Ginsbourger Frederico Guimar˜ aes Martin Holena Amitay Issacs Jayadeva Wee Tat Koo Slawomier Koziel Jouni Lampinen Xiaodong Li

Dudy Lim Passi Luukka Pramod Kumar Meher Hirotaka Nakayama Ferrante Neri Thai Dung Nguyen Alberto Ochoa Yew-Soon Ong Khaled Rasheed Tapabrata Ray Abdellah Salhi Vui Ann Shim Ofer M. Shir Dimitri Solomatine Sanjay Srivastava Janusz Starzyk Stephan Stilkerich Haldun S¨ ural Mohammhed B. Trabia Massimiliano Vasile Lingfeng Wang Chee How Wong

Contents

1

2

New Hybrid Intelligent Systems to Solve Linear and Quadratic Optimization Problems and Increase Guaranteed Optimal Convergence Speed of Recurrent ANN . . . . . . . . . . . . . . . . . . . . . . . . . Otoni N´obrega Neto, Ronaldo R.B. de Aquino, Milde M.S. Lira 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Neural Network of Maa and Shanblatt: Two-Phase Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Hybrid Intelligent System Description . . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Method of Tendency Based on the Dynamics in Space-Time (TDST) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.2 Method of Tendency Based on the Dynamics in State-Space (TDSS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4.1 Case 1: Mathematical Linear Programming Problem – Four Variables . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4.2 Case 2: Mathematical Linear Programming Problem – Eleven Variables . . . . . . . . . . . . . . . . . . . . . . . . 1.4.3 Case 3: Mathematical Quadratic Programming Problem – Three Variables . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A Novel Optimization Algorithm Based on Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Janusz A. Starzyk, Yinyin Liu, Sebastian Batog 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Optimization Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Basic Search Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1 4 7 9 13 16 16 16 18 19 23 25 27 27 30 30

XIV

Contents

2.2.2

Extracting Historical Information by Weighted Optimized Approximation . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.3 Predicting New Step Sizes . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.4 Stopping Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.5 Optimization Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Simulation and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Finding Global Minimum of a Multi-variable Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.2 Optimization of Weights in Multi-layer Perceptron Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.3 Micro-saccade Optimization in Active Vision for Machine Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

4

The Use of Opposition for Decreasing Function Evaluations in Population-Based Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mario Ventresca, Shahryar Rahnamayan, Hamid Reza Tizhoosh 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Theoretical Motivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Definitions and Notations . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Consequences of Opposition . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 Lowering Function Evaluations . . . . . . . . . . . . . . . . . . . . . 3.2.4 Comparison to Existing Methods . . . . . . . . . . . . . . . . . . . 3.3 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Differential Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Opposition-Based Differential Evolution . . . . . . . . . . . . . 3.3.3 Population-Based Incremental Learning . . . . . . . . . . . . . . 3.3.4 Oppositional Population-Based Incremental Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 Evolutionary Image Thresholding . . . . . . . . . . . . . . . . . . . 3.4.2 Parameter Settings and Solution Representation . . . . . . . 3.5 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 ODE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 OPBIL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Search Procedure Exploiting Locally Regularized Objective Approximation: A Convergence Theorem for Direct Search Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marek Bazan 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 The Search Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Zangwill’s Method to Prove Convergence . . . . . . . . . . . . . . . . . . . .

30 36 37 38 39 39 42 43 45 46 49 49 50 51 52 53 54 55 56 57 57 58 59 59 63 64 64 65 68 69

73 73 74 75

Contents

XV

4.4

The Main Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 4.4.1 Closedness of the Algorithmic Transformation . . . . . . . . 78 4.4.2 A Perturbation in the Line Search . . . . . . . . . . . . . . . . . . . 80 4.5 The Radial Basis Appproximation . . . . . . . . . . . . . . . . . . . . . . . . . . 87 4.5.1 Detecting Dense Regions . . . . . . . . . . . . . . . . . . . . . . . . . . 87 4.5.2 Regularization Training . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 4.5.3 Choice of the Regularization Parameter λ Value . . . . . . . 90 4.5.4 Error Bounds for Radial Basis Approximation . . . . . . . . 91 4.6 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 4.6.1 Test Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 4.6.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 4.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 5

6

Optimization Problems with Cardinality Constraints . . . . . . . . . . . . Rub´en Ruiz-Torrubiano, Sergio Garc´ıa-Moratilla, Alberto Su´arez 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Approximate Methods for the Solution of Optimization Problems with Cardinality Constrains . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 Simulated Annealing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.2 Genetic Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.3 Estimation of Distribution Algorithms . . . . . . . . . . . . . . . 5.3 Benchmark Optimization Problems with Cardinality Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.1 The Knapsack Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.2 Ensemble Pruning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.3 Portfolio Optimization with Cardinality Constraints . . . . 5.3.4 Index Tracking by Partial Replication . . . . . . . . . . . . . . . . 5.3.5 Sparse Principal Component Analysis . . . . . . . . . . . . . . . 5.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Learning Global Optimization through a Support Vector Machine Based Adaptive Multistart Strategy . . . . . . . . . . . . . . . . . . . Jayadeva, Sameena Shah, Suresh Chandra 6.1 Introduction and Background Research . . . . . . . . . . . . . . . . . . . . . . 6.2 Global Optimization with Support Vector Regression Based Adaptive Multistart (GOSAM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.1 One Dimensional Wave Function . . . . . . . . . . . . . . . . . . . 6.3.2 Two Dimensional Case: Ackley’s Function . . . . . . . . . . . 6.3.3 Comparison with PSO and GA on Higher Dimensional Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Extension to Constrained Optimization Problems . . . . . . . . . . . . . . 6.4.1 Sequential Unconstrained Minimization Techniques . . . 6.5 Design Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

105 105 108 108 110 111 113 114 116 119 122 124 127 128 131 132 134 136 137 140 141 143 143 147

XVI

7

8

Contents

6.5.1 Sample and Hold Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5.2 Folded Cascode Amplifier . . . . . . . . . . . . . . . . . . . . . . . . . 6.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.7 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

148 149 149 152 153

Multi-objective Optimization Using Surrogates . . . . . . . . . . . . . . . . . Ivan Voutchkov, Andy Keane 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Surrogate Models for Optimization . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Multi-objective Optimization Using Surrogates . . . . . . . . . . . . . . . 7.4 Pareto Fronts - Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 Response Surface Methods, Optimization Procedure and Test Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.6 Update Strategies and Related Parameters . . . . . . . . . . . . . . . . . . . . 7.7 Test Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.8 Pareto Front Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.8.1 Generational Distance ([3], pp.326) . . . . . . . . . . . . . . . . . 7.8.2 Spacing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.8.3 Spread . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.8.4 Maximum Spread . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.9 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.9.1 Understanding the Results . . . . . . . . . . . . . . . . . . . . . . . . . 7.9.2 Preliminary Calculations . . . . . . . . . . . . . . . . . . . . . . . . . . 7.9.3 The Effect of the Update Strategy Selection . . . . . . . . . . . 7.9.4 The Effect of the Initial Design of Experiments . . . . . . . . 7.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

155

A Review of Agent-Based Co-Evolutionary Algorithms for Multi-Objective Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rafał Dre˙zewski, Leszek Siwik 8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Model of Co-Evolutionary Multi-Agent System . . . . . . . . . . . . . . . 8.2.1 Co-Evolutionary Multi-Agent System . . . . . . . . . . . . . . . 8.2.2 Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.3 Species . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.4 Sex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.5 Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3 Co-Evolutionary Multi-Agent Systems for Multi-Objective Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.1 Co-Evolutionary Multi-Agent System with Co-Operation Mechanism (CCoEMAS) . . . . . . . . . . . . . . 8.3.2 Co-Evolutionary Multi-Agent System with Predator-Prey Interactions (PPCoEMAS) . . . . . . . . . . . . . 8.4 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

155 157 158 159 161 163 164 165 165 165 165 166 166 166 167 167 171 174 174 177 177 179 180 180 181 182 183 187 187 190 196

Contents

XVII

8.4.1

Test Suite, Performance Metric and State-of-the-Art Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.2 A Glance at Assessing Co-operation Based Approach (CCoEMAS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.3 A Glance at Assessing Predator-Prey Based Approach (PPCoEMAS) . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5 Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

A Game Theory-Based Multi-Agent System for Expensive Optimisation Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ¨ Abdellah Salhi, Ozgun T¨oreyen 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.1 Optimisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.2 Game Theory: The Iterated Priosoners’ Dilemma . . . . . . 9.2.3 Multi-Agent Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3 Constructing GTMAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.1 GTMAS at Work: Illustration . . . . . . . . . . . . . . . . . . . . . . 9.4 The GTMAS Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.1 Solver-Agents Decision Making Procedure . . . . . . . . . . . 9.5 Application of GTMAS to TSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6 Tests and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.7 Conclusion and Further Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

10 Optimization with Clifford Support Vector Machines and Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . N. Arana-Daniel, C. L´opez-Franco, E. Bayro-Corrochano 10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2 Geometric Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2.1 The Geometric Algebra of n-D Space . . . . . . . . . . . . . . . . 10.2.2 The Geometric Algebra of 3-D Space . . . . . . . . . . . . . . . 10.3 Linear Clifford Support Vector Machines for Classification . . . . . 10.4 Non Linear Clifford Support Vector Machines for Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5 Clifford SVM for Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6 Recurrent Clifford SVM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7.1 3D Spiral: Nonlinear Classification Problem . . . . . . . . . . 10.7.2 Object Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7.3 Multi-case Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7.4 Experiments Using Recurrent CSVM . . . . . . . . . . . . . . . . 10.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

196 197 200 207 207 211 211 213 213 213 214 215 216 218 219 221 228 229 230 233 233 234 235 237 237 242 243 245 247 247 250 256 257 260 260

XVIII

Contents

11 A Classification Method Based on Principal Component Analysis and Differential Evolution Algorithm Applied for Prediction Diagnosis from Clinical EMR Heart Data Sets . . . . . . . . . . . . . . . . . . Pasi Luukka, Jouni Lampinen 11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Heart Data Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Classification Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.1 Dimension Reduction Using Principal Component Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.2 Classification Based on Differential Evolution . . . . . . . . 11.3.3 Differential Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4 Classification Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5 Discussion and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 An Integrated Approach to Speed Up GA-SVM Feature Selection Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tianyou Zhang, Xiuju Fu, Rick Siow Mong Goh, Chee Keong Kwoh, Gary Kee Khoon Lee 12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.1 Parallel/Distributed GA . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.2 Parallel SVM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.3 Neighbor Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.4 Evaluation Caching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3 Experiments and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Computation in Complex Environments; Optimizing Railway Timetable Problems with Symbiotic Networks . . . . . . . . . . . . . . . . . . Kees Pieters 13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.1 Convergence Inducing Process . . . . . . . . . . . . . . . . . . . . . 13.1.2 A Classification of Problem Domains . . . . . . . . . . . . . . . . 13.2 Railway Timetable Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3 Symbiotic Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.1 A Theory of Symbiosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.2 Premature Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4 Symbiotic Networks as Optimizers . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5 Trains as Symbiots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5.1 Trains in Symbiosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5.2 The Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5.3 The Trains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5.4 The Optimizing Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5.5 Computational Complexity . . . . . . . . . . . . . . . . . . . . . . . .

263 264 267 268 268 269 271 272 280 281 285

285 288 288 290 291 292 292 297 298 299 299 300 301 302 304 306 311 313 314 315 315 316 318 319

Contents

XIX

13.5.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5.7 A Symbiotic Network as a CCGA . . . . . . . . . . . . . . . . . . . 13.5.8 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

319 321 322 322

14 Project Scheduling: Time-Cost Tradeoff Problems . . . . . . . . . . . . . . . Sanjay Srivastava, Bhupendra Pathak, Kamal Srivastava 14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.1.1 A Mathematical Description of TCT Problems . . . . . . . . 14.2 Resource-Constrained Nonlinear TCT . . . . . . . . . . . . . . . . . . . . . . . 14.2.1 Artificial Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . 14.2.2 Working of ANN and Heuristic Embedded Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.2.3 ANNHEGA for a Case Study . . . . . . . . . . . . . . . . . . . . . . 14.3 Sensitivity Analysis of TCT Profiles . . . . . . . . . . . . . . . . . . . . . . . . 14.3.1 Working of IFAG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.3.2 IFAG for a Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.4 Hybrid Meta Heuristic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.4.1 Working of Hybrid Meta Heuristic . . . . . . . . . . . . . . . . . . 14.4.2 HMH Approach for Case Studies . . . . . . . . . . . . . . . . . . . 14.4.3 Standard Test Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

325

15 Systolic VLSI and FPGA Realization of Artificial Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pramod Kumar Meher 15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.2 Direct-Design of VLSI for Artificial Neural Network . . . . . . . . . . 15.3 Design Considerations and Systolic Building Blocks for ANN . . . 15.4 Systolic Architectures for ANN . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.4.1 Systolic Architecture for Hopfield Net . . . . . . . . . . . . . . . 15.4.2 Systolic Architecture for Multilayer Neural Network . . . 15.4.3 Systolic Implementation of Back-Propagation Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.4.4 Implementation of Advance Algorithms and Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

325 328 329 330 331 334 336 339 339 343 345 348 352 354 355 359 360 362 364 371 371 373 373 376 376 377

16 Application of Coarse-Coding Techniques for Evolvable Multirobot Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 Jekanthan Thangavelautham, Paul Grouchy, Gabriele M.T. D’Eleuterio 16.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 16.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384

XX

Contents

16.2.1 The Body and the Brain . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.2.2 Task Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.2.3 Machine-Learning Techniques and Modularization . . . . 16.2.4 Fixed versus Variable Topologies . . . . . . . . . . . . . . . . . . . 16.2.5 Regularity in the Environment . . . . . . . . . . . . . . . . . . . . . . 16.3 Artificial Neural Tissue Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.3.1 Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.3.2 The Decision Neuron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.3.3 Evolution and Development . . . . . . . . . . . . . . . . . . . . . . . . 16.3.4 Sensory Coarse Coding Model . . . . . . . . . . . . . . . . . . . . . 16.4 An Example Task: Resource Gathering . . . . . . . . . . . . . . . . . . . . . . 16.4.1 Coupled Motor Primitives . . . . . . . . . . . . . . . . . . . . . . . . . 16.4.2 Evolutionary Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.5.1 Evolution and Robot Density . . . . . . . . . . . . . . . . . . . . . . . 16.5.2 Behavioral Adaptations . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.5.3 Evolved Controller Scalability . . . . . . . . . . . . . . . . . . . . . . 16.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

385 385 386 387 388 389 389 390 391 393 395 397 399 399 403 403 406 407 409 410

Chapter 1

New Hybrid Intelligent Systems to Solve Linear and Quadratic Optimization Problems and Increase Guaranteed Optimal Convergence Speed of Recurrent ANN Otoni N´obrega Neto, Ronaldo R.B. de Aquino, and Milde M.S. Lira

Abstract. This chapter deals with the study of artificial neural networks (ANNs) and Heuristic Rules (HR) to solve optimization problems. The study of ANN as optimization tools for solving large scale problems was due to the fact that this technique has great potential for hardware VLSI implementation, in which it may be more efficient than traditional optimization techniques. However, the implementation of computational algorithm has shown that the proposed technique should have been efficient but slow as compared with traditional mathematical methods. In order to make it a fast method, we will show two ways to increase the speed of convergence of the computational algorithm. For analyzes and comparison, we solved three test cases. This paper considers recurrent ANN to solve linear and quadratic programming problems. These networks are based on the solution of a set of differential equations that are obtained from a transformation of an augmented Lagrange energy function. The proposed hybrid systems combining recurrent ANN and HR presented a reduced computational effort in relation to the one using only the recurrent ANN.

1.1 Introduction The early 1980’s were marked by a resurgence of interest in artificial neural networks (ANNs). At that time, the development of ANNs had the important characteristic of temporal processing. Many researchers have attributed the resumption of researches on ANNs in the eighties to the Hopfield model presented in 1982 [1]. This recurrent Hopfield model has so far constituted, a great progress in the threshold of knowledge in the area of neural networks, until then. Nowadays, it is known that there are two ways of incorporating the temporal computation in a neural network: the first one is possible by using a statistical neural nets to accomplish a dynamical mapping in a structure of short-term memory; and the Otoni N´obrega Neto · Ronaldo R.B. de Aquino · Milde M.S. Lira Electrical Engineering Department, Federal University of Pernambuco, Brazil e-mail: [email protected],[email protected]

Y. Tenne and C.-K. Goh (Eds.): Computational Intelligence in Optimization, ALO 7, pp. 1–26. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

2

O.N. Neto, R.R.B. de Aquino, and M.M.S. Lira

second one is by making internal feedback connections that may be made by single or multi-loop feedback in which the neural network can be fully connected. Artificial neural networks that have feedback connections in their topology are known as recurrent neural networks [2]. The theoretical study and applications of recurrent neural nets were developed in several subsequent works [3, 4, 5, 6, 7, 8, 9]. Actually, the progress provided by Hopfield’s works have shown that a value of energy could be associated to each state of the net and that this energy decreases monotonically as the path is described within the state-space towards a fixed point. These fixed points are therefore stable points of energy [10], i.e., the described energy function behaves as Lyapunov functions for the model described in detail in Hopfield’s works. At this exact moment, it is observed subjects of stabilities in recurrent neural nets. Considering the stability in a non-linear dynamical system, we usually think about stability in the sense of Lyapunov. The Direct Method of Lyapunov is broadly used for stability analysis of linear and non-linear systems which may be either time-variant or time-invariant. Therefore, it can be directly applicable to the stability analysis of ANNs [2]. In 1985, Hopfield solved the traveling salesman problem [7] that is a problem in combinatorial optimization using a continuous model of the recurrent neural network as an optimization tool. In 1986, Hopfield proposed a specialized ANN to solve specific problems of linear programming (LP) [9] based on analog circuits, studied since 1956 by Insley B. Pyne and presented in [11]. On that occasion, Hopfield demonstrated that the dynamics involved in recurrent artificial neural nets were described by a Lyapunov function and that for this reason, it was demonstrated that this network is stable and also that the point of stability is the solution to the problem for which the ANN was modeled. In 1987, Kennedy and Chua demonstrated that the ANN which was proposed by Hopfield in 1986, in spite of searching for the minimum level of the energy function, it had not been modeled to offer an inferior limit, but only when the saturation of an operational amplifier of the circuit was reached [12]. Due to this deficiency, Kennedy and Chua proposed a new circuit for LP problems that also proved to be able to solve quadratic programming (QP) problems. These circuits were nominated as “canonical non-linear programming circuit”, which are based on the Kuhn-Tucker (KT) conditions [12]. In this kind of ANN-based optimization, the problem has to be “hard-wired” in the network and the convergence behavior of the ANN depends greatly on how the cost function is modeled. Later on, hard studies [13, 14] confirmed that for non-linear programming problems, the proposed model by Kennedy and Chua [15] has completely satisfied the optimization of KT conditions and the penalty method. Besides, under appropriate conditions this net is stable. In spite of the important progresses presented in Kennedy and Chua’s studies, a deficiency was observed in the model, which appears when the equilibrium point of the net happens in the neighborhood of the optimal point of the original problem, but the distance between the optimal point and the equilibrium point of the network can be reduced by increasing the penalty parameter (s), as in [14] and [16]. Even so, Kennedy and Chua’s network is able to solve a great class of optimization problems with and without constraints. However, when

1

New HIS to Solve LP and QP Optimization and Increase Convergence Speed

3

the solutions of constrains optimization problems are in the neighborhood of the feasible region, i.e., equality constraints are close to happen, then the network only converges on an approximate solution that can be out of the feasible region [17]. This is explained by the application of the penalty function theorem [16]. For applications in which an unfeasible solution cannot be tolerated, the usefulness of this technique (Kennedy and Chua’s neural networks) is seriously jeopardized. With the intention of overcoming this difficulty, Maa and Shanblatt proposed the two-phase method [14]. This work reveals an innovation in the method presented by David W. Tank and John J. Hopfield [18] and it guarantees that, in certain conditions, the proposed network evolves towards the exact solution of the optimization problem. Since Kennedy and Chua network contains a finite penalty parameter, it generates only approximate solutions and presents an implementation problem when the penalty parameter is very large. To reach an exact solution Maa and Shanblatt method uses another penalty parameter in the second phase. Therefore, to avoid using penalty parameters, some significant works have been done in recent years. Among them, a few primal-dual neural networks with two-layer and onelayer structure were developed for solving linear and quadratic programming problems [18, 19, 20, 21]. These neural networks were proved to be globally convergent to an exact solution when the objective function is convex. Nowadays, recurrent ANN have been used to solve real world problems such as, the hydrothermal Scheduling [22] based on the augmented Lagrange Hopfield network and [23, 24, 25] based on Maa and Shanblatt Two-phase Neural Network. In this work, an ANNs was approached to solve optimization problems and the proposed method by Maa and Shanblatt was applied. The study of ANN as optimization tools for solving large scale problems was due to the fact that this technique has a great potential for hardware VLSI implementation, in which it can be more efficient than traditional optimization techniques. However, the implementation of the method in software has shown that, in spite of the technique being efficient in the solution of optimization problems, the speed of convergence could become slow when compared with traditional mathematical methods. In this regard, it was created and proposed heuristic rules in a hybrid form to aid and accelerate the convergence of the two-phase method in the software. It is important to point out that the software implementation of the method is an important part of the development and analysis of the method in hardware. An important characteristic for choosing Maa and Shanblatt network is that it is ready to solve linear and quadratic optimization problems with equality and inequality linear constraints without using mathematical transformations, which would increase the dimension of the problem. As we plan to apply the developed HIS in this work to solve the hydrothermal scheduling problem [24, 25], which does not need an exact solution, Maa and Shanblatt network first phase was chosen for this implementation. In future works, we may try other kinds of recurrent ANNs. Decision trees and classification rules are important and common methods used for knowledge representation in the expert systems [26]. Heuristic rules are rules which have no particular foundation in a scientific theory, but which are only based on the observation of general patterns and derived from facts. These rules are

4

O.N. Neto, R.R.B. de Aquino, and M.M.S. Lira

applicable to many problems as shown in [27, 28, 29, 30]. Here the basis of the proposed heuristic rules is the dynamical behavior of neural networks. From the convergence analysis, we identified the parameters and their relationships, which are then transformed into a set of heuristic rules. We developed an algorithm based on the heuristic rules and carried out some experiments to evaluate and support the proposed technique. In this work, two possible implementations were developed, tested and compared; and a high reduction in computational effort was observed by using the proposed heuristic rules. This reduction is related to the decrease in the number of ODE computed during the convergence process. Other possible implementations are also indicated. This work is organized beginning with a revision of the two-phase method of Maa and Shanblatt; following, we present the proposed heuristic rules and show the solutions for test cases using the previously discussed techniques; next, the simulation results are presented and analyzed; and finally, we draw conclusions about the proposed work.

1.2 Neural Network of Maa and Shanblatt: Two-Phase Optimization The operation of the Hopfield network model and the subsequent models is based on a constraint violation of optimization problem. When a constraint violation occurs, the magnitude and the direction of the violation are fed back to adjust the states of the neurons of the network so that the overall energy function of the network always decreases until it reaches a minimum level. These ANN models have dynamical characteristics according to the Lyapunov function. Therefore, it can be demonstrated that these networks are stable and that the equilibrium point is the solution of LP and QP problems that the network represents. This type of neural network was firstly improved in [15] and later in [14]. The network used in the last version is the one used in this work. Maa and Shanblatt network is able to solve constrained or unconstrained convex quadratic and linear programming problems. Consider the following convex P problem: 1 T x Qx + cT x 2 s.t. g(x) = Dx − b ≤ 0

(P) min f (x) =

h(x) = Hx − w = 0 x ∈ Rn

(1.1)

where c ∈ Rn , D ∈ R pxn , b ∈ R p , H ∈ Rqxn , w ∈ Rq , p and q ≤ n and Q ∈ Rnxn is symmetric and positive definite or positive semidefinite, f , gi ’s and h j ’s are functions on Rn → R. Assume that the feasible domain of P is not empty and the objective function is bounded below over the domain of P.

1

New HIS to Solve LP and QP Optimization and Increase Convergence Speed

5

Particularly, P is said to be convex programming if f and gi ’s are convex functions, and h j ’s are affine functions. Another particularity of the formulation can be observed when Q is a zero matrix and the cost function is thus reduced to f (x) = cT x. In this case, if the inequality and equality constraints have linear formulation then the problem P becomes a linear programming problem. The method presented by Maa and Shanblatt [14] is composed in to two phases. The first phase of the method aims to initialize the problem and to converge quickly without high accuracy towards the neighborhood of the optimal point, while the second phase aims to reach the exact solution of the problem. To this end, the dynamic of the first phase is based on the exact penalty of Lagrangian function or energy function L(s, x):   q p   s 2 2 (1.2) L(s, x) = f (x) + ∑ g+i (x) + ∑ (h j (x)) 2 i=1 j=1 where s is a large positive real number, and the function g+ i (x(t)) = max{0, gi (x(t))}, + T whose notation was simplified to g+ = [g+ 1 , . . . , gm ] , according to [14]. As long as the system converges x(t) → xˆ , sg+ i (x(t)) → λi and sh j (x(t)) → μ j which are the Lagrange multipliers associated with each corresponding constraint. In the first phase, an approximation of the Lagrange multipliers is already obtained. The block diagram of a two-phase optimization network is shown in Fig. 1.1. The dynamics that happen in the first phase are in the time range 0 ≤ t ≤ t1 (t1 is the time instant when the switch is closed connecting the first phase to the second one). The network operates according to the following dynamics:   p  q  dx + = −∇ f (x) − s ∑ ∇gi (x)gi (x) + ∑ (∇h j (x)h j (x)) (1.3) dt i=1 j=1 In the second phase (t ≥ t1 ) the network begins to shift the directional vector sg+ i (x) gradually to λi , and sh j (v) to μ j . By imposing a small positive real value ε , the update rate of d λi /dt and d μi /dt that are represented in (1.6) and (1.7), respectively, is comparatively much slower than that of dx/dt (1.5). Approximation of such dynamics is possible by considering λ and μ to be fixed. Then it can be seen that (1.6) is seeking a minimum point of the augmented Lagrangian function La (s, x): La (s, x) = f (x) + λ T g(x) + μ T h(x) +

 s +  g (x) 2 +  h(x) 2 2

(1.4)

In the block diagram of Fig. 1.1, in the first phase, the subsystems within the two large rectangles do not contribute during t ≤ t1 and in the second phase, when t > t1 , the dynamics of the network become:   p  q   + dx = −∇ f (x) − ∑ ∇gi (x) sgi (x) + λi + ∑ (∇h j (x) (sh j (x) + μ j )) dt i=1 j=1 (1.5)

6

O.N. Neto, R.R.B. de Aquino, and M.M.S. Lira

Fig. 1.1 Block diagram of the dynamical system to Maa and Shanblatt network

The Lagrange multipliers are updated as: d λi (t + Δ t) = ε sg+ i (gi (x(t))), dt

to

i = 1, . . . p,

and

(1.6)

d μ j (t + Δ t) = ε sh j (x(t)), to j = 1, . . . q. (1.7) dt A practical value is ε = 1/s according to [14], what leaves the network with just one adjustment parameter. However, using ε independently of s gives more freedom to control the dynamics of the network. During the first phase, the Lagrange multipliers are null, thus there is not restriction on the initial value of x(t). According to the theorem of penalty function, the solution achieved in the first phase is not equivalent to the minimum of the function f (x), unless the penalty parameter s is infinite. In this way, the use of the second phase of optimization is necessary to any finite value of s. The system reaches equilibrium when:

1

New HIS to Solve LP and QP Optimization and Increase Convergence Speed

7

g+ i =0 h j = 0 and p

q

i=1

j=1

∇ f (x) + ∑ ∇gi (x)λi (x) + ∑ ∇h j (x)μ j (x) = 0,

(1.8)

that is identical to optimality condition of the KT theorem and thus the equilibrium point of the two-phase network is precisely a global minimum point to a convex problem (P). In [12] it is demonstrated that the Kennedy and Chua network for linear and quadratic programming problems satisfies completely the optimality conditions of KT and the function penalty method. It shows also that under appropriate conditions this network is completely stable. Moreover, it is shown that the equilibrium point happens in the neighborhood of the optimal point of the original problem and that the distance among them can be made arbitrarily small, selecting sufficiently a large value of penalty parameter (s). For problems that cannot tolerate a solution in the infeasible region, due to physical limits of operational amplifiers, a two-phase optimization network model is proposed. In the second phase, we can obtain both, the exact solution for these problems and the corresponding Lagrange multipliers associated with each restriction.

1.3 Hybrid Intelligent System Description The proposed network by Maa and Shanblatt has two attractive features which are the property of guaranteed global convergence of the mathematical programming problem and the possibility of physical implementation of the neural network in a circuit with electrical components where the response time of the dynamic of the circuit would be imposed by the capacitance in the circuit, thus the convergence time would be negligible. In spite of these attractive characteristics, the time required for processing the computational algorithm becomes a barrier in the ANN-based applications for solving large-scale mathematical programming problems, then it is necessary the resolution of several differential equations. In this regard, problems with larger number of variables and constraints will involve a fair amount of differential equations to be solved. In order to mitigate this problem, heuristic rules were developed to accelerate the convergence of the computational algorithm involved in recurrent neural networks. The combination of recurrent ANN with heuristic rules forms the Hybrid Intelligent System (HIS), in which these two techniques interact and exchange information with one another while the optimal solution of the problem is not achieved. The basis of the proposed heuristic rules is the dynamical behavior of the neural networks. The control theory of system studies deeply the dynamical behavior of a process. With the aid of control theory and gathering information of the Lyapunov theorem for the network [2], it can be stated that from any given initial state of the state vectors x(0), the network will always change the values of the state variables xi (t) in the direction in which the value of the Lyapunov function for the network

8

O.N. Neto, R.R.B. de Aquino, and M.M.S. Lira

decreases continuously until it reaches a stationary point, which corresponds to a global minimum of the programming problem. The trajectory of the variables is illustrated in Fig. 1.2. It depicts a two-dimensional trajectory (orbit) of a dynamical system, where it is possible to observe the state variables of the system at certain time instants (t0 ,t1 , . . . ,t5 ). The dotted vector can be understood as the tendency of the convergence (indicated by the gradient vector) of the variables in the dynamics of the system.

Fig. 1.2 A two-dimensional trajectory (orbit) of a dynamical system

Trajectories of the state variables for the same system are exemplified graphically in Fig. 1.3. These trajectories are distinct due to the fact that the state variables have different initial states. The dynamic of recurrent ANN has the same properties and, therefore, it is similar to the dynamics showed in Fig. 1.3. In spite of Maa and Shanblatt model which deals with continuous-time recurrent network, in a computational algorithm the calculations of the iterations are performed in a discrete-time form, since the calculations of the integral equations demand a small step size for calculations, but not null. Therefore, we take total control at the course of the iteration of the algorithm in the network. Detailed observations were carried out during test of the algorithm of Maa and Shanblatt model showing that the computational convergence is slow and the trajectories in the state-space of convergence of recurrent networks are smooth and possibly predictable. Then, we observed that, in certain conditions, it is not only possible to estimate a point closer to the minimum point of the function energy of the network, but it is also possible to estimate a point that instead of the initial orbit of convergence turns to an initial point of a new orbit of convergence. This new orbit would have a shorter curvature and, consequently, a smaller Euclidean distance to the optimal point. In this regard, the number of steps to calculate the convergence of the computational algorithm can be reduced and, consequently, the time to compute the equilibrium point of the network.

1

New HIS to Solve LP and QP Optimization and Increase Convergence Speed

9

Fig. 1.3 An illustration of a two-dimensional state (phase) of a dynamical system and the associated vectorial field

To achieve the equilibrium point, we use two methods. In the first one, the point is calculated starting from the evolution of the dynamics in space-time plan (in this work, we considered only autonomous systems). In the second method, the calculation is performed by observing the evolution of the variable in state-space. The mechanism of these two methods and the way they operate in the proposed HIS are described as follows.

1.3.1 Method of Tendency Based on the Dynamics in Space-Time (TDST) Consider the convergences of dynamical systems of first order according to the graphs in Fig. 1.4. Observing the curves of Fig. 1.4 and pointing out that the time is a variable that is always increasing, i.e., the following point x1 (t) is always in front of x1 (t − Δ t), we can reach the conclusion that a closer point to the convergence would be located outside the internal area of the convergence curve concavity, in case the convergence curve behaves as shown in graph Fig. 1.4(a). For example, for t = 0.060, x1 (0.060) ≈ 0.3; in this case, a better estimation to the point would be x1 (0.061) = 0.45 for Δ t = 0.001s. Restarting the network with this initial state would generate a convergence curve as illustrated in graph Fig. 1.5. However, this rule would be applied only for curves of type (a) and (c) as shown in Fig. 1.4, since curves (b) and (d), a better estimation to the point is located inside the internal area of the concavity. Observing the particularities of the possible curvatures of the convergence curve in the space-time plan, the following parameters were created in relation to: • curvature {curve, straight line}; • concavity, when it exist, {concave downwards, concave upwards};

O.N. Neto, R.R.B. de Aquino, and M.M.S. Lira

1

1

0.9

0.9

0.8

0.8

0.7

0.7

0.6

0.6

x1(t)

x1(t)

10

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0

1

0

0.1

0.2

0.3

0.4

0.5

1

1

0.9

0.9

0.8

0.8

0.7

0.7

0.6

0.6

x1(t)

x1(t)

0.7

0.8

0.9

1

0.6

0.7

0.8

0.9

1

(b)

(a)

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1 0

0.6

t

t

0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0

0.1

0.2

0.3

0.4

0.5

t

t

(c)

(d)

Fig. 1.4 Dynamical convergences of first order (single variable systems): graphs of evolution of state variables in time

• time rating {high, mean, low}; • variable xi (t) {increasing, decreasing}. In order to assess these parameters, the network must provide at least three points (P0 , P1 , P2 ) in the convergence curve. Next, these three points are normalized in the horizontal axis and also in the vertical axis, in order to avoid problems in the algorithm used to estimate a better point. The chosen normalization equation is presented in (1.9), where M is the maximum and m is the minimum of the three values to be normalized, a and b are chosen according to the range of the normalized values. In this work, the values are normalized into value in the range [0, 1], then a = 0 and b = 1. z is the value to be normalized, and zN is the normalized value. zN =

b(z − m) − a(z − M) M−m

(1.9)

1

New HIS to Solve LP and QP Optimization and Increase Convergence Speed

11

1

0.8

x1(t)

Original Dynamic 0.6

Advanced Dynamic 0.4

Predict Point by Heuristic Rule (HIS−1)

0.2

The Algorithm Time Gain Time in which the ANN is Restarted

0

0

0.2

0.4

t

0.6

0.8

1

Fig. 1.5 Action of the HIS to calculate a better point in dynamics evolving over time

→ → From the normalized points, two spatial vectors − v 1 and − v 2 are computed, and the following relevant information are obtained from them: Euclidian norm, angle (θi ) of each vector in relation to the horizontal axis; and finally, angle between them (Δ θ = θ2 − θ1 ). Therefore, when there is a space curvature over the normalized points, classification regions (decision regions) are generated as shown in Fig. 1.6. To understand the illustration in Fig. 1.6 better, consider that the normalized initial point (P0N ) is always found in the beginning of each area (S4, S5, S6, S7, S8 and S9). Besides these six possibilities, there are more three others that occur in case of straightforward convergences where Δ θ is approximately zero.

Fig. 1.6 Classification regions to patterns that have spatial curvature

Region 4 (S4) is similar to the beginning of the convergence shown in Fig. 1.4(a). While region 5 (S5) describes a behavior close to the curve formed by the end of the convergence shown in Fig. 1.4(a) and the beginning of the convergence shown in Fig. 1.4(b). Region 6 (S6) has a convergence similar to that shown in Fig. 1.4(b).

12

O.N. Neto, R.R.B. de Aquino, and M.M.S. Lira

Region 7 (S7) describes the dynamic of the type shown in the Fig. 1.4(d) and region 8 (S8) describes a behavior close to the curve formed by the end of the convergence shown in the Fig. 4(c) and the beginning of the convergence shown in Fig. 1.4(d). While region 9 (S9) represents a behavior close to that shown in Fig. 1.4(c). The straightforward regime can be of three types: increasing - the derivative of the curve is positive and not close to zero, corresponding to region 1 (S1); constant - the derivative of the curve is approximately zero, corresponding to region 2 (S2); and decreasing - derivative is negative and not close to zero, corresponding to region 3 (S3). We can note that the regimes described by regions S5 and S8 can be considered close to the constant straightforward regime. Thus, we modeled the following heuristic rules: Rule 1: if <curvature is a straight line> and and than . Rule 2: if <curvature is a straight line> and than . Rule 3: if <curvature is a straight line> and and than . Rule 4: if <curvature is a curve> and and and than . Rule 5: if <curvature is a curve> and and than . Rule 6: if <curvature is a curve> and and and than . Rule 7: if <curvature is a curve> and and and than . Rule 8: if <curvature is a curve> and and than < Action II >. Rule 9: if <curvature is a curve> and and and than . The actions shown in the rules lead to sub-functions that return a better value for the next initialization point of the network. The straight line condition implicates that either the system is converging very slowly or the step size of the integration algorithm is very small. In this case, the linear function shown in (1.10) can be applied as shown below: P3N = a(P2N − P0N ) + P2N ,

(1.10)

−v (− → where a is a constant that yields a gain in magnitude of → 3 v 3 = P3 − P2 ). The rules above are summarized in Table 1.1.

1

New HIS to Solve LP and QP Optimization and Increase Convergence Speed

13

Table 1.1 Description of the actions to be taken due to the heuristic rules according to each decision region

Actions Description of the Actions I II III IV

V

VI

P3N is calculated according to (1.6) with a = a1 . P3N is calculated according to (1.6) with a = a2 . P3N is calculated according to (1.6) with a = a3 . P3N takes the coordinates of the superior point in the circumference that passes through the normalized points P0N , P1N and P2N . P3N takes the coordinates of the point fixed at the most right position in the circumference that passes through the normalized points P0N , P1N and P2N . P3N takes the coordinates of the inferior point in the circumference that passes through the normalized points P0N , P1N and P2N .

Regions S1 S2, S5, S8 S3 S4

S6, S7

S9

Having the normalized point P3N estimated by the heuristic rules, we need to unnormalize it to obtain the P3 value. This value will be used to start the recurrent network.

1.3.2 Method of Tendency Based on the Dynamics in State-Space (TDSS) To calculate a better point using the dynamics in state-space, two facts must be pointed out: firstly, the convergence of the variables depends on the convergence of other variables; secondly, when we are working in the state-space, the variations in the state of the variables are mapped by taking one variable xi as reference, so that the curve is free to evolve in all directions, which does not happen when we are working in the space-time plan. Perceiving previous facts and observing the convergence orbits in the state-space shown Fig. 1.2 and Fig. 1.3, we reached the conclusion that to calculate a better point in the state-space, it must be located inside the concavity of the orbit of the system dynamics. To calculate in such a state-space, we do the following: first we take one of the variables as a reference (for example, x1 (t)) and draw n − 1 complex plan (for a system with n variable), thus we have the plans x1 (t)0x2 (t), x1 (t)0x3 (t), . . ., x1 (t)0xn−1(t). Having the three points P2 = x(t), P1 = x(t − Δ t) and P0 = x(t − 2Δ t) provided by the network, we can create state vectors in each plan of all n − 1 complex plan. For example, for the plan x1 (t)0x2 (t), we have: − → v 1 (t) = (x1 (t) + i · x2(t)) − (x1 (t − Δ t) + i · x2(t − Δ t)) and

(1.11)

− → v 2 (t) = (x1 (t − Δ t) + i · x2(t − Δ t)) − (x1(t − 2Δ t) + i · x2(t − 2Δ t)).

(1.12)

14

O.N. Neto, R.R.B. de Aquino, and M.M.S. Lira

From these vectors, we carried out a rotation transformation in the axes using (1.13) and (1.14), according to the Fig. 1.7. − → → v 1’ = − v 1 exp(−θ1 )

(1.13)

→ − → v 2 exp(−θ1 ) v 2’ = −

(1.14)

→ v 1 (t) angle and also a translation transformation using: where θ1 is the − → v 1| x1 ’(t − 2Δ t) = −|− x2 ’(t − 2Δ t) = 0 x1 ’(t − Δ t) = 0 x2 ’(t − Δ t) = 0 x1 ’(t) = x1 (t) − x1 (t − 2Δ t) x2 ’(t) = x2 (t) − x2 (t − 2Δ t)

(1.15)

Fig. 1.7 Translation and rotation transformation in state-space

The rotation and translation transformation facilitates the behavior analysis of the → → v 1 . Therefore, heuristic rules can be applied to vector − v 2 in relation to vector − → perform the gain in module and angle of the state vectors yielding vector − v 3 ’ to each n − 1 complex plan. Finally, a strategy is created to determine which will be the final value of the reference variable. An effective strategy is to add the value of the reference variable and the average of the calculated increments of this variable in the complex plans. Fig. 1.8 shows two pictures associated with two examples of a set of heuris→ tic rules that can be used to produce vector − v 3 ’. In each picture, the point that is closer to the left position in the circumference is the point P0 ’, in the center of the circumference is fixed the point P1 ’ and the points marked with tiny circle in the circumference symbolize several possibilities of point P2 ’. And finally, the results

1

New HIS to Solve LP and QP Optimization and Increase Convergence Speed

15

3

3

2

2

1

1 X2’(t)

X2’(t)

of the heuristic rules are marked with green circle, points P3 ’. In order to obtain the final point P3 , we apply the inverse transformation of translation and rotation to the point P3 ’, thus generating the appropriated value to initialize the recurrent network. Fig. 1.9 shows an example of application of the heuristic rules to estimate a better point through the dynamics in the state-space. The external curve represents the dynamics of the recurrent network without the heuristic rules, and the internal one represents the dynamics using the HIS (ANN and TDSS) which is based on heuristic rules (TDSS). We point out that, in the internal curve, the points with circle as a marker are the iteration points performed by the network and the points with a plus sign are the points estimated by the heuristic rules (P3 ).

0

0

−1

−1

−2

−2

−3

−3 0

2 X1’(t)

4

0

2 X1’(t)

(a)

4

(b)

Fig. 1.8 Variations of the rules applied to points P0 , P1 , P2 to calculate the estimated point P3 2.5

2

Original Orbit

x2(t)

1.5

1

Predict Point by Heuristic Rule (HIS−2) 0.5

Advanced Orbit Point by Recurrent NN 0

−0.5 −1

0

1

2

3

4

5

x1(t)

Fig. 1.9 Graph of the convergence orbit of the state variables x1 and x2 : external curve ANN; internal curve - HIS

16

O.N. Neto, R.R.B. de Aquino, and M.M.S. Lira

By iteratively applying the ANN and the HR, the system will perform to reduce the curvature of the orbit in the state-space jumping from one orbit to another until it achieves the solution of the problem (point of equilibrium of the recurrent network).

1.4 Case Studies In order to test the proposed hybrid intelligent system, we chose mathematical programming problems previously solved in [16, 31, 32]. The following cases were solved using the HIS which was implemented in the MATLAB. In addition, to solve the differential equations involved in the problem, we implemented the Richardson extrapolation method of level four, which has in its structure the classical Runge-Kutta method of order four, and uses a fixed step-size of integration [33].

1.4.1 Case 1: Mathematical Linear Programming Problem – Four Variables A classical linear programming problem [31] was used to choose the parameters of the developed heuristic rules. In [31] the problem was solved to compare the performance of several recurrent network models. In this case we deal with the following problem LP1 : (LP1 ) min f (x) = −8x1 − 8x2 − 5x3 − 5x4 s.t.

x1 + x3 = 40

x2 + x4 = 60 −5x1 + 5x2 ≤ 0 2x1 − 3x2 ≤ 0 x≥0 x ∈ R4

(1.16)

1.4.2 Case 2: Mathematical Linear Programming Problem – Eleven Variables With the heuristic rules parameters already defined in case 1, we chose a larger scale problem to compare the models performance. In this case, we chose a flow problem of minimum cost (LP2 ) solved in [32]. In this problem there are several nodes (points) representing several consumers and suppliers which are connecting through paths (arcs). The aim of the problem is to calculate the flow through all paths in order to minimize the total cost, whose value is calculated by the sum of the products of the cost and the flow performed in each arch. The problem can be represented as a graph as shown in Fig. 1.10 where the amount of flow on a node cannot exceed the capacity of the node. A network is a set of elements called nodes

1

New HIS to Solve LP and QP Optimization and Increase Convergence Speed

17

and a set of elements called arcs, each arc ei j being an ordered pair (i, j) of distinct nodes i and j. If ei j is an arc, then the node i is called the tail of ei j and the node j is called the head of ei j . The directed graph shown in Fig. 1.10 is formed of 6 nodes and 11 arcs. e12

1 e13 e15

2 e23

3

e42

e53

e62

e43 5

e54

4 e64 e56

6

Fig. 1.10 A directed graph of a minimum cost flow problem

The cost of each arc is represented by vector c and its maximum capacity flow by vector b, and the demands by vector w. If wi ≤ 0, the node is a supplier (sources) and, if wi > 0, the node is consumers (sinks). Suppose, for instance, that we have wT = [−9 4 17 1 − 5 − 8]. The matrix H of the network is called the incidence matrix of our network. More generally, the incidence matrix of a network with n nodes and m arcs has n rows and m columns. Thus, our matrix H has size 6 x 11 and is formed as follows: ⎤ ⎡ −1 −1 −1 0 0 0 0 0 0 0 0 ⎢ 1 0 0 −1 1 1 0 0 0 0 0 ⎥ ⎥ ⎢ ⎢ 0 1 0 1 0 0 1 1 0 0 0⎥ ⎥ (1.17) H =⎢ ⎢ 0 0 0 0 −1 0 0 −1 1 1 0 ⎥ ⎥ ⎢ ⎣ 0 0 1 0 0 0 −1 0 −1 0 −1 ⎦ 0 0 0 0 0 −1 0 0 0 −1 1 Considering that there are no losses in the network, i.e., everything that is produced is consumed then the sum of all the elements wi j of the graph is zero. This condition turns the matrix H into a linearly dependent (LD) matrix, in other words, any row can be obtained by a linear combination of the other rows. In order to overcome this problem, we remove a row of the matrix H and one element of the column vector w. Here, the last row of the matrix H was removed, turning this matrix and the vector w into a truncated incidence matrix and a truncated vector, according to [32]. ⎡ ⎤ −1 −1 −1 0 0 0 0 0 0 0 0 ⎢ 1 0 0 −1 1 1 0 0 0 0 0 ⎥ ⎢ ⎥ ⎥ (1.18) H =⎢ ⎢ 0 1 0 1 0 0 1 1 0 0 0⎥ ⎣ 0 0 0 0 −1 0 0 −1 1 1 0 ⎦ 0 0 1 0 0 0 −1 0 −1 0 −1

18

O.N. Neto, R.R.B. de Aquino, and M.M.S. Lira

Problem LP2 has the following form: (LP2 ) min f (x) = 3x1 + 5x2 + x3 + x4 + 4x5 + x6 + 6x7 + x8 + x9 + x10 + x11 s.t. − 1x1 − 1x2 − 1x3 = 9 −x1 − x4 + x5 + x6 = −4 x2 + x4 + x7 + x8 = −17 −x5 − x8 + x9 + x10 = −1 x3 − x7 − x9 − x11 = 5 x ≤ [2 10 10 6 8 7 9 9 10 8 6]T x≥0 x ∈ R11

(1.19)

1.4.3 Case 3: Mathematical Quadratic Programming Problem – Three Variables As an example of quadratic programming (QP), the economic dispatch problem was solved as formulate in [16]. In this problem, the aim is to minimize the total cost and respond to the demand of the power system. The formulation contemplates 3 thermal generators (n = 3) connected to just one load. Defining fi as the generation cost of the i-th generation unit, xi as the generated power by the i-th generation unit, w as the total power demand of the load; the limits xi,min , xi,max are defined by the physical limitation of the i-th generation unit. The power economic dispatch is expressed as QP: (QP) min f (x) =

n

∑ fi (xi )

i=1 n

s.t.

∑ xi − w = 0

i=1

xmin ≤ x ≤ xmax x ∈ Rn

(1.20)

Data were obtained from [16]: xmin = [150 100 50]T in MW, xmax = [600 400 200]T in MW, w = 850 MW and the following costs for the generator units: f1 (x1 ) = 561 + 7.92x1 + 0.00156x21 f2 (x2 ) = 310 + 7.85x2 + 0.00194x22 f3 (x3 ) = 78 + 7.95x3 + 0.00482x23

(1.21)

1

New HIS to Solve LP and QP Optimization and Increase Convergence Speed

19

1.5 Simulations The chosen parameters to simulate the problems LP1 and LP2 were: integration step size of 1e−3 , 100 for the parameter s of the neural network in the first and the second phase, value 1.1 for the parameter e of the network in the second phase. The main results of LP1 and LP2 simulation are presented in Table 1.2.

Table 1.2 Simulation Results of LP1 and LP2 LP1 LP2 ANN HIS-1a HIS-2b ANN HIS-1a HIS-2b No. points by the ANN (Phase 1) 8316 1000 2391 8670 No. points by the HR (Phase 1) 331 795 Total no. points (Phase 1) 8316 1331 3186 8670 Normalized computer processing time (Phase 1) 1.00 0.12 0.29 1.00 8.32 1.33 3.19 8.67 Time instant when the switch is closed (s) = t1 No. of calculated points by the ANN (Phase 2) 8607 8277 8316 7692 No. of calculated points in both phases 16923 9608 11502 16362 Initial cost (Phase 1) = f (x(0)) -260.00 -260.00 -260.00 0.00 -741.77 -741.82 -741.78 55.67 Final cost (Phase 1) = f (x(t1 )) Final cost (Phase 2) = f (x(tend )) -740.00 -740.00 -740.00 56.00

2965 986 3951 0.33 3.95 7251 11202 0.00 55.65 55.99

3099 1031 4130 0.35 4.13 7263 11393 0.00 55.63 56.00

a

HIS-1 = ANN and method of Tendency based on the Dynamics in Space-Time(TDST). b HIS-2 = ANN and method of Tendency based on the Dynamics in State-Space (TDSS).

The results shown in Table 1.2 at row 5 point out that both proposed hybrid systems (HIS-1 and HIS-2) were able to advance the dynamics of the simulated linear problems efficiently. This reduces greatly the time to process the network algorithm since at each integration step size, n ODEs are solved, where n is the number of variables in the problem. For instance, for the problem LP1 , the total number of points calculated at the end of the first phase by using the ANN was 8316, then 33264 ODEs were solved while using the HIS-1. It was necessary 1000 points yielding 4000 ODEs to be solved in order to reach the end of the first phase, in other words, the HIS-1 reduced the computational effort by approximately 88% compared to the ANN. Besides, for the LP2 problem, this rate was approximately 66%. The rates of computational effort reduction for problems LP1 and LP2 when comparing the ANN to the HIS-2 were 71% and 64%, respectively. Fig. 1.11-1.13 presents the simulation results for the LP1 problem and Fig. 1.14-1.16 for the LP2 problem. The chosen parameters to simulate the problems QP: integration step size of 1e−2 ; 50 for the parameter s of the neural network in the first phase. The initial condition used was x(0) = [400 300 150]T .

20

O.N. Neto, R.R.B. de Aquino, and M.M.S. Lira

40

40

x1

x2

X1(t) x X2(t)

35

35

30

30

X1(t) x X4(t) 25

25 x4

20

20

15

15

10

10

5

0

2

4

6

8

10

X1(t) x X3(t)

5

x3

0

12

14

0

16

10

15

20

(a)

25

30

35

40

(b)

Fig. 1.11 Dynamics of the problem LP1 obtained by the ANN with the initial condition x(0) = [10 10 10 10]T : (a) Dynamic in time-state plan; (b) Dynamic in state-space plan, taking the variable x1 (t) as reference

40

40 x1

x2

X1(t) x X4(t)

35

35

30

30

25

25 x4

20

15

10

10

5

0

1

2

3

4

X1(t) x X3(t)

5

x3

0

X1(t) x X2(t)

20

15

5

(a)

6

7

8

9

10

0

10

15

20

25

30

35

40

(b)

Fig. 1.12 Dynamics of the problem LP1 , obtained by the HIS-1 with the initial condition x(0) = [10 10 10 10]T : (a) Dynamic in time-space plan; (b) Dynamic in state-space plan, taking the variable x1 (t) as reference

The results shown in Table 1.3 at row 5 point out that both proposed hybrid systems were able to advance the dynamics of the simulated quadratic problems efficiently. This reduces greatly the time to process the network algorithm since at each step size integration, n ODEs are solved, where n is the number of variables in the problem. For instance, for the problem QP, the total number of points calculated at the end of the first phase by using the ANN was 172629, then 517887 ODEs were solved, while using the HIS-1, it was necessary 8257 points yielding 24771 ODEs to be solved in order to reach the end of the first phase, in other words, the HIS-1 reduced the computational effort by approximately 95% compared to the

1

New HIS to Solve LP and QP Optimization and Increase Convergence Speed

21

40

40 x1

x2

35

35

X1(t) x X4(t)

30

30

25

25 x4

20 15

15

10

10

5

0

2

4

X1(t) x X3(t)

5

x3

0

X1(t) x X2(t)

20

6

8

10

12

0

10

15

20

25

(a)

30

35

40

(b)

Fig. 1.13 Dynamics of the problem LP1 , obtained by the HIS-2 with the initial condition x(0) = [10 10 10 10]T : (a) Dynamic in time-space plan; (b) Dynamic in state-space plan, taking the variable x1 (t) as reference

9

9 x8

8

8

x9 x6

7

7 6

6 x4

5

5

x3

4

4

x2

3

3

x1

2 x7

1

2

x10

1

x11

0

0

x5

-1 0

2

4

6

8

(a)

10

12

14

16

0

0.5

1

1.5

2

2.5

(b)

Fig. 1.14 Dynamics of the problem LP2 , obtained by the ANN with the initial condition x(0) = [0 0 0 0 0 0 0 0 0 0 0]T : (a) Dynamic in time-space plan; (b) Dynamic in state-space plan, taking the variable x1 (t) as reference

ANN. In addition to HIS-2, and the QP problem, this rate was approximately 70%. Fig. 1.17-1.19 presents the simulation results for the problem QP. All case studies were carried out in the same computer, thus we take the processing time of the ANN in phase 1 as the base to normalize the hybrid case in the same phase. Here, we point out that the hybrid systems were not used in phase 2. As a result, we have observed that for the LP1 case, the HIS-1 has taken a processing time of 12%, while the HIS-2 has taken 29%; for the LP2 case, the HIS-1 has taken a processing time of 33%, while the HIS-2 has taken 35%; and for the QP case, the HIS-1 has taken a processing time of 6%, while the HIS-2 has taken 45%. It is important

22

O.N. Neto, R.R.B. de Aquino, and M.M.S. Lira

16

16

14

14

12

12 10

10

x8 x9

8

8

x6

6

x4 x3 x2 x1 x10 x5 x7 x11

4 2 0 -2

6 4 2 0 -2

-4 0

2

4

6

8

10

-1.5

-1

-0.5

0

(a)

0.5

1

1.5

2

2.5

3

(b)

Fig. 1.15 Dynamics of the problem LP2 , obtained by the HIS-1 with the initial condition x(0) = [0 0 0 0 0 0 0 0 0 0 0]T : (a) Dynamic in time-space plan; (b) Dynamic in state-space plan, taking the variable x1 (t) as reference

9

9

x8

x9

8

8 x6

7

7 6

6 x4

5

5

x3

4

4

x2

3

3

x1

2

2

x10

1

x5

0

x7

x11

1 0

-1 0

2

4

6

(a)

8

10

0

0.5

1

1.5

2

2.5

(b)

Fig. 1.16 Dynamics of the problem LP2 , obtained by the HIS-2 with the initial condition x(0) = [0 0 0 0 0 0 0 0 0 0 0]T : (a) Dynamic in time-space plan; (b) Dynamic in state-space plan, taking the variable x1 (t) as reference

to note that the heuristic rule efficiency varies according to the type of the problem. In this work, the results have showed that the HIS-1 yielded better performance, specifically, in the LP1 and QP problems. Therefore, we could observe a decrease in the processing time yielded by the implemented heuristic rules, while reducing the number of ODE computed. These rules estimated the next values to each variable of the problem throughout the convergence. We can highlight that as ODE also calculates points during the application of the HIS, the proposed systems can correct themselves in case of incorrect estimative, showing the high performance for being resilient.

1

New HIS to Solve LP and QP Optimization and Increase Convergence Speed

23

Table 1.3 Simulation Results of QP

QP HIS-1a

ANN

HIS-2b

No. points by the ANN (Phase 1) 172629 8257 51900 No. points by the HR (Phase 1) 2750 17299 Total no. points (Phase 1) 172629 11007 69199 Normalized computer processing time (Phase 1) 1.00 0.06 0.45 1726.29 110.07 691.99 Time instant when the switch is closed (s) = t1 Final cost (Phase 1) = f (x(t1 )) 22680.05 22680.05 22680.05 a

HIS-1 = ANN and method of Tendency based on the Dynamics in Space-Time(TDST). b HIS-2 = ANN and method of Tendency based on the Dynamics in State-Space (TDSS).

400

320

x1

X1(t) x X3(t)

300

350

280 x2

260

300

240 250

220 200

200

180 160

150

140

x3 0

200

400

600

800

(a)

1000

1200

1400

1600

1800

120 393

X1(t) x X2(t) 394

395

396

397

398

399

400

401

(b)

Fig. 1.17 Dynamics of the problem QP, obtained by the ANN with the initial condition x(0) = [400 300 150]T : (a) Dynamic in time-space plan; (b) Dynamic in state-space plan, taking the variable x1 (t) as reference

1.6 Conclusion In this paper, two Hybrid Intelligent Systems have been proposed. These systems combined Maa and Shamblatt network with heuristic rules. Maa and Shamblatt network is a two-phase recurrent neural network that provides the exact solution for linear and quadratic programming problem. When compared to conventional linear and nonlinear optimization techniques, the two-phase network formulation becomes advantageous as there is no matrix inversion required. The main aim of the proposed HIS is to increase the speed of convergence towards the optimal point which is guaranteed by the ANN. In the cases presented, the optimal convergence

24

O.N. Neto, R.R.B. de Aquino, and M.M.S. Lira

x1

400

320

X1(t) x X2(t)

300

350

x2

280 260

300

240 250

220 200

200

180 160

150

140

x3 0

20

40

60

80

120 393

100

X1(t) x X3(t) 394

395

396

(a)

397

398

399

400

401

(b)

Fig. 1.18 Dynamics of the problem QP, obtained by the HIS-1 with the initial condition x(0) = [400 300 150]T : (a) Dynamic in time-space plan; (b) Dynamic in state-space plan, taking the variable x1 (t) as reference

x1

400

320

X1(t) x X2(t)

300 350

x2

280 260

300

240 250

220 200

200

180 160

150

140

x3 0

100

200

300

(a)

400

500

600

700

120 393

X1(t) x X3(t) 394

395

396

397

398

399

400

401

(b)

Fig. 1.19 Dynamics of the problem QP, obtained by the HIS-2 with the initial condition x(0) = [400 300 150]T : (a) Dynamic in time-space plan; (b) Dynamic in state-space plan, taking the variable x1 (t) as reference

was reached. The proposed systems have both advantages. The simulation analyses show a reduction in the computational effort by approximately 95% compared to the ANN in the QP case solved in this paper which has a guaranteed optimal convergence, without inverting matrices. The implementation of the proposed HIS has been developed to solve operational planning problems of large-scale, which may be applied in future works. That is, a large-scale economic power dispatch, with the scheduling of hydro, thermal and wind power plants to minimize the overall production cost, while satisfying the load demand in the mid-term operation planning of hydrothermal generation systems. In future works, we will propose the combination of these heuristic rules and/or the application in the second phase of Maa and Shanblatt method.

1

New HIS to Solve LP and QP Optimization and Increase Convergence Speed

25

References 1. Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79, 2552–2558 (1982) 2. Haykin, S.: Neural networks: a comprehensive foundation, 2nd edn. Prentice Hall, USA (1999) 3. Hopfield, J.J.: Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. USA 81, 3088–3092 (1984) 4. Hopfield, J.J.: Learning algorithms and probability distributions in feed-forward and feed-back networks. Proc. Natl. Acad. Sci. USA 84, 8429–8433 (1987) 5. Hopfield, J.J.: The effectiveness of analogue neural network hardware. Network: Computation in Neural Systems 1(1), 27–40 (1990) 6. Hopfield, J.J., Feinstein, D.I., Palmer, R.G.: Unlearning has a stabilizing effect in collective memories. Nature 304, 158–159 (1983) 7. Hopfield, J.J., Tank, D.W.: Neural Computation of Decisions in Optimization Problem. Biological Cybernetics 52, 141–152 (1985) 8. Hopfield, J.J., Tank, D.W.: Computing with Neural Circuits: A Model. Science 233(8), 625–633 (1986) 9. Tank, D.W., Hopfield, J.J.: Simple Neural Optimization Networks: An A/D Converter, Signal Decision Circuit, and a Linear Programming Circuit. IEEE Trans. on Circuits and Systems 33(5), 533–541 (1986) 10. Ludemir, T.B., Braga, A.P., Carvalho, A.C.P.L.F.: Redes Neurais Artificiais Teoria e Aplicacoes. 1st ed. Rio de Janeiro, RJ: LTC - Livros Tecnicos e Cientificos Editora S.A. (2000) 11. Pyne, I.B.: Linear Programming on an electronic analogue computer. Trans. AIEE. Part I (Comm. & Elect.) 75, 139–143 (1956) 12. Kennedy, M.P., Chua, L.O.: Unifying Tank and Hopfield Linear Programming Circuit and the Canonical Nonlinear Programming Circuit of Chua and Lin. IEEE Trans. on Circuits and Systems 34(2), 210–214 (1987) 13. Chiu, C., Maa, C.Y., Shanblatt, M.A.: An artificial neural network algorithm for dynamic programming. Int. J. Neural Syst. 1(3), 211–220 (1990) 14. Maa, C.Y., Shanblatt, M.A.: A Two-Phase Optimization Neural Network. IEEE Transactions on Neural Networks 3(6), 1003–1009 (1992) 15. Kennedy, M.P., Chua, L.O.: Neural Networks for Nonlinear Programming. IEEE Trans. on Circuits and Systems 35(5), 210–220 (1988) 16. Maa, C.Y., Shanblatt, M.A.: Linear and Quadratic Programming Neural Network Analysis. IEEE Transactions on Neural Networks 3(4), 580–594 (1992) 17. Chiu, C., Maa, C.Y., Shanblatt, M.A.: Energy Function Analysis of Dynamic Programming Neural Networks. IEEE Transactions on Neural Networks 2(4) (July 1991) 18. Xia, Y.S.: A New Neural Network for Solving Linear and Quadratic Programming Problems. IEEE Transactions on Neural Networks 7(6), 1544–1547 (1996) 19. Tao, Q., Cao, J.D., Xue, M.S., Qiao, H.: A High Performance Neural Network for Solving Nonlinear Programming Problems with Hybrid Constraints. Phys. Lett. A 288(2), 88–94 (2001) 20. Xia, Y.S., Wang, J.: A General Methodology for Designing Globally Convergent Optimization Neural Networks. IEEE Transactions on Neural Networks 9(6), 1331–1343 (1998)

26

O.N. Neto, R.R.B. de Aquino, and M.M.S. Lira

21. Xia, Y.S., Wang, J.: A Recurrent Neural Network for Solving Nonlinear Convex Programs Subject to Linear Constraints. IEEE Transactions on Neural Networks 16(2), 379–386 (2005) 22. Dieu, V.N., Ongsakul, W.: Enhanced Merit Order and Augmented Lagrange Hopfield Network for Hydrothermal Scheduling. Electrical Power and Energy Systems 30, 93–101 (2008) 23. Naresh, R., Dubey, J., Sharma, J.: Two-phase Neural Network Based Modeling Framework of Constrained Economic Load Dispatch. IEE Proc. Gener. Transm. Distrib. 151(3) (May 2004) 24. Aquino, R.R.B.: Recurrent Artificial Neural Networks: an application to optimization of hydro thermal power systems (in Portuguese), Ph.D. Thesis, COPELE/UFPE, Campina Grande, Brazil (January 2001) 25. Rosas, P., Aquino, R.R.B., et al.: Study of Impacts of a Large Penetration of Wind Power and Distributed Power Generation as a Whole on the Brazilian Power System. In: European Wind Energy Conference (EWEC), London (November 2004) 26. Witten, I.H., Frank, E.: Data Mining Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005) 27. Mitra, S., Mitra, M., Chaudhuri, B.B.: Pattern Defined Heuristic Rules and Directional Histogram Based Online ECG Parameter Extraction. Measurement 42, 150–156 (2009) 28. Tuncel, G.: A Heuristic Rule-Based Approach for Dynamic Scheduling of Flexible Manufacturing Systems. In: Levner, E. (ed.) Multiprocessor Scheduling: Theory and Applications, December 2007, p. 436. Itech Education and Publishing, Vienna (2007) 29. Baykasoglu, A., Ozbakir, L., Dereli, T.: Multiple Dispatching Rule Based Heuristic for Multi-Objective Scheduling of Job Shops Using Tabu Search. In: Proceedings of MIM 2002: 5th International Conference on Managing Innovations in Manufacturing (MIM) Milwaukee, Wisconsin, USA, September 9-11, pp. 1–6 (2002) 30. Idris, N., Baba, S., Abdullah, R.: Using Heuristic Rules from Sentence Decomposition of Experts Summaries to Detect Students Summarizing Strategies. International Journal of Human and Social Sciences 2, 1 (Winter 2008), www.waset.org 31. Zak, S.H., Upatising, V., Hui, S.: Solving Linear Programming Problems with Neural Networks: A Comparative Study. IEEE Transactions on Neural Networks 6(1), 94–104 (1995) 32. Chvatal, V.: Linear Programming. W.H. Freman and Company, New York (1983) 33. Lastman, G.J., Sinha, N.K.: Microcomputer-based numerical methods for science and engineering. Saunders Colleg Pubblishing, USA (1988)

Chapter 2

A Novel Optimization Algorithm Based on Reinforcement Learning Janusz A. Starzyk, Yinyin Liu, and Sebastian Batog

Abstract. In this chapter, an efficient optimization algorithm is presented for the problems with hard to evaluate objective functions. It uses the reinforcement learning principle to determine the particle move in search for the optimum process. A model of successful actions is build and future actions are based on past experience. The step increment combines exploitation of the known search path and exploration for the improved search direction. The algorithm does not require any prior knowledge of the objective function, nor does it require any characteristics of such function. It is simple, intuitive and easy to implement and tune. The optimization algorithm was tested using several multi-variable functions and compared with other widely used random search optimization algorithms. Furthermore, the training of a multi-layer perceptron, to find a set of optimized weights, is treated as an optimization problem. The optimized multi-layer perceptron was applied to Iris database classification. Finally, the algorithm is used in image recognition to find a familiar object with retina sampling and micro-saccades.

2.1 Introduction Optimization is a process to find the maximum or the minimum function value within given constraints by changing values of its multiple variables. It can be the essential for solving complex engineering problems in such areas as computer science, aerospace, machine intelligence applications, etc. When the analytical relation Janusz A. Starzyk · Yinyin Liu Ohio University, School of Electrical Engineering and Computer Science, U.S.A. e-mail: [email protected],[email protected] Sebastian Batog Silesian University of Technology, Institute Of Computer Science, Poland e-mail: [email protected]

Y. Tenne and C.-K. Goh (Eds.): Computational Intelligence in Optimization, ALO 7, pp. 27–47. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

28

J.A. Starzyk, Y. Liu, and S. Batog

between the variables and the objective function value is explicitly known, analytical methods, such as Lagrange multiplier methods [1], interior point methods [18], Newton methods [30], gradient descent methods [25], etc., can be applied. However, in many practical applications, analytical methods do not apply. This happens when the objective functions are unknown, when relations between variables and function value are not given or difficult to find, when the functions are known while their derivatives are not applicable, or when the optimum value of function cannot be verified. In these cases, iterative search processes are required to find the function optimum. Direct search algorithms [10] contain a set of optimization methods that do not require derivatives and do not approximate either the objective functions or their derivatives. These algorithms find locations with better function values following a search strategy. They only need to compare the objective function values in successive iterative steps to make the move decision. Within the category of direct search, distinctions can be made among three classes including pattern search methods [28], simplex methods [6], and adaptive sets of search directions [23]. In pattern search methods, the variables of the function are varied by either steps of predetermined magnitude or the steps sizes are reduced at the same degree [15]. Simplex methods construct a simplex in ℜN using N+1 points and use the simplex to drive the search for optimum. The methods with adaptive sets of search directions, proposed by Rosenbrock [23] and Powell [21], construct conjugate directions using the information about the curvature of the objective function during the search. In order to avoid local minima, random search methods are developed utilizing randomness in setting the initial search points and other search parameters like the search direction or the step size. In Optimized Step-Size Random Search (OSSRS) [24], the step size is determined by fitting a quadratic function for the optimized function values in each of the random directions. The random direction is generated with a normal distribution of a given mean and standard deviation. Monte-Carlo optimizations adopted randomness in the search process to generate the possibilities to escape from the local minima. Simulate Annealing (SA) [13] is one typical kind of Monte-Carlo algorithm. It exploits the analogy between the search for a minimum in the optimization problem and the annealing process in which a metal cools and stabilizes into a minimum energy crystalline structure. It accepts the move to a new position with worse function value with a probability, which is controlled by the ”temperature” parameter, and the probability decreases along the ”cooling process”. SA can deal with highly nonlinear, chaotic problems provided that the cooling schedule and other parameters are carefully tuned. Particle Swarm Optimization (PSO) [11] is a population-based evolutionary computational algorithm. It exploits the cooperation within the solution population instead of the competition among them. At each iteration in PSO, a group of search particles make moves in a mutually coordinated fashion. The step size of a particle is a function of both the best solution found by that particle and the best solution found so far by all the particles in the group. The use of a population of search particles and the cooperation among them enable the algorithm to evaluate function values in a wide range of variables in the input space and to find the optimum position. Each

2

A Novel Optimization Algorithm Based on Reinforcement Learning

29

particle only remembers its best solution and the global best solution of the group to determine its step sizes. Generally, during the course of search, a sequence of decisions on the step sizes is made and a number of function values are obtained in these optimization methods. In order to implement an efficient search for the optimum point, it is desired that such historical information can be utilized in the optimization process. Reinforcement Learning (RL) [27] is a type of learning process to maximize certain numerical values by combining exploration and exploitation and using rewards as learning stimuli. In the reinforcement learning problem, the learning agent performs the experiments to interact with the unknown environment and accumulate the knowledge during this process. It is a trial-and-error exploratory process with the objective to find the optimum action. During this process, an agent can learn to build the model of the environment to instruct its search, so that the agent can predict the environment’s response to its actions and choose the most useful actions for its objectives based on its past exploring experience. Surrogate based optimization refers to an idea of speeding optimization process by using surrogates for the objectives and constraints functions. The surrogates also allow for the optimization of problems with non-smooth or noisy responses, and can provide insight into the nature of the design space. The max-min SAGA approach [20] is to search for designs that have the best worst case performance in the presence of parameter uncertainty. By leveraging a trust-region approach which uses computationally cheap surrogate models, the present approach allows for the possibility of achieving robust design solutions on a limited computational budged. Another example of a surrogate based optimization is the surrogate assisted Hooke-Jeeves (SAHJA) algorithm [8] which can be used as a local component of a global optimization algorithm. This local searcher uses the Hooke-Jeeves method, which performs its exploration of the input space intelligently employing both the real fitness and an approximated function. The idea of building knowledge about an unknown problem through exploration can be applied in the optimization problems. To find the optimum of an unknown multivariable function, an efficient search procedure can be performed using only historical information from conducted experiments to expedite the search. In this chapter, a novel and efficient optimization algorithm based on reinforcement learning is presented. This algorithm uses simple search operators and will be called reinforcement learning optimization (RLO) in the later sections. It does not require any prior knowledge of the objective function or function’s gradient information, nor does it require any characteristics of the objective function. In addition, it is conceptually very simple and easy to implement. This approach to optimization is compatible with the neural networks and learning through interaction, thus it is useful for systems of embodied intelligence and motivated learning as presented in [26]. The following section presents the RLO method and illustrates it within several machine learning applications.

30

J.A. Starzyk, Y. Liu, and S. Batog

2.2 Optimization Algorithm 2.2.1 Basic Search Procedure A N-variable optimization objective function V = f (p1 , p2 , ..., pN ) (p1 , p2 , ..., pN ,V ∈ ℜ1 ) could have several local minima and several global minima Vopt1 , ...,VoptN . It is desired that the search process, initiated from a random point, finds a path to the global optimum point. Unlike particle swarm optimization [11], this process can be performed with a single search particle that learns how to find its way to the optimum point. It does not require the cooperation among a group of particles, although implementing the cooperation among several search particles may further enhance the search process in this method. At each point of the search, the search particle intends to find a new location with a better value within a searching range around it and then determines the direction and the step size for the next move. It tries to reach the optimum by exploring weighted random search of each variable (coordinate). The step size of search in each variable is randomly generated with its own probability density function. These functions are gradually learned during the search process. It is expected that at the later stage of search, the probability density functions are approximated for each variable. Then the stochastically randomized path to the minimum point of the function from the start point is learned. The step sizes of all the coordinates determine the center of the new searching area and the standard deviations of the probability functions determine the size of the new searching area around the center. In the new searching area, several locations PS are randomly generated. If there is a location p’ with better value than the current one, the search operator moves to it. From this new location, new step sizes and new searching range are determined, so that the search for optimum continues. If in the current searching area, there is no point with better value that the search particle can move to, another set of random points are generated until no improvement is obtained after several, say M, trials. Then the searching area size and step sizes are modified in order to find a better function value. If no better value is found after K trials of generating different searching areas or the proposed stopping criterion is met, we can claim that the optimum point has been found. The algorithm of searching for the minimum point is schematically shown in the Figure 2.1.

2.2.2 Extracting Historical Information by Weighted Optimized Approximation After the search particle makes a sequence of n moves, the step sizes of these moves d pti (t = 1, 2, ..., n; i = 1, 2, ..., N) are available for learning. These historical steps have made the search particle move towards better values of the objective function and hopefully get closer to the optimum location. In this sense, these steps are the

2

A Novel Optimization Algorithm Based on Reinforcement Learning

31

Fig. 2.1 The algorithm of RLO searching for the minimum point

successful actions during the trial. It is proposed that the successful actions which result in a positive reinforcement (as the step sizes of each coordinate) follow a function of the iterative steps t, as in (2.1), where dpi represents the step sizes on ith coordinate and f i (t) is the function for coordinate i. d pi = fi (t) (i = 1, 2, ..., N),

(2.1)

These unknown functions f i (t) can be approximated, for example, using polynomials through the least-squared fit (LSF) process. ⎫ ⎧ a0 ⎪ ⎪ ⎪ ⎪ ⎡ ⎤ ⎪ ⎧ i⎫ ⎪ 1 t1 t12 ... t1B ⎪ ⎬ ⎨ d p1 ⎬ ⎨ a1 ⎪ ⎣ ... ... ... ... ... ⎦ a2 = ... (2.2) ⎩ i⎭ ⎪ ⎪ ⎪ d p ... 1 tn tn2 ... tnB ⎪ ⎪ ⎪ n ⎪ ⎪ ⎭ ⎩ aB In (2.2), the step sizes from d pi1 to d pin are the step sizes on a certain coordinate during n steps and are fitted as unknown function values using polynomials of order B. The polynomial coefficients a0 to aB can be obtained and will represent the function f i (t) to estimate dpi , d pi =

B

∑ a jt j.

(2.3)

j=0

Using polynomials for function approximation could be easy and efficient. However, considering the characteristic of optimization problems, we have two concerns. First, in order to generate a good approximation while avoiding overfitting, a proper order of polynomials must be selected. In the optimized approximation algorithm (OAA) presented in [17], the goodness of fit is determined by the so-called signalto-noise ratio figure (SNRF). Based on SNRF, an approximation stopping criterion

32

J.A. Starzyk, Y. Liu, and S. Batog

was developed. Using a certain set of basis functions for approximation, the error signal, computed as the difference between the approximated function and the sampled data, can be examined by SNRF to determine how much useful information it contains. The SNRF for the error signal, denoted as SNRF e , is compared to the precalculated SNRF for white Gaussian noise (WGN), denoted as SNRFW GN . If SNRF e is higher thanSNRFWGN , more basis functions should be used to improve the learning. Otherwise, the error signal shows the characteristic of WGN and should not be reduced any more to avoid fitting into the noise, and the obtained approximated function is the optimum function. Such process can be applied to determine the proper order of the polynomial. The second concern is that in the case of reinforcement learning, the knowledge about originally unknown environment is gradually accumulated throughout the learning process. The information that the learning system obtains at the beginning of the process is mostly based on initially random exploration. During the process of interaction, the learning system collects the historical information and builds the model of the environment. The model can be updated after each step of interaction. The decisions made at the later stages of the interaction are more based on the built model rather than a random exploration. This means that the recent results are more important and should be weighted more heavily than the old ones. For example, the weights applied can be exponentially increasing from the initial trials to the recent ones, as wt =

αt n

(t = 1, 2, ..., n),

(2.4)

where we can define α n = n. As a result, the weights are in the open interval (0:1], and weight is 1 for the most recent sample. Applying the weights in the LSF, we have the weighted least-squared fit (WLSF), expressed as follows: ⎫ ⎧ a0 ⎪ ⎪ ⎪ ⎪ ⎫ ⎤ ⎡ ⎪ ⎧ ⎪ 1 · w1 t1 w1 t12 w1 ... t1B w1 ⎪ ⎬ ⎨ d p1 w1 ⎬ ⎨ a1 ⎪ ⎣ ... ... ... ... ... ⎦ a2 = ... (2.5) ⎪ ⎩ ⎪ ⎭ B 2 ⎪ ... w d p 1 · wn tn wn tn wn ... tn wn ⎪ ⎪ ⎪ n n ⎪ ⎪ ⎭ ⎩ aB Due to the weights applied to the given samples, the approximated function will fit to the recent data better than to the old ones. Utilizing the concept of OAA to obtain optimized WLSF, the SNRF for the error signal or WGN has to be estimated considering the sample weights. In the original OAA for one-dimensional problem [17], the SNRF for error signal was calculated as, C(e j , e j−1 ) (2.6) SNRFe = C(e j , e j ) − C(e j , e j−1 ) where C represents the correlation calculation, e j represents the error signal (j=1,2, ...,n), e j−1 represents the (circular) shifted version of the e j . The characteristics

2

A Novel Optimization Algorithm Based on Reinforcement Learning

33

of SNRF for WGN, expressed through the average value and the standard deviation, can be estimated from Monte-Carlo simulation, as (see derivation at [17])

μSNRF W GN (n) = 0

(2.7)

1 σSNRF W GN (n) = √ . n

(2.8)

Then the threshold, which determines whether SNRF e shows the characteristic of SNRFW GN and the fitting error should not be further reduced, is, thSNRF W GN (n) = μSNRF W GN (n) + 1.7σSNRF W GN (n).

(2.9)

For the weighted approximation, the SNRF for the error signal is calculated as, SNRFe =

C(e j · w j , e j−1 · w j−1 ) . C(e j · w j , e j · w j ) − C(e j · w j , e j−1 · w j−1 )

(2.10)

In Fig.2.2(a), σSNRF W GN (n) from a 200-run Monte-Carlo simulation is shown in the logarithmic scale. The σSNRF W GN (n) can be estimated as

σSNRF

W GN (N)

2 =√ . n

(2.11)

It is found that the 5% significance level can be approximated by the average value plus 1.5 times standard deviations for an arbitrary n. Fig.2.2(b) illustrates the histogram of SNRFW GN with 216 samples, as an example. The threshold in this case of a dataset with 216 samples can be calculated using μ + 1.5σ = 0 + 1.5 × 0.0078 = 0.0117. Therefore, to obtain an optimized weighted approximation in one-dimensional case, the following algorithm is performed. Optimized weighted approximation algorithm (OWAA) Step (2.1). Assume that an unknown function F, with input space t ⊂ ℜ1 is described by n training samples as d pt , (t = 1, 2, ..., n). Step (2.2). The signal detection threshold is pre-calculated for the given number of samples n based on SNRFW GN . For a one-dimensional problem, 1.5 · 2 thSNRF W GN (n) = √ . n Step (2.3). Take a set of basis functions, for example, polynomials of order from 0 up to order B. Step (2.4). Use these B+1 basis functions to obtain the approximated function, dˆpt =

B+1

∑

l=1

fl (xt ) (t = 1, 2, ..., n).

(2.12)

34

J.A. Starzyk, Y. Liu, and S. Batog

Fig. 2.2 Characteristic of SNRF for WGN in weighted approximation

Step (2.5). Calculate the approximation error signal, et = d pt − dˆpt

(t = 1, 2, ..., n).

(2.13)

Step (2.6). Determine SNRF of the error signal using (2.10). Step (2.7). Compare the SNRF e with thSNRF W GN . If the SNRF e is equal to or less than thSNRF W GN , or if B exceeds the number of samples, stop the procedure. In such case Fˆ is the optimized approximation. Otherwise, add one basis function, in this example increase the order of the approximating polynomial to B+1 and repeat Steps (2.4)-(2.7). Using the above algorithm, the proper order of polynomial is determined to extract the useful (but not the noise) information from the historical data. Also, the extracted information will fit into the recent results better than to the old ones. We illustrate this process of learning historical information by considering a 2variable function as an example. Example The function V (p1 , p2 ) = p22 sin(1.5p2 ) + 2p21 sin(2p1 ) + p1 sin(2p2 ) has several local minima, but only one global minimum, as shown in Fig. 2.3. In the process of interaction, the historical information after each iteration is collected. The historical step sizes of 2 coordinates are separately approximated, as shown in the Fig. 2.4 (a) and 2.4 (b). The step sizes of two coordinates are approximated by quadratic polynomials which are determined by OWAA and the coefficients of polynomials are obtained using WLSF. In Fig. 2.4, the approximated functions are compared with the quadratic polynomials whose coefficients are obtained from LSF. Again, it is

2

A Novel Optimization Algorithm Based on Reinforcement Learning

35

observed that, the function obtained using WLSF is fitted closer to the data in later iterations than the function obtained using LSF.

Fig. 2.3 A 2-variable function V (p1 , p2 )

Fig. 2.4 Function approximation for historical step sizes

The level of the approximation error signal et for step sizes of a certain coordinate dpi , which is the difference between the observed sampled data and the approximated function, can be measured by its standard deviation, as shown in (2.14).     1 n σ pi =  (2.14) ∑ (et − e)¯ 2 n t=1 This standard deviation will be called the approximation deviation in the following discussion. It represents the maximum deviation of the location of the search particle from the prediction by the approximated function in the unknown function optimization problem.

36

J.A. Starzyk, Y. Liu, and S. Batog

2.2.3 Predicting New Step Sizes The approximated functions will be used to determine the step sizes for the next iteration, as shown in (2.15) and Fig. 2.5 along with the approximated functions. i d pt+1 = f i (t + 1)

(2.15)

Fig. 2.5 Prediction of the step sizes for the next iteration

The step size functions are the model of environment that the learning system builds during the process of interaction based on historical information. The future step size determined by such model can be employed as exploitation of the existing model. However, such model built during the learning process cannot be treated as exact. Besides exploitation which best utilizes the obtained model, exploration is desired to a certain degree in order to improve the model and discover better solutions. The exploration can be implemented using Gaussian random generator (GRG). As a good trade-off between exploitation and exploration is needed, we propose to use the step sizes for the next iteration determined by the step size functions as the mean value and the approximation deviation as the standard deviation of the random generator. Gaussian random generators give several random choices of the step sizes. Effectively, the determined step sizes of multiple coordinates generate the center of the searching area, and the size of the searching range is determined by the standard deviations of GRG for the coordinates. The multiple random values generated by GRG for each coordinate effectively create multiple locations within the searching area. The objective function values of these locations will be compared and the location with the best value, called current best location, will be chosen as the place from which the search particle will continue searching in the next iteration. Therefore, the actual step sizes are calculated using the distance from the “previous best location” to the “current best location”. The actual step sizes will be added in the historical step sizes and used to update the model of the unknown environment.

2

A Novel Optimization Algorithm Based on Reinforcement Learning

37

Several locations of the search particle in this approach are illustrated in Fig. 2.6 using a 2-variable function as an example. The search particle was located at previous best location p prev (p1prev , p2prev ) and the previous step size was found as d p prev (d p1prev , d p2prev ) after current best location p(p1 , p2 )is found as the best location in previous searching area (an area with p(p1 , p2 ) in it, not shown in the figure). At current best position p(p1 , p2 ), using the environment model built with historical step sizes, the current step size is determined to be dp1 on coordinate 1 and dp2 on coordinate 2, so that the center of the searching area is determined. The approximation deviations of two coordinates σ p1 and σ p2 give the size of the searching range. Within the searching range, several random points are generated in order to find a better position to which the search operator will move.

Fig. 2.6 Step sizes and searching area

2.2.4 Stopping Criterion Search particle moves from every “previous best location” to “current best location” and step sizes actually taken are used for model learning. As new step sizes are generated, the search particle is expected to move to locations with better objective function values. In the proposed algorithm, the search particle only makes the move when a location with a better function value is found. However, if all the points generated in the current searching range have no better function values than the current best value, the search particle does not move and the GRG will repeat generating groups of particle locations for several trials. If no better location is found after M trials, we suspect that the current searching range is too small or the current step size is too large, which makes us miss the locations with better function values. In such case, we should enlarge the size of the searching area, and reduce the step size, as in (2.16),

38

J.A. Starzyk, Y. Liu, and S. Batog

σ pi = α σ pi d pi = ε d pi

(i = 1, 2, ..., N),

(2.16)

where α > 1, and ε < 1. If this new search is still not successful, the searching range and the step size will continue changing until some points with better function values are found. If at certain step of the search process, in order to find the new location with better function values, the current step size is reduced to be too small to make the search particle move anywhere, it indicates that the optimum point has been reached. The stop criterion can be defined by the current step size being β times smaller than the previous step size, as, d p < β d p prev

(0 < β < 1, β is usually small).

(2.17)

2.2.5 Optimization Algorithm Based on previous discussion, the proposed optimization algorithm (RLO) can be described as follows. (a). The procedure starts from a random point of the objective function with Nvariables V = f (p1 , p2 , ..., pN ) . It will try to make a series of moves to get closer to the global optimum point. (b). To change from the current location, the step size dpi (i=1, 2, . . . ,N) and the standard deviation σ pi (i = 1, 2, ..., N) for each coordinate are generated from the uniform probability distribution. (c). The step sizes dpi (i=1, 2, . . . N) determine the center of the searching area. The deviations of all the coordinates σ pi (i = 1, 2, ..., N) determine the size of the searching area. Several points Ps in this range are randomly chosen from Gaussian distribution using dpi as mean values and σ pi as standard deviations. (d). The objective function values are evaluated at these new points. Compare the objective function values on random points with that at the current location. (e). If the new points generated in Step (c) have no better values than the current position, Step (c ) is repeated for up to M trials until point with better function value is found. (f). If the search fails after M trials, enlarge the size of the searching area, and reduce the step size, as in (2.16). (g). If the search with the updated searching area size and the step sizes from Step (f) is not successful, the range and the step size will keep being adjusted until either some points with better values are found, the current step sizes are much smaller than previous step sizes as in (2.17), or function value changed by less than a prespecified threshold. If any of these conditions happens then the algorithm terminates. This also indicates that the optimum point has been reached. (h). Move the search particle to the point p(p1 , p2 ) with the best function value V b (a local minimum or maximum depending on the optimization objective). The distance between previous best point p prev (p1prev , p2prev ) and current best point p(p1 , p2 ) gives the actual step size dpi (i=1, 2, . . . , N). Collect the historical information of the step sizes taken during the search process.

2

A Novel Optimization Algorithm Based on Reinforcement Learning

39

(i). Approximate the function of the step sizes as a function of iterative steps using weighted least-square fit as in (2.5). The proper maximum order of the basis functions is determined using SNRF described in section 2.2.2 to avoid overfitting. (j). Use the modeled function to determine the step sizes dpi (i=1, 2, . . . ,N) for the next iteration step. The approximation deviation difference between the approximated step sizes and the actual step sizes σ pi (i = 1, 2, ..., N) gives the approximation deviation. Repeat Step (c) to (j). In general, the optimization algorithm based on the reinforcement learning builds the model of successful moves for a given objective function. The model is built based on historical successful actions and it is used to determine new actions. The algorithm combines the exploitation and exploration of searching using random generators. The optimization algorithm does not require any prior knowledge of the objective function or its derivatives nor there are any special requirements put on the objective function. The use of search operator is conceptually very simple and intuitive. In the following section, the algorithm is verified using several experiments.

2.3 Simulation and Discussion 2.3.1 Finding Global Minimum of a Multi-variable Function 2.3.1.1

A Synthetic Bivariate Function

A synthetic bivariate function V (p1 , p2 ) = p22 sin(1.5p2 ) + 2p21 sin(2p1 ) + p1 sin(2p2 ), used previously in the example in section 2.2.2, is used as the objective function. This function has several local minima and one global minimum equal to -112.2586. The optimization algorithm starts at a random point and performs the search process looking for the optimum point (minimum in this example). The number of random points Ps generated in the searching area in each step is 10. The scaling factors α and ε in (2.16) are 1.1 and 0.9. The β in (2.17) is 0.005. One possible search path is shown in Fig. 2.7 from the start location to the final optimum location as found by RLO algorithm. The global optimum is found in 13 iterative steps. The historical locations are shown in the figure as well. During the search process, the historical step sizes taken are shown in Fig. 2.8 with their approximation by WLSF. Example of another search process starting from another random point is performed and is shown in Fig. 2.9. The global optimum is found in 10 iterative steps. Table 2.1 shows changes in the numerical function values and adjustment of the step sizes dp1 and dp2 for p1 and p2 in the successive search steps. Notice how the step size was initially reduced to be increased again once the algorithm started to follow a correct path towards the optimum.

40

J.A. Starzyk, Y. Liu, and S. Batog

Fig. 2.7 Search path from start point to optimum

Fig. 2.8 Step sizes taken during the search process

Fig. 2.9 Search path from start point to optimum

2

A Novel Optimization Algorithm Based on Reinforcement Learning

41

Table 2.1 Function values and step sizes in a searching process

Search steps Function value V (p1 , p2 ) Step size d p1 Step size d p2 1 2 3 4 5 6 7 8 9 10

1.4430 -34.8100 -61.4957 -69.8342 -70.5394 -71.5813 -109.0453 -110.8888 -112.0104 -112.1666

2.9455 0.3570 -0.0508 -0.0477 -0.1232 0.0000 -0.0281 0.0495 0.0438

0.8606 -1.7924 -0.7299 -0.3114 0.2015 4.4358 0.3408 -0.0531 -0.0772

Such search process was performed for 300 random trials. The success rate of finding the global optimum is 93.78%. On average, it takes 5.9 steps and 4299 function evaluations to find the optimum in this problem. The same problems are tested on several other direct search based optimization algorithms, including SA [29], PSO [14] and OSSRS [2]. The success rate of finding global optimum and the average number of function evaluations are compared in Tables 2.2, 2.3, 2.4. All the simulations were performed using an Intel Core Duo 2.2GHz based PC, with 2GB of RAM.

Table 2.2 Comparison of optimization performances on synthetic function

RLO

SA

PSO

OSSRS

Success rate of finding the global optimum 93.78% 29.08% 94.89% 52.21% Number of function evaluations 4299 13118 4087 313 CPU time consumption [s] 28.4 254.35 20.29 1.95

2.3.1.2

Six-Hump Camel Back Function

The classic 2D six-hump camel back function [5] has 6 local minima and 2 global minima. The function is given as V (p1 , p2 ) = (4 − 2.1p21 + [−2, 2]).

p41 2 ) p1 + p1 p2 + (−4 + 4p22 ) p22 (p1 ∈ [−3, 3], p2 ∈ 3

Within the specified bounded region, the function has 2 global minima equal to 1.0316. The optimization performances of these algorithms from 300 random trials are compared in Table 2.3.

42

J.A. Starzyk, Y. Liu, and S. Batog Table 2.3 Comparison of optimization performances on six-hump camel back function

RLO

SA

PSO

OSSRS

Success rate of finding the global optimum 80.33% 45.22% 86.44% 42.67% Number of function evaluations 5016 8045.5 3971 256 CPU time consumption [s] 33.60 151.86 20.35 1.63 2.3.1.3

Banana Function

The Rosenbrock’s famous “banana function” [23], as V (p1 , p2 ) = 100(p2 − p21 )2 + (1 − p1)2 , has 1 global minimum equal to 0 lying inside a narrow, curved valley. The optimization performances of these algorithms from 300 random trials are compared in Table 2.4. Table 2.4 Comparison of optimization performances on banana function

RLO

SA

PSO

Success rate of finding the global optimum 74.55% 3.33% 41% Number of function evaluations 48883.7 28412 4168 CPU time consumption [s] 320.74 539.38 20.27

OSSRS 88.89% 882.4 5.15

In these optimization problems, RLO demonstrates consistently satisfactory performance without particular tuning of the parameters. However, other methods show different level of efficiency and capabilities of handling various problems.

2.3.2 Optimization of Weights in Multi-layer Perceptron Training The output of a multi-layer perceptron (MLP) can be looked at as the value of a function with the weights as the approximation variables. Training the MLP, in the sense of finding optimal values of weights to accomplish the learning task, can be treated as an optimization problem. We can take the Iris plant database [22] as a testing case. The Iris database contains 3 classes, 5 numerical features and 150 samples. In order to accomplish the classification of the iris samples, a 3-layered MLP with an input layer, a hidden layer and an output layer can be used. The size of the input layer should be equal to the number of features. The size of the hidden layer is chosen to be 6, and since the class IDs are numerical values equal to 1, 2 and 3, the size of the output layer is 1. The weight matrix between the input layer and the hidden layer contains 30 elements, and the one between the hidden layer and the

2

A Novel Optimization Algorithm Based on Reinforcement Learning

43

output layer contains 6 elements. Overall, there are 36 weight elements (parameters) to be optimized. In a typical trial, the optimization algorithm finds the optimal set of weights after only 3 iterations. In the testing stage, the outputs of the MLP are rounded to be the nearest integers to indicate predicted class IDs. Comparing the given class IDs and the predicted class IDs from the MLP in Fig. 2.10, it is obtained that 146 out of 150 iris samples can be correctly classified by such set of weights and the percentage of correct classification is 97.3%. A single support vector machine (SVM) achieved 96.73% classification rate [12]. In addition, a MLP with the same structure, training by back-propagation (BP) achieved 96% on Iris test case. The MLP and BP are implemented using MATLAB neural network toolbox.

Fig. 2.10 RLO performance on neural network training on Iris problem

2.3.3 Micro-saccade Optimization in Active Vision for Machine Intelligence In the area of machine intelligence, active vision becomes an interesting topic. Instead of taking in the whole scene captured by the camera and making sense of all the information in the conventional computer vision approach, active vision agent focuses on small parts of the scene and moves its fixation frequently. Human and other animals use such quick movement of both eyes, which is called saccade [3], to focus on the interesting part of the scene and efficiently use its own resources. The interesting parts are usually important features of the input, and with the important features being extracted, the high-resolution scene is analyzed and recognized with relatively small number of samples. In a saccade movement network (SMN) presented in [16], the original images are transformed into a set of low resolution images after saccade movements and retina

44

J.A. Starzyk, Y. Liu, and S. Batog

sampling. The set of images, as the sampled features, are fed to the self-organizing winner-take-all classifier (SOWTAC) network for recognition. To find interesting features of the input image and to direct the movements of saccade, image segmentation, edge detection and basic morphology tools [4] are utilized.

Fig. 2.11 Face image and its interesting features in active vision [16]

Fig. 2.11 (a) shows a face image from [7] with 320×240 pixels. The interesting features found are shown in Fig. 2.11 (b). The stars represent the center of the four interesting features found on a face image and the rectangles represent the feature boundaries. Then, the retina sampling model [16] places its fovea at the center of each interesting feature, so that these features will be extracted. Practically, the centers of the interesting features found by image processing tools [4] are not guaranteed to be the accurate centers, which will affect the accuracy of feature extraction and pattern recognition process. In order to help to find the optimum sampling position, RLO algorithm can be used to direct the move of the fovea of the retina and find the closest match between the obtained sample features and pre-stored reference sample features. These slight moves during fixation to find the optimum sampling positions can be called microsaccades in the active vision process, although the actual role of microsaccades has been a debate topic unsolved for several decades [19]. Fig. 2.12 (a) shows a group of ideal samples of important features in face recognition. Fig. 2.12 (b) shows the group of sampled features with initial sampling positions. In the optimization process, the x-y coordinates need to be optimized so that the sampled images have the optimum similarity to the ideal images. The level of similarity can be measured by the sum of squared intensity difference [9]. In this metric, increased similarity will have decreased intensity difference. Such problem can be also perceived as an image registration problem. The two-variables objective function V(x, y), the sum of squared intensity difference, needs to be minimized

2

A Novel Optimization Algorithm Based on Reinforcement Learning

45

Fig. 2.12 Image sampling by micro-saccade

through RLO algorithm. It is noted that the only information available is that V can be the function of x and y coordinates. How the function would be expressed and what are its characteristics are totally unknown. The minimum value of the objective function is not known either. RLO would be the suitable algorithm for such optimization problem. Fig. 2.12 (c) shows the optimized sampled images using RLO-directed microsaccades. The optimized feature samples are closer to the ideal feature samples, which will help the processing of the face image. After the featured images are obtained through RLO-directed microsaccades, these low-resolution images, instead of the entire high-resolution face image, are sent to the SOWTAC network for further processing or recognition.

2.4 Conclusions In this chapter, a novel and efficient optimization algorithm is presented for the problems in which the objective functions are unknown. The search particle is able to build the model of successful actions and choose its future action based on the past exploring experience. The decisions on the step sizes (and directions) are made based on a trade-off between exploitation of the known search path and exploration for the improved search direction. In this sense, this algorithm falls into a category of reinforcement learning based optimization (RLO) methods. The algorithm does not require any prior knowledge of the objective function, nor does it require any characteristics of such function. It is conceptually very simple and intuitive as well as very easy to implement and tune. The optimization algorithm was tested and verified using several multi-variable functions and compared with several other widely used random search optimization

46

J.A. Starzyk, Y. Liu, and S. Batog

algorithms. Furthermore, the training of a multi-layer perceptron (MLP), based on finding a set of optimized weights to accomplish the learning, is treated as an optimization problem. The proposed RLO was used to find the weights of MLP in the training problem on Iris database. Finally, the algorithm is used in the image recognition process to find a familiar object with retina sampling and micro-saccades. The performance of RLO, will depend to a certain degree on the values of several parameters that this algorithm uses. With certain preset parameters, the performance of RLO can meet our requirements in several machine learning problems involved in our current research. In the future research, a theoretical and systematic analysis of the effect of these parameters will be conducted. In addition, using a group of search particles and their cooperation and competition, a population based RLO can be developed. With the help of model approximation techniques and the trade-off between exploration and exploitation proposed in this work, the population based RLO is expected to have better performance.

References 1. Arfken, G.: Lagrange Multipliers, 3rd edn. §17.6 in Mathematical Methods for Physicists, pp. 945–950. Academic Press, Orlando (1985) 2. Belur, S.: A random search method for the optimization of a function of n variables. MATLAB central file exchange, http://www.mathworks.com/ matlabcentral/fileexchange/loadFile.do?objectId=100 3. Cassin, B., Solomon, S.: Dictionary of Eye Terminology. Triad Publishing Company, Gainsville (1990) 4. Detecting a Cell Using Image Segmentation. Image Processing Toolbox, the Mathworks, http://www.mathworks.com/products/image/demos.html 5. Dixon, L.C.W., Szego, G.P.: The optimization problem: An introduction. Towards Global Optimization II. North Holland, New York (1978) 6. Eelder, J.A., Mead, R.: A simplex method for function minimization. The Computer Journal 7, 308–313 (1965) 7. Facegen Modeller. Singular Inversions, http://www.facegen.com/products.htm 8. del Toro Garcia, X., Neri, F., Cascella, G.L., Salvatore, N.: A surrogate associated Hooke-Jeeves algorithm to optimize the control system of a PMSM drive. IEEE ISIE, 347–352 (July 2006) 9. Hill, D.L.G., Batchelor, P.: Registration methodology: concepts and algorithms. In: Hajnal, J.V., Hill, D.L.G., Hawkes, D.J. (eds.) Medical Image Registration. Medical Image Registration. CRC, Boca Raton (2001) 10. Hooke, R., Jeeves, T.A.: Direct search solution of numerical and statistical problems. Journal of the Association for Computing Machinery 8, 212–229 (1961) 11. Kennedy, J., Eberhart, R.C.: Particle swarm optimization. In: Proc. IEEE Int. Conf. Neural Networks, Perth, Australia, December 1995, vol. 4, pp. 1942–1948 (1995) 12. Kim, H., Pang, S., Je, H.: Support vector machine ensemble with bagging. In: Lee, S.-W., Verri, A. (eds.) SVM 2002. LNCS, vol. 2388, p. 397. Springer, Heidelberg (2002) 13. Kirkpatrick, S., Gelatt Jr., C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)

2

A Novel Optimization Algorithm Based on Reinforcement Learning

47

14. Leontitsis, A.: Hybrid Particle Swarm Optimization, MATLAB central file exchange, http://www.mathworks.com/matlabcentral/fileexchange/ loadFile.do?objectId=6497 15. Lewis, R.M., Torczon, V., Trosset, M.W.: Direct search methods: Then and now. Journal of Computational and Applied Mathematics 124(1), 191–207 (2000) 16. Li, Y.: Active Vision through Invariant Representations and Saccade Movements. Master thesis, School of Electrical Engineering and Computer Science, Ohio University (2006) 17. Liu, Y., Starzyk, J.A., Zhu, Z.: Optimized Approximation Algorithm in Neural Networks without overfitting. IEEE Trans. on Neural Networks 19(4), 983–995 (2008) 18. Lustig, I.J., Marsten, R.E., Shanno, D.F.: Computational Experience with a Primal-Dual Interior Point Method for Linear Programming. Linear Algebra and its Application 152, 191–222 (1991) 19. Martinez-Conde, S., Macknik, S.L., Hubel, D.H.: The role of fixational eye movements in visual perception. Nature Reviews Neuroscience 5(3), 229–240 (2004) 20. Ong, Y.-S.: Max-min surrogate-assisted evolutionary algorithm for robust design. IEEE Trans. on Evolutionary Computation 10(4), 392–404 (2006) 21. Powell, M.J.D.: An efficient method for finding the minimum of a function of several variables without calculating derivatives. The Computer Journal 7, 155–162 (1964) 22. Fisher, R.A.: Iris Plants Database (July 1988), http://faculty.cs.byu.edu/˜cgc/Teaching/CS_478/iris.arff 23. Rosenbrock, H.H.: An automatic method for finding the greatest or least value of a function. The Computer Journal 3, 175–184 (1960) 24. Sheela, B.V.: An optimized step-size random search. Computer Methods in Applied Mechanics and Engineering 19(1), 99–106 (1979) 25. Snyman, J.A.: Practical Mathematical Optimization: An Introduction to Basic Optimization Theory and Classical and New Gradient-Based Algorithms. Springer, Heidelberg (2005) 26. Starzyk, J.A.: Motivation in Embodied Intelligence. In: Frontiers in Robotics, Automation and Control, October 2008, pp. 83–110. I-Tech Education and Publishing (2008), http://www.intechweb.org/ book.php?%20id=78&content=subject&sid=11 27. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998) 28. Torczon, V.: On the Convergence of Pattern Search Algorithms. SIAM Journal on Optimization 17(1), 1–25 (1997) 29. Vandekerckhove, J.: General simulated annealing algorithm, MATLAB central file exchange, http://www.mathworks.com/matlabcentral/fileexchange/ loadFile.do?objectId=10548 30. Ypma, T.J.: Historical development of the Newton-Raphson method. SIAM Review 37(4), 531–551 (1995)

Chapter 3

The Use of Opposition for Decreasing Function Evaluations in Population-Based Search Mario Ventresca, Shahryar Rahnamayan, and Hamid Reza Tizhoosh

Abstract. This chapter discusses the application of opposition-based computing to reducing the amount of function calls required to perform optimization by population-based search. We provide motivation and comparison to similar, but different approaches including antithetic variates and quasi-randomness/ low-discrepancy sequences. We employ differential evolution and population-based incremental learning as optimization methods for image thresholding. Our results confirm improvements in required function calls, as well as support the oppositional princples used to attain them.

3.1 Introduction Global optimization is concerned with discovering an optimal (minimum or maximum) solution to a given problem generally within a large search space. In some instances the search space may be simple (i.e. concave or convex optimization can be used). However, most real world problems are multi-modal and deceptive [5] which often causes traditional optimization algorithms to become trapped at local optima. Many strategies have been developed to overcome this for global optimization including, but not limited to simulated annealing [9], tabu search [4], evolutionary algorithms [7] and swarm intelligence [3]. Some of these methods employ a single solution per iteration methodology whereby only one solution is generated and successively perturbed towards more Mario Ventresca · Hamid Reza Tizhoosh Department of Systems Design Engineering, The University of Waterloo, Ontario, Canada e-mail: [email protected],[email protected] Shahryar Rahnamayan Faculty of Engineering and Applied Science, The University of Ontario Institute of Technology, Ontario, Canada e-mail: [email protected]

Y. Tenne and C.-K. Goh (Eds.): Computational Intelligence in Optimization, ALO 7, pp. 49–71. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

50

M. Ventresca, S. Rahnamayan, and H.R. Tizhoosh

appropriate solutions (i.e. simulated annealing and tabu search). Single-solution methods inherently require low computational time per iteration, however they often lack sufficient diversity to adequately explore the search space. An alternative is via population-based techniques where many solutions are generated at each iteration and all (or most) are used to determine the search direction (i.e. evolutionary and swarm intelligence algorithms). By considering many solutions per iteration these methods will have a higher diversity, however will require a large amount of computation. In this chapter we focus on population-based optimization. Real-world problems also present the possible issue of complexity in the evaluation of a solution. That is, determining the quality of a given solution is computationally expensive. Investigation into simpler evaluation metrics is one possible direction, however, it may be found that evaluation is still expensive. Then, reducing the time spent (i.e. number of function evaluations) becomes an important goal. Opposition-based computing (OBC) is a newly devised concept, having as one of its aims, the improvment of convergence rates of algorithms by defining and simultaneously considering pairs of opposite solutions [24]. This advanced convergence rate is also usually accompanied by a more desirable final result. To date, OBC has shown improvements in reinforcement learning [20, 22, 23], evolutionary algorithms [14, 15, 16], ant colony optimization [12], simulated annealing [28], estimation of distribution [29], and neural networks [26, 27, 30]. In this chapter we discuss the application of OBC to reducing the number of function calls required to achieve a desired accuracy for population-based searches. We show theoretical reasoning behind OBC (which has roots in monotonic optimization) and provide mathematical motivations for its ability to reduce function calls and improve accuracy of simulation results. The key factor to accomplishing this is via simultaneous consideration of negatively associated variables and their affect on the target evaluation function and search process. We choose to highlight the improvements for the task of image thresholding using differential evolution and population-based incremental learning. The rest of this chapter is organized as follows: Section 3.2 discusses the theoretical motivations behind our approach. Differential evolution and population-based incremental learning are discussed in Section 3.3 as are their respective oppositionbased counterparts. The experimental setup is given in Section 3.4 and results are presented in Section 8.4. Conclusions are give in Section 3.6.

3.2 Theoretical Motivations In this section we introduce notations and definitions required to explain the concept of opposition and its ability to reduce the number of function calls. We also provide a brief comparison of OBC to antithetic variates and quasi-random/low-discrepancy sequences.

3

The Use of Opposition for Decreasing Function Evaluations

51

3.2.1 Definitions and Notations In the following definitions, assume that A ⊆ ℜd is non-empty, compact and ddimensional. Without loss of generality, f : A → ℜ is a continuous function to be maximized. We assume all A are feasible. The purpose of a global search method is to discover the global optima (either minimum or maximum) of a given function and not converge to one of the local optima. Definition 1 (Global Optima). A solution θ ∗ ∈ A is a global optima if f (θ ) ≤ f (θ ∗ ) for all θ ∈ A . There may exist more than one global optima. Definition 2 (Local Optima). A solution θ ∈ A is a local optima if there exists a ε -neighborhood Nε (θ ) with radius ε > 0 where g(θ , θ ) < ε for distance function g and θ ∈ A ∩ Nε (θ ), and f (θ ) ≤ f (θ ). Recent research [1, 25, 32] has shown the benefit of utilizing monotonic transformations of the evaluation criteria as a means of discovering global optima. This causes a reordering of the solutions and a gradient-based method can be used to search the reordered space. An issue with these convexification and concavification methods which transform certain functions to a monotonic form is that the mapping must be known a priori. Otherwise, optimization on the transformed function is unreliable. Definition 3 (Monotonicity). Function φ : ℜ → ℜ is monotonic if for x, y ∈ ℜ and x < (>)y, then φ (x) ≤ (≥)φ (y). A strictly monotonic function is one which does not permit equality, (i.e. φ (x) < (>)φ (y)). Theoretically, a monotonic transformation is ideal, however, OBC does not require it. Instead, opposition extends the monotonic global search idea through the use of opposites solutions, which are simultaneously considered and the more desirable (w.r.t. f and the problem definition) is used during the search. Definition 4 (Opposite). A pair (x, x) ˘ ∈ A are opposites if there exists a function Φ : A → A where Φ (x) = y and Φ (y) = x. The breve notation will be used to denote the opposite element (i.e. x˘ = Φ (x) = y). The function Φ referred to in Definition 4 is the key to employing opposition-based techniques. This determines which elements are opposites, and a poorly selected function could lead to poor performance (see the following section). Definition 5 (Opposite Mapping). A one-to-one function Φ : A → A where every pair x, x˘ ∈ A are unique (i.e. for z ∈ A , if Φ (x) = y and Φ (y) = x then there does not exist Φ (y) = z or Φ (x) = z).

Φ can be determined via prior knowledge, intuition or through some a priori or online learning procedure. Simultaneous use of the opposites (for example, a maximization problem) is easily accomplished by allowing f (θ ) = f (θ˘ ) = max( f (θ ), f (θ˘ )) and searching for a solution S which corresponds to the most desirable solution. (3.1) S = arg max f (θ ). θ

52

M. Ventresca, S. Rahnamayan, and H.R. Tizhoosh

3.2.2 Consequences of Opposition As with a monotonic transformation the “optimal” opposition map is one which effectively reorders elements of A such that φ ( f ) is monotonic. Under this trans˘ ≤ 0 for X, X˘ ⊂ A , as is shown in Figure 3.1. formation cor(X, X)

1 0.9 0.8 0.7

Evaluation

0.6 0.5

X

X

0.4 0.3 0.2 0.1 0

Solution

Fig. 3.1 Example of a transformed evaluation funtion to a monotonic function. The values in X and X˘ are negatively correlated. The original function (not shown) could have been any nonlinear, continuous function

The implementation of opposition for an optimization problem involves the simultaneous examination of x, x˘ and returns the most desirable. That is, we aim for a negative correlation between the two guesses. A consequence of Φ is an effective halving of possible evaluations (as shown in Figure 3.2) where f (θ ) = f (θ˘ ) = max( f (θ ), f (θ˘ )). The halving results from full information of the transformed function such that the opposite solution is not required to be observed to compute the max operation. In the more general case, it is sufficient to determine a function such that Pr(max( f (θ1 ), f (θ˘1 ) < max( f (θ1 ), f (θ2 ))) > 0.5, for θ1 , θ2 ∈ A . A further consequence of simultaneous consideration of opposites is a provably more desirable E[ f ] and lower variance [28]. Alone this does not guarantee a higher quality outcome; the probability density function of the opposite-transformed function values should also be more desirable than the joint p.d.f. of random sampling (i.e. the distribution corresponding to Pr(max(x1 , x2 ))).

3

The Use of Opposition for Decreasing Function Evaluations

53

1 0.9 0.8 0.7

Function representing the maximum of a solution and its opposite

Evaluation

0.6 0.5

Transformed monotonic function

0.4 0.3 0.2 0.1 0

Solution

Fig. 3.2 Taking f (x) = max( f (x), f (−x)), we see that the possible evaluations in the search space have been halved in the optimal situation of full knowledge. In the general case, the transformed function will have a more desirable mean and lower variance

While not investigated in this chapter we make the observation that successive applications of different opposite maps will lead to further smoothing of f . For example, (3.2) f 2 (θ ) = max( f (z = arg max f (θ )), f (˘z)) θ

where z˘ is determined via opposite map Φ2 = Φ1 and superscript f 2 indicates the two applications of opposite mappings. In the limit, lim f i (θ ) = max( f i (zi = arg max f i−1 (θi )), f (˘zi )) = f (θ ∗ )

i→∞

θi

(3.3)

for i > 0 and global optima f (θ ∗ ). Effectively, this flattens the entire error surface of f , except for the global optimal(s). A more feasible alternative is to use k > 0 transformations which give reasonable results where 0 ≤ | f k−1 − f k | < ε does not diminish greatly.

3.2.3 Lowering Function Evaluations We briefy discuss conditions on the design of the opposition map Φ which often lead to improvements over purely random sampling with respect to lowering function evaluations. If using an algorithm solely based on randomly generating solutions to the problem at hand then we require that for some ε > 0 and δ > 0.5

54

M. Ventresca, S. Rahnamayan, and H.R. Tizhoosh

Pr (max( f (x), f (x)) ˘ − max( f (x), f (y)) > ε ) > δ

(3.4)

where x, y, x˘ ∈ A . That is, the distribution max(x, x) ˘ must be less desirable than the distribution of i.i.d. random guesses. If this condition is met, then the probability that the optimal solution (or higher quality) is discovered is higher using opposite guesses. The goal in developing Φ is to maximize ε and δ . A similar goal is to determine Φ such that E[g(x, x)] ˘ is maximized for some distance function g. Satisifying this condition implies (3.4). Thus, probabilistically we expect a lower number of function calls to find a solution of a given thresholded quality. If employing this strategy within a guided search method the dynamics of the algorithm must be considered to assure the guarantee (i.e. the algorithm adds bias to the search which affects the convergence rate). Practically, the simplest manner to decide a satisfactory Φ is through intuition or prior knowledge about f . A possibility is to utilize modified low-discrepancy sequences (see below), which aim to distribute guesses evenly throughout the search space.

3.2.4 Comparison to Existing Methods While opposition may sometimes employ methods from antithetic variates and lowdiscrepancy sequences, in general that is not the case. To elucidate the uniqueness of opposition in the following we distinguish it from these two methods. 3.2.4.1

Antithetic Variates

Suppose we desire to estimate ξ = E[ f ] = E[Y1 ,Y2 ] with unbiased estimator Y1 + Y2 ξˆ = . 2

(3.5)

If Y1 ,Y2 are i.i.d. then var(ξˆ ) = var(Y1 +Y2 )/2. However, if cov(Y1 ,Y2 ) < 0 then the variance can be further reduced. One method to accomplish this is through the use of a monotonic function h. Then, generate Y1 as an i.i.d. value as before, but utilizing h our two variables are h(Y1 ) and h(1 − Y1), which are monotonic over interval [0,1]. And h(Y1 ) + h(Y2 ) ξˆ = 2

(3.6)

will be an unbiased estimator of E[ f ]. Opposition is similar in its selection of negatively correlated samples. However, in the antithetic approach there is no guideline to construct such a monotonic function, although such a function has been proven to exist [17]. Opposition provides various means to accomplish this, as well as to incorporate the idea into optimization while guaranteeing more desirable expected values and lower variance in the target function.

3

The Use of Opposition for Decreasing Function Evaluations

55

Further, opposition extends beyond the generation of solutions in random sampling-based algorithms. It can also be applied to algorithm behavior and can be used to relate concepts expressed linguistically, where we have not found evidence that antithetic variates have application. 3.2.4.2

Quasi-Randomness and Low-Discrepancy Sequences

These methods aim to combine the randomness from pseudorandom generators which select values i.i.d. with the advantages of generating points distributed in a grid-like fashion. The goal is to uniformly distribute values over the search space by achieving a low-discrepancy. However, this is achieved at the cost of statistical independence. The discrepancy of a sequence is a measure of its uniformity and can be calculated via [6]:    |B ∩ X|   − λd (B) DN (X) = sup  (3.7) N I∈J where λd is the d-dimensional Lebesque measure, |B ∩ X| is the number of points of X = (x1 , ..., xN ) that fall into interval I, and J is the set of d-dimensional intervals defined as: d

∏[ai, bi ) = {x ∈ ℜd : ai ≤ xi ≤ bi }

(3.8)

i=1

for 0 ≤ ai < bi ≤ 1. That is, the actual number of points within each interval for a given sample is close to the expected number. Such sequences have been widely studied in quasiMonte Carlo methods [10]. Opposition may utilize low-discrepancy sequences in some situations. Though in general, low-discrepancy sequences are simply a means for attaining uniform distribution without regard to the correlation between the evaluations at these points. Further, opposition-based techniques simultaneously consider two points in order to smooth the evaluation function and improve performance of the sampling algorithm whereas quasi-random sequences often are concerned with many more points. These methods have been applied to evolutionary algorithms where it was found that by a performance study of the different sampling methods such as Uniform, Normal, Halton, Sobol, Faure, and Low-Discrepancy is valuable only for low-dimensional (d < 10 and so non-highly-sparse) populations [11].

3.3 Algorithms In this section we describe Differential Evolution (DE) and Population-¿ It seems, performance study of the different sampling methods such as Uniform, Normal, Halton, Sobol, Faure, and Low-Discrepancy [11] is valuable only for low-dimensional (D < 10 and so non-highly-sparse) populations. Learning (PBIL), which are the

56

M. Ventresca, S. Rahnamayan, and H.R. Tizhoosh

parent algorithms for this study. We also describe the oppositional variants of these methods.

3.3.1 Differential Evolution Differential Evolution (DE) was proposed by Price and Storn in 1995 [21]. It is an effective, robust, and simple global optimization algorithm [8]. DE is a populationbased directed search method. Like other evolutionary algorithms, it starts with an initial population vector. If no preliminary knowledge about the solution space is available the population is randomly generated. Each vector of the initial population can be generated as follows [8]: Xi, j = a j + rand j (0, 1) × (a j − b j ); j = 1, 2, ..., d,

(3.9)

where d is the problem dimension; a j and b j are the lower and the upper boundaries of the variable j, respectively. rand(0, 1) is the uniformly generated random number in [0, 1]. Assume Xi,t (i = 1, 2, ..., N p ) are candidate solution vectors in generation t and N p : is the population size. Successive populations are generated by adding the weighted difference of two randomly selected vectors to a third randomly selected vector. For classical DE (DE/rand/1/bin), the mutation, crossover, and selection operators are straightforwardly defined as follows: Mutation: For each vector Xi,t in generation t a mutant vector Vi,t is defined by Vi,t = Xa,t + F(Xc,t − Xb,t ),

(3.10)

where i = {1, 2, ..., N p } and a, b, and c are mutually different random integer indices selected from {1, 2, ..., N p }. Further, i, a, b, and c are unique such that it is necessary for N p ≥ 4. F ∈ [0, 2] is a real constant which determines the amplification of the added differential variation of Xc,t − Xb,t . Larger values for F result in higher diversity in the generated population and lower values lead to faster convergence. Crossover: By shuffling competing solution vectors DE utilizes the crossover operation to generate new solutions and also to increase the population diversity. For the classical DE (DE/rand/1/bin), the binary crossover (shown by ‘bin’ in the notation) is utilized. It defines the following trial vector: Ui,t = (U1i,t ,U2i,t , ...,Udi,t ), where, U ji,t

 V ji,t = X ji,t

if rand j (0, 1) ≤ Cr ∨ j = k, . otherwise,

(3.11)

(3.12)

for Cr ∈ (0, 1) the predefined crossover rate, and rand j (0, 1) is the jth evaluation of a uniform random number generator. k ∈ {1, 2, ..., d} is a random parameter index,

3

The Use of Opposition for Decreasing Function Evaluations

57

chosen once for each i to make sure that at least one parameter is always selected from the mutated vector, V ji,t . Most popular values for Cr are in the range of (0.4, 1) [14]. Selection: This decides which vector (Ui,t or Xi,t ) should be a member of next (new) generation, t + 1. For a minimization problem, the vector with the lower value of objective function is chosen (greedy selection). This evolutionary cycle (i.e., mutation, crossover, and selection) is repeated N p (population size) times to generate a new population. These successive generations are produced until meeting the predefined termination criteria.

3.3.2 Opposition-Based Differential Evolution By utilizing opposite points, we can obtain fitter starting candidate solutions even when there is no a priori knowledge about the solution(s) according to: 1. Random initialization of population P(NP ), 2. Calculate opposite population by OPi, j = a j + b j − Pi, j ,

(3.13)

i = 1, 2, ..., N p ; j = 1, 2, ..., D, where Pi, j and OPi, j denote jth variable of the ith vector of the population and the opposite-population, respectively. 3. Selecting the N p fittest individuals from {P ∪ OP} as the initial population. The general ODE scheme also employs generation jumping, but it has not be used in this work in lieu of only population-initialization and sample generation.

3.3.3 Population-Based Incremental Learning PBIL is a stochastic search which abstracts the population of samples found in evolutionary computation with a probability distribution for each variable of the solution [2]. At each generation a new sample population is generated based on the current probability distribution. The best individual is retained and the probability model is updated accordingly to reflect the belief regarding the final solution. The update rule is similar to that found in reinforcement learning. A population is represented by probability matrix M := (mi, j )d×c which stores the probability distribution over each possible element in the solution. If considering a binary problem then solution S := (si, j )d×c ∈ {0, 1} and mi, j ∈ [0, 1] is the probability element si, j = 1. For continuous problems probability distributions are used instead of a threshold value as is the case for discrete problems [18]. Learning consists of utilizing M to generate population P of k samples. After evaluation of each sample according to function f the “best” (B∗ ) solution is retained and M is updated according to

58

M. Ventresca, S. Rahnamayan, and H.R. Tizhoosh

Mt = (1 − α )Mt−1 + α B∗

(3.14)

where 0 < α < 1 is the learning rate and t ≥ 1 is the iteration. Initially, mi, j = 0.5 to reflect that lack of prior information. To abstract the crossover and mutation operators of evolutionary computation, PBIL employs a randomization of M. Let 0 < β , γ < 1 be the probability of mutation and degree of mutation, respectively. Then with probability β mi, j = (1 − γ )mi, j + γ · random(0 or 1).

(3.15)

Algorithm 1 provides a summary of this approach.

Algorithm 1. Population-Based Incremental Learning [2] 1: {Initialize probabilities} 2: M0 := (mi, j ) = 0.5 3: for t = 1 to ω do 4: {Generate samples} 5: G1 = generate samples(k,Mt−1 ) 6: 7:

{Find best sample} B∗ = select best({B∗ } ∪ G1)

8: 9:

{Update M} Mt = (1 − α )Mt−1 + α B∗

10: {Mutate probability vector} 11: for i = 0...d and j = 0...c do 12: if random(0, 1) < β then 13: mi, j = (1 − γ )mi, j + γ · random(0 or 1) 14: end if 15: end for 16: end for

3.3.4 Oppositional Population-Based Incremental Learning The opposition-based version of PBIL (OPBIL) shown in Algorithm 2 employs the opposite concept to improve diversity within the sample generation phase. A direct effect on convergence rate is observed as a consequence of this mechanism. Further, OPBIL has an ability to escape local optima which estimation of distribution algorithms such as PBIL are prone to becoming trapped on [19]. The description provided here is brief and the interested reader is invited to read [29] for a more detailed description. The general structure of the PBIL algorithm remains, however aside from the sampling procedure the update and mutation rules are altered to reflect a degrading

3

The Use of Opposition for Decreasing Function Evaluations

59

degree of opposition with respect to the number of iterations. That is, as the number of iterations t → ∞ the amount two opposite solutions differ approaches 1 bit (w.r.t. Hamming distance). Sampling is accompished using an opposite guessing strategy whereby half of the population R1 is generated using probability matrix M and the other half is generated via a change in Hamming distance to a given element of R1 . The distance is calculated using an exponentially decaying function in the flavor of

ξ (t) = le(ct) ,

(3.16)

where l is the maximum number of bits in a guess and c < 0 is a user defined constant. Updating of M is performed in lines 14-28. If a new global best solution has been discovered (i.e. η = B∗ ), or with probability pamp the sample best solution is used to focus the search, respectively. When no new optima have been discovered this strategy tends away from B∗ . The actual update is performed in line 16 and is based on a reinforcement learning update using the sample best solution. The degree to which M is updated is controlled by the user defined parameter 0 < ρ < 1. Should the above criteria for update fail, a decay of M with probability pdecay is attempted in line 17. The decay, performed in lines 21-27 slowly tends M away from B∗ . This portion of the algorithm has the intention to prevent convergence an aide in the exploration ability through small, smooth updates. Parameter 0 < τ < 1 is user defined where often τ  ρ . Equations in lines 11 and 12 were determined experimentally and no argument regarding their optimality is provided. Indeed, there likely exists many functions which will yield more desirable results. These have been decided because they tend to lead to a good behavior and outcome of PBIL.

3.4 Experimental Setup In this section we provide a discussion of the image thresholding problem, and the application of evolutionary algorithms to solving it. Additionally, the evaluation measure we employ to grade the quality of a segmentation is presented. Parameter settings and problem representation are also given.

3.4.1 Evolutionary Image Thresholding Image segmentation involves partitioning an image I into a set of segments with the goal of locating objects in the image which are sufficiently similar. Thresholding is a subset problem of image segmentation, with only 2 classes defined by whether a given pixel is above or below a specific threshold value ω . This task has numerous applications and several general segmentation algorithms have been proposed [33]. Due to the variety of image types there does not exist a single algorithm for segmenting all images optimally.

60

M. Ventresca, S. Rahnamayan, and H.R. Tizhoosh

Algorithm 2. Pseudocode for the OPBIL algorithm Require: Maximum iterations, ω Require: Number of samples per iteration, k 1: {Initialize probabilities} 2: M0 = mi..l = 0.5 3: for t = 1 to ω do 4: {Generate samples} 5: R1 = generate samples(k/2,M) 6: R˘ 1 = generate opposites(R1) 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24: 25: 26: 27: 28: 29: 30:

{Find best sample} η = select best({R1 ∪ R˘ 1 }) B∗ = select best(B∗ , η ) {Compute probabilities} pamp (Δ ) = 1 − e−bΔ pdecay (Δ ;

f (B∗ ),

∗

)− f (η ) 1 − f (Bf (B ∗) √ f (η )) = Δ +1

{Update M} if η = B∗ OR random(0, 1) < pamp then Δ =0 Mt = (1 − ρ )Mt−1 + ρη else if random(0, 1) < pdecay then if random(0, 1) < pdecay then use B∗ in line 23 instead of η end if for all i, j each with probability < pdecay do if ηi, j = B∗i, j then  1 − τ · random(0, 1), if ηi, j = 1, mi, j = mi, j · 1 + τ · random(0, 1), otherwise else  1 + τ · random(0, 1), if ηi, j = 1, mi, j = mi, j · 1 − τ · random(0, 1), otherwise end if end for end if Δ = Δ +1 end for

3

The Use of Opposition for Decreasing Function Evaluations

61

Many general purpose segmentation algorithms are histogram based and aim to discover a deep valley between two peaks, and setting ω equal to that value. However, many real world problems will have multimodal histograms and deciding which value (i.e. valley) will correspond to the best thresholing is not obvious. The difficulty is compounded by the fact that the relative size of peaks may be large (and the valley becomes hard to distinguish) or valleys could be very broad. Several algorithms have been proposed to overcome this [33]. Other methods based on information theory and other statistical methods have been proposed as well [13]. Typically, the problem of segmentation involves a high degree of uncertainty which makes solving the problem difficult. Stochastic searches such as evolutionary algorithms and population-based incremental learning often cope well with uncertainty in optimization, hence they provide an interesting alternative approach to traditional methods. The main difficulty associated with the use of population-based methods is that they are computationally expensive due to the large amount of function calls required during the optimization process. One approach to minimizing uncertainty is by splitting the image into subimages which (hopefully) have characteristics allowing for an easy segmentation. Combining the subimages together then forms the entire segmented image, although this requires extra function calls to analyze each subimage. An important caveat is that the local image may represent a good segmentation, but may not be useful with respect to the image as a whole. In this chaper we investigate thresholding with population-based techniques. Using ODE we do not perform any splitting into subimages and for OPBIL we split I into four equal-sized subregions, each having it’s own threshold value. In both cases we require a single evaluation to perform the segmentation and we show that the opposition-based techniques reduce the required number of function calls. As stated above, there exist many different segmentation algorithms. Further, numerous methods for evaluating the quality of a seqmentation have also been put forth [34]. In this paper we use a simple method which aims to minimize the discrepancy between the original M × N gray-level image I and its thresholded image T [31]: M

N

∑ ∑ |Ii, j − Ti, j |

(3.17)

i=1 j=1

where | · | represents the absolute value operation. Using different evaluations will change the outcome of the algorithm, however, the problem of segmentation in this manner nonetheless remains computationally expensive. We use the images shown in Figure 3.3 to evaluate the algorithms. The first column represents the original image, then the gold and the third column corresponds to the approximate target image for ODE and OPBIL (i.e. the value-to-reach targets, discussed below). We show the gold image for completeness, it is not required in the experiments. Both experiments employ a value-to-reach (VRT) stopping criteria which measures the time or function calls required to reach a specific value. The VTR values have been experimentally determined and are given in the following table. Due to

62

M. Ventresca, S. Rahnamayan, and H.R. Tizhoosh

Fig. 3.3 The images used to benchmark the algorithms. The first column is the original graylevel image, the second is the gold and the third colum is the target image of the optimization within the required function calls

3

The Use of Opposition for Decreasing Function Evaluations

63

the respective algorithm ability of solving this problem, given the representation and bahavior of convergence these values differ for ODE and OPBIL.

Table 3.1 Value-to-reach (VRT) for O/DE and O/PBIL experiments

Image 1 2 3 4 5 6

O/DE O/PBIL 19579 19850 3391 4925 7139 7175 19449 19850 19650 19700 22081 22700

3.4.2 Parameter Settings and Solution Representation The ODE and OPBIL algorithms differ in that the former is a real optimization algorithm and the latter operates in the binary domain. Therefore, the solution representations will also differ and consequently directly affect the quality of results. However, the goal of this chapter is to show the ability of opposition to decrease the required number of function evaluations, and so fine-tuning aspects these algorithms is not the focus of this investigation. ODE Settings The differential evolution experiments follow standard enconding guidelines. The user-defined parameter settings were determined empirically as shown in Table 3.2.

Table 3.2 Parameter settings for differential evolution-based experiments

Parameter Value Population size Np = 5 Amplification factor F = 0.9 Crossover probability Cr = 0.9 Mutation strategy DE/rand/1/bin Maximum function calls MAXNFC = 200 Jumpring rate (no jumping) Jr = −1

In order to maintain a reliable and fair comparison, these settings are kept unchanged for all conducted experiments for both DE and ODE algorithms.

64

M. Ventresca, S. Rahnamayan, and H.R. Tizhoosh

OPBIL Settings As stated above, OPBIL requires a binary solution representation. However, thresholding aims to discover an integer value 0 ≤ T ≤ 255, to perform the segmentation operation of I > T . Additionally, we use an approach of splitting I into subimages I1,...,16 where each Ii is an equal sized square region of the original image. Encoding was determined to be a matrix R := (ri, j )16×8 which corresponds to 16 subimages having a gray-level value < 28 = 256. Each row of R is converted to an integer which is used to segment the respective region of I. The extra regions increase problem difficulty as they result in more deceptive and multimodal problems. Parameter settings for PBIL and OPBIL are as follows:

Table 3.3 Parameter settings for population-based incremental learning experiments

Parameter Value Maximum iterations t = 150 Sample size k = 24 PBIL Only Learning rate α = 0.35 Mutation probability β = 0.15 Mutation degree γ = 0.25 OPBIL Only Update frequency control b = 0.1 Learning rate ρ = 0.25 Probability decay τ = 0.0005

3.5 Experimental Results Using the test images and parameter settings stated above we show the ability of oppositional concepts to decrease the required function calls. Only the results for OPBIL are more detailed due to space limitations, but similar behavior should be observed in ODE. All results presented correspond to the average of 30 runs, unless otherwise noted.

3.5.1 ODE Table 3.4 presents a summary of the results obtained regarding function calls for ODE versus DE. Except for image 2, we show an decrease in function calls for all images. Images 4 and 5 have statistically significant improvements with respect to the decreased number of function calls, using a t-test at 0.9 confidence level. Further except for image 6, we show a lower standard deviation which indicates higher reliability in the results.

3

The Use of Opposition for Decreasing Function Evaluations

65

Computing the overall mean function calls we show an improvement of 322277=45 function calls. This equates to an average of 45/6 = 7.5 saved function calls per image. This implies a savings of 322/277 ≈ 1.16, indicating approximately 16% less function calls. For expensive optimization problems this can correspond to a great amount of savings with respect to algorithm run-time. Table 3.4 Summary results for DE vs. ODE with respect to required function calls. μ and σ correspond to the mean and standard deviation of the subscripted algorithm, respectively

Image 1 2 3 4 5 6 Total

μDE 74 32 42 74 47 63 322

σDE 41 20 23 36 37 26

μODE 60 35 37 54 45 46 277

σODE 34 16 19 30 22 31

3.5.2 OPBIL Table 3.5 shows the expected number of iterations (each iteration has 24 function calls) to attain the value-to-reach given in Table 3.1. In all cases OPBIL reaches its goal in fewer iterations that PBIL, where results for images 2,5,6 are found to be statistically significant using a t-test at a 0.9 confidence interval. Additionally, in all cases we find a lower standard deviation indicating a more reliable behavior for OPBIL. Overall we have 444-347=97 saved iterations using OPBIL, which is an average of 16*24=384 function calls per image. The approximate savings is 444/347 ≈ 1.28 which is about a 28% improvement in required iterations.

Table 3.5 Summary results for PBIL vs. OPBIL with respect to required iterations calls. μ and σ correspond to the mean and standard deviation of the subscripted algorithm, respectively

Image μPBIL σPBIL μOPBIL σOPBIL 1 62 19 53 12 2 80 25 65 9 3 61 12 60 5 4 47 14 40 10 5 68 13 53 9 6 128 21 76 14 Total 444 347

66

M. Ventresca, S. Rahnamayan, and H.R. Tizhoosh

In the following we analyze the correlation and distance for each sample per iteration. This is to examine whether the negative correlation and larger distance properties between a guess and its opposite are found in the sample. If true, we have supported (although not confirmed) the hypothesis that the observed improvements can be due to these characteristics. Figure 3.4 shows the averaged correlation cor(Rt1 , R˘ 1 ), for randomly generated t R1 and respective opposites R˘t1 at iteration t. The solid line corresponds to OPBIL and the dotted line is PBIL, respectively. These plots show that OPBIL indeed has a lower correlation (with respect to evaluation function) than PBIL (where we generate R1 as above, and let R˘ 1 also be randomly generated). In all cases the correlation is much stronger for PBIL (noting that if the algorithm reaches the VTR then we set the correlation to 0).

Image 2 1

0.8

0.8 correlation

correlation

Image 1 1

0.6 0.4 0.2 0

0.6 0.4 0.2 0

0

50

100

150

0

50

1

1

0.8

0.8

0.6 0.4 0.2 0

150

100

150

100

150

0.6 0.4 0.2 0

0

50

100

150

0

50

iterations Image 5

iterations Image 6

1

1

0.8

0.8 correlation

correlation

100 iterations Image 4

correlation

correlation

iterations Image 3

0.6 0.4 0.2 0

0.6 0.4 0.2 0

0

50

100 iterations

150

0

50 iterations

Fig. 3.4 Sample mean correlation over 30 trials for PBIL (dotted) versus OPBIL (solid). We find OPBIL indeed yields a lower correlation than PBIL

3

The Use of Opposition for Decreasing Function Evaluations

67

We also examine the mean distance, g¯ =

2 k/2 ∑ g(Rt1,i , R˘t1,i ) k i=1

(3.18)

which computes the fitness-distance between the ith guess R1,i and its opposite R˘ 1,i at iteration t, which is shown in Figure 3.5. The distance for PBIL is relatively low throughout the 150 iterations, gently decreasing as the algorithm converges. However, as a consequence of OPBIL’s ability to mainain diversity the distances between samples increases during the early stanges of the search and similarily rapidly decreases. Indeed, this implies the lower correlation shown above.

Image 1

Image 2

4000

1500

distance

distance

3000 2000 1000 0

0

50

100

1000

500

0

150

0

50

100 iterations Image 4

150

0

50

100 iterations Image 6

150

0

50

100 iterations

150

iteration Image 3 4000 3000

2000

distance

distance

3000

1000

0

1000

0

50

100 iterations Image 5

0

150

4000 3000

4000

distance

distance

6000

2000

0

2000

2000 1000

0

50

100 iterations

150

0

Fig. 3.5 Sample mean distance over 30 trials for samples of PBIL (dotted) versus OPBIL (solid). We find OPBIL indeed yields a higher distance between paired samples than PBIL

68

M. Ventresca, S. Rahnamayan, and H.R. Tizhoosh

The final test is to examine the standard deviation (w.r.t. evaluation function) of the distance between samples, given in Figure 3.6. Both algorithms have similarly formed plots with respect to this measure, reflecting the convergence rate of the respective algorithms. It seems as though the use of opposition aides by infusing diversity early during the search and quickly focuses once a high quality optima is found. Conversely, the basic PBIL does not include this bias, therefore convergence is less rapid.

Image 1

Image 2

2000

1500

std. dev.

std. dev.

1500 1000

1000

500

500 0

0

50

100

0

150

0

50

iterations Image 3 2000

std. dev.

std. dev.

1000 500

0

50

100

150

100

150

500

0

150

0

50 iterations Image 6

3000

1500

2000

1000

std. dev.

std. dev.

100

1000

iterations Image 5

1000

0

150

1500

1500

0

100 iterations Image 4

0

50

100 iterations

150

500

0

0

50 iterations

Fig. 3.6 Sample mean standard deviations over 30 trials for samples of PBIL (dotted) versus OPBIL (solid)

3.6 Conclusion In this chapter we have discussed the application of opposition-based computing techniques to reducing the required number of function calls. Firstly, a brief

3

The Use of Opposition for Decreasing Function Evaluations

69

introduction to the underlying concepts of opposition were given, along with conditions under which opposition-based methods should be successful. A comparison to similar concepts of antithetic variates and quasi-random/low-discrepancy sequences made obvious the uniqueness of our method. Two recently proposed algorithms, ODE and OPBIL were briefly introduced and the manner in which opposition is used to improve their parent algorithms, DE and PBIL was given, respectively. The manner in which opposites are used in both cases differed, but the underlying concepts are the same. Using the expensive optimization problem of image thresholding as a benchmark, we examine the ability of ODE and PBIL to lower the required function calls to reach the pre-specified target value. It was found that both algorithms reduce the expected number of function calls, ODE by approximately 16% (function calls) and OPBIL by 28% (iterations), respectively. Further, concentrating on OPBIL, we show the hypothesized lower correlation and higher fitness-distance measures for a quality opposite mapping. Our results are very promising, however their requires future work in various regards. Firstly, a further theoretical basis for opposition and choosing opposite mappings is needed. This could possibly lead to general strategies of implementation when no prior knowledge is available. Further application to different real-world problems is also desired.

Acknowledgements This work has been partially supported by Natural Sciences and Engineering Research Council of Canada (NSERC).

References 1. Bai, F., Wu, Z.: A novel monotonization transformation for some classes of global optimization problems. Asia-Pacific Journal of Operational Research 23(3), 371–392 (2006) 2. Baluja, S.: Population Based Incremental Learning - A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning. Tech. rep., Carnegie Mellon University, CMU-CS-94-163 (1994) 3. Engelbrecht, A.: Fundamentals of Computational Swarm Intelligence. Wiley, Chichester (2005) 4. Glover, F., Laguna, M.: Tabu Search. Kluwer, Dordrecht (1997) 5. Goldberg, D.E., Horn, J., Deb, K.: What makes a problem hard for a classifier system? Tech. rep. In: Collected Abstracts for the First International Workshop on Learning Classifier Systems (IWLCS 1992), NASA Johnson Space (1992) 6. Niederreiter, H.: Random Number Generation and Quasi-Monte Carlo Methods. Society for Industrial and Applied Mathematics (1992) 7. Holland, J.H.: Adaptation in Natural and Artificial Systems. The University of Michigan Press (1975) 8. Price, K., Storn, R., Lampinen, J.A.: Differential Evolution: A Practical Approach to Global Optimization. Springer, Heidelberg (2005)

70

M. Ventresca, S. Rahnamayan, and H.R. Tizhoosh

9. Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by Simulated Annealing. Science 220(4598), 671–680 (1983) 10. Lemieux, C.: Monte Carlo and Quasi-Monte Carlo Sampling. Springer, Heidelberg (2009) 11. Maaranen, H., Miettinen, K., Penttinen, A.: On intitial populations of genetic algorithms for continuous optimization problems. Journal of Global Optimization 37(3), 405–436 (2007) 12. Montgomery, J., Randall, M.: Anti-pheromone as a tool for better exploration of search space. In: Third International Workshop, ANTS, pp. 1–3 (2002) 13. O’Gormam, L., Sammon, M., Seul, M. (eds.): Practical Algorithms for Image Analysis. Cambridge University Press, Cambridge (2008) 14. Rahnamayan, S., Tizhoosh, H.R., Salama, M.M.A.: Opposition-based differential evolution. IEEE Transactions on Evolutionary Computation 12(1), 64–79 (2008) 15. Rahnamayn, S., Tizhoosh, H.R., Salama, S.: Opposition-based Differential Evolution Algorithms. In: IEEE Congress on Evolutionary Computation, pp. 7363–7370 (2006) 16. Rahnamayn, S., Tizhoosh, H.R., Salama, S.: Opposition-based Differential Evolution Algorithms for Optimization of Noisy Problems. In: IEEE Congress on Evolutionary Computation, pp. 6756–6763 (2006) 17. Rubinstein, R.: Monte Carlo Optimization, Simulation and Sensitivity of Queueing Networks. Wiley, Chichester (1986) 18. Sebag, M., Ducoulombier, A.: Extending Population-Based Incremental Learning to Continuous Search Spaces. In: Eiben, A.E., B¨ack, T., Schoenauer, M., Schwefel, H.-P. (eds.) PPSN 1998. LNCS, vol. 1498, pp. 418–427. Springer, Heidelberg (1998) 19. Shapiro, J.: Diversity loss in general estimation of distribution algorithms. In: Parallel Problem Solving in Nature IX, pp. 92–101 (2006) 20. Shokri, M., Tizhoosh, H.R., Kamel, M.: Opposition-based Q(lambda) Algorithm. In: IEEE International Joint Conference on Neural Networks, pp. 646–653 (2006) 21. Storn, R., Price, K.: Differential evolution- a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization 11, 341–359 (1997) 22. Tizhoosh, H.R.: Reinforcement Learning Based on Actions and Opposite Actions. In: International Conference on Artificial Intelligence and Machine Learning (2005) 23. Tizhoosh, H.R.: Opposition-based Reinforcement Learning. Journal of Advanced Computational Intelligence and Intelligent Informatics 10(4), 578–585 (2006) 24. Tizhoosh, H.R., Ventresca, M. (eds.): Oppositional Concepts in Computational Intelligence. Springer, Heidelberg (2008) 25. Toh, K.: Global optimization by monotonic transformation. Computational Optimization and Applications 23(1), 77–99 (2002) 26. Ventresca, M., Tizhoosh, H.R.: Improving the Convergence of Backpropagation by Opposite Transfer Functions. In: IEEE International Joint Conference on Neural Networks, pp. 9527–9534 (2006) 27. Ventresca, M., Tizhoosh, H.R.: Opposite Transfer Functions and Backpropagation Through Time. In: IEEE Symposium on Foundations of Computational Intelligence, pp. 570–577 (2007) 28. Ventresca, M., Tizhoosh, H.R.: Simulated Annealing with Opposite Neighbors. In: IEEE Symposium on Foundations of Computational Intelligence, pp. 186–192 (2007) 29. Ventresca, M., Tizhoosh, H.R.: A diversity maintaining population-based incremental learning algorithm. Information Sciences 178(21), 4038–4056 (2008)

3

The Use of Opposition for Decreasing Function Evaluations

71

30. Ventresca, M., Tizhoosh, H.R.: Numerical condition of feedforward networks with opposite transfer functions. In: IEEE International Joint Conference on Neural Networks, pp. 3232–3239 (2008) 31. Weszka, J., Rosenfeld, A.: Threshold evaluation techniques. IEEE Transactions on Systems, Man and Cybernetics 8(8), 622–629 (1978) 32. Wu, Z., Bai, F., Zhang, L.: Convexification and concavification for a general class of global optimization problems. Journal of Global Optimization 31(1), 45–60 (2005) 33. Yoo, T. (ed.): Insight into Images: Principles and Practice for Segmentation, Registration, and Image Analysis. AK Peters (2004) 34. Zhang, H., Fritts, J., Goldman, S.: Image segmentation evaluation: A survey of unsupervised methods. Computer Vision and Image Understanding 110, 260–280 (2008)

Chapter 4

Search Procedure Exploiting Locally Regularized Objective Approximation: A Convergence Theorem for Direct Search Algorithms Marek Bazan

Abstract. The Search Procedure Exploiting Locally Regularized Objective Approximation is a method to speed-up local optimization processes in which the objective function evaluation is expensive. It was introduced in [1] and further developed in [2]. In this paper we present the convergence theorem of the method. The theorem is proved for the EXTREM [6] algorithm but applies to any Gauss-Siedle algorithm that uses sequential quadratic interpolation (SQI) as a line search method. After some extension it can also be applied to conjugate direction algorithms. The proof is based on the Zangwill theory of closed transformations. This method of the proof was chosen instead of sufficient decrease approach since the crucial element of the presented proof is an extension of the SQI convergence proof from [14] which is based on this approach.

4.1 Introduction Optimization processes with objective functions that are expensive to evaluate – since usually their evaluation requires to solve a large system of linear equations or to simulate some physical process – occur in many fields of modern design. The main strategy in speeding up such processes via constructing a model to approximate an objective function are trust region methods [4]. The application of radial basis function approximation as an approximation model in trust region methods was discussed in [13]. The standard method to prove a convergence of a trust region method is the method of sufficient decrease. In [1] and [2] we presented the search procedure which can be viewed as an alternative to trust region methods. It relies on combining the direct search algorithm EXTREM [6] with the locally regularized radial basis approximation. The Marek Bazan Institute of Informatics, Automatics and Robotics, Department of Electronics, Wrocław University of Technology, ul. Janiszewskiego 11/17, 50-372 Wrocław, Poland e-mail: [email protected]

Y. Tenne and C.-K. Goh (Eds.): Computational Intelligence in Optimization, ALO 7, pp. 73–103. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

74

M. Bazan

method achieves a speed-up of the optimization by exchanging some number of direct function evaluations by radial basis approximation. The method combined with the EXTREM algorithm was implemented within the computer program ROXIE for superconducting magnets design and optimization [3], however it is a general framework and as we shall see, any search algorithm that is based on the GaussSeidle non-gradient algorithm or conjugate direction algorithm, can be used. We will call our method the Search Procedure Exploiting Locally Regularized Objective Approximation (SPELROA). In this paper we give the convergence theorem for the SPELROA method, combined with the EXTREM minimization algorithm under the assumption that the radial basis approximation has a realtive error less than ε . The method of proof is different from those used in proofs of trust region methods since it is based on the Zangwill theory of closed transformations. The crucial element of the proof is the modification of the proof of the convergence of the quadratic interpolation as a line search method (c.f. [14]) under the assumption that a pertubation of the function value may be introduced to the algorithm at each step. We give the conditions on the function values as well as on ε to maintain convergence. The proof of the convergence of the sequential quadratic interpolation in [14] is based on Zangwill’s theory and as we essentially extend it we chose this method also to prove the convergence of the whole SPELROA method. The plan of this chapter is the following. In the next section we sketch the SPELROA method. In the third section we give some theory from [21] to be used in the next section to prove the main result. Finally in the fourth section we discuss the radial basis approximation and heuristics used for its construction. We also describe difficulties in establishing strict error bound in the current state of the development of radial basis function approximation for sparse data. In the reminder of the paper we give numerical results for three test functions of 6,8 and 11 variables from a set of test functions proposed in [12].

4.2 The Search Procedure Let there be given a direct search optimization algorithm A that uses the quadratic interpolation as a line search method. The SPELROA method combined with algorithm A can be written in the form of the following algorithm (c.f. [2]). While generating the set Z we have to take care that data points are not placed too close to each other. When two points are too close to each other – where a distance is controlled by a user-supplied parameter whose value is relative to the diameter of the set Z – one of the points has to be replaced by another point not yet included. Such procedure of constructing Z keeps the separation distance (c.f. [16]) greater than the user-supplied parameter value and therefore guarantees that the radial basis function interpolation matrix is not singular. The crucial step of the scheme is point 3, containing a threefold check for whether the approximation f˜(xk ) can be used in the algorithm A instead of f (xk ) evaluated directly. Conditions being checked in steps 3.a) and 3.b) are related to the radial basis approximation and will be discussed

4

Search Procedure Exploiting Locally Regularized Objective Approximation

75

1 2

3

The Search Procedure exploiting Locally Regularized Objective Approximation Input : f : Rd → R – the objective function, x 0 ∈ Rd – a starting point, ε >0 – prescribed accuracy of the approximation, f˜(·) – a radial basis function approximation of f (·), Is – number of initial steps performed by the direct optimization algorithm A, N < Is – size of the dataset to construct the approximation f˜(·),. ε -check – a procedure to check conditions required by the convergence theorem to hold. 0. Perform Is initial steps of the algorithm A. 1. In the k-th step generate point xk for which the function value is supposed to be evaluated using algorithm A. 2. Generate a set Z from N nearest points for which function was evaluated directly. 3. Judge, whether for point xk a reliable approximation of f (xk ) can be constructed. a. If point xk is located in a reliable region of the search domain then construct the approximation f˜ and evaluate f˜(xk ). b. If the approximation f˜(xk ) was correctly constructed then perform an ε -check. c. If the ε -check is positive (i.e. the procedure returns true) then substitute f (xk ) ← f˜(xk ) to the algorithm A. d. Else evaluate f (xk ) directly. 4. If the stopping criterion of algorithm A is satisfied then stop. Else replace k := k + 1 and go to 1.

in section 4.5 whereas the ε -check procedure from step 3.c) is associated with the convergence theorem and will be discussed in section 4.4.

4.3 Zangwill’s Method to Prove Convergence Feasible direction optimization algorithms can be written as xk+1 = xk + τk dk

76

M. Bazan

where dk is a search direction and τk is a search step. The k-th iteration of such algorithms can be viewed as a composite algorithmic transformation A = M 1 D where D : Rd → Rd × Rd is a search direction generating transform D(x) = (x, d) where d ∈ Rd is a direction vector, whereas M 1 : Rd × Rd → Rd is a line minimization transform M 1 (x, d) = {y : f (y) = min f (x + τ d), y = x + τ 0d} τ ∈J

where (x, d) ∈ Rd × Rd and J is a variability interval for a scalar τ . In his monography [21] Zangwill gave a method of proving the convergence of feasible directions optimization algorithms based on properties of an algorithmic transformation A. In this section we sketch all crucial definitions and lemmas used to state the main convergence theorem. Definition 1. A transformation A : V → V is a transformation of a point into a set when each point x ∈ V is assigned a set A(x) of points from V . The result of application of the transform A(·) to a point xk can be any point xk+1 from a set A(xk ) thus xk+1 ∈ A(xk ). A transformation A = M 1 D defining a feasible direction optimization algorithm is a transformation of a point into a set. Definition 2. We say that a transformation A : V → V is closed in a point x∞ , if the following implication holds true: 1. xk → x∞ , k ∈ K , 2. yk ∈ A(xk ), k ∈ K , 3. yk → y∞ , imply 4. y∞ ∈ A(x∞ ), where K is a sequence of natural numbers. We say that transformation A is closed on X ∈ V if it is closed in any point x ∈ X. The property of closedness for algorithmic transformations is an analogue of the property of continuity for ”usual” functions. Theorem 1. (see [21] page 99) Let a transformation A : V → V of a point into a set be an algorithm, which for a given point x1 , generates a sequence {xk }∞ k=1 . Let S ⊂ V be a set of solutions. Let us assume 1. All points xk are in a compact set X ⊂ V . 2. There exists a function Z : V → R such, as

4

Search Procedure Exploiting Locally Regularized Objective Approximation

77

a. if point x is not a solution, then for any y ∈ A(x) there is Z(y) < Z(x) b. if point x is a solution, then either the algorithm finishes or for any y ∈ A(x) Z(y) ≤ Z(x). 3. Transformation A is closed for a point x if it is not a solution. Then either the algorithm finishes in a point which is a solution or any convergent subsequence generated by the algorithm has its limit in the solution set S. Now we additionally need two lemmas from [21] concerning the closedness of a composition of closed transforms. Lemma 1. Let C : W → X be a given function and B : X → Y be a transform of a point into a set. If function C is continuous in point w∞ and B is closed in C(w∞ ), then a composition A = BC : W → Y is closed in w∞ . Lemma 2. Let f be a continuous function. Then the transform M 1 is closed, if J is a compact and bounded interval.

4.4 The Main Result In practice usually another line search operator is considered, since implementation of the operator M 1 (·, ·) is expensive. Let’s consider a line search operator M ∗ defined as (4.1) M ∗ (x, d) = M 1 (x, d) ∪ {y = x + τ d : f (y) ≤ f (x) − Δ , τ ∈ J}. Its value is a set of points for which function f is decreased by Δ on the direction d from point x or when this set is empty its value is a minimum of function f . A suggestion of a practical application of the operator M ∗ can be found in one of the exercises in [21]. In the algorithm EXTREM such an operator is used instead of M 1 . The operator M ∗ is implemented to be the parabolic search algorithm. Application of the operator M ∗ is more practical, in particular in the initial part of the optimization process, where the first steps of the parabolic interpolation search give the most significant decrease of the function value whereas the next steps usually make the function f decrease much slower. The main result of this note is the following theorem. Theorem 2. Let a function f : Ω → R; Ω ⊂ Rd be an objective function of the optimization problem. Let be given a method to approximate the objective function f in certain points of the domain Ω with a relative error ε > 0. If function f is strictly convex the SPELROA method combined with the Gauss-Seidel search algorithm using approximated sequential quadratic interpolation as a line search method converges to a stationary point x0 ∈ S where S = {x : ∇ f (x) = 0}.

78

M. Bazan

Proof of Theorem 2 will be based on Theorem 1 where transformation A will be A = RM ∗ Dd M ∗ Dd−1 . . . M ∗ D2 M ∗ D1 where Di chooses i-th direction from the orthogonal direction base of the k-th iteration and R is an orthogonalization step to produce a new base along the direction k−1 xk−1 from k − 1 iteration. 0 xd

4.4.1 Closedness of the Algorithmic Transformation To show that point 3 of the theorem 1 is fullfilled we first have to show that the transformation M ∗ defined in (4.1) is closed. We prove the following lemma. Lemma 3. Let f be a continuous function. Then the transform M ∗ is closed, if J is a compact and bounded interval. Proof. According to Definition 2 let us consider sequences {(xk , dk )}∞ k=1 and {yk }∞ k=1 . We assume that 1. (xk , dk ) → (x∞ , d∞ ), k ∈ K 2. yk ∈ M ∗ (xk , dk ), k ∈ K 3. y → y∞ , k ∈ K So we have that

yk = xk + τ k dk

where τk ∈ J k for which f (xk + τ k dk ) ≤ f (xk ) − Δ . Because τ k ∈ J for k ∈ K and J is compact, thus there exists a convergent subsequence τ k → τ ∞, k ∈ K 1, where K 1 ⊂ K and τ ∞ ∈ J. For a fixed τ ∈ J from the definition of yk it folllows

or

f (yk ) < f (xk + τ dk )

(4.2)

f (yk ) − f (xk + τ dk ) ≤ Δ .

(4.3)

Note that if (4.3) is not satisfied then (4.2) is satisfied. Since f is continuous in a limit we get f (y∞ ) = lim f (yk ) < lim f (xk + τ dk ) = f (x∞ + τ d∞ ) k∈K 1

and in the same way

k∈K 1

f (y∞ ) − f (x∞ + τ d∞ ) ≤ Δ .

(4.4)

(4.5)

4

Search Procedure Exploiting Locally Regularized Objective Approximation

79

Because the above for any τ (4.4) or (4.5) is fulfilled then for any point y∗ ∈ M ∗ (x∞ , d∞ ) we can f (y∞ ) < f (y∗ ), (4.6) or

f (y∞ ) − f (y∗ ) ≤ Δ .

(4.7)

On the other hand in point y∗ ∈ M ∗ (x∞ , d∞ ) function f for τ ∈ J attains the least value y∞ = x∞ + τ ∞ d∞ , τ ∞ ∈ J or, therefore

f (y∞ ) < f (x∞ ) − Δ

(4.8)

f (y∗ ) − f (y∞ ) ≤ Δ .

(4.9)

Comparing (4.8) and (4.9) with (4.6) and (4.7) we get the result y∞ ∈ M ∗ (x∞ , d∞ ). To make use of Lemma 1 in order to show the closedness of transformation A we also have to notice that the transformations Di (x) = (x, di ) (i = 1, . . . , d) are continuous functions. For direct search algorithms transformations Di generate orthogonal directions. They are the same as in the Gauss-Seidle algorithm (c.f. [21]). For conjugate direction search algorithms transformations Di generate succesive conjugate directions. Transformation R that generates the orthogonal serach directions new [dnew 0 , . . . , dd−1 ] for the step k is defined as k new new R(xk−1 0 , [d0 , . . . , dd−1 ]) = (x0 , [d0 , . . . , dd−1 ]),

where the sequence of the new orthogonal vectors is uniquely defined by the orthogonalization of the vectors w0 , w1 , . . . , wd−1 w0 = s0 d0 + s1 d1 + . . . + sd−1 dd−1 w1 = + s1 d1 + . . . + sd−1 dd−1 ... wd−1 = + sd−1 dd−1 where scalars s0 , s1 , . . . , sd−1 correspond to step sizes in all directions in the step k − 1. Transformation R is uniquely defined without any conditions on scalars s0 , s1 , . . . , sd−1 as long as the orthogonalization is performed using the algorithm presented in [15]. In this case it is also a continuous function. Finally then since the transformation A is a composition of closed transformations M ∗ with continuous functions Di (i = 0, . . . , d − 1) and R then the assumptions of the Lemma 1 are satisfied we see that transformation A is closed. That proves that assumption 3 of the Theorem 1 holds for unperturbed M ∗ . In the succeeding subsection we show that transformation M ∗ can be realized by the perturbed transformation M 1 .

80

M. Bazan

4.4.2 A Perturbation in the Line Search For non-gradient optimization algorithms with an orthogonal basis as a set of search directions transformation M ∗ is the only place in the algorithm where the perturbation from approximation of the objective function is introduced by the SPELROA method. Therefore to show that point 2 of the Theorem 1 is fullfilled it is sufficient that implementation of the transformation M ∗ that allows perturbation of function value on the level ε > 0 minimizes function f along the direction d. A proof of the convergence of the parabolic interpolation line search method can be found in [8] or [14]. Here we will give conditions on the perturbation of the function so that the proof given in [14] holds true. We will keep the notation as close as possible to that in [14]. (i) (i) (i) Let function f : R → R be unimodal. Let for a triplet ζ (i) = (ζ1 , ζ2 , ζ3 ) be (i)

(i)

(i)

(i)

(i)

f (ζ2 ) ≤ min{ f (ζ1 ), f (ζ3 )} i.e. the interval [ζ1 , ζ3 ] contains an unique minimizer of function f . For a non-perturbed objective function we define a set of feasible triplets T ⊂ R3 defining an interval [ζ1 , ζ3 ] that contains the minimizer λˆ as T := {ζ ∈ R3 : ζ1 < ζ2 < ζ3 , f (ζ2 ) ≤ min{ f (ζ1 ), f (ζ3 )} ∪{ζ ∈ R3 : ζ1 = ζ2 < ζ3 , f (ζ1 ) ≤ 0, f (ζ3 ) ≥ f (ζ1 )} ∪{ζ ∈ R3 : ζ1 < ζ2 = ζ3 , f (ζ3 ) ≥ 0, f (ζ1 ) ≥ f (ζ3 )} ∪{ζ ∈ R3 : ζ1 = ζ2 = ζ3 = λˆ }. For ζ ∈ T with ζ1 < ζ2 < ζ3 the minimum of the quadratic interpolating points (ζ1 , f (ζ1 )), (ζ2 , f (ζ2 )), (ζ3 , f (ζ3 )) equals

λ ∗ (ζ ) =

1 (ζ32 − ζ22 ) f (ζ1 ) + (ζ32 − ζ12 ) f (ζ2 ) + (ζ22 − ζ12 ) f (ζ3 ) 2 (ζ3 − ζ2 ) f (ζ1 ) + (ζ3 − ζ1 ) f (ζ2 ) + (ζ2 − ζ1 ) f (ζ3 )

Then the set of admissible replacement triplets A(ζ ) is a set of candidate triplets that may replace ζ ∈ T defining a smaller interval containing λˆ in the next iteration of the algorithm. A0 (ζ ) is defined as (ζ1 , λ ∗ (ζ ), ζ2 ), (ζ2 , λ ∗ (ζ ), ζ3 ), A0 (ζ ) := T ∩ {u1 (ζ ), u2 (ζ ), u3 (ζ ), u4 (ζ )} where (λ ∗ (ζ ), ζ2 , ζ3 ), (ζ1 , ζ2 , λ ∗ (ζ )). (4.10) The crucial assumption on the perturbation that we introduce into the triplet used to construct a quadratic is that we always allow a perturbation in only one point of three points. Without loss of generality let us assume that a minimum of the perturbed quadratic is constructed only for ζ such that ζ1 < ζ2 < ζ3 . For the perturbation of the value of the function f on the level ε = 0 we define three sets of triplets: u1 (ζ ) u2 (ζ ) u3 (ζ ) u4 (ζ )

= = = =

4

Search Procedure Exploiting Locally Regularized Objective Approximation

81

T1 (ε ) := {ζ ∈ R3 : ζ1 < ζ2 < ζ3 , f (ζ2 ) ≤ min{ f (ζ1 )(1 − |ε |), f (ζ3 )} } (4.11) T2 (ε ) := {ζ ∈ R3 : ζ1 < ζ2 < ζ3 , f (ζ2 )(1 + |ε |) ≤ min{ f (ζ1 ), f (ζ3 )} } (4.12) T3 (ε ) := {ζ ∈ R3 : ζ1 < ζ2 < ζ3 , f (ζ2 ) ≤ min{ f (ζ1 ), f (ζ3 )(1 − |ε |)} } (4.13) with the minima of the underlying perturbed quadratics 1 (ζ32 − ζ22 ) f (ζ1 )(1 + |ε |) + (ζ32 − ζ12 ] f (ζ2 ) + (ζ22 − ζ12 ) f (ζ3 ) , λ˜ 1∗ (ε ; ζ ) = 2 (ζ3 − ζ2 ) f (ζ1 )(1 + |ε |) + (ζ3 − ζ1 ) f (ζ2 ) + (ζ2 − ζ1 ) f (ζ3 ) 1 (ζ32 − ζ22 ) f (ζ1 ) + (ζ32 − ζ12 ) f (ζ2 )(1 + |ε |) + (ζ22 − ζ12 ) f (ζ3 ) , λ˜ 2∗ (ε ; ζ ) = 2 (ζ3 − ζ2 ) f (ζ1 ) + (ζ3 − ζ1 ) f (ζ2 )(1 + |ε |) + (ζ2 − ζ1 ) f (ζ3 ) 1 (ζ32 − ζ22 ) f (ζ1 ) + (ζ32 − ζ12 ) f (ζ2 ) + (ζ22 − ζ12 ) f (ζ3 )(1 + |ε |) λ˜ 3∗ (ε ; ζ ) = 2 (ζ3 − ζ2 ) f (ζ1 ) + (ζ3 − ζ1 ) f (ζ2 ) + (ζ2 − ζ1 ) f (ζ3 )(1 + |ε |) for a perturbation of function f in points ζ1 , ζ2 and ζ3 respectively. Such definitions of sets Tl (ε ) (l ∈ {1, 2, 3}) provide that the perturbed minima are contained within the interval [ζ1 , ζ3 ]. Corresponding sets of admissible triplets are defined as A˜ l (ε ; ζ ) := Tl (ε ) ∩ {u˜l1 (ε ; ζ ), u˜l2 (ε ; ζ ), u˜l3 (ε ; ζ ), u˜l4 (ε ; ζ )} where

u˜l1 (ε ; ζ ) u˜l2 (ε ; ζ ) u˜l3 (ε ; ζ ) u˜l4 (ε ; ζ )

= = = =

(ζ1 , λ˜ l∗ (ε ; ζ ), ζ2 ), (ζ2 , λ˜ l∗ (ε ; ζ ), ζ3 ), (λ˜ l∗ (ε ; ζ ), ζ2 , ζ3 ), (ζ1 , ζ2 , λ˜ l∗ (ε ; ζ )),

(4.14)

where l ∈ {1, 2, 3}. Lemma 4. Let S denote the set of stationary points of function f S := {ζ ∈ T : f (ζ1 ) = 0 or f (ζ2 ) = 0 or f (ζ3 ) = 0}. The following statements hold: 1. For every ζ ∈ T, the set A(ζ ) = A0 (ζ ) ∪ A˜ 1 (ε ; ζ ) ∪ A˜ 2 (ε ; ζ ) ∪ A˜ 3 (ε ; ζ ) is nonempty. 2. The set valued map A(·) is closed. / S} such that ζ1 < ζ2 < ζ3 there is c(y) < c(ζ ) 3. For every ζ ∈ T\S := {ζ ∈ T : ζ ∈ for all y ∈ A(ζ ) for c(ζ ) = f˜1 (ζ1 ) + f˜2 (ζ2 ) + f˜3 (ζ3 ) where f˜l (·) = f or f˜l (·) = f (·)(1 + |ε |) depending on whether a value at point ζl was exact or perturbed. Proof. Let us introduce the following notation f˜l (·) = f (·) when evaluated at points x = ζl or f˜l (·) = f (·)(1 + ε ) when evaluated at point x = ζl . At the beginning let us note that Tl (ε ) ⊂ T for (l ∈ {1, 2, 3}) and ε > 0.

82

M. Bazan

1. Let ζ = (ζ1 , ζ2 , ζ3 ), ∈ T be fixed. If f (ζ1 ), f (ζ2 ) and f (ζ3 ) are computed without perturbation then A(ζ ) is not empty by proof in [14]. Let us consider then that a function value was approximated with the relative error equal ε . The minimum of the quadratic constructed in this case will be λ˜ l∗ (ε ; ζ ) where l ∈ {1, 2, 3} depending on at which point a function value is approximated. Let us consider the case when λ˜ l∗ (ε , ζ ) ∈ [ζ1 , ζ2 ] providing moreover that the minimum λ ∗ (ζ ) obtained as if the function was evaluated without any perturbation also belongs to [ζ1 , ζ2 ]. Then A(ζ ) is empty if and only if both u˜1 (ε ; ζ ) and u˜3 (ε ; ζ ) are not in A(ζ ), i.e. if and only if f (λ˜ 1∗ (ε ; ζ )) > min{ f (ζ1 )(1 + |ε |), f (ζ2 )} = f (ζ2 )

(4.15)

and f (ζ2 ) > min{ f (λ˜ 1∗ (ε ; ζ )), f (ζ3 )} ≥ min{ f (λ˜ 1∗ (ε ; ζ )), f (ζ2 )}

(4.16)

if a function value is approximated at ζ1 (analogue inequalities are considered for perturbations in ζ2 and ζ3 ). Since the inequalities (4.16) imply, that f (ζ2 ) ≥ f (λ˜ 1∗ (ε ; ζ )) we get a contradiction with the inequality (4.15) which proves the thesis when the perturbation is in the point ζ1 . The similiar contradiction we get for analogue inequalities for perturbations in ζ2 and ζ3 . Please note that the condition that the corresponding unperturbed λ ∗ (ζ ) is also in [ζ1 , ζ2 ] guarantees that if we compare y˜ = (y˜1 , y˜2 , y˜3 ) ∈ A˜ l (ζ ) (l ∈ {1, 2, 3}) with y = (y1 , y2 , y3 ) ∈ A0 (ζ ) we get y˜1 ≤ y1 and y˜3 ≥ y3 . The case when λ˜ ∗ (ζ ) belongs to [ζ2 , ζ3 ] is symmetric. This shows that A(ζ ) is not empty. 2. We will prove the closedness of A from Definition 2. Let us assume that {ζ (i) }∞ i=0 and ζ (i) → ζ∗ ∈ T and there exists ζ∗∗ ∈ T and an infinite subsequence K ⊂ N such that ζ (i+1) ∈ A(ζ (i) ) for every i ∈ K such that ζ (i+1) →K ζ∗∗ as i → ∞. Then there must exist k ∈ {1, 2, 3, 4} and an inifinite subsequence K ⊂ K such that ζ (i+1) = uk (ζ (i) ) or ζ (i+1) = u˜lki (ε ; ζ (i) ) (where li ∈ {1, 2, 3}) for every i ∈ K . As shown in [14] functions uk (·) are continuous and therefore when the sequence (i) generated by approximated values then from the {ζ (i) }∞ i=0 does not contain ζ continuity of uk (·) with respect to ζ and the closedness of the set T it follows that uk (ζ (i) ) → uk (ζ∗ ) = ζ∗∗ ∈ A(ζ∗ ). This proves the closedness of transformation A if no approximation is used in any sequence ζ (i) . Now we will consider the sequences containing approximated triplets. Introducing the approximated triplets ζ (i+1) (i.e. ζ (i+1) ∈ A˜ l (ε ; ζ (i) )) introduces discontinuities of the first kind into functions uk (ζ ) and the argument based on continuity of functions uk (ζ ) cannot be applied directly. We will show that using algorithm A introduces a finite number of isolated points discontinuities and that guarantees that from any sequence {ζ (i) }∞ i=1 after removing some finite number of initial elements we can apply the proof from [14]. We have to consider two cases

4

Search Procedure Exploiting Locally Regularized Objective Approximation

83

• When a number of occurences of u˜lki in the sequence {ζ (i) }∞ i=0 is finite then the proof of the closedness of transformation A given in [14] applies after removing from {ζ (i) }∞ i=0 some number of initial elements. • Let us then assume that u˜lki occur an inifinite number of times in {ζ (i) }∞ i=0 . (i) ∞ Since {ζ }i=1 is convergent then for any δ ∈ R there exists i0 such that for all i > i0 we have (i )

(i )

(i )

(i +1)

|ζ (i0 ) − ζ (i0 +1) | = |(ζ1 0 , ζ2 0 , ζ3 0 ) − (ζ1 0

(i +1)

, ζ2 0

(i +1)

, ζ3 0

)| < δ (4.17)

Let us choose such i1 for which inequality (4.17) holds. Since the approximation in the next iterations is used infnitely many times then there exists a subsequence K = {i : i > i1 } ⊂ N such that if i ∈ K then

ζ (i+1) ∈ A˜ l (ε ; ζ (i) ) for some l ∈ {1, 2, 3}. We will show that for any δ it is possible to choose ε such that ||ζ (i1 ) − ζ (i1 +1) ||2 > 2δ .

(4.18)

This will give us at the end a contradiction with assumed convergence. To give conditions on ε we solve inequalities (4.18) using expressions (4.14) for ulk (ε , ζ ) for k = 1, . . . , 4 and l ∈ {1, 2, 3} and applying a method of parabola transformation qˆ from Appendix A. (i) For example for perturbation in ζ1 we get

ε1 >

(B − C)(ζ(r) − Ka ) − (B − ζ(r)C) − A(Ka − 1) A(Ka − 1)

and one of the system of inequalities is fullfiled   A(1 + ε1) + B − C < 0 A(1 + ε1) + B − C > 0 or . A(Ka − 1) < 0 A(Ka − 1) > 0 

where

ε1 =

ε (i)

f (ζl )

, Ka =

2δ ζ3 − ζ1

2

(4.19)

(4.20)

− [1 − ζ(r)]2 .

ζ1 with A, B and C defined in Appendix A. We leave to the and ζ(r) = ζζ2 − 3 −ζ1 reader showing that these conditions are not contradicting and also derivation of the analogue inequalities for l = 2 and l = 3. The above considerations mean for any δ the value ε can be chosen so that (4.18) holds. But it contradicts with the assumption that {ζi }∞ i=0 converges since we can choose two subsequences from {ζi }∞ i=0 that converge to two different accumulation points. The first subsequence is formed from ζi1 s and the second from ζi1 +1 s. Both are infinite. From this we conclude that from

84

M. Bazan

certain i0 the sequence {ζi }∞ i=0 cannot contain approximated points and therefore the proof from [14] also applies in this case. This finishes the proof of the closedness of transformation A(ζ ). 3. Let us assume that ζ ∈ T\S. Then λ˜ ∗ (ζ ) ∈ (ζ1 , ζ3 ). Hereafter in this point we consider the quadratic obtained with a transformation qˆ from Appendix A. This transformation preserves all properties used in this proof. In the following consideration we will use the expression Δl (ε ; ζ ) which is the distance between the unperturbed and perturbed minimum i.e. Δl (ε ; ζ ) = |λ˜ l∗ (ζ ) − λ ∗ (ζ )|. See Appendix B for expression Δl (ε ; ζ ). Here the situation is also symmetric with respect to ζ1 and ζ3 and we will consider the case where λ ∗ (ζ ) ∈ (ζ1 , ζ2 ] as well as λ˜ l∗ (ζ ) ∈ (ζ1 , ζ2 ] (l ∈ {1, 2, 3}). Firstly we observe that f (ζ2 ) ≤ f (ζ3 ), because if there was f (ζ2 ) = f (ζ3 ), then λ˜ ∗ (ζ ) = 1/2(ζ2 + ζ3 ) since no value is perturbed or only the value at point ζ2 is perturbed and there is no effect of this perturbation on a minimum. Both cases contradict the assumption that λ˜ l∗ (ζ ) ∈ (ζ1 , ζ2 ] (l ∈ {1, 2, 3}). When λ˜ l∗ (ζ )∈(ζ1 , ζ2 ] (l ∈{1, 2, 3}) then only u1(ζ ) and u˜l1 (ε ; ζ ) (l ∈ {1, 2, 3}) and u3 (ζ ) and u˜l3 (ζ ) (l ∈ {1, 2, 3}) can belong to A(ζ ). For unperturbed function values we have λ ∗ (ζ ) < ζ2 and for perturbed function values we have

λl∗ (ε ; ζ ) + Δl (ε ; ζ ) < ζ2

(l ∈ {1, 2, 3}).

(4.21)

In Appendix B we solve (4.21) with respect to ε establishing conditions for which λl∗ (ε ; ζ ) < ζ2 and provide the line of proof from [14] is valid. We get only three cases / A(ζ ) as well as a. A(ζ ) = {u1 (ζ ), u˜11 (ζ ), u˜21 (ζ ), u˜31 (ζ )}. Then since u3 (ζ ) ∈ u˜l3 (ε ; ζ ) ∈ / A(ζ ) (l ∈ {1, 2, 3}) and f (λ ∗ (ζ )) < f (ζ2 ) as well as f (λ˜ l∗ (ε ; ζ )) < f (ζ2 ) (l = 1, 3) (4.22) f (λ˜ 2∗ (ε ; ζ )) < f (ζ2 )(1 + |ε |) we must have c(u˜l1 (ζ )) = f˜l (ζ1 ) + f (λ˜ l∗ (ζ )) + f˜l (ζ3 ) < f˜l (ζ1 ) + f˜l (ζ2 ) + f˜l (ζ3 ) = c(ζ ), where f˜l (·) = f (·) or f˜l (·) = f (·)(1 + ε ) depending on which value was perturbed. / A(ζ ) as well as b. A(ζ ) = {u3 (ζ ), u˜13 (ζ ), u˜23 (ζ ), u˜33 (ζ )}. Then, since u1 (ζ ) ∈ u˜l1 (ε ; ζ ) ∈ / A(ζ ) (l ∈ {1, 2, 3}) we must have that f (ζ2 ) ≤ f (λ ∗ (ζ )) as well as f˜(ζ2 ) ≤ f (λ˜ l∗ (ζ )) (l ∈ {1, 2, 3}) depending on which value was perturbed. Also f (λ ∗ (ζ )) < f (ζ1 ) as well as fl (λ˜ l∗ (ζ )) < f˜l (ζ1 ) (l ∈ {1, 2, 3}) since otherwise we would have a local maximum in [ζ1 , ζ2 ] contradicting unimodality. Therefore in this case we must have c(u˜l3 (ζ )) = f (λ˜ l∗ (ζ )) + f˜2 (ζ2 ) + f˜3 (ζ3 ) < f˜l (ζ1 ) + f˜l (ζ2 ) + f˜l (ζ3 ) = c(ζ ) with perturbation at point ζl .

4

Search Procedure Exploiting Locally Regularized Objective Approximation

85

c. Finally, we can have a case A(ζ ) = {u1 (ζ ), u3 (ζ )}. In this case we are not able to include to A(ζ ) any of the approximated triplet u˜l (ε ; ζ ) (i = 1, 2, 3). This is because of the following properties i. f (ζ2 ) < f (ζ1 ) by assumption, ii. λ ∗ (ζ ) ≤ ζ2 , iii. f (λ ∗ (ζ )) = f (ζ2 ) which implies λ ∗ (ζ ) = ζ2 . These equalities hold since otherwise, we would have a contradiction with the unimodality of f (·). Approximating any of the value would mean that we would not be able to guarantee the property iii. Therefore since f (ζ2 ) < min{ f (ζ1 ), f (ζ3 )} we get c(u1 (ζ )) < c(ζ ) and c(u3 (ζ )) < c(ζ ). From a practical point of view for a given ζi where one of the coordinates is approximated or all are exact, we can see whether we have to use exact values i.e. remove the approximation, checking whether |λ˜ l∗ (ε ; ζ ) − ζ2 | > Δl (ε ; ζ )

(l ∈ {1, 2, 3}).

This exhausts all the possibilities and finishes the proof of the third point. We are now in a position to formulate the Sequential Quadratic Interpolation Algorithm with Perturbation (c.f. [14] for the unperturbed version). Please note that in the following we use the expressions an “approximated” and “perturbed” function values interchangably – they are synonymous here. Moreover the adjective “exact” here means perturbed on the level of the machine precision ε where ε << ε . The main condition under which we can make use of the available approximation of the function in one of the points is the separation property, i.e. for the approximation used in a point with an index l ∈ {1, 2, 3} the triplet ζi belongs to Tl where Tl is defined by (4.11), (4.12) or (4.13) respectively. Now we can formulate a convergence theorem for the above algorithm analogous to that in ([14] p. 155). Theorem 3. Suppose that {ζi }∞ i=0 is a sequence constructed by Algorithm 2 in minimizing a continuously differentiable and unimodal function f : R → R. Then ζ (i) → ζˆ as i → ∞ with ζˆ ∈ S. To prove the above theorem we will show the way in which we can apply the proof given in [14] making use of the Lemma 4. Proof. The main difficulty in application of the method of the proof from [14] is the fact that using approximation in certain points of the domain causes a discontinuity of the cost function c as well as a discontinuity of the functional of calculating the minimum of the quadratics with respect to a parameter triplet ζ (i) . Let us observe first that the proof of the third point of Lemma 4 shows that allow(i) ing perturbation according to (4.23) provides that {ζ1 }∞ i=0 is monotone increasing (i) is monotone decreasing. Since both these sequences are bounded as well as {ζ3 }∞ i=0 (i) they are both convergent. Moreover keeping Δ (ε ; ζ ) on a level so that the above

86

M. Bazan

1 2

3

The Sequential Quadratic Interpolation Algorithm with the Objective Function Perturbation Input : ζ0 ∈ T – a starting point, ε > 0 – the relative error of the approximation available in certain points of the function evaluation. 0. Set i = 0. 1. Compute λ ∗ = λ ∗ (ζ (i) ) or λ ∗ = λ˜ l∗ (ε ; ζ (i) ) depending on whether the function (i) (i) (i) value was exact in all points ζ1 , ζ2 , ζ3 or it was perturbed in point (i)

ζl (l ∈ {1, 2, 3}) respectively. (i) (i) 2. If λ ∗ = ζ1 or λ ∗ = ζ3 then STOP, else construct the set A(ζ (i) ) and a. If the approximation to any value of the triplet ζ (i) is not available then A(ζ (i) ) = A0 according to (4.10). (i) b. If the approximation in point ζl (l ∈ {1, 2, 3}) is available i. Compute transformation qˆ as described in Appendix A. ii. Compute Δl (ε ; ζ (i) ) iii. If (i) |λl∗ (ε ; ζ ) − ζ2 | < Δl (ε ; ζ ). then A(ζi ) = A0 and go to 3. iv. If

λl∗ (ε ; ζ ) + Δl (ε ; ζ ) < ζi2

or

λl∗ (ε ; ζ ) − Δl (ε ; ζ ) > ζi2

(4.23)

then A = A0 ∪ A˜ l 3. Compute

ζ (i+1) ∈ arg min{c(ζ ) : ζ ∈ A(ζ (i) )}

4. Replace i := i + 1 and go to step 1.

property of the mentioned sequences would be fulfilled when the approximation (i) (i) would not be used, guarantees that ζˆ ∈ [ζ1 , ζ3 ] for any i ∈ N. We have to distinguish between two nontrivial cases ˆ ˆ ˆ ˆ ˆ ˆ 1. When {ζ (i) }∞ i=1 → ζ and ζ = (ζ1 , ζ2 , ζ3 ) is such an accumulation point that ζ1 < ζˆ2 < ζˆ3 . In this case the contradiction is obtained using the continuity of the cost function c if an unperturbed algorithm was used. We can use the continuity argument here since as it was shown in the proof of Lemma 4 the approximation can only be used in a finite number of steps. Therefore after removing a certain number of initial steps the proof from [14] applies. 2. The first case, therefore does not apply, i.e. the sequence constructed by Algorithm 2 can have two accumulation points. In [14] it is shown that it is the same

4

Search Procedure Exploiting Locally Regularized Objective Approximation

87

accumulation point. Lemma 8 guarantees that the argument contained in [14] holds true if the constructed sequence ends up in the stopping criterion with an unperturbed triplet. Since the sequence is not infinite we have to consider one more case, namely, when the algorithm stops with a perturbed triplet. Two accumulation points are ζ∗ = (ζˆ1 , ζˆ1 , ζˆ3 ) and ζ∗∗ = (ζˆ1 , ζˆ3 , ζˆ3 ). To provide the separation of function values in the first case the perturbation can be only in the point ζˆ3 . In the second case the perturbation can be only in the point ζˆ1 . (i) (i) (i) (i) In the above sequences we have therefore ζ2 → ζ1 or ζ2 → ζ3 . On the other (i)

hand, the separation between λ ∗ (ε ; ζ (i) ) and ζ2 has to be greater than Δ3 (ε ; ζ (i) ) and Δ1 (ε ; ζ (i) ) for two cases respectively. This gives the contradiction with the convergence.

From the practical point of view to provide the convergence of the perturbed SQI algorithm and therefore the whole SPELROA method it is sufficient to provide that the procedure ε -check in step 3.c) of Algorithm 1 to be launched in each iteration of the perturbed SQI algorithm. It takes the last three points xi−2 , xi−1 , xi and compute x −xk−1 ζk = (0,t1 ,t2 ) such that t1 = k−2 xk−2 −xk , t2 = 1 and then compute the transformation qˆ defined by (4.34) to obtain points (0, −2), (ζ(r) , q( ˆ ζ(r) )), (1, −1) when f˜(xk−2 ) < f˜(xk ) (for the case when f˜(xk−2 ) > f˜(xk ) it is sufficient to rotate the scaled parabola around point 0.5) and scales ε ⎧ ε when a perturbation is in point xk−2 , ⎪ ⎨ f (xk−2 ) , ε when a perturbation is in point xk−1 , ε := f (xk−1 ) , ⎪ ⎩ 2ε , when a perturbation is in point xk f (x ) k

depending on at which point the function value was approximated. Then if 1. a perturbation is at 0 then if ε satisfies (4.35) and fullfills (4.36) or (4.37) then the approximation can be used else calculate the objective function deterministically, 2. a perturbation is at ζ(r) then if ε satisfies (4.38) and fullfills (4.37) or (4.40) then the approximation can be used else calculate the objective function deterministically, 3. a perturbation is at 1 then ε has to satisfy analogue conditions (left to the reader) for the approximation to be used by the algorithm.

4.5 The Radial Basis Appproximation In this section we will discuss the main aspects and possibilities of constructing a radial basis approximation of the objective function in Algorithm 1.

4.5.1 Detecting Dense Regions Since the data set Z = {xi }Ni=1 is very sparse then a method to detect regions rich in data within the convex hull Ω of Z is required. In [2] we introduced a number of merit

88

M. Bazan

γ (x, Z ) :=

∑Nj=1,i< j ai jWi j ∑Nj=1,i< j Wi j

(4.24)

d

ij 1 where ai j = r j +r , Wi j = r j +r , and di j = ||x j − xi ||2 and r j = ||x − x j ||2 . γ (x, Z ) i i measures how well data points from Z surround the evaluation point x. Numbers ai j measure how far point x is placed from the interval xi x j . The maximal value equal 1 is attained by ai j if point x is on this interval. The weighting Wi j emphasize in γ (x, Z ) the impact of intervals xi x j that are close to point x. The additional denominating by a sum of weights Wi j provides that for any x ∈ Rd the range of values of γ (x, Z ) is (0, 1]. If the value of γ (x, Z ) is greater than a certain threshold value then in the evaluation point x a construction of an approximation with a good local quality can be expected.

0.9 0.8 0.7 0.6 X2

0.5 0.4 0.3 0.2 0.1 0 −1

−0.9 −0.8 −0.7 −0.6 −0.5 −0.4 −0.3 X1

Fig. 4.1 γ (x, Z ) defined by (4.24). Here the set Z is a data set constructed on the 50-th optimization step of a 2-parametric Rosenbrock function optimization with the EXTREM algorithm

4.5.2 Regularization Training For a given data set Z = {xi , f (xi )}Ni=1 ⊂ Rd × R of pairwise different data points xi , and for the Gaussian radial basis function φ (x) = exp(−x2 /r2 ), r ≥ 0 (see [9]) a radial basis function interpolatant s(x) is defined as N

s(x) = ∑ wi φi (||x − xi||), i=1

where s(xi ) = f (xi ) = fi ,

i = 1, . . . , N.

(4.25)

4

Search Procedure Exploiting Locally Regularized Objective Approximation

89

The interpolation conditions s(xi ) = fi imply the matrix formulation

Φ w = f,

(4.26)

where Φ = [φ (xi − x j )]i=1,...,N; j=1,...,N , f = [ f1 , f2 , . . . , fN ]T , w = [w1 , w2 , . . . , wN ]T . Positive definiteness [9] of φ guarantees nonsingularity of matrix Φ and thus there exists the unique solution w for which s(x) interpolates data from the set Z. Although the other choice of positive definite radial basis functions is possible we choose the Gaussian function due to its natural interpretation of r parameter which can be set for a value proportional to the diameter of the data set Z = {xi }Ni=1 . There are two reasons for which we solve an approxmiation problem rather than interpolation. Firstly when the data set Z is very irregular and φ is strictly positive definite the matrix Φ is very ill-conditioned. Secondly the problem of solving (4.26) yields solution s(x) which may oscillate between data points where data is sparse. The approximation solution is sought by means of Tikhonov regulatization. Suppose that we are given a linear mapping T : H → R and define the regularization operator J : H → R by J(s) = ||T s||2R where H is a native space ot the underlying radial basis function φ (·). For a function f ∈ H and for the prescribed value of the single regulatization parameter λ its approximation is a function fλ (x) ∈ H in a form (4.25) which is the solution of the minimization problem   N

min fλ

∑ [ f (xi ) − fλ (xi)]2 + λ J( fλ ) : fλ ∈ H

.

(4.27)

i=1

In the matrix form the problem (4.27) is written as (Φ T Φ − λ I)w = Φ T t

(4.28)

for a λ > 0 governing a trade-off between a data reproduction and the desired smoothness of solution fλ (x). Due to the ill-conditioning of matrix Φ a direct inversion of matrix (Φ T Φ − λ I) in (4.28) is not numerically stable. To solve (4.28) in a numerically stable way we use the singular value decomposition of the matrix Φ defined as

Φ = USVT ,

(4.29)

where U ∈ RN×N , V ∈ RN×N are orthogonal matrices and S = diag(σ1 , . . . , σN ), where σ1 ≥ σ2 ≥ · · · ≥ σN are singular values of Φ . The singular value decomposition is unique up to the signs of the columns of matrices U and VT . Using the above decompostion we express the inverse matrix in (4.28) as

Φ † = (Φ T Φ + λ 2 I)−1 Φ T = V((ST S + λ 2I)−1 ST )UT = VΩλ UT , where

 Ωλ = diag

 σ1 σ2 σN , ,..., 2 . σ12 + λ 2 σ22 + λ 2 σN + λ 2

(4.30)

90

M. Bazan

Using the above equation the weight vector wλ is expressed (see [5]) by the expansion with respect to singular vectors of the matrix Φ

σi (uT t)vi . 2 +λ2 i σ i=1 i N

wλ = VT Ωλ† UT t = ∑

(4.31)

Comparing the expansion (4.31) for λ > 0 with the expansion for λ = 0 i.e. for the interpolation problem which reads N

1 T (ui t)vi . i=1 σi

w = VS−1 UT t = ∑

(4.32)

one can see what is the role of the regularization parameter λ . For σ p ≥ λ ≥ σ p+1 we have 1 σi  2 (k > p), σk σk + λ 2 and hence the impact of the singular vectors corresponding to singular values σk < λ is dumped in the expansion (4.31). This enables us to avoid oscillation of the solution that are introduced by inverting small singular values in the expansion of the weight vector. Another approach in solving the problem of ill-conditioning of the interpolation matrix generated by multiquadratic functions was presented in [7].

4.5.3 Choice of the Regularization Parameter λ Value An appropriate procedure to choose λ parameter is a crucial issue in the Tikhonov regularization. A chosen procedure should be able to find a λ for which the solution s(x) reproduces well the data set Z as well as it generalizes well in-between data points. These goals are conflicting and therefore a method should give us the right balance between reproduction of the data set as well as a generalization for data from outside the data set. 4.5.3.1

Weighted Gradient Variance and Local Mean Square Error

In [2] we introduced a method that relates a choice of λ to the reproduction quality on the data points close to the evaluation point x. The method was defined as the following. Let us define r j = ||x − x j ||2 ,

j = 1, . . . , N.

For a sequence of λ s that covers the singular value spectrum (σN , σ1 ) of the matrix S from the decomposition (4.29) we calculate the Normalized Local Mean Square Error

4

Search Procedure Exploiting Locally Regularized Objective Approximation

    [sλ (x j )− f j ]2 1  N · r2  ∑ j=1 f 2j  j NLMSEλ ,Z (x) =  , ∑Nj=1 r12

91

(4.33)

j

and the Weighted Gradient Variance

WGVλ ,Z (x) =

∑Nj=1

||∇sλ (x j )−Gλ ,Z (x)||2 r2j

∑Nj=1 r12

,

j

where Gλ ,Z (x) is a mean gradient variance at point x defined as Gλ ,Z (x) =

∑Nj=1

∇sλ (x j ) r2j

∑Nj=1 r12

,

j

where sλ (x j ) is a value of the constructed approximation for λ at point x j from the data set Z, and ∇ is the gradient operator. It is the discrepancy method since only these λ ’s are considered for which the value of NLMSE is smaller than a prescribed, user-defined threshold value NLMSEthr . The threshold NLMSEthr value says which approximation quality has to be preserved on the points from Z nearest to the evaluation point x. From the set of λ ’s for which NLMSEλ ,Z (x) is smaller than NLMSEthr the minimizer of the oscillation measure WGVλ ,Z (x) is chosen as the optimum. Figure 4.2 shows a) the data set of 30 points from the 2-parametric Rosenbrock function optimization by the EXTREM algorithm, b) a plot of NLMSEλ ,Z (x) for two different points of the domain for a data set, c) a corresponding plot with W GVλ ,Z(A) and W GVλ ,Z (B). Figure 4.3 shows a) the obtained approximation and b) the chosen value of λ parameter.

4.5.4 Error Bounds for Radial Basis Approximation In the previous section we presented a method to choose the value of the regularization parameter λ . Unfortunately, using this method does not provide us that the generalization error is smaller than a prescribed value ε as well as we are not able directly to estimate the generalization error. This method similarly as the Generalized Cross Validation (c.f. [18]) estimates an error on the data points (WGV for nearest points to the evaluation point and GCV for all points from a data set). The error bounds for radial basis interpolation has been studied extensively for more than two decades. The first bounds establishing the rate of convergence of a radial basis interpolant for functions from a native space of the underlaying radial basis function were given in [10]. The latter paper gave the foundations for the development of the theory of convergence of the radial basis interpolation (see

92

M. Bazan

a)

b) 0.98

−1

A B

0.96 −2

A

0.94

−3 log10(NLMSE)

0.92

x2

0.9 0.88

B

0.86 0.84

−4

−5

−6

0.82 −7 0.8 0.78 −1.02

−1

−0.98

−0.96 x

−0.94

−0.92

−8 −18

−0.9

−16

−14

−12

−10

−8

−6

−4

−2

log10(λ)

1

c) 1.8 1.7

A

log10(WGVλ, Z( x))

1.6 1.5 1.4 1.3 1.2 1.1

B 1 −18

−16

−14

−12

−10

−8

−6

−4

−2

log10(λ)

Fig. 4.2 a) Data set consisting of 30 points from the optimization path from EXTREM algorithm optimizing the 2-parametric Rosenbrock function. b) Local reproduction of the data near points A and B measured by NLMSEλ ,Z (A) and NLMSEλ ,Z (B) respectively. Here NLMSEthr = 10−5 is depicted by a straight line. c) A measure of the oscillation of the solution W GVλ ,Z (x) in points A and B. By dots the optimal λ for points A and B are depicted

e.g. [17], [19] and references therein). The rate of the convergence which is considered is with respect to dataset density i.e. a global fill distance h(Z , Ω ) := max min ||y − xi||2 . y∈Ω 1≤i≤N

4

Search Procedure Exploiting Locally Regularized Objective Approximation

a)

93

b) λ for approximation constructed with WGV

approximation error of the approximation−5constructed with WGV for NLMSE thr = 10

−1

−4

x 10

−0.95

2

0.98

0.96

0.94

0.92

0.9

0.88

0.86 −12

y −0.9

−10 −8

1

−6 −4 −2

0 −1 −0.98 −0.96 −0.94 −0.92

0

−0.9 0.86

0.88

0.9

0.92

0.94

0.96

0.98

Fig. 4.3 a) The approximation error for λ chosen by the measure W GVλ ,Z (x) for NLMSEthr = 5.0 · 10−6 . b) Chosen value of λ parameter – it can be noticed that in data regions where data is sparse the method suggests a greater value of λ

where Z = {xi }Ni=1 ⊆ Ω which is a mesh norm measuring the radius of the biggest ball contained in the domain Ω and not containing any data points inside, and where the domain Ω satisfies the interior cone condition with radius r and angle θ . Definition 3. A set Ω ⊆ Rd is said to satisfy an interior cone condition if there exists an angle θ ∈ (0, π /2) and a radius r > 0 so that for every x ∈ Ω a unity vector ξ (x) exists so that the cone C(x, ξ (x), θ , r) := {x + η y : y ∈ Rd , ||y||2 = 1, yT ξ (x) ≥ cos θ , η ∈ [0, r]} is contained in Ω . The summary of error bounds for various radial basis functions was given in [16]. A very precise derivation of the error bounds for Gauss radial basis function interpolation was given in [20]. In the latter paper one can find a derivation of all constants involved in the bound. The analogue error estimates (with all constants involved) of the approximation with positive defined radial basis function constructed with Tikhonov regularization for functions from the Sobolew space Wpτ was presented in [19]. Here we will show that the error estimates contained in [19] cannot be used in our scheme due to the small number of points in the optimization process and therefore using heuristic methods from previous sections is justified. All of the error bounds rely on a common property of local polynomial reproduction that has to be guaranteed by an approximation procedure (c.f. [19]). The error bounds can be formulated in the form of the following theorem. Theorem 4. Suppose that Ω ⊂ Rd is bounded and satisfies an interior cone condition with angle θ and radius r. Let m be a maximal degree of polynomials reproduced by fλ in a form (4.25) defined as the solution of (4.27). Define the following quantities

94

M. Bazan



 sin θ , 4(1 + sin θ ) sin θ sin ϑ . Q(m, θ ) := 2 8m (1 + sin θ )(1 + sin ϑ )

ϑ := 2arcsin

If the global fill distance h(Ω , Y ) satisfies h(Z , Ω ) ≤ Q(m, θ )r. the approximation error can be bounded as || f − fλ ||L∞ (Ω ) ≤ C[h(Z , Ω )]τ −d/p | f |Wpτ (Ω ) + 2ε ,

and ε = max | f (x j )− fλ (x j )|. x j ∈Z

Let us consider the unity ball B(0, 1) as a domain of the approximation. It satisfies the interior cone condition with r = 1 and θ = π /3. It can be seen to reproduce only the linear polynomials i.e. m = 1 and therefore for the above bounds to be satisifed there has to be h(Z , Ω ) ≤ Q(1, π /3) < 0.012613. It means that from one data point to another there must be a distance not greater than 1.3% of the radius of the ball containing the whole data set. Such a number of points cannot be generated by any local optimization algorithm. The above consideration shows that the exisiting accurate error bounds for the regularized approximation with radial basis functions cannot be applied to estimate the approximation in the SPELROA method. It is due to the sparseness of the data in data sets constructed by local optimization algorithms.

4.6 Numerical Results Numerical results of the performance for real optimization problems from the LHC magnet design process with up to 5 parameters were presented in [2]. Here we present results for three test problems from a set of test problems proposed in [12].

4.6.1 Test Problems 1. Six variable problem and eight variable problem I As a six and first eight variable problem we considered the Extended Rosenbrock Function (i.e. problem no 21 in [12]). It is defined as d/2

f ([x1 , x2 , . . . , xd ]) = ∑ 100(x2i − x22i−1)2 + (1 − x2i−1)2 . i=1

A standard starting point is x0 = (ξ j ) where ξ2 j−1 = −1.2 and ξ2 j = 1 and the minimum equals f ∗ = 0 at (1, . . . , 1).

4

Search Procedure Exploiting Locally Regularized Objective Approximation

95

2. Eight variable problem II As a second eight variable problem we chose Chebquad function (i.e. problem no 35 in [12]). It is defined as d

f (x1 , . . . , xd ) = ∑ fi (x)2 ,

where

i=1

fi (x) =

1 d

d

 1

j=1

0

∑ Ti (x j ) −

Ti (x)dx

and Ti is the i-th Chebyshev polynomial shifted to the interval [0, 1]. The standard starting point is x0 = (ξi ) where ξ j = j/(d + 1) and the minimum for d = 8 equals to f ∗ = 3.51687 · 10−3. 3. Eleven variable problem As en eleven variable problem we chose Osborne 2 function (i.e. problem no 19 in [12]). It is defined as 11

f (x1 , . . . , x11 ) = ∑ fi (x)2 ,

where

fi (x) = yi − (xi e−ti x5 + x2e−(ti −x9 )

2x 6

+

i=1

+x3 e−(ti −x10 )

2x 7

+ x4e−(ti −x11 )

2x 8

)

ti = (i − 1)/10 and yi for i = 1, . . . , 65 are constants that can be found in [12]. The standard starting point is x0 = [1.3, 0.65, 0.65, 0.7, 0.6, 3, 5, 7, 2, 4.5, 5.5] and the minimum is f ∗ = 4.01377 · · ·10−2.

4.6.2 Results For each problem we run Algorithm 1 combined with EXTREM algorithm with the three following objective function approximation methods 1. Radial basis function aproximation without regularization, 2. Radial basis function aproximation with regularization using Generalized Cross Validation [18] to choose λ parameter, 3. Radial basis function aproximation with regularization using Weighted Gradient Variance to choose λ parameter. To construct the approximation of the objective function with one of these methods we used 30 Gaussian radial basis functions with an equal shape parameter set to half of the distance between the most distant centers. No additional parameters were required for the first and the second method. The single user-defned threshold NLMSEthr for measure (4.33) was used in the third method. Apart from parameters concerning the construction of the radial basis approximation we had to set-up three parameters related directly to the Algorithm 1 itself. These are: the number of initial steps Is = 50 and ε = 10−3 for the ε -chcek procedure and γthr – the threshold value for a measure (4.24) to detect the reliable region. In the tables below we show performance of the Algorithm 1 for problems defined in the previous subsection with the above objective function approximation strategies compared to the EXTREM algorithm. The first column shows the number

96

M. Bazan

Table 4.1 6-variable Rosenbrock function optimization using: Left) pure EXTREM, Right) Algorithm 1 without regularization and with γthr = 0.65

step num. 250 500 750 1000 1250 1523

EXTREM ||x − x∗|| f 2.267853 2.885768 0.650821 0.109817 0.418187 0.031570 0.246522 0.012666 0.029985 0.001228 0.002014 0.000001

step num. 250 500 750 933

Alg. 1 no regularization num. ||x − x∗|| f approx. 2.459117 3.711824 22 1.318398 0.559350 45 0.675244 0.089372 35 0.130808 0.005606 10

Table 4.2 6-variable Rosenbrock function optimization using: Left) Algorithm 1 with regularization using GCV, Right) Algorithm 1 with regularization using WGV with NLMSEthr = 5 · 10−6 . In both cases γthr = 0.65

step num. 250 500 750 1000 1250 1288

Alg. 1 with GCV Alg. 1 with WGV num. step num. ∗ ∗ ||x − x || f approx. num. ||x − x || f approx. 2.363547 3.991352 11 250 2.292762 2.923941 12 1.094954 0.334820 28 500 1.250487 0.435774 16 0.778636 0.162569 48 750 0.630858 0.150665 28 0.218992 0.009666 47 1000 0.513698 0.048269 38 0.025537 0.000131 49 1250 0.094284 0.001763 47 0.025537 0.000131 10 1262 0.094284 0.001763 4

Table 4.3 8-variable Rosenbrock function optimization using: Left) pure EXTREM, Right) Algorithm 1 without regularization and γthr = 0.65

step num. 250 500 750 1000 1250 1500 1750 2000 3471

EXTREM ||x − x∗|| f 3.923525 10.438226 2.731706 5.606541 1.771076 1.032648 1.009230 0.251387 0.484663 0.050215 0.274941 0.015386 0.262434 0.012950 0.208392 0.007465 0.000553 0.000000

step num. 250 500 750 1000 1250 1500 1537

Alg. 1 no regularization num. ||x − x∗|| f approx. 3.030179 6.376938 22 2.153959 1.855113 23 1.380208 0.471618 30 0.910852 0.215283 46 0.361361 0.028169 43 0.186309 0.006926 55 0.186309 0.006926 7

4

Search Procedure Exploiting Locally Regularized Objective Approximation

97

Table 4.4 8-variable Rosenbrock function optimization using: Left) Algorithm 1 with regularization using GCV, Right) Algorithm 1 with regularization using WGV with NLMSEthr = 10−6 . In both cases γthr = 0.65

step num. Alg. 1 with GCV step num. ∗ num. ||x − x || f approx. 250 500 250 3.100123 7.932378 17 750 500 2.353671 3.580721 28 1000 750 1.810286 1.085058 26 1250 1000 0.946269 0.240704 41 1500 1200 0.705151 0.118365 43 1750 1861

Alg. 1 with WGV num. ||x − x∗|| f approx. 3.263247 9.379781 11 2.288176 2.394216 15 1.272541 0.435303 17 0.767617 0.143009 35 0.505116 0.057994 42 0.146317 0.004114 51 0.028930 0.000169 35 0.028958 0.000169 12

Table 4.5 8-variable Chebquad function optimization using: Left) pure EXTREM Right) Algorithm 1 without regularization and with γthr = 0.65

step num. 1 250 500 750 1000 1237

EXTREM ||x − x∗|| f 0.161512 0.038618 0.097709 0.006187 0.059121 0.004681 0.016460 0.003698 0.000351 0.003517 0.000000 0.003517

step num. 250 500 750 1000 1197

Alg. 1 no regularization num. ||x − x∗|| f approx. 0.098420 0.006216 4 0.059421 0.004684 18 0.009926 0.003584 10 0.000454 0.003517 48 0.000141 0.003517 97

of steps of the algorithm i.e. a sum of the number of the direct functions evaluations and the number of steps in which the radial basis function approximation was used. The second column shows the distance from the minium and the third column shows the objective function value. The last column shows the number of steps in which objective function approximation was used within the previous 250 steps. As we can see in all cases SPELROA required considerably fewer steps than the pure EXTREM algorithm to stop. Using the WGV method to build radial basis approximation gave the best convergence results, i.e. the stopping point for the SPELROA with WGV compared to stopping points given by the method with the other methods. For all problems NLMSEthr was chosen intuitively to ∼ 10−6 . It means that the reconstruction of the training set in the vicinity of the evaluation point was at the level of 10−6. To compute the reliable region γthr set up to 0.65 was sufficient to preserve convergence of the method. In the optimization of 8-variable Chebquad function it turned out that it was possible to reduce γthr to 0.6. That gave 16 points in which the objective function was approximated in the first 250 steps

98

M. Bazan

Table 4.6 8-variable Chebquad function optimization using: Left) Algorithm 1 with regularization using GCV and γthr = 0.65, Right) Algorithm 1 with regularization using WGV with NLMSEthr = 10−6 and with γthr = 0.60

step num. 250 500 750 1000 1095

Alg. 1 with GCV num. step ||x − x∗ || f approx. num. 0.098419 0.006216 4 250 0.059477 0.004684 17 500 0.008713 0.003584 11 750 0.00156 0.003518 118 1000 0.00156 0.003518 68 1077

Alg. 1 with WGV num. ||x − x∗ || f approx. 0.109306 0.064139 0.009802 0.001266 0.001051

0.007110 0.004759 0.003586 0.003519 0.003517

16 73 46 31 52

Table 4.7 11-variable Osborne 2 function optimization using: Left) pure EXTREM, Right) Algorithm 1 without regularization and with γthr = 0.65

step num. 1 250 500 750 1000 1250 1434

EXTREM ||x − x∗|| f 4.755269 2.093420 1.004450 0.081672 0.099411 0.041034 0.073530 0.041772 0.007715 0.040138 0.001092 0.040138 0.000000 0.040138

step num. 250 500 710

Alg. 1 no regularization num. ||x − x∗|| f approx. 1.379902 0.977272 29 0.568322 0.059211 31 0.335485 0.041512 43

Table 4.8 11-variable Osborne 2 function optimization using: Left) Algorithm 1 with regularization using GCV and γthr = 0.65 b) Algorithm 1 with regularization using WGV with NLMSEthr = 10−6 . In both cases γthr = 0.65

step num. 250 500 750 1000 1106

Alg. 1 with GCV num. ||x − x∗ || f approx. 1.673133 0.975217 0.496683 0.068066 0.062328

0.365797 0.045721 0.041805 0.040462 0.040177

32 31 43 35 25

step num. 250 500 750 919

Alg. 1 with WGV num. ||x − x∗ || f approx. 0.953416 0.334653 0.0441106 0.037960

0.106864 0.047136 0.040209 0.040185

14 38 49 29

instead 4 such points when γthr = 0.65. An interesting result was also obtained for Osborne 2 function. The EXTREM algorithm found a better solution than that suggested in [12]. Algorithm 1 with any approximation method did not converge to this minimum. The method without regularization did not converge at all whereas GCV

4

Search Procedure Exploiting Locally Regularized Objective Approximation

99

and WGV converged rather to the minimum suggested in [12] than to that found by EXTREM.

4.7 Summary The Search Procedure Exploiting Locally Regularized Objective Approximation is a method to speed-up local optimization processes. The method combines a nongradient optimization algorithm with the regularized local radial basis function approximation. It relies on using a local regularized radial basis function approximation instead of a direct objective function evaluation in a certain number of function evaluation steps of the optimization algorithm. In this chapter we presented the proof of the convergence of the Search Procedure Exploiting Regularized Objective Approximation which applies to any Gauss-Siedle and conjugate direction search algorithm that uses the sequanetial quadratic interpolation as a line search procedure. The convergence is proven under assumption that the approximation of the objective function with the prescribed approximation relative error is exploited only in the seqential quadratic interplation. The performance of the method was presented on 6 and 8-parametric Rosenbrock function, 8-parametric Chebquad function and 11-parametric Osborne 2 function. Further studies will be to compare the method with trust region methods. Acknowledgements. I would like to thank all referees for very valuable comments.

Appendix A The minimum of the quadratic q(ζ ) built of three points (ζ1 , f (ζ1 )), (ζ2 , f (ζ2 )) and (ζ3 , f (ζ3 )) where {ζ1 , ζ2 , ζ3 } ⊂ R equals

λ∗ =

1 (ζ32 − ζ22 ) f (ζ1 ) + (ζ32 − ζ12 ) f (ζ2 ) + (ζ22 − ζ12 ) f (ζ3 ) , 2 (ζ3 − ζ2 ) f (ζ1 ) + (ζ3 − ζ1 ) f (ζ2 ) + (ζ2 − ζ1 ) f (ζ3 )

Let us transform quadratic q(x) = ax2 + bx + c; x = ζ1 + t(ζ3 − ζ1 );t = (−∞, ∞) to ˆ + cˆ assuming the following quadratic q(x) ˆ = ax ˆ 2 + bx 1. A transformation will be of the form q(x) ˆ = E(p(x)) + D,

p(x) = L2 (x),

(4.34)

where L2 (x) is the Lagrange interpolation parabola constructed on points ζ1 (0, f (ζ1 )), (ζ(r) , f (ζ2 )) and (1, f (ζ3 ) whereas ζ(r) = ζζ2 − . 3 −ζ1 2. a) If f (ζ1 ) > f (ζ3 ) b) if f (ζ1 ) < f (ζ3 )   q(0) ˆ = −1, q(0) ˆ = −2, q(1) ˆ = −2, q(1) ˆ = −1.

100

M. Bazan

From the condition 1. we get L2 (x) = a x2 + b x + c where a =

f (ζ3 ) f (ζ1 ) f (ζ2 ) + + , ζ(r) ζ(r) (ζ(r) − 1) 1 − ζ(r)

! f (ζ3 )ζ(r) f (ζ1 )(ζ(r) + 1) f (ζ2 ) + + , b =− ζ(r) ζ(r) (ζ(r) − 1) 1 − ζ(r)

c = f (ζ1 ). From the conditions 2. we get a)  1 E = a +b , c D = 2 − Ec = 2 − a +b .

b) 

1 E = − a +b , c D = 1 + Ec = 1 + a +b .

ˆ + cˆ we have aˆ = Ea , bˆ = Eb , cˆ = Ec + Then in the canonical form q(x) ˆ = ax ˆ 2 + bx D. The crucial properties of this transformation are q( ˆ ζ(r) ) < −1.   p ζ(r) = f (ζ2 ). q( ˆ ζ(r) ) does not depend on f (ζ2 ). For the minimum point λ ∗ of q such that λ ∗ ∈ [ζ1 , ζ3 ] we get a minimum point ∗ −ζ 1 λˆ ∗ = λζ − where λˆ ∗ is a minimum point of q. ˆ 3 ζ1 5. The transformation has a singularity when a = b .

1. 2. 3. 4.

This transformation reduces the number of free parameters from 6 to 3.

Appendix B B.1

Expression for Δ (ε ; ζ )

The unperturbed minimum λ ∗ (ζ ) is related to the perturbed minimum λ˜ l∗ (ε ; ζ ) as

λ˜ l∗ (ε ; ζ ) = λ ∗ (ζ ) ± Δl (ε ; ζ ). Here we assume that both λ ∗ (ζ ) and λ˜ l∗ (ε ; ζ ) are minima of the quadratics obtained by the transformation q(·) ˆ from Appendix A with the assumption that f (ζ1 ) < f (ζ3 ) i.e. if the opposite is true then we have to rotate the quadratics with respect to the center of the interval [ζ1 , ζ3 ]. Let us introduce a notation to simplify derivations. Let us denote A = 2(ζ(r) − 1), B = q( ˆ ζ(r) ),C = ζ(r) . Then for the unperturbed quadratic we have that

4

Search Procedure Exploiting Locally Regularized Objective Approximation

λ∗ =

101

1 A(ζ(r) + 1) + B − ζ(r)C 2 A + B −C

whereas for the perturbed one we have   1 A(ζ(r) + 1)(1 + ε ) + B − ζ(r)C ∗ λ1 (ε ; ζ ) = 2 A(1 + ε ) + B − C   A( ζ + 1 (r) 1) + B(1 + ε ) − ζ(r)C λ2∗ (ε ; ζ ) = 2 A + B(1 + ε ) − C   ζ + A( 1 (r) 1) + B − ζ(r)C(1 + ε ) λ3∗ (ε ; ζ ) = 2 A + B − C(1 + ε ) for the perturbation in 0, ζ(r) and 1 respectively where ζ(r) = the expression |λ ∗ − λ˜ ∗ (ε ; ζ )| to get

Δ1 (ε ; ζ ) =

ζ2 −ζ1 ζ3 −ζ1 . We

can simplify

  C − ζ(r) B Aε · 2(A + B − C) Aε + A + B − C

and Δ2 (ε ; ζ ) and Δ3 (ε ; ζ ) similarly.

B.2 The Main Condition If inequalities (4.21) are satisfied then we have a guarantee that λ ∗ (ζ ) < ζ2 . It is because if λ˜ l∗ (ε ; ζ ) is shifted by Δl (ε ; ζ ) to the left then shifting it back to the right will not make it greater than ζ2 . If it is shifted to the right then we have a margin of 2Δl (ε ; ζ ). As previously mentioned we will consider only perturbation in 0 and ζ(r) i.e. l = 1 and l = 2. For l = 1 we get A(ζ(r) + 1)(1 + ε ) + B − ζ(r)C < 2ζ2 A(1 + ε ) + B − C We have to consider two cases depending on the sign of the denominator. 1. If A(1 + ε ) + B − C) > 0 then we get A(1 − ζ(r))ε < A(ζ(r) − 1) + B(2ζ(r) − 1) − C(2ζ(r) − 1) which again is divided into two cases depending on the sign of the cooeficient by ε a. When A(1 − ζ(r)) > 0 then

ε<

A(ζ(r) − 1) + B(2ζ(r) − 1) − C(2ζ(r) − 1) . A(1 − ζ(r))

b. When A(1 − ζ(r)) < 0 then

102

M. Bazan

ε>

A(ζ(r) − 1) + B(2ζ(r) − 1) − C(2ζ(r) − 1) . A(1 − ζ(r))

2. If A(1 + ε ) + B − C) < 0 then we get two cases a. When A(1 − ζ(r)) > 0 then

ε>

A(ζ(r) − 1) + B(2ζ(r) − 1) − C(2ζ(r) − 1) . A(1 − ζ(r))

b. When A(1 − ζ(r)) < 0 then

ε<

A(ζ(r) − 1) + B(2ζ(r) − 1) − C(2ζ(r) − 1) . A(1 − ζ(r))

Only upper bounds for ε , i.e. 1.a) and 2.b), are interpretable as a solution to our problem. So finally we get two regions for ε and ζ(r) where

ε< for

or

A(ζ(r) − 1) + B(2ζ(r) − 1) − C(2ζ(r) − 1) , A(1 − ζ(r))  

(4.35)

A(1 + ε ) + B − C > 0 , A(1 − ζ(r)) > 0

(4.36)

A(1 + ε ) + B − C < 0 . A(1 − ζ(r)) < 0

(4.37)

In the same way we can obtain conditions for l = 2:

ε< for

or

 

−B(1 − ζ(r)) − A , B(1 − ζ(r))

(4.38)

A + B(1 + ε ) − C > 0 , B(1 − ζ(r)) < 0

(4.39)

A + B(1 + ε ) − C < 0 . B(1 − ζ(r)) > 0

(4.40)

References 1. Bazan, M., Russenschuck, S.: Using neural networks to speed up optimization algorithms. Eur. Phys. J. AP 12, 109–115 (2000) 2. Bazan, M., Aleksa, M., Russenschuck, S.: An improved method using radial basis function neural networks to speed up optimization algorithms. IEEE Trans. on Magnetics 38, 1081–1084 (2002)

4

Search Procedure Exploiting Locally Regularized Objective Approximation

103

3. Bazan, M., Aleksa, M., Lucas, J., Russenschuck, S., Ramberger, S., V¨ollinger, C.: Integrated design of superconducting magnets with the CERN field computation program ROXIE. In: Proc. 6th International Computational Accelarator Physics Conference, Darmstadt, Germany (September 2000) 4. Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust region methods. SIAM, Philadelphia (2005) 5. Hansen, P.C.: Rank-deficient and Discrete Ill-posed Problems. SIAM, Philadelphia (1998) 6. Jacob, H.G.: Rechnergest¨utzte Optimierung statischer und dynamischer Systeme. Springer, Heidelberg (1982) 7. Kansa, E.J., Hon, Y.C.: Circumventing the ill-conditionning problem with multiquadratic radial basis: Applications to elliptic partial differentail equations. Comp. Math. with App. 39(7-8), 123–137 (2000) 8. Luenberger, D.G.: Introduction to linear and nonlinear programming, 2nd edn. AddisonWesley, New York (1984) 9. Micchelli, C.A.: Interpolation of Scattered Data: Distance Matrices and Conditionally Positive Definite Functions. Constructive Approximation 2, 11–22 (1986) 10. Madych, W.R., Nelson, S.A.: Multivariate interpolation and conditionally positive definite functions II. Math. Comp. 4(189), 211–230 (1990) 11. Madych, W.R.: Miscellaneous error bounds for multiquadric and related interpolators. Comp. Math. with Appl. 24(12), 121–138 (1992) 12. Mor´e, J.J., Garbow, B.S., Hillstorm, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Software 7(1), 17–41 (1981) 13. Oeuvray, R.: Trust region method based on radial basis functions with application on biomedical imaging, Ecole Polytechnique Federale de Lausanne (2005) 14. Polak, E.: Optimization. Algorithms and Consistent Approximations. Applied Mathematical Sciences, vol. 124. Springer, Heidelberg (1997) 15. Powell, M.J.D.: On calculation of orthogonal vectors. The Computer Journal 11(3), 302–304 (1968) 16. Schaback, R.: Error estimates and condition number for radial basis function interpolation. Adv. Comput. Math. 3, 251–264 (1995) 17. Schaback, R.: Native Hilbert Spaces for Radial Basis Functions I. The new development in Approximation Theory. Birkh¨auser, Basel (1999) 18. Wahba, G.: Spline models for obsevational data. SIAM, Philadelphia (1990) 19. Wendland, H., Rieger, C.: Approximate interpolation with applications to selecting smoothing parameters. Numerische Mathematik 101, 643–662 (2005) 20. Wendland, H.: Gaussian Interpolation Revisited. In: Kopotun, K., Lyche, T., Neamtu, M. (eds.) Trends in Approximation Theory, pp. 427–436. Vanderbilt University Press, Nashville (2001) 21. Zangwill, W.I.: Nonlinear Programming; a Unified Approach. Prentice-Hall International Series. Prentice-Hall, Englewood Cliffs (1969)

Chapter 5

Optimization Problems with Cardinality Constraints Rub´en Ruiz-Torrubiano, Sergio Garc´ıa-Moratilla, and Alberto Su´arez

Abstract. In this article we review several hybrid techniques that can be used to accurately and efficiently solve large optimization problems with cardinality constraints. Exact methods, such as branch-and-bound, require lengthy computations and are, for this reason, infeasible in practice. As an alternative, this study focuses on approximate techniques that can identify near-optimal solutions at a reduced computational cost. Most of the methods considered encode the candidate solutions as sets. This representation, when used in conjunction with specially devised search operators, is specially suited to problems whose solution involves the selection of optimal subsets of specified cardinality. The performance of these techniques is illustrated in optimization problems of practical interest that arise in the fields of machine learning (pruning of ensembles of classifiers), quantitative finance (portfolio selection), time-series modeling (index tracking) and statistical data analysis (sparse principal component analysis).

5.1 Introduction Many practical optimization problems involve the selection of subsets of specified cardinality from a collection of items. These problems can be solved by exhaustive enumeration of all the candidate solutions of the specified cardinality. In practice, only small problems of this type can be exactly solved within a reasonable amount of Rub´en Ruiz-Torrubiano Computer Science Department, Universidad Aut´onoma de Madrid, Spain e-mail: [email protected] Sergio Garc´ıa-Moratilla Computer Science Department, Universidad Aut´onoma de Madrid, Spain e-mail: [email protected] Alberto Su´arez Computer Science Department, Universidad Aut´onoma de Madrid, Spain e-mail: [email protected] Y. Tenne and C.-K. Goh (Eds.): Computational Intelligence in Optimization, ALO 7, pp. 105–130. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

106

R. Ruiz-Torrubiano, S. Garc´ıa-Moratilla, and A. Su´arez

time. The number of steps required to find the optimal solution can be reduced using branch-and-bound techniques. Nevertheless, the computational complexity of the search remains exponential, which means that large problems cannot be handled by these exact methods. It is therefore important to design algorithms that can identify near-optimal solutions at a reduced computational cost. In this article we present an unified framework for handling optimization problems with cardinality constraints. A number of approximate methods within this framework are analyzed and their performance is tested in extensive benchmark experiments. In its general form, an optimization problem with cardinality constraints can be formulated in terms of a vector of binary variables z = {z1 , z2 , . . . , zD }, zi ∈ {0, 1}. The goal is to minimize a cost function that depends on z, subject to a constraint that specifies the number of non-zero bits in z D

∑ zi = k.

min {F(z)} z

(5.1)

i=1

Optimization problems with cardinality constraints given by an inequality ∑D i=1 zi ≤ K can be solved by selecting the best of the solutions of K optimization problems with the equality constraint ∑D i=1 zi = k; k = 1, 2, . . . , K. Finally, the solution of a combinatorial optimization problem without restrictions can be obtained by solving the sequence of problems with cardinality constraints ∑D i=1 zi = k; k = 1, 2, . . . , D. Continuous optimization tasks with cardinality constraints can also be analyzed within this framework. Consider the problem of minimizing a function that depends on a D-dimensional continuous parameter θ . We search for solutions with exactly k non-zero components of θ min {F(θ )} θ

θ ∈ RD ,

D

∑ I(θi = 0) = k,

(5.2)

i=1

where I(·) is an indicator function (I(true) = 1, I(false) = 0). This hybrid problem can be transformed into a purely combinatorial one of the type (5.1) by introducing a D-dimensional binary vector z whose i-th component indicates whether variable i is allowed to take a non-zero value (zi = 1) or is set to zero (zi = 0). min [F ∗ (z)] , z

∑ zi = k.

(5.3)

i

The function F ∗ (z) is the solution of an auxiliary continuous optimization problem in the reduced space defined by z F ∗ (z) = min F(θ [z] ), [θ |z]

(5.4)

5

Optimization Problems with Cardinality Constraints

107

where θ [z] denotes the k-dimensional vector formed by the components of θ for which the value of the corresponding component of z is 1. The remaining components of θ are set to zero in the auxiliary problem. This decomposition makes it clear how hybrid methods that combine techniques for combinatorial and continuous optimization can be applied to identify the solution of the subset selection problem with a continuous objective function: For a given value of z, the optimal θ [z] is calculated by solving the surrogate problem defined by (5.4), where z determines which components of θ are allowed to take a non-zero value. The final solution is obtained by searching in the purely combinatorial space of possible values of z, using the optimal function value that is a solution of (5.4) to guide the exploration. The success of this hybrid approach depends on the availability of a continuous optimization algorithm that can efficiently identify the globally optimal solution of the auxiliary optimization problem defined in (5.4) and on the efficiency of the algorithm used to address the combinatorial part of the search. For simple forms of the continuous objective function and of the remaining restrictions (other than the cardinality constraint), the auxiliary problem can be efficiently solved by exact optimization techniques. For instance, efficient linear and quadratic programming algorithms are available if the function is linear or quadratic, respectively [1]. For more complex objective functions, general non-linear optimization techniques (such as quasi-Newton [2] or interior-point methods [3]) may be necessary. In these cases, there is no guarantee that the solution of the auxiliary problem be globally optimal. As a consequence, if the solutions found are far from the global optimum, the combinatorial search that is used to solve the original problem (5.3) can be seriously misled. In this work, we assume that the continuous optimization task defined by (5.4) can be solved exactly and focus on the solution of the combinatorial part of the original problem. Section 5.2 describes how standard combinatorial optimization techniques can be adapted to handle the cardinality constraints considered. Emphasis is placed on the use of an appropriate encoding for the search states in terms of sets. This set-based encoding is particularly well-suited for the definition of search operators that preserve the cardinality of the candidate solutions. With this adaptation, the approximate methods described provide a practicable alternative to identifying the exact solution by exhaustive search, which becomes computationally infeasible in large problems, or to computationally inexpensive optimization methods, such as greedy search, which tend to find suboptimal solutions. The experiments presented in Section 5.3 illustrate how the techniques reviewed find near-optimal solutions with limited computational resources and can therefore be used to address optimal subset selection problems of practical interest. Novel results regarding the application of these techniques to some of these problems (ensemble pruning and sparse PCA) are also provided. Finally, Section 8.5 summarizes the conclusions of this work.

108

R. Ruiz-Torrubiano, S. Garc´ıa-Moratilla, and A. Su´arez

5.2 Approximate Methods for the Solution of Optimization Problems with Cardinality Constrains In this section we describe how simulated annealing (SA), genetic algorithms (GA), and estimation of distribution algorithms (EDA) can be used to solve large optimization problems with cardinality constraints. They are stochastic search methods that involve the generation of candidate solutions, which are then rejected or selected according to their performance. In their standard formulation, no particular consideration is given to the number of non-zero components in the candidate solutions generated. Cardinality constraints can be taken into account using one of the following approaches: (i) No candidate solution violating the constraint is generated at any time by the algorithm. Enforcing this property requires the design of appropriate genetic and neighborhood operators, such that the space of solutions of a given cardinality is closed under these search operators [4] [5]. (ii) Solutions that violate the cardinality constraint can be generated by the successor operators. Whenever a violation occurs, a repair algorithm is applied to transform the infeasible solution into a solution of the desired cardinality. Typically, a local search is used to obtain the closest feasible solution, but random repair mechanisms can be used as well [6] [7]. (iii) Solutions that violate the cardinality constraint can be generated by the successor operators. In contrast with the previous approach, infeasible solutions are not repaired. Instead, a penalty term is introduced on the evaluation function so that infeasible candidate solutions have worse scores than feasible ones with an equivalent performance [6]. In the experiments described in Section 5.3, the best overall performance is obtained by methods that use a set-based representation together with appropriately designed successor operators that preserve the cardinality of the solutions. These results underscore the importance of using a representation that properly reflects the structure of the problem. Therefore, the focus of this study is on the design of search operators that preserve the cardinality of the candidate solutions. These specially adapted methods are generally preferable to standard schemes that take into account the restrictions by either ad-hoc repair mechanisms or by including a term in the cost function that penalizes violations of the constraints.

5.2.1 Simulated Annealing Simulated annealing (SA) is an optimization technique inspired by the field of thermodynamics [8]. The main idea is to mimic the physical process of melting a solid and then cooling it to allow the formation of a regular crystalline structure that attains a minimum of the system’s free energy. In simulated annealing the function to be minimized F(z) (objective or cost function) takes the role of the free energy in the physical system. The physical configuration space is replaced by the space of candidate solutions, which are connected by transitions defined by a neighborhood

5

Optimization Problems with Cardinality Constraints

109

operator. The stochastic search proceeds by considering transitions from the current state z(cur) to a neighboring configuration zl ∈ N (z(cur) ) generated at random. The proposed transition is accepted if the value of the objective function decreases. Otherwise, if the candidate configuration is of higher cost, the transition is accepted only with a certain probability. This probability is expressed as a Boltzmann factor   (cur) ) − F(z ) F(z l , (5.5) Paccept (zl , z(cur) ; Tk ) = exp − Tk where the parameter Tk plays the role of a temperature. A general version of this technique is given as Algorithm 1. In this pseudocode, the function annealingSchedule returns the temperature Tk for the following epoch. It is common to use a geometrical schedule Tk = γ Tk−1 , where γ smaller, but usually close to one, regulates how fast the temperature is decreased.

Algorithm 1. Simulated annealing • • • •

Generate initial configuration z(0) and initial temperature T0 z(cur) ← z(0) i←0 While convergence criteria are not met [Annealing loop] – – – –

i ← i+1 Fix temperature for epoch i: Ti = annealingSchedule(Ti−1 ) Fix length for epoch i: Li For l = 1, . . . , Li [Epoch loop] 1. Select randomly an element zl ∈ N (z(cur) ). 2. If F(zl ) < F(z(cur) ), then z(cur) ← zl 3. Else, generate u ∼ U[0, 1] If u < Paccept (zl , z(cur) ; Ti ), then z(cur) ← zl

• Return the best value found.

Cardinality constraints can be handled in SA by selecting an appropriate encoding for z and a corresponding neighborhood N (z). In particular, the candidate solutions can be encoded as sets of specified cardinality. The components of the binary vector z are then interpreted as indicating membership to the set: if zi = 1, the ith element is included in the solution. Otherwise, if zi = 0 it is excluded from the selection. It is also necessary to design a neighborhood operator that preserves the cardinality constraints, so that no penalty or repair mechanisms are needed. A simple design is to exchange an element included in the current candidate solution with an element excluded from it. This is the version of SA that will be used in Section 5.3.

110

R. Ruiz-Torrubiano, S. Garc´ıa-Moratilla, and A. Su´arez

5.2.2 Genetic Algorithms Genetic algorithms are a class of optimization methods that mimic the process of the natural evolution [9]. Optimization is achieved by selection from a population that exhibits some random variability. The outline of a general genetic algorithm is shown in Algorithm 2.

Algorithm 2. Genetic Algorithm • Generate an initial population P0 with P individuals. • For each individual I j ∈ P0 , calculate fitness Φ (I j ). • Initialize the generation counter t ← 0. • While convergence criteria are not met: – Increase the generation counter t ← t + 1. – Select a parent set Πt ⊂ Pt composed of nP individuals from the population. / – While Πt = 0: · Extract two individuals I1 and I2 from Πt . · Apply the crossover operator Θ (I1 , I2 ) and generate nC children (with probability pC ). · Apply the mutation operator to the nC children (with probability pM ). – Calculate the fitness value of the new individuals. – Add the new individuals to the population. – Select P individuals that make up Pt+1 , the population for generation t + 1.

For problems with cardinality constraints, two alternative encodings for the candidate solutions are considered. A first possibility is a standard binary representation, where the chromosomes are bit-strings. The difficulty with this encoding is that standard mutation and crossover operators do not preserve the number of nonzero bits of the parents. A possible solution to this problem is to assign a lower fitness value to individuals in the population that violate the cardinality constraint. Assuming that a problem with an inequality cardinality constraint is considered, a penalized fitness function can be built by subtracting from the standard fitness function a penalty term that depends on the magnitude of the violation of the cardinality constraint Δk (z) = |Card(z) − k| (5.6) The penalized fitness function is

Φ p (z) = Φ (z) − β f p (Δk (z)),

(5.7)

where f p : N → R+ is a monotonically increasing function of Δk (z) and β ≥ 0 represents the strength of the penalty. Another option is to repair infeasible individuals when they are generated. Several repair mechanisms can be defined for this purpose. For instance, an individual

5

Optimization Problems with Cardinality Constraints

111

can be repaired by randomly setting some bits to 0 or 1, as needed, until the cardinality constraint is satisfied (random repair). Another alternative is to use a heuristic to determine which bits must be set to 0 or to 1 (heuristic repair). The results of a greedy optimization or the solutions of a relaxed version of the problem can also be used to achieve this objective [10]. An alternative to binary encoding is to use the set representation introduced in simulated annealing. The use of this representation simplifies the design of crossover and mutation operators that preserve the cardinality of the individuals. The neighboring operator defined in SA can be used to construct mutated individuals. Since this operator swaps a variable in the set of selected variables with another variable in the complement of this set, the cardinality of the original chromosome is preserved by the mutation. Some crossover operators on sets were introduced in [5]. They are defined taking into account the properties of respect and assortment [11]. Respect ensures that the offspring inherit the common genetic material of the parents. Assortment guarantees that every combination of the alleles of the two parents is possible in the child, provided that these alleles are compatible. When cardinality constraints are considered, it is no longer possible to design crossover operators that guarantee both respect and assortment. A crossover operation that provides a good balance of these properties and ensures that the cardinality of the parents is preserved in the offspring is random assorting recombination (RAR). RAR crossover is described in Algorithm 3. In this algorithm, the integer parameter w ≥ 0 determines the amount of common information from both parents that is retained by the offspring. For w = 0, elements that are present in the chromosomes of both parents are not allowed in the child. Higher values of w assign more importance to the elements in the intersection of the parents’ sets (chromosomes). In the limit w → ∞, the child contains every element that is in both of the parents’ chromosomes with a probability that approaches 1.

5.2.3 Estimation of Distribution Algorithms Estimation of distribution algorithms (EDAs) are a class of evolutionary methods in which diversity is generated by a probabilistic sampling scheme [12]. Depending on the nature of this sampling scheme, different variants of EDAs can be designed. In this work, we consider the Population Based Incremental Learning (PBIL) algorithm as a representative algorithm of the EDA family [13]. It operates on binary chromosomes of fixed length (z) and assumes statistical independence among the genes {zi ; i = 1, 2, . . . , D}. In generation g, the genotype of the population is characterized by the probability vector p(g) , whose ith component is the probability of assigning the value 1 to the gene in the ith position. The update of the probability distribution using DSe g (see Algorithm 4) in PBIL is p(g+1) = α

1 M (gim ) ∑ z + (1 − α )p(g), M m=1

(5.8)

112

R. Ruiz-Torrubiano, S. Garc´ıa-Moratilla, and A. Su´arez

Algorithm 3. Random Assortment Recombination algorithm 1 2

Input: Two parents I1 and I2 , and a fixed cardinality k. Output: A child chromosome Θ . • Create auxiliary sets A, B,C, D, E: – – – –

A = elements present in both parents. B = elements not present in any of the parents. C ≡ D elements present in only one parent. E = 0. /

• Build set G = {w copies of elements from A and B, and 1 copy of elements in C and D} / • While |Θ | < k and G = 0: – Extract g ∈ G without replacement. – If g ∈ A or g ∈ C, and g ∈ / E, Θ = Θ ∪ {g}. – If g ∈ B or g ∈ D, E = E ∪ {g}. • If |Θ | < k, add elements chosen at random from U − Θ until chromosome is complete.

Algorithm 4. Estimation of distribution algorithm(EDA) • Initialize the distribution that characterizes the population P(0) (z) • Initialize generation counter g ← 0. • While convergence criteria are not met – Sample population of P individuals using P(g) (z) Dg = {z(g1) , . . . , z(gP) } – Sort population by non-increasing fitness value

Dg = {z(gi1 )) , z(gi2 ) , . . . , z(giP ) }, where i1 , i2 , . . . , iP is a reordering of the indices 1, 2, . . . , P such that

Φ (z(gi1 ) ) ≥ Φ (z(gi2 ) ) ≥ · · · ≥ Φ (z(giP ) ) – Select the first M ≤ P individuals from the sorted population (gi1 ) (gi2 ) DSe ,z , . . . , z(giM ) } g = {z

– Estimate the new probability P(g+1) (z) distribution using DSe g – Update generation counter g ← g + 1 • Return the best solution found.

5

Optimization Problems with Cardinality Constraints

113

where z(gim ) represents the individual in the im −th position on generation g, and α ∈ (0, 1] is a smoothing parameter included to avoid strong fluctuations in the estimates of the probability distribution. Individuals are sorted by decreasing fitness values. The Univariate Marginal Distribution Algorithm (UMDA [14]) algorithm from the EDA family is recovered when α = 1. Even though the encoding is binary, the cardinality constraints can be enforced in the sampling of individuals. Algorithm 5 describes a sampling method that generates individuals of a specified cardinality k from a distribution of bits characterized by the probability vector p. The application of this method to sample new individuals guarantees that the algorithm is closed with respect to the cardinality constraint.

Algorithm 5. Sampling individuals of a specified cardinality from p. • Initialize pˆ ← p • Initialize individual x = 0. • For i = 1, 2, . . . , k – – – – –

Generate a random number u ∼ U[0, 1] j−1 pˆi < u ≤ ∑D Determine the value of j such that ∑i=1 i= j+1 pˆi . Set x j = 1. Update the value pˆ j ← 0 Renormalize pˆi pˆi ← D , i = 1, 2, . . . , D, ∑k=1 pˆk

so that pˆ can be interpreted as a probability vector ∑D i=1 pˆi = 1. – Return the generated individual x.

5.3 Benchmark Optimization Problems with Cardinality Constraints This section introduces a collection of optimization problems with cardinality constraints that are used to illustrate the application of the methods described in the previous section. These include standard combinatorial optimization problems, such as the knapsack problem, and real-world problems that arise in the fields of machine learning (ensemble pruning), quantitative finance (cardinality-constrained portfolio optimization), time series modeling (index tracking by partial replication) and statistical data analysis (sparse principal components analysis). The problems considered are either purely combinatorial or involve the optimization of continuous parameters. In a purely combinatorial optimization problem F(z) can be directly evaluated once z is known. The knapsack problem and ensemble pruning are of this type. The remaining problems considered (portfolio selection, index tracking and sparse PCA) are hybrid optimization tasks, in which the evaluation of F(z) for a fixed value of the binary vector z requires the solution of an auxiliary continuous optimization

114

R. Ruiz-Torrubiano, S. Garc´ıa-Moratilla, and A. Su´arez

problem. While it is possible to address the combinatorial and the continuous optimization problems simultaneously, we concentrate on strategies that handle these aspects separately. Therefore, the outcome of the continuous optimization algorithm is used to guide the combinatorial optimization search, as in (5.4). For the hybrid problems considered, the secondary optimization task can be efficiently solved in an exact manner by quadratic programming. Nonetheless, the scheme can be directly generalized when the evaluation of F(z) requires a more complex programming solution, possibly without guarantee of convergence to the global solution of the surrogate optimization problem. Under these conditions the algorithm used to address the combinatorial part can actually be misled by the suboptimal solutions found in the auxiliary problem.

5.3.1 The Knapsack Problem Knapsack problems are a family of combinatorial optimization problems that involve selecting a subset from a pool of items [15]. In this work, we consider the 0/1 knapsack problem, which can be shown to be NP-complete [16]. In an instance of the 0/1 knapsack D items are available to fill up a knapsack. A profit pi and a weight wi are associated to the i-th item, i = 1, 2, . . . , D. The objective is to identify the subset of items whose accumulated profit is maximum and whose overall weight does not exceed a given capacity W D

max

∑ pi zi

i=1

D

s.t. ∑ wi zi ≤ W zi ∈ {0, 1} , i = 1, 2, . . . , D.

(5.9)

i=1

Both exact and approximate methods have been used to address the 0/1 knapsack problem. Exact algorithms based on branch-and-bound approaches and dynamic programming are reviewed in [17]. Genetic algorithms [18, 19] and EDAs [12] have also been used to address this problem. Cardinality constraints are generally not considered in the standard 0/1 knapsack problem. Nevertheless, the optimum of the unconstrained problem can be obtained by solving D cardinality-constrained knapsack problems ∑D i=1 zi = k; k = 1, 2, . . . D. The k-th element in this sequence is a knapsack problem with the restriction that only k items can be included in the knapsack. To compare the performance of the different optimization methods analyzed in this work, we use the testing protocol proposed in [20] [18]. Three types of problems, defined in terms of two parameters v, r ∈ R+ , v > 1, are considered: (1) Uncorrelated: Weights and profits are generated randomly in [1, v]. (2) Weakly correlated: Weights are generated randomly in [1, v] and profits are generated in the interval [wi − r, wi + r]. (3) Strongly correlated: Weights are generated randomly in [1, v] and profit pi = wi + r. In general, knapsack problems with correlations between weights and profits are more difficult to solve than problems in which the weights and profits are independent. We

5

Optimization Problems with Cardinality Constraints

115

use v = 10, r = 5 and a capacity W = 2v, which tends to include very few items in the solution. The results reported are averages over 25 realizations of each problem, which are solved using the different approximate methods: SA, a standard GA with linear penalty, a GA using set encoding and the RAR operator (w = 1), and PBIL. The conditions under which the search is conducted are determined on exploratory experiments. A geometric annealing schedule Tk = γ Tk−1 with γ = 0.9 is used in SA. The GAs evolve populations composed of 100 individuals. The probabilities of crossover and mutation are pc = 1, pm = 10−2 , respectively. In PBIL, a population composed of 1000 individuals is used. The probability distribution is updated using 10% of the individuals. The smoothing parameter α is 0.1. Exact results obtained with the solver SYMPHONY from the COIN-OR project [21] implementing a branch-andcut (B&C) approach [22], are also reported for reference. In the strongly correlated problems it was not possible to find the exact solutions within a reasonable amount of time. Table 5.1 Results for the 0-1 Knapsack problem with restrictive capacity

Corr. No.Items

none none none weak weak weak strong strong strong

100 250 500 100 250 500 100 250 500

Algorithm GA Lin. GA RAR SA PBIL Profit Time Profit Time Profit Time Profit Time 79.36 26.2 82.09 54.5 80.70 98.4 81.89 24.1 90.63 38.1 105.34 134.9 102.91 284.4 104.51 47.4 95.93 57.2 119.88 261.9 118.07 531.7 117.28 91.5 52.97 26.9 54.38 53.9 53.53 99.87 54.33 24.2 59.07 38.3 66.24 130.4 65.13 286.4 65.85 47.7 60.40 56.1 74.17 266.1 73.40 531.9 72.05 87.9 76.19 26.2 79.77 57.8 79.73 98.7 78.99 24.0 83.98 37.8 94.20 139.5 94.15 286.0 92.39 47.3 84.52 55.3 101.40 272.2 102.16 525.6 96.60 86.9

B&C (exact) Profit Time 82.11 62.0 106.43 178.7 123.93 568.8 54.43 81.8 67.10 180.3 76.61 560.1 − − − − − −

Table 5.1 displays the average profit obtained and the time (in seconds) to reach a solution for each method. The experiments were performed on an AMD Turion computer with 1.79 Ghz processor speed and 1 Gb RAM. None of the approximate methods reaches the optimal profit, which is calculated using an exact branch-andcut method. The highest profit obtained by an approximate optimization is highlighted in boldface. In all cases, the algorithms that use a set encoding (GA with RAR crossover and SA) exhibit the best performance. They also require longer times to reach a solution, specially SA. PBIL obtains good results only in small uncorrelated knapsack problems. This is explained by the fact that the sampling and estimation of probability distributions becomes progressively more difficult as the dimensionality of the problem increases. Furthermore, PBIL assumes statistical independence between the variables, which makes the algorithm perform worse on problems in which correlations are present. The standard GA with linear penalty has a very poor performance in all the knapsack problems analyzed.

116

R. Ruiz-Torrubiano, S. Garc´ıa-Moratilla, and A. Su´arez

5.3.2 Ensemble Pruning Consider the problem of automatic induction of classifiers from a collection of instances {(xn , yn ); n = 1, 2, . . . , N}, where yn is the class label of the example characterized by the vector of attributes xn . The goal is to induce from these data an autonomous system that accurately predicts the class label on the basis of the vector of attributes of a previously unseen instance. There are a number of algorithms that can be used for learning different types of classifiers: decision trees, neural networks, support vector machines, etc. In practice, one of the most successful paradigms is ensemble learning [23]. Ensembles are composed of a diverse collection of classifiers that are generated from the same training data by introducing variations in the algorithm used for induction or in the conditions under which learning takes place. The outputs of the individual classifiers are then combined (for instance, by majority voting) to produce the prediction of the ensemble. Pooling the decisions of the ensemble members has the potential of improving the generalization capacity of a single learner. However, ensembles are costly to generate and have large storage requirements. Furthermore, the time required to classify an unlabeled instance increases linearly with the size of the ensemble. Recent work has shown that the storage requirements and classification times can be significantly reduced by selecting a subset of classifiers whose generalization capacity is equivalent and sometimes superior to the original complete ensemble. This process receives the name of ensemble pruning [24], selection, [25] or thinning [26]. Ensemble pruning has been a subject of great interest in the recent literature on machine learning (see refs. in [27]). Most studies focus on the definition of appropriate quantities that can be optimized on the training set to obtain pruned ensembles with good generalization performance. The individual properties of classifiers are not useful to guide the selection process. The generalization capacity of the pruned ensemble crucially depends on the complementarity of the classifiers that are part of it. The search in the space of subensembles is usually greedy. A notable exception is [28, 29, 30], where genetic algorithms, generally with real-valued chromosomes, are used. Another exception is [31], where ensemble pruning is formulated as a quadratic integer programming problem. Consider an ensemble composed of D classifiers and a set of labeled instances. Define a matrix G, whose element Gi j is the number of common errors between classifier i and classifier j, where i, j = 1, 2, . . . , D. The value of the diagonal term Gii is the number of errors made by classifier i. The matrix is then symmetrized and its elements normalized so that they are in the same scale   Gii 1 Gi j G ji ˜ ˜ , Gi j,i= j = Gii = + . (5.10) N 2 Gii G j j Intuitively, ∑i G˜ ii measures the overall strength of the ensemble classifiers and ∑i j,i= j G˜ i j measures their diversity. The subensemble selection problem of size k can now be formulated as a quadratic integer programming problem

5

Optimization Problems with Cardinality Constraints

˜ · z, argmin zT · G z

D

s.t. ∑ zi = k,

117

zi ∈ {0, 1}.

(5.11)

i=1

The binary variable zi indicates whether classifier i should be selected. The size of the pruned ensemble, k, is specified beforehand. The selection process is a combinatorial optimization problem whose exact requires the evaluation of the  solution  D performance of the exponentially large subensembles of size k that can be k extracted from an ensemble of size D. In [31] the solution is approximated in polynomial time by applying semi-definite programming (SDP) to a convex relaxation of the original problem. To investigate the performance in the ensemble pruning problem of the optimization methods described in Section 5.2, we generate bagging ensembles for five representative benchmark problems from the UCI repository: heart, pima, satellite, waveform and wdbc (Breast Cancer Wisconsin) [32]. The individual classifiers in the ensemble are trained on different bootstrap samples of the original data [33]. If the classifiers used as base learners are unstable the fluctuations in the bootstrap sample lead to the induction of different predictors. Assuming that the errors of these classifiers are uncorrelated, pooling their decisions by majority voting should improve the accuracy of the predictions. In the experiments performed, bagging ensembles of 101 CART trees are built [34]. The original ensemble is pruned to k = 21 decision trees. The strength-diversity measure G, the time consumed in seconds and the number of evaluations are averaged over 5 ten-fold cross-validations for heart, pima, satellite and wdbc, and over 50 independent partitions for waveform. The success rate is the average over 50 repetitions of the optimization for a given partition of the data into training and testing sets. The parameters for the metaheuristic optimization methods are determined in exploratory experiments using the results of SDP as a gauge. For the GAs, populations with 100 individuals are evolved using a steady state generational substitution scheme. The crossover probability is set to 1. The mutation probability is 10−2 for GAs with binary representation and 10−3 for GAs with set representation. The strength of the penalty term in the GA with linear penalties is β = 400. If the best individual of the final population does not satisfy the cardinality constraint, a greedy search is performed to fulfill the restriction. The value w = 1 is used in RAR-GA. A geometric annealing schedules with γ = 0.9 is used in SA. In these experiments, the best solution in 10 independent executions of the SA algorithm is chosen. For PBIL, a population of 1000 individuals is generated, where 10% of the individuals are used to update the probability distribution. The smoothing constant is set to α = 0.1. The results of the ensemble pruning experiments performed are summarized in Table 5.2. Most of the optimization methods analyzed reach similar solutions in all the classification problems considered, with the exception of the standard GA with linear penalty, which obtains the worst values of the objective function. In terms of this quantity, the best overall results correspond to SA and SDP. In terms of efficiency, SDP should be preferred. In machine learning, the relevant measure of performance is the generalization capacity of the classifiers generated. The test

118

R. Ruiz-Torrubiano, S. Garc´ıa-Moratilla, and A. Su´arez

Table 5.2 Results for the GA, SA and EDA approaches in the ensemble pruning problem

Algorithm

Problem

heart pima SA satellite waveform wdbc heart pima GA satellite Linear Penalty waveform wdbc heart pima GA Heuristic Repair satellite waveform wdbc heart pima GA satellite RAR (w = 1) waveform wdbc heart pima PBIL satellite waveform wdbc heart pima SDP satellite waveform wdbc

Best G Success Time (s) Test Error rate 156.2940 1.00 7.575 18.06 234.8931 0.98 14.644 23.99 185.1163 1.00 63.490 13.15 105.2465 1.00 37.621 19.64 121.9183 0.88 34.085 4.50 157.8668 0.98 2.017 17.96 235.0572 0.92 2.072 23.86 186.0706 1.00 0.870 12.85 105.8294 0.02 5.179 19.70 122.7625 0.06 4.963 4.45 156.3127 1.00 0.851 17.87 234.8931 1.00 1.665 23.99 185.1163 1.00 0.520 13.15 105.2465 0.80 8.168 19.64 121.9183 0.90 7.875 4.50 156.3860 1.00 0.697 17.96 234.9190 1.00 1.381 24.09 185.1163 1.00 0.910 13.15 105.2510 0.48 6.880 19.62 121.9399 0.40 6.449 4.50 156.4111 1.00 38.409 17.69 234.9358 0.96 38.426 24.05 185.1163 1.00 16.400 13.15 105.2663 0.36 16.086 19.67 122.0467 0.34 38.392 4.34 156.3034 1.00 1.137 18.15 234.8956 1.00 1.159 24.09 185.1163 1.00 1.230 13.15 104.9984 0.90 1.230 19.60 121.9143 0.90 1.117 4.39

error displayed in the last column of the table provides an estimate of the error rate in examples that have not been used to train the classifiers. Lower test errors indicate better generalization capacity. According to this measure the ranking of methods is rather different: classifiers that were optimal according to the objective function are suboptimal in terms of their generalization capacity. This indicates that the learning process is affected by overfitting, because the objective function is estimated on the training data. Nevertheless, the generalization performance of the pruned ensembles is very similar for all the optimization methods considered. Table 5.3 shows the test error of a single CART tree, of a complete bagging ensemble and the range of values

5

Optimization Problems with Cardinality Constraints

119

Table 5.3 Test errors for CART, standard bagging and pruned bagging

Problem heart pima satellite waveform wdbc

CART 23.63 24.84 13.80 30.27 7.28

Bagging 21.48 24.67 14.25 22.53 5.68

Pruned bagging [17.69,18.15] [23.86,24.09] [12.85,13.15] [19.62,19.67] [4.34,4.50]

of the test error obtained by pruned bagging ensembles of size k = 21. In all the classification problems considered, pruned ensembles have a lower test error than CART and complete bagging.

5.3.3 Portfolio Optimization with Cardinality Constraints The selection of optimal investment portfolios is a problem of great interest in the area of quantitative finance and has attracted much attention in the scientific community (see refs. in [10]). It is a multiobjective optimization task with two opposed goals: The maximization of profit and the minimization of risk. Several methods have been proposed to address this problem, mostly within the classical meanvariance model developed by H. Markowitz [35]. In this framework, the returns of the assets considered for investment are modeled as white noise. Profit is quantified in terms of the expected return of the portfolio. The variance of the portfolio returns is used as a measure of risk. In its simplest version, the problem can be solved by quadratic programming [1]. However, if cardinality constraints are included, the problem becomes a mixed-integer quadratic problem, which can be shown to be NP-Complete [10] T

min

w[z] · Σ[z,z] · w[z]

s.t.

w[z] · r¯ [z] = R∗ a[z] ≤ w[z] ≤ b[z] , a[z] ≥ 0, b[z] ≥ 0

(5.13) (5.14)

l ≤ A[z] · w[z] ≤ u

(5.15)

z ·1 ≤ K wT · 1 = 1, w ≥ 0.

(5.16) (5.17)

z

T

T

(5.12)

The inputs of the algorithm are r¯ , the vector of expected asset returns and Σ, the covariance matrix of the asset returns. The goal is to determine the optimal weights of the assets in the portfolio; i.e. the value of w that maximizes the variance of the portfolio returns (5.12), for a given value of the expected return of the portfolio, R∗ (5.13). The elements of the binary vector z specify whether asset i is included in the final portfolio (zi = 1) or not (zi = 0). Column vectors x[z] are obtained by removing from the corresponding vector x those components i for which zi = 0. Similarly,

120

R. Ruiz-Torrubiano, S. Garc´ıa-Moratilla, and A. Su´arez

the matrix A[z] is obtained by eliminating the i-th column of A whenever zi = 0. Finally, Σ[z,z] is obtained by removing from Σ the rows and columns for which the corresponding indicator is zero (zi = 0). The symbols 0 and 1 denote vectors of the appropriate size whose entries are all equal to 0 or to 1, respectively. Minimum and maximum investment constraints, which set a lower and an upper bound on the investment of each asset in the portfolio are captured by (5.14). Vectors a and b are D × 1 column vectors with the lower and upper bounds on the portfolio weights, respectively. Inequality (5.15) summarizes the M concentration of capital constraints. The m-th row of the M × D matrix A is the vector of coefficients of the linear combination that defines the constraint. The M × 1 column vectors l and u correspond to the lower and upper bounds of the M linear restrictions, respectively. Concentration of capital constraints can be used, for instance, to control the amount of capital invested in a group of assets, so that investor preferences or limits for investment in certain asset classes can be formally expressed. Since these constraints are linear, they do not increase the difficulty of the problem, which can still be solved efficiently by quadratic programming. Expression (5.16) corresponds to the cardinality constraint, which limits the number of assets