Evolution and Optimum Seeking Hans-Paul Schwefel
2
Preface In 1963 two students at the Technical University of Berli...
113 downloads
2627 Views
6MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
Evolution and Optimum Seeking Hans-Paul Schwefel
2
Preface In 1963 two students at the Technical University of Berlin met and were soon to collaborate on experiments which used the wind tunnel of the Institute of Flow Engineering. During the search for the optimal shapes of bodies in a ow, which was then a matter of laborious intuitive experimentation, the idea was conceived of proceeding strategically. However, attempts with the coordinate and simple gradient strategies were unsuccessful. Then one of the students, Ingo Rechenberg, now Professor of Bionics and Evolutionary Engineering, hit upon the idea of trying random changes in the parameters de ning the shape, following the example of natural mutations. The evolution strategy was born. A third student, Peter Bienert, joined them and started the construction of an automatic experimenter, which would work according to the simple rules of mutation and selection. The second student, I myself, set about testing the eciency of the new methods with the help of a Zuse Z23 computer for there were plenty of objections to these random strategies. In spite of an occasional lack of nancial support, the Evolutionary Engineering Group which had been formed held rmly together. Ingo Rechenberg received his doctorate in 1970 for the seminal thesis: Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. It contains the theory of the two membered evolution strategy and a rst proposal for a multimembered strategy, which in the nomenclature introduced here, is of the (+1) type. In the same year nancial support from the Deutsche Forschungsgemeinschaft (German Research Association) enabled the initiation of the work which comprises most of the present book. This work was concluded, at least temporarily, in 1974 with the thesis Evolutionsstrategie und numerische Optimierung and published by Birkhauser, Basle, Switzerland, in 1977 under the title Numerische Optimierung von Computer-Modellen mittels der Evolutionsstrategie as well as by Wiley, Chichester, in 1981 as monograph Numerical optimization of computer models. Between 1976 and 1985 the author was not able to continue his work in the eld of Evolution Strategies (nowadays abbreviated: ESs). The general interest in this type of optimum seeking algorithms was not broad enough for there to be nancial support. On the other hand, the number of articles, journals, and books devoted to (mathematical) optimization has increased tremendously. Looking back upon the development from 1964 on, when the rst ES version was devoted to experimental optimization, i.e., upon 30 years, or roughly one human generation, reveals three interesting facts:
First, ESs are not at all outdated. On the contrary, three consecutive conferences on Parallel Problem Solving from Nature (PPSN) in 1990 (see Schwefel and Manner, 1991), 1992 (Manner and Manderick, 1992), and 1994 (Davidor, Schwefel, and Manner, 1994) have demonstrated a revived and increasing interest.
Secondly, the computational environment has changed over time, not only with respect to the number of (also personal) computers and their data processing power, but even more with respect to new architectures. MIMD (Multiple Instructions v
vi Multiple Data) machines with many processors working in parallel for one task seem to wait for inherently parallel problem solving concepts like ESs. Biological metaphors prevail within the new branch of Arti cial Intelligence, called Articial Life (AL).
Third, updating this dissertation from 1974/1975 once more (after adding only a few pages to Chapter 7 in 1981) can be done without rewriting the bulk of the chapters on traditional approaches. Since the emphasis always has been centered on derivativefree direct optimum-seeking methods, it should be sucient to add material on three concepts now, i.e., Genetic Algorithms (GAs), Simulated Annealing (SA), and Tabu Search (TS). This was done with the new Sections 5.3 to 5.5 in Chapter 5. Another innovation is a oppy disk with all those procedures which had been used for the test series in the 1970s, along with a users' manual. Hopefully, some incorrectnesses have been deleted now, too. A rst thank goes again to my friend Dr. Mike Finnis whose translation of my German original into English still forms the core of this book. Thanks go also to those who helped me in completing this update, especially Ms. Heike Bracklo, who brought the scanned ASCII text into LaTeX formats, Mr. Ulrich Hermes, Mr. Jorn Mehnen, and Mr. Joachim Sprave for the many graphs and ready for use computer programs, as well as all those who helped in the process of proofreading the complete work. Finally, I would like to thank the Wiley team for the fruitful collaboration during the process of editing the camera-ready script.
Dortmund, Autumn 1994
Hans-Paul Schwefel
Contents Preface 1 Introduction 2 Problems and Methods of Optimization
2.1 General Statement of the Problems : : : : : : : : : : : : 2.2 Particular Problems and Methods of Solution : : : : : : 2.2.1 Experimental Versus Mathematical Optimization 2.2.2 Static Versus Dynamic Optimization : : : : : : : 2.2.3 Parameter Versus Functional Optimization : : : : 2.2.4 Direct (Numerical) Versus Indirect (Analytic) Optimization : : : : : : : : : 2.2.5 Constrained Versus Unconstrained Optimization : 2.3 Other Special Cases : : : : : : : : : : : : : : : : : : : : :
3 Hill climbing Strategies
v 1 5
: : : : :
5 6 6 9 10
: : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
13 16 18
: : : : :
: : : : :
: : : : :
: : : : :
3.1 One Dimensional Strategies : : : : : : : : : : : : : : : : : : : : 3.1.1 Simultaneous Methods : : : : : : : : : : : : : : : : : : : 3.1.2 Sequential Methods : : : : : : : : : : : : : : : : : : : : : 3.1.2.1 Boxing in the Minimum : : : : : : : : : : : : : 3.1.2.2 Interval Division Methods : : : : : : : : : : : : 3.1.2.2.1 Fibonacci Division. : : : : : : : : : : : 3.1.2.2.2 The Golden Section. : : : : : : : : : : 3.1.2.3 Interpolation Methods : : : : : : : : : : : : : : 3.1.2.3.1 Regula Falsi Iteration. : : : : : : : : : 3.1.2.3.2 Newton-Raphson Iteration. : : : : : : 3.1.2.3.3 Lagrangian Interpolation. : : : : : : : 3.1.2.3.4 Hermitian Interpolation. : : : : : : : : 3.2 Multidimensional Strategies : : : : : : : : : : : : : : : : : : : : 3.2.1 Direct Search Strategies : : : : : : : : : : : : : : : : : : 3.2.1.1 Coordinate Strategy : : : : : : : : : : : : : : : 3.2.1.2 Strategy of Hooke and Jeeves: Pattern Search : 3.2.1.3 Strategy of Rosenbrock: Rotating Coordinates : 3.2.1.4 Strategy of Davies, Swann, and Campey (DSC) 3.2.1.5 Simplex Strategy of Nelder and Mead : : : : : vii
: : : : :
: : : : : : : : : : : : : : : : : : :
: : : : :
: : : : : : : : : : : : : : : : : : :
: : : : :
: : : : : : : : : : : : : : : : : : :
: : : : :
: : : : : : : : : : : : : : : : : : :
: : : : :
: : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : :
23 25 26 27 28 29 29 32 33 34 35 35 37 38 40 41 44 48 54 57
viii 3.2.1.6 Complex Strategy of Box : : : : : : : : : : : : : : 3.2.2 Gradient Strategies : : : : : : : : : : : : : : : : : : : : : : : 3.2.2.1 Strategy of Powell: Conjugate Directions : : : : : : 3.2.3 Newton Strategies : : : : : : : : : : : : : : : : : : : : : : : : 3.2.3.1 DFP: Davidon-Fletcher-Powell Method (Quasi-Newton Strategy, Variable Metric Strategy) 3.2.3.2 Strategy of Stewart: Derivative-free Variable Metric Method : : : : : : : 3.2.3.3 Further Extensions : : : : : : : : : : : : : : : : : :
: : : :
61 65 69 74
: : : :
77
: : : : : : : :
78 81
: : : :
4 Random Strategies 5 Evolution Strategies for Numerical Optimization
5.1 The Two Membered Evolution Strategy : : : : : : : : : : : : : : : : : 5.1.1 The Basic Algorithm : : : : : : : : : : : : : : : : : : : : : : : 5.1.2 The Step Length Control : : : : : : : : : : : : : : : : : : : : : 5.1.3 The Convergence Criterion : : : : : : : : : : : : : : : : : : : : 5.1.4 The Treatment of Constraints : : : : : : : : : : : : : : : : : : 5.1.5 Further Details of the Subroutine EVOL : : : : : : : : : : : : 5.2 A Multimembered Evolution Strategy : : : : : : : : : : : : : : : : : : 5.2.1 The Basic Algorithm : : : : : : : : : : : : : : : : : : : : : : : 5.2.2 The Rate of Progress of the (1 , ) Evolution Strategy : : : : : 5.2.2.1 The Linear Model (Inclined Plane) : : : : : : : : : : 5.2.2.2 The Sphere Model : : : : : : : : : : : : : : : : : : : 5.2.2.3 The Corridor Model : : : : : : : : : : : : : : : : : : 5.2.3 The Step Length Control : : : : : : : : : : : : : : : : : : : : : 5.2.4 The Convergence Criterion for > 1 Parents : : : : : : : : : : 5.2.5 Scaling of the Variables by Recombination : : : : : : : : : : : 5.2.6 Global Convergence : : : : : : : : : : : : : : : : : : : : : : : : 5.2.7 Program Details of the ( + ) ES Subroutines : : : : : : : : : 5.3 Genetic Algorithms : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5.3.1 The Canonical Genetic Algorithm for Parameter Optimization 5.3.2 Representation of Individuals : : : : : : : : : : : : : : : : : : 5.3.3 Recombination and Mutation : : : : : : : : : : : : : : : : : : 5.3.4 Reproduction and Selection : : : : : : : : : : : : : : : : : : : 5.3.5 Further Remarks : : : : : : : : : : : : : : : : : : : : : : : : : 5.4 Simulated Annealing : : : : : : : : : : : : : : : : : : : : : : : : : : : 5.5 Tabu Search and Other Hybrid Concepts : : : : : : : : : : : : : : : :
: : : :
: : : : : : : : : : : : : : : : : : : : : : : : :
: : : :
: : : : : : : : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : : :
87 105 105 106 110 113 115 115 118 118 120 124 127 134 142 145 146 149 149 151 152 153 155 157 158 160 162
6 Comparison of Direct Search Strategies for Parameter Optimization 165 6.1 Diculties : : : : : : : : : : : 6.2 Theoretical Results : : : : : : 6.2.1 Proofs of Convergence 6.2.2 Rates of Convergence : 6.2.3 Q-Properties : : : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
165 166 167 168 169
ix 6.2.4 Computing Demands : : : : : : : : : : : : : : : 6.3 Numerical Comparison of Strategies : : : : : : : : : : : 6.3.1 Computer Used : : : : : : : : : : : : : : : : : : 6.3.2 Optimization Methods Tested : : : : : : : : : : 6.3.3 Results of the Tests : : : : : : : : : : : : : : : : 6.3.3.1 First Test: Convergence Rates for a Quadratic Objective Function : : 6.3.3.2 Second Test: Reliability : : : : : : : : 6.3.3.3 Third Test: Non-Quadratic Problems with Many Variables : : : : : : : : : : 6.4 Core storage required : : : : : : : : : : : : : : : : : : :
: : : : :
170 173 174 175 179
: : : : : : : : : : : : : : : : : : : : : :
179 204
: : : : : : : : : : : : : : : : : : : : : :
217 232
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
: : : : :
7 Summary and Outlook 8 References Appendices A Catalogue of Problems
235 249 325 325
B Program Codes
367
C Programs
415
A.1 Test Problems for the First Part of the Strategy Comparison : : : : : : : : 325 A.2 Test Problems for the Second Part of the Strategy Comparison : : : : : : : 327 A.3 Test Problems for the Third Part of the Strategy Comparison : : : : : : : 361
B.1 (1+1) Evolution Strategy EVOL : : : : : : : : : : : : : : : : : : : : : : : : 367 B.2 ( , ) Evolution Strategies GRUP and REKO : : : : : : : : : : : : : : : : 375 B.3 ( + ) Evolution Strategy KORR : : : : : : : : : : : : : : : : : : : : : : : 386 C.1 Contents of the Floppy Disk : : : : : : : : : : : : : : : : : : : : : C.2 About the Program Disk : : : : : : : : : : : : : : : : : : : : : : : C.3 Running the C Programs : : : : : : : : : : : : : : : : : : : : : : : C.3.1 How to Install OptimA on a PC Using LINUX or on a UNIX Workstation : : : : : : : : : : : : : : : : : : C.3.2 How to Install OptimA on a PC Under DOS : : : : : : : C.3.3 Running OptimA : : : : : : : : : : : : : : : : : : : : : : : C.4 Description of the Programs : : : : : : : : : : : : : : : : : : : : : C.4.1 How to Incorporate New Functions : : : : : : : : : : : : : C.5 Examples : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : C.5.1 An Application of the Multimembered Evolution Strategy to the Corridor Model : : : : : : : : : : : : : : : : : : : : C.5.2 OptimA Working in Batch Mode : : : : : : : : : : : : : :
Index
: : : : : : : : : : : : : : :
415 416 417
: : : : : :
417 418 418 418 419 421
: : : : : : : : : :
421 422
: : : : : :
: : : : : :
: : : : : :
: : : : : :
425
x
Chapter 1 Introduction There is scarcely a modern journal, whether of engineering, economics, management, mathematics, physics or the social sciences, in which the concept optimization is missing from the subject index. If one abstracts from all specialist points of view, the recurring problem is to select a better or best (Leibniz, 1710 eventually, he introduced the term optimal ) alternative from among a number of possible states of a airs. However, if one were to follow the hypothesis of Leibniz, as presented in his Theodicee, that our world is the best of all possible worlds, one could justi ably sink into passive fatalism. There would be nothing to improve or to optimize. Biology, especially since Darwin, has replaced the static world picture of Leibniz' time by a dynamic one, that of the more or less gradual development of the species culminating in the appearance of man. Paleontology is providing an increasingly complete picture of organic evolution. So-called missing links repeatedly turn out to be not missing, but rather hitherto undiscovered stages of this process. Very much older than the recognition that man is the result (or better, intermediate state) of a meliorization process is the seldom-questioned assumption that he is a perfect end product, the \pinnacle of creation." Furthermore, long before man conceived of himself as an active participant in the development of things, he had unconsciously inuenced this evolution. There can be no doubt that his ability and e orts to make the environment meet his needs raised him above other forms of life and have enabled him, despite physical inferiority, to nd, to hold, and to extend his place in the world{so far at least. As long as mankind has existed on our planet, spaceship earth, we, together with other species have mutually inuenced and changed our environment. Has this always been done in the sense of meliorization? In 1759, the French philosopher Voltaire (1759), dissatis ed with the conditions of his age, was already taking up arms against Leibniz' philosophical optimism and calling for conscious e ort to change the state of a airs. In the same way today, when we optimize we nd that we are both the subject and object of the history of development. In the desire to improve an object, a process, or a system, Wilde and Beightler (1967) see an expression of the human striving for perfection. Whether such a lofty goal can be attained depends on many conditions. It is not possible to optimize when there is only one way to carry out a task{then one has no alternative. If it is not even known whether the problem at hand is soluble, the 1
2
Introduction
situation calls for an invention or discovery and not, at that stage, for any optimization. But wherever two or more solutions exist and one must decide upon one of them, one should choose the best, that is to say optimize. Those independent features that distinguish the results from one another are called (independent) variables or parameters of the object or system under consideration they may be represented as binary, integer, otherwise discrete, or real values. A rational decision between the real or imagined variants presupposes a value judgement, which requires a scale of values, a quantitative criterion of merit, according to which one solution can be classi ed as better, another as worse. This dependent variable is usually called an objective (function) because it depends on the objective of the system{the goal to be attained with it{and is functionally related to the parameters. There may even exist several objectives at the same time{the normal case in living systems where the mix of objectives also changes over time and may, in fact, be induced by the actual course of the evolutionary paths themselves. Sometimes the hardest part of optimization is to de ne clearly an objective function. For instance, if several subgoals are aimed at, a relative weight must be attached to each of the individual criteria. If these are contradictory one only can hope to nd a compromise on a trade-o subset of non-dominated solutions. Variability and distinct order of merit are the unavoidable conditions of any optimization. One may sometimes also think one has found the right objective for a subsystem, only to realize later that, in doing so, one has provoked unwanted side e ects, the rami cations of which have worsened the disregarded total objective function. We are just now experiencing how narrow-minded scales of value can steer us into dangerous plights, and how it is sometimes necessary to consider the whole Earth as a system, even if this is where di erences of opinion about value criteria are the greatest. The second diculty in optimization, particularly of multiparameter objectives or processes, lies in the choice or design of a suitable strategy for proceeding. Even when the objective has been suciently clearly de ned, indeed even when the functional dependence on the independent variables has been mathematically (or computationally) formulated, it often remains dicult enough, if not completely impossible, to nd the optimum, especially in the time available. The uninitiated often think that it must be an easy matter to solve a problem expressed in the language of mathematics, that most exact of all sciences. Far from it: The problem of how to solve problems is unsolved{and mathematicians have been working on it for centuries. For giving exact answers to questions of extremal values and corresponding positions (or conditions) we are indebted, for example, to the dierential and variational calculus, of which the development in the 18th century is associated with such illustrious names as Newton, Euler, Lagrange, and Bernoulli. These constitute the foundations of the present methods referred to as classical, and of the further developments in the theory of optimization. Still, there is often a long way from the theory, which is concerned with establishing necessary (and sucient) conditions for the existence of minima and maxima, to the practice, the determination of these most desirable conditions. Practically signi cant solutions of optimization problems rst became possible with the arrival of (large and) fast programmable computers in the mid-20th century. Since then the ood of publications on the subject of optimization has been steadily rising in volume it is a
Introduction
3
simple matter to collect several thousand published articles about optimization methods. Even an interested party nds it dicult to keep pace nowadays with the development that is going on. It seems far from being over, for there still exists no all-embracing theory of optimization, nor is there any universal method of solution. Thus it is appropriate, in Chapter 2, to give a general survey of optimization problems and methods. The special r^ole of direct, static, non-discrete, and non-stochastic parameter optimization emerges here, for many of these methods can be transferred to other elds the converse is less often possible. In Chapter 3, some of these strategies are presented in more depth, principally those that extract the information they require only from values of the objective function, that is to say without recourse to analytic partial derivatives (derivative-free methods). Methods of a probabilistic nature are omitted here. Methods which use chance as an aid to decision making, are treated separately in Chapter 4. In numerical optimization, chance is simulated deterministically by means of a pseudorandom number generator able to produce some kind of deterministic chaos only. One of the random strategies proves to be extremely promising. It imitates, in a highly simpli ed manner, the mutation-selection game of nature. This concept, a two membered evolution strategy, is formulated in a manner suitable for numerical optimization in Chapter 5, Section 5.1. Following the hypothesis put forward by Rechenberg, that biological evolution is, or possesses, an especially advantageous optimization process and is therefore worthy of imitation, an extended multimembered scheme that imitates the population principle of evolution is introduced in Chapter 5, Section 5.2. It permits a more natural as well as more e ective speci cation of the step lengths than the two membered scheme and actually invites the addition of further evolutionary principles, such as, for example, sexual propagation and recombination. An approximate theory of the rate of convergence can also be set up for the (1 , ) evolution strategy, in which only the best of descendants of a generation become parents of the following one. A short excursion, new to this edition, introduces nearly concurrent developments that the author was unaware of when compiling his dissertation in the early 1970s, i.e., genetic algorithms, simulated annealing, and tabu search. Chapter 6 then makes a comparison of the evolution strategies with the direct search methods of zero, rst, and second order, which were treated in detail in Chapter 3. Since the predictive power of theoretical proofs of convergence and statements of rates of convergence is limited to simple problem structures, the comparison includes mainly numerical tests employing various model objective functions. The results are evaluated from two points of view:
Eciency, or speed of approach to the objective
Eectivity, or reliability under varying conditions
The evolution strategies are highly successful in the test of e ectivity or robustness. Contrary to the widely held opinion that biological evolution is a very wasteful method of optimization, the convergence rate test shows that, in this respect too, the evolution methods can hold their own and are sometimes even more ecient than many purely deterministic methods. The circle is closed in Chapter 7, where the analogy between
4
Introduction
iterative optimization and evolution is raised once again for discussion, with a look at some natural improvements and extensions of the concept of the evolution strategy. The list of test problems that were used can be found in Appendix A, and FORTRAN codes of the evolution strategies, with detailed guidance for users, are in Appendix B. Finally, Appendix C explains how to use the C and FORTRAN programs on the oppy disk.
Chapter 2 Problems and Methods of Optimization 2.1 General Statement of the Problems According to whether one emphasizes the theoretical aspect (existence conditions of optimal solutions) or the practical (procedures for reaching optima), optimization nowadays is classi ed as a branch of applied or numerical mathematics, operations research, or of computer-assisted systems (engineering) design. In fact many optimization methods are based on principles which were developed in linear and non-linear algebra. Whereas for equations, or systems of equations, the problem is to determine a quantity or set of quantities such that functions which depend on them have speci ed values, in the case of an optimization problem, an initially unknown extremal value is sought. Many of the current methods of solution of systems of linear equations start with an approximation and successively improve it by minimizing the deviation from the required value. For non-linear equations and for incomplete or overdetermined systems this way of proceeding is actually essential (Ortega and Rheinboldt, 1970). Thus many seemingly quite di erent and apparently unrelated problems turn out, after a suitable reformulation, to be optimization problems. Into this class come, for example, the solution of di erential equations (boundary and initial value problems) and eigenvalue problems, as well as problems of observational calculus, adaptation, and approximation (Stiefel, 1965 Schwarz, Rutishauser, and Stiefel, 1968 Collatz and Wetterling, 1971). In the rst case, the basic problem again is to solve equations in the second, the problem is often reduced to minimize deviations in the Gaussian sense (sum of squares of residues) or the Tchebyche sense (maximum of the absolute residues). Even game theory (Vogelsang, 1963) and pattern or shape recognition as a branch of information theory (Andrews, 1972 Niemann, 1974) have features in common with the theory of optimization. In one case, from among a stored set of idealized types, a pattern will be sought that has the maximum similarity to the one presented in another case, the search will be for optimal courses of action in conict situations. Here, two or more interests are competing. Each player tries to maximize his chance of winning with regard to the way in which his opponent supposedly plays. Most optimization 5
6
Problems and Methods of Optimization
problems, however, are characterized by a single interest, to reach an objective that is not inuenced by others. The engineering aspect of optimization has manifested itself especially clearly with the design of learning robots, which have to adapt their operation to the prevailing conditions (see for example Feldbaum, 1962 Zypkin, 1970). The feedback between the environment and the behavior of the robot is e ected here by a program, a strategy, which can perhaps even alter itself. Wiener (1963) goes even further and considers self-reproducing machines, thus arriving at a consideration of robots that are similar to living beings. Computers are often regarded as the most highly developed robots, and it is therefore tempting to make comparisons with the human brain and its neurons and synapses (von Neumann, 1960, 1966 Marfeld, 1970 Steinbuch, 1971). They are nowadays the most important aid to optimization, and many problems are intractable without them.
2.2 Particular Problems and Methods of Solution The lack of a universal method of optimization has led to the present availability of numerous procedures that each have only limited application to special cases. No attempt will be made here to list them all. A short survey should help to distinguish the parameter optimization strategies, treated in detail later, from the other procedures, but while at the same time exhibiting some features they have in common. The chosen scheme of presentation is to discuss two opposing concepts together.
2.2.1 Experimental Versus Mathematical Optimization
If the functional relation between the variables and the objective function is unknown, one is forced to experiment either on the real object or on a scale model. To do so one must be as free as possible to vary the independent variables and have access to measuring instruments with which the dependent variable, the quality, can be measured. Systematic investigation of all possible states of the system will be too costly if there are many variables, and random sampling of various combinations is too unreliable for achieving the desired result. A procedure must be signi cantly more e ective if it systematically exploits whatever information is retained about preceding attempts. Such a plan is also called a strategy. The concept originated in game theory and was formulated by von Neumann and Morgenstern (1961). Many of the search strategies of mathematical optimization to be discussed later were also applied under experimental conditions{not always successfully. An important characteristic of the experiment is the unavoidable e ect of (stochastic) disturbances on the measured results. A good experimental optimization strategy has to take account of this fact and approach the desired extremum with the least possible cost in attempts. Two methods in particular are most frequently mentioned in this connection: The EVOP (evolutionary operation) method proposed by G. E. P. Box (1957), a development of the experimental gradient method of Box and Wilson (1951) The strategy of arti cial evolution designed by Rechenberg (1964)
Particular Problems and Methods of Solution
7
The algorithm of Rechenberg's evolution strategy will be treated in detail in Chapter 5. In the experimental eld it has often been applied successfully: for example, to the solution of multiparameter shaping problems (Rechenberg, 1964 Schwefel, 1968 Klockgether and Schwefel, 1970). All variables are simultaneously changed by a small random amount. The changes are (binomially or) normally distributed. The expected value of the random vector is zero (for all components). Failures leave the starting condition unchanged, only successes are adopted. Stochastic disturbances or perturbations, brought about by errors of measurement, do not a ect the reliability but inuence the speed of convergence according to their magnitude. Rechenberg (1973) gives rules for the optimal choice of a common variance of the probability density distribution of the random changes for both the unperturbed and the perturbed cases. The EVOP method of G. E. P. Box changes only two or three parameters at a time{if possible those which have the strongest inuence. A square or cube is constructed with an initial condition at its midpoint its 22 = 4 or 23 = 8 corners represent the points in a cycle of trials. These deterministically established states are tested sequentially, several times if perturbations are acting. The state with the best result becomes the midpoint of the next pattern of points. Under some conditions, one also changes the scaling of the variables or exchanges one or more parameters for others. Details of this altogether heuristic way of proceeding are described by Box and Draper (1969, 1987). The method has mainly been applied to the dynamic optimization of chemical processes. Experiments are performed on the real system, sometimes over a period of several years. The counterpart to experimental optimization is mathematical optimization. The functional relation between the criterion of merit or quality and the variables is known, at least approximately to put it another way, a more or less simpli ed mathematical model of the object, process or system is available. In place of experiments there appears the manipulation of variables and the objective function. It is sometimes easy to set up a mathematical model, for example if the laws governing the behavior of the physical processes involved are known. If, however, these are largely unresearched, as is often the case for ecological or economic processes, the work of model building can far exceed that of the subsequent optimization. Depending on what deliberate inuence one can have on the process, one is either restricted to the collection of available data or one can uncover the relationships between independent and dependent variables by judiciously planning and interpreting tests. Such methods (Cochran and Cox, 1950 Kempthorne, 1952 Davies, 1954 Cox, 1958 Fisher, 1966 Vajda, 1967 Yates, 1967 John, 1971) were rst applied only to agricultural problems, but later spread into industry. Since the analyst is intent on building the best possible model with the fewest possible tests, such an analysis itself constitutes an optimization problem, just as does the synthesis that follows it. Wald (1966) therefore recommends proceeding sequentially, that is to construct a model as a hypothesis from initial experiments or given a priori information, and then to improve it in a stepwise fashion by a further series of tests, or, alternatively, to sometimes reject the model completely. The tting of model parameters to the measured data can be considered as an optimization problem insofar as the expected error or maximum risk is to be minimized. This is a special case of optimization, called calculus of observations , which involves sta-
8
Problems and Methods of Optimization
tistical tests like regression and variance analyses on data subject to errors, for which the principle of maximum likelihood or minimum 2 is used (see Heinhold and Gaede, 1972). The cost of constructing a model of large systems with many variables, or of very complicated objects, can become so enormous that it is preferable to get to the desired optimal condition by direct variation of the parameters of the process, in other words to optimize experimentally. The fact that one tries to analyze the behavior of a model or system at all is founded on the hope of understanding the processes more fully and of being able to solve the synthesis problem in a more general way than is possible in the case of experimental optimization, which is tied to a particular situation. If one has succeeded in setting up a mathematical model of the system under consideration, then the optimization problem can be expressed mathematically as follows:
F (x) = F (x1 x2 : : : xn) ! extr The round brackets symbolize the functional relationship between the n independent variables fxi i = 1(1)ng1 and the dependent variable F , the quality or objective function. In the following it is always a scalar quantity. The variables can be scalars or functions of one or more parameters. Whether a maximum or a minimum is sought for is of no consequence for the method of optimization because of the relation maxfF (x)g = ; minf;F (x)g Without loss of generality one can concentrate on one of the types of problem usually the minimum problem is considered. Restrictions do arise, insofar as in many practical problems the variables cannot be chosen arbitrarily. They are called constraints. The simplest of these are the non-negative conditions:
xi 0
for all i = 1(1)n
They are formulated more generally like the objective function: 8 9 > <> = Gj (x) = Gj (x1 x2 : : : xn ) > = > 0 for all j = 1(1)m : The notation chosen here follows the convention of parameter optimization. One distinguishes between equalities and inequalities. Each equality constraint reduces the number of true variables of the problem by one. Inequalities, on the other hand, simply reduce the size of the allowed space of solutions without altering its dimensionality. The sense of the inequality is not critical. Like the interchanging of minimum and maximum problems, one can transform one type into the other by reversing the signs of the terms. It is sucient to limit consideration to one formulation. For minimum problems this is 1
The term 1(1)n stands for 1,2,3,...,n.
Particular Problems and Methods of Solution
9
normally the type Gj (x) 0: Points on the edge of the (closed) allowed space are thereby permitted. A di erent situation arises if the constraint is given as a strict inequality of the form Gj (x) > 0: Then the allowed space can be open if Gj (x) is continuous. If for Gj (x) 0, with other conditions the same, the minimum lies on the border Gj (x) = 0, then for Gj (x) > 0, there is no true minimum. One refers here to an inmum, the greatest lower bound, at which actually Gj (x) = 0. In analogous fashion one distinguishes between maxima and suprema (smallest upper bounds). Optimization in the following means always to nd a maximum or a minimum, perhaps under inequality constraints.
2.2.2 Static Versus Dynamic Optimization
The term static optimization means that the optimum is time invariant or stationary. It is sucient to determine its position and size once and for all. Once the location of the extremum has been found, the search is over. In many cases one cannot control all the variables that inuence the objective function. Then it can happen that these uncontrollable variables change with time and displace the optimum (non-stationary case). The goal of dynamic optimization2 is therefore to maintain an optimal condition in the face of varying conditions of the environment. The search for the extremum becomes a more or less continuous process. According to the speed of movement of the optimum, it may be necessary, instead of making the slow adjustment of the independent variables by hand{as for example in the EVOP method (see Chap. 2, Sect. 2.2.1), to give the task to a robot or automaton. The automaton and the process together form a control loop. However, unlike conventional control loops this one is not required to maintain a desired value of a quantity but to discover the most favorable value of an unknown and time-dependent quantity. Feldbaum (1962), Frankovic et al. (1970), and Zach (1974) investigate in detail such automatic optimization systems, known as extreme value controllers or optimizers. In each case they are built around a search process. For only one variable (adjustable setting) a variety of schemes can be designed. It is signi cantly more complicated for an optimal value loop when several parameters have to be adjusted. Many of the search methods are so very costly because there is no a priori information about the process to be controlled. Hence nowadays one tries to build adaptive control systems that use information gathered over a period of time to set up an internal model of the system, or that, in a sense, learn. Oldenburger (1966) and, in more detail, Zypkin (1970) tackle the problems of learning and self-learning robots. Adaptation is said to take place if the change in the control characteristics is made on the basis of measurements of those input quantities to the process that cannot be altered{also known as disturbing variables. If the output quantities themselves are used (here the objective function) to adjust the control system, the process is called self-learning or self-adaptation. The latter possibility is more reliable but, because of the time lag, slower. Cybernetic engineering is concerned with learning processes in a more general form and always sees or even seeks links with natural analogues. An example of a robot that adapts itself to the environment is the homeostat of Ashby 2
Some authors use the term dynamic optimization in a dierent way than is done here.
10
Problems and Methods of Optimization
(1960). Nowadays, however, one does not build one's own optimizer every time there is a given problem to be solved. Rather one makes use of so-called process computers, which for a new task only need another special program. They can handle large and complicated problems and are coupled to the process by sensors and transducers in a closed loop (online) (Levine and Vilis, 1973 McGrew and Haimes, 1974). The actual computer usually works digitally, so that analogue-digital and digital-analogue converters are required for input and output. Process computers are employed for keeping process quantities constant and maintaining required pro les as well as for optimization. In the latter case an internal model (a computer program) usually serves to determine the optimal process parameters, taking account of the latest measured data values in the calculation. If the position of the optimum in a dynamic process is shifting very rapidly, the manner in which the search process follows the extremum takes on a greater signi cance for the overall quality. In this case one has to go about setting up a dynamic model and specifying all variables, including the controllable ones, as functions of time. The original parameter optimization goes over to functional optimization.
2.2.3 Parameter Versus Functional Optimization
The case when not only the objective function but also the independent variables are scalar quantities is called parameter optimization. Numerical values fxi i
= 1(1)ng
of the variables or parameters are sought for which the value of the objective function
F = F (x) = extrfF (x)g is an optimum. The number of parameters describing a state of the object or system is
nite. In the simplest case of only one variable (n = 1), the behavior of the objective function is readily visualized on a diagram with two orthogonal axes. The value of the parameter is plotted on the abscissa and that of the objective function on the ordinate. The functional dependence appears as a curve. For n = 2 a three dimensional Cartesian coordinate system is required. The state of the system is represented as a point in the horizontal plane and the value of the objective function as the vertical height above it. A mountain range is obtained, the surface of which expresses the relation of dependent to independent variables. To further simplify the representation, the curves of intersection between the mountain range and parallel horizontal planes are projected onto the base plane, which provides a contour diagram of the objective function. From this three dimensional picture and its two dimensional projection, concepts like peak, plateau, valley, ridge, and contour line are readily transferred to the n-dimensional case, which is otherwise beyond our powers of description and visualization. In functional optimization, instead of optimal points in three dimensional Euclidean space, optimal trajectories in function spaces (such as Banach or Hilbert space) are to be determined. Thus one refers also to in nite dimensional optimization as opposed to the nite dimensional parameter optimization. Since the variables to be determined are
Particular Problems and Methods of Solution
11
themselves functions of one or more parameters, the objective function is a function of a function, or a functional. A classical problem is to determine the smooth curve down which a point mass will slide between two points in the shortest time, acted upon by the force of gravity and without friction. Known as the brachistochrone problem, it can be solved by means of the ordinary variational calculus (Courant and Hilbert, 1968a,b Denn, 1969 Clegg, 1970). If the functions to be determined depend on several variables it is a multidimensional variational problem (Klotzler, 1970). In many cases the time t appears as the only parameter. The objective function is commonly an integral, in the integrand of which will appear not only the independent variables x(t) = fx1(t) x2(t) : : : xn(t)g but also their derivatives x_ i(t) = @xi=@t and sometimes also the parameter t itself: Z t2 F (x(t)) = t f (x(t) x_ (t) t) dt ! extr 1
Such problems are typical in control theory, where one has to nd optimal controlling functions for control processes (e.g., Chang, 1961 Lee, 1964 Leitmann, 1964 Hestenes, 1966 Balakrishnan and Neustadt, 1967 Karreman, 1968 Demyanov and Rubinov, 1970). Whereas the variational calculus and its extensions provide the mathematical basis of functional optimization (in the language of control engineering: optimization with distributed parameters), parameter optimization (with localized parameters) is based on the theory of maxima and minima from the elementary di erential calculus. Consequently both branches have followed independent paths of development and become almost separate disciplines. The functional analysis theory of Dubovitskii and Milyutin (see Girsanov, 1972) has bridged the gap between the problems by allowing them to be treated as special cases of one fundamental problem, and it could thus lead to a general theory of optimization. However di erent their theoretical bases, in cases of practical signi cance the problems must be solved on a computer, and the iterative methods employed are then broadly the same. One of these iterative methods is the dynamic programming or stepwise optimization of Bellman (1967). It was originally conceived for the solution of economic problems, in which time-dependent variables are changed in a stepwise way at xed points in time. The method is a discrete form of functional optimization in which the trajectory sought appears as a steplike function. At each step a decision is taken, the sequence of which is called a policy. Assuming that the state at a given step depends only on the decision at that step and on the preceding state{i.e., there is no feedback{, then dynamic programming can be applied. The Bellman optimum principle implies that each piece of the optimal trajectory that includes the end point is also optimal. Thus one begins by optimizing the
nal decision at the transition from the last-but-one to the last step. Nowadays dynamic programming is frequently applied to solving discrete problems of optimal control and regulation (Gessner and Spremann, 1972 Lerner and Rosenman, 1973). Its advantage compared to other, analytic methods is that its algorithm can be formulated as a program suitable for digital computers, allowing fairly large problems to be tackled (Gessner and
12
Problems and Methods of Optimization
Wacker, 1972). Bellman's optimum principle can, however, also be expressed in di erential form and applied to an area of continuous functional optimization (Jacobson and Mayne, 1970). The principle of stepwise optimization can be applied to problems of parameter optimization, if the objective function is separable (Hadley, 1969): that is, it must be expressible as a sum of partial objective functions in which just one or a very few variables appear at a time. The number of steps (k) corresponds to the number of the partial functions at each step a decision is made only on the (`) variables in the partial objective. They are also called control or decision variables. Subsidiary conditions (number m) in the form of inequalities can be taken into account. The constraint functions, like the variables, are allowed to take a nite number (b) of discrete values and are called state variables. The recursion formula for the stepwise optimization will not be discussed here. Only the number of required operations (N ) in the calculation will be mentioned, which is of the order
N k bm+` For this reason the usefulness of dynamic programming is mainly restricted to the case ` = 1 k = n and m = 1. Then at each of the n steps, just one control variable is speci ed with respect to one subsidiary condition. In the other limiting case where all variables have to be determined at one step, the normal case of parameter optimization, the process goes over to a grid method (complete enumeration) with a computational requirement of order O(b(n+m) ). Herein lies its capability for locating global optima, even of complicated multimodal objective functions. However, it is only especially advantageous if the structure of the objective function permits the enumeration to be limited to a small part of the allowed region. Digital computers are poorly suited to solving continuous problems because they cannot operate directly with functions. Numerical integration procedures are possible, but costly. Analogue computers are more suitable because they can directly imitate dynamic processes. Compared to digital computers, however, they have a small numerical range and low accuracy and are not so easily programmable. Thus sometimes digital and analogue computers are coupled for certain tasks as so-called hybrid computers. With such systems a set of di erential equations can be tackled to the same extent as a problem in functional optimization (Volz, 1965, 1973). The digital computer takes care of the iteration control, while on the analogue computer the di erentiation and integration operations are carried out according to the parameters supplied by the digital computer. Korn and Korn (1964), and Bekey and Karplus (1971), describe the operations involved in trajectory optimization and the solution of di erential equations by means of hybrid computers. The fact that random methods are often used for such problems has to do with the computational imprecision of the analogue part, with which deterministic processes usually fail to cope. If requirements for accuracy are very high, however, purely digital computation has to take over, with the consequent greater cost in computation time.
Particular Problems and Methods of Solution
13
2.2.4 Direct (Numerical) Versus Indirect (Analytic) Optimization
The classi cation of mathematical methods of optimization into direct and indirect procedures is attributed to Edelbaum (1962). Especially if one has a computer model of a system, with which one can perform simulation experiments, the search for a certain set of exogenous parameters to generate excellent results asks for robust direct optimization methods. Direct or numerical methods are those that approach the solution in a stepwise manner (iteratively), at each step (hopefully) improving the value of the objective function. If this cannot be guaranteed, a trial and error process results. An indirect or analytic procedure attempts to reach the optimum in a single (calculation) step, without tests or trials. It is based on the analysis of the special properties of the objective function at the position of the extremum. In the simplest case, parameter optimization without constraints, one proceeds on the assumption that the tangent plane at the optimum is horizontal, i.e., the rst partial derivatives of the objective function exist and vanish in x: @F = 0 for all i = 1(1)n (2.1) @xi x=x This system of equations can be expressed with the so-called Nabla operator (r) as a single vector equation for the stationary point x: rF (x) = 0
(2.2)
Equation (2.1) or (2.2) transforms the original optimization problem into a problem of solving a set of, perhaps non-linear, simultaneous equations. If F (x) or one or more of its derivatives are not continuous, there may be extrema that do not satisfy the otherwise necessary conditions. On the other hand not every point in IRn{the n-dimensional space of real variables{ that satis es conditions (2.1) need be a minimum it could also be a maximum or a saddle point. Equation (2.2) is referred to as a necessary condition for the existence of a local minimum. To give sucient conditions requires further processes of di erentiation. In fact, di erentiations must be carried out until the determinant of the matrix of the second or higher partial derivatives at the point x is non-zero. Things remain simple in the case of only one variable, when it is required that the lowest order non-vanishing derivative is positive and of even order. Then and only then is there a minimum. If the derivative is negative, x represents a maximum. A saddle point exists if the order is odd. For n variables, at least the n2 (n + 1) second partial derivatives
@ 2F (x) @xi @xj
for all i j = 1(1)n
must exist at the point x. The determinant of the Hessian matrix r2F (x) must be positive, as well as the further n ; 1 principle subdeterminants of this matrix. While MacLaurin had already completely formulated the sucient conditions for the existence of minima and maxima of one parameter functions in 1742, the corresponding theory
14
Problems and Methods of Optimization
for functions of several variables was only completed nearly 150 years later by Schee er (1886) and Stolz (1893) (see also Hancock, 1960). Sucient conditions can only be applied to check a solution that was obtained from the necessary conditions. The analytic path thus always leads the original optimization problem back to the problem of solving a system of simultaneous equations (Equation (2.2)). If the objective function is of second order, one is dealing with a linear system, which can be solved with the aid of one of the usual methods of linear algebra. Even if noniterative procedures are used, such as the Gaussian elimination algorithm or the matrix decomposition method of Cholesky, this cannot be done with a single-step calculation. Rather the number of operations grows as O(n3 ): With fast digital computers it is certainly a routine matter to solve systems of equations with even thousands of variables however, the inevitable rounding errors mean that complete accuracy is never achieved (Broyden, 1973). One can normally be satis ed with a suciently good approximation. Here relaxation methods, which are iterative, show themselves to be comparable or superior. It depends in detail on the structure of the coecient matrix. Starting from an initial approximation, the error as measured by the residues of the equations is minimized. Relaxation procedures are therefore basically optimization methods but of a special kind, since the value of the objective function at the optimum is known beforehand. This a priori information can be exploited to make savings in the computations, as can the fact that each component of the residue vector must individually go to zero (e.g., Traub, 1964 Wilkinson and Reinsch, 1971 Hestenes, 1973 Hestenes and Stein, 1973). Objective functions having terms or members of higher than second order lead to non-linear equations as the necessary conditions for the existence of extrema. In this case, the stepwise approach to the null position is essential, e.g., with the interpolation method, which was conceived in its original form by Newton (Chap. 3, Sect. 3.1.2.3.2). The equations are linearized about the current approximation point. Linear relations for the correcting terms are then obtained. In this way a complete system of n linear equations has to be solved at each step of the iteration. Occasionally a more convenient approach is to search for the minimum of the function !2 n ~F (x) = X @F @x i=1
i
with the help of a direct optimization method. Besides the fact that F~ (x) goes to zero, not only at the sought for minimum of F (x) but also at its maxima and saddle points, it can sometimes yield non-zero minima of no interest for the solution of the original problem. Thus it is often preferable not to proceed via the conditions of Equation (2.2) but to minimize F (x) directly. Only in special cases do indirect methods lead to faster, more elegant solutions than direct methods. Such is, for example, the case if the necessary existence condition for minima with one variable leads to an algebraic equation, and sectioning algorithms like the computational scheme of Horner can be used or if objective functions are in the form of so-called posynomes, for which Dun, Peterson, and Zener (1967) devised geometric programming, an entirely indirect method.
Particular Problems and Methods of Solution
15
Subsidiary conditions, or constraints, complicate matters. In rare cases equality constraints can be expressed as equations in one variable, that can be eliminated from the objective function, or constraints in the form of inequalities can be made superuous by a transformation of the variables. Otherwise there are the methods of bounded variation and Lagrange multipliers, in addition to penalty functions and the procedures of mathematical programming. The situation is very similar for functional optimization, except that here the indirect methods are still dominant even today. The variational calculus provides as conditions for optima di erential instead of ordinary equations{actually ordinary di erential equations (Euler-Lagrange) or partial di erential equations (Hamilton-Jacobi). In only a few cases can such a system be solved in a straightforward way for the unknown functions. One must usually resort again to the help of a computer. Whether it is advantageous to use a digital or an analogue computer depends on the problem. It is a matter of speed versus accuracy. A hybrid system often turns out to be especially useful. If, however, the solution cannot be found by a purely analytic route, why not choose from the start the direct procedure also for functional optimization? In fact with the increasing complexity of practical problems in numerical optimization, this eld is becoming more important, as illustrated by the work of Daniel (1969), who takes over methods without derivatives from parameter optimization and applies them to the optimization of functionals. An important point in this is the discretization or parameterization of the originally continuous problem, which can be achieved in at least two ways:
By approximation of the desired functions using a sum of suitable known functions or polynomials, so that only the coecients of these remain to be determined (Sirisena, 1973)
By approximation of the desired functions using step functions or sides of polygons, so that only heights and positions of the connecting points remain to be determined
Recasting a functional into a parameter optimization problem has the great advantage that a digital computer can be used straightaway to nd the solution numerically. The disadvantage that the result only represents a suboptimum is often not serious in practice, because the assumed values of parameters of the process are themselves not exactly known (Dixon, 1972a). The experimentally determined numbers are prone to errors or to statistical uncertainties. In any case, large and complicated functional optimization problems cannot be completely solved by the indirect route. The direct procedure can either start directly with the functional to be minimized, if the integration over the substituted function can be carried out (Rayleigh-Ritz method) or with the necessary conditions, the di erential equations, which specify the optimum. In the latter case the integral is replaced by a nite sum of terms (Beveridge and Schechter, 1970). In this situation gradient methods are readily applied (Kelley, 1962 Klessig and Polak, 1973). The detailed way to proceed depends very much on the subsidiary conditions or constraints of the problem.
16
Problems and Methods of Optimization
2.2.5 Constrained Versus Unconstrained Optimization
Special techniques have been developed for handling problems of optimization with constraints. In parameter optimization these are the methods of penalty functions and mathematical programming. In the rst case a modi ed objective function is set up, which For the minimum problem takes the value F (x) = + 1 in the forbidden region, but which remains unchanged in the allowed (feasible) region (barrier method e.g., used within the evolution strategies, see Chap. 5) Only near the boundary inside the allowed region, yields values di erent from F (x) and thus keeps the search at a distance from the edge (partial penalty function e.g., used within Rosenbrock's strategy, see Chap. 3, Sect. 3.2.1.3) Di ers from F (x) over the whole space spanned by the variables (global penalty function) This last is the most common way of treating constraints in the form of inequalities. The main ideas here are due to Carroll (1961 created response surface technique) and to Fiacco and McCormick (1964, 1990 SUMT, sequential unconstrained minimization technique). For the problem
F (x) ! min Gj (x) 0 for all j = 1(1)m Hk (x) = 0 for all k = 1(1)` the penalty function is of the form (with r vj wk > 0 and Gj > 0) ` m X X F~ (x) = F (x) + r G v(jx) + 1r wk "Hk (x)]2 j =1 j
k=1
The coecients vj and wk are weighting factors for the individual constraints and r is a free parameter. The optimum of F~ (x) will depend on the choice of r, so it is necessary to alter r in a stepwise way. The original extreme value problem is thereby solved by a sequence of optimizations in which r is gradually reduced to zero. One can hope in this way at least to nd good approximations for the required minimum problem within a
nite sequence of optimizations. The choice of suitable values for r is not, however, easy. Fiacco (1974) and Fiacco and McCormick (1968, 1990) give some indications, and also suggest further possibilities for penalty functions. These procedures are usually applied in conjunction with gradient methods. The hemstitching method and the riding the constraints method of Roberts and Lyvers (1961) work by changing the chosen direction whenever a constraint is violated, without using a modi ed objective function. They orient themselves with respect to the gradient of the objective and the derivatives of the constraint functions (Jacobian matrix). In hemstitching, there is always a return into the feasible region, while in riding the constraints the search runs along the active constraint boundaries. The variables are
Particular Problems and Methods of Solution
17
reset into the allowed region by the complex method of M. J. Box (1965) (a direct search strategy) whenever explicit bounds are crossed. Implicit constraints on the other hand are treated as barriers (see Chap. 3, Sect. 3.2.1.6). The methods of mathematical programming, both linear and non-linear, treat the constraints as the main aspect of the problem. They were specially evolved for operations research (Muller-Merbach, 1971) and assume that all variables must always be positive. Such non-negativity conditions allow special solution procedures to be developed. The simplest models of economic processes are linear. There are often no better ones available. For this purpose Dantzig (1966) developed the simplex method of linear programming (see also Krelle and Kunzi, 1958 Hadley, 1962 Weber, 1972). The linear constraints, together with the condition on the signs of the variables, span the feasible region in the form of a polygon (for n = 2) or a polyhedron, sometimes called simplex. Since the objective function is also linear, except in special cases, the desired extremum must lie in a corner of the polyhedron. It is therefore sucient just to examine the corners. The simplex method of Dantzig does this in a particularly economic way, since only those corners are considered in which the objective function has progressively better values. It can even be thought of as a gradient method along the edges of the polyhedron. It can be applied in a straightforward way to many hundreds, even thousands, of variables and constraints. For very large problems, which may have a particular structure, special methods have also been developed (Kunzi and Tan, 1966 Kunzi, 1967). Into this category come the revised and the dual simplex methods, the multiphase and duplex methods, and decomposition algorithms. An unpleasant property of linear programs is that sometimes just small changes of the coecients in the objective function or the constraints can cause a big alteration in the solution. To reveal such dependencies, methods of parametric linear programming and sensitivity analysis have been developed (Dinkelbach, 1969). Most strategies of non-linear programming resemble the simplex method or use it as a subprogram (Abadie, 1972). This is the case in particular for the techniques of quadratic programming, which are conceived for quadratic objective functions and linear constraints. The theory of non-linear programming is based on the optimality conditions developed by Kuhn and Tucker (1951), an extension of the theory of maxima and minima to problems with constraints in the form of inequalities. These can be expressed geometrically as follows: at the optimum (in a corner of the allowed region) the gradient of the objective function lies within the cone formed by the gradients of the active constraints. To start with, this is only a necessary condition. It becomes sucient under certain assumptions concerning the structure of the objective and constraint functions. For minimum problems, the objective function and the feasible region must be convex, that is the constraints must be concave. Such a problem is also called a convex program. Finally the Kuhn-Tucker theorem transforms a convex program into an equivalent saddle point problem (Arrow and Hurwicz, 1956), just as the Lagrange multiplier method does for constraints in the form of equalities. A complete theory of equality constraints is due to Apostol (1957). Non-linear programming is therefore only applicable to convex optimization, in which, to be precise, one must distinguish at least seven types of convexity (Ponstein, 1967). In addition, all the functions are usually required to be continuously di erentiable, with an
18
Problems and Methods of Optimization
analytic speci cation of their partial derivatives. There is an extensive literature on this subject, of which the books by Arrow, Hurwicz, and Uzawa (1958), Zoutendijk (1960), Vajda (1961), Kunzi, Krelle, and Oettli (1962), Kunzi, Tzschach, and Zehnder (1966, 1970), Kunzi and Krelle (1969), Zangwill (1969), Suchowitzki and Awdejewa (1969), Mangasarian (1969), Stoer and Witzgall (1970), Whittle (1971), Luenberger (1973), and Varga (1974) are but a small sample. Kappler (1967) considers some of the procedures from the point of view of gradient methods. Kunzi and Oettli (1969) give a survey of the more extended procedures together with an extensive bibliography. FORTRAN programs are to be found in McMillan (1970), Kuester and Mize (1973), and Land and Powell (1973). Of special importance in control theory are optimization problems in which the constraints are partly speci ed as di erential equations. They are also called non-holonomous constraints. Pontrjagin et al. (1967) have given necessary conditions for the existence of optima in these problems. Their trick was to distinguish between the free control functions to be determined and the local or state functions which are bound by constraints. Although the theory has given a strong foothold to the analytic treatment of optimal control processes, it must be regarded as a case of good luck if a practical problem can be made to yield an exact solution in this way. One must usually resort in the end to numerical approximation methods in order to obtain the desired optimum (e.g., Balakrishnan and Neustadt, 1964, 1967 Rosen, 1966 Leitmann, 1967 Kopp, 1967 Mufti, 1970 Tabak, 1970 Canon, Cullum, and Polak, 1970 Tolle, 1971 Unbehauen, 1971 Boltjanski, 1972 Luenberger, 1972 Polak, 1973).
2.3 Other Special Cases According to the type of variables there are still other special areas of mathematical optimization. In parameter optimization for example the variables can sometimes be restricted to discrete or integer values. The extreme case is if a parameter may only take two distinct values, zero and unity. Mixed variable types can also appear in the same problem hence the terms discrete, integer, binary (or zero-one), and mixed-integer programming. Most of the solution procedures that have been worked out deal with linear integer problems (e.g., those proposed by Gomory, Balas, and Beale). An important class of methods, the branch and bound methods, is described for example by Weinberg (1968). They are classed together with dynamic programming as decision tree strategies. For the general non-linear case, a last resort can be to try out all possibilities. This kind of optimization is referred to as complete enumeration. Since the cost of such a procedure is usually prohibitive, heuristic approaches are also tried, with which usable, not necessarily optimal, solutions can be found (Weinberg and Zehnder, 1969). More clever ways of proceeding in special cases, for example by applying non-integer techniques of linear and non-linear programming, can be found in Korbut and Finkelstein (1971), Greenberg (1971), Plane and McMillan (1971), Burkard (1972), Hu (1972), and Gar nkel and Nemhauser (1972, 1973). By stochastic programming is meant the solution of problems with objective functions, and sometimes also constraints, that are subject to statistical perturbations (Faber, 1970). It is simplest if such problems can be reduced to deterministic ones, for example by working
Other Special Cases
19
with expectation values. However, there are some problems in which the probability distributions signi cantly inuence the optimal solution. Operational methods at rst only existed for special cases such as, for example, warehouse problems (Beckmann, 1971). Their numbers as well as the elds of application are growing steadily (Hammer, 1984 Ermoliev and Wets, 1988 Ermakov, 1992). In general, one has to make a clear distinction between deterministic solution methods for more or less noisy or stochastic situations and stochastic methods for deterministic but dicult situations like multimodal or fractal topologies. Here we refer to the former in Chapter 4 we will do so for the latter, especially under the aspect of global optimization. In a rather new branch within the mathematical programming eld, called non-smooth or non-dierentiable optimization, more or less classical gradient-type methods for nding solutions still persist (e.g., Balinski and Wolfe, 1975 Lemarechal and Mi$in, 1978 Nurminski, 1982 Kiwiel, 1985). For successively approaching the zero or extremum of a function if the measured values are subject to uncertainties, a familiar strategy is that of stochastic approximation (Wasan, 1969). The original concept is due to Robbins and Monro (1951). Kiefer and Wolfowitz (1952) have adapted it for problems in which the maximum of a unimodal regression function is sought. Blum (1954a) has proved that the method is certain to converge. It distinguishes between test or trial steps and work steps. With one variable, starting at the point x(k), the value of the objective function is obtained at the two positions x(k) c(k). The slope is then calculated as (k) (k) (k ) (k) y(k) = F (x + c )2c;(kF) (x ; c ) A work step follows from the recursion formula (for minimum searches) x(k+1) = x(k) ; 2a(k)y(k) The choice of the positive sequences c(k) and a(k) is important for convergence of the process. These should satisfy the relations lim c(k) ! 0 k!1 1 X a(k) = 1 k=1 1 X a(k)c(k) < 1 k=1 1 X a(k) !2 < 1 c(k) k=1
One chooses for example the sequences (0) a(k) = ak a(0) > 0 c(0) c(0) > 0 k > 0 c(k) = p 4 k
20
Problems and Methods of Optimization
This means that the work step length goes to zero very much faster than the test step length, in order to compensate for the growing inuence of the perturbations. Blum (1954b) and Dvoretzky (1956) describe how to apply this process to multidimensional problems. The increment in the objective function, hence an approximation to the gradient vector, is obtained from n + 1 observations. Sacks (1958) uses 2 n trial steps. The stochastic approximation can thus be regarded, in a sense, as a particular gradient method. Yet other basic strategies have been proposed these adopt only the choice of step lengths from the stochastic approximation, while the directions are governed by other criteria. Thomas and Wilde (1964) for example, combine the stochastic approximation with the relaxation method of Southwell (1940, 1946). Kushner (1963) and Schmitt (1969) even take random directions into consideration. All the proofs of convergence of the stochastic approximation assume unimodal objective functions. A further disadvantage is that stability against perturbations is bought at a very high cost, especially if the number of variables is large. How many steps are required to achieve a given accuracy can only be stated if the probability density distribution of the stochastic perturbations is known. Many authors have tried to devise methods in which the basic procedure can be accelerated: e.g., Kesten (1958), who only reduces the step lengths after a change in direction of the search, or Odell (1961), who makes the lengths of the work steps dependent on measured values of the objective function. Other attempts are directed towards reducing the e ect of the perturbations (Venter, 1967 Fabian, 1967), for example by making only the direction and not the size of the gradients determine the step lengths. Bertram (1960) describes various examples of applications. More of such work is that of Krasulina (1972) and Engelhardt (1973). In this introduction many classes of possible or practically occurring optimization problems and methods have been sketched briey, but the coverage is far from complete. No mention has been made, for example, of broken rational programming, nor of graphical methods of solution. In operations research especially (Henn and Kunzi, 1968) there are many special techniques for solving transport, allocation, routing, queuing, and warehouse problems, such as network planning and other graph theoretical methods. This excursion into the vast realm of optimization problems was undertaken because some of the algorithms to be studied in more depth in what follows, especially the random methods of Chapter 4, owe their origin and nomenclature to other elds. It should also be seen to what extent methods of direct parameter optimization permeate the other branches of the subject, and how they are related to each other. An overall scheme of how the various branches are interrelated can be found in Saaty (1970). If there are two or more objectives at the same time and occasion, and especially if these are not conict-free, single solution points in the decision variable space can no longer give the full answer to an optimization question, not even in the otherwise simplest situation. How to look for the whole subset of ecient, non-dominated, or Paretooptimal solutions can be found under keywords like vector optimization, polyoptimization or multiple criteria decision making (MCDM) (e.g., Bell, Keeney, and Rai a, 1977 Hwang and Masud, 1979 Peschel, 1980 Grauer, Lewandowski, and Wierzbicki, 1982 Steuer, 1986). Game theory comes into play when several decision makers have access to di erent
Other Special Cases
21
parts of the decision variable set only (e.g., Luce and Rai a, 1957 Maynard Smith, 1982 Axelrod, 1984 Sigmund, 1993). No consideration is given here to these special elds.
22
Problems and Methods of Optimization
Chapter 3 Hill climbing Strategies In this chapter some of the direct, mathematical parameter optimization methods will be treated in more detail for static, non-discrete, non-stochastic, mostly unconstrained functions. They come under the general heading of hill climbing strategies because their manner of searching for a maximum corresponds closely to the intuitive way a sightless climber might feel his way from a valley up to the highest peak of a mountain. For minimum problems the sense of the displacements is simply reversed, otherwise uphill or ascent and downhill or descent methods (Bach, 1969) are identical. Whereas methods of mathematical programming are dominant in operations research and the special methods of functional optimization in control theory, the hill climbing strategies are most frequently applied in engineering design. Analytic methods often prove unsuitable in this eld
Because the assumptions are not satis ed under which necessary conditions for extrema can be stated (e.g., continuity of the objective function and its derivatives)
Because there are diculties in carrying out the necessary di erentiations
Because a solution of the equations describing the conditions does not always lead to the desired optimum (it can be a local minimum, maximum, or saddle point)
Because the equations describing the conditions, in general a system of simultaneous non-linear equations, are not immediately soluble
To what extent hill climbing strategies take care of these particular characteristics depends on the individual method. Very thorough presentations covering some topics can be found in Wilde (1964), Rosenbrock and Storey (1966), Wilde and Beightler (1967), Kowalik and Osborne (1968), Box, Davies, and Swann (1969), Pierre (1969), Pun (1969), Converse (1970), Cooper and Steinberg (1970), Ho mann and Hofmann (1970), Beveridge and Schechter (1970), Aoki (1971), Zahradnik (1971), Fox (1971), C%ea (1971), Daniel (1971), Himmelblau (1972b), Dixon (1972a), Jacoby, Kowalik and Pizzo (1972), Stark and Nicholls (1972), Brent (1973), Gottfried and Weisman (1973), Vanderplaats (1984), and Papageorgiou (1991). More variations or theoretical and numerical studies of older methods can be found as individual publications in a wide variety of journals, or in the volumes of collected articles such as Graves and Wolfe (1963), Blakemore and Davis (1964), Lavi 23
24
Hill climbing Strategies
and Vogl (1966), Klerer and Korn (1967), Abadie (1967, 1970), Fletcher (1969a), Rosen, Mangasarian, and Ritter (1970), Geo rion (1972), Murray (1972a), Lootsma (1972a), Szego (1972), and Sebastian and Tammer (1990). Formulated as a minimum problem without constraints, the task can be stated as follows: n min (3.1) x fF (x) j x 2 IR g The column vector x (at the extreme position) is required 2 x 3 66 x12 77 x = 66 .. 77 = (x1 x2 : : : xn)T 4 . 5 xn and the associated extreme value F = F (x) of the objective function F (x), in this case the minimum. The expression x 2 IRn means that the variables are allowed to take all real values x can thus be represented by any point in an n-dimensional Euclidean space IRn. Di erent types of minima are distinguished: strong and weak, local and global. For a local minimum the following relationship holds:
F (x) F (x) for 0
v u n uX kx ; x k = t (xi ; xi )2 i=1
and
(3.2) "
x 2 IRn This means that in the neighborhood of x de ned by the size of " there is no vector x for which F (x) is smaller than F (x). If the equality sign in Equation (3.2) only applies when x = x, the minimum is called strong, otherwise it is weak. An objective function that only displays one minimum (or maximum) is referred to as unimodal. In many cases, however, F (x) has several local minima (and maxima), which may be of di erent heights. The smallest, absolute or global minimum (minimum minimorum) of a multimodal objective function satis es the stronger condition F (x) F (x)
for all x 2 IRn
(3.3)
This is always the preferred object of the search. If there are also constraints, in the form of inequalities or equalities
Gj (x) 0
for all j = 1(1)m
(3.4)
Hk (x) = 0
for all k = 1(1)`
(3.5)
One Dimensional Strategies
25
then IRn in Equations (3.1) to (3.3) must either be replaced by the hopefully non-empty subset M 2 IRn to represent the feasible region in IRn de ned by Equation (3.4), or by IRn;` , the subspace of lower dimensionality spanned by the variables that now depend on each other according to Equation (3.5). If solutions at in nity are excluded, then the theorem of Weierstrass holds (see for example Rothe, 1959): \In a closed compact region a x b every function which is continuous there has at least one (i.e., an absolute) minimum and maximum." This can lie inside or on the boundary. In the case of discontinuous functions, every point of discontinuity is also a potential candidate for the position of an extremum.
3.1 One Dimensional Strategies The search for a minimum is especially easy if the objective function only depends on one variable. F(x)
x
a
b c
d
e
f
g
Figure 3.1: Special points of a function of one variable
a: b: c: d-e: f: g: h:
local maximum at the boundary local minimum at a point of discontinuity of Fx(x) saddle point, or point of inection weak local maximum local minimum maximum (may be global) at a point of discontinuity of F (x) global minimum at the boundary
h
26
Hill climbing Strategies
This problem would be of little practical interest, however, were it not for the fact that many of the multidimensional strategies make use of one dimensional minimizations in selected directions, referred to as line searches. Figure 3.1 shows some possible ways minima and other special points can arise in the one dimensional case.
3.1.1 Simultaneous Methods
One possible way of discovering the minimum of a function with one parameter is to determine the value of the objective function at a number of points and then to declare the point with the smallest value the minimum. Since in principle all trials can be carried out at the same time, this procedure is referred to as simultaneous optimization. How closely the true minimum is approached depends on the choice of the number and location of the trial points. The more trials are made, the more accurate the solution can be. One will be concerned, however, to obtain a result at the lowest cost in time and computation (or material). The two requirements of high accuracy and lowest cost are contradictory thus an optimum compromise must be sought. The e ectiveness of a search method is judged by the size of the largest remaining interval of uncertainty (in the least favorable case) relative to the position of the minimum for a given number of trials (the so-called minimax concept, see Wilde, 1964 Beamer and Wilde, 1973). Assuming that the points in the series of trials are so densely distributed that several at a time are in the neighborhood of a local minimum, then the length of the interval of uncertainty is the same as the distance between the two points in the neighborhood of the smallest value of F (x). The number of necessary trials can thus get very large unless one has at least some idea of whereabouts the desired minimum is situated. In practice one must limit investigation of the objective function to a nite interval "a b]. It is obvious, and it can be proved theoretically, that the optimal choice of all simultaneous search methods is the one in which the trial points are evenly distributed over the interval "a b] (Boas, 1962, 1963a{d). If N equidistant points are used, the interval of uncertainty is of length `N = N 2+ 1 (b ; a) and the e ectiveness takes the value = N 2+ 1 Put another way: To be sure of achieving an accuracy of " > 0, the equidistant search (also called lattice, grid, or tabulation method ) requires N trials, where 2(b ; a) ; 1 < N 2(b ; a) N integer (3.6) " " Apart from the requirements that the chosen interval "a b] should contain the absolute minimum being sought and that N should be big enough in relation to the \wavyness" of the objective function, no further conditions need to be ful lled in order for the grid method to succeed.
One Dimensional Strategies
27
Even more e ective search schemes can be devised if the objective function is unimodal in the interval "a b]. Wilde and Beightler (1967) describe a procedure, using evenly distributed pairs of points, which is also referred to as a simultaneous dichotomous search. The distance between two points of a pair must be chosen to be suciently large that their objective function values are di erent. As ! 0 the dichotomous search with an even number of trials (even block search) is the best. The number of trials required is 2(b ; a) ; 2 < N 2(b ; a) ; 1 N integer (3.7) " " This is one less than for equidistant searches with the same accuracy requirement. Such a scheme is referred to as optimal in the sense of the -minimax concept. Investigations of arrangements of trials also for uneven numbers (odd block search), with non-vanishing , can be found in Wilde (1966) and Wilde and Beightler (1967). The one dimensional search procedures dealt with so far, which strive to reduce the interval of uncertainty, are called by Wilde (1964) direct elimination procedures. As a second category one can consider the various interpolation methods. These can be much more e ective, but only when the objective function F (x) satis es certain \smoothness" conditions, or can be suciently well approximated in the interval under consideration by a polynomial P (x). In this case, instead of the minimum of F (x), a zero of Px (x) = dP (x)=dx is determined. The number of points at which the objective function must be investigated depends on the order of the chosen polynomial and on the type of information at the points that determine it. Besides the values of the objective function itself, consideration is given to its rst, second, and, less frequently, higher derivatives. In general, no exact statements can be made regarding the quality of the approximation to the desired minimum for a given number of trials. Details of the various methods of locating real zeros of rational functions of one variable, such as regula falsi, Lagrangian and Newtonian interpolation, can be found in books on practical or numerical mathematics under the heading of non-linear equations: e.g., Booth (1967), Faddejew and Faddejewa (1973), Saaty and Bram (1964), Traub (1964), Zurmuhl (1965), Walsh (1966), Ralston and Wilf (1967, 1969), Householder (1970), Ortega and Rheinboldt (1970), and Brent (1973). Although from a fundamental point of view interpolation methods represent indirect optimization procedures, they are of interest here as line search strategies, especially when they are applied iteratively and obtain information about derivatives from function values.
3.1.2 Sequential Methods
If the trials for determining a minimum can be made sequentially, the intermediate results retained at a given time can be used to locate the next trial of the sequence more favorably than would be possible without this information. With the digital computers usually available nowadays, which work in a serial way, one is actually obliged to execute all steps one after the other. Sequential methods, in which the solution is approached in a stepwise, or iterative, manner, are advantageous here. The evaluation of the intermediate results and prediction
28
Hill climbing Strategies
of favorable conditions for the next trial presupposes a more or less precise internal model of the objective function the better the model corresponds to reality, the better will be the results of the interpolation and extrapolation processes. The simplest assumption is that the objective function is unimodal, which means that local minima also always represent global minima. On this basis a number of sequential interval-dividing procedures have been constructed (Sect. 3.1.2.2). Iterative interpolation methods demand more \smoothness" of the objective function (Sect. 3.1.2.3). In the former case it is necessary, in the latter useful, to determine at the outset a suitable interval, "a(0) b(0)], in which the desired extremum lies (Sect. 3.1.2.1).
3.1.2.1 Boxing in the Minimum
If there are no clues as to whereabouts the desired minimum might be situated, one can start with two points x(0) and x(1) = x(0) + s and determine the objective function there. If F (x(1)) < F (x(0)) one proceeds in the chosen direction keeping the same step length: x(k+1) = x(k) + s until F (x(k+1)) > F (x(k)) If, however, F (x(1)) > F (x(0)), one chooses the opposite direction: x(2) = x(0) ; s and x(k+1) = x(k) ; s for k 2 similarly, until a step past the minimum is taken, one has thus determined the minimum of the unimodal function to within an uncertainty interval of length 2 s (Beveridge and Schechter, 1970). In numerical optimization problems the values of the variables often run through several powers of 10, or alternatively they must be precisely determined at many points. In this case the boxing-in method with a very small xed step length is too costly. Box, Davies, and Swann (1969) therefore suggest starting with an initial step length s(0) and doubling it at each successful step. Their recursion formula is as follows: x(k+1) = x(0) + 2k s(0) It is applied as long as F (x(k+1)) F (x(k)) holds. As soon as F (x(k+1)) > F (x(k)) is registered, however, b(0) = x(k+1) is set as the upper bound to the interval and the starting point x(0) is returned to. The lower bound a(0) is found by a corresponding process with negative step lengths going in the opposite direction. In this way a starting interval "a(0) b(0)] is obtained for the one dimensional search procedure to be described below. It can happen, because of the convention for equality of two function values, that the search for a bound to the interval does not end if the objective function reaches a constant horizontal level. It is therefore useful to specify a maximum step length that may not be exceeded.
One Dimensional Strategies
29
The boxing-in method has also been proposed occasionally as a one dimensional optimization strategy (Rosenbrock, 1960 Berman, 1966) in its own right. In order not to waste too many trials far from the target when the accuracy requirement is very high, it is useful to start with relatively large steps. Each time a loop ends with a failure the step length is reduced by a factor less than 0:5, e.g., 0:25. If the above rules for increasing and reducing the step lengths are combined, a very exible procedure is obtained. Dixon (1972a) calls it the success/failure routine. If a starting interval "a(0) b(0)] is already at hand, however, there are signi cantly better strategies for successively reducing the size of the interval.
3.1.2.2 Interval Division Methods
If an equidistant division method is applied repeatedly, the interval of uncertainty is reduced at each step by the same factor , and thus for k steps by k . This exponential progression is considerably stronger than the linear dependence of the value of on the number of trials per step. Thus as few simultaneous trials as possible would be used. A comparison of two schemes, with two and three simultaneous trials, shows that except in the rst loop, only two new objective function values must be obtained at a time in both cases, since of three trial points in one step, one coincides with a point of the previous step. The total number of trials required with sequential application of the equidistant three point scheme is 2 log b;" a 2 log b;" a N odd (3.8) 1 + log 2 < N 3 + log 2 Even better results are provided by the sequential dichotomous search with one pair per step. For the limiting case ! 0 one obtains 2 log b;" a 2 log b;" a N even (3.9) log 2 < N 2 + log 2 Detailed investigations of the inuence of on various equidistant and dichotomous search schemes can be found in Avriel and Wilde (1966b), and Beamer and Wilde (1970). Of greater interest are the two purely sequential elimination methods described in the following chapters, which only require a single objective function value per step. They require that the objective function be unimodal, otherwise they only guarantee that a local minimum or maximum will be found. Shubert (1972) describes an interval dividing procedure that is able to locate all local extrema, including the global optimum. To use it, however, an upper bound to the slope of the function must be known. The method is rather costly, especially with regard to the storage space required.
3.1.2.2.1 Fibonacci Division. This interval division strategy was introduced by
Kiefer (1953). It operates with a series due to Leonardo of Pisa, which bears his pseudonym Fibonacci: f0 = f1 = 1
30
Hill climbing Strategies
fk = fk;1 + fk;2 for k 2 An initial interval "a(0) b(0)] is required, containing the extremum together with a number N , which represents the total number of intended interval divisions. If the general interval is called "a(k) b(k)], the lengths s(k) = t(k) (b(k) ; a(k)) = (b(k+1) ; a(k+1)) are subtracted from its ends, with the reduction factor t(k) = ffN ;k;1 (3.10) N ;k giving c(k) = a(k) + s(k) d(k) = b(k) ; s(k) The values of the objective function at c(k) and d(k) are compared and whichever subinterval contains the better (in a minimum search, lower) value is taken as de ning the interval for the next step. If F (d(k)) < F (c(k)) then a(k+1) = a(k) and b(k+1) = c(k) If F (d(k)) > F (c(k)) then a(k+1) = d(k) and b(k+1) = b(k) A consequence of the Fibonacci series is that, except for the rst interval division, at all of the following steps one of the two new points c(k+1) and d(k+1) is always already known. If F (d(k)) < F (c(k)) then c(k+1) = d(k) and if F (d(k)) > F (c(k)) then d(k+1) = c(k)
One Dimensional Strategies
31
F(x)
x a
d
a
d
c
a
d c
b
c
b
b
Step k Step k+1 Step k+2
Figure 3.2: Interval division in the Fibonacci search
so that each time only one new value of the objective function needs to be obtained. Figure 3.2 illustrates two steps of the procedure. The process is continued until k = N ; 2. At the next division, because f2 = 2 f1 d(k) and c(k) coincide. A further interval reduction can only be achieved by slightly displacing one of the test points. The displacement must be at least big enough for the two objective function values to still be distinguishable. Then the remaining interval after N trials is of length `N = f1 (b(0) ; a(0)) + N As ! 0 the e ectiveness tends to fN;1 . Johnson (1956) and Kiefer (1957) show that this value is optimal in the sense of the -minimax concept, according to which the Fibonacci search is the best of all sequential interval division procedures. However, by taking account of the displacement, not only at the last but at all the steps, Oliver and Wilde (1964) give a recursion formula that for the same number of trials yields a slightly smaller residual interval. Avriel and Wilde (1966a) provide a proof of optimality. If one has a priori information about the structure of the objective function it can be exploited to advantage (Gal, 1971) in order to reduce further the number of trials. Overholt (1967a, 1973) suggests that in general there is no a priori information available to x suitably, and it is therefore better to omit the nal division using a displacement rule and to choose N one bigger from the start. In order to obtain the minimum with accuracy " > 0 one
32
Hill climbing Strategies
should choose N such that
(0) (0) fN > b ;" a fN ;1 Then the e ectiveness of the procedure becomes =f2 N +1 and since (Lucas, 1876) 2 p !N +1 p !N +13 p !N +1 1 1 + 1 ; 1 5 5 5' p 1+ 5 fN = p 4 2 ; 2 2 5 5
the number of trials is approximately
p
b(0) ;a(0) + log 5 log " p N' log 1+2 5
(0) b ; a(0) log "
(3.11)
Overholt (1965) shows by means of numerical tests that the procedure must often be terminated prematurely as F (d(k+1)) becomes equal to F (c(k+1)), for example because of computing with a nite number of signi cant gures. Further divisions of the interval of uncertainty are then pointless. For the boxing-in method of determining the initial interval one would x an initial step length of about 10 " and a maximum step length of about 5 109 ", so that for a 36-bit computer the number range of integers is not exceeded by the largest required Fibonacci number. Finally, two further applications of the Fibonacci procedure may be mentioned. By reversing the scheme, Wilde and Beightler (1967) obtain a method of boxing in the minimum. Kiefer (1957) shows how to proceed if values of the objective function can only be obtained at discrete, not necessarily equidistant, points. More about such lattice search problems can be found in Wilde (1964), and Beveridge and Schechter (1970).
3.1.2.2.2 The Golden Section. It can sometimes be inconvenient to have to specify in advance the number of interval divisions. In this case Kiefer (1953) and Johnson (1956) propose, instead of the reduction factor t(k), which varies with the iteration number in the Fibonacci search, a constant factor (3.12) t = 2p ' 0:618 (positive root of: t2 + t = 1) 1+ 5 For large N ; k t(k) reduces to t. In addition, t is identical to the ratio of lengths a to b, which is obtained by dividing a total length of a + b into two pieces such that the smaller, a, has the same ratio to the larger, b, as the larger to the total. This harmonic division (after Euclid) is also known as the golden section, which gave the procedure its name (Wilde, 1964). After N function calls the uncertainty interval is of length
`N = tN ;1 (b(0) ; a(0))
One Dimensional Strategies For the limiting case N
! 1,
33 since lim (tN ;1 fN ) = 1:17
N !1
the number of trials compared to the Fibonacci procedure is about 17% higher. Compared to the Fibonacci search without displacement, since 1 N ;1 f lim t N +1 ' 0:95 N !1 2 the number of trials is about 5% lower. It should further be noted that, when using the Fibonacci method on digital computers, the Fibonacci numbers must rst be generated, or a sucient number of them must be provided and stored. The number of trials needed for a sequential golden section is 2 b(0);a(0) 3 (0) (0) N = 66 loglog "t 77 ; 1 log b ;" a (3.13) 6 7 Other properties of the iteration sequence, including the criterion for termination at equal function values, are the same as those of the method of interval division according to Fibonacci numbers. Further details can be found, for example, in Avriel and Wilde (1968). Complete programs for the interval division procedures have been published by Pike and Pixner (1965), and Overholt (1967b,c) (see also Boothroyd, 1965 Pike, Hill, and James, 1967 Overholt, 1967a).
3.1.2.3 Interpolation Methods
In many cases one is dealing with a continuous function, the minimum of which is to be determined. If, in addition to the value of the objective function, its slope can be speci ed everywhere, many methods can be derived that may converge faster than the optimal elimination methods. One of the oldest schemes is based on the procedure named after Bolzano for determining the zeros of a function. Assuming that one has two points at which the slopes of the objective function have opposite signs, one bisects the interval between them and determines the slope at the midpoint. This replaces the interval end point, which has a slope of the same sign. The procedure can then be repeated iteratively. At each trial the interval is halved. If the slope has to be calculated from the di erence of two objective function values, the bisection or midpoint strategy becomes the sequential dichotomous search. Avriel and Wilde (1966b) propose, as a variant of the Bolzano search, evaluating the slope at two points in the interval so as to increase the reduction factor. They show that their diblock strategy is slightly superior to the dichotomous search. If derivatives of the objective function are available, or at least if it can be assumed that these exist, i.e., the function F (x) is continuous and di erentiable, far better strategies for the minimum search can be devised. They determine analytically the minimum of a trial function that coincides with the objective function, and possibly also its derivatives, at selected argument values. One distinguishes linear, quadratic, and cubic models according to the order of the trial polynomial. Polynomials of higher order are virtually never used.
34
Hill climbing Strategies
They require too much information about the function F (x). Furthermore, it turns out that in contrast to all the methods referred to so far such strategies do not always converge, for reasons other than rounding error.
3.1.2.3.1 Regula Falsi Iteration. Given two points a(k) and b(k), with their function values F (a(k)) and F (b(k)), the simplest approximation formula for a zero c(k) of F (x) is
(k) b ; a(k) c = a ; F (a ) F (b(k)) ; F (a(k)) This technique, known as regula falsi or regula falsorum, predicts the position of the zero correctly if F (x) depends linearly on x. For one dimensional minimization it can be applied to nd a zero of Fx(x) = dF (x)=dx: (k)
(k)
(k)
(k) a(k) c(k) = a(k) ; Fx(a(k)) F (bb(k)) ; ; F (a(k))
x
x
(3.14)
The underlying model here is a second order polynomial with linear slope. If Fx(a(k)) and Fx(b(k)) have opposite sign, c(k) lies between a(k) and b(k). If Fx(c(k)) 6= 0, the procedure can be continued iteratively by using the reduced interval "a(k+1) b(k+1)] = "a(k) c(k)] if Fx(c(k)) and Fx(b(k)) have the same sign, or using "a(k+1) b(k+1)] = "c(k) b(k)] if Fx(c(k)) and Fx(a(k)) have the same sign. If Fx(a(k)) and Fx(b(k)) have the same sign, c(k) must lie outside "a(k) b(k)]. If Fx(c(k)) has the same sign again, c(k) replaces the argument value at which jFxj is greatest. This extrapolation is also called the secant method. If Fx(c(k)) has the opposite sign, one can continue using regula falsi to interpolate iteratively. As a termination criterion one can apply Fx(c(k)) = 0 or jFx(c(k))j ", " > 0. A minimum can only be found reliably in this way if the starting point of the search lies in its neighborhood. Otherwise the iteration sequence can also converge to a maximum, at which, of course, the slope also goes to zero if Fx(x) is continuous. Whereas in the Bolzano interval bisection method only the sign of the function whose zero is sought needs to be known at the argument values, the regula falsi method also makes use of the magnitude of the function. This extra information should enable it to converge more rapidly. As Ostrowski (1966) and Jarratt (1967, 1968) show, for example, this is only the case if the function corresponds closely enough to the assumed model. The simpler bisection method is better, even optimal (as a zero method in the minimax sense), if the function has opposite signs at the two starting points, is not linear and not convex. In this case the linear interpolation sometimes converges very slowly. According to Stanton (1969), a cubic interpolation as a line search in the eccentric quadratic case often yields even worse results. Dixon (1972a) names two variants of the regula falsi recursion formula, but it is not known whether they lead to better convergence. Fox (1971) proposes a combination of the Bolzano method with the linear interpolation. Dekker (1969) (see also Forsythe, 1969) accredits this procedure with better than linear convergence. Even greater reliability and speed is attributed to the algorithm of Brent (1971), which follows Dekker's method by a quadratic interpolation process as soon as the latter promises to be successful.
One Dimensional Strategies
35
It is inconvenient when dealing with minimization problems that the derivatives of the function are required. If the slopes are obtained from function values by a di erence method, diculties can arise from the nite accuracy of such a process. For this reason Brent (1973) combines regula falsi iteration with division according to the golden section. Further variations can be found in Schmidt and Trinkaus (1966), Dowell and Jarratt (1972), King (1973), and Anderson and Bjorck (1973).
3.1.2.3.2 Newton-Raphson Iteration. Newton's interpolation formula for improv-
ing an approximate solution x(k) to the equation F (x) = 0 (see for example Madsen, 1973) (k) x(k+1) = x(k) ; FF ((xx(k))) x uses only one argument value, but requires the value of the derivative of the function as well as the function itself. If F (x) is linear in x, the zero is correctly predicted here, otherwise an improved approximation is obtained at best, and the process must be repeated. Like regula falsi, Newton's recursion formula can also be applied to determining Fx(x) = 0, with of course the reservations already stated. The so-called Newton-Raphson rule is then (k) (3.15) x(k+1) = x(k) ; FFx((xx(k))) xx
If F (x) is not quadratic, the necessary number of iterations must be made until a termination criterion is satis ed. Dixon (1972a) for example uses the condition jx(k+1) ; x(k)j < ". To set against the advantages that only one argument value is required, and, for quadratic objective functions, one iteration is sucient to nd a point where Fx(x) = 0, there are several disadvantages:
If the derivatives Fx and Fxx are obtained approximately by numerical di erentiation, the eciency of the procedure is worsened not only by rounding errors but also by inaccuracies in the approximation. This is especially true in the neighborhood of the minimum being sought, since Fx becomes vanishingly small there. Minima, maxima, and saddle points are not distinguished. The starting point x(0) must already be located as close as possible to the minimum being sought. If the objective function is of higher than second order, the Newton-Raphson iteration can diverge. The condition for convergence towards a minimum is that Fxx(x) > 0 for all x.
3.1.2.3.3 Lagrangian Interpolation. Whereas regula falsi and Newton-Raphson it-
eration attempt to approximate the minimum using information about the derivatives of the objective function at the argument values and can therefore be classi ed as indirect optimization methods, in Lagrangian interpolation only values of the objective function itself are required. In its general form the procedure consists of tting a pth order polynomial through p + 1 points (Zurmuhl, 1965). One usually uses three argument values and a
36
Hill climbing Strategies
parabola as the model function (quadratic interpolation). Assuming that the three points are a(k) < b(k) < c(k), with the objective function values F (a(k)) F (b(k)) and F (c(k)), the trial parabola P (x) has a vanishing rst derivative at the point (k) 2 (k) 2 (k) (k) 2 ; (a(k))2 ] F (b(k)) + "(a(k))2 ; (b(k) )2 ] F (c(k)) d(k) = 12 "(b ) ;"b((ck) ;) c](Fk)](Fa (a)(k+))"(+c "c()k) ; a(k)] F (b(k)) + "a(k) ; b(k)] F (c(k)) (3.16) This point is a minimum only if the denominator is positive. Otherwise d(k) represents a maximum or a saddle point. In the case of a minimum, d(k) is introduced as a new argument value and one of the old ones is deleted: 9 ( (k) (k) (k) a(k+1) = a(k) > = if a < d < b b(k+1) = d(k) > and F (d(k)) < F (b(k)) c(k+1) = b(k)
9 ( (k) (k) (k) > a(k+1) = d(k) = if a < d < b ( k +1) ( k ) b =b > and F (d(k)) > F (b(k)) c(k+1) = c(k) 9 ( (k) (k) (k) a(k+1) = b(k) > = if b < d < c (k+1) (k) b =d > and F (d(k)) < F (b(k)) c(k+1) = c(k) 9 ( (k) (k) (k) a(k+1) = a(k) > = if b < d < c b(k+1) = b(k) > (3.17) and F (d(k)) > F (b(k)) c(k+1) = d(k) Figure 3.3 shows one of the possible cases. It is useful to determine the ends of the interval a(0) and c(0) at the start of the one dimensional search by a procedure such as one of those described in Section 3.1.2.1. The third argument value is best positioned at the center of the interval. It can be seen from the recursion formula (Equation (3.17)) that only one value of the objective function needs to be obtained at each iteration, at the current position of d(k). The three points clearly do not in general remain equidistant. When the interpolation formula (Equation (3.16)) predicts a minimum that coincides to the desired accuracy " with one of the argument values, the search can be terminated. As the result, one will select the smallest objective function value from among the argument values at hand and the computed minimum. There is little prospect of success if the minimum is predicted to lie outside the interval "a(k) c(k)], at any rate when a bounding procedure of the type described in Section 3.1.2.1 has been applied initially. In this case too the procedure is best terminated. The same holds if a negative denominator in Equation (3.16) indicates a maximum, or if a point of inection is expected because the denominator vanishes. If the measured objective function values are subject to error, for example in experimental optimization, special precautions must be taken. Hotelling (1941) treats this problem in detail. How often the interpolation must be repeated until the minimum is suciently well approximated cannot be predicted in general it depends on the level of agreement between
One Dimensional Strategies F P
F(x)
37
Minimum P(x)
P(x)
x a
(k)
(k+1)
a
d
(k)
b
(k)
c
(k)
(k+1) (k+1)
b
c
Figure 3.3: Lagrangian quadratic interpolation
the objective function and the trial function. In the most favorable case the objective function is also quadratic. Then one iteration is sucient. This is why it can be advantageous to use an interpolation method rather than an interval division method such as the optimal Fibonacci search. Dijkhuis (1971) describes a variant of the basic procedure in which four argument values are taken. The two inner ones and each of the outer ones in turn are used for two separate quadratic interpolations. The weighted mean of the two results yields a new iteration point. This procedure is claimed to increase the reliability of the minimum search for non-quadratic objective functions.
3.1.2.3.4 Hermitian Interpolation. If one chooses, instead of a parabola, a third
order polynomial as a test function, more information is needed to make it agree with the objective function. Beveridge and Schechter (1970) describe such a cubic interpolation procedure. In place of four argument values and associated objective function values, two points a(k) and b(k) are enough, if, in addition to the values of the objective function, values of its slope, i.e., the rst order di erentials, are available. This Hermitian interpolation is mainly used in conjunction with gradient or quasi-Newton methods, because in any case they require the partial derivatives of the objective function, or they approximate them using nite di erence methods. The interpolation formula is: (k) c(k) = a(k) + (b(k) ; a(k)) 2 w +w F; F(bx((ka)) ;) F; (za(k)) x x
38 where and
Hill climbing Strategies (k) (k ) z = 3 "F ((aa(k));;bF(k)()b )] ; Fx(a(k)) ; Fx(b(k))
(3.18)
q w = + z2 ; Fx(a(k)) Fx(b(k)) Recursive exchange of the argument values takes place according to the sign of Fx(c(k)) in a similar way to the Bolzano method. It should also be veri ed here that a(k) and b(k) always bound the minimum. Fletcher and Reeves (1964) use Hermitian interpolation in their conjugate gradient method as a subroutine to approximate a relative minimum in speci ed directions. They terminate the iteration as soon as ja(k) ; b(k)j < ". As in all interpolation procedures, the speed and reliability of convergence depend on the degree of agreement between the model function and the objective function. Pearson (1969), for example, reports that the Fibonacci search is superior to Hermitian interpolation if the objective function is logarithmic or a polynomial of high order. Guilfoyle, Johnson, and Wheatley (1967) propose a combination of cubic interpolation and either the Fibonacci search or the golden section.
3.2 Multidimensional Strategies There have been frequent attempts to extend the basic ideas of one dimensional optimization procedures to several dimensions. The equidistant grid strategy, also known in the experimental eld as the method of factorial design, places an evenly meshed grid over the space under consideration and evaluates the objective function at all the nodes. If n is the dimension of the space under consideration and Ni (i = 1 2 : : : n) is the number of discrete values that the variable xi can take, then the number of combinations to be tested is given by the product n Y N = Ni (3.19) i=1
If Ni = N1 for all i = 1(1)n, one obtains N = N1n. This exponential increase in the number of trials and the computational requirements is what provoked Bellman's now famous curse of dimensions (see Wilde and Beightler, 1967). On a traditional computer, which works sequentially, the trials must all be carried out one after another. The computation time therefore increases as O(cn ), in which the constant c depends on the required accuracy and the size of the interval to be investigated. Proceeding sequentially brought considerable advantages in the one dimensional case if it could only be assumed that the objective function was unimodal. Krolak and Cooper (1963) (see also Krolak, 1968) and Sugie (1964) have given an extension to several dimensions of the Fibonacci search scheme. For n = 2, two points are chosen on one of the two coordinate axes within the given interval in the same way as for the usual one dimensional Fibonacci search. The values of this variable are rst held constant while two complete Fibonacci searches are made to nd the relative optima with respect to
Multidimensional Strategies
39
the second variable. Both end results are then used to reject one of the values of the
rst variable that were held constant, and to reduce the size of the interval with respect to this parameter. By analogy, a three dimensional minimization consists of a recursive sequence of two dimensional Fibonacci searches. If the number of function calls to reduce the uncertainty interval "ai bi] suciently with respect to the variable xi is Ni, then the total number N also obeys Equation (3.19). The advantage compared to the grid method is simply that Ni depends logarithmically on the ratio of initial interval size to accuracy (see Equation (3.11)). Aside from the fact that each variable must be suitably xed in advance, and that the unimodality requirement of the objective function only guarantees that local minima are approached, there is furthermore no guarantee that a desired accuracy will be reached within a nite number of objective function calls (Kaupe, 1964). Other elimination procedures have been extended in a similar way to the multivariable case, such as, for example, the dichotomous search (Wilde, 1965) and a sequential boxingin method (Berman, 1969). In each case the e ort rises exponentially with the number of variables. Another elimination concept for the multidimensional case, the method of contour tangents, is due to Wilde (1963) (see also Beamer and Wilde, 1969). It requires, however, the determination of gradient vectors. Newman (1965) indicates how to proceed in the two dimensional case, and also for discrete values of the variables (lattice search). He requires that F (x) be convex and unimodal. Then the cost should only increase linearly with the number of variables. For n 3, however, no applications of the contour tangent method are as yet known. Transferring interpolation methods to the n-dimensional case means transforming the original minimum problem into a series of problems, in the form of a set of equations to be solved. As non-linear equations can only be solved iteratively, this procedure is limited to the special case of linear interpolation with quadratic objective functions. Practical algorithms based on the regula falsi iteration can be found in Schmidt and Schwetlick (1968) and Schwetlick (1970). The procedure is not widely used as a minimization method (Schmidt and Vetters, 1970). The slopes of the objective function that it requires are implicitly calculated from function values. The secant method described by Wolfe (1959b) for solving a system of non-linear equations also works without derivatives of the functions. From n + 1 current argument values, it extracts the required information about the structure of the n equations. Just as the transition from simultaneous to sequential one dimensional search methods reduces the e ort required at the expense of global convergence, so each further acceleration in the multidimensional case is bought by a reduction in reliability. High convergence rates are achieved by gathering more information and interpreting it in the form of a model of the objective function. If assumptions and reality agree, then this procedure is successful if they do not agree, then extrapolations lead to worse predictions and possibly even to abandoning an optimization strategy. Figure 3.4 shows the contour diagram of a smooth two parameter objective function. All the strategies to be described assume a degree of smoothness in the objective function. They do not converge with certainty to the global minimum but at best to one of the local minima, or sometimes only to a saddle point.
40
Hill climbing Strategies x2
b
c d
e a
a:
Global minimum
b:
Local minimum
c:
Local maxima
d,e:
Saddle points
c
x1
Figure 3.4: Contour lines of a two parameter function (
F x1 x2
)
Various methods are distinguished according to the kind of information they need, namely: Direct search methods, which only need objective function values F (x) Gradient methods, which also use the rst partial derivatives rF (x) ( rst order strategies) Newton methods, which in addition make use of the second partial derivatives r2F (x) (second order strategies) The emphasis here will be placed on derivative-free strategies, that is on direct search methods, and on such higher order procedures as glean their required information about derivatives from a sequence of function values. The recursion scheme of most multidimensional strategies is based on the formula: x(k+1) = x(k) + s(k) v(k) (3.20) They di er from each other with regard to the choice of step length s(k) and search direction v(k), the former being a scalar and the latter a vector of unit length.
3.2.1 Direct Search Strategies
Direct search strategies do without constructing a model of the objective function. Instead, the directions, and to some extent also step lengths, are xed heuristically, or by
Multidimensional Strategies
41
a scheme of some sort, not always in an optimal way under the assumption of a speci ed internal model. Thus the risk is run of not being able to improve the objective function value at each step. Failures must accordingly be planned for, if something can also be \learned" from them. This trial character of search strategies has earned them the name of trial-and-error methods. The most important of them that are still in current use will be presented in the following chapters. Their attraction lies not in theoretical proofs of convergence and rates of convergence, but in their simplicity and the fact that they have proved themselves in practice. In the case of convex or quadratic unimodal objective functions, however, they are generally inferior to the rst and second order strategies to be described later.
3.2.1.1 Coordinate Strategy
The oldest of multidimensional search procedures trades under a variety of names (e.g., successive variation of the variables, relaxation, parallel axis search, univariate or univariant search, one-variable-at-a-time method, axial iteration technique, cyclic coordinate ascent method, alternating variable search, sectioning method, Gauss-Seidel strategy) and manifests itself in a large number of variations. The basic idea of the coordinate strategy, as it will be called here, comes from linear algebra and was rst put into practice by Gauss and Seidel in the single step relaxation method of solving systems of linear equations (see Ortega and Rocko , 1966 Ortega and Rheinboldt, 1967 VanNorton, 1967 Schwarz, Rutishauser, and Stiefel, 1968). As an optimization strategy it is attributed to Southwell (1940, 1946) or Friedmann and Savage (1947) (see also D'Esopo, 1959 Zangwill, 1969 Zadeh, 1970 Schechter, 1970). The parameters in the iteration formula (3.20) are varied in turn individually, i.e., the search directions are xed by the rule: ( if k = p n p integer v(k) = e` with ` = nk (mod n) otherwise where e` is the unit vector whose components have the value zero for all i 6= `, and unity for i = `. In its simplest form the coordinate strategy uses a constant step length s(k). Since, however, the direction to the minimum is unknown, both positive and negative values of s(k) must be tried. In a rst and easy improvement on the basic procedure, a successful step is followed by further steps in the same direction, until a worsening of the objective function is noted. It is clear that the choice of step length strongly inuences the number of trials required on the one hand and the accuracy that can be achieved in the approximation on the other. One can avoid the problem of the choice of step length most e ectively by using a line search method each time to locate the relative optimum in the chosen direction. Besides the interval division methods, the Fibonacci search and the golden section, Lagrangian interpolation can also be used, since all these procedures work without knowledge of the partial derivatives of the objective function. A further strategy for boxing in the minimum must be added, in order to establish a suitable starting interval for each one dimensional minimization.
42
Hill climbing Strategies
The algorithm can be described as follows: Step 0: (Initialization) Establish a starting point x(00) and choose an accuracy bound " > 0 for the one dimensional search. Set k = 0 and i = 1. Step 1: (Boxing in the minimum) Starting from x(ki;1) with an initial step length s = smin (e.g., smin = 10 "), box in the minimum in the direction ei. Double the step length at each successful trial, as long as s < smax (e.g., smax = 5 109 "). De ne the interval limits a(ik) and b(ik). Step 2: (Line search) By varying xi within the interval "a(ik) b(ik)] search for the relative minimum x0 with the required accuracy " (line search with an interval division procedure or an iterative interpolation method): (ki;1) + s e )g F (x0) = min i s fF (x Step 3: (Check for improvement) If F (x0) F (x(ki;1)), then set x(ki) = x0 otherwise set x(ki) = x(ki;1). Step 4: (Inner loop over all coordinates) If i < n, increase i i + 1 and go to step 1. Step 5: (Termination criterion, outer iteration loop) 0) = x(kn) . Set x(k+1(k+1 If all jxi 0) ; x(ik0)j 5 " for all i = 1(1)n, then end the search otherwise increase k k + 1, set i = 1, and go to step 1. Figure 3.5 shows a typical sequence of iteration points for n = 2 variables. In theory the coordinate strategy always converges if F (x) has continuous partial derivatives and the line searches are carried out exactly (Kowalik and Osborne, 1968 Schechter, 1968 Ortega and Rheinboldt, 1970). Rapid convergence is only achieved, however, if the contour surfaces F (x) = const: are approximately concentric surfaces of a hypersphere, or, in the case of elliptic contours, if the principle axes almost coincide with the coordinate axes. If the signi cant system parameters inuence each other (non-separable objective function), then the distances covered by each line search are small without the minimum being within striking distance. This is especially true when the number of variables is large. At discontinuities in the objective function it may happen that improvements in the objective function can only be made in directions that are not along a coordinate axis. In this case the coordinate strategy fails. There is a similar outcome at steep \valleys" of a continuously di erentiable function, i.e., if the step lengths that would enable successful
Variable values
k
i
x x
(0)
0
0
0
9
(1)
0
1
3
9
(2)
0
(2)
1
0
3
5
(3)
1
1
7
5
(4)
1
(4)
2
0
7
3
(5)
2
1
9
3
(6)
2
2
9
2
Numbering
Direction index
43 Iteration index
Multidimensional Strategies
e2
Start x x
(0,1)
(0,0)
x
(1,1)
(0,2) (1,0)
x
=
x
x
(2,1)
(1,2) (2,0)
x
=
x
x
(2,2) =
(3,0)
x
End
1
2
2 3 5 carried over
2 7 3 carried over
carried over
e1
(6)
3
0
9
2
Figure 3.5: Coordinate strategy
convergence are so small that the number of signi cant gures to which data are handled by the computer is insucient for the variables to be signi cantly altered. Numerical tests with the coordinate strategy show that an exact determination of the relative minima is unnecessary, at least at distances far from the objective. It can even happen that one inaccurate line search can make the next one particularly e ective. This phenomenon is exploited in the procedures known as under- or overrelaxation (Engeli, Ginsburg, Rutishauser, and Stiefel, 1959 Varga, 1962 Schechter, 1962, 1968 Cryer, 1971). Although the relative optimum is determined as before, either an increment is added on in the same direction or an iteration point is de ned on the route between the start and nish of the one dimensional search. The choice of the under- or overrelaxation factor requires assumptions about the structure of the problem. The necessary information is available for the problem of solving systems of linear equations with a positive de nite matrix of coecients, but not for general optimization problems. Further possible variations of the coordinate strategy are obtained if the sequence of searches parallel to the axes is not made to follow the cyclic scheme. Southwell (1946), for example, always selects either the direction in which the slope of the objective function Fxi (x) = @F@x(x) i is maximum, or the direction in which the largest step can be taken. To evaluate the choice
44
Hill climbing Strategies
of direction, Synge (1944) uses the ratio Fxi =Fxixi of rst to second partial derivatives at the point x(k). Whether or not the additional e ort for this scheme is worthwhile depends on the particular topology of the contour surface. Adding directions other than parallel to the axes is also often found to accelerate the convergence (Pinsker and Tseitlin, 1962 Elkin, 1968). Its great simplicity has always made the coordinate strategy attractive, despite its sometimes slow convergence. Rules for handling constraints{not counting here penalty function methods{have been devised, for example, by Singer (1962), Murata (1963), and Mugele (1961, 1962, 1966). Singer's maze method departs from the coordinate directions as soon as a constraint is violated and progresses into the feasible region or along the boundary. For this, however, the gradient of the active constraints must be known. Mugele's poor man's optimizer, a discrete coordinate strategy without line searches, not only handles active constraints, but can also cope with narrow valleys that do not run parallel to the coordinate axes. In this case diagonal steps are permitted. Similar to this strategy is the direct search method of Hooke and Jeeves, which because it has become very widely used will be treated in detail in the following chapter.
3.2.1.2 Strategy of Hooke and Jeeves: Pattern Search
The direct pattern search of Hooke and Jeeves (1961) was originally devised as an automatic experimental strategy (see Hooke, 1957 Hooke and VanNice, 1959). It is nowadays much more widely used as a numerical parameter optimization procedure. The method by which the direct pattern search works is characterized by two types of move. At each iteration there is an exploratory move, which represents a simpli ed Gauss-Seidel variation with one discrete step per coordinate direction. No line searches are made. On the assumption that the line joining the rst and last points of the exploratory move represents an especially favorable direction, an extrapolation is made along it (pattern move) before the variables are varied again individually. The extrapolations do not necessarily lead to an improvement in the objective function value. The success of the iteration is only checked after the following exploratory move. The length of the pattern step is thereby increased each time, while the optimal search direction only changes gradually. This pays o to most advantage where there are narrow valleys. An ALGOL implementation of the strategy is due to Kaupe (1963). It was improved by Bell and Pike (1966), as well as by Smith (1969) (see also DeVogelaere, 1968 Tomlin and Smith, 1969). In the rst case, the sequence of plus and minus exploratory steps in the coordinate directions is modi ed to suit the conditions at any instant. The second improvement aims at permitting a retrospective scaling of the variables as the step lengths can be chosen individually to be di erent from each other. The algorithm runs as follows: Step 0: (Initialization) Choose a starting point x(00) = x(;1n), an accuracy bound " > 0, and initial (0) step lengths si 6= 0 for all i = 1(1)n (e.g., s(0) = 1 if no more plausible 1 values are at hand). Set k = 0 and i = 1.
Multidimensional Strategies
45
Step 1: (Exploratory move) Construct x0 = x(ki;1) + s(ik) ei (discrete step in positive direction). If F (x0) < F (x(ki;1)), go to step 2 (successful rst trial) otherwise replace x0 x0 ; 2 s(ik) ei (discrete step in negative direction). If F (x0) < F (x(ki;1)), go to step 2 (success) otherwise replace x0 x0 + s(ik) ei (back to original situation). Step 2: (Retention and switch to next coordinate) Set x(ki) = x0. If i < n, increase i i + 1 and go to step 1. Step 3: (Test for total failure in all directions) If F (x(kn)) F (x(k0)), set x(k+10) = x(k0) and go to step 9. Step 4: (Pattern move) Set x(k+10) = 2 x(kn) ; x(k;1n) (extrapolation), and s(ik+1) = s(ik) sign(x(ikn) ; x(ik;1n)) for all i = 1(1)n. (This may change the sequence of positive and negative directions in the next exploratory move.) Increase k k + 1 and set i = 1. (Observe: There is no success control of the pattern move so far.) Step 5: (Exploration after extrapolation) Construct x0 = x(ki;1) + s(ik) ei. If F (x0) < F (x(ki;1)), go to step 6 otherwise replace x0 x0 ; 2 s(ik) ei. If F (x0) < F (x(ki;1)), go to step 6 otherwise replace x0 x0 + s(ik) ei. Step 6: (Inner loop over coordinates) Set x(ki) = x0. If i < n, increase i i + 1 and go to step 5. Step 7: (Test for failure of pattern move) If F (x(kn)) F (x(k;1n)) (back to position before pattern move), set x(k+10) = x(k;1n) s(ik+1) = s(ik) for all i = 1(1)n, and go to step 10. Step 8: (After successful pattern move, retention and rst termination test) If jx(ikn) ; x(ik;1n)j 12 js(ik)j for all i = 1(1)n, set x(k+10) = x(k;1n) and go to step 9 otherwise go to step 4 (for another pattern move). Step 9: (Step size reduction and termination test) If js(ik)j " for all i = 1(1)n, end the search with result x(k0) otherwise set s(ik+1) = 12 s(ik) for all i = 1(1)n.
46
Hill climbing Strategies
Step 10: (Iteration loop) Increase k k + 1, set i = 1, and go to step 1. Figure 3.6, together with the following table, presents a possible sequence of iteration points. From the starting point (0), a successful step (1) and (3) is taken in each coordinate direction. Since the end point of this exploratory move is better than the starting point, it serves as a basis for the rst extrapolation. This leads to (4). It is not checked here whether or not any improvement over (3) has occurred. At the next exploratory move, from (4) to (5), the objective function value can only be improved in one coordinate direction. It is now checked whether the condition (5) is better than that of point (3). This is the case. The next extrapolation step, to (8), has a changed direction because of the partial failure of the exploration, but maintains its increased length. Now it will be assumed that, starting from (8) with the hitherto constant exploratory step length, no success will be scored in any coordinate direction compared to (8). The comparison with (5) shows that a reduction in the value of the objective function has nevertheless occurred. Thus the next extrapolation to (13) remains the same as the previous one with respect to direction and step length. The next exploratory move leads to a point (15), which although better than (13) is worse than (8). Now there is a return to (8). Only after the exploration again has no success here, are the step lengths halved in order to make further progress possible. The fact that at some points in this case the objective function was tested several times is not typical for n > 2. Starting point
(2)
Success Failure Extrapolation
(1) (0)
Final point (7) (3) (21) (12)
(5) (4)
(18) (17) (22) (9) (6)
(10) (19)
(23) (8) (24)
(25)
(11) (20)
(15)
(13)
(16)
Figure 3.6: Strategy of Hooke and Jeeves
(14)
Multidimensional Strategies
47
Numbering Iteration Direction Variable Comparison Step index index values point lengths k i x1 x2 s1 s2 (0) (1) (2) (3) (4) (5) (6) (7) (8)
0 0 0 0 1 1 1 1 2
0 1 2 2 0 1 2 2 0
0 9 2 9 2 11 2 7 4 5 6 5 6 3 6 7 10 3
(0) (1) (1) (4),(3) (5) (5) -,(5)
(9) (10) (11) (12) (13) (14) (15) (16) (17) (8) (18) (19) (20) (21) (8)
2 2 2 2 3 3 3 3 3 4 4 4 4 4 5
1 1 2 2 0 1 1 2 2 0 1 1 2 2 0
12 3 8 3 10 1 10 5 14 1 16 1 12 1 12 ;1 12 3 10 3 12 3 8 3 10 1 10 5 10 3
(8) (8) (8) (8) (13) (13),(8) (15) (15) (8) (8) (8) (8) -
(22) (23) (24) (25)
5 5 5 6
1 1 2 0
11 9 9 8
(8) (8) (23),(8) -
3 3 2 1
2
2
2
;2
2
;2
2
;2
2
;2
1
;1
Remarks starting point success failure success extrapolation success, success failure failure extrapolation, success failure failure failure failure extrapolation failure success, failure failure failure return failure failure failure failure step lengths halved failure success success, success extrapolation
;1 ;1 A proof of convergence of the direct search of Hooke and Jeeves has been derived by C%ea (1971) it is valid under the condition that the objective function F (x) is strictly convex and continuously di erentiable. The computational operations are very simple and even in unforeseen circumstances cannot lead to invalid arithmetical manipulations such as, for example, division by zero. A further advantage of the strategy is its small storage requirement. It is of order O(n). The selected pattern accelerates the search in valleys, provided they are not sharply bent. The extrapolation steps follow, in an approximate way, the gradient trajectory. However, the limitation of the trial steps to coordinate directions can also lead to a premature termination of the search here, as in the coordinate strategy. Further variations on the method, which have not achieved such popularity, are due to, among others, Wood (1960, 1962, 1965 see also Weisman and Wood, 1966 Weisman,
48
Hill climbing Strategies
Wood, and Rivlin, 1965), Emery and O'Hagan (1966 spider method), Fend and Chandler (1961 moment rosetta search), Bandler and MacDonald (1969 razor search see also Bandler, 1969a,b), Pierre (1969 bunny-hop search), Erlicki and Appelbaum (1970), and Houston and Hu man (1971). A more detailed enumeration of older methods can be found in Lavi and Vogl (1966). Some of these modi cations allow constraints in the form of inequalities to be taken into account directly. Similar to them is a program designed by M. Schneider (see Drenick, 1967). Aside from the fact that in order to use it one must specify which of the variables enter the individual constraints, it does not appear to work very e ectively. Excessively long computation times and inaccurate results, especially with many variables, made it seem reasonable to omit M. Schneider's procedure from the strategy comparison (see Chap. 6). The problem of how to take into account constraints in a direct search has also been investigated by Klingman and Himmelblau (1964) and Glass and Cooper (1965). The resulting methods, to a greater or lesser extent, transform the original problem. They have nowadays been superseded by the general penalty function methods. Automatic \optimizators" for on-line optimization of chemical processes, which once were well known under the names Opcon ( Bernard and Sonderquist, 1959) and Optimat (Weiss, Archer, and Burt, 1961), also apply modi ed versions of the direct search method. Another application is described by Sawaragi at al. (1971).
3.2.1.3 Strategy of Rosenbrock: Rotating Coordinates
Rosenbrock's idea (1960) was to remove the limitation on the number of search directions in the coordinate strategy so that the search steps can move parallel to the axes of a coordinate system that can rotate in the space IRn . One of the axes is set to point in the direction that appears most favorable. For this purpose the experience of successes and failures gathered in the course of the iterations is used in the manner of Hooke and Jeeves' direct search. The remaining directions are xed normal to the rst and mutually orthogonal. To start with, the search directions comprise the unit vectors vi(0) = ei for all i = 1(1)n Starting from (0the point x(00), a trial is made in each direction with the discrete initial 0) step lengths si for all i = 1(1)n. When a success is scored (including equality of the objective function values), the changed variable vector is retained and the step length is multiplied by a positive factor > 1 for a failure, the vector of variables is left unchanged and the step length is multiplied by a negative factor ;1 < < 0. Rosenbrock proposes the choice = 3 and = ;0:5. This process is repeated until at least one success followed (not necessarily immediately) by a failure is registered in each direction. As a rule several cycles must be run through, because if there is a failure in any particular direction the opposite direction is not tested in the same cycle. Following this rst part of the search, the coordinate axes are rotated. Rosenbrock uses for this the orthogonalization procedure of Gram and Schmidt (see, for example, Birkho and MacLane, 1953 Rutishauser, 1966 Nake, 1966). The recursion formulae are as follows: vi(k+1) = kwwik for all i = 1(1)n i
Multidimensional Strategies where and
49
8 > < ai i;1 wi = > ai ; P (aT v(k+1)) v(k+1) : j j j =1 i ai =
n X j =i
for i = 1 for i = 2(1)n
(3.21)
d(jk) vj(k) for all i = 1(1)n
A scalar d(ik) represents the distance covered in direction vi(k) in the kth iteration. Thus v1(k+1) points in the overall successful direction of the step k. It is expected that a particularly large search step can be taken in this direction at the next iteration. The requirement of waiting for (at least one success in each direction has the e ect that no k) direction is lost, and the vi always span the full n-dimensional Euclidean space. The termination rule or convergence criterion is determined by the lengths of the vectors a(1k) and a(2k). Before each orthonormalization there is a test whether ka(1k)k < " and ka(2k) k > 0:3 ka(1k) k. When this condition is satis ed in six consecutive iterations, the search is ended. The second condition is designed to ensure that a premature termination of the search does not occur just because the distances covered have become small. More signi cantly, the requirement is also that the main success direction changes suciently rapidly something that Rosenbrock regards as a sure sign of the proximity of a minimum. As the strategy comparison will show (see Chap. 6), this requirement is often too strong. It even hinders the ending of the procedure in many cases. In his original publication Rosenbrock has already given detailed rules as to how inequality constraints can be treated. His procedure for doing this can be viewed as a partial penalty function method, since the objective function is only altered in the neighborhood of the boundaries. Immediately after each variation of the variables, the objective function value is tested. If the comparison is unfavorable, a failure is registered as in the unconstrained case. For equality or an improvement, however, if the iteration point lies near a boundary of the region, the success criterion changes. For example, for constraints of the form Gj (x) 0 for all j = 1(1)m, the extended objective function F~ (x) takes the form (this is one of several suggestions of Rosenbrock): m X ~ F (x) = F (x) + 'j (x) (fj ; F (x)) j =1
in which and
8 > < 0 'j (x) = > 3 ; 4 2 + 2 3 : 1
if Gj (x) if 0 < Gj (x) < if Gj (x) 0
(3.22)
= 1 ; 1 Gj (x) The auxiliary item fj is the value of the objective function belonging to the last success of the search that did not fall in the region of the j th boundary. As a reasonable value for the boundary zone one can take = 10;4 . (Rosenbrock sets = 10;4 "bj (x(k)) ; aj (x(k))]
50
Hill climbing Strategies
for constraints of the form aj (x) Gj (x) bj (x) this kind of double sided bounding is not always given however). The basis of the procedure is fully described in Rosenbrock and Storey (1966). Using the notations xi object variables si step sizes vi direction components di distances travelled i success/failure indications the extended algorithm of the strategy runs as follows: Step 0: (Initialization) Choose a starting point x(00) such that Gj (x(00)) Choose an accuracy parameter " > 0 (Rosenbrock takes " = 10;4 = 10;4 ). Set vi(0) = ei for all i = 1(1)n. Set k = 0 (outer loop counter), = 0 (inner loop counter). If there are constraints (m > 0), set fj = F (x(00)) for all j = 1(1)m.
> 0 for all j = 1(1)m.
Step 1: (Initialization of step sizes, distances travelled, and indicators) Set s(ik0) = 0:1, d(ik) = 0, and i(k) = ;1 for all i = 1(1)n. Set ` = 0 and i = 1. Step 2: (Trial step) Construct x0 = x(kn` + i ; 1) + s(ik`) vi(k). If F (x0) > F (x(kn` + i ; 1)), go to step 6 otherwise ( 0 go to step 5 if m = 6= 0 set F~ = F (x0) and j = 1: Step 3: (Test of feasibility) 8 > 0 > < If Gj (x0) > otherwise, > : Step 4: (Constraints loop) If j < m increase j
go to step 7 set fj = F (x0) and go to step 4 replace F~ F~ + 'j (x0) (fj ; F~ ) as Equation (3.22). If F~ > F (x(kn` + i ; 1)) go to step 6.
j + 1 and go to step 3.
Multidimensional Strategies
51
Step 5: (Store the success and update the internal memory) Set x(kn` + i) = x0 s(ik` + 1) = 3 s(ik`) , and replace d(ik) d(ik) + s(ik`). If i(k) = ; 1 set i(k) = 0. Go to step 7. Step 6: (Internal memory update in case of failure) kn` + i) = x(kn` + i ; 1), Set x((k` si + 1) = ; 21 s(ik`). If i(k) = 0 set i(k) = 1. Step 7: (Main loop) If j(k) = 1 for all j = 1(1)n, go to step 8 otherwise ( n increase i i + 1 if i < = n increase ` ` + 1 and set i = 1: Go to step 2. Step 8: (Preparation for the orthogonalization and check for termination) n Set x(k + 10) = x(kn` + i) = x(k0) + P d(jk) vj(k). j=1 n (k) (k) P (k ) Construct the vectors a = d v for all i = 1(1)n i
j=i
j
j
(a1 is the total progress during the loop just nished). If ka(1k)k < " and (if n > 1)ka(2k)k > 0:3 ka(1k)k, increase + 1 otherwise set = 0. If = 6, end the search. Step 9: (Orthogonalization) If n > 1, construct new direction vectors vi(k + 1) for i = 1(1)n according to the recursion formula (Equation (3.21)) of the Gram-Schmidt orthogonalization. Increase k k + 1 and go to step 1.
52
Hill climbing Strategies
(2) (0)
v2
(0)
v
1
(3)
(1)
(0) (4)
v
(5)
(1)
Starting point
1
Success
(1)
v2
(7)
Failure Overall success (8) (6)
(9) (2)
v2
(2)
v
1
(10)
Figure 3.7: Strategy of Rosenbrock
(11)
Multidimensional Strategies
53
Numbering Iteration Test index/ index iteration k nl + i (0) (1) (2) (3) (4) (5) (6) (4)
0 0 0 0 0 0 0 1
0 1 2 3 4 5 6 0
Variable values x1 x2 0 2 2 8 2 ;1 2 2
9 9 11 9 8 8 5 8
Step lengths s1 s2 2 2 6 ;3 2
2 2 ;1 ;3 2
Remarks
starting point success failure failure success failure failure transformation and orthogonalization (7) 1 1 3.8 7.1 2 success (8) 1 2 2.9 5.3 2 success (9) 1 3 8.3 2.6 6 success (10) 1 4 5.6 ;2.7 6 failure (11) 1 5 24.4 ;5.4 18 failure (9) 2 0 8.3 2.6 2 2 transformation and orthogonalization In Figure 3.7, including the following table, a few iterations of the Rosenbrock strategy for n = 2 are represented geometrically. At the starting point x(00) the search directions are the same as the unit vectors. After three runs through (6 trials), the trial steps in each direction have led to a success followed by a (1) failure. At the best condition thus attained, (4) at x(04) = x(10), new direction vectors v1 and v2(1) are generated. Five further trials lead to the best point, (9) at x(13) = x(20), of the second iteration, at which a new choice of directions is again made. The complete sequence of steps can be followed, if desired, with the help of the accompanying table. Numerical experiments show that within a few iterations the rotating coordinates become oriented such that one of the axes points along the gradient direction. The strategy thus allows sharp valleys in the topology of the objective function to be followed. Like the method of Hooke and Jeeves, Rosenbrock's procedure needs no information about partial derivatives and uses no line search method for exact location of relative minima. This makes it very robust. It has, however, one disadvantage compared to the direct pattern search: The orthogonalization procedure of Gram and Schmidt is very costly. It requires storage space of order O(n2) for the matrices A = faij g and V = fvij g, and the number of computational operations even increases with O(n3 ). At least in cases where the objective function call costs relatively little, the computation time for the orthogonalization with many variables becomes highly signi cant. Besides this, the number of parameters is in any case limited by the high storage space requirement. If there are constraints, care must be taken to ensure that the starting point is inside the allowed region and suciently far from the boundaries. Examples of the application of
54
Hill climbing Strategies
Rosenbrock's strategy can be found in Storey (1962), and Storey and Rosenbrock (1964). Among them is also a discretized functional optimization problem. For unconstrained problems there exists the code of Machura and Mulawa (1973). The Gram-Schmidt orthogonalization has been programmed, for example, by Clayton (1971). Lange-Nielsen and Lance (1972) have proposed, on the basis of numerical experiments, two improvements in the Rosenbrock strategy. The rst involves not setting constant step lengths at the beginning of a cycle or after each orthogonalization, but rather modifying them and simultaneously scaling them according to the successes and failures during the preceding cycle. The second improvement concerns the termination criterion. Rosenbrock's original version is replaced by the simpler condition that, according to the achievable computational accuracy, several consecutive trials yield the same value of the objective function.
3.2.1.4 Strategy of Davies, Swann, and Campey (DSC)
A combination of the Rosenbrock idea of rotating coordinates with one dimensional search methods is due to Swann (1964). It has become known under the name Davies-SwannCampey (abbreviated DSC) strategy. The description of the procedure given by Box, Davies, and Swann (1969) di ers somewhat from that in Swann, and so several versions of the strategy have arisen in the subsequent literature. Preference is given here to the original concept of Swann, which exhibits some features in common with the method of conjugate directions of Smith (1962) (see also Sect. 3.2.2). Starting from x(00), a line search is made in each of the unit directions vi(0) = ei for all i = 1(1)n. This process is followed by a one dimensional minimization in the direction of the overall success so far achieved (0n) x(00) vn(0)+1 = kxx(0n) ; ; x(00)k
with the result x(0n + 1). The orthogonalization follows this, e.g., by the Gram-Schmidt method. If one of the line searches was unsuccessful the new set of directions would no longer span the complete parameter space. Therefore only those old direction vectors along which a prescribed minimum distance has been moved are included in the orthogonalization process. The other directions remain unchanged. The DSC method, however, places a further hurdle before the coordinate rotation. If the distance covered in one iteration is smaller than the step length used in the line search, the latter is reduced by a factor 10, and the next iteration is carried out with the old set of directions. After an orthogonalization, one of the new directions (the rst) coincides with that of the (n +1)-th line search of the previous step. This can therefore also be interpreted as the
rst minimization in the new coordinate system. Only n more one dimensional searches need be made to nish the iteration. As a termination criterion the DSC strategy uses the length of the total vector between the starting point and end point of an iteration. The search is ended when it is less than a prescribed accuracy bound.
Multidimensional Strategies The algorithm runs as follows: Step 0: (Initialization) Specify a starting point x(00) and an initial step length s(0) (the same for all directions). De ne an accuracy requirement " >(0)0. Choose as a rst set of directions vi = ei for all i = 1(1)n. Set k = 0 and i = 1. Step 1: (Line search) (ki) Starting from x(ki(k;1) , seek the relative minimum x in the direction vi ) such that (k ) F (x(ki)) = F (x(ki;1) + d(ik) vi(k)) = min fF (x(ki;1) + d vi )g: d Step 2: (Main 8 loop) > < < n If i > = n : = n + 1 Step 3:
Step 4: Step 5:
Step 6:
Step 7:
increase i i + 1, and go to step 1 go to step 3 go to step 4. (Eventually one more line search) Construct z = x((kkn) ) ; x(k0). If kzk > 0, set vn+1 = z = kzk i = n + 1 , and go to step 1 otherwise set x(kn+1) = x(kn) d(nk+1) = 0 , and go to step 5. (Check appropriateness of step length) If kx(kn+1) ; x(k0)k s(k), go to step 6. (Termination criterion) Set s(k+1) = 0:1 s(k) . If s(k+1) " end the search otherwise set x(k+10) = x(kn+1), increase k k + 1, set i = 1, and go to step 1. (Check appropriateness of orthogonalization) (k) (k ) Reorder ( the directions vi and associated distances di such that > " for all i = 1(1)p (k) jdi j " for all i = p + 1 (1)n: If p < 2, thus always for n = 1, go to step 5. (Orthogonalization) Construct new direction vectors vi(k+1) for i = 1(1)p by means of the orthogonalization process of Gram and Schmidt (Equation (3.21)). Set s(k+1) = s(k) d(1k+1) = d(nk+1) , and x(k+10) = x(kn) x(k+11) = x(kn+1). Increase k k + 1 set i = 2, and go to step 1.
55
56
Hill climbing Strategies
No geometric representation has been attempted here, since the ne deviations from the Rosenbrock method would hardly be apparent on a simple diagram. The line search procedure of the DSC method has been described in detail by Box, Davies, and Swann (1969). It boxes in the minimum in the chosen direction using three equidistant points and then applies a single Lagrangian quadratic interpolation. The authors state that, in their experience, this is more economical with regard to the number of objective function calls than an exact line search with a sequence of interpolations. The algorithm of the line search is: Step 0: (Initialization) Specify a starting point x0, a step length s, and a direction v (all given from the main program). Step 1: (Step forward) Construct x = x0 + s v. If F (x) F (x0), go to step 3. Step 2: (Step backward) Replace x x ; 2 s v and s ;s. If F (x) F (x0), go to step 3 otherwise (both rst trials without success) go to step 5. Step 3: (Further steps) Replace s 2 s and set x0 = x. Construct x = x0 + s v. If F (x) F (x0), repeat step 3. Step 4: (Prepare interpolation) Replace s 0:5 s. Construct x = x0 + s v. Of the four points just generated, x0 ; s x0 x0 + s, and x0 + 2 s, reject the one which is furthest from the point that has the smallest value of the objective function. Step 5: (Interpolation) De ne the three available equidistant points x1 < x2 < x3 and the associated function values F1 F2, and F3 (this can be done in the course of the trial steps to box in the minimum). Fit a trial parabola through the three points and solve the necessary condition for its minimum. Because the argument values are equidistant, Equation (3.16) for the Lagrangian quadratic interpolation simpli es to x = x2 + 2 (Fs (;F12;F F+3) F ) 1
2
3
If the denominator vanishes or if it turns out that F (x) > F2, then the line search ends with the result x00 = x2 F~0 = F2 otherwise with the result x00 = x and F~0 = F (x).
Multidimensional Strategies
57
A numerical strategy comparison by M. J. Box (1966) shows the method to be a very e ective optimization procedure, in general superior both to the Hooke and Jeeves and the Rosenbrock methods. However, the tests only refer to smooth objective functions with few variables. If the number of parameters is large, the costly orthogonalization process makes its inconvenient presence felt also in the DSC strategy. Several suggestions have been made to date as to how to simplify the Gram-Schmidt procedure and to reduce its susceptibility to numerical rounding error (Rice, 1966 Powell, 1968a Palmer, 1969 Golub and Saunders, 1970, Householder method). Palmer replaces the conditions of Equation (3.21) by: 8 (k) Pn d2 = 0 otherwise > v if > i > j =1 j > > > < Pn (k) s Pn 2 (k+1) for i = 1 vi = > j=1 dj vj = j=1 dj > > > ! sn > n n 2 P Pn P > (k) (k) P > : di;1 j=i dj vj ; vi;1 j=i dj = j=i d2j j=i;1 d2j for i = 2(1)n He shows that even if no success was obtained in one of the directions vi(k), that is di = 0, the new vectors vi(k+1) for all i = 1(1)n still span the complete parameter space, because k+1) vi(+1 is set equal to ;vi(k). Thus the algorithm does not need to be restricted to directions for which di > ", as happens in the algorithm with Gram-Schmidt orthogonalization. The signi cant advantage of the revised procedure lies in the fact that the number of computational operations remains only of the order O(n2). The storage requirement is also somewhat less since one n n matrix as an intermediate storage area is omitted. For problems with linear constraints (equalities and inequalities) Box, Davies, and Swann (1969) recommend a modi cation of the orthogonalization procedure that works in a similar way to the method of projected gradients of Rosen (1960, 1961) (see also Davies, 1968). Non-linear constraints (inequalities) can be handled with the created response surface technique devised by Carroll (1961), which is one of the penalty function methods. Further publications on the DSC strategy, also with comparison tests, are those of Swann (1969), Davies and Swann (1969), Davies (1970), and Swann (1972). Hoshino (1971) observes that in a narrow valley the search causes zigzag movements. His remedy for this is to add a further search, again in direction v1(k), after each set of n line searches. With the help of two examples, for n = 2 and n = 3, he shows the accelerating e ect of this measure.
3.2.1.5 Simplex Strategy of Nelder and Mead
There are a group of methods called simplex strategies that work quite di erently to the direct search methods described so far. In spite of their common name they have nothing to do with the simplex method of linear programming of Dantzig (1966). The idea (Spendley, Hext, and Himsworth, 1962) originates in an attempt to reduce, as much as possible, the number of simultaneous trials in the experimental identi cation procedure
58
Hill climbing Strategies
of factorial design (see for example Davies, 1954). The minimum number according to Brooks and Mickey (1961) is n + 1. Thus instead of a single starting point, n + 1 vertices are used. They are arranged so as to be equidistant from each other: for n = 2 in an equilateral triangle for n = 3 a tetrahedron and in general a polyhedron, also referred to as a simplex. The objective function is evaluated at all the vertices. The iteration rule is: Replace the vertex with the largest objective function value by a new one situated at its re ection in the midpoint of the other n vertices. This rule aims to locate the new point at an especially promising place. If one lands near a minimum, the newest vertex can also be the worst. In this case the second worst vertex should be reected. If the edge length of the polyhedron is not changed, the search eventually stagnates. The polyhedra rotate about the vertex with the best objective function value. A closer approximation to the optimum can only be achieved by halving the edge lengths of the simplex. Spendley, Hext, and Himsworth suggest doing this whenever a vertex is common to more than 1:65 n + 0:05 n2 consecutive polyhedra. Himsworth (1962) holds that this strategy is especially advantageous when the number of variables is large and the determination of the objective function prone to error. To this basic procedure, various modi cations have been proposed by, among others, Nelder and Mead (1965), Box (1965), Ward, Nag, and Dixon (1969), and Dambrauskas (1970, 1972). Richardson and Kuester (1973) have provided a complete program. The most common version is that of Nelder and Mead, in which the main di erence from the basic procedure is that the size and shape of the simplex is modi ed during the run to suit the conditions at each stage. The algorithm, with an extension by O'Neill (1971), runs as follows: Step 0: (Initialization) Choose a starting point x(00), initial step lengths s(0) i for all i = 1(1)n (0) (if no better scaling is known, si = 1), and an accuracy parameter " > 0 (e.g., " = 10;8 ). Set c = 1 and k = 0. Step 1: (Establish the initial simplex) x(k) = x(k0) + c s(0) e for all = 1(1)n. Step 2: (Determine worst and best points for the normal reection) Determine the indices w (worst point) and b (best point) such that (k ) F (x(kw)) = max fF (x ) = 0(1)ng (k ) F (x(kb)) = min fF (x ) = 0(1)ng n Construct x( = n1 P x(k) and x0 = 2 x( ; x(kw) (normal reection). =0 6=w F (x(kb)), go to
If F (x0) < step 4. Step 3: (Compare trial with other vertices) Determine the number for which F (x0) F (x(k)) holds for all = 0(1)n. 8 > < > 1 set x(k+1w) = x0 and go to step 8 If > = 1 go to step 5 : = 0 go to step 6:
Multidimensional Strategies
59
Step 4: (Expansion) Construct x00 = 2 x0 ; x(. If F (x00) < F (x(kb)), set x(k+1w) = x00 otherwise set x(k+1w) = x0. Go to step 8. Step 5: (Partial outside contraction) Construct x00 = 0:5 ((x + x0). If F (x00) F (x0), set x(k+1w) = x00 and go to step 8 otherwise go to step 7. Step 6: (Partial inside contraction) Construct x00 = 0:5 ((x + x(kw)). If F (x00) F (x(kw)), set x(k+1w) = x00 and go to step 8. Step 7: (Total contraction) Construct x(k+1) = 0:5 (x(kb) + x(k)) for all = 0(1)n. Go to step 9. Step 8: (Normal iteration loop) Assign x(k+1) = x(k) for all = 0(1)n except = w. Step 9: (Termination criterion) Increase n k k + 1. Pn 2 ! P 1 F 2(x(k)) ; n+1 F (x(k)) < "2, go to step 10 If n1 =0
=0
otherwise go to step 2. Step 10: (Restart test note that index w points to the new vertex) Test whether any vector (kw)). x = x(kw) 0:001 s(0) i ei for all i = 1(1)n exists, such that F (x) < F (x (k0) If so, set x = x c = 0:001, and go to step 1 (restart) otherwise end the search with result x(kw). The criterion for ending the minimum search is based on testing whether the variance of the objective function values at the vertices of the simplex is less than a prescribed limit. A few hypothetical iterations of the procedure for two variables are shown in Figure 3.8 including the following table. The sequence of reections, expansions, and contractions is taken from the accompanying table, in which the simplex vertices are numbered sequentially and sorted at each stage in order of their objective function values.
60
Hill climbing Strategies
(1)
(2)
Starting point Vertex point First and last simplex
(3)
(4) (5)
(9) (6) (7) (15) (17) (14) (12)
(11)
(16) (10) (8)
(13)
Figure 3.8: Simplex strategy of Nelder and Mead
Multidimensional Strategies Iteration index
Simplex vertices worst best
0
1
1
2
2
3
3
6
4
5
5 6 7
61
9 14 15
11 17
2 2 2 3 3 6 6 6 5 5 9 10 9 9 11 11 11 7 7 7 17 16
6 8 9 11 13 16
3 3 3 5 5 5 5 5 7 7 7 7 7 7 7 7 12 12 12 12 12
Remarks 4 5
start simplex reection expansion (successful) reection
7
reection expansion (unsuccessful) reection reection partial outside contraction
12
reection expansion (unsuccessful) reection partial inside contraction total contraction
The main di erence between this program and the original strategy of Nelder and Mead is that after a normal ending of the minimization there is an attempt to construct a new starting simplex. To this end, small trial steps are taken in each coordinate direction. If just one of these tests is successful, the search is started again but with a simplex of considerably reduced edge lengths. This restart procedure recommends itself because, especially for a large number of variables, the simplex tends to no longer span the complete parameter space, i.e., to collapse, without reaching the minimum. For few variables the simplex method is known to be robust and reliable, but also to be relatively costly. There are n + 1 parameter vectors to be stored and the reection requires a number of computational operations of order O(n2). According to Nelder and Mead, the number of function calls increases approximately as O(n2:11) however, this empirical value is based only on test results with up to 10 variables. Parkinson and Hutchinson (1972a,b) describe a variant of the strategy in which the real storage requirement can be reduced by about half (see also Spendley, 1969). Masters and Drucker (1971) recommend altering the expansion or contraction factor after consecutive successes or failures respectively.
3.2.1.6 Complex Strategy of Box
M. J. Box (1965) calls his modi cation of the polyhedron strategy the complex method, an abbreviation for constrained simplex, since he conceived it also for problems with
62
Hill climbing Strategies
inequality constraints. The starting point of the search does not need to lie in the feasible region. For this case Box suggests locating an allowed point by minimizing the function m X F~ (x) = ; Gj (x) j (x) j =1
with until
(
j (x) = 01
if Gj (x) 0 otherwise
(3.23)
F~ (x) = 0 The two most important di erences from the Nelder-Mead strategy are the use of more vertices and the expansion of the polyhedron at each normal reection. Both measures are intended to prevent the complex from eventually spanning only a subspace of reduced dimensionality, especially at active constraints. If an allowed starting point is given or has been found, it de nes one of the n + 1 N 2 n vertices of the polyhedron. The remaining vertex points are xed by a random process in which each vector inside the closed region de ned by the explicit constraints has an equal probability of selection. If an implicit constraint is violated, the new point is displaced stepwise towards the midpoint of the allowed vertices that have already been de ned until it satis es all the constraints. Implicit constraints Gj (x) 0 are dealt with similarly during the course of the minimum search. If an explicit boundary is crossed, xi ai, the o ending variable is simply set back in the allowed region to a value near the boundary. The details of the algorithm are as follows: Step 0: (Initialization) Choose a starting point x(0) and a number of vertices N n + 1 (e.g., N = 2 n). Number the constraints such that the rst j m1 each depend only on one variable, x`j (Gj (x`j ), explicit form). Test whether x(0) satis es all the constraints. If not, then construct a substitute objective function according to Equation (3.23). Set up the initial complex as follows: x(01) = x(0) n and x(0) = x(0) + P z e for = 2(1)N , i=1
i i
where the zi are uniformly distributed random numbers from the range ( "a b ], if constraints are given in the form a x b i i i i i h (0) (0) otherwise xi ; 0:5 s xi + 0:5 s ] where, e.g., s = 1: If Gj (x(0)) < 0 for any j m1 > 1 replace x(0`j ) 2 x(0`j 1) ; x(0`j ): If Gj (x(0)) < 0 for any j m > 1 ;1 1 P (0) replace x(0) 0:5 "x(0) + ;1 x ]. =1
Multidimensional Strategies
63
(If necessary repeat this process until Gj (x(0)) 0 for all j = 1(1)m.) Set k = 0. Step 1: (Reection) Determine the index w (worst vertex) such that (k ) F (x(kw)) = max fF (x ) = 1(1)N g: N Construct x( = 1 P x(k) N ;1
Step 2: Step 3:
Step 4:
Step 5:
Step 6: Step 7:
Step 8:
=1
6=w ; x(kw))
and x0 = x( + ((x (over-reection factor = 1:3): (Check for constraints) If m = 0, go to step 7 otherwise set j = 1. If m1 = 0 , go to step 5. (Set vertex back into bounds for explicit constraints) Obtain g = Gj (x0) = Gj (x0`j ). If g 0, go to step 4 otherwise replace x0`j x0`j + g + " (backwards length " = 10;6 ). If Gj (x0) < 0, replace x0`j x0`j ; 2 (g + "). (Explicit constraints loop) Increase j j + 1. 8 > go to step 3 < j m1 If > m1 < j m go to step 5 : j > m go to step 7: (Check implicit constraints) If Gj (x0) 0, go to step 6 otherwise go to step 8, unless the same constraint caused a failure ve times in a row without its function value Gj (x0) being changed. In this case go to step 9. (Implicit constraints loop) If j < m, increase j j + 1 and go to step 5. (Check for improvement) If F (x0) < F (x((k)) for at least one = 1(1)N except = w, (k ) for all = 1(1)N except = w set x(k+1) = xx0 for = w increase k k + 1 and go to step 1 otherwise go to step 8, unless a failure occurred ve times in a row with no change in the objective function value F (x0). In this case go to step 9. (Contraction) Replace x0 0:5 ((x + x0). Go to step 2.
64
Hill climbing Strategies
Step 9: (Termination) Determine the index b (best vertex) such that (k ) F (x(kb)) = min fF (x ) = 1(1)N g: End the search with the result x(kb) and F (x(kb)): Box himself reports that in numerical tests his complex strategy gives similar results to the simplex method of Nelder and Mead, but both are inferior to the method of Rosenbrock with regard to the number of objective function calls. He actually uses his own modi cation of the Rosenbrock method. Investigation of the e ect of the number of vertices of the complex and the expansion factor (in this case 2 n and 1.3 respectively) lead him to the conclusion that neither value has a signi cant e ect on the eciency of the strategy. For n > 5 he considers that a number of vertices N = 2 n is unnecessarily high, especially when there are no constraints. The convergence criterion appears very reliable. While Nelder and Mead require that the standard deviation of all objective function values at the polyhedron vertices, referred to its midpoint, must be less than a prescribed size, the complex search is only ended when several consecutive values of the objective function are the same to computational accuracy. Because of the larger number of polyhedron vertices the complex method needs even more storage space than the simplex strategy. The order of magnitude, O(n2 ), remains the same. No investigations are known of the computational e ort in the case of many variables. Modi cations of the strategy are due to Guin (1968), Mitchell and Kaplan (1968), and Dambrauskas (1970, 1972). Guin de nes a contraction rule with which an allowed point can be generated even if the allowed region is not convex. This is not always the case in the original method because the midpoint to which the worst vertex is reected is not tested for feasibility. Mitchell nds that the initial con guration of the complex inuences the results obtained. It is therefore better to place the vertices in a deterministic way rather than to make a random choice. Dambrauskas combines the complex method with the step length rule of the stochastic approximation. He requires that the step lengths or edge lengths of the polyhedron go to zero in the limit of an in nite number of iterations, while their sum tends to in nity. This measure may well increase the reliability of convergence however, it also increases the cost. Beveridge and Schechter (1970) describe how the iteration rules must be changed if the variables can take only discrete values. A practical application, in which a process has to be optimized dynamically, is described by Tazaki, Shindo, and Umeda (1970) this is the original problem for which Spendley, Hext, and Himsworth (1962) conceived their simplex EVOP (evolutionary operation) procedure. Compared to other numerical optimization procedures the polyhedra strategies have the disadvantage that in the closing phase, near the optimum, they converge rather slowly and sometimes even stagnate. The direction of progress selected by the reection then no longer coincides at all with the gradient direction. To remove this diculty it has been suggested that information about the topology of the objective function, as given by function values at the vertices of the polyhedron, be exploited to carry out a quadratic interpolation. Such surface tting is familiar from the related methods of test planning and
Multidimensional Strategies
65
evaluation (lattice search, factorial design), in which the task is to set up mathematical models of physical or other processes. This territory is entered for example by G. E. P. Box (1957), Box and Wilson (1951), Box and Hunter (1957), Box and Behnken (1960), Box and Draper (1969, 1987), Box et al. (1973), and Beveridge and Schechter (1970). It will not be covered in any more detail here.
3.2.2 Gradient Strategies
The Gauss-Seidel strategy very straightforwardly uses only directions parallel to the coordinate axes to successively improve the objective function value. All other direct search methods strive to advance more rapidly by taking steps in other directions. To do so they exploit the knowledge about the topology of the objective function gleaned from the successes and failures of previous iterations. Directions are viewed as most promising in which the objective function decreases rapidly (for minimization) or increases rapidly (for maximization). Southwell (1946), for example, improves the relaxation by choosing the coordinate directions, not cyclically, but in order of the size of the local gradient in them. If the restriction of parallel axes is removed, the local best direction is given by the (negative) gradient vector rF (x) = (Fx1 (x) Fx2 (x) : : : Fxn (x))T with @F (x) for all i = 1(1)n Fxi (x) = @x i
at the point x . All hill climbing procedures that orient their choice of search directions v(0) according to the rst partial derivatives of the objective function are called gradient strategies. They can be thought of as analogues of the total step procedure of Jacobi for solving systems of linear equations (see Schwarz, Rutishauser, and Stiefel, 1968). So great is the number of methods of this type which have been suggested or applied up to the present, that merely to list them all would be dicult. The reason lies in the fact that the gradient represents a local property of a function. To follow the path of the gradient exactly would mean determining in general a curved trajectory in the ndimensional space. This problem is only approximately soluble numerically and is more dicult than the original optimization problem. With the help of analogue computers continuous gradient methods have actually been implemented (Bekey and McGhee, 1964 Levine, 1964). They consider the trajectory x(t) as a function of time and obtain it as the solution of a system of rst order di erential equations. All the numerical variants of the gradient method di er in the lengths of the discrete steps and thereby also with regard to how exactly they follow the gradient trajectory. The iteration rule is generally rF (x(k)) x(k+1) = x(k) ; s(k) kr F (x(k))k It assumes that the partial derivatives everywhere exist and are unique. If F (x) is continuously di erentiable then the partial derivatives exist and F (x) is continuous. (0)
66
Hill climbing Strategies
A distinction is sometimes drawn between short step methods, which evaluate the gradients again after a small step in the direction rF (x(k)) (for maximization) or ;rF (x(k)) (for minimization), and their equivalent long step methods. Since for nite step lengths s(k) it is not certain whether the new variable vector is really better than the old, after the step the value of the objective function must be tested again. Working with small steps increases the number of objective function calls and gradient evaluations. Besides F (x) n partial derivatives must be evaluated. Even if the slopes can be obtained analytically and can be speci ed as functions, there is no reason to suppose that the number of computational operations per function call is much less than for the objective function itself. Except in special cases, the total cost increases roughly as the weighting factor (n + 1) and the number of objective function calls. This also holds if the partial derivatives are approximated by di erential quotients obtained by means of trial steps
Fxi (x) = @F@x(x) = F (x + e i) ; F (x) + O( 2) i
for all i = 1(1)n
Additional diculties arise here since for values of that are too small the subtraction is subject to rounding error, while for trial steps that are too large the neglect of terms O( 2) leads to incorrect values. The choice of suitable deviations requires special care in all cases (Hildebrand, 1956 Curtis and Reid, 1974). Cauchy (1847), Kantorovich (1940, 1945), Levenberg (1944), and Curry (1944) are the originators of the gradient strategy, which started life as a method of solving equations and systems of equations. It is rst referred to as an aid to solving variational problems by Hadamard (1908) and Courant (1943). Whereas Cauchy works with xed step lengths s(k), Curry tries to determine the distance covered in the (not normalized) direction v(k) = ;rF (x(k)) so as to reach a relative minimum (see also Brown, 1959). In principle, any one of the one dimensional search methods of Section 3.1 can be called upon to nd the optimal value for s(k): (k) (k) F (x(k) + s(k) v(k)) = min s fF (x + s v )g
This variant of the basic strategy could thus be called a longest step procedure. It is better known however under the name optimum gradient method, or method of steepest descent (for maximization, ascent). Theoretical investigations of convergence and rate of convergence of the method can be found, e.g., in Akaike (1960), Goldstein (1962), Ostrowski (1967), Forsythe (1968), Elkin (1968), Zangwill (1969), and Wolfe (1969, 1970, 1971). Zangwill proves convergence based on the assumptions that the line searches are exact and the objective function is continuously twice di erentiable. Exactness of the one dimensional minimization is not, however, a necessary assumption (Wolfe, 1969). It is signi cant that one can only establish theoretically that a stationary point will be reached (rF (x) = 0) or approached (krF (x)k < " " > 0). The stationary point is a minimum, only if F (x) is convex and three times di erentiable (Akaike, 1960). Zellnik, Sondak, and Davis (1962), however, show that saddle points are in practice an obstacle, only if the search is started at one, or on a straight gradient trajectory passing through one. In other cases numerical rounding errors ensure that the path to a saddle point is unstable.
Multidimensional Strategies
67
The gradient strategy, however, cannot distinguish global from local minima. The optimum at which it aims depends only on the choice of the starting point for the search. The only chance of nding absolute extrema is to start suciently often from various initial values of the variables and to iterate each time until the convergence criterion is satis ed (Jacoby, Kowalik, and Pizzo, 1972). The termination rules usually recommended for gradient methods are that the absolute value of the vector krF (x(k))k < "1 "1 0 or the di erence F (x(k;1)) ; F (x(k)) < "2 "2 0 vanishes or falls below a given limit. The rate of convergence of the strategy of steepest descent depends on the structure of the objective function, but is no better than rst order apart from exceptional cases like contours that are concentric, spherically symmetric hypersurfaces (Forsythe, 1968). In the general quadratic case (3.24) F (x) = 21 xT A x + xT b + c with elliptic contours, i.e., positive de nite matrix A, the convergence rate depends on the ratio of smallest to greatest eigenvalue of A, or geometrically expressed, on the oblateness of the ellipsoid. It can be extremely small (Curry, 1944 Akaike, 1960 Goldstein, 1962) and is in principle no better than the coordinate strategy with line searches (Elkin, 1968). In narrow valleys both procedures execute zigzag movements with very small changes in the variable values in relation to the distance from the objective. The individual steps can even become too small to e ect any change in the objective function value if this is de ned with a nite number of decimal places. Then the search ends before reaching the desired optimum. To obviate this diculty, Booth (1949, 1955) has suggested only going 90% of the way to the relative minimum in each line search (see also Stein, 1952 Kantorovich, 1952 Faddejew and Faddejewa, 1973). In fact, one often obtains much better results with this kind of underrelaxation. Even more advantageous is a modi cation due to Forsythe and Motzkin (1951). It is based on the observation that the search movements in the minimization of quadratic objective functions oscillate between two asymptotic directions (Stiefel, 1952 Forsythe, 1968). Forsythe and Motzkin therefore from time to time include a line search in the direction v(k) = x(k) ; x(k;2) for k 2 in order to accelerate the convergence. For n = 2 the gradient method is thereby greatly improved with many variables the eciency advantage is lost again. Similar proposals for increasing the rate of convergence have been made by Baer (1962), Humphrey and Cottrell (1962, 1966), Witte and Holst (1964), Schinzinger (1966), and McCormick (1969). The Partan (acronym for parallel tangents) methods of Buehler, Shah, and Kempthorne (1961, 1964) have attracted particular attention. One of these, continued gradient Partan, alternates between gradient directions v(k) = ;rF (x(k)) for k = 0 as well as k 1 odd
68
Hill climbing Strategies
and those derived from previous iteration points
v(k) = x(k) ; x(k;3) for k 2 even (with x(;1) = x(0)) For quadratic functions the minimum is reached after at most 2 n ; 1 line searches (Shah, Buehler, and Kempthorne, 1964). This desirable property of converging after a
nite number of iterations, that is also called quadratic convergence, is only shared by strategies that apply conjugate gradients, of which the Partan methods can be regarded as forerunners (Pierre, 1969 Sorenson, 1969). In the fties, simple gradient strategies were very popular, especially the method of steepest descent. Today they are usually only to be found as components of program packages together with other hill climbing methods, e.g., in GROPE of Flood and Leon (1966), in AID of Casey and Rustay (1966), in AESOP of Hague and Glatt (1968), and in GOSPEL of Huelsman (1968). McGhee (1967) presents a detailed ow diagram. Wasscher (1963a,b) has published two ALGOL codings (see also Haubrich, 1963 Wallack, 1964 Varah, 1965 Wasscher, 1965). The partial derivatives are obtained numerically. A comprehensive bibliography by Leon (1966b) names most of the older versions of strategies and gives many examples of their application. Numerical comparison tests have been carried out by Fletcher (1965), Box (1966), Leon (1966a), Colville (1968, 1970), and Kowalik and Osborne (1968). They show the superiority of rst (and second) order methods over direct search strategies for objective functions with smooth topology. Gradient methods for solving systems of di erential equations are described for example by Talkin (1964). For such problems, as well as for functional optimization problems, analogue and hybrid computers have often been applied (Rybashov, 1965a,b, 1969 Sydow, 1968 Fogarty and Howe, 1968, 1970). A literature survey on this subject has been compiled by Gilbert (1967). For the treatment of variational problems see Kelley (1962), Altman (1966), Miele (1969), Bryson and Ho (1969), C%ea (1971), Daniel (1971), and Tolle (1971). In the experimental eld, there are considerable diculties in determining the partial derivatives. Errors in the values of the objective function can cause the predicted direction of steepest descent to lie almost perpendicular to the true gradient vector (Kowalik and Osborne, 1968). Box and Wilson (1951) attempt to compensate for the perturbations by repeating the trial steps or increasing their number above the necessary minimum of (n + 1). With 2n trials, for example, a complete factorial design can be constructed (e.g., Davies, 1954). The slope in one direction is obtained by averaging the function value di erences over 2n;1 pairs of points (Lapidus et al., 1961). Another possibility is to determine the coecients of a linear polynomial such that the sum of the squares of the errors between measured and model function values at N n + 1 points is a minimum. The linear function then represents the tangent plane of the objective function at the point under consideration. The cost of obtaining the gradients when there are many variables is too great for practical application, and only justi ed if the aim is rather to set up a mathematical model of the system than simply to perform the optimization. In the EVOP (acronym for evolutionary operation) scheme, G. E. P. Box (1957) has presented a practical simpli cation of this gradient method. It actually counts as a direct search strategy because it does not obtain the direction of the gradient but only one of a
nite number of especially good directions. Spendley, Hext, and Himsworth (1962) have
Multidimensional Strategies
69
devised a variant of the procedure (see also Sections 3.2.1.5 and 3.2.1.6). Lowe (1964) has gathered together the various schemes of trial steps for the EVOP strategy. The philosophy of the EVOP strategy is treated in detail by Box and Draper (1969). Some examples of applications are given by Kenworthy (1967). The eciency of methods of determining the gradient in the case of stochastic perturbations is dealt with by Mlynski (1964a,b, 1966a,b), Sergiyevskiy and Ter-Saakov (1970), and others.
3.2.2.1 Strategy of Powell: Conjugate Directions
The most important idea for overcoming the convergence diculties of the gradient strategy is due to Hestenes and Stiefel (1952), and again comes from the eld of linear algebra (see also Ginsburg, 1963 Beckman, 1967). It trades under the names conjugate directions or conjugate gradients. The directions fvi i = 1(1)ng are said to be conjugate with respect to a positive de nite n n matrix A if (Hestenes, 1956) viT A vj = 0 for all i j = 1(1)n i 6= j A further property of conjugate directions is their linear independence, i.e., n X
i vi = 0 i=1
only holds if all the constants f i i = 1(1)ng are zero. If A is replaced by the unit matrix, A = I , then the vi are mutually orthogonal. With A = r2F (x) (Hessian matrix) the minimum of a quadratic function is obtained exactly in n line searches in the directions vi. This is a factor two better than the gradient Partan method. For general non-linear problems the convergence rate cannot be speci ed. As it is frequently assumed, however, that many problems behave roughly quadratically near the optimum, it seems worthwhile to use conjugate directions. The quadratic convergence of the search with conjugate directions comes about because second order properties of the objective function are taken into account. In this respect it is not, in fact, a rst order gradient method, but a second order procedure. If all the n rst and n2 (n + 1) second partial derivatives are available, the conjugate directions can be generated in one process corresponding to the Gram-Schmidt orthogonalization (Kowalik and Osborne, 1968). It calls for expensive matrix operations. Conjugate directions can, however, be constructed without knowledge of the second derivatives: for example, from the changes in the gradient vector in the course of the iterations (Fletcher and Reeves, 1964). Because of this implicit exploitation of second order properties, conjugate directions has been classi ed as a gradient method. The conjugate gradients method of Fletcher and Reeves consists of a sequence of line searches with Hermitian interpolation (see Sect. 3.1.2.3.4). As a rst search direction v(0) at the starting point x(0), the simple gradient direction v(0) = ;rF (x(0)) is used. The recursion formula for the subsequent iterations is v(k) = (k) v(k;1) ; rF (x(k)) for all k = 1(1)n (3.25)
70
Hill climbing Strategies
with the correction factor (k) T F (x(k))
(k) = rFr(Fx((kx;1)))T r rF (x(k;1))
For a quadratic objective function with a positive de nite Hessian matrix, conjugate directions are generated in this way and the minimum is found with n line searches. Since at any time only the last direction needs to be stored, the storage requirement increases linearly with the number of variables. This often signi es a great advantage over other strategies. In the general, non-linear, non-quadratic case more than n iterations must be carried out, for which the method of Fletcher and Reeves must be modi ed. Continued application of the recursion formula (Equation (3.25)) can lead to linear dependence of the search directions. For this reason it seems necessary to forget from time to time the accumulated information and to start afresh with the simple gradient direction (Crowder and Wolfe, 1972). Various suggestions have been made for the frequency of this restart rule (Fletcher, 1972a). Absolute reliability of convergence in the general case is still not guaranteed by this approach. If the Hessian matrix of second partial derivatives has points of singularity, then the conjugate gradient strategy can fail. The exactness of the line searches also has an important e ect on the convergence rate (Kawamura and Volz, 1973). Polak (1971) de nes conditions under which the method of Fletcher and Reeves achieves greater than linear convergence. Fletcher (1972c) himself has written a FORTRAN program. Other conjugate gradient methods have been proposed by Powell (1962), Polak and Ribi)ere (1969) (see also Klessig and Polak, 1972), Hestenes (1969), and Zoutendijk (1970). Schley (1968) has published a complete FORTRAN program. Conjugate directions are also produced by the projected gradient methods (Myers, 1968 Pearson, 1969 Sorenson, 1969 Cornick and Michel, 1972) and the memory gradient methods (Miele and Cantrell, 1969, 1970 see also Cantrell, 1969 Cragg and Levy, 1969 Miele, 1969 Miele, Huang, and Heidemann, 1969 Miele, Levy, and Cragg, 1971 Miele, Tietze, and Levy, 1972 Miele et al., 1974). Relevant theoretical investigations have been made by, among others, Greenstadt (1967a), Daniel (1967a, 1970, 1973), Huang (1970), Beale (1972), and Cohen (1972). Conjugate gradient methods are encountered especially frequently in the elds of functional optimization and optimal control problems (Daniel, 1967b, 1971 Pagurek and Woodside, 1968 Nenonen and Pagurek, 1969 Roberts and Davis, 1969 Polyak, 1969 Lasdon, 1970 Kelley and Speyer, 1970 Kelley and Myers, 1971 Speyer et al., 1971 Kammerer and Nashed, 1972 Szego and Treccani, 1972 Polak, 1972 McCormick and Ritter, 1974). Variable metric strategies are also sometimes classi ed as conjugate gradient procedures, but more usually as quasi-Newton methods. For quadratic objective functions they generate the same sequence of points as the Fletcher-Reeves strategy and its modi cations (Myers, 1968 Huang, 1970). In the non-quadratic case, however, the search directions are di erent. With the variable metric, but not with conjugate directions, Newton directions are approximated. For many practical problems it is very dicult if not impossible to specify the partial derivatives as functions. The sensitivity of most conjugate gradient methods to imprecise
Multidimensional Strategies
71
speci cation of the gradient directions makes it seem inadvisable to apply nite di erence methods to approximate the slopes of the objective function. This is taken into account by some procedures that attempt to construct conjugate directions without knowledge of the derivatives. The oldest of these was devised by Smith (1962). On the basis of numerical tests by Fletcher (1965), however, the version of Powell (1964) has proved to be better. It will be briey presented here. It is arguable whether it should be counted as a gradient strategy. Its intermediate position between direct search methods that only use function values, and Newton methods that make use of second order properties of the objective function (if only implicitly), nevertheless makes it come close to this category. The strategy of conjugate directions is based on the observation that a line through the minimum of a quadratic objective function cuts all contours at the same angle. Powell's idea is then to construct such special directions by a sequence of line searches. The unit vectors are taken as initial directions for the rst n line searches. After these, a minimization is carried out in the direction of the overall result. Then the rst of the old direction vectors is eliminated, the indices of the remainder are reduced by one and the direction that was generated and used last is put in the place freed by the nth vector. As shown by Powell, after n cycles, each of n + 1 line searches, a set of conjugate directions is obtained provided the objective function is quadratic and the line searches are carried out exactly. Zangwill (1967) indicates how this scheme might fail. If no success is obtained in one of the search directions, i.e., the distance covered becomes zero, then the direction vectors are linearly dependent and no longer span the complete parameter space. The same phenomenon can be provoked by computational inaccuracy. To prevent this, Powell has modi ed the basic algorithm. First of all, he designs the scheme of exchanging directions to be more exible, actually by maximizing the determinant of the normalized direction vectors. It can be shown that, assuming a quadratic objective function, it is most favorable to eliminate the direction in which the largest distance was covered (see Dixon, 1972a). Powell would also sometimes leave the set of directions unchanged. This depends on how the value of the determinant would change under exchange of the search directions. The objective function is here tested at the position given by doubling the distance covered in the cycle just completed. Powell makes the termination of the search depend on all variables having changed by less than 0:1 " within an iteration, where " represents the required accuracy. Besides this rst convergence criterion, he o ers a second, stricter one, according to which the state reached at the end of the normal procedure is slightly displaced and the minimization repeated until the termination conditions are again ful lled. This is followed by a line search in the direction of the di erence vector between the last two endpoints. The optimization is only nally ended when the result agrees with those previously obtained to within the allowed deviation of 0:1 " for each component. The algorithm of Powell runs as follows: Step 0: (Initialization) Specify a starting point x(0) and accuracy requirements "i > 0 for all i = 1(1)n.
72
Hill climbing Strategies
Step 1: (Specify rst set of directions) Set vi(0) = ei for all i = 1(1)n and set k = 0. Step 2: (Start outer loop) Set x(k0) = x(k) and i = 1. Step 3: (Line search) Determine x(ki) such that (ki;1) F (x(ki)) = min + s vi(k))g: s fF (x Step 4: (Inner loop) If i < n increase i i + 1 and go to step 3. Step 5: (First termination criterion) Increase k k + 1. (k) Set x(k) = x(k;1n) and vi = vi(k;1) for all i = 1(1)n. If jx(ik) ; x(ik;1)j < 0:1 "i for all i = 1(1)n, go to step 9. Step 6: (First test for direction exchange) Determine F~ = F (2 x(k) ; x(k;1)). If F~ F (x(k;1)), go to step 2. Step 7: (Second test for direction exchange) Determine the index ` 1 ` n such that '` = max f'i i = 1(1)ng where 'i = F (x(k;1i;1)) ; F (x(k;1i)). i If "F (x(k;1)) ; 2 F (x(k)) + F~ ] "F (x(k;1)) ; F (x(k)) ; '` ]2 21 '` "F (x(k;1)) ; F~ ]2, go to step 2. Step 8: (New direction set and additional line search) Eliminate v`(k) from the list of directions so that vn(k) becomes free. Set vn(k) = x(k) ; x(k;1) = x(k;1n) ; x(k;10). Determine a new x(k) such that (k;1n) F (x(k)) = min + s vn(k))g: s fF (x Go to step 2. Step 9: (Second termination criterion) n Set y(1) = x(k) and replace x(0) y(1) + P 10 " ei . i=1 Repeat steps 1 to 8 until the convergence criterion (step 5) is ful lled again and call the result y(2). Determine y(3) such that (2) (2) (1) F (y(3)) = min s fF (y + s (y ; y ))g: If jyi(3) ; yi(2)j < 0:1 "i for all i = 1(1)n and jyi(3) ; yi(1)j < 0:1 "i for all i = 1(1)n, then end the search with the result y(3) and F (y(3))
Multidimensional Strategies
73
otherwise set x(0) = y(3) v1(0) = y(3) ; y(1) vi(0) = vi(k) for i = 2(1)n, k = 0, and go to step 2. Figure 3.9 illustrates a few iterations for a hypothetical two parameter function. Each of the rst loops consists of n + 1 = 3 line searches and leads to the adoption of a new search direction. If the objective function had been of second order, the minimum would certainly have been found by the last line search of the second loop. In the third and fourth loops it has been assumed that the trial steps have led to a decision not to exchange directions, thus the old direction vectors, numbered v3 and v4 are retained. Further loops, e.g., according to step 9, are omitted. The quality of the line searches has a strong inuence on the construction of the conjugate directions. Powell uses a sequence of Lagrangian quadratic interpolations. It is terminated as soon as the required accuracy is reached. For the rst minimization within an iteration three points and Equation (3.16) are used. The argument values taken in direction vi are: x (the starting point), x + si vi, and either x + 2 si vi or x ; si vi, according to whether F (x + si vi) < F (x) or not. The step length si is given initially by the associated accuracy "i multiplied by a maximum factor and is later adjusted in the course of the iterations. In the direction constructed from the successful results of n one dimensional searches, the argument values are called x(k0) x(kn), and 2 x(kn) ; x(k0). With three points (a < b < c) and associated objective function values (Fa Fb Fc), not only the minimum but also the second derivative of the quadratic trial function P (x) can be speci ed. In the notation of Section 3.1.2.3.3, the formula for the curvature i in the direction vi is Starting point End points of line searches Tests for direction exchange v 1
End points of iterations
(0,1)
(0,0)
v 2 (0,2)
v 3
(0,3)=(1,0)
(1,1) (3,1)
(2,1)
(1,2) v 4
(1,3)=(2,0)
(3,3)=(4,0) v (3,2) 5
(2,2)=(3,0)
Figure 3.9: Strategy of Powell, conjugate directions
74
Hill climbing Strategies
@ 2 (P (x + s v )) = ;2 (b ; c) Fa + (c ; a) Fb + (a ; b) Fc i = @s i 2 (b ; c) (c ; a) (a ; b) Powell uses this quantity i for all subsequent interpolations in the direction vi as a scale for the second partial derivative of the objective function. He scales the directions vi, p which in his case are not normalized, by 1= i. This allows the possibility of subsequently carrying out a simpli ed interpolation with only two argument values, x and x + si vi. It is a worthwhile procedure, since each direction is used several times. The predicted minimum, assuming that the second partial derivatives have value unity, is then x0 = x + 12 si ; s1 "F (x + si vi) ; F (x)] vi i For the trial step lengths si, Powell uses the empirical recursion formula q s(ik) = 0:4 F (x(k)) ; F (x(k;1)) Because of the scaling, all the step lengths actually become the same. A more detailed justi cation can be found in Ho mann and Hofmann (1970). Contrary to most other optimization procedures, Powell's strategy is available as a precise algorithm in a tested code (Powell, 1970f). As Fletcher (1965) reports, this method of conjugate directions is superior for the case of a few variables both to the DSC method and to a strategy of Smith, especially in the neighborhood of minima. For many variables, however, the strict criterion for adopting a new direction more frequently causes the old set of directions to be retained and the procedure then converges slowly. A problem which had a singular Hessian matrix at the minimum made the DSC strategy look better. In a later article, Fletcher (1972a) de nes a limit of n = 10 to 20, above which the Powell strategy should no longer be applied. This is con rmed by the test results presented in Chapter 6. Zangwill (1967) combines the basic idea of Powell with relaxation steps in order to avoid linear dependence of the search directions. Some results of Rhead (1971) lead to the conclusion that Powell's improved concept is superior to Zangwill's. Brent (1973) also presents a variant of the strategy without derivatives, derived from Powell's basic idea, which is designed to prevent the occurrence of linear dependence of the search directions without endangering the quadratic convergence. After every n + 1 iterations the set of directions is replaced by an orthogonal set of vectors. So as not to lose all the information, however, the unit vectors are not chosen. For quadratic objective functions the new directions remain conjugate to each other. This procedure requires O(n3 ) computational operations to determine orthogonal eigenvectors. As, however, they are only performed every O(n2 ) line searches, the extra cost is O(n) per function call and is thus of the same order as the cost of evaluating the objective function itself. Results of tests by Brent con rm the usefulness of the strategy.
3.2.3 Newton Strategies
Newton strategies exploit the fact that, if a function can be di erentiated any number of times, its value at the point x(k+1) can be represented by a series of terms constructed at
Multidimensional Strategies
75
another point x(k): where
F (x(k+1)) = F (x(k)) + hT rF (x(k)) + 12 hT r2F (x(k)) h + : : :
(3.26)
h = x(k+1) ; x(k) In this Taylor series, as it is called, all the terms of higher than second order are zero if F (x) is quadratic. Di erentiating Equation (3.26) with respect to h and setting the derivative equal to zero, one obtains a condition for the stationary points of a second order function: rF (x(k+1)) = rF (x(k)) + r2 F (x(k)) (x(k+1) ; x(k) ) = 0
or
x(k+1) = x(k) ; "r2F (x(k))];1 rF (x(k)) (3.27) If F (x) is quadratic and r2F (x(0)) is positive-de nite, Equation (3.27) yields the solution x(1) in a single step from any starting point x(0) without needing a line search. If Equation (3.27) is taken as the iteration rule in the general case it represents the extension of the Newton-Raphson method to functions of several variables (Householder, 1953). It is also sometimes called a second order gradient method with the choice of direction and step length (Crockett and Cherno , 1955) v(k) = ;"r2F (x(k))];1 rF (x(k)) s(k) = 1
(3.28)
The real length of the iteration step is hidden in the non-normalized Newton direction v . Since no explicit value of the objective function is required, but only its derivatives, the Newton-Raphson strategy is classi ed as an indirect or analytic optimization method. Its ability to predict the minimum of a quadratic function in a single calculation at rst sight looks very attractive. This single step, however, requires a considerable e ort. Apart from the necessity of evaluating n rst and n2 (n + 1) second partial derivatives, the Hessian matrix r2F (x(k)) must be inverted. This corresponds to the problem of solving a system of linear equations r2 F (x(k)) 4 x(k) = ;rF (x(k)) (3.29) for the unknown quantities 4x(k). All the standard methods of linear algebra, e.g., Gaussian elimination (Brown and Dennis, 1968 Brown, 1969) and the matrix decomposition method of Cholesky (Wilkinson, 1965), need O(n3) computational operations for this (see Schwarz, Rutishauser, and Stiefel, 1968). For the same cost, the strategies of conjugate directions and conjugate gradients can execute O(n) steps. Thus, in principle, the Newton-Raphson iteration o ers no advantage in the quadratic case. If the objective function is not quadratic, then v (0) does not in general point towards a minimum. The iteration rule (Equation (3.27)) must be applied repeatedly. (k )
76
Hill climbing Strategies
s(k) = 1 may lead to a point with a worse value of the objective function. The search diverges, e.g., when r2F (x(k)) is not positive-de nite.
It can happen that r2F (x(k)) is singular or almost singular. The Hessian matrix cannot be inverted.
Furthermore, it depends on the starting point x(0) whether a minimum, a maximum, or a saddle point is approached, or the whole iteration diverges. The strategy itself does not distinguish the stationary points with regard to type. If the method does converge, then the convergence is better than of linear order (Goldstein, 1965). Under certain, very strict conditions on the structure of the objective function and its derivatives even second order convergence can be achieved (e.g., Polak, 1971) that is, the number of exact signi cant gures in the approximation to the minimum solution doubles from iteration to iteration. This phenomenon is exhibited in the solution of some test problems, particularly in the neighborhood of the desired extremum. All the variations of the basic procedure to be described are aimed at increasing the reliability of the Newton iteration, without sacri cing the high convergence rate. A distinction is made here between quasi-Newton strategies, which do not evaluate the Hessian matrix explicitly, and modied Newton methods, for which rst and second derivatives must be provided at each point. The only strategy presently known which makes use of higher than second order properties of the objective function is due to Biggs (1971, 1973). The simplest modi cation of the Newton-Raphson scheme consists of determining the step length s(k) by a line search in the Newton direction v(k) (Equation (3.28)) until the relative optimum is reached (e.g., Dixon, 1972a): (k) (k) F (x(k) + s(k) v(k)) = min s fF (x + s v )g
(3.30)
To save computational operations, the second partial derivatives can be redetermined less frequently and used for several iterations. Care must always be taken, however, that v(k) always points \downhill," i.e., the angle between v(k) and ;rF (x(k)) is less than 900 . The Hessian matrix must also be positive-de nite. If the eigenvalues of the matrix are calculated when it is inverted, their signs show whether this condition is ful lled. If a negative eigenvalue appears, Pearson (1969) suggests proceeding in the direction of the associated eigenvector until a point is reached with positive-de nite r2F (x). Greenstadt (1967a) simply replaces negative eigenvalues by their absolute value and vanishing eigenvalues by unity. Other proposals have been made to keep the Hessian matrix positivede nite by addition of a correction matrix (Goldfeld, Quandt, and Trotter, 1966, 1968 Shanno, 1970a) or to include simple gradient steps in the iteration scheme (Dixon and Biggs, 1972). Further modi cations, which operate on the matrix inversion procedure itself, have been suggested by Goldstein and Price (1967), Fiacco and McCormick (1968), and Matthews and Davies (1971). A good survey has been given by Murray (1972b). Very few algorithms exist that determine the rst and second partial derivatives numerically from trial step operations (Whitley, 1962 see also Wasscher, 1963c Wegge, 1966). The inevitable approximation errors too easily cancel out the advantages of the Newton directions.
Multidimensional Strategies
77
3.2.3.1 DFP: Davidon-Fletcher-Powell Method (Quasi-Newton Strategy, Variable Metric Strategy)
Much greater interest has been shown for a group of second order gradient methods that attempt to approximate the Hessian matrix and its inverse during the iterations only from
rst order data. This now extensive class of quasi-Newton strategies has grown out of the work of Davidon (1959). Fletcher and Powell (1963) improved and translated it into a practical procedure. The Davidon-Fletcher-Powell or DFP method and some variants of it are also known as variable metric strategies. They are sometimes also regarded as conjugate gradient methods, because in the quadratic case they generate conjugate directions. For higher order objective functions this is no longer so. Whereas the variable metric concept is to approximate Newton directions, this is not the case for conjugate gradient methods. The basic recursion formula for the DFP method is
x(k+1) = x(k) + s(k) v(k) with and
v(k) = ;H (k)T rF (x(k)) H (0) = I
H (k+1) = H (k) + A(k) The correction A(k) to the approximation for the inverse Hessian matrix, H (k), is derived from information collected during the last iteration thus from the change in the variable vector y(k) = x(k+1) ; x(k) = s(k) v(k) and the change in the gradient vector z(k) = rF (x(k+1)) ; rF (x(k)) it is given by
(k) (k) (k) (k) T (k) (k)T A(k) = yy(k)Tyz(k) ; H z(zk)T H(H(k) z(zk) )
(3.31)
The step length s(k) is obtained by a line search along v(k) (Equation (3.30)). Since the rst partial derivatives are needed in any case they can be made use of in the one dimensional minimization. Fletcher and Powell do so in the context of a cubic Hermitian interpolation (see Sect. 3.1.2.3.4). A corresponding ALGOL program has been published by Wells (1965) (for corrections see Fletcher, 1966 Hamilton and Boothroyd, 1969 House, 1971). The rst derivatives must be speci ed as functions, which is usually inconvenient and often impossible. The convergence properties of the DFP method have been thoroughly investigated, e.g., by Broyden (1970b,c), Adachi (1971), Polak (1971), and Powell (1971, 1972a,b,c). Numerous suggestions have thereby been made for improvements. Convergence is achieved if F (x) is convex. Under stricter conditions it can be proved that the convergence rate is greater than linear and the sequence of iterations
78
Hill climbing Strategies
converges quadratically, i.e., after a nite number (maximum n) of steps the minimum of a quadratic function is located. Myers (1968) and Huang (1970) show that, if the same starting point is chosen and the objective function is of second order, the DFP algorithm generates the same iteration points as the conjugate gradient method of Fletcher and Reeves. All these observations are based on the assumption that the computational operations, including the line searches, are carried out exactly. Then H (k) always remains positivede nite if H (0) was positive-de nite and the minimum search is stable, i.e., the objective function is improved at each iteration. Numerical tests (e.g., Pearson, 1969 Tabak, 1969 Huang and Levy, 1970 Murtagh and Sargent, 1970 Himmelblau, 1972a,b), and theoretical considerations (Bard, 1968 Dixon, 1972b) show that rounding errors and especially inaccuracies in the one dimensional minimization frequently cause stability problems the matrix H (k) can easily lose its positive-de niteness without this being due to a singularity in the inverse Hessian matrix. The simplest remedy for a singular matrix H (k), or one of reduced rank, is to forget from time to time all the experience stored within H (k) and to begin again with the unit matrix and simple gradient directions (Bard, 1968 McCormick and Pearson, 1969). To do so certainly increases the number of necessary iterations, but in optimization as in other activities it is wise to put safety before speed. Stewart (1967) makes use of this procedure. His algorithm is of very great practical interest since he obtains the required information about the rst partial derivatives from function values alone by means of a cleverly constructed di erence scheme.
3.2.3.2 Strategy of Stewart: Derivative-free Variable Metric Method
Stewart (1967) focuses his attention on choosing the length of the trial step d(ik) for the approximation (k) @F (x) (k) gi ' Fxi x = @x (k) i x to the rst partial derivatives in such a way as to minimize the inuence of rounding errors on the actual iteration process. Two di erence schemes are available: h i gi(k) = (1k) F (x(k) + d(ik) ei) ; F (x(k)) (forward di erence) di and h i gi(k) = 1(k) F (x(k) + d(ik) ei) ; F (x(k) ; d(ik) ei) (central di erence) (3.32) 2 di Application of the one sided (forward) di erence (Equation (3.32)) is preferred, since it only involves one extra function evaluation. To simplify the computation, Stewart introduces the vector h(k), which contains the diagonal elements of the matrix (H (k));1 representing information about the curvature of the objective function in the coordinate directions. The algorithm for determining the gi(k) i = 1(1)n runs as follows:
Multidimensional Strategies Step 0:
Step 1:
Step 2:
(
79
) (k) j jxi j F (x(k))
(k;1)
jgi
Set = max "b "c ("b represents an estimate of the error in the calculation of F (x). Stewart sets "b = 10;10 and "c = 5 10;13 :)
2 If gi(k;1) h(ik;1) F (x(k)) v (k) 0 1 u 0 (k;1) u h F ( x ) i i A t (k;1) and i = i0 @1 ; 0 (k;1) de ne i0 = 2 u hi 3 i hi + 4 gi(k;1) otherwise de ne v (k) (k;1) (k;1) 0 1 u u F ( x ) g 2 g i i0 = 2 t3 and i = i0 @1 ; 0 (k;1)i (k;1) A : (k;1) 2 (hi ) 3 i hi + 4 gi
Set di(k) = i sign(h(ik;1)) sign(gi(k;1)) ( 0 (k) 0 (k ) d if d (k) i i0 6= 0 and di = (k;1) di if di(k) = 0: (k;1) (k) di 10; , use Equation (3.32) otherwise If hi (k;1) 2 gi r replace d(ik) (k1;1) ; gi(k;1) + (gi(k;1))2 + 2 10 F (x(k)) h(ik;1) hi and use Equation (3.33). (Stewart chooses = 2.) Stewart's main algorithm takes the following form: Step 0: (Initialization) Choose an initial value x(0), accuracy requirements "ai > 0 i = 1(1)n, and initial step lengths d(0) i for the gradient determination, e.g., 8 if x(0) 6= 0 < 0:05 x(0) (0) i i di = : 0:05 if x(0) i = 0: Calculate the vector g(0) from Equation (3.32) using the step lengths d(0) i : (0) (0) Set H = I hi = 1 for all i = 1(1)n and k = 0: Step 1: (Prepare for line search) Determine v(k) = ;H (k) g(k): If k = 0, go to step 3. If g((kk))T v(k) < 0, go to step 3. If hi > 0 for all i = 1(1)n, go to step 3. 0
80
Hill climbing Strategies
Step 2: (Forget second order information) Replace H(0)(k) H (0) = I (k) hi hi = 1 for all i = 1(1)n and v(k) ;g(k): Step 3: (Line search and eventual break-o ) Determine x(k+1) such that (k) (k) F (x(k+1)) = min s fF (x + s v )g: If F (x(k+1)) F (x(k)), end the search with result x(k) and F (x(k)). Step 4: (Prepare for inverse Hessian update) Determine g(k+1) by the above di erence scheme. Construct y(k) =(k)x (k+1) ; x(k) and z(k) = g(k+1) ; g(k): (k) If k > n and vi < "ai and yi < "ai for all i = 1(1)n, end the search with result x(k+1) F (x(k+1)). Step 5: (Update inverse Hessian) Construct H (k+1) = H (k) + A(k) using Equation (3.31) and ! (k) (k) (k)T (k) z (k) (k) (k+1) (k) s g v ( k ) i hi = hi + (k)T (k) zi "1 ; (k)T (k) ] + 2 s gi v z v z for all i = 1(1)n: Step 6: (Main loop / termination criterion) If the denominators are non-zero, increase k k + 1, and go to step 1 otherwise terminate with result x(k+1) F (x(k+1)). In place of the cubic Hermitian interpolation, Stewart includes a quadratic Lagrangian interpolation as used by Powell in his conjugate directions strategy. Gradient calculations at the argument values are thereby avoided. One point, x(k), is given each time by the initial vector of the line search. The second, x(k) + s v(k), is situated in the direction v(k) at a distance ( ) (k) 2 ( F ( x ) ; Fm) s = min 1 ; g(k)T v(k)
Fm is an estimate of the value of the objective function at the minimum being sought. It must be speci ed beforehand. s = 1 is an upper bound corresponding to the length of a Newton-Raphson step. The third argument value is to be calculated from knowledge of the points x(k) and x(k) + s v(k), their associated objective function values, and g(k)T v(k), the derivative of the objective function at point x(k) in direction v(k). The sequence of Lagrangian interpolations is broken o if, at any time, the predicted minimum worsens the situation or lies at such a distance outside the interval that it is more than twice as far from the next point as the latter is from the midpoint. Lill (1970, 1971) (see also Kov%acs and Lill, 1971) has published a complete ALGOL program for the derivative-free DFP strategy of Stewart it di ers slightly from the original only in the line search. Fletcher (1969b) reports tests that demonstrate the superiority of Stewart's algorithm to Powell's as the number of variables increases.
Multidimensional Strategies
81
Brown and Dennis (1972) and Gill and Murray (1972) have suggested other schemes for obtaining the partial derivatives numerically from values of the objective function. Stewart himself reports tests that show the usefulness of his rules insofar as the results are completely comparable to others obtained with the help of analytically speci ed derivatives. This may be simply because rounding errors are in any case more signi cant here, due to the matrix operations, than for example in conjugate gradient methods. Kelley and Myers (1971), therefore, recommend carrying out the matrix operations with double precision.
3.2.3.3 Further Extensions
The ability of the quasi-Newton strategy of Davidon, Fletcher, and Powell (DFP) to construct Newton directions without needing explicit second partial derivatives makes it very attractive from a computational point of view. All e orts in the further rapid and intensive development of the concept have been directed to modifying the correction Equation (3.31) so as to reduce the tendency to instability because of rounding errors and inexact line searches while retaining as far as possible the quadratic convergence. There has been a spate of corresponding proposals and both theoretical and experimental investigations on the subject up to about 1973, for example: Adachi (1973a,b) Bass (1972) Broyden (1967, 1970a,b,c, 1972) Broyden, Dennis, and Mor%e (1973) Broyden and Johnson (1972) Davidon (1968, 1969) Dennis (1970) Dixon (1972a,b,c, 1973) Fiacco and McCormick (1968) Fletcher (1969a,b, 1970b, 1972b,d) Gill and Murray (1972) Goldfarb (1969, 1970) Goldstein and Price (1967) Greenstadt (1970) Hestenes (1969) Himmelblau (1972a,b) Hoshino (1971) Huang (1970, 1974) Huang and Chambliss (1973, 1974) Huang and Levy (1970) Jones (1973) Lootsma (1972a,b) Mamen and Mayne (1972) Matthews and Davies (1971) McCormick and Pearson (1969)
82
Hill climbing Strategies
McCormick and Ritter (1972, 1974) Murray (1972a,b) Murtagh (1970) Murtagh and Sargent (1970) Oi, Sayama, and Takamatsu (1973) Oren (1973) Ortega and Rheinboldt (1972) Pierson and Rajtora (1970) Powell (1969, 1970a,b,c,g, 1971, 1972a,b,c,d) Rauch (1973) Ribi)ere (1970) Sargent and Sebastian (1972, 1973) Shanno and Kettler (1970a,b) Spedicato (1973) Tabak (1969) Tokumaru, Adachi, and Goto (1970) Werner (1974) Wolfe (1967, 1969, 1971) Many of the di erently sophisticated strategies, e.g., the classes or families of similar methods de ned by Broyden (1970b,c) and Huang (1970), are theoretically equivalent. They generate the same conjugate directions v(k) and, with an exact line search, the same sequence x(k) of iteration points if F (x) is quadratic. Dixon (1972c) even proves this identity for more general objective functions under the condition that no term of the sequence H (k) is singular. The important nding that under certain assumptions convergence can also be achieved without line searches is attributed to Wolfe (1967). A recursion formula satisfying these conditions is as follows: H (k+1) = H (k) + B (k) where (k ) (k) z (k)) (y (k) ; H (k) z (k) )T (3.33) B (k) = (y ;(H y(k) ; H (k) z(k)) z(k)T The formula was proposed independently by Broyden (1967), Davidon (1968, 1969), Pearson (1969), and Murtagh and Sargent (1970) (see Powell, 1970a). The correction matrix B (k) has rank one, while A(k) in Equation (3.31) is of rank two. Rank one methods, also called variance methods by Davidon, cannot guarantee that H (k) remains positive-de nite. It can also happen, even in the quadratic case, that H (k) becomes singular or B (k) increases without bound. Hence in order to make methods of this type useful in practice a number of additional precautions must be taken (Powell, 1970a Murray, 1972c). The following compromise proposal H (k+1) = H (k) + A(k) + (k) B (k) (3.34) where the scalar parameter (k) > 0 can be freely chosen, is intended to exploit the advantages of both concepts while avoiding their disadvantages (e.g., Fletcher, 1970b). Broyden
Multidimensional Strategies
83
(1970b,c), Shanno (1970a,b), and Shanno and Kettler (1970) give criteria for choosing suitable (k). However, the mixed correction, also known as BFS or Broyden-Fletcher-Shanno formula, cannot guarantee quadratic convergence either unless line searches are carried out. It can be proved that there will merely be a monotonic decrease in the eigenvalues of the matrix H (k). From numerical tests, however, it turns out that the increased number of iterations is usually more than compensated for by the saving in function calls made by dropping the one dimensional optimizations (Fletcher, 1970a). Fielding (1970) has designed an ALGOL program following Broyden's work with line searches (Broyden, 1965). With regard to the number of function calls it is usually inferior to the DFP method but it sometimes also converges where the variable metric method fails. Dixon (1973) de nes a correction to the chosen directions,
v(k) = ;H (k) rF (x(k)) + w(k) where and
w(0) = 0
(k+1) (k) T (k+1) w(k+1) = w(k) + (x (x(k;+1)x ; )x(kr))FT (zx(k) ) (x(k+1) ; x(k)) by which, together with a matrix correction as given by Equation (3.35), quadratic convergence can be achieved without line searches. He shows that at most n + 2 function calls and gradient calculations are required each time if, after arriving at v(k) = 0, an iteration x(k+1) = x(k) ; H (k) rF (x(k)) is included. Nearly all the procedures de ned assume that at least the rst partial derivatives are speci ed as functions of the variables and are therefore exact to the signi cant
gure accuracy of the computer used. The more costly matrix computations should wherever possible be executed with double precision in order to keep down the e ect of rounding errors. Just two more suggestions for derivative-free quasi-Newton methods will be mentioned here: those of Greenstadt (1972) and of Cullum (1972). While Cullum's algorithm, like Stewart's, approximates the gradient vector by function value di erences, Greenstadt attempts to get away from this. Analogously to Davidon's idea of approximating the Hessian matrix during the course of the iterations from knowledge of the gradients, Greenstadt proposes approximating the gradients by using information from objective function values over several subiterations. Only at the starting point must a di erence scheme for the rst partial derivatives be applied. Another interesting variable metric technique described by Elliott and Sworder (1969a,b, 1970) combines the concept of the stochastic approximation for the sequence of step lengths with the direction algorithms of the quasi-Newton strategy. Quasi-Newton strategies of degree one are especially suitable if the objective function is a sum of squares (Bard, 1970). Problems of minimizing a sum of squares arise for example from the problem of solving systems of simultaneous, non-linear equations,
84
Hill climbing Strategies
or determining the parameters of a mathematical model from experimental data (nonlinear regression and curve tting). Such objective functions are easier to handle because Newton directions can be constructed straight away without second partial derivatives, as long as the Jacobian matrix of rst derivatives of each term of the objective function is given. The oldest iteration procedure constructed on this basis is variously known as the Gauss-Newton (Gauss, 1809) method, generalized least squares method, or Taylor series method. It has all the advantages and disadvantages of the Newton-Raphson strategy. Improvements on the basic procedure are given by Levenberg (1944) and Marquardt (1963). Wolfe's secant method (Wolfe, 1959b see also Jeeves, 1958) is the forerunner of many variants which do not require the Jacobian matrix to be speci ed at the start but construct it in the course of the iterations. Further details will not be described here the reader is referred to the specialist literature, again up to 1973: Barnes, J.G.P. (1965) Bauer, F.L. (1965) Beale (1970) Brown and Dennis (1972) Broyden (1965, 1969, 1971) Davies and Whitting (1972) Dennis (1971, 1972) Fletcher (1968, 1971) Golub (1965) Jarratt (1970) Jones (1970) Kowalik and Osborne (1968) Morrison (1968) Ortega and Rheinboldt (1970) Osborne (1972) Peckham (1970) Powell (1965, 1966, 1968b, 1970d,e, 1972a) Powell and MacDonald (1972) Rabinowitz (1970) Ross (1971) Smith and Shanno (1971) Spath (1967) (see also Silverman, 1969) Stewart (1973) Vitale and Taylor (1968) Zeleznik (1968) Brent (1973) gives further references. Peckham's strategy is perhaps of particular interest. It represents a modi cation of the simplex method of Nelder and Mead (1965) and Spendley (1969) and in tests it proves to be superior to Powell's strategy (1965) with regard to the number of function calls. It should be mentioned here at least that non-linear regression, where parameters of a model that enter the model in a non-linear way (e.g., as exponents) have to be estimated, in general requires a global optimization
Multidimensional Strategies
85
method because the squared sum of residuals de nes a multimodal function. Reference has been made to a number of publications in this and the preceding chapter in which strategies are described that can hardly be called genuine hill climbing methods they would fall more naturally under the headings of mathematical programming or functional optimization. It was not, however, the intention to give an introduction to the basic principles of these two very wide subjects. The interested reader will easily nd out that although a nearly exponentially increasing number of new books and journals have become available during the last three decades, she or he will look in vain for new direct search strategies in that realm. Such methods form the core of this book.
86
Hill climbing Strategies
Chapter 4 Random Strategies One group of optimization methods has been completely ignored in Chapter 3: methods in which the parameters are varied according to probabilistic instead of deterministic rules even the methods of stochastic approximation are deterministic. As indicated by the title there is not one random strategy but many, some of which dier considerably from each other. It is common to resort to random decisions in optimization whenever deterministic rules do not have the desired success, or lead to a dead end on the other hand random strategies are often supposed to be essentially more costly. The opinion is widely held that with careful thought leading to cleverly constructed deterministic rules, better results can always be achieved than with decisions that are in some way made randomly. The strategies that follow should show that randomness is not, however, the same as arbitrariness, but can also be made to obey very rened rules. Sometimes only this kind of method solves a problem eectively. Profound considerations do not underlie all the procedures used in hill climbing strategies. The cyclic choice of coordinate directions in the Gauss-Seidel strategy could just as well be replaced by a random sequence. One can also consider increasing the number of directions used. Since there is no good reason for preferring to search for the optimum along directions parallel to the axes, one could also use, instead of only n dierent unit vectors, any number of randomly chosen direction vectors. In fact, suggestions along these lines have been made (Brooks, 1958) in order to avoid a premature termination of the minimum search in narrow oblique valleys (compare Chap. 3, Sect. 3.2.1.1). Similar concepts have been developed for example by O'Hagan and Moler (after Wilde and Beightler, 1967), Emery and O'Hagan (1966), Lawrence and Steiglitz (1972), and Beltrami and Indusi (1972), to improve the pattern search of Hooke and Jeeves (1961, see Chap. 3, Sect. 3.2.1.2). The limitation to a nite number of search directions is not only a disadvantage in narrow oblique valleys but also at the border of the feasible region as determined by inequality constraints. All the deterministic remedies against prematurely ending the iteration sequence assume that more information can be gathered, for example in the form of partial derivatives of the constraint functions (see Klingman and Himmelblau, 1964 Glass and Cooper, 1965 Paviani and Himmelblau, 1969). Providing this information usually means a high extra cost and is sometimes not possible at all. 87
88
Random Strategies
Random directions that are not oriented with respect to the structure of the objective function and the allowed region also imply a higher cost because they do not take optimal single steps. They can, however, be applied in every case. Many deterministic optimization methods, especially those which are guided by the gradient of the objective function, have convergence diculties at points where the partial derivatives are discontinuous. On the contour diagram of a two parameter objective function, of which the maximum is sought, such positions correspond to sharp ridges leading to the summit (e.g., Zwart, 1970). A narrow valley{the geometric picture in the case of minimization{leads to the same problem if the nite step lengths are greater than its width. Then all attempts fail to make improvements in the coordinate directions or, from trial steps in these directions, fail to predict a locally best direction in which to continue (gradient direction). The same phenomenon can also occur when the partial derivatives are specied analytically, because of the rounding errors involved in computing with a nite number of signicant gures. To avoid premature termination of a search in such cases, Norkin (1961) has suggested the following procedure. When the optimization according to the conventional scheme has ended, a step is taken away from the supposed optimum in an arbitrary coordinate direction. The extremum is sought again, excluding this one variable, and the search is only nally ended when deviations in all directions have led back to the same point. This rule should also prevent stagnation at saddle points. Even the simplex method of linear programming makes random decisions if the search for the extremum threatens to be endless because the problem is degenerate. Then following Dantzig's suggestion (1966) the iteration scheme should be interrupted in favor of a random exchange step. A problem is only degenerate, however, because the general rules do not cover the special case (see also Chap. 6, Sect. 6.2). A further example of resorting to chance when a dead end has been reached is Brent's modication of the strategy with conjugate directions (Brent, 1973). Powell's algorithm (Powell, 1964) when applied to problems in many dimensions tends to generate linearly dependent directions and then to proceed within a subspace of IRn. For this reason Brent now and then interrupts the line searches with steps in randomly chosen directions (see also Chap. 3, Sect. 3.2.2.1). One very frequently comes across proposals to let chance take control when the problem is to nd global minima of multimodal objective functions. Such problems frequently crop up in process design (Motskus, 1967 Mockus, 1971) but can also be the result of recasting discrete problems into continuous form (Katkovnik and Shimelevich, 1972). Practically all sequential search procedures can only lead to a local optimum{as a rule, the one nearest to the starting point. There are a few proposals for ensuring global convergence of sequential optimization methods (e.g., Motskus and Feldbaum, 1963 Chichinadze, 1967, 1969 Goldstein and Price, 1971 Ueing, 1971, 1972 Branin and Hoo, 1972 McCormick, 1972 Sutti, Trabattoni, and Brughiera, 1972 Treccani, Trabattoni, and Szego, 1972 Brent, 1973 Hesse, 1973 Opacic, 1973 Ritter and Tui as mentioned by Zwart, 1973). They are often in the form of additional, heuristic rules. Gran (1973), for example, considers gradient methods that are supposed to achieve global convergence by the addition of a random process to the deterministic changes. Hill (1964 see also Hill and Gibson, 1965) suggests subdividing the interval to be explored and gathering sucient information in each section to carry out a cubic interpolation. The best of the results for the
Random Strategies
89
parts is taken as an approximation to the global optimum. However, for n-dimensional interpolations the cost increases rapidly with n this scheme thus looks impractical for more than two variables. To work with several, randomly chosen starting points and to compare each of the local minima (or maxima) obtained is usually regarded as the only course of action for determining the global optimum with at least a certain probability (so-called multistart techniques). Proposals along these lines have been made by, among others, Gelfand and Tsetlin (1961), Bromberg (1962), Bocharov and Feldbaum (1962), Zellnik, Sondak, and Davis (1962), Krasovskii (1962), Gurin and Lobac (1963), Flood and Leon (1964, 1966), Kwakernaak (1965), Casey and Rustay (1966), Weisman and Wood (1966), Pugh (1966), McGhee (1967), Crippen and Scheraga (1971), and Brent (1973). A further problem faces deterministic strategies if the calculated or measured values of the objective function are subject to stochastic perturbations. In the experimental eld, for example in the on-line optimum search, or for control of the optimal conditions in processes, perturbations must be taken into account from the start (e.g., Tovstucha, 1960 Feldbaum, 1960, 1962 Krasovskii, 1963 Medvedev, 1963, 1968 Kwakernaak, 1966 Zypkin, 1967). However, in computational optimization too, where the objective function is analytically specied, a similar eect arises because of rounding errors (Brent, 1973), especially if one uses hybrid analogue computers for solving functional optimization problems (e.g., Gilbert, 1967 Korn and Korn, 1964 Bekey and Karplus, 1971). A simple, if expensive (in the sense of cost in computations or trials) method of dealing with this is the repetition of measurements until a denite conclusion is possible. This is the procedure adopted by Box and Wilson (1951) in the experimental gradient method, and by Box (1957) in his EVOP strategy. Instead of a xed number of repetitions, which while on the safe side may be unnecessarily high, one can follow the concept of sequential analysis of statistical data (Wald, 1966 see also Zigangirov, 1965 Schumer, 1969 Kivelidi and Khurgin, 1970 Langguth, 1972), which is to make only as many trials as the trial results seem to make absolutely necessary. More detailed investigations on this subject have been made, for example, by Mlynski (1964a,b, 1966a,b). As opposed to attempting to improve the decisive data, Brooks and Mickey (1961) have found that one should work with the minimum number of n + 1 comparison points in order to determine a gradient direction, even if this is a perturbed one. One must however depart from the requirement that each step should yield a success, or even the locally greatest success. The motto that following locally the best possible route seldom leads to the best overall result is true not only for rst order gradient strategies but also for Newton and quasi-Newton methods . Harkins (1964), for example, maintains that inexact line searches not only do not worsen the convergence of a minimization procedure but in some cases actually improve it. Similar experiences led Davies, Swann, and Campey in their strategy (see Chap. 3, Sect. 3.2.1.4) to make only one quadratic interpolation in each direction. Also Spendley, Hext, and Himsworth (1962), in the formulation of their simplex method, which generates only near-optimal directions, work on the assumption that random decisions are not necessarily a total disadvantage (see also Himsworth, 1962). Based on similar arguments, the modication of this strategy by M. J. Box (1965) sets up the initial simplex or complex by means of random numbers. Imamura et al. (1970) even go so far as to superimpose articial stochastic variations on an objective function
90
Random Strategies
in order to prevent convergence to inferior local optima. The rigidity of an algorithm based on a xed internal model of the objective function, with which the information gathered during the iterations is interpreted, is advantageous if the objective function corresponds closely enough to the model. If this is not the case, the advantage disappears and may even turn into a disadvantage. Second order methods with quadratic models seem more sensitive in this respect than rst order methods with only linear models. Even more robust are the direct search strategies that work without an explicit model, such as the strategy of Hooke and Jeeves (1961). It makes no use of the sizes of the changes in the objective function values, but only of their signs. A method that uses a kind of minimal model of the objective function is the stochastic approximation (Schmetterer, 1961 see also Chap. 2, Sect. 2.3). This purely deterministic method assumes that the measured or calculated function values are samples of a normally distributed random quantity, of which the expectation value is to be minimized or maximized. The method feels its way to the optimum with alternating exploratory and work steps, whose lengths form convergent series with prescribed bounds and sums. In the multidimensional case this standard concept can be the basis of various strategies for choosing the directions of the work steps (Fabian, 1968). Usually gradient methods show themselves to best advantage here. The stochastic approximation itself is very versatile. Constraints can be taken into account (Kaplinskii and Propoi, 1970), and problems of functional optimization can be treated (Gersht and Kaplinskii, 1971) as well as dynamic problems of maintaining or seeking optima (Chang, 1968). Tsypkin (1968a,b,c, 1970a,b see also Zypkin, 1966, 1967, 1970) discusses these topics very thoroughly. There are also, however, arguments against the reliability of convergence for certain types of objective function (Aizerman, Braverman and Rozonoer, 1965). The usefulness of the strategy in the multidimensional case is limited by its high cost. Hence there has been no shortage of attempts to accelerate the convergence (Fabian, 1967 Berlin, 1969 Saridis, 1968, 1970 Saridis and Gilbert, 1970 Janac, 1971 Kwatny, 1972 see also Chap. 2, Sect. 2.3). Ideas for using random directions look especially promising some of the many investigations of this topic which have been published are Loginov (1966), Stratonovich (1968, 1970), Schmitt (1969), Ermoliev (1970), Svechinskii (1971), Tsypkin (1971), Antonov and Katkovnik (1972), Berlin (1972), Katkovnik and Kulchitskii (1972), Kulchitskii (1972), Poznyak (1972), and Tsypkin and Poznyak (1972). The original method is not able to determine global extrema reliably. Extensions of the strategy in this direction are due to Kushner (1963, 1972) and Vaysbord and Yudin (1968). The sequence of work steps is so designed that the probability of the following state being the global optimum is maximized. In contrast to the gradient concept, the information gathered is not interpreted in terms of local but of global properties of the objective function. In the case of two local minima, the eort of the search is gradually concentrated in their neighborhood and only when one of them is signicantly better is the other abandoned in favor of the one that is also a global minimum. In terms of the cost of the strategy, the acceleration of the local search and the reliability of the global search are diametrically opposed. Hill and Gibson (1965) show that their global strategy is superior to Kushner's, as well as to one of Bocharov and Feldbaum. However, they only treat cases with n 2 parameters. More recent research results have been presented by
Random Strategies
91
Pardalos and Rosen (1987), Torn and Z ilinskas (1989), Floudas and Pardalos (1990), Zhigljavsky (1991), and Rudolph (1991, 1992b). Now there are even specialized journals established in the eld, see Horst (1991). All the strategies mentioned so far are fundamentally deterministic. They only resort to chance in dead-end situations, or they operate on the assumption that the objective function is stochastically perturbed. Jarvis (1968), who compares deterministic and probabilistic optimization methods, nds that random methods that do not stick to any particular model are most suitable when an optimum must be located under particularly dicult conditions, such as a perturbed objective function or a \pathological" problem structure with several extrema, discontinuities, plateaus, forbidden regions, etc. The homeostat of Ashby (1960) is probably the oldest example of the application of a random strategy. Its objective is to maintain a condition of equilibrium against stochastic disturbances. It may happen that no optimum is sought, but only a point in an allowed region (today one calls such task a constraints satisfaction problem or CSP). Nevertheless, corresponding solution methods are closely tied to optimization, and there are a series of various heuristic planning methods available (e.g., Weinberg and Zehnder, 1969). Ashby's strategy, which he calls a blind homeostatic process, becomes active whenever the apparatus strays from equilibrium. Then the controllable parameters are randomly varied until the desired condition is restored. The nite number (in this case) of discrete settings of the variables all enter the search process with equal probability. Chichinadze (1960) later constructed an electronic model on the same principle and used it for synthesizing simple optimal control systems. Brooks (1958), probably stimulated by R. L. Anderson (1953), is generally regarded as the initiator of the use of random strategies for optimization problems. He describes the simple, later also called blind or pure random search for nding a minimum or maximum in the experimental eld. In a closed interval a x b several points are chosen at random. The probability density w(x) is constant everywhere within the region and zero outside. ( for all a x b w(x) = 10=V otherwise V , the volume of the cube with corners ai and bi for i = 1(1)n, is given by n Y V = (bi ; ai) i=1
The value of the objective function must be determined at all selected points. The point that has the lowest or highest function value is taken as optimum. How well the true extremum is approximated depends on the number of trials as well as on the actual random results. Thus one can only give a probability p that the optimum will be found within a given number N of trials with a prescribed accuracy.
p = 1 ; (1 ; v=V )N
(4.1)
The volume v < V < 1 contains all points that satisfy the accuracy requirement. By
92
Random Strategies
rearranging Equation (4.1), the number of trials is obtained N = ln (1 ; vp) (4.2) ln (1 ; V ) that is required in order to place with probability p at least one trial in the volume v. Brooks concludes from this that the cost is independent of the number of variables. In their criticism Hooke and Jeeves (1958) point out that it is not feasible to consider the accuracy in terms of the volumev ratio for problems with many variables. For n = 100 parameters, a volume ratio of V = 0:1 corresponds to a length ratio of the side length D of V and d of v of s d = n v ' 0:98 D V This means that the uncertainty in the variables xi is 98% of the original interval ai bi], although the volume containing the optimum has been reduced to one tenth of the original. Shimizu (1969) makes the same mistake as Brooks and attempts to implement the strategy for problems with more general constraints. A comparison of the pure random search and deterministic search methods known at the time for experimental optimization problems (Brooks, 1959) also shows no advantage of the stochastic strategy. The test only covers four dierent objective functions, each with two variables. Brooks then recommends applying his random method if the number of parameters is large or if the determination of objective function values is subject to large perturbations. McArthur (1961) concludes on the basis of numerical experiments that the random strategy is also preferable for complicated problem structures. Just this circumstance has led to the use, even today, of the pure random search, often called the Monte-Carlo method, for example in computer optimization of building construction (Golinski and Lesniak, 1966 Lesniak, 1970 Hupfer, 1970). In principle, all the trials of the simple random strategy can be made simultaneously. It is thus numbered among the simultaneous optimization methods. The decision to choose a particular state vector of variables does not depend on the results of preceding trials, since the probability of scoring according to the uniform distribution is the same at all times. However, in applications on the traditional, serially operating computers, the trials must be made sequentially. This can be used to advantage by storing the current best value of the objective function and its associated variable value. In Chapter 3, Section 3.1.1 and 3.2 the grid or tabulation method was referred to as optimal in the minimax sense. The blind random strategy should thus not be any better. Dening the interval length Di = bi ; ai for the variable xi, with required accuracy di , and assuming that all the Di = D and di = d for i = 1(1)n, then for the volume ratio in Equations (4.1) and (4.2) ! v= d n V D If Vv is small, which when there are many variables must be the case, one can use the approximation ln (1 + y) ' y for y 1
Random Strategies
93
to write the number of required trials as
n N ' ; ln(1 ; p) Dd
Assuming that Dd is an integer, the grid method requires D n N= d trials (compare Chap. 3, Sect. 3.2, Equation (3.19)). The value is the same for both procedures if p ' 0:63. Supposing that the probability of at least one score of the required accuracy is p = 0:90, then the random strategy results in D n N ' 2:3 d which is clearly worse than the grid strategy (Spang, 1962). The reason for the extra cost, however, should not be attributed to the randomness of decisions itself, but to the fact that for an equiprobable, continuous selection of variables, the trials can be very close together or, in the discrete case, they can repeat themselves. If one can avoid that, the disadvantage would no longer exist. A randomized sequence of trials even might hit upon the optimal result earlier than an ordered one. Nevertheless Spang's proof has for some time brought all random methods, not only the simple Monte-Carlo strategy, into disrepute. Nowadays the term Monte-Carlo methods is understood to cover, in general, simulation methods that have to do with stochastic events. They are applied eectively to solving dicult dierential equations (Little, 1966) or for evaluating integrals (Cowdrey and Reeves, 1963 McGhee and Walford, 1968). Besides the simple hit-or-miss scheme, however, greatly improved variants have been developed (e.g., W. F. Bauer, 1958 Hammersley and Handscomb, 1964 Korn, 1966, 1968 Hull, 1967 Brandl, 1969). Amann (1968a,b) reports a Monte-Carlo method with information storage and a sequential extension for the solution of a linear boundary value problem, and Curtiss (1956) describes a Monte-Carlo procedure for solving systems of linear equations. Both are supposed to be less costly than comparable deterministic strategies. Pinkham (1964) and Pincus (1970) describe modications for the problems of nding zeros of a non-linear function and of constrained optimization. Since only relatively few publications treat random optimization methods in any depth (Karnopp, 1961, 1963 Idelsohn, 1964 Dickinson, 1964 Rastrigin, 1963, 1965a,b, 1966, 1967, 1968, 1969, 1972 Lavi and Vogl, 1966 Schumer, 1967 Jarvis, 1968 Heydt, 1970 Cockrell, 1970 White, 1970, 1971 Aoki, 1971 Kregting and White, 1971), the improved strategies will be brie!y presented here. They all operate with sequential and sometimes both simultaneous and sequential random trials and in one way or another exploit the information from preceding trials to accelerate the convergence. Brooks himself already suggests several improvements. Thus to exclude repetitions or closely situated trials, the volume to be investigated can be subdivided into, for example, cubic subspaces, into each of which only one random trial is placed. According to one's
94
Random Strategies
knowledge of the approximate position of the optimum, the subspaces will be assigned dierent sizes (Idelsohn, 1964). The original uniform distribution is thereby replaced by one with a greater density in the neighborhood of the expected optimum. Karnopp (1961, 1963, 1966) has treated this problem in detail without, however, giving any practical procedure. Mathematically based investigations of the same topic are due to Motskus (1965), Hupfer (1970), Pluznikov, Andreyev, and Klimenko (1971), Yudin (1965, 1966, 1972), Vaysbord (1967, 1968, 1969), Taran (1968a,b), Karumidze (1969), and Meerkov (1972). If after several (simultaneous) samples the search is continued in an especially promising looking subregion, the procedure becomes sequential in character. Suggestions of this kind have been made for example by McArthur (1961), Motskus (1965), and Hupfer (1970) (shrinkage random search). Zakharov (1969, 1970) applies the stochastic approximation for the successive shrinkage of the region in which Monte-Carlo samples are placed. The most thoroughly worked out strategy is that of McMurtry and Fu (1966, probabilistic automaton see also McMurtry, 1965). The problem considered is to adjust the variable parameters of a control system for a dynamic process in such a way that the optimum of the system is found and maintained despite perturbations and (slow) drift (Hill, McMurtry, and Fu, 1964 Hill and Fu, 1965). Initially the probabilities are equal for all subregions, at the center of which the function values are measured (assumed to be stochastically perturbed). In the course of the iterations the probability matrix is altered so that regions with better objective function values are tested more often than others. The search ends when only one subregion remains: the one with the highest probability of containing the global optimum. McMurtry and Fu use a so-called linear intensication to adjust the probability matrix. Suggestions for further improving the convergence rate have been made by Nikolic and Fu (1966), Fu and Nikolic (1966), Shapiro and Narendra (1969), Asai and Kitajima (1972), Viswanathan and Narendra (1972), and Witten (1972). Strongin (1970, 1971) treats the same problem from the point of view of decision theory. All these methods lay great emphasis on the reliability of global convergence. The quality of the approximation depends to a large extent on the number of subdivisions of the n-dimensional region under investigation. High accuracy requirements cannot be met for many variables since, at least initially, the number of subregions to investigate rises exponentially with the number of parameters. To improve the local convergence properties, there are suggestions for replacing the midpoint tests in a subvolume by the result of an extreme value search. This could be done with one of the familiar search strategies such as a gradient method (Hill, 1969) or any other purely sequential random search method (Jarvis 1968, 1970) with a high convergence rate, even if it were only guaranteed to converge locally. Application, however, is limited to problems with at most seven or eight variables, as reported. Another possibility for giving a sequential character to random methods consists of gradually shifting the expectation value of a random variable with a restricted probability density distribution. Brooks (1958) calls his proposal of this type the creeping random search. Suitable random numbers are provided for example by a Gaussian distribution with expectation value and standard deviation . Starting from a chosen initial condition x(0), several simultaneous trials are made, which most likely fall in the neighborhood of the starting point ( = x(0)). The coordinates of the point with the best function value form
Random Strategies
95
the expectation value for the next set of random trials. In contrast to other procedures, the data from the other trials are not exploited to construct a linear or even quadratic model from which to calculate a best possible step (e.g., Brooks and Mickey, 1961 Aleksandrov, Sysoyev, and Shemeneva, 1968 Pugachev, 1970). For small and a large number of samples, the best value will in any case fall in the locally most favorable direction. In order to approach a solution with high accuracy, the variance 2 must be successively reduced. Brooks, however, gives no practical rule for this adjustment. Many algorithms have since been published that are extensions of Brooks' basic concept of the creeping random search. Most of them no longer choose the best of several trials they accept each improvement and reject each worsening (Favreau and Franks, 1958 Munson and Rubin, 1959 Wheeling, 1960). The iteration rule of a creeping random search is, for the minimum search: ( (k) (k) F (x(k) + z(k)) F (x(k)) (success) ( k +1) x = xx(k) + z ifotherwise (failure) The random vector z(k), which in this notation eects the change in the state vector x, belongs to an n-dimensional (0 2) normal distribution with the expectation value = 0 and the variance 2, whichp in the simplest case is the same for all components. One can thus regard , or better n, as a kind of average step length. The direction of z(k) is uniformly distributed in IRn , i.e., purely random. Gaussian distributions for the increments are also used by Bekey et al. (1966), Stewart, Kavanaugh, and Brocker (1967), and De Graag (1970). Gonzalez (1970) and White (1970) use instead of a normal distribution a uniform distribution that covers a small region in the form of an n-dimensional cube centered on the starting point. This clearly favors the diagonal directions, in which the total p step lengths are on average a factor n greater than in the coordinate directions. Pierre (1969) therefore restricts the uniformly distributed random probe to an n-dimensional hypersphere of xed radius. Rastrigin (1960{1972) gives the total step length v u n uX s = t zi2 i=1
a xed value. Instead of the normal distribution he thus obtains a circumferential or hypersphere-surface distribution. In addition, he repeats the evaluation of the objective function when there is a failure in order to reduce the eect of stochastic perturbations. Taking two model functions n X F1(x) = F1(x1 : : : xn) = xi (inclined plane) iv =1 u n uX F2(x) = F2(x1 : : : xn) = t x2i (hypercone) i=1
he investigates the average convergence rate of his strategy and compares it with that of an experimental gradient method, in which the partial derivatives are approximated by quotients of dierences obtained from exploratory steps . He shows that for a linear
96
Random Strategies
p
problem structure like F1 the random strategy needs only O( n) trials, whereas the gradient strategy needs O(n) trials to cover a prescribed distance. For n > 3, the random strategy is always superior to the deterministic method. Whereas Rastrigin shows that the random search always does better than the gradient search in the spherically symmetric eld F2, Movshovich (1966) maintains the opposite. The discrepancy can be traced to diering assumptions about the choice of step length (see also Yvon, 1972 Gaviano and Fagiuoli, 1972). To choose suitable step lengths or variances poses the same problems for sequential random searches as are familiar from deterministic strategies. Here too, a closely related problem is to achieve global convergence with reference to a suitable termination rule, the convergence criterion, and with a degree of reliability. Khovanov (1967) has conceived an individual manner of controlling the random step lengths. He accepts every random change, irrespective of success or failure, increases the variance at each failure and reduces it otherwise. The objective is to increase the probability of lingering in the more promising regions and to abandon states that are irrelevant to the optimum search. No applications of the strategy are known to the author. Favreau and Franks (1958), Bekey et al. (1966), and Adams and Lew (1966) use a constant ratio between i and xi for i = 1(1)n. This measure does have the eect of continuously altering the \step lengths," but its merit is not obvious. Just because a variable value xi is small in no way indicates that it is near to the extreme position being sought. Karnopp (1961) was the rst to propose a step length rule based on the number of successes or failures, according to which the i or s are all uniformly reduced or enlarged such that a success always occurs after two or three trials. Schumer (1967), and Schumer and Steiglitz (1968), submit Rastrigin's circumferential random direction method to a thorough examination by probability theory. For the model n X F3(x) = x2i = r2 i=1
with the condition n 1 and the continuously optimal step length s ' 1:225 prn they obtain a rate of progress ', which is the average distance covered in the direction of the objective (minimum) per random step: ' ' 0:203 nr and a success rate ws which is the average number of successes per trial: ws ' 0:270 They are only able to treat the general quadratic case theoretically for n = 2. Their result can be interpreted in the sense that ' is dependent on the smallest radius of curvature of the elliptic contour passing through r. Since neither r nor s can be assumed to be known in advance, it is not clear how to keep to the optimal step length. Schumer and Steiglitz (1968) give an adaptive method with which the correct size of s can be
Random Strategies
97
maintained at least approximately during the course of the iterations. At the starting point x(0) two random changes are made with step lengths s(0) and s(0) (1 + a), where 0 < a 1. If both samples are successful, for the next iteration s(1) = s(0) (1 + a) is taken, i.e., the greater value. If only one sample yields an improvement in the objective function, its step length is taken nally if no success is scored, s(1) remains equal to s(0). A reduction in s is only made if several consecutive trials are unsuccessful. This is also the procedure of Maybach (1966). This adjustment to the local conditions assists the strategy in achieving high convergence rates but reduces the chances of locating global optima among several local ones. For this reason a sample with a signicantly larger step length (a > 1) should be included from time to time. Numerical tests show that the computation cost, or number of trials, actually only increases linearly with the number of variables. Schumer and Steiglitz have tested this using the model functions F3 and n X F4(x) = x4i i=1
A comparison with a Newton-Raphson strategy, in which the partial rst and second derivatives are determined numerically and the cost increases as O(n2 ), favors the random method when n > 78 for F3 and when n > 2 for F4. For the second, biquadratic model function, Nelder and Mead (1965) state that the number of trials or function evaluations in their simplex strategy grows as O(n2:11), so that the sequential random method is superior from n > 10. White and Day (1971) report numerical tests in which the cost in iterations with Schumer's strategy increases more sharply than linearly with n, whereas a modication by White (1970) shows exact linear dependence. A comparison with the strategy of Fletcher and Powell (1963) favors the latter, especially for truly quadratic functions. Rechenberg (1973), with an n-dimensional normal distribution (see Chap. 5, Sect. 5.1), reaches almost the same theoretical results as Schumer for the circumferential distribution, if one notes that the overall step length v u n p uX tot = t i2 = n i=1
for equal variances i2 = 2 in each random component zi is proportional to the square root of the number of variables. The reason for this lies in the property of Euclidean space that, as the number of dimensions increases, the volume of a hypersphere becomes concentrated more and more in the boundary region near the surface. Rechenberg's adaptation rule is founded on the relation between optimal variance and probability of success derived from two essentially dierent models of the objective function. The adaptation rule which is thereby formulated makes the frequency and size of the increments respectively dependent on the number of variables and independent of the structure of the objective function. This will be discussed in more detail in Chapter 5, Section 5.1. Convergence proofs for the sequential random strategy have been given by Matyas (1965, 1967) and Rechenberg (1973) only for the case of constant variance 2. Gurin (1966) has proved convergence also for stochastically perturbed objective functions. The
98
Random Strategies
convergence rate is still reduced by perturbations (Gurin and Rastrigin, 1965), but not as much as in gradient methods. Global convergence can be achieved if the reference value of the objective function is measured more than once at the comparison point (Saridis and Gilbert, 1970). As soon as any attempt is made to achieve higher rates of convergence by adjusting the variances or step lengths, the chance of nding a global optimum diminishes. Then the random strategy itself becomes a path-oriented instead of a volume-oriented strategy. The probability of global convergence still always remains nite it may simply become very small, especially in the case of many dimensions. Apart from adjusting the step lengths, one can consider modifying the directions. Several proposals of this kind have been published: Satterthwaite (1959a following McArthur, 1961), Wheeling (1960), Smith and Rudd (1964 following Dickinson, 1964), Matyas (1965, 1967), Bekey et al. (1966), Stewart, Kavanaugh, and Brocker (1967), De Graag (1970), and Lawrence and Emad (1973). They are all heuristic in nature. In the simplest case of a directed random search, a successful random direction is maintained until a failure occurs (Satterthwaite). Bekey, Lawrence, and Rastrigin actually make use of each random direction. If the rst step leads to a failure, they use the opposite direction (positive and negative absolute biasing). Smith and Rudd store the two currently best points from a larger series of samples and obtain from their separation a step length for continuing the optimization. Wheeling's history vector method adds to each random increment a deterministic portion, derived from experience. This additional vector is initially zero. It is increased at each success by a fraction of the increment vector, and correspondingly decreased at each failure. Such a learning and forgetting process also forms the basis of the algorithms of De Graag and Matyas. The latter has received the most attention, in spite of the fact that it gives no precise guidance on how to choose the variances. Schrack and Borowski (1972), who apply their own step length rule in Matyas' strategy, were able to show by numerical tests that the simple algorithm of Schumer and Steiglitz, without direction orientation, is at least as good as Matyas' for unperturbed as well as perturbed measurements of the objective function. A quite dierent kind of method, due to Kjellstrom (1965), in which the random search takes place in varying three dimensional subspaces of the space IRn, shows itself here to be very much worse. Another method that sets out to accept only especially favorable directions is the threshold strategy of Stewart, Kavanaugh and Brocker (1967), in which only those random changes are accepted that result in a specied minimum improvement in the objective function value. A more recent version of the same idea has been given by Dueck and Scheuer (1990). The simultaneous adjustment of step lengths and directions has seldom been attempted. The suggestions of Favreau and Franks (1958) and Matyas (1965, 1967) remain too imprecise to be practicable. Gaidukov (1966 see also Hupfer, 1970) and Furst, Muller, and Nollau (1968) provide more exact information for this purpose, based on either the concepts of Rastrigin or Matyas. Modication of the expectation values and variances of the random vectors is made according to the success or failure of iterations. No applications of the strategy are known, however, so that for the time being the observation of Schrack and Borowski (1972) still stands, namely that a careful choice of the step lengths is the most important prerequisite for the rapid convergence of a random method. A method devised by Rastrigin (1965a,b, 1968) and developed further by Heydt (1970)
Random Strategies
99
works entirely with a restricted choice of directions. With a xed step length, a direction can be randomly selected only from within an n-dimensional hypercone. The angle subtended by the cone and its height (and thus the overall step length) are controlled in an adaptive way. For a spherical objective function, e.g., the model functions F2 (hypercone), F3 (hypersphere), or F4 (something intermediate between hypersphere and hypercube), there is no improvement in the convergence behavior. Advantages can only be gained if the search has to follow a particular direction for a long time along a narrow valley. Sudden changes in direction present a problem, however, which leads Heydt to consider substituting for the cone conguration a hyper-parabolic or hyper-hyperbolic distribution, with which at least small step lengths would retain sucient freedom of direction. In every case the striving for rapid convergence is directly opposed to the reliability of global convergence. This has led Jarvis (1968, 1970) to investigate a combination of the method of Matyas (1965, 1967) with that of McMurtry and Fu (1966). Numerical tests by Cockrell (1969, 1970 see also Fu and Cockrell, 1970) show that even here the basic strategy of Matyas (1965) or Schumer and Steiglitz (1967) is clearly the better alternative. It oers high convergence rates besides a fair chance of locating global optima, at least for a small number of variables. In the case of many dimensions, every attempt to reach global reliability is thwarted by the excessive cost. This leaves the globally convergent stochastic approximation method of Vaysbord and Yudin (1968) far behind the rest of the eld. Furthermore, the sequential or creeping random search is the least susceptible if perturbations act on the objective function. Users of random strategies always draw attention to their simplicity, !exibility and resistance to perturbations. These properties are especially important if one wishes to construct automatic optimalizers (e.g., Feldbaum, 1958 Herschel, 1961 Medvedev and Ruban, 1967 Krasnushkin, 1970). Rastrigin actually built the rst optimalizer with a random search strategy, which was designed for automatic frequency control of an electric motor. Mitchell (1964) describes an extreme value controller that consists of an analogue computer with a permanently wired-in digital part. The digital part serves for storage and !ow control, while the analogue part evaluates the objective function. The development of hybrid analogue computers, in which the computational inaccuracy is determined by the system, has helped to bring random methods, especially of the sequential type, into more general use. For examples of applications besides those of the authors mentioned above, the following publications can be referred to: Meissinger (1964), Meissinger and Bekey (1966), Kavanaugh, Stewart, and Brocker (1968), Korn and Kosako (1970), Johannsen (1970, 1973), and Chatterji and Chatterjee (1971). Hybrid computers can be applied to best advantage for problems of optimal control and parameter identication, because they are able to carry out integrations and dierentiations more rapidly than digital computers. Mutseniyeks and Rastrigin (1964) have devised a special algorithm for the dynamic control problem of keeping an optimum. Instead of the variable position vector x, a velocity vector with components @xi=@t is varied. A randomly chosen combination is retained as long as the objective function is decreasing in value (for minimization @F=@t < 0). As soon as it begins to increase again, a new velocity vector is chosen at random. It is always striking, if one observes living beings, how well adapted they are in shape, function, and lifestyle . In many cases, biological structures, processes, and systems even
100
Random Strategies
surpass the capabilities of highly developed technical systems. Recognition of this has for years led many authors to suspect that nature is in possession of optimal solutions to her problems. In some cases the optimality of biological subsystems can even be demonstrated mathematically, for example for the ratios of diameters in branching arteries (Cohn, 1954), for the hematocrit value (the volume fraction of solid particles in the blood Lew, 1972), and the position of branch points in a level system of blood vessels (Kamiya and Togawa, 1972 see also Grassmann, 1967, 1968 Rosen, 1967 Rein and Schneider, 1971). According to the theory of the descent of the species, all organisms that exist today are the (intermediate) result of a long process of development: evolution. Based on the multitude of nds of transitional species that have since become extinct, paleontology is providing a gradually more complete picture of this development. Leaving aside supernatural explanations, one must assume that the development of optimal or at least very good structures is a property of evolution, i.e., evolution is, or possesses, an optimization (or better, meliorization) strategy. In evolution, the mechanism of variation is the occurrence of random exchanges, even \errors," in the transfer of genetic information from one generation to the next. The selection criterion favors the better suited individuals in the so-called struggle for existence. The similarity of variation and selection to the iteration rules of direct optimization methods is, in fact, striking. This analogy is most often drawn for random strategies, since mutations can best be interpreted as random changes. Thus Ashby (1960) regards as mutations the stochastic parameter variations in his blind homeostatic process. For many variables, however, the pure or blind random search requires so many trials that it offers no acceptable explanation of the capabilities of natural structures, processes, and systems. With the highest possible physical rate of transfer of information, as given by Bremermann (1962 see also Ashby, 1965, 1968) of 1047 bits per second and gram of computer mass, the mass of the earth and the extent of its lifetime up to now would not be sucient to solve even simple combinatorial problems by complete enumeration or a blind random search, never mind to determine the optimal conguration of the 104 to 105 genes with their information content of around 1010 bits (Bremermann, 1963). Evolution must rather be considered as a sequential process that exploits the information from preceding successes and failures in order to follow a trajectory, although not a completely deterministic one, in the n-dimensional parameter space. Brooks (1958) and Favreau and Franks (1958) are therefore right to compare their creeping random search with biological evolution. Yet it is also certainly a very much simplied imitation of the natural process of development. In the 1960s, two proposals that consciously think of higher evolution principles as optimization rules to be simulated are due to Rechenberg (1964, 1973) and Bremermann (1962, 1963, 1967, 1968a,b,c, 1970, 1971, 1973a,b see also Bremermann, Rogson, and Sala, 1965, 1966 Bremermann and Lam, 1970). Bremermann reasons from the (nowadays!) low mutation rates observed in nature that only one component of the variable vector should be varied at a time. He then encounters with this scheme the same diculties as arise in the coordinate method. On the basis of his failure with the mutation-selection scheme, for example on linear programming problems, he comes to the conclusion that ecological niches are actually only stagnation points in development, and they do not represent optimal states of adaptation. None of his many attempts to invoke
Random Strategies
101
the principles of population, sexual inheritance, recombination, dominance, and recessiveness to improve the convergence behavior yield the hoped for breakthrough. He thus eventually resigns himself to a largely deterministic strategy. In the linear programming problem, he chooses from the starting point several random directions and follows these in turn up to the boundary of the feasible region. The best states on the individual bounding hyperplanes are used to determine a new starting point by taking the arithmetic mean of the component parameters. Because of the convexity of the allowed region, the new starting point is always within it. The simultaneous choice of several search directions is supposed to be the analogue of the population principle and the construction of the average the analogue of recombination in sexual propagation. To tackle the problem of nding the minimum or maximum of an unconstrained, non-linear function, Bremermann even applies a ve point Lagrangian interpolation to determine relative extrema in the random directions. Rechenberg's evolution strategy changes all the components of the variable vector at each mutation. In his case, the low mutation rate for many dimensions is expressed by choosing small values for the step lengths, or the spread in the random changes. On the basis of theoretical work with two model functions he nds that the standard deviations of the random components are set optimally when they are inversely proportional to the number of parameters. His two membered evolution strategy resembles the scheme of Schumer and Steiglitz (1968), which is acknowledged to be particularly good, except that a (0 2) normally distributed random quantity replaces the xed step length s. He has also added to it a step length modication rule, again derived from theory, which makes this look a very promising search method. It is rened in Chapter 5, Section 5.1 to meet the requirements of numerical optimization with digital computers. A multimembered strategy is treated in Section 5.2, which follows the same basic concept however, by imitating the principles of population and recombination, it can operate without external control of the step lengths. Incorporating more than one descendant at a time and forgetting \parental wisdom" at the end of each iteration loop has provoked erce objections against a more natural evolution strategy. Box (1957) also considers that his EVOP (evolutionary operation) strategy resembles the biological mutation-selection process. He regards the vertices of his pattern of trial points, of which the best becomes the center of the next pattern, as individuals of a population, of which only the best \survives." The \ospring" are, however, generated by purely deterministic rules. Random decisions, as used by Satterthwaite (1959a after Lowe, 1964) in his REVOP (random evolutionary operation) variant, are actually explicitly rejected by Box (see Youden et al., 1959 Satterthwaite, 1959b Budne, 1959 Anscombe, 1959). From a biological or cybernetic point of view, Pask (1962, 1971), Schmalhausen (1964), Berg and Timofejew-Ressowski (1964), Dobzhansky (1965), Moran (1967), and Kussul and Luk (1971) among others have examined the analogy between optimization and evolution. The fact that no practical algorithms have come out of this is no doubt because the evolutionary processes are described only verbally. Although they sometimes even include their more subtle eects, they have so far not produced a really quantitative, predictive theory. Exceptions, such as the work of Eigen (1971 see also Schuster, 1972), Merzenich
102
Random Strategies
(1972), and Papentin (1972) are so dierent in emphasis that they are not applicable to the kind of problems considered here. The ways in which a process of mathematization can be implemented in theoretical biology are documented in for example the books by Waddington (1968) and Locker (1973), which contain a number of contributions of interest from the optimization point of view, as well as many articles in the journal Mathematical Biosciences, which has been published by R. W. Bellman since 1967, and some papers from two Berkeley symposia (LeCam and Neyman, 1967 LeCam, Neyman, and Scott, 1972). Whereas many modern books on biology, such as Riedl (1976) and Roughgarden (1979), still give mainly verbal explanations of organic evolution, in general, this is no longer the case. Physicists like Ebeling and Feistel (see Feistel and Ebeling, 1989) and biologists like Maynard Smith (1982, 1989) meanwhile have contributed mathematical models. The following two paragraphs thus no longer represent the actual situation, but before we add some new aspects they will be presented, nevertheless, to characterize the situation as perceived by the author in the early 1970s (Schwefel, 1975a): Relationships have been seen between random strategies and biological evolution on the one hand and the psychology of recognition processes on the other, for example, by Campbell (1960) and Khovanov (1967). The imitation of such processes{the catch phrase is articial intelligence{always leads to the problem of choosing or designing a suitable search algorithm, which should rather be heuristic than strictly deterministic. Their simplicity, reliability (even in extreme, unfamiliar situations), and !exibility give the random strategies a special r^ole in this eld. The topic will not be discussed more fully here, except to mention some publications that explicitly deal with the relationship to optimization strategies: Friedberg (1958), Friedberg, Dunham, and North (1959), Minsky (1961), Samuel (1963), J. L. Barnes (1965), Vagin and Rudelson (1968), Thom (1969), Minot (1969), Ivakhnenko (1970), Michie (1971), and Slagle (1972). A particularly impressive example is given by the work of Fogel, Owens, and Walsh (1965, 1966a,b), in which imitation of the biological evolutionary principles of mutation and selection gives a (mathematical) automaton the ability to recognize prescribed sequences of numbers. It may be that in order to match the capabilities of the human brain{and to understand them{there must be a move away from the digital methods of present serial computers to quite dierent kinds of switching elements and coupling principles. Such concepts, as pursued in neurocybernetics and neurobionics, are described, for example, by Brajnes and Svecinskij (1971). The development of the perceptron by Rosenblatt (1958) can be seen as a rst step in this direction. Two research teams that have emphasized the adaptive capacity of evolutionary procedures and who have shown interesting computer simulations are Allen and McGlade (1986), and Galar, Kwasnicka, and Kwasnicki (see Galar, Kwasnicka, and Kwasnicki, 1980 Galar, 1994). In terms of the optimization tasks looked at throughout this book, one might call their point of view dynamic or on-line optimization, including optimum holding against environmental changes. As Schwefel and Kursawe (1992) have pointed
Random Strategies
103
out, a limited life span of all individuals is an important ingredient in such cases (principle of forgetting). Two others who have tried to explain brain processes on an evolutionary, at least selectionist, basis are Edelman (1987) and Conrad (1988). Though their approach has not yet been embraced by the main stream of neural network research, this might happen in the near future (e.g., Banzhaf and Schmutz, 1992). An even more general paradigm shift in the eld of articial intelligence (AI) has emerged under the label articial life (AL see Langton, 1989, 1994a,b Langton et al., 1992 Varela and Bourgine, 1992). Whereas Lindenmayer (see Prusinkiewicz and Lindenmayer, 1990) demonstrates the possibility of (re-)creating plant forms by means of rather simple computer algorithms, the AL community tries to imitate animal behavior computationally. In most cases the goal is to design \intelligent" robots, sometimes called knowbots or animats (Meyer and Wilson, 1991 Meyer, 1992 Meyer, Roitblat, and Wilson, 1993). The attraction of even simple evolutionary models (re-)producing fairly complex behavior of multi-individual systems simulated on computers is already spreading across the narrow bounds of computer science as such. New ideas are emerging from evolutionary computation, not only towards the organization of software development (Huberman, 1988), but also into the eld of economics (e.g., Witt, 1992 Nissen, 1993, 1994) and even beyond (Schwefel, 1988 Haefner, 1992). It may be questionable whether worthwhile conclusions from the new ndings can reach as far as that, but ecology at least should be a eld that could benet from a consequent use of evolutionary thinking (see Wol, Soeder, and Drepper, 1988). Computers have opened a third way of systems analysis aside from the classical mathematical/analytical and experimental/empirical main roads: i.e., numerical and/or symbolical simulation experiments. There is some hope that we may learn this way quickly enough so that we can maintain life on earth before we more or less unconsciously destroy it. Real evolution always had to deal with unpredictable environmental changes, not only those resulting from exogenous in!uences, but also self-induced endogenous ones. The landscape is some kind of n-dimensional trampoline, and every good imitation of organic evolution, whether it be called adaptive or meliorizing, must be able to work properly under such hard conditions. The multimembered evolution strategy (see Chap. 5, Sect. 5.2) with limited life span of the individuals fullls that requirement to some extent.
104
Random Strategies
Chapter 5 Evolution Strategies for Numerical Optimization The task of mimicking biological structures and processes with the object of solving technical problems is as old as engineering itself. Mimicry itself, as a natural \strategy", is even older than mankind. The legend of Daedalus and Icarus bears early witness to such human endeavor. A sign of its scientic coming of age is the formation of the distinct branch of science known as bionics (e.g., Hertel, 1963 Gerardin, 1968 Beier and Gla%, 1968 Nachtigall, 1971 Heynert, 1972 Zerbst, 1987), which is concerned with the recognition of existing biological solutions to problems that also happen to arise in engineering, and with the adequate emulation of these examples. It is always thereby supposed that evolution has found particularly good, perhaps even optimal solutions. This assumption has often proved to be correct, or at any rate useful. Only a few attempts to imitate the actual methods of natural development are known (Ashby, 1960 Bremermann, 1962{1973 Rechenberg, 1964, 1973 Fogel, Owens, and Walsh, 1965, 1966a,b Holland, 1975 see also Chap. 4) since they are curiously regarded a priori as being especially bad, meaning costly. Rechenberg proposed the hypothesis \that the method of organic evolution represents an optimal strategy for the adaptation of living things to their environment," and he concludes \it should therefore be worthwhile to take over the principles of biological evolution for the optimization of technical systems."
5.1 The Two Membered Evolution Strategy Rechenberg's two membered evolution scheme, suggested in similar form by other authors as a random strategy (see Chap. 4) will be expressed in this chapter as an algorithm for solving non-discrete, non-stochastic, parameter optimization problems. As in Chapter 3, the problem is F (x) ! min where x 2 IRn. In the constrained case x has to be in an allowed region G IRn, where
G = fx 2 IRn j Gj (x) 0 j = 1(1)n Gj restriction functionsg 105
106
Evolution Strategies for Numerical Optimization
In this, as in all direct search methods, it is not possible to deal with constraints in the form of equalities.
5.1.1 The Basic Algorithm
The two membered scheme is the minimal concept for an imitation of organic evolution. The two principles of mutation and selection, which Darwin (1859) recognized to be most important, are taken as rules for variation of the parameters and for ltering during the iteration sequence respectively. In the language of biology, the rules are as follows: Step 0: (Initialization) A given population consists of two individuals, one parent and one descendant. They are each identied by their genotype according to a set of n genes. Only the parental genotype has to be specied as starting point. Step 1: (Mutation) The parent E (g) of the generation g produces a descendant N (g), whose genotype is slightly dierent from that of the parent. The deviations refer to the individual genes and are random and independent of each other. Step 2: (Selection) Because of their dierent genotypes, the two individuals have a dierent capacity for survival (in the same environment). Only one of them can produce further descendants in the next generation, namely the one which represents the higher survival value. It becomes the parent E (g+1) of the generation g + 1. Thus the simplest possible assumptions are made: The population size remains constant An individual has in principle an innitely long life span and capacity for producing descendants (asexually) No dierence exists between genotype (encoding) and phenotype (appearanc