This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
{1,2}" mapping the truth assignments for Monotone2SAT onto the bundles for 2Cons2. ip(B(v)) := x with Xi = 1 •£> Vi = false. ip is one-to-one and B(v) is a satisfying truth assignment for F iff ip{B{v)) is a consistent bundle for A. Details of the proof can be found in [FS97], • Since mCons2 is a generalization of 2Cons2 we conclude that we cannot expect to get polynomial time algorithms for the counting problem considered here. R, given by
3.3
mCons3
Next, we will show that the problem to find the most consistent bundle according to a consistency matrix A is A r P-complete even if we restrict our problems to have
63 binary key factors only. This is done by reducing the following well known problem to 2Cons3: Problem SimpleMaxCut: In: Out:
k € IN, an undirected graph G = (V, E). Is there a partition Vi U Vi of V such that the number of edges between Vi and Vi is at least k ?
SimpleMaxCut is known to be NP-complete ([GJ79, p. 210]). Theorem 3 2Cons3 is
NP-complete.
Sketch of proof: Let (A;, G) be an instance of SimpleMaxCut. Define S := V, n := |V|, m := 2 and A € /C 2 n x 2 n by _ J 2 A{s,bUt,d) •- | !
if {s,t} € £ a n d & ^ d otherwiSe
Define c:= k + \n(n — 1). This transformation can be computed in polynomial time. We refer to [FS97] for the technical details of the proof that G has a cut of size at least k iff there exists a consistent bundle x for A having consistency value 7(1) > c. Observing that 2Cons3 G NP we have the desired result.
•
It follows that mCons3 is iVP-complete for m > 2.
4
Algorithms
In this section we will present algorithms for the three problems mConsl, mCons2 and mCons3. Although we know from Section 3 that polynomial algorithms cannot be expected we will show that, for instances appearing in practice, fast algorithms can be developed. These algorithms are based on standard techniques to solve ./VP-hard problems.
4.1
mConsl
From Theorem 1 we know that mConsl is computationally equivalent to mSAT. To develop an algorithm for mConsl we use the reduction function from Lemma 2 to
64
generate an instance for mSAT from an instance for mConsl. The instances for mSAT are then solved using the public domain SAT-solver SATO [Zha93]. Since a large fraction of the clauses generated by the reduction function are 2-clauses, SATO efficiently solves even large instances. • n-20, m - 2 n*=20, ra-^3 v~r.-i***^"^"*
' n**20, m^4
Jr1
•i**
£i!l
>-Vu»^,A«.'»»-*^* m v i'"
„„*««K ,
v
/ - A ""> »"
200
400 number of inconsistencies
Figure 1: Run times to solve mConsl with SATO To test the performance of our approach we generated three classes of random consistency matrices. Figure 1 shows the average performance of SATO on these matrices. We fixed n = 20 for all three classes and varied m from 2 to 4. For each class random matrices with up to 600 total inconsistencies have been generated. The inconsistencies have been distributed uniformly over the matrices. In addition we included the performance of SATO on two real scenario projects performed recently by the authors of [GFS95]. These measurements are marked A and B in the figure. The consistency matrices of these projects are shown in the appendix of [FS97]. The experiments were run on a Pentium 133MHz processor. Note that SATO solves all the problem instances in a fraction of a second. Thus we have established our Irst goal to solve practically relevant instances of mConsl in real time.
4.2
mCons2
In Section 3 we have shown that mCons2 is at least as hard as TVP-complete problems. Therefore we cannot expect to find polynomial algorithms for the problem. In [FS97] we describe two algorithms to count the number of consistent bundles for a consistency matrix A, Both have exponential run times in the worst case but are fast enough for real time requirements on problem instances of practical relevance. The first algorithm is a backtracking algorithm counting the number of consistent bundles. The algorithm successively constructs all bundles by fixing the key factors one after another. Thus, a tree is generated with the leaves of the tree corresponding
65 to single bundles. The algorithm cuts off the search at a tree node if the last key factor fixed is totally inconsistent to one of the key factors fixed in levels above. The second algorithm is based on the inclusion-exclusion principle and successively generates all subsets of the sets of total inconsistencies. The run time of the algorithm is 0(2' 7 I) if i" is the set of inconsistencies in A. In [FS97] we found that the first algorithm is more efficient than the second one if the number of inconsistencies exceeds \nm2 when running on randomly generated instances. Our solution applies the backtracking algorithm if this threshold is exceeded and the inclusion-exclusion algorithm in the other case. Due to the limited space we refer the reader to [FS97] for details.
4.3
mCons3
In this section we will develop a Branch&Bound algorithm which solves the optimization problem Problem m C o n s 3 a : In: n,m £lN, and a consistency matrix A 6 fCnmxnm. Out: A consistent bundle x with 7(1) > 7(2/) for all consistent ye {l,...,m}s. In order to avoid the optimization process to suppress projections the scenario managers often are interested in a set X of constant size c of optimal bundles for each projection as formalized in Problem c-mCons3b: In: n,m £]N, and a consistency matrix A € ]Cnmxnm. Out: For each projection (s, b) a set X(a,b) of consistent bundles with l^s.ft)! = c such that for all x € <-f(s,f>) a n d all consistent y € { 1 , . . . ,m}s \ /f(s,6), with ys = b we have xs = b and •y(x) > 7(2/). All consistent bundles x with xs = b should be delivered, if there are less than c such bundles. The latter problem is especially important, when many different scenarios are under investigation even if they are considered to be unlikely, e.g. if the effect of a crisis shall be studied. Variations of c-mCons3b with different additional constraints are sometimes used but are not considered here.
66
Both problems are computationally equivalent to mCons3 and therefore are known to be ATP-complete from Theorem 3. In what follows we will first present a BranchfeBound algorithm for mCons3a. Later we will describe some modifications which allow the algorithm to solve c-mCons3b. The general structure of the algorithm is presented in Figure 2.
Branch&Bound(consMatrix A, nt n, int m ) { bundle opt, x, y; opt = initialSolution(A,n,m); lower = 7(opt);
* = (); Open = {(x, MAXINT)}; while (lempty(Open)) { x = best entry from Open; y = x; next = first unfixed key factor of x; for (b=l; b < m; b + + ) { y[next] = b; k = Bound (y); if (k > lower) { if (next < n-1) Open = Open U {(y,K)}; else { opt = y; lower = k; Open = Open \ {(z i P) 1 P < lower }; } } } } return opt; }
Figure 2: A Branch&Bound-algorithm for mCons3a Some questions remain open in this general approach, which will be investigated in more detail in the next paragraphs.
Finding an initial solution To find a good initial solution we use a state-of-the-art Hill Climbing algorithm: For a number of HCRounds rounds the algorithm starts with a random bundle. It then tries to improve on a given bundle by testing all neighbors (bundles differing in exactly one component) and proceeding with the one having the largest consistency value. This is done as long as there is any neighbor having a better consistency value than the current bundle.
67 We tested the Hill Climbing algorithm on 6000 random inputs with the same parameters as a recent scenario project: we generated matrices corresponding to 20 key factors, 9 of them having 2 projections the other 11 having three projections. The consistency matrix contains 26 total inconsistencies uniformly distributed over the set of entries and all the other entries were randomly chosen from the set of consistency values without zero. The quality of the solution returned by the function "initialSolution" is excellent. Even with HCRounds = 20 a best solution is found in more than 80 % of the instances. With HCRounds = 100 a best solution is found in more than 99 % of the instances. The run time of the function is smaller than one second even for HCRounds = 100. In our experiments we obtained the best results with HCRounds = 5. These excellent results may well be due to the fact that the instances used are random instances. However, the algorithm behaves equally well on instances appearing in practice.
Computing a lower bound The second question is how to compute a lower bound f5(x) for a bundle x that is fixed only in the first h key factors. We are interested only in monotone bounds, i.e. if x' is a bundle resulting from x by fixing a key factor then P{x') should not be larger than P(x). This is guaranteed by the following: Definition 3 Let n,m G IN and A be an n x m-consistency matrix. A function /3 : { 1 , . . . , m}~n —>W is called a bound function for A iff for each x G { 1 , . . . , m}h we have f3(x) > y(y) \/y € { 1 , . . . , m}", x is prefix of y. Consider the following function: Definition 4 Let n,m G W, and S = { 0 , . . . , n — 1} be a set of key factors. Let A be an n x m consistency matrix. Let x be a bundle with the key factors s G { 0 , . . . , h} fixed and all the others unfixed. We define h-l
h
s=0 t=s+l
'
. s,t fixed h
Ps,b(x) :=
Y A(t,xt),(s,b) t fixed
'
68 s-1
Kbix)
•=
max
Y.
AA(t4),{s,b)}
J
t unfixed
P(x) := ft(z) +
V
max (/3s,6(x) + J3s,b{x)) 6e{l,...,m}
s=h+1
/? sums up the consistency values of all pairs of projections that have already been fixed. For any unfixed key factor s and all projections (s, 6) the consistency values obtained from the key factors already fixed and the maximum possible consistency values that can be expected from the unfixed key factors are computed under the assumption that xs will become b. For each unfixed key factor s the maximum over all projections (s, 6) is added to obtain the value /3(x). Therefore, f3 is clearly a bound function. In [Sen97] we compared j3 to other functions. Our experiments show that (5 is one of the best functions considered there but that there are several plausible alternatives. In order to compute /3 efficiently during the search it is updated incrementally with the help of several numbers as described in the following definition. Definition 5 Let n G IN, m G IN and let A be annxm consistency matrix. Let x be a bundle with the key factors s 6 { 0 , . . . , h} fixed and all the others unfixed. /i-l
Xi(x,h)
h
:= 5Z H As,*»),(t,z«) s=0 t=s+l h
X2(x,h,s,b)
:= J2 A(t<Xt),(Sfi) s-l
\3{h,s,b)
:=
Y,
™ax M(t,d),(s,b)}
t=h+1de{i,...,m]
for all h G {0,.. .,n — l } , s G {h + 1 , . . . ,n - 1},6 G { 1 , . .
.,m}.
It is now easy to see that f3(x) =
\1(x,h)+
n-l
y^
max
,=/H-l6e{1
(A2(a;, h, s, b) + A3(/i, s, b)) m}
for a bundle x having the first key factors { 0 , . . . , h} already fixed.
69 The values of A3 can be computed offline by using the entries of A in time 0(n2 • m) and can be stored in a table of size 0(n2 • m). If x is a bundle having the first key factors s € {0,... ,h} already fixed and y is a bundle resulting from x by fixing yu+\ then the values of Ai and A2 can be efficiently computed during the search using the equations Xi{y,h+l) X2(y,h + l,s,b)
=
\\{x,h)
+
X2(y,h,h+l,yh+i)
-
X2(x,h,s,b)
Thus, the bound function /3(x) can be computed in 0(n-m) y in algorithm Branch&Bound of Figure 2.
and
time for any new bundle
Some more implementation details are contained in [FS97].
Finding a set of optimal bundles The scenario manager is not only interested in the optimal bundle. He is much more interested to compute for each projection (s, b) a set of solutions. All the bundles in this set must have key factor s fixed to b. The cardinality c of these sets of solutions is the same for all projections. These constraints have been described as problem c-mCons3b. The Branch&Bound algorithm presented above can easily be modified to solve this problem: 1. Initial solution: Computing an initial solution is no longer done. Although the function for computing an initial solution can easily be modified to compute a set of solutions, it turned out that for the instances considered here this computation is too time consuming. This inefficiency is caused by the effort to manage the set of solutions found so far, since the Hill Climbing algorithm may visit solutions twice or even more often. Thus, the Branch&Bound algorithm without computing an initial solution outperforms the one with initial solution on instances of moderate size. 2. Set of optimums: The Branch&Bound algorithm keeps a set of optimal solutions instead of a single solution only. This is easily done since the BranchfeBound algorithm delivers every solution at most once. A small heap is used to keep the set of optimums, with the worst optimum located in the root. The mechanism of heap sort is used to exchange the root entry for a better bundle found during the search. 3. Lower Bound: If the solution heap already contains c solutions, the lower bound can easily be found as the smallest consistency value of a bundle in the
70 heap. As described above this is the consistency value of the root of the heap. If less than c bundles are contained in the heap then the lower bound is set to 0. 4. One set for each projection: In order to deliver a set of optimum bundles for each projection (s, 6) we introduced a set of heaps for the optima found so far, one for each projection. Each complete bundle found during the search is then compared with the root of each of the solution heaps and inserted into the heap, if its consistency value is larger than the consistency value of the root. When generating an incomplete bundle x during the search, its upper bound has to be compared with the solution heaps. For the projections already fixed the bound is compared to the root entries of the corresponding heaps. For the projections still unfixed all possible solution heaps must be checked. The bundle may be pruned if the bundle is no longer relevant for all solution heaps, i.e. its upper bound is smaller than the consistency values of all the roots of the solution heaps. To speed up the comparisons of the upper bound with the solution heap roots of the unfixed key factors, a list of bounds is maintained, one for each key factor. For a key factor s the corresponding entry contains the minimal root bound of the solution heaps corresponding to projections of s. The list entry for s is updated, whenever the consistency value of a root entry changes in a solution heap corresponding to a projection of s. After branching an incomplete bundle, the upper bound of the newly generated bundle is then compared only with the minimum bound for the last key factor fixed. 5. Deleting obsolete entries from the Open list: In the regular Branch&Bound algorithm entries may be deleted from the Open list whenever their upper bound is smaller than the best solution found so far. In the modified algorithm this would be an inefficient operation, since each entry has to be compared with many roots. This makes the deletion operation so inefficient that we decided to omit the operation in the modified algorithm at the cost of an increase in memory requirement. Instead of deleting obsolete entries from the Open list we compare an entry with the roots of the corresponding solution heaps immediately before it is branched. With this we compare all entries that are branched twice but avoid the comparisons with the solution heap roots.
Experimental Results We again generated random matrices to evaluate the performance of our algorithm. In addition we tested the algorithm on instances from real scenario projects. The random matrices were constructed as consistency matrices for 20 key factors, 10 of them having 2 projections and the other 10 having 3 projections. The experiments were run on a Pentium 133MHz processor.
71
"4*
s-^^^^.^
40.0 60.0 number of inconsistencies
Figure 3: Run times to solve c-mCons3b depending on the number of inconsistencies
400 600 number of bundles to compute
Figure 4: Run times to solve c-mCons3b depending on the number of bundles to compute
In Figures 3 and 4 we present the run times of the optimization algorithm depending on the number of total inconsistencies in the matrix and depending on the number of bundles to compute. In the first experiment we fixed the number of bundles to compute to 100, in the latter experiment we fixed the number of inconsistencies in the matrix to 10. Again we included the performance on the three real scenario projects in the figure. As can be seen from the graphs the run times are always smaller than 10 seconds. The experiments show that the problem mCons3 can be solved in real time for input instances that are in the magnitude of the instances generated by real existing scenario projects.
72
5
Conclusions
In our paper we addressed the topic of the consistency analysis in scenario projects. We analyzed the three most important problems posted to us by the scenario managers. For all three problems we showed that in a general form they are at least NP-ha,rd. Nevertheless, we presented algorithms that solve all three problems very quickly. Moreover, for instances occurring in practice, the solution times are usually in the order of some seconds on a state-of-the-art notebook. This means, that • the work of the human experts to determine the key factors, projections and the consistency matrix can be efficiently supported by the computer. • the time consuming consistency check can be done in real time and • mistakes in early stages of a scenario project can be corrected online. Therefore, our algorithms are an important step forward towards a real time scenario management system.
References [AL83] R. Amara and A.J. Lipinski. (1983), Business Planning for an Uncertain Future: Scenarios & Strategies, Pergamon Press, New York, USA. [Bec83] H.S. Becker. (1983), "Scenarios: A Tool of Growing Importance to Policy Analysists in Government and Industry," Technological forecasting & social change, Vol. 23/83, pp 95-120. [DL80] C. Ducot and G.J. Lubben. (1980), "A Typology for Scenarios," Future, Vol. 12, pp 51-57. [FS97] R. Feldmann and N. Sensen. (1997), "Efficient Algorithms for the Consistency Analysis in Scenario Projects," Technical Report, Univ. of Paderborn, Germany. [Fos90] M.J. Foster. (1993), "Scenario Planning for Small Business," Longe Range Planning, Vol. 26, pp 123-129. [Fre84] R.E. Freeman. (1984), Strategic Management: A Stakeholder Approach, Pitman Publishers, Boston, USA. [GJ79] M. R. Garey und D. S. Johnson. (1979), Computers and Intractability, Freeman and company, New York, USA. [GFS95] J. Gausemeier, A. Fink, and O. Schlake. (1995), Szenario Management, Carl Hanser Verlag, Miinchen, Wien. [God87] M. Godet. (1987) Scenarios and strategic management, Butterworths, London, England.
73 [Goe93] U. Gotze. (1993), Szenario Technik in der strategischen Unternehmensplanung, Deutscher Universitats Verlag, Wiesbaden, Germany. [Mey92] M. Meyer-Schonherr. (1992), Szenario Technik als Instrument der strategischen Planung, Verlag Wissenschaft & Praxis, Ludwigsburg, Germany. [Schae78] T.J. Schaefer. (1978), "The complexity of satisfiability problems,", Proc. 10th Ann. ACM Symp. on Theory of Computing, pp 216-226. Association for Computing Machinery. [Sen97] N. Sensen. (1997), Konsistenzanalyse beim Szenario Management: Ein Existenz-, Zdhl- und Optimierungsproblem, Master thesis (in German), Univ. of Paderborn, Germany. [Val79a] L. G. Valiant. (1979), "The complexity of computing the permanent," Theoretical Computer Science, Vol 8, pp 189-201. [Val79b] L. G. Valiant. (1979), "The complexity of enumeration and reliability problems," SIAM Journal of Computing, Vol 8, pp 410-421. [Zha93] H. Zhang. (1993), "Sato: a decision procedure for propositional logic," Association for Automated Reasoning Newsletter, Vol. 22, 1993.
This page is intentionally left blank
Combinatorial and Global Optimization, pp. 75—95 P.M. Pardalos, A. Migdalas and R. Burkard, Editors © 2002 World Scientific Publishing Co.
Assignment of Reusable and Non-Reusable Frequencies Dimitris A. Fotakis 1 ' 2 (fotakisficti.gr) (1) Department of Computer Engineering and Informatics University of Patras, 265 00 Rion, Greece Paul G. Spirakis 1 ' 2 ( s p i r a k i s S c t i . g r ) (2) Computer Technology Institute Kolokotroni 3, 262 21 Patras, Greece
Abstract Graph radio coloring and graph radio labelling are combinatorial models for two interesting cases of Frequency Assignment. In both problems positive integer labels (channels) must be assigned to all the vertices of a graph such that adjacent vertices get labels at distance at least two. In radio labelling all the labels must be distinct, while in radio coloring only the vertices being at distance no more than two in the input graph must be assigned distinct labels. For both problems the objective is to minimize the maximum label used. We first prove that both radio coloring and radio labelling remain A/"'P-complete for graphs of diameter two, and we show that a 1aapproximation algorithm for radio coloring can be obtained from any aapproximation algorithm for coloring squares of graphs. We also show that radio labelling is equivalent to Hamiltonian Path with distances one and two (HP(1,2)), and we present a polynomial-time algorithm for computing an optimal radio labelling, given a coloring of the input graph with constant number of colors. Thus we prove that radio labelling is in V for planar graphs. Additionally, we present competitive algorithms and a lower bound for on-line radio labelling. Keywords: Frequency Assignment, Hamiltonian Cycle, PolynomialTime Algorithms, On-line Algorithms.
76
1
Introduction
The Frequency Assignment Problem (FAP) arises from the fact that radio transmitters operating at the same or closely related frequency channels have the potential to interfere with each other. FAP can be formulated as an optimization problem as follows: Given a collection of transmitters to be assigned operating channels and a set of interference constraints on transmitter pairs, find an assignment that fulfills all the interference constraints and minimizes the allocated bandwidth. A common model for FAP is the interference graph. Each vertex of an interference graph represents a transmitter, while each edge represents an interference constraint between the adjacent transmitters. The frequency channels are usually assumed to be uniformly spaced in the spectrum and are labelled using positive integers. Frequency channels with adjacent integer labels are assumed adjacent in the spectrum [6, 7, 10]. Clearly, FAP is a generalization of graph coloring [7]. Instead of specifying the interference constraints for each pair of transmitters, FAP can be defined by specifying a minimum allowed spatial distance for each channel/spectral separation that is a potential source of interference [6, 7]. In [7] these are called Frequency-Distance (FD) constraints. FD-constraints can be given as a set of distances {D0,Di,...,DK}, Do > D\ > • • • > D^, where the distance Dx, x G { 0 , 1 , . . •, K}, is the minimum distance between transmitters using channels at distance x. The FD(/t)-coloring problem has been proposed as a model for FAP in unweighted interference graphs [10, 17]. In FD(/t)-coloring we seek a function XK : V (->• { 1 , . . ., u} that fulfills the FD-constraints with respect to the graph distances Do = K + 1, Dx = K,...,DK = 1, and minimizes the largest color v used (color span). Alternatively, v,u G V are only allowed to get colors at distance x, |XK(u) — XK(w)| = x, x € { 0 , 1 , . . . , K}, if v and u are at distance at least K — x + 1 from each other in the interference graph. A polynomial-time exact algorithm for a variant of FD(/t)-coloring in lattices is presented in [10]. FD(2)-coloring, which is also called radio coloring, is a combinatorial model for the widely used "co-channel" and "adjacent-channel" interference constraints. A problem similar to FD(2)-coloring is studied [13] in the context of mobile networks, where each vertex may require more than one colors (multicoloring). A polynomial-time approximation algorithm for triangular lattices is presented in [13]. In some practical applications the transmitters cover a local or metropolitan area. Hence, the transmitters are not allowed to operate at the same channel (D0 = oo, nonreusable frequency channels). The problem of radio labelling is the equivalent of radio coloring in the context of non-reusable frequency assignment. In particular, a valid radio labelling fulfills the FD-constraints with respect to DQ = oo, D\ = 2, D 2 = 1. In
77 radio labelling we seek an assignment of distinct integer labels to all the vertices of a graph, such that adjacent vertices get labels at distance at least two. The objective is to minimize the maximum label used (label span). The definition of radio labelling is communicated to us by [9]. Radio labelling and similar combinatorial models for non-reusable frequency assignment are used for obtaining lower bounds on the optimal values of general FAP instances [2, 17]. Since general instances of FAP are intractable, there exist many attempts to develop heuristic approximation algorithms [1]. Hence, some lower bounding techniques are necessary for assessing the quality of the solutions found by these algorithms.
1.1
Summary of Results
We start with proving that both radio coloring and radio labelling remain MVcomplete for graphs of diameter two. We also show that a polynomial-time 1aapproximation algorithm for radio coloring can be obtained from any polynomial-time a-approximation algorithm for coloring squares of graphs. We proceed to study radio labelling that is shown equivalent to Hamiltonian Path with distances one and two (HP(1,2)). Hence, radio labelling is MAX-SNP-hard and approximable in polynomial time within 7/6 [16]. Next, we present a polynomial-time algorithm for computing an optimal radio labelling, given a coloring of the graph with constant number of colors. Therefore, we prove that radio labelling is in V for planar graphs and graphs colorable with constant number of colors in polynomial time. As a side effect, we show that, given a partition of the vertices of a graph into constant number of cliques, we can decide if the graph is Hamiltonian in polynomial time. We are not aware of another algorithm that exploits a partition into cliques for deciding Hamiltonicity. Motivated by the practical applications of on-line frequency assignment in mobile networks [15], we define two on-line variations of radio labelling, and we prove that the greedy algorithm achieves a competitive ratio of two for both variations. We also obtain a lower bound of 3/2 on the competitive ratio of any on-line algorithm for the first variation.
2
Definitions and Techniques
Given a graph G(V, E), d(v, u) denotes the length of the shortest path between v, u £ V, and diam(G) denotes the diameter of G defined as diam(G) = max„ >u6 y{d(u, u)}.
78 The square of G, denoted G 2 , is a graph on the vertex set V that contains an edge between v, u £ V, if v and u are at distance at most two in G. The complementary graph G is a graph on the vertex set V that contains an edge {v, u}, iff {v, u} £ E. Hamiltonian Path with distances one and two (HP(1,2)) is the problem of finding a Hamiltonian path of minimum length in a complete graph, where all the edge lengths are either one or two. A Hamiltonian path is a simple path visiting each vertex of a graph exactly once. An instance of HP(1,2) can also be defined by an unweighted graph G(V, E), if the edges of E are considered of length one, and the edges not in E (non-edges) of length two. Therefore, HP(1,2) is a generalization of the Hamiltonian path problem. HP(1,2) is MAX-SNP-hard and approximable in polynomial time within a factor of 7/6 [16]. In the graph coloring problem, we seek an assignment of colors to the vertices of a graph such that no pair of adjacent vertices get the same color. The objective is to minimize the number of colors used. Given a graph G, the value of an optimal coloring is called the chromatic number of G and is denoted by x{G). It is AfV-h&id even to efficiently approximate the chromatic number of general graphs [14]. However, there exist polynomial-time approximation algorithms whose performance guarantees are not so good [8, 12]. The following generalized version of graph coloring is derived by applying the FDconstraints to unweighted graphs with respect to the distances DQ = K + 1,-Di = K,...,DK
= l.
Definition 2.1 (FD(re)-coloring) INSTANCE: A graph
G(V,E).
SOLUTION: A valid FD(«)-coloring, i.e. a function XK : V — i >• { 1 , . . . , v} such that, for all v, u e V, |XK(i>) — XK(w)| = x, x € { 0 , 1 , . . . , K], only if d(v, u) > K — x + 1. O B J E C T I V E : Minimize the maximum color v used (color span). Clearly, the FD(«)-coloring problem allows a pair of non-adjacent vertices to be assigned the same color/channel, provided they are located far apart. In the sequel, we shall concentrate on the study of FD(2)-coloring, also called radio coloring, modelling the widely used "co-channel" and "adjacent-channel" interference constraints. We remark that the problem of radio coloring a graph G is not equivalent to the problem of coloring G 2 . In particular, the objective in coloring G 2 is to minimize the number of different colors used, while the objective in radio coloring G is to minimize the maximum color assigned to a vertex. For example, if G is Km^m (i.e. the complete bipartite graph with m vertices on each class), then G 2 is the complete graph on 2m vertices. Therefore, x{G2) = 1m, while X2(G) = 2m + 1.
79 A valid FD(/c)-labelling fulfills the FD-constraints with respect to the graph distances Do = oo, A = K, . . . , DK — 1. Definition 2.2 ( F D (/^-labelling) INSTANCE: A graph
G(V,E).
SOLUTION: A valid FD(n)-labelling, i.e. a function LK : V H > {1,...,U} such that, for all v,u € V, |LK(w) — L K (M)| = x, x € { 1 , . . . , «}, only if d(v,u) > K — x + 1. O B J E C T I V E : Minimize the maximum label v used (label span). In the sequel, we shall concentrate on the FD(2)-labelling problem, also called radio labelling. Given a graph G(V,E), RL(G) denotes the value of an optimal radio labelling for G and, for any v € V, RL(v) denotes the label assigned to v by a valid radio labelling. Obviously, given a coloring of G with x colors, it is easy to find a radio labelling of value at most \V\ + \ ~ 1- Therefore, |V|
X
(G)-1
It is not hard to verify that RL(G) = |V| for all graphs G such that G contains a Hamiltonian path. On the other hand, RL(G) = |V| + x(G) — 1 for any complete r-partite graph G, r > 2.
2.1
On-line Radio Labelling
We proceed to define the problem of on-line radio labelling (cf. Chapter 13 of [11] for the basic definitions concerning on-line computation). In the on-line setting of the problem, an induced subgraph of the interference graph appears to the on-line algorithm in a vertex-by-vertex fashion. In case of on-line radio labelling, a request is a newly arrived vertex that has to be assigned a radio label. A request is accepted if the vertex actually gets a valid label. Otherwise, the request is rejected. We define two variations of on-line radio labelling. Definition 2.3 (On-line Radio Labelling — Benefit Version) INSTANCE: A graph G(V, E) and an integer bound p > 0. A new vertex v € V is presented to the on-line algorithm at every step. The adversary may choose to request any subset V C V in any order. SOLUTION: The algorithm has to decide to either accept v by assigning a label not greater than /3 to it, or reject v. At any step, the labels of the set Va of accepted vertices must form a valid radio labelling for the subgraph of G induced by Va.
80 O B J E C T I V E : Maximize the number of accepted vertices. Definition 2.4 (On-line Radio Labeling — Assignment Version) INSTANCE: A graph G(V,E). At every step, a new vertex v e V is presented to the on-line algorithm. The adversary may choose to request any subset V C V in any order. SOLUTION: The algorithm must assign labels to all the vertices requested. At any step, the labels of the set Vp of the vertices presented so far must form a valid radio labelling for the subgraph of G induced by Vv. O B J E C T I V E : Minimize the maximum label used.
2.2
Competitive Analysis and Lower Bounds
The usual approach of analyzing the performance of on-line algorithms is the competitive analysis [18], In competitive analysis, the performance of the on-line algorithm is compared to the performance of the optimal off-line algorithm on every sequence of requests, and the worst-case ratio is considered. Let A be an on-line algorithm for a benefit (maximization) problem and a any sequence of requests. Also, let A(a) denote the benefit accrued by A when presented with a, and OPT(cr) denote the benefit accrued by the optimal off-line algorithm OPT on the same sequence. We say that the on-line algorithm A is c-competitive if there exists a constant b such that on every request sequence a, c • A{a)
+b>
OPT(CT)
The competitive ratio for cost (minimization) problems is defined similarly. In case of randomized on-line algorithms, the competitive ratio is defined with respect to the expected benefit of the algorithm. A standard technique used in competitive analysis is to employ an adversary which plays against the algorithm A and constructs an input which incurs a high cost for A and a low cost for OPT. We say that an adversary is oblivious if it constructs the request sequence in advance, before A starts working on the sequence. However, the oblivious adversary knows the probability distribution of the actions taken by the algorithm A. A common technique for proving lower bounds on the competitive ratio of a randomized on-line algorithm against the oblivious adversary is to bound the performance of the "best" deterministic on-line algorithm on inputs generated from the "worst" probability distribution. In particular, let V be a probability distribution over sequences of requests a, let Ep[A(cr)] denote the expected value of the benefit of an algorithm A,
81 and E E I[OPT((T)] be the expected benefit of the optimal off-line algorithm on inputs generated from V. An algorithm A (for a benefit problem) is c-competitive against V if there exists a constant b such that, c • Ev[A(a)] +b>
Ev[OPT{a)}
Lemma 2.5 ([4]) A real number c is a lower bound on the competitive ratio of randomized on-line algorithms against the oblivious adversary if and only if there exists a probability distribution V such that c is a lower bound on the competitive ratio of any deterministic on-line algorithm against T>.
3
The Complexity of Radio Coloring and Radio Labelling
We proceed to show that both radio coloring and radio labelling remain AA'P-complete, even for a very restricted class of instances, namely graphs of diameter two. Lemma 3.1 Radio coloring and radio labelling restricted to graphs of diameter two are HV -complete. Proof. Let G(V, E), \V\ = n, be any graph of diameter two. Since, for all v,u € V, d(v, u) < 2, any valid radio coloring must assign distinct colors to all the vertices of G. Moreover, if {v, u) e E, then \X2(v) - X 2 ( M ) | > 2. Therefore, if diam(G) = 2, the problem of radio coloring in G is equivalent to the problem of radio labelling in G. Hence, it suffices to show that radio labelling is jV"P-complete for graphs of diameter two. Clearly, radio labelling is in MV. Additionally, it is not hard to verify (see also Lemma 3.3) that RL(G) < |V| if and only if the complementary graph G contains a Hamiltonian path. Thus, in order to show that radio labelling is jVP-complete for graphs G of diameter two, it suffices to show that the Hamiltonian Path problem remains A/"'P-complete for complementary graphs G such that diam(G) = 2. Let G'(V',E') be any graph and let s,t e V' be any pair of non-adjacent vertices. The problem of deciding if G' contains a Hamiltonian path starting from s and ending to t is A/"'P-complete (Hamiltonian Path between Two Vertices [5]). Let G^(V' U {vs,vt},E' U {(s, vs), (t, vt)}) be the graph obtained from G' by adding two nonadjacent vertices vs,vt, and connecting vs to s and vt to t (Figure 1). The graph G^ is the complement of a graph of diameter two. In particular, the following observations justify that diam (G(cn = 2, since all the vertices are at distance at most two from each other in G(c\
82
Figure 1: The complementary graph of G' c ' has diameter two. 1. The vertex pairs (s,i), (v„vt), (s,vt) and (t,vs) are connected by edges in
G^.
2. Any pair of vertices u, w £ V — {s, t] are at distance at most two from each other, because they are connected to both vs,vt. 3. Any vertex u € V U {vs} — {s, £} is at distance at most two from s, because both u and s are connected to vt. 4. Any vertex u € V U {v(} — {s,£} is at distance at most two from t, because both u and t are connected to vs. Additionally, G^ contains a Hamiltonian path if and only if G' contains a Hamiltonian path from s to t. Therefore, Hamiltonian path is A/'T'-complete for complements of graphs of diameter two. • Notice that a set C C V is a valid color class of G2, iff it is a valid radio color class of G, because any pair of vertices in C is at distance at least three in G. Moreover, if we assign the colors 1 , 3 , . . . , 2\{G2) — 1 to the color classes of G 2 , we obtain a valid radio coloring of G. Therefore, X(G2) < X 2 (G) < 2 X (G 2 ) - 1 Additionally, if A is a polynomial-time a-approximation algorithm for coloring G 2 and | J 4 ( G 2 ) | denotes the number of colors used by A, then we can easily compute a valid radio coloring of G of value no more than 2| J 4(G 2 )| — 1. Since X 2 (G)
< 2|A(G 2 )| - 1 < 2 Q X ( G 2 ) - 1 < 2 Q X 2 ( G ) - 1 ,
this is a 2a-approximation for radio coloring in G.
83 Lemma 3.2 For any graph G and real number a > 1, a polynomial-time laapproximation algorithm for radio coloring in G can be obtained from any polynomialtime a-approximation algorithm for coloring G2. Since radio labelling assigns distinct integer labels to all the vertices of a graph, it is a vertex arrangement problem. In particular, we show that radio labelling is equivalent to HP(1,2) in the complementary graph. Lemma 3.3 Graph radio labelling and HP(1,2) are equivalent. Proof. Given an instance of radio labelling, i.e. a graph G(V, E), the corresponding instance of HP(1,2) is a complete graph G on the vertex set V, and the distance function d is defined for all v, u £ V, v / u, by
d(v u) =\2l1 ' > a[V U
*M*E
if ( « , « ) € £
Given any valid radio labelling L (for G) of value RL(L), we can obtain a Hamiltonian path H for G by traversing all the vertices in increasing order of their labels. Moreover, the following claim implies that the length of the Hamiltonian path H is exactly RL(L) - 1. Claim 1 The length of the path up to any vertex of label i, i — 1 , . . . , RL(L), is exactly i — 1. Proof of the Claim. We prove the claim by induction on i. Clearly, it is true for the first vertex, where i = 1. Assume inductively that it is true for any vertex v of label i > 1, and let u be the next vertex in the Hamiltonian path. We proceed by case analysis: 1. If the label of u is i + 1, the edge {v, u] is not present in G and, by construction, d(v, u) = 1. Thus, the length of the path up to vertex u is exactly i. 2. If the label of u is i + 2, by the construction of the Hamiltonian path, there does not exist a vertex of label i + 1. Therefore, the edge {v,u} 6 E, and d(v,u) = 2. Consequently, the path up to u has length exactly i + 1. D Conversely, given an instance (G,d) of HP(1,2) on the vertex set V, \V\ = n, an instance GiV, E) of radio labelling can be obtained by only connecting the vertex pairs that are at distance 2. Furthermore, given a Hamiltonian path H = (vx, v2, • • •, v„) for G of length 1(H), we obtain a valid radio labelling L (for G) as follows:
84 1. RL(vi) = 1. 2. For i = l , . . . , n - l , (a) RL(ui+i) = RL{Vi) + 1, if d(vi, vi+\) = 1. By the construction of G(V, E), in this case {vt, vi+1} £ E. (b) RL(« i+1 ) = RL(^) + 2, if d(vi, vi+1) = 2. By the construction of G(V, E), in this case {«,, i»i+i} 6 -EBy construction, all the vertices get distinct labels, while, if an edge {i>,,i>;+1} is present in E, the vertices vi and vi+i are assigned non-adjacent labels. Therefore, the resulting radio labelling is a valid one. Additionally, the last vertex of the path vn is assigned the label 1(H) + 1, that is the largest label used. Hence, RL(L) = 1(H) + 1.
• The previous lemma implies that radio labelling is M AX-SNP-hard and approximable in polynomial time within 7/6 [16].
4
An Exact Algorithm for Constant Number of Colors
Since radio labelling is equivalent to HP(1,2) and a coloring corresponds to a partition into cliques of the complementary graph, we present the technical part of the proof in the context of Hamiltonian paths/cycles and partitions into cliques. In particular, we prove that, given a partition of a graph G(V, E) into constant number of cliques, we can decide if G is Hamiltonian in polynomial time.
4.1
Hamiltonian Cycles and Partitions into Cliques
Theorem 4.1 Given a graph G(V,E), \V\ = n, and a partition of V into K > 1 cliques, there exists a deterministic algorithm that runs in time 0(nK^2li~1A and decides ifG is Hamiltonian. IfG is Hamiltonian, the algorithm outputs a Hamiltonian cycle. Proof. A set of inter-clique edges M C E is an HC-set if M can be extended to a Hamiltonian cycle using only clique-edges, i.e. there exists an M' c ' C E of clique edges such that M U M^"1 is a Hamiltonian cycle. We first show that, given a set of inter-clique edges M, we can decide if M is an HCset and construct a Hamiltonian cycle from M in poly(n, K) time (Proposition 4.2).
85 Then, we prove that G is Hamiltonian iff there exists an HC-set of cardinality at most K(K — 1) (Lemmas 4.3 and 4.4). The algorithm exhaustively searches all the sets of inter-clique edges of cardinality at most K(K — 1) for an HC-set. Additionally, we conjecture that, if G is Hamiltonian, then there exists an HC-set of cardinality at most 2(K — 1). We prove this conjecture for a special case (Lemma 4.5) and we use Lemma 4.3 to show the equivalence to Conjecture 1. Let C — {C\,... ,CK} be a partition of V into K > 1 cliques. Given a set M C E of inter-clique edges, the clique graph T(C, M) contains exactly K vertices, that correspond to the cliques of C, and represents how the edges of M connect the different cliques. If M is an HC-set, then the corresponding clique graph T(C, M) is connected and eulerian. However, the converse is not always true. Given a set of inter-clique edges M, we color an edge RED, if it shares a vertex of G with another edge of M. Otherwise, we color it BLUE. The corresponding edges of G are colored with the same colors, while the remaining edges (E — M) are colored BLACK. Additionally, we color RED each vertex v 6 V, which is the common end vertex of two or more RED edges. We color BLUE each vertex » e l ^ t o which exactly one edge of M (RED or BLUE) is incident. The remaining vertices of G are colored BLACK (Figure 2).
Let H be any Hamiltonian cycle of G and let M be the corresponding set of interclique edges. Obviously, RED vertices cannot be exploited for visiting any BLACK vertices belonging to the same clique. If H visits a clique C; through a vertex v, and leaves C, through a vertex u, then v,u € Ct consist of a BLUE vertex pair. A BLUE pass through a clique Ct is a simple path of length at least one, that entirely consists of non-RED vertices of Cj. A clique d is covered by M, if all the vertices of C; have degree at most two in M, and the existence of a non-RED vertex implies the existence of at least one BLUE vertex pair. The following proposition characterizes HC-sets. P r o p o s i t i o n 4.2 A set of inter-clique edges M is an HC-set iff the corresponding clique graph T(C, M) is connected, eulerian, and, (a) For all i = 1 , . . . , K, Ci is covered by M; and (b) There exists an eulerian trail R for T such that: For any RED vertex v G V, R passes through v exactly once using the corresponding RED edge pair. Proof. Any HC-set corresponds to a connected, eulerian clique graph T(C, M) that fulfills both (a) and (b). Conversely, we can extend M into a Hamiltonian cycle H following the eulerian trail R. All the RED vertices can be included in H with degree two because of (b). Moreover, since R is an eulerian trail and (a) holds for M, all the BLUE and the BLACK vertices can be included in H with degree two. Therefore, H is a Hamiltonian cycle. •
86
(a) A graph partitioned into 3 cliques
0 9 ^
(b) An HC-set with 5
(c) A RED Hamiltonian Cycle
A black vertex A red vertex A blue vertex
-
— A blue edge A red edge A black edge A black edge of the cycle - - - • An edge to be removed
(d) A cycle using less RED edges
(e) An HC-set with 3 edges
Figure 2: An application of Lemmas 4.3 and 4.4. The proof of Proposition 4.2 implies a deterministic procedure for deciding if a set of inter-clique edges is an HC-set in poly(n, K) time. Moreover, in case that M is an HC-set, this procedure outputs a Hamiltonian cycle.
Lemma 4.3 Let BK > 2 be some integer only depending on K such that, for any graph G(V,E) and any partition of V into K cliques, if G is Hamiltonian and \V\ > BK, then G contains at least one Hamiltonian cycle not entirely consisting of inter-clique edges (RED vertices). Then, for any graph G(V, E) and any partition of V into K cliques, G is Hamiltonian iff it contains a Hamiltonian cycle with at most BK inter-clique edges.
Proof. Let H be the Hamiltonian cycle of G containing the minimum number of interclique edges and let M be the corresponding set of inter-clique edges. Assume that \M\ > BK. The hypothesis implies that H cannot entirely consist of RED vertices. Therefore, H should contain at least one BLUE vertex pair.
87 We substitute any BLUE pass of H through a clique C* with a single RED super-vertex v, that also belongs to the clique C*. Hence, v is connected to all the remaining vertices of C, using BLACK edges. These substitutions result in a cycle H' that entirely consists of RED vertices, and contains exactly the same set M of inter-clique edges with H. Obviously, the substitutions of all the BLUE passes of H with RED super-vertices result in a graph G'(V, E') that is also Hamiltonian, |V'| > BK, and V is partitioned into K cliques. Moreover, for any Hamiltonian cycle He of G', the reverse substitutions of all the RED super-vertices v with the corresponding BLUE passes result in a Hamiltonian cycle of G that contains exactly the same set of inter-clique edges with He (Figure 2). Since H' is a Hamiltonian cycle that entirely consists of inter-clique edges and \V'\ > BK, the hypothesis implies that there exists another Hamiltonian cycle of G' that contains strictly less inter-clique edges than H'. Therefore, there exists a Hamiltonian cycle of G that contains less inter-clique edges than H. • Lemma 4.3 implies that, in order to prove the upper bound on the cardinality of a minimum HC-set, it suffices to prove the same upper bound on the number of vertices of Hamiltonian graphs that (i) can be partitioned into K cliques, and (ii) only contain Hamiltonian cycles entirely consisting of inter-clique edges. It should be intuitively clear that such graphs cannot contain an arbitrarily large number of vertices. L e m m a 4.4 Given a graph G(V, E) and a partition of V into K cliques, G is Hamiltonian iff there exists an HC-set M such that \M\ < K(K — 1). Proof. By definition, the existence of an HC-set implies that G is Hamiltonian. Conversely, let H be the Hamiltonian cycle of G that contains the minimum number of inter-clique edges, and let M be the corresponding HC-set. If \M\ < K(K - 1), then we are done. Otherwise, Lemma 4.3 implies that it suffices to prove the same upper bound on |V\ for graphs G(V, E) that only contain Hamiltonian cycles entirely consisting of inter-clique edges. Assume that \V\ = \M\ and the coloring of V under M entirely consists of RED vertices, and consider an arbitrary orientation of the Hamiltonian cycle H (e.g. a traversal of the edges of H in the clockwise direction). If there exist a pair of cliques C; and Cj and four vertices V\, v-i e Q and u\, %2 € Cj, such that both vx are followed by ux (x = 1, 2) in a traversal of H, then the BLACK edges {vi, v2} and {ui, M2} can be used instead of {vi, u^} and {^2, W2} in order to obtain a Hamiltonian cycle containing less inter-clique edges than H. The previous situation can be avoided only if, for all i = 1 , . . . , K, and j = 1 , . . . , K, j / i, at most one vertex u, € Cj is followed by a vertex u € Cj in any traversal of H. Hence, if |V| > K(K— 1), then G contains at least one Hamiltonian cycle not entirely consisting of inter-clique edges. Alternatively, any HC-set M of minimum cardinality contains at most two edges between any pair of cliques Cj and Cj. Thus, M contains at most K(K — 1) inter-clique edges. D
88
Figure 3: An HC-set containing exactly 2(K — 1) edges. Therefore, we can decide if G is Hamiltonian in time 0[n^2K~r>\ because the number of the different edge sets containing at most K(K — 1) inter-clique edges is at most n 2 "'" - 1 ), and we can decide if a set of inter-clique edges is an HC-set in time poly(n, K) = 0(nK). • A BLUE Hamiltonian cycle is a Hamiltonian cycle that does not contain any RED vertices or edges. We can substantially improve the bound of K(K — 1) for graphs containing a BLUE Hamiltonian cycle. Lemma 4.5 Given a graph G(V, E) and a partition ofV into K cliques, ifG contains a BLUE Hamiltonian cycle, then there exists an HC-set M entirely consisting of BLUE edges, such that \M\ < 2(K — 1). Proof. Let H be the BLUE Hamiltonian cycle of G that contains the minimum number of inter-clique edges and let M be the corresponding HC-set. Assume that \M\ > 2(K — 1), otherwise we are done. Notice that, since RED vertices cannot be created by removing edges, any M' C M that corresponds to an eulerian, connected, clique graph T(C, M') is an HC-set that only contains BLUE edges. Let ST{C,MS) be any spanning tree of T(C,M). Since \Ms\ = K — 1, the graph T< S '(C, M-Ms), which is obtained by removing the edges of the spanning tree from T, contains at least K edges. Therefore, T ' s ' contains a simple cycle L. The removal of the edges of L does not affect connectivity (the edges of L do not touch the spanning tree ST), and subtracts two from the degrees of the involved vertices/cliques. Clearly, the clique graph T'(C, M — L) is connected and eulerian, and M — L, \M — L\ < \M\, is an HC-set. • Figure 3 shows an HC-set of cardinality exactly 2(K— 1) that corresponds to a Hamiltonian cycle using the minimum number of inter-clique edges. Therefore, the bound of 2(K — 1) is tight. However, we are not able to construct HC-sets that contain more than 2(K — 1) edges and correspond to Hamiltonian cycles using the minimum number of inter-clique edges. Hence, we conjecture that the bound of 2(K — 1) holds for any graph and any partition into K cliques. An inductive (on |V|) application of Lemma 4.3 suggests that this conjecture is equivalent to the following: Conjecture 1 For any Hamiltonian graph G(V,E) of 2K — 1 vertices and any partition of V into K cliques, there exists at least one Hamiltonian cycle not entirely
89 consisting of inter-clique edges. The previous conjecture is very similar to a theorem proved by C.A.B. Smith in 1946. This theorem states that the number of Hamiltonian cycles that contain any given edge of a cubic graph (i.e. a simple regular graph of degree 3) is even. A simple algebraic proof of this theorem can be found in [3]. Smith's Theorem implies Conjecture 1 in case that the partition C contains at least k — 1 cliques of even cardinality and at most 1 clique of odd cardinality. If we only consider Hamiltonian graphs, Smith's Theorem can be applied to graphs G(V, E) consisting of a Hamiltonian cycle and a perfect matching. Therefore, Smith's Theorem is applicable to graphs that contain a RED Hamiltonian cycle, such that V is partitioned into n=^ cliques of two vertices (perfect matching). A more general class of graphs fulfills the hypothesis of Conjecture 1. In particular, Conjecture 1 is applicable,to the graphs consisting of a RED Hamiltonian cycle and K cliques of arbitrary cardinalities. However, the conclusion is weaker than the conclusion of Smith's Theorem, in the sense that it only claims the existence of a Hamiltonian cycle avoiding at least one inter-clique (RED) edge.
4.2
A Reduction from Radio Labelling to Hamiltonian Cycle
L e m m a 4.6 Given a graph G(V,E), \V\ = n, and a coloring of G with K colors, an optimal radio labelling can be computed in O l n l l ( & + 1 ' ) time.
Proof. Let G(V,E) be the complement of the input graph G. Obviously, RL(G) < n + K — 1. Therefore, at most K — 1 labels remain unused by an optimal radio labelling. Hence, any optimal solution to the corresponding HP(1,2) instance (see also the proof of Lemma 3.3) contains at most K — 1 non-edges (of G), and there exists a Hamiltonian cycle containing at most K non-edges (of G). Then, we show how to compute a Hamiltonian cycle containing the minimum number of non-edges to the complementary graph G. Let A be the algorithm of Theorem 4.1. We call A at most M = 0(n2li) times, with input the graphs G;(V, E U Ni), i = 1 , . . . , M. The sets N, are all possible subsets of non-edges of G with at most K elements, including the empty one. Let Gi be a Hamiltonian graph that corresponds to a set N, of minimum cardinality. Obviously, the Hamiltonian cycle produced by A(Gi) is a Hamiltonian cycle with the minimum number of non-edges for G. D Since any planar graph can be colored with constant number of colors in polynomial time, the following theorem is an immediate consequence of Lemma 4.6.
90 T h e o r e m 4.7 An optimal radio labelling of a planar graph can be computed in polynomial time. Remark. Conjecture 1 implies that, given a graph G(V,E) and a coloring with K colors, an optimal radio labelling can be computed in time noi-K\
5 5.1
Algorithms for On-line Radio Labelling On-line Radio Labelling - Benefit Version
We first analyze the performance of the greedy algorithm B G R E E D Y . B G R E E D Y assigns to a newly arrived request the least integer j , such that j has not been assigned to any previously accepted request, and the resultant radio labelling is a valid one. B G R E E D Y accepts the request, if and only if j is no more than the bound
L e m m a 5.1 The competitive ratio of B G R E E D Y is /S § Proof. The B G R E E D Y algorithm always accepts at least [f 1 requests, because, for i = 1, 2 , . . . , 111, B G R E E D Y can always assign a label no more than 2i — 1 to the i-th request. Since the optimal off-line algorithm cannot accept more than (5 requests, an upper bound of /?[fl on the competitive ratio of B G R E E D Y can be established. We prove that this ratio is precisely /3 | by exhibiting a special sequence of requests. Let Sm be any graph on m vertices containing a Hamiltonian path and Hm = Sm be the complementary graph. Lemma 3.3 implies that RL(Hm) = m. Additionally, let Lm be the graph obtained from the complete graph on y vertices, denoted Km/2, and Hm/2 by connecting any vertex of Km/2 to any vertex of Hm/2. By construction, RL(L m ) = ^ . For some bound /? > 0, let the request sequence consist of the vertices of the graph L2p, such that all the /3 vertices of Kp are requested before all the vertices of Hp. Clearly, B G R E E D Y can only accept the first f requests, while the optimal algorithm accepts the last j3 requests. This establishes the competitive ratio of B GREEDY. • Notice that B G R E E D Y does not take into account the structure of the optimal solution on the current set of requests. The previous instance shows that unless a deterministic on-line algorithm takes into account the optimal solution on the current set of requests, it cannot achieve competitive ratio better than /3 [ | 1 . Therefore
91 B G R E E D Y is optimal among on-line algorithms that do not reject by choice. We next prove that no randomized on-line algorithm can achieve a competitive ratio less than | against any adversary. Lemma 5.2 No randomized algorithm for the benefit version of on-line radio labelling can achieve competitive ratio less than | . Proof. We prove the lower bound against the oblivious adversary; since the oblivious adversary is the least powerful one, this implies a lower bound of § against any adversary. The proof consists of defining an appropriate probability distribution on the request sequences and applying Lemma 2.5. Let { 1 , . . . ,/3} be the set of available labels. We consider the following probability distribution on the request sequences: 1. With probability p, the request sequence consists of the vertices of the graph K0. 2. With probability 1 — p, the request sequence consists of the vertices of the graph L2g, such that the vertices of Kg precede the vertices of Hg. If the input is Kg, the optimal off-line algorithm accepts exactly f 1 requests, while if the input is L2/3, it accepts exactly /3 requests. Thus, the expected number of accepted requests for the optimal algorithm is
E(OPT) = p
+(1-P)0
Let A be any deterministic algorithm. On input Kg, the algorithm A will accept x requests using labels from the set { 1 , . . . , 2x — 1}, where x is any integer number between 1 and | . The value of x is fixed after the choice of the input distribution, in order for A to be the deterministic on-line algorithm maximizing the expected number of accepted requests with respect to the specific probability distribution on the request sequences, i.e. A to be the "best" deterministic algorithm. Thus, with probability p, A accepts x requests. Additionally, A has to accept exactly x requests from the graph Kg on input L2g, because ^4 is a deterministic on-line algorithm and the vertices of Kg precede the vertices of Hg. Consequently, A can accept at most (/3 — 2x) requests from the subgraph Hg. Thus, the expected number of accepted requests of the best deterministic algorithm in this distribution is at most
E{A) = px + {1 - p){p ~ x)
92 For p = | , this reduces to E{A) = 2 ,
and E(OPT) = 1
+0) >
3/3
Thus, any deterministic on-line algorithm cannot achieve a competitive ratio less than | against this probability distribution. Consequently, Lemma 2.5 implies that | is a lower bound on the performance of any randomized on-line algorithm against the oblivious adversary. D In the on-line setting, we mainly face information-theoretic questions that have to do with the value of information on the computation of a minimum cost labelling. Thus, the lower bound holds for any on-line algorithm A, and it does not depend on the running time of A. On the other hand, even if we know the entire request sequence, radio labelling is AfP-complete, and, therefore, not expected to be solvable optimally in polynomial time. However, the proof of Lemma 5.2 does not take into account the Af'P-completeness of radio labelling in Hp. Consequently, a polynomial-time on-line algorithm with competitive ratio | is unlikely to exist.
5.2
On-line Radio Labelling — Assignment Version
We continue to analyze the performance of the greedy algorithm AGREEDY for the assignment version of on-line radio labelling, where the algorithm is not allowed to reject requests. AGREEDY assigns to a newly arrived request i, i = 1 , . . . , n, the least integer j such that both j has not been assigned to a previous request, and the resulting radio labelling is a valid one. Since there does not exist an upper bound on the labels used, such a label j always exist. L e m m a 5.3 The competitive ratio CA of A G R E E D Y is 2 — -
— -.
Proof. For any sequence of n requests, the value of a labelling computed by AGREEDY is at most 2n — 1, since the labelling 1 , 3 , . . . , 2n — 1 is always a valid one. The upper bound on the competitive ratio follows from the fact that the value of an optimal labelling cannot be less than n. To show the lower bound, for any even integer n > 2, let Un be the graph on n vertices obtained from Kn by removing the edges of a Hamiltonian path. Hence, the complement of Un is a Hamiltonian path on n vertices, and RL([/„) = n. Let Mi,u 2 ,... ,un be an ordering of the vertices of U(n) according to their appearance in the Hamiltonian path contained by the complementary graph; that is, for i = 1,... n — 1, the vertices ut and u i + 1 are not adjacent in Un. The adversary requests
93 all the vertices of Un in the following order: for i = 0 , . . . , f - 1 , it requests the vertex Mj+i followed by the vertex un-{. Claim 2 Fori = 0 , . . . , | - 2 , A G R E E D Y assigns to the vertexui+i the label4(i+l)-3 and to un-i the label 4(i + 1) — 1. Moreover, A G R E E D Y computes a radio labelling of Un of value 2n — 3. Proof of the Claim. We prove the first part of the claim by induction on i. Clearly, for i = 0, the vertex ui gets the label 1, and the vertex un gets the label 3, since u\ and un are adjacent in Un. Assume inductively that the claim holds for any i, 0 < i < | — 2. Then, for i + 1, the adversary requests the vertices Mi+2 and un_(j+i). In [/„, the vertex ui+2 is adjacent to all the previously requested vertices except u; +1 . Since the vertex u n _j has been assigned the label 4(i + 1) — 1, A G R E E D Y assigns to ui+2 the label 4(i + 1) + 1 = 4(i + 2) — 3. Similarly, since all the previously requested vertices except « n _j are adjacent to the vertex «„_( i+1 ), A G R E E D Y assigns to M„_(i+1j the label 4(i + 2) — 1. Therefore, for i — | — 2, the A G R E E D Y algorithm assigns to the vertex u„/2^i the label 2n — 7, and to wn/2+2 the label 2n — 5. Then, the adversary requests the vertices un/2 and Mn/2+i that get the labels 2n — 3 and 2n — 4 respectively. • Since the optimal off-line algorithm can easily compute a radio labelling of value exactly n, we obtain a lower bound of 2 — | on the competitive ratio of the algorithm AGREEDY.
6
•
Open Problems
An interesting direction for further research is to obtain a polynomial-time approximation algorithm for radio coloring in planar graphs using our exact algorithm for radio labelling. One approach may be to decompose the planar graph to subgraphs, such that almost all vertices of each subgraph get distinct colors by an optimal (or near optimal) assignment. Then, our exact algorithm can be used for computing an optimal radio labelling for each of the resulting subgraphs. Obviously, the decomposition of the planar graph and the combination of the partial assignments to a near optimal solution require an appropriate planar separator theorem. Another research direction is the conjecture that, given a graph G(V, E) and a partition of V into K cliques, G is Hamiltonian iff there exists a Hamiltonian cycle containing at most 2(K — 1) inter-clique edges. This would imply an n°^ exact algorithm for radio labelling in the complementary graph G, given a coloring of G with K
94 colors. In this paper, we prove the conjectured bound for graphs and partitions that contain at least one Hamiltonian cycle H, such that any vertex v £ V has degree at most one in the set of inter-clique edges of H. Additionally, it may be possible to improve the complexity of the algorithm of Theorem 4.1 to 0(f(K)p(n)), where /(re) is a fixed function of re, e.g. /(re) = 2" , and p(n) is a fixed polynomial in n of degree not depending on re, e.g. p(n) = n.
References [1] K. Aardal, C.A.J. Hurkens, J.K. Lenstra, and S.R. Tiourine (1996), "Algorithms for frequency assignment problems", CWI Quarterly 9, pp. 1-9. [2] S.M. Allen, D.H. Smith, and S. Hurley (1997), "Lower Bounding Techniques for Frequency Assignment", Submitted to Discrete Mathematics. [3] C. Berge (1985), Graphs (second edition), North Holland. [4] A. Borodin, N. Linial, and M. Saks (1992), "An optimal on-line algorithm for metrical task systems", Journal of the Association for Computing Machinery 39(4), pp. 745-763. [5] M.R. Garey and D.S. Johnson (1979), Computers and Intractability: A Guide to the Theory of MV'-completeness, Freeman, San Francisco. [6] R.A.H. Gower and R.A. Leese (1997), "The Sensitivity of Channel Assignment to Constraint Specification", Proc. of the 12th International Symposium on Electromagnetic Compatibility, pp. 131-136. [7] W.K. Hale (1980), "Frequency Assignment: Theory and Applications", Proceedings of the IEEE 68(12), pp. 1497-1514. [8] M. Halldorsson (1993), "A still better performance guarantee for approximate graph coloring", Information Processing Letters 45, pp. 19-23. [9] F. Harary (1997), Personal Communication. [10] J. van den Heuvel, R.A. Leese, and M.A. Shepherd (1996), "Graph Labelling and Radio Channel Assignment", Manuscript available from http://www.maths.ox.ac.uk/users/gowerr/preprints.html. [11] D.S. Hochbaum (1997), Approximation Algorithms for NV-hard Publishing.
Problems, PWS
95 [12] D. Karger, R. Motwani, and M. Sudan (1994), "Approximate graph coloring by semidefinite programming", Proc. of the 35th IEEE Symposium on Foundations of Computer Science, pp. 2-13. [13] S. Khanna and K. Kumaran (1998), "On Wireless Spectrum Estimation and Generalized Graph Coloring", Proc. of the 17th Joint Conference of IEEE Computer and Communications Societies - INFOCOM. [14] C. Lund and M. Yannakakis (1994), "On the Hardness of Approximating Minimization Problems", Journal of the Association for Computing Machinery 4 1 , pp. 960-981. [15] G. Pantziou, G. Pentaris, and P. Spirakis (1997), "Competitive Control in Mobile Networks", Proc. of the 8th International Symposium on Algorithms and Computation, pp. 404-413. [16] C.H. Papadimitriou and M. Yannakakis (1993), "The Traveling Salesman Problem with Distances One and Two", Mathematics of Operations Research 18(1), pp. 1-11. [17] A. Raychaudhuri (1985), Intersection assignments, graphs, PhD Thesis, Rutgers University.
T-colourings and powers of
[18] D. Sleator and R.E. Tarjan (1985), "Amortized Efficiency of List Update and Paging Rules", Communications of the Association for Computing Machinery 28, pp. 202-208.
This page is intentionally left blank
Combinatorial and Global Optimization, pp. 97-110 P.M. Pardalos, A. Migdalas and R. Burkard, Editors © 2002 World Scientific Publishing Co.
Image Space Analysis for Vector Optimization and Variational Inequalities. Scalarization F . Giannessi Department of Mathematics, University of Pisa, Via F. Buonarroti 2 56127 Pisa, Italy
L. Pellegrini Institute of Mathematics, University of Verona, Via dell'Artigliere 19 37129 Verona, Italy
Abstract
Vector constrained extremum problems and Vector Variational Inequalities are considered, and a separation scheme is introduced. Starting from such a scheme, several theoretical aspects can be developed as well as methods of solution; for instance, scalarization of Vector Optimization is analysed and a method is described which does not require any assumption on the problem. The analysis is extended to a Vector Variational Inequality; this turns out to be equivalent to a scalar Quasi-Variational Inequality. K e y w o r d s : Vector Optimization, Vector Variational Inequalities, Separation, Scalarization.
98
1
Introduction
In the last few years there has been a growing interest in vector problems, both from a theoretical point of view and as it concerns the applications to real problems. This paper aims to contribute to the definition of a general scheme for carrying out the analysis of vector problems as well as for finding methods of solution. Such a scheme was proposed in [10]; it has produced some developments in the field of Vector Variational Inequalities [13] besides that of Vector Optimization [1,7]. As far as Vector Optimization, some papers have introduced some aspects of the image space analysis proposed in [10], independently from each other and from [10]. In the next section we will briefly recall the general scheme which we pose as the basis for analysing a vector extremum problem. In Sect.3, with reference to scalarization, it will be shown how the above scheme can help us in carrying out the analysis of several topics. In Sect.4 we discuss how to extend the approach to Variational Inequalities. It is shown that a Vector Variational Inequality is equivalent to a scalar Quasi-Variational Inequality. Let the positive integers £, m, n and the cone C C R be given. In the sequel it will be assumed that C is convex, closed and pointed with apex at the origin and with i n t C ^ 0; more precisely, with nonempty interior 1 . Consider the vector-valued functions / : R™ -> R £ , g : R n -> R m , and the subset X C R n . We will consider the following vector minimization problem, which is called generalized Pareto problem: (1.1)
minC\{0} f(x)
, subject to x €. R := {a; £ X : g(x) > 0},
where minc\{o} marks vector minimum with respect to the cone C\{0} : y £ R is a (global) vector minimum point (in short, v.m.p.) of (1.1), iff2
(1-2)
/(») 2c\{o}/(*) , Vzetf,
where the inequality means
f(v) ~ f(x) i C*\{0}. At C = Re+ (1.1) becomes the classic Pareto vector problem. A vector minimization problem which is often associated to (1.1) is the following one 3 , called weak vector problem: (1.3) 1
min i n t c/(a;) , s.t. x € R,
Some of the propositions which will be established do not require all these assumptions on C. Without cutting off the apex of C, (1.2) would require that y be the unique v.m.p. of (1.1): at i Vi f{z) — f(y) inequality (1.2) - with C instead of C\{0} - would become 0 $ C, which is false. 3 Different from (1.1), since different cones obviously identify different vector problems. 2
z L
99 where minj nt c marks vector minimum with respect to the cone i n t C : y G R is a (global) v.m.p. of (1.3), iff (1-4)
f(y)hntcf(x) , VxeR,
where the inequality means f(y) — f(x) ^ i n t C . The term weak comes from the following tradition. Notwithstanding the fact that (1.1) and (1.3) are distinct problems, since the solutions of (1.1) are solutions also of (1.3) (but not necessarily vice versa), then the solutions of (1.3) are often called "weak solutions" of (1.1). At C = R ^ (1.3) is called weak vector Pareto problem. In the sequel we will outline a separation scheme for (1.1). It would also be interesting to carry out such a scheme for (1.3) too. Connections between the two schemes might be useful.
2
A Separation Scheme
Let us now consider problem (1.1). It is trivial to note that (1.2) is satisfied iff the system (in the unknown x): (2.1)
f(y) - f(x) e C , f(y) - f(x) ± 0 , g(x) > 0. x £ X
is impossible. Consider the sets: - t t : = ( C \ { 0 } ) x R ™ , « : = { ( « , « ) e R ( x E " : « = / ( } ) - / ( i ) , v = g(x), x € X}. System (2.1) is impossible iff (2.2)
ftn/C
= 0.
Hence, to prove that y is a v.m.p. of (1.1) it is equivalent to show (2.2). Since in general this is too difficult, an approach for proving (2.2) consists in obtaining the existence of a hyperplane, such that % belongs to an open halfspace (defined by the hyperplane) and K. to its complement. The above hyperplane (if any) separates (in a disjunctive way) % and /C. More generally, we can look for a functional, such that H belongs to its positive level set and K, to the nonpositive one. % and /C are subsets of the space Re+m, which is called image space; K. is called the image of problem (1.1). It is easy to see that (1.1) is equivalent to the vector maximization problem: (2.3)
max C V [ 0 } u , s.t. (u,v) e K. n (Re x R™),
where maxc\{o} marks vector maximum with respect to the cone C\{0}; more precisely, (u, s ) e / C n (R* x R™) is a vector maximum point of (2.3) iff (2.4)
w^c\{o}W , V ( « , « ) e ^ f l ( R ' x K " ) .
100 (2.3) is called image problem associated to (1.1). In [10] the above approach was proposed both for Scalar (£ = 1) and Vector (C = R ^ i ^ l ) Optimization as well as for Variational Inequalities. Such a proposal has led to some developments in the scalar case (see, e.g., [2,6,11,14,21] and references therein), and to some initial results [1,7,13] also in the vector case. Recently, some papers (see, e.g., [3,8]) have been published in the field of Vector Optimization, which introduce, more or less explicitly and independently from each other, the image space (called objective or balance space). The Authors of such papers seem not to know the proposal made in [10] and the subsequent developments. For this reason, the present paper aims to contribute to avoid the re-introduction of already existing theories and to stimulate research in the field. Now, we will briefly outline a way of deriving several topics from the separation scheme; for details one is referred to [11]. Let C* := {c* € R* : (c*,c) > 0, Vc e C} denote the positive polar of C. Consider a function w : He x R m —• R given by w = w(u,v;0,\) = (9,u) + (\,v), where J e R ( and A G R m are parameters. We then have (u,v) €H
, 9 e int C\
A > 0 => {9, u) + (A, v) > 0,
or (2.5)
9 e int C* , A e R!? =4- U
Therefore, it is easy to see that the existence of 9 E int C* and A 6 R™, such that (2.6)
(9, f(y) - f(x)) + (A, g(x))
<0,VxeX,
is a sufficient condition for (2.2), and thus for the impossibility of (2.1), and hence for y to be a v.m.p. of (1.1). Under suitable assumptions, it is also a necessary condition. Likewise what has been done in [11], from (2.6) we can derive, besides optimality conditions, duality, penalty and gap functions theory. In fact, (2.6) can be considered as coming from the minimization of the scalar function (9,f(x)) over R, 9 being a (scalarizing) parameter; indeed, this is precisely the classic way of scalarizing (1.1). Of course, the above way of separating % and K, is not the only one. Instead of a linear separation function, like the above w, we can introduce a nonlinear ones; among these, a particularly interesting one is a piecewise linear separation function, which has been studied in [4]. If this piecewise separation scheme is adopted, then we recover the classic vector duality theory [22].
101
3
On the Scalarization of Vector Optimization
Now, let us discuss one of the most analysed topics in Vector Optimization: scalarization of (1.1), namely how to set up a scalar minimization problem, which leads to detecting all the solutions of (1.1) or at least one. Assume that X be convex. Let us recall that / is called C-function, iff Vx',x" G X we have [11]: (3.1)
(1 - a)f(x')
+ af(x")
- / ( ( l - a)x' + ax") G C , Va G [0,1].
When C D R ^ o r C C R ' + , then / is called C-convex. At £ = 1 and C = R+ we recover the classic definition of convexity. Now, Vy 6 X, consider the sets 4 : S(y) :={x€X:
f(x) G f(y) - C} , Sp(y) := {x G X : (p, f(x))
< (p,
f(y))},
where p G C*. When X = R " and C = R+, then the above sets are level sets of / and (p, / ) , respectively. If / is linear, then S(y) is a cone with apex at y, and Sp(y) a supporting halfspace of S(y) at its apex. Proposition 1. If / is a C-function, then S(y) is convex My G X. Proof, x', x" <E S(y) => 3c', c" e C such that f(x') = }{y) - c' and f(x") = / ( ? / ) - c". From these equalities, since the convexity of C implies c := (1 — a)d + ac" € C, Va e [0,1], we find: (3.2)
{l-<x)f(x,)
+ af(x!') = f(y)-c
, Vae[0,l].
From (3.1) we have that 3c G C, such that: / ( ( l - a)x' + ax") = (1 - a)f(x')
+ af(x")
- c = f(y) - c - c = f(y) - c,
where (1 — a)x' + ax" G X (since X is convex), c := c + c G C because C is a convex cone, and the last but one equality comes from (3.2). It follows that (1 - a)x' + ax" G S(y) , Va e [0,1], W, x" G S{y). Now, consider any fixed p G C*, and introduce the (scalar) minimization problem: (3.3)
min(p, f(x)}
, s.t.
x€RnS(y),
which depends on the parameter y. 4
In the sequel, p will not play the role of a parameter and will be considered fixed.
D
102 Proposition 2. Let X be convex. If / is a C-function, g is concave and p G C*, then (3.3) is convex. Proof. We have to show that (p, / ) and RnS(y)
W,x"
are convex, p e C* and (3.1) imply,
eX, (p, (1 - a ) / ( a 0 + a / ( i " ) - / ( ( l - a ) z ' + ax")) > 0 , Va 6 [0,1],
or (p, / ( ( l - a)a:' + ax")) < (1 - a)(p, /(*')) + a(p, / ( * " ) } , Va G [0,1], which expresses the convexity of (p, f{x)). The convexity of X and the concavity of g give the convexity of R. Because of Proposition 1 we obtain the convexity of S(y) and hence that of R n S(y). D
Proposition 3 . If p € C*, then (3.4)
S(y)CSp(y)
, y e S(y)nSp(y)
, Vy e R» .
Proof, a; G 5(y) => 3c € C such that f(x) = f(y) — c. From this equality, taken into account that p G C* and c G C imply (p, c) > 0, we have: (P,f(x))
= (p,/(l/)) -,
The 1st of (3.4) follows. O e C o j e (3.4) holds.
S(y);y
G S p (y) is trivial; hence the 2nd of O
Now, let us state some properties; they might be useful in defining a method for finding one or all the solutions of (1.1) by solving (3.3). Proposition 4. Let p G intC* be fixed. Then, (2.1) is impossible - and hence y is a solution of (1.1) - iff the system (in the unknown x): (3.5)
xeX
is impossible. Furthermore, the impossibility of (3.5) is a necessary and sufficient condition for y to be a (scalar) minimum point of (3.3). Proof. The 1st of (3.5)=> f(y) - f(x) / 0, so that the possibility of (3.5) implies that of (2.1). The 1st of (2.1) and p G intC* imply the 1st of (3.5), so that the possibility of (2.1) implies that of (3.5). By replacing the 1st of (3.5) equivalently with (p,f(x)) < (p,f(y)), we immediately see the triviality of the 2nd part of the statement. Dl
103 Proposition 4 shows that y° is a v.m.p. of (1.1) iff it is a (scalar) minimum point of (3.3) at y = y°. For every y G R let A(y) denote the set of solutions of (3.3); Proposition 4 states that y° is a v.m.p. of (1.1) iff it is a fixed point of the point-toset map A : X =t X. The following results show how it is possible to determine such a fixed point. Proposition 5. We have: (3.6)
x° G S(y°)
=• S(x°) C S(y°).
Proof. x° G S(y°) => 3c° 6 C such that f(x°) = f{y°) - c°. x € S(x°) => 3c G C such that / ( £ ) = /(a; 0 ) — c. Summing side by side the two equalities we obtain f(x) = f(y°) — c, where c := c° + c G C since C is a convex cone. It follows that x G S{y°) and hence 5(a:°) C S(y°). O
Proposition 6. If x° is a (global) minimum point of (3.3) at y — y°, then a;0 is a (global) minimum point of (3.3) also at y = x°. Proof. Ab absurdo, suppose that a;0 is not a (global) minimum point of (3.3) at y = x°. Then (3.7)
3xeRnS(x°)
s.t.
(pj(x))<(p,f(x0)).
Because of Proposition 5, x° G S{y°) =^> S(x°) C S{y°). This inclusion and (3.7) imply xeRnS(y°) and (p,f(x))<(p,f(x0)), which contradict the assumption.
D
Proposition 6 suggests a method for finding a v.m.p. of (1.1). Let us choose any p G intC*; p will remain fixed in the sequel. Then, we choose any y° G R and solve the (scalar) problem (3.3) at y = y°. We find a solution a;0 (if any). According to Proposition 6, a:0 is a fixed point of the point-to-set map A and then it is a v.m.p. of (1.1). If we want to find all the solutions of (1.1), we must look at (3.3) as a parametric problem with respect to y; Propositions 4 and 6 guarantee that all the solutions of (1.1) will be reached. Note that such a scalarization method does not require any assumption on (1.1). Example 1. Let us set I = 2, m = 2, n = 1, X = R, C = R+, and /i(x) = 2 x - a ; 2 , f2{x) = 1 - x 2 , gi{x) = x, g2(x) = l - x , / = ( / i , / 2 ) , g= {gi,g2).
104 We find S(y) = {y} Vy £ [0,1]. Hence, the unique solution of (3.3) is y itself. By varying y, (3.3) gives, with its solutions, the interval [0,1], which is the set of v.m.p. of (1.1), as it is trivial to check. Now, let us use the classic scalarization [5,21], i.e. the scalar parametric problem which, here, becomes: (3.8)
min[cifi(x)
+ c2f2(x) = - ( c i + c2)x2 + 2cxx + c2], s.t. x G [0,1],
where (ci,C2) € intC* = int R^. are parameters. Such a scalarization aims to find all the v.m.p. of (1.1) by solving (3.8) with respect to all possible pairs of parameters (ci,c 2 ). In the present example it is easy to see that the only solutions of (3.8) are x = 0, or a; = 0 and x = 1, or x = 1, according to respectively c2 < C\, or c2 = ci, or c2 > C\. Hence, the scalarized problem (3.8) does not detect all the solutions of (1.1). In order to stress the differences between the classic scalarization of a Vector Optimization Problem and the present one, let us consider the following: Example 2. Let us set £ = 2,m = l , n = 2,X = R 2 , C = R ^ , x = {x1,x2),y (2/1,2/2), and
=
fi(x) = xi+ 2x 2 , f2(x) = 4xx + 2x2 , g{x) = ~\xx\ + x2. Choose p = (1,1) and y° = (0,1). Then (3.3) becomes: (3.9)
min(5xi + 4x 2 ) , s.t. - |xi| + x2 > 0, xx + 1x2 < 2,2x\ + x2 < 1.
The (unique) solution of (3.9) is easily found to be x° = (—2, 2). Because of Proposition 6 x° is a v.m.p. of (1.1) in the present case. Furthermore, we have R!~\ S(x°) = {x0}, namely the parametric system (in the unknown x): (3.10)
-|a;i I + x2 > 0 , x1 + 2x2
+ 2y2 , 2xx + x2 < 2yx + y2
has the (unique) solution a;0. In order to find all the v.m.p. of (1.1) we have to search for all y € R such that (3.10) has y itself as the (unique) solution. (3.10) is equivalent to /3
11 - )
/ kil < 1
x2 < ~^xx + ^(yi + 2y2) x2< - 2 x i + 2j/i + y2.
With xi > 0, (3.11) cannot have y as (unique) solution. Hence, we consider the case Xi < 0; by using Motzkin elimination method and by requiring a unique solution, (3.11) becomes: ~~2Xl
=
2^Vl
+ 2y2
^
'
Xl = 2yi
+ V2
'
Xl
~ °
105 and leads us to y\ + y2 = 0, y\ < 0 or
y = (yi = -t, V2 = t) , te [o, +oo[, which gives us all the v.m.p. of (1.1). Now, let us use the classic scalarization [5,21], i.e. the scalar parametric problem which, here, becomes: (3.12) min[ci/i(a:) + c2f2{x) = ( d + 4c 2 )zi + (2c : + 2c 2 )x 2 ], s.t.
-\x1\+x2>0,
where (ci,c2) £ C*\{0} = R+ \{0} are parameters. Such a scalarization aims detect all the v.m.p. of (1.1) by solving (3.12) with respect to all possible pairs parameters (ci,c 2 ). In the present example, it is trivial to see that the minimum (3.12) exists iff —\c\ < c2 < \c\, and that at 0 < c2 < | c i , the minimum points (3.12) are all the v.m.p. of (1.1).
4
to of of of
Vector Variational Inequalities
The approach outlined in Sect.2 in reference to (1.1) can be adopted also in fields other than Optimization. Indeed, the starting point is the impossibility of a system; (2.1) is a special case. Now, let F : R " —> R £ x n be a matrix-valued function, and consider the following Vector Variational Inequality: find y G K := {x £ X : g(x) > 0} such that (4.1)
F(y){x - y) £c\{0} 0 , Vz e K.
At £ = 1 (4.1) becomes the classic Stampacchia Variational Inequality 5 [15]. At £ > 1 and C = R ^ , the study of (4.1) has been proposed in [10]. Obviously, y is a solution of (4.1) iff the system (in the unknown x): (4.2)
F(y)(y - x) 6 C , F(y){y - x) # 0 , g(x) > 0 , x € X
is impossible. Starting from this point several topics can be developed; see, for instance, [13]. Here, we take into consideration the scalarization of (4.1). To this aim consider the sets: T(y) := { i 6 R " : F(y)x e F{y)y-C}
, Tp{y) := {x 6 R" :
(PF(y),y)},
where p G C* is considered a row-vector. When C = R^_, then the above sets are the level sets of the vector function F{y)x and of the scalar function (pF(y),x). Tp(y) is a supporting halfspace of T(y) at y, as Proposition 11 will show. If F(y) is a constant matrix and C is polyhedral, then T(y) is a polyhedron. 5
It would be interesting to extend the present analysis to the Minty Vector Variational Inequality
[12,16].
106 Proposition 7. T(y) is convex \/y e X. Proof. x',x" e T(y) => 3c', c" € C such that F{y)x' F(y)y — c". From these equalities, Va € [0,1] we have:
= F{y)y - d,F(y)x"
=
F(y)[(l - a)x' + ax"] = F{y)y - c(a), where c(a) := (1 — a)d + ad' e C since C is convex. Hence (1 — a)x' + ax" € T{y) for each a 6 [0,1]. O
Now, let us introduce the (scalar) Quasi-Variational Inequality which consists in finding y e K(y) := K II T(y) such that (4.3)
{Fp(y),x-y)>0,VxeK(y),
where Fp(y) := pF(y) and p has to be considered fixed; (4.3) is a scalarization of (4.1). F will be called C-operator iff (4.4)
[F{x') - F{x")]{x' - x") € C , Va:', x" G R n .
When C D R+ or C C R^_, then F will be called C'-monotone; when £ = 1, the notion of C-operator collapses to classic ones: F becomes monotone or antitone, according to C = R + or C = R_, respectively. Proposition 8. If X is convex, F is a C-operator, g is concave, and p e C*, then (4.3) is monotone. Proof. We have to show that K(y) is convex and Fp monotone. The assumptions on X and g imply the convexity of K; because of Proposition 7, T(y) is convex Vy € R n . Hence K(y) is convex Vy G R". p € C* and (4.4) imply (p, [ i V ) - F{x")}{x' - x")) > 0 W,x"
€ Rn,
or (Fp(x') - Fp{x"),x' - x") > 0 Va;',x" e E " .
Proposition 9. If p £ C*, then (4-5)
T(y)CTp(y)
, T(y)nTp(u)
D {y}.
D
107 Proof, x 6 T(y) =>• 3c € C such that F(y)x = F(y)y — c. From this equality, account being taken that p € C* and c € C imply (p, c) > 0, we have: (PF(y),x)
= (pF(y),y)
- (p,c) <
(pF(y),y).
The 1st of (4.5) follows. The 2nd of (4.5) is a consequence of the obvious relations V £ T(y) (due to the closure of C) and y € Tp(y). D
Now, let us state some preliminary properties, which might help in finding methods for solving (4.1) through (4.3). Proposition 10. If p € intC*, then (4.2) is impossible - and hence y is a solution of (4.1) - iff the system (in the unknown x): (4.6)
(PF(y), y-x)>0
, F{y){y - i ) e C , g(x) > 0 , x € X
is impossible. Furthermore, the impossibility of (4.6) is a necessary and sufficient condition for y to be a solution of (4.3). Proof. The 1st of (4.6)=> F(y)(y — x) =f= 0, so that the possibility of (4.6) implies that of (4.2). The 1st of (4.2) and p e intC* imply the 1st of (4.6). The last part of statement is trivial. D Note that Proposition 10 shows that a Vector Variational Inequality can be equival e n t ^ replaced with a scalar Quasi-Variational one. Consider, now, the function
(Fp{y),x-y),
and the problem (where y € K(y) is a parameter): (4.7)
mm(p(x;y)
, s.t. x £ K{y).
Since ip(y; y) = 0, for any fixed y the minimum in (4.7) is < 0. Proposition 11. y € K is a solution of (4.1) iff it is a global minimum point of (4.7). Proof. Let y be a global minimum point of (4.7), so that tp(y; y) = 0 and tp(x; y) > 0 for each x e K(y). Hence, because of Proposition 10, y is a solution of (4.1). Now, let y be a solution of (4.1). Because of Proposition 10, system (4.6) is impossible, so that y is a solution of (4.3); this implies (p(x; y) > 0 Mx e K(y). Since tp(y; y) = 0, y is a global minimum point of (4.7). D
108 Example 3. Let us set I = 2, m = 1, n = 2, C = R2+ = C, x — (xi,x2), g(x) = Xi + x2 — 2, and F ( v ) = w
(201-2 V2y: + 2
X = K2+, y = {yu y2),
2y2 + 2 2y2~2j-
We choose p = (pi,P2) = (1> l)i so that (fi(x; y) = 4(2/13;! + y2x2 - y\ - y^), f (1 - Vl)Xl - (1 + 2/2)^2 + y? + 2/1 - 2/i + 2/2 > 0 •^(2/) : S - ( 1 + yi)a:i + (1 - 2/2)^2 + 2/1 + y\ + 2/1 - y2 > 0 [ X\ + x2 > 2 , i i > 0 , x2 > 0. It is easy to check that any y° which belongs to the segment ](2,0), (0, 2)[c R 2 is a solution of (4.7) at y = y° and hence of (4.1). At y = y = (2,0), problem (4.7) becomes min4(xi — 2) , s.t. x\ + x2 = 2, 3 ^ ! — x2 < 6, x\ > 0, x2 > 0, whose unique global minimum point is x = (0, 2) / y. Hence, because of Proposition 11, y is not solution of (4.1). A quite analogous conclusions can be drawn at y = (0, 2). Note that in Example 3 the operator F(y) is the Jacobian matrix of the vector function / : R 2 -> R 2 , given by fr
, A _ ( / i ( 2 / h _ ( ( 2 / i - l ) 2 + (2/2 + l ) 2 \
/W
~ U ( 2 / ) J ~ V ( 2 / i + l) 2 + (2/ 2 -l) 2 J '
Consider (1.1) with the above function and with R equal to K of Example 3, i.e. K = {x 6 R ^ : Xi + x2 > 2}. It is easy to check that all the v.m.p. of (1.1) are given by the segment [(2,0), (0, 2)], whose extrema are not solutions of the related Vector Variational Inequality (4.1). Hence, such inequality is a sufficient (but not necessary) condition for y to be v.m.p. of (1.1).
References 1 G. Bigi and M. Pappalardo, "Regularity conditions in Vector Optimization". Jou. Optimization Theory and Appls., Plenum, New York, to appear. 2 R. Conti et al. (Eds.), "Optimization and related fields". Lecture Notes in Mathematics No. 1190, Springer-Verlag, Berlin, 1986, pp. 57-93.
109 3 J.P. Dauer, "Analysis of the objective space in multiple objective linear programming". Jou. Mathematical Analysis and Appls.", Vol. 126, 1987, pp. 579-593. 4 P.H. Dien, G. Mastroeni, M. Pappalardo and P.H. Quang, "Regularity conditions for constrained extremum problems via image space: the linear case". In Lecture Notes in Sc. and Mathem. Systems, No. 405, Komlosi, Rapcsack, Schaible Eds., Springer-Verlag, 1994, pp. 145-152. 5 Dinh The Luc, "Theory of Vector Optimization". Lecture Notes in Ec. and Math. Systems No. 319, Springer-Verlag, Berlin, 1989. 6 G. Di Pillo et al. (Eds.), "Nonlinear optimization and Applications". Plenum, New York, 1996, pp. 13-26 and 171-179. 7 P. Favati and M. Pappalardo, "On the reciprocal Vector Optimization problems". Jou. Optimization Theory Appls., Plenum, New York, Vol. 47, No. 2, 1985, pp. 181-193. 8 P.A.V. Ferreira and M.E.S. Machado, "Solving multiple-objective problems in the objectice space". Jou. Optimization Theory Appls., Plenum, New York, Vol. 89, No. 3, 1996, pp. 659-680. 9 E.A. Galperin, "Nonscalarized multiobjective global optimization". Jou. Optimization Theory Appls., Plenum, New York, Vol. 75, No. 1, 1992, pp. 69-85. 10 F. Giannessi, "Theorems of the alternative, quadratic programs and complementarity problems". In "Variational Inequalities and complementarity problems", R.W. Cottle et al. Eds., J. Wiley, 1980, pp. 151-186. 11 F. Giannessi, "Theorems of the alternative and optimality conditions". Jou. Optimization Theory Appls., Plenum, New York, Vol. 42, No. 11, 1984. pp. 331-365. 12 F. Giannessi, "On Minty variational principle". In "New trends in mathematical programming", F. Giannessi, S. Komlosi and T. Rapcsack Eds., Kluwer Acad. Publ., Dordrecht, 1998, pp. 93-99. 13 F. Giannessi Ed., "Vector Variational Inequalities and Vector equilibria. Mathematical Theories". Kluwer Acad. Publ., New York 1999. 14 F. Giannessi and A. Maugeri (Eds.), "Variational Inequalities and Networks equilibrium problems". Plenum, New York, 1995, pp. 1-7, 21-31, 101-121 and 195-211. 15 D. Kinderleherer and G. Stampacchia, "An introduction to Variational inequalities". Academic Press, New York, 1980.
110
16 S. Komlosi and L. Pellegrini, "On the Stampacchia and Minty Vector Variational Inequalities". Submitted to [13]. 17 G. Mastroeni, "Separation methods for Vector Variational Inequalities. Saddle point and gap function". To appear in "Nonlinear Optimization and Applications", G. Di Pillo at al. (Eds.), Kluwer Acad. Publ., Dordrecht, 1999. 18 M. Pappalardo, "Stationarity in Vector Optimization". Rendiconti del Circolo Matematico di Palermo, Serie II, No. 48, 1997, pp. 195-200. 19 A. Pascoletti and P. Serafini, "Scalarizing Vector Optimization problems". Jou. Optimization Theory Appls., Plenum, New York, Vol. 42, No. 4, 1984, pp. 499-523. 20 L. Pellegrini, "On a general approach to Vector Optimization. Duality". To appear in "Nonlinear Optimization and Applications", G. Di Pillo et al. (Eds.), Kluwer Acad. Publ., Dordrecht, 1999. 21 T. Rapcsack, "Smooth Nonlinear Optimization in R n " . Series "Nonconvex Optimization and its Applications", No. 19, Kluwer Acad. Publ., Dordrecht, 1997. 22 S. Wang and Z. Li, "Scalarization and Lagrange duality in multiobjective optimization". Optimization, Vol. 26, Gordon and Breach Publ., 1992, pp. 315324.
Combinatorial and Global Optimization, pp. 111-121 P.M. Pardalos, A. Migdalas and R. Burkard, Editors © 2002 World Scientific Publishing Co.
Solving Quadratic Knapsack Problems by Reformulation and Tabu Search. Single Constraint Case Fred Glover University of Colorado at Boulder Gary Kochenberger University of Colorado at Derive Bahrain Alidaee University of Mississippi Mohammad Amini University of Memphis
Abstract Several recent papers have presented new approaches to special cases of the quadratic knapsack (QK) problem. Despite the advances reported, these problems remain, in general, very challenging to solve. Models containing more than 100 variables tax current methods to their limits. Moreover, current methods can only handle special instances of quadratic objective functions, leaving a large range of important problems unconsidered. In this paper, we show that it is possible to handle quadratic knapsack problems of dramatically greater size than those within the capacity of previous methods, and in addition to handle entirely general quadratic objectives. This outcome results by the device of reformulating the QK problem as an unconstrained binary quadratic problem, and applying a recently developed metaheuristic to this latter version. We report computational experience that discloses the viability of our approach. Keywords: quadratic knapsack problem, unconstrained binary quadratic problem, tabu search
112
1
Introduction
The Quadratic (multidimensional) Knapsack problem is one of several classic NPhard problems that has been the object of considerable research over the years. The general problem can be stated as QK : max xQx st Ax < b xe{o,i} We assume the coefficients of the matrix A are all non-negative, and those of b are all positive. No sign conditions are assumed for Q. Note that any linear term originally appearing in the objective function can be absorbed into the diagonal of Q, since Xj = x? for any 0-1 variable Xj. In this paper we focus on the single knapsack case, where A consists of a single row vector, although the approach taken can be applied to the more general case with multiple knapsack constraints. Program QK has many applications, having been shown to address problems arising in capital budgeting, flexible manufacturing, earth station site selection for satellite communication systems, task assignments in host satellite systems, and a variety of other problems. The references given, and particularly the paper by Gallo, Hammer and Simeone [4], give discussions of various applications. A significant number of these applications involve the single knapsack case we examine here, and most of the algorithms developed for QK also focus on this case. It is also possible, via the types of re-formulations recently highlighted by Kochenberger, Alidaee, and Amini [12], to recast many traditional models into the form of QK. Thus, the applicability of QK is quite broad in its scope. Several notable papers have recently appeared in the literature advancing our ability to solve special cases of the single knapsack QK. Billionnet and Calmels [2] discuss a branch and bound algorithm based on linearizations while Michelon and Veilleax [14] present a branch and bound approach based on Lagrangian decomposition and Lagrangian relaxation. Each of these algorithms assumes the matrix Q is positive. Both approaches employ variable fixing routines and both report computational experience with random problems of size up to 40 variables. More recently, Hammer and Rader [8] gave a branch and bound algorithm for the single knapsack QK based on repeated use of the L2 -best linear approximation. Variable fixing procedures, based on Lagrangian relaxation, are explored and computational experience on problems with up to 100 variables is reported. The Hammer and Rader algorithm, like the two methods cited above, assumes a positive Q matrix. Since QK is NP-Hard, exact algorithms are likely to degrade in performance as the
113 problem size grows. As expected, the experience reported in the three algorithms referenced above illustrates the rapid growth in computation times as problem size grows. The state of the art, even for this special class of problems (positive Q) with a single knapsack constraint, is rather limited in terms of the size of problem that can be solved by exact methods. Larger problems, and those involving a general Q matrix, are computationally demanding and more amenable to heuristics than exact methods. The approach we take in this paper is to reformulate QK as an unconstrained binary quadratic program (QP) and then solve the reformulated, equivalent model by recently developed heuristics designed to solve QP. This approach is motivated by the success reported on new solution approaches for QP. See for example, the recent papers by Pardalos and Rodgers [15], Chardaire and Sutter [3], Glover, Kochenberger and Alidaee [5], Glover, Kochenberger, Alidaee, and Amini [6], Hammer, Boros, and Sun [7], and Lodi, Allemand, and Liebling [13]. In this case we make use of the method of [6]. Although heuristic, this method has been demonstrated to be so effective that it has obtained optimal solutions to all benchmark problems that exact methods have been able to solve, while requiring greatly reduced computational time by comparison to the exact methods. In the sections that follow, we present the transformation used to convert QK into QP. We then present our computational experience followed by some summary comments.
2
Reformulation
Transformations for reformulating integer programs in the manner employed here have been proposed by several authors. Discussions relevant to our work, and upon which we draw, are given by Bazarra and Goode [1], Hammer and Rudeanu [9], Hansen [10], Hansen et al [11], and Sinclair [16]. For a recent paper highlighting such reformulations in a variety of model settings, the reader is referred to Kochenberger, Alidaee, and Amini [12]. In the work reported here, we convert Quadratic Knapsack models into an unconstrained quadratic program (QP) by employing a quadratic infeasibility penalty. While other representations are possible, we restrict our efforts to quadratic penalties in order to take advantage of our ability to solve QP. Adding slack variables (in binary expansion form), we can write the quadratic knapsack problem as: QK : max Xo = xQx st Ax = b xe{0,l}
114 where x, A, and Q have been augmented to include the slack variables. Then, the equality constraints can be accommodated by introducing a positive penalty, PEN, and the associated penalty term PEN(Ax - b)'(Ax - b). Subtracting this term from the objective function yields the equivalent penalized program QK(PEN) : max x 0 = = st
xQx - PEN(Ax - b)'(Ax - b) xQx — xDx + ex + constant x€{0,l} l
where D = PEN * A*A, c = 2PEN * b A, and the constant = - P E N * b'b. The additive constant does not effect the optimization and can be ignored for the time being. The remaining three terms can be combined into a single term allowing us to represent the equivalent unconstrained quadratic program simply as: QK(PEN) : max xQx st xe{o,i} where qjj = q5j — dy for i ^ j and qj, = qn—dn+Ci for the diagonal terms. As previously remarked, we solve QK(PEN) using the Tabu Search (TS) approach reported in Glover, Kochenberger, Alidaee, and Amini [6]. An overview of our TS method is given in the appendix. Example 1. To illustrate the reformulation procedure, consider the example used by Hammer and Rader [8]: maximize xo =
2xi + 5x2 + 2x3 + 4x4 + 8x1X2 + 6x1X3 +10xix 4 + 2x 2 x 3 + 6x2X4 + 4x 3 x 4
st 8xi + 6x2 + 5x 3 + 3x4 < 16 Introducing a slack activity, bounded above by 3, as lx 5 + 2x6, the transformation can be used to give an equivalent unconstrained problem in 6 binary variables. (The upper bound on the slack is selected to be large enough to admit an optimal solution. This can be done by several approaches, which we do not address here.) Choosing PEN to be 10, we get QP(PEN) with Q given by " 1922 -476 -397 -235 -80 -476 1565 -299 -177 -60 -397 -299 1352 -148 -50 874 -30 -235 -177 -148
-80
-60
-50
-160 -120 -100
-30 310 -60 -20
-160 " -120 -100 -60 -20 600
115 and an additive constant of —2560. Solving this unconstrained problem, QP(PEN), gives the solution x 0 = 2588 which is found at x = {1,0,1,1,0,0} . Adjusting for the additive constant, we have the original objective function value of 28. In this example, both slack variables are equal to zero and thus the knapsack constraint is tight at the optimal solution given. We now turn to the computational work we carried out and the results obtained.
3
Computational Experiments
To test our approach to quadratic knapsack problems, 44 random quadratic knapsack problems were generated, re-formulated as unconstrained quadratic binary programs, and solved using our tabu search algorithm. Each problem was generated with qy between 0 and 25, aj between 1 and 10, and b chosen between 10 and X) a j- Problem sizes ranged from 10 to 500 variables with densities varying from 25% to 100 %. Each problem had a single knapsack constraint that was converted to an equality with the addition of six additional binary slack variables. This allowed for a slack activity up to size 63 (which proved to more than sufficient). Tables 1 and 2 summarize our results. For each problem, we report problem characteristics, along with the solution obtained in the allotted number of Tabu Search oscillation cycles and computation times. The solutions shown in Table 1, with the exception of problem C03, are in fact optimal, as verified by the branch and bound algorithm of Pardalos and Rodgers (P&R) [15]. Problem C03 could not be solved to completion by the exact method of [15] and thus an optimal solution to this problem is not known. Likewise, optimal solutions for the problems of Table 2 are yet to be established. These problems, while readily solved by our Tabu Search heuristic, proved to be beyond the capability of the branch and bound algorithm. (Branch and bound (P&R) run times in excess of six hours failed to produce optimal solutions for the smallest of these problems.) The results shown in Table 2 are the best values found within the arbitrary cycle limit imposed in our tabu search approach. Certainly the choice of the parameter PEN is key to the approach we are taking. Theoretically, it is sufficient to choose PEN to be greater than the sum of the absolute values of the elements of Q. Our experience indicates, however, that feasible solutions can be obtained with much smaller values for PEN. PEN needs to be large enough to make infeasible choices unattractive to the search/optimization process. Yet choosing PEN somewhat larger than necessary may cause the elements of Q to become unnecessarily large, masking the relevance of Q and making the problem, in principle, harder to solve. This potential difficulty, as discussed later in this section, did not prove to be a problem for us. The approach taken in the work reported here was to start with a small value for PEN and raise it as needed in order to produce
116
Problem ID
A01 A01 A03 A04 A05 A06 A07 A08 B01 B02 B03 B04 B05 B06 B07 B08 C01 C02 C03 C04 C05 C06 C07 C08
Table 1: COMPUTATIONAL RESULTS # variables Density PEN Solution # TS Cycles 10 0.25 50 100 6220 10 50 100 0.25 24373 10 100 0.5 50 1357 10 0.5 50 45342 100 10 50 3408 100 0.75 100 10 0.75 50 34220 1 58612 100 10 50 10 1 50 42462 100 50 180947 100 20 0.25 20 50 76794 100 0.25 20 50 73279 100 0.5 20 0.5 50 226432 100 20 50 347331 100 0.75 20 100 0.75 50 55570 20 1 50 194715 100 20 1 50 32605 100 30 100 0.25 250 156990 30 50 100 0.25 2635 30 250 1262937 100 0.5 30 250 814340 100 0.5 30 50 4631 100 0.75 100 30 0.75 250 273512 100 30 1 250 422510 6472 100 30 1 50
Time(sec)
<1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <1 <2 <2 <2 <2 <2 <2 <2 <2
feasible solutions. We arbitrarily took PEN to be 50, 250, 500, 750, 1000, or 1500. For the "A", " B " , "C", "D", and "E" problems, PEN was initially set to 50. Each problem was then solved and the solutions obtained were checked for feasibility. Problems that were infeasible were solved again with successively larger values of PEN until the knapsack constraint was satisfied. The " F " problems, based on what we learned from A-E, were solved with PEN = 1500 from the outset. Clearly PEN = 1500 (or something larger) would have worked for all the problems. Tables 1 and 2 report the values of PEN used to yield the solutions shown. In all cases, the solutions listed are feasible with respect to the original knapsack constraints. Choosing PEN too large, as mentioned above, has the potential to make a problem more difficult to solve than an appropriately chosen smaller value of PEN. For the testing we have done to date, this potential difficulty did not materialize. This is illustrated in Table 3 where we report results obtained by solving a particular problem,
117
Problem ID
D01 D02 D03 D04 D05 D06 D07 D08 E01 E02 E03 E04 E05 E06 E07 E08 F01 F02 F03 F04
Table 2: COMPUTATIONAL EXPERIENCE (CON'T) # variables Density PEN Solution # T S cycles Time(sec) 40 0.25 50 888033 100 <4 40 50 1607975 0.25 100 <4 40 0.5 50 177190 100 <4 40 50 1272126 100 <4 0.5 40 0.75 50 1076775 100 <4 40 0.75 50 1389799 100 <4 1 40 50 1549279 100 <4 1 <4 40 50 40931 100 100 50 2659975 0.25 100 <9 100 100 0.25 50 1953310 <9 100 100 0.5 500 4241440 <9 100 0.5 750 10993576 100 <9 100 500 2119812 100 <9 0.75 100 0.75 1000 11040166 100 <9 100 1 750 13696762 100 <9 100 1 1000 5943174 100 <9 500 0.25 1500 35491046 500 240 500 0.5 1500 35544448 500 240 500 0.75 1500 76078805 500 240 500 1 1500 76145674 500 240
C08, with various values of PEN ranging from 250 to 3000. Table 3 lists the value of PEN, the optimal solution value, and the Tabu Search cycle at which the optimal solution was obtained for these runs. In all cases, the solutions shown in Table 3 are feasible and optimal. Note that the performance of the heuristic was fairly uniform over the range of PEN values. While there is some variation in the cycle count required to reach the optimal solution, no discernible pattern is suggested by the table. Similar computations on other problems have also yielded only small variations in solution burden associated with differing PEN values. This is a somewhat surprising discovery since the folklore about using " infeasibility penalties" in discrete and nonlinear optimization is that their size can materially effect computational efficiency. The absence of this effect here suggest that our TS method is highly robust. Our approach had no trouble finding high quality, feasible solutions to all problems in the test set. And, compared to alternative methods from the literature, these solutions were found very quickly. The papers by Michelon and Veilleux [14] and Billionnet and Calmels [2] report computational experience on problems up to size n = 40 only. Their problems are comparable in size and density to our "A", " B " , "C", and "D"
118
Table 3: PERFORMANCE ON PROBLEM C08 FOR VARIOUS VALUES OF PEN PEN 250 500 750 1000 1500 3000
Optimal Value 30500 60750 91000 121250 181750 363250
Cycles Yielding Optimal 16 17 29 10 19 29
problems. For the 40 variable problems, Michelon and Veilleax report solution times of more than 900 seconds (SUN 4.0) while Billionet and Calmels report solutions times of approximately 500 seconds (HP 9000) for their 40 variable problems. Hammer and Rader (H&R) [8] report computational experience with problems up to size n = 100. They report solving completely dense, 40 variable problems in roughly 20 seconds (SPARC station 1+). For their 100 variable problems, H&R solve low density problems with admirable efficiency. For instance, 25% problems are solved in little more than 2 minutes. However, 100 % dense problems, of size 100, were solved in roughly 24 minutes. None of these authors report experience on larger problems. Since our approach is heuristic in nature, comparing the solution times we report in Tables 1 and 2 with the branch and bound times given above must be done with considerable care. Such comparisons are further confounded by the fact that different computers were used in the various studies and that our Tabu Search approach was run for an arbitrary number of oscillation cycles. Nonetheless, a quick comparison gives the reader some indication of the speed with which our approach generates high quality solutions to these problems. As shown in Tables 1 and 2, we produce optimal solutions to test problems up to 30 variables (and varying densities) in about 2 seconds on a Pentium 200 computer. Thus, as in the case of unconstrained problems, we obtain optimal solutions to all problems where such solutions are known, and we obtain such solutions in at least an order of magnitude faster than the exact approach. In many cases we are faster by two orders of magnitude. The remaining problems belong to the group that could not be solved by the exact methods. We produced high quality feasible solutions to the 40 variable and 100 variable problems in roughly 4 and 9 seconds, respectively. For our 500 variable problems, which we ran for 500 oscillation cycles, run times were approximately 4 minutes. To the best of our knowledge, ours is the first study to report on problems as large as 500 variables. The previous limit is represented by the 100 variable problems in the Hammer and Rader paper.
119 It is interesting to note that our solution times for a given problem size are not very sensitive to problem density. Problems with 500 variables are solved in about 4 minutes whether they are 25% dense or 100% dense. This behavior is in sharp contrast to the branch and bound algorithms whose performance degrades considerably with density. For example, on their 100 variable problems, H&R report approximate times of 2, 3, 8, and 24 minutes, respectively, for densities 25, 50, 75, and 100%. The time invariant nature of our approach is due to the fact that regardless of the density of the original matrix Q, the transformation produces a matrix Q which is very dense. This is illustrated in the example presented earlier where Q was fairly sparse and Q was 100 % dense. It should also be pointed out that the transformation destroys any nice properties (like all non-negative elements) that Q may have exhibited. Neither the absence of special structure nor presence of high density pose a problem for our approach.
4
Summary and Conclusions
For quadratic knapsack problems of modest size and special properties, efficient exact methods, like that of Hammer and Rader, exist. However, quadratic knapsack problems with hundreds of variables pose computational difficulties in excess of what can reasonably be handled by exact methods. Current exact methods also are unable to handle QK problems that have negative elements in their objective functions. For these larger and/or more general problems, heuristic methods must be employed. In this paper we have demonstrated that the unconstrained binary quadratic program, QP, is a viable alternative approach for modeling and solving QK problems. The reformulated problems are readily solved by our tabu search algorithm to yield high quality feasible solutions, which are optimal in all cases that can be solved by exact methods. Our results illustrate that large instances can be readily solved. For the class of problems considered here, choosing an appropriate value for the parameter PEN was straightforward. An appropriate choice, which for any problem depends on both problem size and problem data, can always be found by a minimum of experimentation, or, more formally, by simple target analysis. However, we found (surprisingly) that taking PEN somewhat larger than necessary did not significantly affect computational performance. It should be noted that no attempt was made to tailor our Tabu Search algorithm for the work carried out here. We used the algorithm of [6] for general QP problems with no modifications. (The method is also not tuned to obtain best results for these QP problems.) For instance, employing an improvement routine, designed specifically for QK problems, at critical points in the search process would likely lead to better results. Our approach likewise does not depend on any particular property of the
120 original Q matrix. Future research will address the solution of QK problems that include multiple knapsack constraints.
A P P E N D I X : Overview of our Tabu Search Algorithm Our algorithm is built around strategic oscillations that alternate between constructive phases (progressively setting variables to 1, ADDS) and destructive phases (progressively setting variables to 0, DROPS). To control the underlying search process, we use a memory structure that is updated at locally optimal solutions called critical events. A parameter span is used to indicate the amplitude of oscillation about a critical event. We begin with span equal to 1 and gradually increase it to some limiting value. For each value of span, a series of alternating constructive and destructive phases is executed before progressing to the next value. At the limiting point, we begin to gradually decrease span, allowing again for a series of alternating constructive and destructive phases. When span reaches the value of 1, a complete span cycle has been completed and the next cycle is launched. Information stored at critical events is used to influence the search process by penalizing adds (during constructive phases) and favoring drops (during destructive phases) for variables that have been in recent critical solutions at a level of one. Cumulative critical event information is also used to introduce a subtle long term bias into the search process. A complete description of the approach is given in the references sited.
References [1] Bazarra, M. and J. Goode, "A Cutting-Plane Algorithm for the Quadratic Set Covering Problem," OR, Vol.23, no. 1, Jan-Feb (1975), pp.150-158. [2] Billionnet, A., and Calmels, A./'Linear Programming for the 0-1 Quadratic Knapsack Problem," EJOR, 92 (1996),310-325. [3] Chardaire, P., and Sutter, A., "A Decomposition Method for Quadratic Zero-One Programming," Management Science, 41(1995),704-712. [4] Gallo, G., Hammer, P., and Simeone,B., " Quadratic Knapsack Problems," Mathematical Programming, 12(1980),132-149. [5] Glover, F., Kochenberger, G., and Alidaee, B., "Adaptive Memory Tabu Search for Binary Quadratic Programs," Management Science, 44(1998), 336-345.
121 [6] Glover, F., Kochenberger,G., Alidaee, B., and, Amini, M., "Tabu Search With Critical Event Memory: An Enhanced Application for Binary Quadratic Programs," (1997), 2nd International Conference on MetaHeuristics. [7] Hammer, P., E. Boros, and X. Sun, "On Quadratic Unconstrained Binary Optimization," paper given at the Seattle INFORMS Meeting, Fall 1998. [8] Hammer, P., and Rader, D., "Efficient Methods for Solving quadratic 0-1 Knapsack Problems," INFOR, 35(1997),170-182 [9] Hammer, P., and Rudeanu, S., Boolean Methods in OR and Related Areas, Springer-Verlag, New York, (1968). [10] Hansen, P., "Methods of Nonlinear 0-1 Programming," Annals of Discrete Mathematics, 5(1079), 53-70. [11] Hansen, P., Jaumard,F., and Mathon,V., "Constrained Nonlinear 0-1 Programming," ORSA Journal on Computing, 5(1993), 97-119. [12] Kochenberger, G., Alidaee, B., and Amini, M., "Applications of the Unconstrained Binary Quadratic Program," Working Paper, University of Colorado at Denver, (1998) [13] Lodi, A., Allemand, K., and Liebling, T., "An Evolutionary Heuristic for Quadratic 0-1 Programming," EJOR (to appear). [14] Michelon, P., and Veilleau, L., "Lagrangean Methods for the 0-1 Quadratic Knapsack Problem," EJOR, 92 (1996), 326-341. [15] Pardalos, P., and Rodgers, G., "Computational Aspects of a Branch and Bound Algorithm for Quadratic zero-One Programming," Computing, 45(1990), 131144. [16] Sinclair, M., "An Exact Penalty Function Approach for Nonlinear Integer Programming Problems," EJOR, 27(1986), 50-56.
This page is intentionally left blank
Combinatorial and Global Optimization, pp. 123-132 P.M. Pardalos, A. Migdalas and R. Burkard, Editors © 2002 World Scientific Publishing Co.
Global optimization using dynamic search trajectories Albert A. Groenwold (Albert.GroenwoldOeng.up.ac.za) De.pt. of Mechanical and Aeronautical Engineering, University of Pretoria, Pretoria, 0002, South Africa.
J.A. Snyman (Jan. SnymanQeng. up. a c . za) Dept. of Mechanical and Aeronautical Engineering, University of Pretoria, Pretoria, 0002, South Africa.
Abstract Two global optimization algorithms are presented. Both algorithms attempt to minimize an unconstrained objective function through the modeling of dynamic search trajectories. The first, namely the Snyman-Fatti algorithm, originated in the 1980's and still appears an effective global optimization algorithm. The second algorithm is currently under development, and is denoted the modified bouncing ball algorithm. For both algorithms, the search trajectories are modified to increase the likelihood of convergence to a low local minimum. Numerical results illustrate the effectiveness of both algorithms. Keywords: Global optimization, Dynamic search trajectories.
124
1
Introduction
The problem of globally optimizing a real valued function is inherently intractable (unless hard restrictions are imposed on the objective function) in that no practically useful characterization of the global optimum is available. Indeed the problem of determining an accurate estimate of the global optimum is mathematically ill-posed in the sense that very similar objective functions may have global optima very distant from each other [1]. Nevertheless, the need in practice to find a relative low local minimum has resulted in considerable research over the last decade to develop algorithms that attempt to find such a low minimum, e.g. see [2]. The general global optimization problem may be formulated as follows. Given a real valued objective function f(x) defined on the set x € D in M n , find the point x* and the corresponding function value /* such that /* = f{x") = minimum {/(x)|x e D}
(1)
if such a point x* exists. If the objective function and/or the feasible domain D are non-convex, then there may be many local minima which are not global. If D corresponds to all 1R" the optimization problem is unconstrained. Alternatively, simple bounds may be imposed, with D now corresponding to the hyper box (or domain or region of interest) defined by D = {x\£ < x < u)
(2)
where £ and u are n-vectors defining the respective lower and upper bounds on x. From a mathematical point of view, Problem (1) is essentially unsolvable, due to a lack of mathematical conditions characterizing the global optimum, as opposed to the local optimum of a smooth continuous function, which is characterized by the behavior of the problem function (Hessians and gradients) at the minimum [3] (viz. the Karush-Kuhn-Tucker conditions). Therefore, the global optimum /* can only be obtained by an exhaustive search, except if the objective function satisfies certain subsidiary conditions [4], which mostly are of limited practical use [5]. Typically, the conditions are that / should satisfy a Lipschitz condition with known constant L and that the search area is bounded, e.g. for all x, x € X \f(x)-f(x)\
< L\\x-x\\
(3)
So called space-covering deterministic techniques have been developed [6] under these special conditions. These techniques are expensive, and due to the need to know L, of limited practical use. Global optimization algorithms are divided into two major classes: deterministic and stochastic 1 [6]. Deterministic methods can be used to determine the global optimum 'Prom the Greek word stokhastikos, i.e. 'governed by the laws of probability'.
125 through exhaustive search. These methods are typically extremely expensive. With the introduction of a stochastic element into deterministic algorithms, the deterministic guarantee that the global optimum can be found is relaxed into a confidence measure. Stochastic methods can be used to assess the probability of having obtained the global minimum. Stochastic ideas are mostly used for the development of stopping criteria, or to approximate the regions of attraction as used by some methods [3]. The stochastic algorithms presented herein, namely the Snyman-Fatti algorithm and the modified bouncing ball algorithm, both depend on dynamic search trajectories to minimize the objective function. The respective trajectories, namely the motion of a particle of unit mass in an n-dimensional conservative force field, and the trajectory of a projectile in a conservative gravitational field, are modified to increase the likelihood of convergence to a low local minimum.
2
The Snyman-Fatti trajectory method
The essentials of the original SF algorithm [5] using dynamic search trajectories for unconstrained global minimization will now be discussed. The algorithm is based on the local algorithms presented in [7, 8]. For more details concerning the motivation of the method, its detailed construction, convergence theorems, computational aspects and some of the more obscure heuristics employed, the reader is referred to the original paper.
2.1 Dynamic trajectories In the SF algorithm successive sample points x^j = 1,2,..., are selected at random from the box D defined by (2). For each sample point x*, a sequence of trajectories T*, i = 1,2,..., is computed by numerically solving the successive initial value problems: x(t) = -Vf(x(t)) (4) x(Q) = xl0 ; x(0) = x'Q This trajectory represents the motion of a particle of unit mass in an n-dimensional conservative force field, where the function to be minimized represents the potential energy. Trajectory Tl is terminated when x(t) reaches a point where f(x(t)) is arbitrarily close to the value f(x'0) while moving "uphill", or more precisely, if x(t) satisfies the
126 conditions f(x(t)) and
> / ( x j ) - eu
x(t)TVf(x(t))
(5) >0
where eu is an arbitrary small prescribed positive value. An argument is presented in [5] to show that when the level set {x\f(x) < f(xl0)} is bounded and V / ( X Q ) ¥" 0, then conditions (5) above will be satisfied at some finite point in time. Each computed step along trajectory T1 is monitored so that at termination the point x'm at which the minimum value was achieved is recorded together with the associated velocity xlm and function value fa. The values of x%m and x'm are used to determine the initial values for the next trajectory T'+l. From a comparison of the minimum values the best point x\, for the current j over all trajectories to date is also recorded. In more detail the minimization procedure for a given sample point x J , in computing the sequence x\, i = 1,2,..., is as follows.
2.2
Minimization procedure MP1
1. For given sample point xj, set x\ := xj and compute T 1 subject to x\ := 0 ; record xlm, xlm and fa ; set x\ := xxm and i := 2, 2. compute trajectory T* with x'0 := \ (x'f1 + sc^M and x'0 := j i ^ 1 , record a4,a4and/4, 3. if fa < }{x\-1)
then x\ := <
; else x\ := x j " 1 ,
4. set i:— i + 1 and go to 2. In the original paper [5] an argument is presented to indicate that under normal conditions on the continuity of / and its derivatives, x\ will converge to a local minimum. Procedure MP1, for a given j , is accordingly terminated at step 3 above if | | V / ( x j ) | | < e, for some small prescribed positive value e, and x\ is taken as the local minimizer x^, i.e. set x^ := x\ with corresponding function value fj := f{x3,). Reflecting on the overall approach outlined above, involving the computation of energy conserving trajectories and the minimization procedure, it should be evident that, in the presence of many local minima, the probability of convergence to a relative low local minimum is increased. This one is expected because, with a small value of tu (see conditions (4)), it is likely that the particle will move through a trough associated with a relative high local minimum, and move over a ridge to record a lower function value at a point beyond. Since we assume that the level set associated
127 with the starting point function is bounded, termination of the search trajectory will occur as the particle eventually moves to a region of higher function values.
3
The modified bouncing ball trajectory method
The essentials of the modified bouncing ball algorithm using dynamic search trajectories for unconstrained global minimization are now presented. The algorithm is in an experimental stage, and details concerning the motivation of the method, its detailed construction, and computational aspects will be presented in future.
3.1
Dynamic trajectories
In the MBB algorithm successive sample points x*,j = 1,2,..., are selected at random from the box D defined by (2). For each sample point asJ', a sequence of trajectory steps Ax' and associated projection points xl+l, i = 1,2,..., are computed from the successive analytical relationships (with x1 := x' and prescribed Vo, > 0):
A*' = IV.cosfliV/^/liy/V)
(6)
where tan-1(||V/(*i)||) + ^ ,
U
1
V0i smOi + {(V0i shift) 2 + 9 L
(7) 1/2
2gh(xi)}
(8)
fc(x') = /(«') + *
(9)
with k a constant chosen such that h(x) > 0 V a; £ D, g & positive constant, and xi+1 = x* + Ax1
(10)
For the next step, select Voi+1 < Vor Each step Ax' represents the ground or horizontal displacement obtained by projecting a particle in a vertical gravitational field (constant g) at an elevation ^(a:1) and speed Voi at an inclination ft. The angle ft represents the angle that the outward normal n to the hypersurface represented by y = h(x) makes, at x' in n + 1 dimensional space, with the horizontal. The time of flight ti is the time taken to reach the ground corresponding to y = 0. More formally, the minimization trajectory for o given sample point x' and some initial prescribed speed V0 is obtained by computing the sequence x\ i = 1,2,..., as follows.
128
3.2
Minimization procedure MP2
1. For given sample point x^, set x 1 := x J and compute trajectory step A x 1 according to (6) - (9) and subject to VQ1 := Vo; record x2 := x1 + Ax1, set i := 2 and V02 := aV0l (a < 1) 2. Compute Ax1 according to (6) - (9) to give x1+1 := x' + Ax\ set V0j+1 := aV0i
record x I + 1 and
3. set i := i + 1 and go to 2 In the vicinity of a local minimum x the sequence of projection points x\ i = 1,2,..., constituting the search trajectory for starting point xj will converge since Ax1 -¥ 0 (see (6)). In the presence of many local minima, the probability of convergence to a relative low local minimum is increased, since the kinetic energy can only decrease for a < 1. Procedure MP2, for a given j , is successfully terminated if | | V / ( x 2 ) | | < e for some small prescribed positive value e, or when aVJ < fiVg, and x 1 is taken as the local minimizer x^ with corresponding function value fj := /i(xj-) — k. Clearly, the condition aVg < fiV^ will always occur for 0 < /3 < a and 0 < a < 1.
4
Global stopping criterion
The above methods require a termination rule for deciding when to end the sampling and to take the current overall minimum function value / , i.e. / = minimum Ifj,
over all j to date }
(11)
as the global minimum value /*. Define the region of convergence of the dynamic methods for a local minimum x as the set of all points x which, used as starting points for the above procedures, converge to x. One may reasonably expect that in the case where the regions of attraction (for the usual gradient-descent methods, see [9]) of the local minima are more or less equal, that the region of convergence of the global minimum will be relatively increased. Let R/c denote the region of convergence for the above minimization procedures MP1 and MP2 of local minimum xk and let ak be the associated probability that a sample point be selected in Rk. The region of convergence and the associated probability for the global minimum x* are denoted by R* and a* respectively. The following basic
129 assumption, which is probably true for many functions of practical interest, is now made. A. Basic assumption: a* > a^ for all local minima x . The following theorem may be proved. B. Theorem (Ref. [5]): Let r be the number of sample points falling within the region of convergence of the current overall minimum / after n points have been sampled. Then under assumption A and a statistically non-informative prior distribution the probability that / corresponds to /* may be obtained from
^"•M'l-^
(I2)
On the basis of this theorem the stopping rule becomes: STOP when Pr I / = /* I > q*, where q* is some prescribed desired confidence level, typically chosen as 0.99.
5
Numerical results No. 1 2 3 4 5 6 7 8 9 10 11 12
Name Griewank Gl Griewank G2 Goldstein-Price Six-hump Camelback Shubert, Levi No. 4 Branin Rastrigin Hartman 3 Hartman 6 Shekel 5 Shekel 7 Shekel 10
ID Gl G2 GP C6 SH BR RA H3 H6 S5 S7 S10
n 2 10 2 2 2 2 2 3 6 4 4 4
Ref. [2,4] [2,4] [2, 10] [2, 11] [12] [2, 13] [2] [2, 10] [2, 10] [2, 10] [2, 10] [2, 10]
Table 1: The Test Functions The test functions used are tabulated in Table 1, and tabulated numerical results are presented in Tables 2 and 3. In the tables, the reported number of function values Nf are the average of 10 independent (random) starts of each algorithm. Unless otherwise stated, the following settings were used in the SF algorithm (see [5]): 7 = 2.0, a = 0.95, e = 10" 2 , w = 10~2, 5 = 0.0, q* = 0.99, and At = 1.0. For
130
No. 1 2 3 4 5 6 7 8 9 10 11 12
SF-- This Study ID N, (r/n)b (r/n)w 4199 6/75 Gl 6/40 G2 25969 6/84 6/312 2092 5/12 GP 4/4 C6 426 4/4 5/9 6/104 8491 6/29 SH 3922 5/12 BR 4/4 4799 RA 6/67 6/117 H3 933 4/4 5/8 1025 4/4 H6 5/10 6/24 1009 S5 4/4 S7 1057 6/37 5/8 6/31 SlO 845 4/4
Ref. [5]
Nf
r/h
1606 6/20 26076 6/60
668 263 — — — 563 871
4/4 4/4 — — — 5/6 5/8
1236 6/17 1210 6/17 1365 6/20
MBB (r/h)b (r/h)w 2629 6/23 5/8 Nf
19817
6/24
592 213
4/4 4/4 5/7 4/4 4/4 5/9 4/4 5/8
1057
286 1873
973 499 2114 2129 1623
6/16
5/7
6/69 5/10 5/10 6/26
5/6 6/42 6/29
5/9 6/39 6/47 6/39
Table 2: Numerical Results
Method TRUST MBB
Test Function C6 G P R A SH 55 31 103 59 72 25 29 74 168 171
BR
H3 58 24
Table 3: Cost (Nf) using a priori Stopping Condition
the MBB algorithm, a = 0.99, e = 10~ 4 , and q* = 0.99 were used. For each problem, the initial velocity VQ was chosen such that Aa; 1 was equal to half the 'radius' of the domain D. A local search strategy was implemented with varying a in the vicinity of local minima. In Table 2, (r/h)b and (r/h)w respectively indicate the best and worst r/h ratios (see equation (12)), observed during 10 independent optimization runs of both algorithms. The SF results compare well with the previously published results by Snyman and Fatti, who reported values for a single run only. For the Shubert, Branin and Rastrigin functions, the MBB algorithm is superior to the SF algorithm. For the Shekel functions (S5, S7 and SlO), the SF algorithm is superior. As a result of the stopping criterion (12), the SF and MBB algorithms found the global optimum between 4 and 6 times for each problem. The results for the trying Griewank functions (Table 2) are encouraging. G l has some 500 local minima in the region of interest, and G2 several thousand. The values used
131 for the parameters are as specified, with At = 5.0 for G l and G2 in the SF-algorithm. It appears that both the SF and MBB algorithms are highly effective for problems with a large number of local minima in D, and problems with a large number of design variables. In Table 3 the MBB algorithm is compared with the recently published deterministic TRUST algorithm [14]. Since the TRUST algorithm was terminated when the global approximation was within a specified tolerance of the (known) global optimum, a similar criterion was used for the MBB algorithm. The table reveals that the two algorithms compare well. Note however that the highest dimension of the test problems used in [14] is 3. It is unclear if the deterministic TRUST algorithm will perform well for problems of large dimension, or problems with a large number of local minima in D.
6
Conclusions
Two stochastic global optimization methods based on dynamic search trajectories are presented. The algorithms are the Snyman-Fatti trajectory method and the modified bouncing ball trajectory method. Numerical results indicate that both algorithms are effective in finding the global optimum efficiently. In particular, the results for the trying Griewank functions are encouraging. Both algorithms appear effective for problems with a large number of local minima in the domain, and problems with a large number of design variables. A salient feature of the algorithms is the availability of an apparently effective global stopping criterion.
References [1] Schoen, F. (1991), "Stochastic Techniques for Global Optimization: A Survey of Recent Advances," J. Global Optim., 1, 207-228. [2] Torn, A. and Zilinskas, A. (1989), Global Optimization: Lecture Notes in Computer Science, 350, Springer-Verlag, Berlin. [3] Arora, J.S., El-wakeil, O.A., Chahande, A.I. and Hsieh, C.C. (1995), "Global Optimization Methods for Engineering Applications: A Review," Preprint. [4] Griewank, A.O. (1981), "Generalized Descent for Global Optimization," J. Optim. Theory Appl., 34, 11-39. [5] Snyman, J.A. and Fatti, L.P. (1987), "A Multi-start Global Minimization Algorithm with Dynamic Search Trajectories," J. Optim. Theory Appl., 54, 121-141.
132 [6] Dixon, L.C.W., Gomulka, J. and Szego, G.P. (1975), "Towards a Global Optimization Technique", Towards Global Optimization, (Dixon, L.C.W. and Szego, G.P., Eds.), North-Holland, Amsterdam, 29-54. [7] Snyman, J.A. (1982), "A New and Dynamic Method for Unconstrained Minimization," Appl. Math. Modeling, 6, 449-462. [8] Snyman, J.A. (1983), "An Improved Version of the Original Leap-Frog Dynamic Method for Unconstrained Minimization: LFOPl(b)," Appl. Math. Modeling, 7, 216-218. [9] Dixon, L.C.W., Gomulka, J. and Hersom, S.E. (1976), "Reflections on the Global Optimization Problem," Optimization in Action, (Dixon, L.C.W., Ed.), Academic Press, London, 398-435. [10] Dixon, L.C.W. and Szego, G.P. (1978), "The Global Optimization Problem: An Introduction," Towards Global Optimization 2, (Dixon, L.C.W. and Szego, G.P., Eds.), North-Holland, Amsterdam, 1-15. [11] Branin, F.H. (1972), "Widely Used Convergent Methods for Finding Multiple Solutions of Simultaneous Equations," IBM J. Research Develop., 504-522. [12] Lucidi, S. and Piccioni, M. (1989), "Random Tunneling by means of Acceptancerejection Sampling for Global Optimization," J. Opt. Theory Appl., 62, 255-277. [13] Branin, F.H. and Hoo, S.K. (1972), "A Method for Finding Multiple Extrema of a Function of n Variables," Numerical Methods of Nonlinear Optimization, (Lootsma, F.A., Ed.), Academic Press, London, 231-237. [14] Barhen, J., Protopopescu, V. and Reister, D. (16 May 1997), "TRUST: A Deterministic Algorithm for Global Optimization," Science, 276, 1094-1097.
Combinatorial and Global Optimization, pp. 133-144 P.M. Pardalos, A. Migdalas and R. Burkard, Editors © 2002 World Scientific Publishing Co.
On Pareto Efficiency. A General Constructive Existence Principle G. Isac ( isac-gQrmc . ca) Department of Mathematics and Computer Science Royal Military College of Canada PO. Box 17000 STN Forces Kingston, Ontario, K7K 7B4 Canada
Abstract We present in this paper a general constructive existence principle for Pareto efficiency. This principle is presented in sequentially complete locally convex spaces. In the second part of the paper we present several realizations of this general principle in some particular cases. This paper can be considered as a starting point for new investigations. Keywords: Constructive existence principle and Pareto efficiency.
1
Introduction
The importance of Pareto efficiency, in vector optimization theory, with respect to a cone, is well known. An important problem in vector optimization is the identification of efficient points of a set. Consider the general case, i.e., the case of infinite dimensional vector spaces. Let F and E be topological vector spaces. Suppose that E is ordered by a closed pointed
134
convex cone K C E. Given a non-empty subset X C F and a function f : X —t E, consider the vector optimization problem j maximize \ x €X
f(x)
,..., [ '
For problem (1), we are interested in finding all solutions that are efficient, i.e., the elements of the set
Xe = eff(f(X),K)
=
{x0£X\f(x)-f(x0)£K,
for some x £ X implies f(x) = /(xo)} We can show that x0 € Xe if and only if f(X)
n (K + f(x0))
= {/(*„)}
(2)
We have that /(zo) is a K-support point for f(X) in the sense of Bishop and Phelps [4]. There also exist two important variants of efficiency. For any set A C E we denote by cone(A) = [JtA and by cZ[cone(A)] the closure of cone(A). A point x0 is said to be t>o
a properly efficient solution of (1) if XQ € Xe and cl [cone (f(X) — K — {f(xQ)})]nK {0}.
=
If we denote by Xpe the set of all properly efficient solutions for (1) we have that Xpe C Xe. For more details about the properly efficiency the reader is referred to [1], [2], [3], [15], [16] and [20]. If E* is the topological dual of E, we denote by
K* = {4>e E*\4>(k) > o Vifc e K \ { o } } . Suppose that K is such that K " is non-empty. We say that xo € X is a positive proper efficient solution of (1) if there exists <j> G K " such that <j>(f(x0)) >
135
2
Preliminaries
Let E{T) be a topological vector space. We say that a subset K C E is a pointed convex cone if the following properties are satisfied: (kl)
A K C K for all A G R+,
(k2)
K + KCK,
(k3)
K n ( - K ) = {0}.
The topological dual of E will be denoted by E*. By definition, the dual cone of K is K* = {y G E*\{x, y) > 0 for all x G K } where (,) is a duality between E and E*. We say that the cone K is well based if there exists a bounded convex set B such that 0 0 B and K = (J XB. We denote by B the topological closure of B. If the cone A>0
K is well based we can show that K has a bounded base, i.e., there exists a bounded convex set B j C K such that for every x G K \ { 0 } , there exists a unique real number \ x > 0 and a unique element bx G B0 such that x = \xbx. We say that E(T) is a locally convex space if the topology r is defined by a sufficient family of seminorms {P*}A<EA> * e ' t n e family {PAIAGA satisfies the following properties: (si) for any x G E, x ^ 0, there exists A G A such that p\{x) ^ 0, and (s2) for any Ai, A2 G A there exists A3 G A such that p^(x),p\2(x)
for all i e £ .
If {E(T), {P\}\EA) is a locally convex space then a closed pointed convex cone is well based, if and only if, there exists, / G K* such that for each A G A there exists a real number 5\ > 0 such that 6\p\(x) < f(x) for all i £ K . We say that the cone K is solid if its topological interior is non-empty. A non-empty subset D of E is said to be K-bounded if D C y - K for some y G E. Denote by < the ordering defined by K, i.e., x < y if and only if y — x G K. If A is a non-empty subset of E, we say that an element x0 G A is a K- efficient point (or a K-conical support point) if and only if A n (K + x0) = {x0} . Let D C E be a non-empty subset. We say that a set-valued mapping V : D —> 2D is a generalized dynamical system if for each x £ D, T(x) is non-empty. A point xt G D is said to be a critical point for r if T(a:») = {a;.}. We observe that x, G D is a K-emcient point if and only if x, is a critical point for the generalized dynamical system T(x) = (x + K) n D, x G D. We say that a locally convex space {E(T), Cauchy sequence in E is convergent.
{PA}ASA) is sequentially complete if every
136
3
The main result
The main result in this paper is the following general constructive existence test for K-efficient points, for a non-empty closed subset. Theorem 1 Let (E(T), {PA}AGA) be a sequentially complete locally convex space, K c E a pointed, closed convex cone and D C E a non-empty closed subset. Consider the generalized dynamical system T : D —¥ 2D defined by T(x) = (x + K) n D. Given XQ G D, if there exists a sequence {xn}^=0 C D such that xn+\ G r(a;„) for all n = 0, 2, 3 , . . ..and lim^.^, 6\(r(xn)) = 0 for all A e A, where 6X{A) = sup{px(x - y)\x,y G A}, OO
then, D has a ~K-efficient point xt such that x0 < xt. Moreover, P | r ( x n ) = {i*} and n=0
nmn_^00 xn = x^. Proof First we remark that r ( x n + 1 ) C T(xn), for all n = 0, 2, 3 , . . . . Indeed, if x G F(xn+i) = (x„+i + K) fl £>, then x = xn+i + k\, with k\ G K and x G D. Since xn^.\ G T(x n ), there exists ^ ^ K such that xn+i = xn + k2. Thus, we have x G D and x = xn + k\ + k% G xn + K, that is x G T(xn). Therefore { r ( x n ) } ^ _ 0 is decreasing. The sequence {xn}^=0 is a Cauchy sequence. Indeed, since limn_too<SA(r(a:n)) = 0 for all A G A, we have that for each e > 0 and A G A, there exists n€ G N such that sup p\(x — y) < e, for all n > n€. x,yer(xn) Because xm G T(xm-i) C T(xn) for all m,n G N such that m > n, P\{xm — Xn) < e, whenever m > n > nt. This implies that {xn}^L0 sequence in E(T). By the sequential completeness of E there exists a such that linin-jooXn = x„. Notice that xm G Y{xn) for all m > n,n
we have that is a Cauchy point x, G E = 0,1,2,....
oo
Hence we have xt G T(xn) for all n = 0 , 1 , 2 , . . . , i.e., x, G P | T(xn).
(We also used the
71=0
fact that T(xn) is closed for each n = 0,1, 2,....) The element x, is unique. Indeed, oo
assume that there exists w„ G f] T(xn) with u„ ^ x*. Since the family of seminorms n=0
{PA}A<EA is sufficient, we have that p\{x) = 0 for all A G A implies x = 0. Hence, because u* — x, ^ 0, there exists some A G A such that px(u* — x*) = a > 0. Since lim n ^ 0 0 5A(r(x n )) = 0 for all A G A there exists n 0 G N such that 6\(r(xno)) < a. In view of x*,u, G Y(xno) we have PA(M» - xt)5x{r(xno))
< px(ut
-
xt),
137 oo
which is a contradiction. Therefore we have that f] V(x) = {x,} . We must show n=0
that r(xt) — {x„}. Indeed, suppose that x E T(xt). We have x = x* + k, with k G K, and x 6 D. Since x„ G r(a;„) for all n = 0,1, 2 , . . . , we deduce that x„ — xn 6 K for all n = 0,1, 2 , . . . , which implies that x = x, + k = xn + k + ki G T(xn) for all oo
n — 0 , 1 , 2 , . . . , (where &i G K). It follows, x G P) r(a;„) = {a;,} . Evidently, we have n=0
x — x*. Since in particular xt € XQ + K, we have XQ < x„ and the proof is complete. • To apply Theorem 1 to particular problems, the art is the ability to construct the sequence with the properties indicated in Theorem 1.
4
Realizations of Theorem 1
We preset now in this section some realizations of the general principle expressed by Theorem 1. Let (E, || ||) be a Banach space and K C E a solid, closed pointed convex cone. For any ko 6 Int(K) and A E]0, 1[ we can define the set K(A, ko) = {x € E\x — A||x||fe0 € K}. We can show that K(A, ko) is a closed pointed convex cone. Because, for every x G K(A, k0) there exists k € K such that x—A||x||A;o = k, it follows that x = A||a;||fco € K, i. e., K(A, k0) C K. The cone K(A, k0) was defined and used in [14]. The next result, proved by A. Gopfert and Chr. Tammer in [14], can be considered in particular as a realization of Theorem 1. Denote by
Let (E, || ||) be a Banach space, K C E a solid, closed pointed convex € Int(K) and A G]0,1[. If D C E is an arbitrary closed K.-bounded any XQ £ D there exists x, 6 Eff(Z3; K(A, k0)) such that x0
Proof We consider on D the generalized dynamical system F(x) = (K(A; k0) + x)nD, for all x e D. Let the sequence be obtained by the construction proposed by Gopfert and Tammer in [14], i.e., (i) Xi = x0 (ii) if defined we choose xn+\ G T(xn) such that x G T(xn) does not exist with x G xn+i + ^nk0 + (K\{0}).
138 In [14] it is proved that this sequence is well defined and for every n € N, T(a;r,+i) C r(xn). If x e r ( x „ + i ) is an arbitrary element we have that x € xn+i +K(A, k0), which implies that ( i — xn+i) — X\\x — a;n+i||A;o G K. We have x
€
x„ + i + A||;r-:r n+1 ||fco + K
(3)
Since r(ai„ + i) C T(xn), we have that x e T(xn) and considering (ii) we deduce x
$ xn+1 + - ^ — k0 + ( K \ { 0 » n+ 1
(4)
Hence, taking into account (4) we have X\\x — xn+i\\ < ^ , which implies that 5(T(xn+i)) < w 2 + 1 ) , where <5(r(a;n+i)) = diam{T(xn+i)). Applying Theorem 1 the theorem is proved. • We will show now that a result similar to Theorem 2 follows from a more general construction, certainly, modulo another definition of the sequence {xn}neN . T h e o r e m 3 Let (E,\\ ||) be a Banach space, e a strictly positive real number and K C E a closed pointed convex cone such that K C {x € E\^(x) + e\\x\\ < 0}, where "J : E —> R is a subadditive continuous mapping. Let D be a non-empty closed subset of E. Then, for every xa € D such that $> is bounded from below on (xa + K)DD, there exists x* e Eff(D;K) such that xa
€.T(xn).
If (i) is the case, we have that xn = xt. Suppose that (ii) is true. Since K C {x 6 E\ib(x) + e\\x\\ < 0} we deduce from (ii) and using the fact that * is subadditive, tf (a;) - * ( x n ) < *(a; -
xn),
•${x) - *(a; n ) + e||ar - xn\\ < * ( i - xn) + e\\x - xn\\ < 0, and
*(a; n ).
(5)
139 Thus, *(xn) -
inf
*(j/) > * ( i n ) - *(x) > 0.
We pick a point xn+\ G T(a;n) such that *(xn+1)
<
inf
*(!/) + - * ( i B ) -
"gr(3:„)
inf
Z
*(i/)
(6)
v€T(x„)
and we have T(xn+i) = (x n + 1 + K) n D, and so on. Since K is a convex cone we can show that T(xn+i) C r(a; n ) for all n = 0 , 1 , 2 , . . . . The sequence {r(z 7 l )}^ = 0 (if it is not stopped after a finite number of steps) is decreasing. From (6) we obtain 9(Xn+1)
-
inf *(!/) < *(Xn+1) ^6r(i„+i)
-
inf 9( ) < - * ( ! „ ) - inf mv) fsr(x„) V z ver(xn)
(7)
Let z G T(a;„), then applying (7) n-times we obtain *(*„)-tf(z)
<
#(z„)-
inf
*(i/)
^€r(x„)
< -
1 - *(i„_i)2
< -
1 —n * ( * « ) 2
*(i/) < A
inf "6r(i„_i)
inf
*(i/)
Since K C {x € £|\t(a;) + e||z|| < 0} and r(a;„) = (xn + K) n D, we have for any z G r ( x n ) , *(z) - W(xn) < # ( 2 - z„), which implies - # ( 2 - xn) < *(a;„) - ^ ( z ) and finally, |iB-z||
< M*(a;„)-*(«)] <
1 ¥ ( ! . ) - inf *(«/) e2n "/er(i0)
¥ ( i a ) - inf *(«/) which implies that <5(r(:rn)) -4 0 e2" as n —>• 00. The theorem is now a consequence of Theorem 1 and the proof is complete. • Therefore 5(T{xn))
<
In [11] J. D. Dauer and O. A. Saleh proved the following result. Theorem 4 Let E(T) be a topological vector space and K a solid pointed convex cone in E. If y € Int{K) then the mapping ^v(x) = inf{a G R|a; G ({-ay} + K ) } , for all x G E, is a continuous suhlinear functional on E which satisfies the following properties:
140 (1) Int(K)
= { i E E\
(2) K = {x(E E\Vy{x)
< 0},
(3) ^y is K-decreasing on E. Applying Theorem 4 we obtain the following variant of Theorem 3. Theorem 5 Let (E, \\ ||) be a Banach space, K C E a solid, closed pointed convex cone and \f „ the continuous sublinear mapping defined in Theorem 4- Let e be a strictly positive (eventually sufficiently small) real number. Consider the closed convex cone K.(^y) = {x £ E\^y(x) + e||a;|| < 0}. If D C E is an arbitrary K-bounded set, then for any xa € D there exists xt € Eff(Z); K(\E,!,)) such that xa < K ( * „ ) X„. Moreover xm is obtained as the limit of a sequence {xn}%L0 C D defined by the method given in Theorem 3. Proof We observe that K(\t ! / ) C K. Since D is K-bounded, we have that ^y is bounded from below on (xa + K($j / )) n D. (This is a consequence of property (3) of mapping tyy. The theorem is now a consequence of Theorem 3.) • Remark Because for a very small e > 0 the cone K(^ , !/ ) is, in some sense, very close to K, the K(3 , !/ )-emciency obtained by Theorem 5 is a kind of e-emciency. For e-efficiency the reader is referred to [19], [25] and [32], Now, we consider the case of locally convex spaces. Let [E(T), {px}xe\) D e a locally convex space as denned in this paper. If K c E is a well based pointed closed convex cone, then there exists / 6 K* with the property that for every A e A there exists ex > 0 such that K C {x € E\f(x) + EAPA(^) < 0}. We have the following result. Theorem 6 Let (E(T),{PX}X6A) be a sequentially complete locally convex space and K C E a closed pointed convex cone. Suppose that there exists a subadditive continuous mapping $ : E —> R with the property that for every A £ A there exists ex > 0 such that K C {x e E\V(x) + txPx(x) < 0}. Let D be a non-empty closed subset of E. Then, for every xa £ D such that \f is bounded from below on (xa + K ) n D , there exists x„ G Eff(£); K) such that xa < K 1 , . Moreover, i» is obtained as the limit of a sequence {xn}n£ff C D, well defined. Proof We consider on D the generalized dynamical system r(a;) = (x + K)flD, x £ D and we define a sequence {xn}^=0 C D inductively as in the proof of Theorem 3. We take x0 = xa. Suppose that xn £ D is defined and r(xfc +1 ) C T{xk) for all k = 0 , 1 , . . . , n — 1. We have two possibilities:
141 (i) either T(xn) = (xn + K) n D = {xn} or (ii) there exists x ^ xn with x e
T(xn).
If (i) is the case, we have that xn = x*. Suppose that (ii) is true. Because the family of seminorms {p\}\eh is sufficient, there exists Ao € A such that p\0(x — xn) / 0 (i.e., P\a{x — xn) > 0). Because ty is subadditive we have $(x) - tf (z„) + cXoPx0(x - xn) < m(x - xn) + eXo{x - xn) < 0, which implies * ( i ) < * ( x n ) - ex0Px0(x - xn) < * ( x n ) . Thus, V(xn) -
inf
V(v) > V(xn) - V(x) > 0.
We pick a point xn+i € T(xn) such that *(xn+i)<
inf * ( » + - * ( * „ ) ^er(o:n) ^
inf
*M
^6r(x„)
We have F(a; n+ i) = (xn+i + K) n Z? and r ( x n + 1 ) C T(xn) for all n = 0,1, 2, It is evident that *0»+i)-
inf
*(i/)
- *(iB)-
<
i/er(x„+i)
/
inf *(i/) "er(i n )
(8)
Let a; G T(a; n ). Applying (8) n-times we obtain * ( ! „ ) - * ( ! )
<
^
tffxJ
-
inf
VM
ver(x„)
Since, for every A £ A we have K
C
{x € E\V(x)
+ £APA(X) < 0}
we can show as in the proof of Theorem 3 that px{xn - x)
<
1 ex2n
*(*„)-
inf
Mu)
v£T{xa)
which implies that Sx(T{xn))
<
e A 2"
^er(a:0)
We have that <5A(r(£n)) -> 0 as n -> oo for all A e A. The theorem is now a consequence of Theorem l.D
142 Corollary 7 Let {E(T), {P\}\£A) be a sequentially complete locally convex space and K C E a well based closed pointed convex cone. Let D be a non-empty closed subset of E. Then, for every xa £ D such that (xa + K) n D is bounded, there exists x, € Eff(£);K) such that xa
References [1] BENSON, H. P.: The vector maximization problem: proper efficiency and stability, SIAM J. Appl. Math. 32 (1977), 64-72. [2] BENSON, H. P.: An improved definition of proper efficiency for vector minimization with respect to cones, J. Math. Anal. Appl. 79 (1979), 232-241. [3] BENSON, H. P.: Efficiency and proper efficiency in vector maximization with respect to cones, J. Math. Anal. Appl. 93 (1983), 273-289. [4] BISHOP, E. and PHELPS, R. R.: The support functionals of a convex set, In.: Proc. Symp. Pure Math., Amer. Math. Soc, Providence, R. I. (1962), 27-35. [5] BITRAN, G. R. and MAGNANTI, T. L.: The structure of admissible points with respect to cone dominance, J. Optim. Theory Appl. 29 Nr. 4 (1978), 573-614. [6] BORWEIN, J. M : On the existence of Pareto efficient points, Math. Oper. Res. 8 Nr. 1 (1983), 64-73. [7] CESARY, L and SURYANARAYANA, M. B.: Existence theorems for Pareto optimization, multivalued and Banach space valued functionals, Trans. Amer. Math. Soc. 244 (1978), 37-65. [8] CHEW, K. L.: Maximal points with respect to cone dominance in Banach spaces and their existence, J. Optim Theory Appl. 44 Nr. 1 (1984), 1-53.
143 CORLEY, H. W.: An existence result for maximization with respect to cones, J. Optim. Theory Appl. 31 Nr. 2 (1980), 277-281. DAUER, J. P. and GALLAGHER, R. J.: Positive proper efficient points and related cone results in vector optimization theory, SI AM J. Control Optim. 28 Nr. 1 (1990), 158-172. DAUER,J. P. and SALEH, O. A.: A characterization of proper minimal points as solutions of sublinear optimization problems, J. Math. Anal. Appl. 178 (1993), 227-246. DAURER, J. P. and STADLER, W.: A survey of vector optimization in infinitedimensional spaces, Part II, J. Optim. Theory Appl, 51 (1986), 205-241. GOPFERT, A, and TAMMER, CHR.: A new maximal point theorem (To appear) . GOPFERT, A, and TAMMER, CHR. :e-Approximate solutions and conical support points. A new maximal point theorem, (Preprint-(1997)). HARTLEY R.: On cone-efficiency, cone-convexity and cone-compactness, SI AM J. Appl. Math., 34 Nr. 2 (1978), 211-222. HENIG, M. I.: Proper efficiency with respect to cones, J. Optim. Theory Appl. 36 (1982), 387-407. ISAC, G.: Sur l'existence de l'optimum de Pareto, Riv. Math. Univ. Parma (4) 9 (1983), 303-325. ISAC, G.: Pareto optimization in infinite dimensional spaces: the importance of nuclear cones, J. Math. Anal. Appl., 182, Nr. 2 (1994), 393-404. ISAC, G.: The Ekeland's principle and the Pareto e-efficiency, In: M. Tamiz (Ed.): Multi-Objective Programming and Goal Programming, Lect. Notes in Econom. Math. Systems Nr. 432, Springer-Verlag (1996), 148-162. ISAC, G. and POSTOLICA, V.: The Best Approximation and Optimization in Locally Convex Spaces, Verlag Peter Lang, Frankfurt (1993). JAHN, J.: Existence theorems in vector optimization, J. Optim. Theory Appl. 50 Nr. 3 (1986), 397-406. JAHN, J.: Mathematical Vector Optimization in Partially Ordered Linear Spaces, Peter Lang, Frankfurt. LUC, D. T.: An existence theorem in vector optimization, Math. Oper. Res. 14 Nr. 4 (1989), 693-699.
144 [24] LUC, D. T.: Theory of Vector Optimization, Lecture Notes in Economics and Math. Systems, Vol. 319, Springer-Verlag, New York, Berlin (1989). [25] NEMETH, A. B.: Between Pareto efficiency and Pareto e-efficiency, Optimization 20 Nr. 5 (1989), 615-637. [26] PENOT, J. P.: L'optimization la Pareto: Deux ou trois choses que je sais d'elle, Publ. Math. Univ. Pau (1978). [27] PERESSINI, A. L.: Ordered Topological Vector Spaces, Harper and Row New York, Evanston and London, (1967). [28] PHELPS, R. R.: Support cones in Banach spaces and their applications, Advances in Math. 13 (1974), 1-19. [29] POSTOLICA, V.: Some existence results concerning the efficient points in locally convex spaces, Babes-Bolyai Univ. Faculty of Math. Seminar on Math. Anal. (1987), 75-80. [30] STAIB, T.: On two generalizations of Pareto minimality, J. Optim. Theory Appl. 59 Nr. 2 (1988), 289-306. [31] STERNA-KARWAT, A.: On existence of cone-maximal points in real topological linear spaces, Israel J. Math. 54 Nr. 1 (1986), 33-41. [32] TAMMER, CHR.: Existence results and necessary conditions for e-efficient elements, In: B. Brosowski, J. Ester, S. Helding and R. Nehse (Eds.), Multicriteria Deci- sion Proc. 14th Meeting of the German Working Group "Mehrkriterielle Entsheidung" , Peter Lang, Frankfurt (1993), 97-110. [33] WAGNER, D. H.: Semi-compactness with respect to an euclidean cone, Canad. J. Math. 29 Nr. 1 (1977), 29-36. [34] YU, P. L.: Cone convexity, cone extreme points and nondominated solutions in decision problems with multiobjectives, J. Optim. Theory Appl. 14 (1974), 319-377.
Combinatorial and Global Optimization, pp. 145-160 P.M. Pardalos, A. Migdalas and R. Burkard, Editors © 2002 World Scientific Publishing Co.
Piecewise Linear Network Flow Problems Dukwon
Kim
([email protected])
Center for Applied Optimization Department of Industrial and Systems 303 Weil Hall, University of Florida Gainesville, FL 32611 USA
P a n o s M. P a r d a l o s
Engineering
(pardalosQufl.edu)
Center for Applied Optimization Department of Industrial and Systems 303 Weil Hall, University of Florida Gainesville, FL 32611 USA
Engineering
Abstract We consider the minimum cost network flow problem, and in particular the case with piecewise linear arc costs. We identify special subclasses of minimum cost network flow problems according to the type of piecewise cost functions, discuss their relationships, and present transformations and reformulations; The general piecewise linear minimum cost network flow problem (PLNFP) is formulated as a fixed charge network flow problem (FCNFP), and it is demonstrated that the indefinite, nonconvex, P L N F P is equivalent to a F C N F P with reduced feasible region. A major advantage of the formulations introduced here is that solutions can be found by solving fixed charge problems instead of solving difficult nonconvex optimization problems. K e y w o r d s : minimum concave cost network flow problem, piecewise linear minimum cost network flow problem, fixed charge network flow problem, minimum indefinite cost network flow problem, mixed integer formulations
146
1
Introduction
In this article, the minimum cost network flow problem [39] is considered with piecewise linear arc costs, so called the Piecewise Linear minimum cost Network Flow Problem (PLNFP). As a special subclass of minimum cost network flow problems, general piecewise linear network problems can be classified according to the type of piecewise cost functions. Using general network flow constraints, PLNFP can be stated as follows: Given a directed graph G = (N, A) consisting of a set N of m nodes and a set A of n arcs, then solve [PLNFP] fij(xa)
min f(x) = Y,
subject to
Y, x/a{k,i)eA
^2 x* = bi'
Vi € N
(!)
(i,k)eA
0 < x y < « y , V{i,j)EA
(2)
where / is separable and each / y is piecewise linear. For instance, the arc cost / y ^ y ) can be defined as follows:
c x
ij ij
Jij\xij)
"r siji
\j
_
x
ij
<
\j
— ' rT*>T.-4- «"^ ij lJ ij '
\Tii~l ij
< r < — l3 —
\Tii ij
where A*^ for k = 1 to r^ — 1 are breakpoints in the given interval [0, My]. The constraints in (1) are called the conservation of flow equations. The constraints in (2) are called capacity constraints on the arc flows. The problem is uncapacitated if ui:i = oo, V(i,j) e A.
147 Three specific classes of PLNFP are identified based on the arc costs, / y - as follows: 1. Convex PLNFP 2. Concave PLNFP 3. Indefinite PLNFP In some cases, the indefinite PLNFP is called discontinuous PLNFP, since it usually results from a set of discontinuities in the arc cost functions. Since the Fixed Charge Network Flow Problem (FCNFP) has a very close relation to the PLNFP, it is important to understand the special structure of FCNFP to solve PLNFP. Due to the global optimality property of concave minimization [32, 34], global solutions can be obtained at extreme points of feasible regions for the cases of FCNFP and concave PLNFP. A survey on minimum concave-cost network flow problems can be found in [15].
1.1
Convex P L N F P
Suppose the constraints are all linear, and the cost function to be minimized is separable and piecewise linear. Then the proportionality assumption is violated for the cost function. However, if the piecewise linear function is convex, the problem can still be modeled as an LP [28]. Let fij(xij) denote the contribution of Xy in the separable objective function. Suppose that there are r y — 1 breakpoints at which /y(xy) changes slope, such that
Let the slope in the subinterval A y _1 < aiy < Ay be c y for k = 1 to r y , and let yfj be the portion of xy lying in the kth subinterval, A y _1 to A y , (i.e., j/ t * is the length of the overlap of the interval 0 to x y with the subinterval Ay rl to Ay), k = 1 to r-y. When defined in this manner, the new variables yh,yfj, • • • ,Vij' partition a;y such that *tf = V y + t / y + ••• + «#'• (3) These variables are subject to the constraints: 0 <
Vh
< %j
0 <
4
< Ay- - Ay.
l
1
o < yT < xr - xr 0 <
2/1/ < A ^ - A ^ " 1
(4)
148 and for every t, if y\j > 0, then each of j/ y is equal to its upper bound
(5)
A*. - A*"1, V k < t. Defining the new variables as shown above, it is clear that fij(xij) is equal to cj.-tA + . .. + q!J yi". If the original separable piecewise linear objective function to be minimized is continuous and convex so that it holds the following increasing conditions
4 <4 <...<#,
(6)
constraints on the new variables of the type (5) can be ignored in the transformed model. Since convex PLNFP is reformulated as an LP using the above technique and has specially structured network constraints, it can be solved efficiently in polynomial time. If fij is not continuous or the slopes do not satisfy the condition (6), then the constraints (5) must be specifically included in the model. Since these constraints are not linear, the transformed model is no longer an LP.
1.2
FCNFP
Due to the similarity of its structure with the piecewise linear case, the Fixed Charge Network Flow Problem (FCNFP) has close relations to PLNFP. It is very important and useful to investigate some features of FCNFP even if FCNFP does not belong to the class of PLNFP. The FCNFP is a special case of Minimum Concave Cost Network Flow Problems (MCNFP) [32], whose arc cost function has a discontinuity at the origin. The arc cost function fij(xij) of the FCNFP has a form
f.(x-
)= I
%i
i = ° >0,
(7) y !
where s y > 0 is a fixed cost for arc (i,j) G A. In many practical problems, the cost of an activity is the sum of a fixed cost and a cost proportional to the level of the activity. FCNFP is obtained by imposing a fixed cost of s y > 0 if there is positive flow on arc (i,j) and a variable cost c v . Due to the discontinuity of / y , the problem can be transformed to a 0-1 mixed integer programming problem by introducing n binary variables, indicating whether the corresponding activity is being carried out or not. Assuming Sy > 0, /y- can be replaced by
149 Jij
CijX^j ~h SijVij
with
Xii
> 0 and !/« = J J ff *jj ~ J
(8)
The above condition (8) can be incorporated into the capacity constraints to yield 0 < Xij < UijVij, yij 6 {0,1}. Hence we obtain the following formulation of the fixed charge network flow problem: [FCNFPJMIP
min
Yl (c
subject to Mx = b
(9)
0 < x^ < Uijyij,
(i,j)€A
(10)
!/ye{0,i},
(iJ)eA,
(ll)
where M is an m x n node-arc incidence matrix and 6 is an m-dimensional column vector. This P C N F P M / P can be solved by using any type of classical branch and bound algorithms that use LP relaxations [30]. These LP relaxations can be solved efficiently by existing linear network algorithms exploiting the special structure of their feasible domain [18]. As we can see in later sections, many concave and indefinite PLNFPs can be reduced to a FCNFP model by introducing new variables and modifying problem structures. It is noticed that FCNFP models reduced from original PLNFPs can be also transformed to a 0-1 mixed integer programming problem. Consequently, the size of the resulting model grows very fast even if the original PLNFP is just of medium size. This stimulates the reason that many researchers have developed new efficient schemes to improve their exact methods (especially the branch and bound method). Indeed, the computational effort and memory requirement to solve large-scale FCNFP models have been gradually reduced in various application areas. Yet, since there is a limitation for improving exact solution methods to solve the problem in practical sense, developing an effective approximate method is still in need.
1.3
Concave P L N F P
As different from the convex case, concave PLNFPs are more difficult to solve since we cannot use the same technique in the convex case to reduce the problem into an LP.
150
V (= ««)
*«
Figure 1: Example of a concave piecewise linear arc cost function. However, a concave PLNFP can be transformed to a fixed charge network problem in an extended network. The size of the extended network depends on the number of linear pieces in each arc cost function. An Arc Separation Procedure (ASP) is required to solve the problem in this way and ASP can be valid due to the concavity of arc cost functions. Let us consider an arc (i,j) £ A and its arc cost / „ , and suppose JV, has r „ linear pieces as defined previously. Then arc (i,j) can be separated into rtj arcs between nodes i and j for (i,j) 6 A. Each separated arc (i, j)k for k = 1 to r^ has a fixed charge cost function ft (see Figure 1) defined by
Jij\*l,)
| 4+4-Iy
if
Xij
= 0 > 0.
This extended network is denoted by Ge(N, Ae) where the number of arcs \Ae\ is given by
ne = \Ae\ = J2 rnAfter the ASP modification shown in Figure 2, the original concave piecewise linear objective function can be expressed as a sum of fixed charge arc cost functions as follows:
/(*) =
E (i,3)eA
fa(xii)
151
fi'W
ffrtl)
S>
<
Figure 2: Arc separation procedure.
= £ E/«(*«) (i,j)eA fc=i
= E X K 4 + 4)-
(12)
It is easy to see that the equality in (12) can not be true in general cases without a set of constraints to restrict a domain for each separated arc cost function. However, due to the following property from the concavity of arc cost functions:
4 > 4 > ... > c$ > 0,
(13)
the equality holds true in this case. More precisely, fij(xij) is equal to at most one arc cost among all separated arc costs between node i and j at the optimality of minimization problems. This argument can be generalized as the following theorem.
Theorem 1 Given an extended network described above, if a positive flow a;*,- is optimal for a minimum concave PLNFP and if Xjf1 < X\A < Ay- for 1 < q < rij, then it takes only one arc, {i,j)q (i.e. q-th arc) among all separated arcs between node i and j , (i,j)k for k = 1 to rtj. Proof: The same notation as the above is used in the proof. Let us consider an arc (i,j) e A with r y pieces of concave piecewise linear cost function / y . And denote that c y a n < i s% a r e slope and fixed charge of the kth piece of linear function for k = 1,2, ...,r y -, respectively. Notice that these slopes hold the property in (13). After ASP, there are r y separated arcs the flow can take. So, let us denote that flowk is the optimal flow of the fcth separated arc (i, j)k for K — i, z , . . . , r^j.
152 Firstly, it is clear that if x*j is less than or equal to the first breakpoint Ay, then the optimal flow cannot be split (i.e. flow1 = a;*.) since the cost of arc (i, j)1, f}j < f£, V Xij < X\j for k = 2 , 3 , . . . , r „ . Secondly, consider the following two cases when A?- < A^"1 < x*j < Xfj, and suppose flowp and flow9 are optimal flows in the extended network problem. Case I : flowp — e and flow9 = x*j — e > 0 for any positive e. Using a contradiction, suppose that the optimal flow is split into two arcs (i,j)p and (hj)9j 1 < V < 1 < rtj- Now, let us evaluate the difference of cost function values in the original PLNFP with x^ and the extended network problem with flowp and flow9:
x x [fm+fU h-c)]-fv( w £ c x £
= [4 + 4 + 4 + U ij - )1 ~ (4 + cijx*ij) >
0.
The inequality in the last term above is true since 4 contradicts to the optimality of flowp and flow9. Case I I : flowp = e = x*j and flow9 = 0.
^
an
d 4
*"- 4 "
Thus, it
Again, using a contradiction, let us evaluate the difference of cost function value between node i and j :
[fm+fiM-uMi) =
\sij + Hje) ~ \Sij + cijxij) lSij ' Hj^ijl ~ \Sij "T" cijxij)
=
>
0.
With the same reason in Case I, we have a contradiction to the optimality of flowp and flow9. Therefore, flow9 = x^ and the proof is completed. D Based on the theorem above, the original concave PLNFP can be reduced to a FCNFP with the objective function given in (12) and some extended network constraints corresponding to the constraints in (1). The resulting formulation of a concave PLNFP is shown as follows: [PLNFP]FC
min /(*)= £ £ 4 ( 4 ) (ij)€A *=1
subject to
£
£ 4 -
(l,i)€Ak=l
£ 5>« = fc> vi 6 jv (i,l)eAk=l
(i4)
153
4>0, o < i y < Uy,
\/{i,j)£A, fc = l,2,...,r y V(i, j ) e A
In the formulation, it is clear that any other set of constraints is not necessary to specify lower and upper bounds for separated arcs {i,j)k £ Ae, V i,j G N due to the above theorem. As a result, the solution of concave PLNFP can be found by solving the fixed charge network problem formulated as above. Thus, developing an efficient algorithm to solve a FCNFP (exactly or approximately) is a key to solve PLNFPs in this approach.
1.4
Indefinite P L N F P
Lastly, we consider a PLNFP with indefinite arc cost functions, which is the most difficult case in this class. The major difficulty to find exact solutions for indefinite PLNFPs is originated from the structure of their arc cost function. Obviously such cost functions are neither convex nor concave, and possibly have a finite set of discontinuities. However, due to the nature of real world applications of the model, we focus on two certain types of arc cost function in indefinite PLNFP models, so called staircase [6, 17, 24] and sawtooth [24] arc cost functions, respectively. Both arc cost functions have a very similar structure in overall shape, however, they have a different aspect at breakpoints (see Figure 1 and 2). It can be described in mathematical form as follows: • "Staircase" arc cost function: fif1^'1)
< fiMif1 + £)>
for
any £ > 0 and fc = 2, 3 , . . . , r y .
• "Sawtooth" arc cost function: / ^ ( A * - 1 - e) > /'(Ay" 1 ), for any e > 0 and k = 2, 3 , . . . , r y . Moreover, it is assumed that the property of slopes shown in (13) is still valid since it is a very general phenomenon in real applications. Note that extreme point solutions are not guaranteed in this case since objective functions are no longer concave. Now, we introduce an equivalent MIP formulation for the problem in a FCNFP model with some additional parameters and binary variables. Let us define the size of interval between adjacent breakpoints as AA*. = A*. - A*ri, V ( U ) € A ,
k = 1, 2 , . . . , r y ,
(15)
154
A / ( = u,j)
Figure 3: Staircase arc cost function.
Figure 4: Sawtooth arc cost function.
*J
155 and define the gap of function values at each breakpoint in arc cost functions as
A 4 = (4A*r1 + 4)-(c^ 1 A* r 1 + s*r1) =
4 - 4 " 1 + (c* - c*ri)A*-\
V(i, j ) e A, Vfc,
(16)
where sy- = 0 and Cy = 0 (also clearly Ady = sy-)- We now let Xy be the part of Xy that lies within level k (i.e. fcth subinterval), in the following sense: (0 4
ifXy
= 1 ^y-^y"1 [ AA*.
ifAy-^XylA^. if xy > A*-,
(17)
and we obtain the following equation for substitution into [ F C N F P ] M I P model. We then introduce new binary variables defined by *
f 1 \ 0
^
if A*"1 < Xy otherwise.
( V
} ;
Using (15) - (18), the indefinite PLNFP under consideration can be formulated as an MIP version of FCNFP as follows: mhl
£
m $>yXy.+Ady4)
(ij)6-4 fc=l
subject to constraints in (1) and V
(M) V(t, j ) , k = 1, . . . , Ty
(19) (20)
xy-i > AAy- 1 ^., V ( i J ) , fc = 2 , . . . , ry
(21)
2y = E ^ l 4 , Xy < AAy»y, 4>°> »y 6 {0,1},
V(t,j), A = l , . . . , r y V(t,j), fc = l , . . . , r y .
(22) (23)
It is noticed that combining one constraint from (20) and one from (21) yields AAy- 1 !^ < x ^ < AAf- 1 ^- 1 , which implies y% < y*" 1 , V(i, j ) , fc > 1. There is another approach to formulate the problem as a concave Minimum Cost Network Flow Problem model. In [24], Lamar described an equivalent formulation of MCNFP with general nonlinear arc costs (including the problem considered in this section) as a concave MCNFP on an extended network. The equivalence between the problems is based on converting each arc with an arbitrary cost function in the original problem into an arc with a concave piecewise linear cost function in series with a set of parallel arcs, each with a linear arc cost function (see [24] for details). Thus, the resulting problem is a concave MCNFP, which is different from the FCNFP formulation model shown above.
156
2
Applications
Piecewise linear network models have a number of applications in various areas such as transportation problems, location problems, distribution problems, communication network design problems, and economic lot-sizing problems. Due to the structure of objective functions (especially in concave and indefinite PLNFP models), many realworld situations listed above can be modeled as PLNFP. There are major applications in the following two fields which have been studied extensively by many researchers.
2.1
Transportation Problems
The first major field is transportation-related problems with concave cost functions including fixed charge cost functions. The concave cost functions in this field are usually assumed to be piecewise linear in many cases. A number of algorithms developed with different schemes and their computational results have been reported. These can be found in a limited list of references [2, 3, 4, 7, 11, 16, 19, 20, 23, 25, 26, 31, 35, 36]. The category of exact solution approach contains diverse techniques based on extreme point ranking, branch and bound, and dynamic programming methods. Since the problems can be formulated as MIP models, branch and bound approach with various branching schemes has been a major interest in the literature. Murty [29] introduced an extreme point ranking method for solving fixed charge problems. McKeown [27] extended Murty's method to avoid some degeneracy of the problem. Recently, Pardalos [33] discussed a range of enumerative techniques based on vertex enumeration or extreme point ranking. Rech and Barton [35] investigated a nonconvex transportation algorithm using branch and bound approach to problems with piecewise linear cost functions. These functions are approximated by a convex envelope and solved using out-of-kilter method. Bornstein and Rust [6] specialized this approach to the problem with staircase cost function, using successive linearizations of the objective function. Thach [38] proposed a method for decomposing the problem with a staircase structure into a sequence of much smaller subproblems. Lamar [25] developed a branch and bound approach for cases of capacitated MCNFP in which the costs consist of piecewise linear segments. The problem is formulated as an MIP, with the branching variables determining which linear cost region an arc flow falls into. Recently, Kim and Pardalos [22] developed a heuristic procedure for solving general FCNFP without formulating it as an MIP. The procedure is consist of solving a series of LPs to update slopes and searching extreme points of the convex feasible region
157 with the updated slopes. This approach provides a potential possibility of parallel implementation with different initial solutions to improve the quality of solutions. Some heuristic approaches can be found in [8, 11, 20, 23].
2.2
Location Problems
Another major application area is to solve location problems. Since the problem in this field is to locate facilities and determine the size of facilities to minimize total cost [13], it naturally involves fixed costs and/or piecewise linear costs. Since solution methods in this field have used network formulation in many cases, they are quite similar to those for solving FCNFP or PLNFP [1, 10, 12, 21, 37]. However, exploiting their certain problem structures, there are some Lagrangian approaches [5, 14] to the problems. Recently, Holmberg [17] proposed a decomposition and linearization approach for solving the facility location problem with staircase costs. A comparison of heuristic and relaxation approaches in this field can be found in [9].
3
Concluding Remarks
In this article, three categories of PLNFP are identified and formulated in general formats. Some properties of problems in each category are investigated to show the insight of problems including FCNFP. The concave PLNFP is formulated as a FCNFP in MIP structure exploiting the concavity of arc cost functions (see section 2.3). Moreover, the indefinite (nonconvex) PLNFP is also transformed to a FCNFP with a reduced feasible region. This implies that the extreme point solution of the transformed FCNFP may not be an extreme point solution of the original indefinite PLNFP. A major advantage of the formulations introduced here is that solutions can be found by solving fixed charge problems instead of solving difficult nonconvex optimization problems. As we can see in the transformation to FCNFP, the size of FCNFP is usually quite large because of new binary variables introduced in the model. Thus, developing an efficient algorithm for large-scale FCNFP can provide a key to solve concave and indefinite PLNFP in practice.
158
References [1] A K I N C , U . , AND KHUMAWALA, B.M.: 'An Efficient Branch and Bound Algorithm for t h e Capacitated Warehouse Location Problem', Management Science 2 3 (1977), 585-594. [2] BALAKRISHNAN, A., AND G R A V E S , S.C.: 'A Composite Algorithm for a Concave-Cost Network Flow Problem', Networks 19 (1989), 175-202. [3] BALINSKI, M . L . : 'Fixed-Cost Transportation Problems', Naval Research Quarterly 8 (1961), 41-54.
Logistics
[4] B A R R , R . S . , G L O V E R , F . , A N D K L I N G M A N , D . : 'A New Optimization Method for
Large Scale Fixed Charge Transportation Problems', Operations Research 2 9 (1981), 448-463. [5] BEASLY, J . E . : 'Lagrangean Heuristics for Location Problem', European Journal of Operational Research 6 5 (1993), 383-399. [6] BORNSTEIN, C . T . , AND R U S T , R.: 'Minimizing a Sum of Staircase Functions under Linear Constraints', Optimization 19 (1988), 181-190. [7] C A B O T , A . V . , AND E R E N G U C , S.S.: 'Some Branch-and-Bound Procedures for FixedCost Transportation Problems', Naval Research Logistics Quarterly 3 1 (1984), 145-154. [8] C O O P E R , L., AND D R E B E S , C : 'An Approximate Solution Method for the Fixed Charge Problem', Naval Research Logistics Quarterly 14 (1967), 101-113. [9] CORNUEJOLS, G., SRIDHARAN, R., AND THIZY, J . M . : 'A Comparison of Heuristics and Relaxations for t h e Capacitated Plant Location Problem', European Journal of Operational Research 5 0 (1991), 280-297. [10] DAVIS, P . S . , AND R A Y , T.L.: 'A Branch-and-Bound Algorithm for the Capacitated Facilities Location Problem', Naval Research Logistics Quarterly 16 (1969), 331-344. [11] DlABY, M.: 'Successive Linear Approximation Procedure for Generalized FixedCharge Transportation Problem', Journal of the Operational Research Society 4 2 (1991), 991-1001. [12] E F R O Y M S O N , M.A., AND R A Y , T . L . : 'A Branch-and-Bound Algorithm for Plant Location', Operations Research 14 (1966), 361-368. [13] FRANCIS, R.L., McGlNNIS, L . F . , AND WHITE, J . A . : Facility Layout and Location: An Analytical Approach, 2nd Ed., Prentice Hall, 1992. [14] G E O F F R I O N , A . , AND M C B R I D E , R.: 'Lagrangean Relaxation applied to Capacitated Facility Location Problems', AIIE Transactions 10 (1978), 40-47. [15] GuiSEWITE, G . M . , AND PARDALOS, P . M . : 'Minimum Concave-cost Network Flow Problems: Applications, Complexity, and Algorithms', Annals of Operations Research 2 5 (1990), 75-100.
159 G R A Y , P . : 'Exact Solution of the Fixed-Charge Transportation Problem', Research 19 (1971), 1529-1538.
Operations
H O L M B E R G , K.: 'Solving the staircase cost facility location problem with decomposition and piecewise linearization', European Journal of Operational Research 75 (1994), 41-61. H O R S T , R., AND P A R D A L O S , P . M . (eds.): Handbook of Global Optimization, Academic Publishers, 1995.
Kluwer
KENNINGTON J . , AND U N G E R , E . : 'A New Branch-and-Bound Algorithm for the Fixed-Charge Transportation Problem', Management Science 2 2 (1976), 1116-1126. K H A N G , D . B . , AND F U J I W A R A , O . : 'Approximate Solutions of Capacitated FixedCharge Minimum Cost Network Flow Problems', Networks 21 (1991), 689-704. KHUMAWALA, B.M.: 'An Efficient Branch-and-Bound Algorithm for t h e Warehouse Location Problem', Management Science 1 8 (1972), B718-B731. K I M , D . AND P A R D A L O S , P . M . : 'A Solution Approach to t h e Fixed Charge Network Flow Problem Using a Dynamic Slope Scaling Procedure' t o appear Operations Research Letters, 1997. KUHN, H . , AND BAUMOL, W . : 'An Approximative Algorithm for t h e Fixed Charge Transportation Problem', Naval Research Logistics Quarterly 9 (1962), 1-15. L A M A R , B. W . : 'A Method for Solving Network Flow Problems with General Nonlinear Arc Costs', in Du, D.-Z. and Pardalos, P.M. (eds.), Network Optimization Problems, World Scientific Publishing Co., 1993. LAMAR, B . W . : 'An Improved Branch and Bound Algorithm for Minimum Concave Cost Network Flow Problems', Journal of Global Optimization 3 (1993), 261-287. L A R S S O N , T . , M I G D A L A S , A . , AND R O N N Q V I S T , M., 'A Lagrangean Relaxation Ap-
proach to Capacitated Minimum Concave Cost Flow Problem', European Journal of Operations Research 7 8 (1994), 116-129 McKEOWN, P . : 'A Vertex Ranking Procedure for Solving the Linear Fixed-Charge problem', Operations Research 2 3 (1975), 1183-1191. M U R T Y , K . G . : Linear and Combinatorial
Programming,
John Wiley and Sons, 1976
M U R T Y , K . G . : 'Solving the Fixed Charge Problem by Ranking the Extreme Points', Operations Research 16 (1968), 268-279. NEMHAUSER, G.L., AND WOLSEY, L.A.: Integer and Combinatorial John Wiley and Sons, 1988.
Optimization,
P A L E K A R , U . S . , K A R W A N , M . H . , A N D Z I O N T S , S.: 'A Branch-and-Bound Method
for the Fixed Charge Transportation Problem', Management 1105.
Science 3 6 (1990), 1092-
160 [32] HORST, R., PARDALOS, P . M . , AND THOAI, N.V.: Introduction tion, Kluwer Academic Publishers, 1995.
to Global
Optimiza-
[33] P A R D A L O S , P . M . : 'Enumerative Techniques for Solving some Nonconvex Global Optimization Problems' OR Spectrum 10 (1988), 29-35. [34] PARDALOS, P . M . AND ROSEN, J . B . : Constrained Global Optimization: Algorithms and Applications, Lecture Notes in Computer Science 268, Springer Verlag, 1987. [35] R E C H , P . , AND B A R T O N , L.G.: 'A Non-Convex Transportation Algorithm', in Beale, E.M.L. (ed.), Applications of Mathematical Programming Techniques, The English Universities Press, 1970. [36] S A , G.: 'Concave Programming in Transportation Networks', Ph.D. dissertation, Sloan School of Management, M.I.T., 1968. 8 (1961), 41-54. [37] SOLAND, R . M . : 'Optimal Facility Location with concave Costs', Operations 22 (1974), 373-382.
Research
[38] THACH, P . T . : 'A Decomposition Method for the Min Concave Cost Flow Problem with a Staircase Structure', Japan J. Appl. Math. 7 (1990), 103-120. [39] ZANGWILL, W . I . : 'Minimum Concave-Cost Flows in Certain Networks', Science 14 (1968), 429-450.
Management
Combinatorial and Global Optimization, pp. 161-176 P.M. Pardalos, A. Migdalas and R. Burkard, Editors © 2002 World Scientific Publishing Co.
Semidefinite Programming Approaches for MAX-2-SAT and MAX-3-SAT: computational perspectives E. de Klerk ( e . d e k l e r k O t w i . t u d e l f t . n l ) Department of Technical Mathematics and Informatics, Delft University of Technology, Mekelweg 4, 2628 CD Delft, The Netherlands. J.P. Warners (j . p. warnersStwi. tudelf t . nl) Department of Technical Mathematics and Informatics, Delft University of Technology, Mekelweg 4, 2628 CD Delft, The Netherlands.
Abstract Semidefinite programming (SDP) relaxations - in conjunction with randomized rounding schemes - yield 7/8 and 0.931 approximation algorithms for MAX3-SAT and MAX-2-SAT respectively. In spite of these powerful theoretical results, it is not clear if SDP can be used as a practical tool for solving MAXSAT problems to optimality. In this regard, the usefulness of the SDP approach will ultimately depend on the ability to exploit sparsity in the SDP relaxations of large dimension. (The dimension corresponds to the number of variables and the sparsity is related to the number of clauses.) We present an investigation of sparsity issues for the SDP relaxations of Goemans and Williamson [7], and Feige and Goemans [6] for MAX-2-SAT. Moreover, we test a branch and cut procedure to solve MAX-2-SAT to optimality, where the dual of the SDP relaxation is solved by interior point methods in order to exploit sparsity. The idea of exploiting sparsity in this way was first investigated 'for other combinatorial optimization problems by Benson, Ye, and Zhang [2]. Finally, based on this numerical experience we discuss possible extensions to MAX-3-SAT using the 7/8 relaxation of Karlow and Zwick [12] for MAX-3-SAT. Keywords: Semidefinite programming, satisfiability, interior point algorithms
162
1
Introduction
An instance of the MAX-SAT problem is defined by a collection of Boolean clauses {C\,..., Ck}, where each clause is a disjunction of literals drawn from a set of variables {x\,... , £ „ } . A literal is either a variable x; or its negation -ix, for some i. Each clause has an associated nonnegative weight, and an optimal solution to a MAX-SAT instance is an assignment of truth values to the variables which maximizes the total weight of the satisfied clauses. M AX-p-SAT is a special case of MAX-SAT where each clause contains at most p literals. There are many algorithms available for the MAX-SAT problem, even though the MAX-2-SAT problem is already NP-complete. Algorithms for MAX-SAT include well-known procedures such as EDPL, and branch and cut; approximation algorithms include GRASP, GSAT, and recently semidefinite programming (SDP) (see e.g. [10] and the references therein). SDP became a powerful tool for MAX-2-SAT problems when Goemans and Williamson [7] proved that a 0.87856 approximation algorithm could be obtained by rewriting the MAX-2-SAT problem as a Boolean quadratic programming problem, and solving a convex SDP relaxation of the resulting problem, followed by a randomized rounding procedure. This rounding procedure may be seen as a heuristic which gives a solution with a quality guarantee, while the optimal value of the SDP gives a bound on the sub-optimality of this heuristic solution. If the heuristic solution is not shown to be optimal by the bound, then the relaxation must be tightened by adding suitable cuts. Recent numerical studies indicate that it is very hard to prove optimality for MAX-2-SAT by only tightening the SDP relaxation [10]. It seems necessary therefore to use the SDP relaxations in some branch and cut framework. Up to now, the bottleneck for such an approach has been that it is hard to exploit sparsity in the solution procedure of the SDP relaxations. In this paper, we show that one can solve the dual of the SDP relaxations efficiently, using the dual interior point method of Benson, Ye and Zhang [2]. Moreover, we show that the relaxed solutions can be used in a branch and cut scheme to solve MAX-2-SAT problems with 50 variables and up to 500 clauses to optimality in a few minutes on a workstation.
2
The SDP relaxation of MAX-2-SAT
The key in the SDP reformulation is to introduce new Boolean variables ( j / i , . . . , yn) S {—1,1}" and to express the number of unsatisfied clauses as a quadratic function of these new variables. Note that a clause Xi V Xj is satisfied if«/, + yj > 0, etc. Thus we can represent the satisfiability problem as a feasibility problem: find y € { - 1 , 1 } " so that Ay > 0,
163 where A is a suitable k x n matrix. Note that
(
1 or — 1
if clause i is satisfied I (1)
—3
otherwise
I
Letting e denote the vector of all-ones, it follows that (Ay — e)T(Ay — e) = 9unsat + (k — unsat) where unsat denotes the number of unsatisfied clauses for the assignment y of truth values. Using eTe = k, we have that unsat — - (yTATAy
- 2eT Ay),
and the MAX-2-SAT problem is simply to minimize this quantity over y G {—1,1}". Thus we have rewritten MAX-2-SAT as a Boolean quadratic programming problem: unsat* :- min j - (yT AT Ay - 2eT Ay) \ y e { - 1 , l } n | . Such problems have standard semidefinite relaxations with provable quality bounds [16]. The first step in deriving the relaxation is to remove the linear term in the objective by adding an additional Boolean variable yn+i € {—1,1} to obtain unsat* = nun { i {f ATAy - 2yn+1eTAy)
| (y, yn+1) e { - 1 , l } n + 1 } .
(2)
Note that the introduction of the auxiliary variable yn+\ does not change the optimal value of the optimization problem. Problem (2) can be further simplified by using the structure of A: Note that one has
J2{ATA)H = 2k, «=i
which shows that unsat* = min {^ (f
[ATA - diag (ATA)) y - 2yn+1eTAy
+ 2k) \ (y, yn+1) e { - 1 , l } n + 1 }
We can rewrite this as: ]-k + mm{yTWy
: y e {-1,1}-+'} ,
where y := [j/i,..., yn, 2/n+i] and W is the (n + 1) x (n + 1) matrix: ATA - diag (ATA) W:=\
-eTA
-ATe 0
(3)
164 Note that Wij can only be nonzero if Xi and Xj appear together in some clause. Thus the fraction of nonzeros of W will never exceed the ratio (k + n) : | n ( n + 1). For example, for a MAX-2-SAT instance with n = 100 variables and k = 400 clauses the upper bound on the density of W is 9.8%. We will show in later sections why this ratio is an important consideration when choosing an algorithm for solving the SDP relaxation. The SDP relaxation of (3) can be derived by rewriting it as unsat* := -k + mm J T r \WyyT)
: diag \yyT] = e | ,
where "Tr' denotes the trace operator. The SDP relaxation is now obtained by replacing the rank one, positive semidefinite matrix yyT by a positive semidefinite matrix X which is not restricted to have rank one. The relaxation therefore takes the form: unsat* > SDP* := -k + min {Tr (WX) : diag {X) = e, X > 0} , (4) 4 x where X > 0 means X is symmetric positive semidefinite. Note that all products j/ij/j are replaced by the matrix entries Xtj. Given a Choleski decomposition of X, say X = VTV, one can write Xtj = (vl)Tv^ where the vl's are the columns of V. This means that the product y^- is in fact relaxed to an inner product (vl)Tv^. This type of relaxation was originally suggested by Lovasz and Schrijver [13]. Goemans and Williamson [7] proved that k — unsat* •k^SDp-* * °'87856-
,% (5)
(We should note that the derivation of the model presented here is different from that in [7], and follows that of Van Maaren and Warners [14].) Also note that Wtj = 0 V(i, j) if all possible clauses are included for the n variables. In this case exactly | of the clauses are satisfiable, i.e. unsat* = \k. In this case one trivially has SDP* = unsat*, i.e. the SDP relaxation is exact. It is therefore reasonable to expect that the SDP relaxation will become even tighter as the ratio k/n grows. This can be observed from numerical experiments shown in Table 1, where the average ratio for the left hand side of expression (5) is given as a function of the number of clauses (for random MAX-2-SAT instances with 50 variables). Goemans and Williamson also proposed the following heuristic for use in conjunction with the SDP relaxation: • Solve the SDP relaxation (4) to obtain an e-optimal X =
VTV.
165
# clauses
k—unsat* k-SDP*
50
0.95658
100
0.97502
200
0.98538
500
0.98911
1000
0.99128
2000
0.99258
3000
0.99400
4000
0.99503
5000
0.99592
Table 1: The average quality of the SDP relaxation improves as the number of clauses grows (50 variables).
• Choose a random r € R " and normalize r. • Set x/i = 1 if rTvl > 0 or set j/, = — 1 otherwise. This randomized algorithm yields an approximate solution to MAX-2-SAT with expected objective value at least 0.87 times the optimal value.
3
Additional valid inequalities
There exist instances of MAX-2-SAT where the ratio in (5) is no better than 0.88889 [6]. The SDP relaxation (4) can be strengthened by adding a number of valid inequalities (cuts) proposed by Feige and Goemans [6]. The first set of inequalities (called triangle inequalities) are based on the observation that for any pair of indices (i,j) there holds: Vn+m + yn+Wj + yiyj
>
-i
-Vn+lVi ~ Vn+lVj + ViVj
>
-1
-Vn+lVi + Vn+lVj - ViVj
>
-1.
166 In the SDP relaxation these inequalities correspond to | n ( n — 1) additional linear constraints: Xn+i,i + Xn+ij -Xn+hi
+ Xij
>
—1
- Xn+1J + X{j
>
-1
•
(6)
—
Xn+i,i + Xn+ij — Xij > —1. Note that the first of these constraints can be rewritten as ^ T r (cy lfl+1 cfj iB+1 A-) - I1- > - 1 where the vector e j J j n + 1 has ones in the positions i, j and n + 1, and zero elsewhere. In other words, we have a constraint of the form Tr(AiX)>l,
(7)
where Ai is a rank one matrix containing only zero's and ± 1 . The other two constraints in (6) can be represented similarly. Feige and Goemans have shown that the addition of these inequalities improves the quality guarantee of the SDP relaxation from 0.87856 to 0.93109 (see (5)). A bound on the worst-case approximation is 0.94513, i.e. there exist problems where the ratio (5) is no larger than 0.94513. The constraints (6) form a subset of an even larger set of | n ( n - l)(n - 2) valid inequalities which follow from: VkVi + VkVj + ViV, > VkVi - VkVj ~ ViVj >
-1 -1
-VkVi + VkVj ~ ViVj > - 1 -VkVi - VkVj + ViVj > - 1 for each distinct triple of indices (i, j , k). The corresponding constraints in the SDP relaxation once again takes the form (7). The quality guarantee for the additional cuts remains 0.93109, but the worst-known behaviour now becomes 0.98462. In practice all these inequalities cannot be added beforehand because of the increase in problem size; it is more feasible to re-solve the SDP relaxation after having added (some of) the violated inequalities.
4
Solving the SDP relaxation of MAX-2-SAT
The SDP relaxations mentioned so far can be cast in the generic form minTr(WX)
167 subject to diag (X) Tr(AtX)
=
e
>
1,
i=
l,...,m
x y o, where the A^s are rank one matrices corresponding to the valid inequalities in Section 3. The associated dual problem is m
n
subject to m
D^
+ ^yiAi
+S
=
W
J/>0,7 6 R " , S ^ 0 , where D(j) denotes the diagonal matrix with the vector 7 £ R " on its diagonal. Note that the dual matrix S will have more or less the same sparsity structure as W, if the number of cuts m is small. Recall further that W will be sparse in general, as discussed in Section 2. This suggests to solve the dual problem instead of the primal in order to exploit this sparsity structure. Dual interior point methods are based on the dual logarithmic barrier function /6(S,i/)=logdet(S) + f:iog(»i), which can be added to the dual objective function in order to replace the (matrix) inequality constraints S >z 0 and y > 0. Thus one can solve a sequence of problems of the form 5Z Vi + Y, 7t + t*h(S, y) \
(8)
subject to m t=i
for decreasing values of fj, > 0. The projected Newton direction for this problem can be calculated from a positive definite linear system with coefficient matrix consisting
168 of four blocks (see the collected works [5, 1, 8, 4, 2]):
BT
C
where 'o' indicates the Hadamard (componentwise) product, and the blocks B and C are respectively of the form
Bij = Tr [AiS^ejejS-1)
=
ejS^AiS^ej
where e, is the j t h standard unit vector, and
dj = Tr (AiS^AjS-1) Once S'1 is known, the computation of [S~ J oS^1}^, multiplication and some additions.
. J3y and C„ all require only one
The matrix M can therefore be assembled quickly once the inverse 5 _ 1 has been computed. Detail of how to compute 5 _ 1 efficiently in general is given by Benson, Ye, and Zhang in [2]. They have implemented a dual scaling method (using the search direction described above 1 ) which requires 0{s/m + n) iterations for convergence. The computation per iteration is dominated by the solution of the linear system with coefficient matrix M. This algorithm is used in the numerical experiments below. To give an impression of how fast the relaxed problem can be solved using this approach, the CPU-times (in seconds) for MAX-2-SAT relaxations of some of the largest benchmark problems from [9] is given in Table 2. The computation was done on a HP 9000/715 workstation. The column 'solution' gives the best obtained solution for the Goemans Williamson heuristic, upper gives the bound k — SDP*, and 'ratio' indicates the ratio of the best obtained heuristic solution to k — SDP*.
5
A branch and cut framework
The SDP relaxations can be used in any branch and cut framework. The framework we have used for our numerical experiments will be described in this section, with reference to Figure 1. 1 The dual scaling method chooses the parameter fi in (8) dynamically by monitoring the progress of the algorithm via the so-called Tanabe-Todd-Ye potential function. For details the reader is referred to [2].
169
problem
clauses
variables
solution
upper
ratio
time sdp
time heuristic
p2300-l
300
99
285
292
0.9762
4.09
0.32
p2300-2
300
101
285
294
0.9710
4.08
0.35
p2300-3
300
101
287
293
0.9808
3.91
0.34
p2300-4
300
101
286
293
0.9772
4.02
0.35
p2300-5
300
101
284
293
0.9694
4.90
0.34
p2300-6
300
101
283
290
0.9774
4.38
0.35
p2300-7
300
101
279
288
0.9709
4.74
0.36
p2300-8
300
101
283
291
0.9757
4.11
0.36
p2300-9
300
101
284
291
0.9761
4.26
0.34
p2400-l
400
101
369
379
0.9761
5.26
0.40
p2400-2
400
101
371
380
0.9780
5.13
0.41
p2400-3
400
101
373
383
0.9764
4.37
0.39
P 2400-4
400
101
371
378
0.9827
4.65
0.40
p2400-5
400
101
370
379
0.9780
5.22
0.39
p2400-6
400
101
372
380
0.9796
4.82
0.41
p2400-7
400
101
373
382
0.9788
4.44
0.40
p2400-8
400
101
366
376
0.9748
4.77
0.40
Table 2: Solution times for the SDP relaxation of MAX-2-SAT and for the GoemansWilliamson heuristic.
At any node in the branching tree, the current set of clauses (obtained after partial assignment of the variables) is denoted by $, and lb and ub contain lower and upper bounds on the minimal number of unsatisfiable clauses respectively. The value unsat is a counter for the number of unsatisfied clauses by the current partial assignment. Note that lb and unsat are local variables that are only valid in the current branch; on the other hand, ub is a global variable which is valid for the whole search tree. At termination of the procedure, ub contains the optimal value of the instance. Before calling node.procedure the values lb, unsat, and ub must be initialized. One can take lb := 0, unsat := 0, ub := k. Following Borchers and Furman, unit resolution is applied if ub — unsat = 1. Subsequently, the semidefinite relaxation of the current
procedure node_procedure ($, lb, unsat); if (ub — unsat = 1) unit_resolution($); (lbsltp, ubsdp)~ SDP-.relaxation(<J>);
(*)
ub := mm{ub, unsat + ubadp};
(*)
lb~max{lb,
(*)
unsat 4- max{0, Z6sdp}};
if (ub — lb < 1) return; x :=branch_rule($); Set x «— TRUE and update $, unsat; if (u6 — unsat > 1) node_procedure ($, Z6, wrasai); Set x 4- FALSE and update $, unsat; if (u6 — unsat > 1) node.procedure ($, Z6, unsat); return;
Figure 1: Branch and cut framework for MAX-2-SAT
171 formula is solved to obtain upper and lower bounds ubslip and lbsip. The current bounds ub and lb are then updated (taking unsat into account), and if ub — lb < 1, then the best known solution so far cannot be improved upon in the current branch, so that we backtrack. Otherwise a variable x is determined to branch on, which is set to TRUE and FALSE respectively. The branching rule for fixing variables is as follows: choose the variable with the highest occurrence in the longest clauses. The formula $ and unsat are subsequently updated; if ub — unsat < 1 this branch need not be further explored. In the other case node_procedure is recursively called.
In each node, the steps marked (*) can be repeated adding violated cuts from Section 3 to the relaxation, to obtain tighter bounds.
6
Numerical experiments
In this section we present some numerical results for the branch and cut SDP algorithm of the previous section. The results presented here are of a preliminary nature, and were obtained without adding extra cuts. The MAX-2-SAT benchmark problems are taken from Borchers and Furman [3] (except for the two largest instances). As before, all reported CPU-times are in seconds on a HP-9000/715 workstation with 160MB internal memory. The SDP branch and cut method presented here are compared to a modified EDPL algorithm [3] and to the Mixed integer linear programming approach using the commercial solver Minto [15]. The respective CPU-times are shown in Table 3. It is immediately clear that the SDP approach is distinctly superior to the other two approaches if the clauses/variables ratio exceeds 4:1. The reason is that the SDP relaxation becomes tighter as this ratio grows, as discussed in Section 2. Note also that the SDP branch and cut algorithm solved each of the problems in a few minutes. It therefore has a very robust performance in comparison to the other two methods. The second set of test problems are weighted M AX-2-S AT problems from Borchers and Furman [3]. The same observations hold as for the unweighted problems, although the difference is now somewhat less pronounced. All the algorithms fare somewhat better on these problems. The results are shown in Table 4.
172
# clauses S D P (nodes)
EDPL
Minto
100
84 (82)
1.36
12.9
150
69 (64)
5.1
18.0
200
91 (70)
395
67.3
250
118 (92)
2218
128
300
170 (128)
29794
687
350
127 (91)
>12hr
2339
400
56 (40)
>12hr
1550
450
276 (210)
>12hr
12634
500
205 (144)
>12hr
8677
2500
331 (184)
not run not run
5000
663 (399)
not run not run
Table 3: Solution times (in seconds) of MAX-2-SAT benchmark problems (n = 50) for different algorithms
7 7.1
Future work Cutting planes
The results from the previous section for MAX-2-SAT can probably be improved upon significantly by adding some of the cuts described in Section 3 to the relaxations. The influence of added cuts is illustrated in Table 5. These results were obtained using the SDP solver CUTSDP [11] in the branching scheme described in Section 5. The solution times are for proving optimality only, and are given for two MAX-2SAT instances from Table 3 and two from Table 4 (weighted). The solver CUTSDP uses a primal-dual predictor-corrector algorithm based on the so-called XS direction. This direction also results in a sparse Newton system at each iteration of the solution of MAX-2-SAT relaxation, but the algorithm still requires additional computation involving the dense primal matrix variable. For this reason, it is not as efficient as the dual scaling method. However, the CUTSDP software automatically adds (some of) the violated triangle inequalities described in Section 3, and therefore gives and indication of the effect of cutting planes on the branching procedure.
173
# clauses
SDP (nodes)
EDPL
Minto
100
101 (125)
1.36
12.9
150
101 (108)
2.04
16.3
200
58 (61)
23.5
34.1
250
137 (117)
235
171
300
61 (44)
874
149
350
161 (122)
40285
2155
400
100 (82)
20233
579
450
53 (44)
>12hr
1420
500
118 (76)
>12hr
3153
Table 4: Solution times (in seconds) for weighted MAX-2-SAT benchmark problems (n = 50) for different algorithms
# clauses
CUTSDP with cuts
(nodes)
CUTSDP without cuts
(nodes)
100
283
(25)
206
(78)
450
396
(38)
403
(191)
100 (weighted)
180
(20)
228
(116)
450 (weighted)
270
(28)
80
(40)
Table 5: Solution times (in seconds) for MAX-2-SAT benchmark problems (n = 50) for the CUTSDP method (with and without cuts) in a branching framework
It is clear from Table 5 that the introduction of cuts reduces the number of nodes in the branching tree significantly, but increases the solution time of the relaxations at the nodes. The total solution time is not improved in general, and all the solution times are worse than those reported in Table 3 and Table 4 for the dual scaling method without cuts. Nevertheless, it is clear that the number of branching nodes can be reduced significantly; the challenge is therefore to extend the dual scaling method to use cuts and to
174 find the optimal trade-off between stronger relaxations and increased solution times.
7.2
Extension to MAX-3-SAT
Another challenging problem is to extend the approach in this paper to MAX-3-SAT problems. The are two possibilities in this regard: (1) One can rewrite MAX-3-SAT as a MAX-2-SAT problem in such a way that the SDP relaxation to the MAX-2-SAT problem yields a 0.801 approximation algorithm for MAX-3-SAT (see [17]); the resulting MAX-2-SAT problem can then be solved using the approach in this paper. (2) One can use the branch and cut formulation of this paper in conjunction with the MAX-3-SAT relaxation of Karlov and Zwick [12]. This relaxation guarantees a 7/8 approximation. The difficulty with approach (1) is that one has seven 2-clauses for each clause of MAX-3-SAT. For example, the clause a V b V c is replaced by the seven (weighted) clauses 2a V z, ->6Vz, &V-.Z, -1CV2, cV^z, b V c, ->b V ->c, where z is an auxiliary variable. If the MAX-3-SAT instance therefore has k clauses and n variables, the associated MAX-2-SAT instance has 7k clauses and n + k variables. The MAX-2-SAT instance is also highly structured, and it remains to be seen if the SDP relaxations are as effective in this case as for random instances. The approach (2) involves the following relaxation of MAX-3-SAT: the clause Xi V Xj V xk will be true if and only if 1 - f (X„+1 + Xi)(Xj + Xk),
!
1 - \{xn+i
+ Xk)(Xi + Xj),
1 - \{xn+l
+ Xj)(Xi + Xk)
1.
1
We can relax the left hand side to four linear matrix inequalities by replacing xt by a vector Vi of norm one, etc., and replacing the products by inner products, as before: t
<
1 - -(Vn+l + Vi)T{Vj + i;*)
t
<
1-
t
<
1 - j K + l + Vk)T{Vi + Vj)
t
< 1.
~{vn+1
+ Vj)T(Vi + Vk)
175 The SDP relaxation involves maximizing the sum of the values t for all the clauses. It is easy to check that one obtains an SDP with 4k inequality constraints where the coefficient matrices of the constraints are of rank 3. One can still solve the resulting problem by the dual scaling method (see [2]), but the assembly of the linear system at each iteration becomes more expensive, and its coefficient matrix becomes more dense. The question is therefore if these relaxations can be solved quickly enough to allow incorporation in a branch and cut scheme.
References [1] K.M. Anstreicher and M. Fampa. A long-step path following algorithm for semidefinite programming problems. Working Paper, Department of Management Sciences, University of Iowa, Iowa City, USA, 1996. [2] S.J. Benson, Y. Ye, and X. Zhang. Solving large-scale sparse semidefinite programs for combinatorial optimization. Working paper, Computational Optimization LAb, Dept. of Management Science, University of Iowa, Iowa City, USA, 1997. [3] B. Borchers and J. Furman. A two-phase exact algorithm for MAX-SAT and weighted MAX-SAT problems. Manuscript, 1997. (To appear in Journal of Combinatorial Optimization). [4] E. de Klerk. Interior Point Methods for Semidefinite Programming. PhD thesis, Delft University of Technology, Delft, The Netherlands, 1997. [5] L. Faybusovich. Semi-definite programming: a path-following algorithm for a linear-quadratic functional. SIAM Journal on Optimization, 6(4):1007-1024, 1996. [6] U. Feige and M. Goemans. Approximating the value of two prover proof systems with applications to MAX 2SAT and MAX DICUT. In Proc. Third Israel Symposium on Theory of Computing and Systems, 1995. [7] M.X. Goemans and D.P. Williamson. Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM, 42(6):1115-1145, 1995. [8] B. He, E. de Klerk, C. Roos, and T. Terlaky. Method of approximate centers for semi-definite programming. Optimization Methods and Software, 7:291-309, 1997.
176 [9] S. Joy, J. Mitchell, and B. Borchers. A branch and cut algorithm for MAX-SAT and weighted MAX-SAT. In M.A. Trick and D.S. Johnson, editors, DIM ACS series in Discrete Mathematics and Theoretical Computer Science, volume 26. American Mathematical Society. [10] S. Joy, J. Mitchell, and B. Borchers. Solving max-sat and weighted max-sat problems using branch-and-cut. Manuscript, 1998. [11] S. Karisch. CUTSDP - A toolbox for a cutting-plane approach based on semidefinite programming. User's Guide/Version 1.0. Technical Report IMM-REP-199810, Dept. Mathematical Modelling, Technical University of Denmark, 1998. [12] H. Karlow and U. Zwick. Manuscript, 1997.
A 7/8-approximation algorithm for max 3sat?
[13] L. Lovasz and A. Schrijver. Cones of matrices and set-functions and 0-1 optimization. SI AM Journal on Optimization, 1(2):166-190, 1991. [14] H. Van Maaren and J.P. Warners. Bounds and fast approximation algorithms for binary quadratic optimization problems with application to MAX 2SAT and MAX CUT. Technical Report 97-35, Delft University of Technology, The Netherlands, 1997. [15] G.L. Nemhauser, M.W.P. Savelsbergh, and G.C. Sigismondi. Minto, a mixed integer optimizer. Operations Research Letters, 15(l):47-58, 1994. [16] Yu. Nesterov. Quality of semidefinite relaxation for nonconvex quadratic optimization. CORE Discussion paper 9719, Belgium, March 1997. [17] L. Trevisan, G.B. Sorkin, M. Sudan, and D.P. Williamson. Gadgets, approximation and linear programming. In Proc. of the 37th Anual IEEE Symposium on Foundations of Computer Science. 1996.
Combinatorial and Global Optimization, pp. 177—204 P.M. Pardalos, A. Migdalas and R. Burkard, Editors © 2002 World Scientific Publishing Co.
On a Data Structure in a Global Description of Sequences Victor Korotkich (v. korotkichQcqu. edu. au) Department of Mathematics and Computing, Central Queensland University Rockhampton, Queensland ^702, Australia
Abstract A data structure in a global description of sequences is presented. The structure consists of infinite hierarchical levels. We can view the elements as those composed of elements from the lower levels. The key interest in the structure is that it has two distinctive representations, that is, algebraic and geometric, which complement each other. In the first one the elements are integer relations suggesting that ultimate building blocks are just integers from which the structure develops as one undivided whole. In the second one the elements are two-dimensional geometric patterns which upon visualization give a picture of hierarchical formations. The picture contains nonlocal order and large symmetry. Global optimization problems formulated in terms of the structure are given to show its descriptive potentialities. Well-known geometric objects appear in a new way as solutions to the problems. Keywords: data structures, binary sequences, global optimization
1
Introduction
Frequently the design of new algorithms in combinatorial and global optimization involves the development of new data structures. In computations the basic carriers of information are finite binary sequences. The paper presents a data structure in a
178 global description of sequences. The structure consists of infinite hierarchical levels. We can view the elements as those composed of elements from the lower levels. The key interest in the structure is that it has two distinctive representations, that is, algebraic and geometric, which complement each other. In the first one the elements are integer relations suggesting that ultimate building blocks are just integers from which the structure develops as one undivided whole. In the second one the elements are two-dimensional geometric patterns which upon visualization give a picture of hierarchical formations. The picture contains nonlocal order and large symmetry. The structure begins to emerge when sequences are described in global terms as presented in Sect. 2. To realize a global description of a sequence we associate it with numbers, called "structural" numbers, that are assumed to capture its global properties. Structural numbers are combinatorial objects in nature. There exists a geometric interpretation of structural numbers which visualizes them and gives their meaning. The geometric interpretation plays the crucial role in the study. The next step in Sect. 3 leads us to consider the ability of structural numbers to completely specify sequences. In particular, it turns out that we should be concerned to find out whether they can be viewed as coordinates of a space. We show that a complete specification of a sequence with respect to another one in terms of structural numbers is reduced to a system of linear equations. These systems of linear equations constitute the main ingredient for the definition of the structure. Our approach to the system is to focus on its geometric understanding. In general terms geometric understanding occurs by visual pictures that address insight as a force for generating new knowledge. In our case this means that we do not algebraically manipulate the system, but instead through visualization we try to understand phenomena involved. It turns out that the geometric interpretation reveals interesting phenomena. With this focus on visualization two-dimensional geometric objects, called integer patterns, are presented in Sect. 4. They are used as a means to unfold the systems of linear equations into visible images. In Sect. 5 for a particular example we show how the system of linear equations can be visualized to generate a heirarchical formation of integer patterns and integer relations. By visualization the formation has order and large symmetry. This order is nonlocal as it concerns integer patterns of each level and across all levels simultaneously. The aim of the example is to demonstrate a language of integer patterns and hopefully provide a visual access to the structure when it is defined formally. Figuratively, such examples are a window through which we can view properties of the structure and see answers that may not be possible to obtain in the formal way. The example helps to understand that the formation constitutes one possible formation among many others admitted by a whole structure. In Sect. 6 we consider and define this whole structure. Firstly, the system of linear equations is associated with a hierarchical set of integer relations. However, transformations of integer relations
179 are not viewed as a good way to describe how things change. They are more perceived visibly through changes in geometric objects. Secondly, we show that there is an isomorphism between the set of integer relations and a hierarchical set of integer patterns. This puts the integer relations in a geometric form and gives a picture for their transformations. The language of integer patterns in this picture implies phenomena with changes. We define a structure that incorporates all these formations into one whole and call the structure a "web of relations" to informally capture its properties. Due to the isomorphism, elements of the web of relations can be equally seen as integer patterns or integer relations. In Sect. 7 we show descriptive potentialities of the structure. The descriptions arise as hierarchical formations of the elements of the structure and the geometric interpretation enables us to actually see them. To make the potentialities more accessible simple global optimization problems formulated in terms of the structure are considered. We demonstrate that well-known geometric objects appear in a new way as solutions to the problems.
2
Structural Numbers and their Geometric Interpretation
The structure begins to emerge when sequences are described in global terms as presented in this section. It is known that sequences occur when physical systems are analysed in terms of Cartesian order. Specifically, in this analysis they give a simple description concerning the dynamics of the systems in a finite time. It is convenient here to have this interpretation of sequences. Let 6 and e be respective spacings of a spacetime lattice (5, e) in 1 + 1 dimensions. Let I be an integer alphabet and In = {s: s = sx...sn, Si £ I, i = 1,..., n} be the set of all sequences of length n > 2. If I = { - 1 , +1} then In is the set of all binary sequences of length n, denoted by the alternative notation Bn. A sequence s = Si...sn € In encodes on a lattice (S, e) the dynamics of a physical system that has a space position Sj<5 at a time instant ie,i = l,...,n. This encoding can always be made clear from a context and explicit when 5 = 1 and e = 1. A global description of a sequence is its representation that uses information about the sequence as a whole object. To realize a global description of a sequence s = Si...sn € In we associate it with numbers #i(s),..., $/t(s),... , called "structural" numbers, that are assumed to capture its global properties. Definition 1. Structural numbers ^ ( s ) , k = 1, 2,... of a sequence s = Si...sn 6 In
180 on a lattice (5, e) are defined by the formula k-l
tik(s) = Y, akmi((m + nfsi
+ (m + n-
lfsi
+ ... + (m + iysn)ek6,
(1)
t=0
where m is an integer, akmi = (1/fc!) ( * ) ( ( - l ) * - ' " 1 ^ + I)*"' + ( - l ) * - ^ * - ' ) - / 0,
i = 0,..., k - 1,
and (A is the binomial coefficient. Let us briefly analyse the formula (1). Firstly, we see that a sequence s = s\...sn comes into play in (1) as a whole object, because all its components Si,i = 1, ...,n are presented in it. Secondly, we observe that structural numbers are combinatorial objects as binomial coefficients are involved in (1). The formula also shows that structural numbers are defined by using powers of integers m + 1, ...,m + n but it does not give immediately a style of thinking based on intuition. Fortunately, there exists a geometric interpretation of structural numbers which visualizes them and gives their meaning. The geometric interpretation plays the crucial role in the study and can be presented by two parts involved. Firstly, finite sequences admit a natural representation in terms of piecewise constant functions. Namely, let As[tm,tm+n], \m\ = 0,1,... denote the partition of an interval [tm,tm+n] C R[e] with n + 1 equally spaced points ti,i = m,...,m + n such that tm = me,
ti+i -U = e, i = m, ...,m + n-
1,
n = {tm+n -
tm)/e
and where R[e] means that R is measured in terms of e. Let Wj(Ae[tm, tm+n}) be the set of all piecewise constant functions / : [tm, tm+n] —» R^j such that / is constant in [tm, tm+i], (ti} ti+i], i = m + 1,..., m + n — 1, where R^j means that R is measured in terms of S. A code of a function / 6 Ws(As[tm,tm+n]), denoted c ( / ) , is a sequence s = S\...sn, Si € R, i = 1, ...,n such that Si6 is the value of the function in (U-i, ti],i = m + 1,..., m + n. Let a mapping pm(E take a sequence s = S\...sn, s, G R, i = 1,..., n to a function / G Ws(Ae[tm,tm+n\), denoted / = p m fe(s), such that c(f) = s and whose kth integral /'*! satisfies the condition fik\tm) = 0, k = 1, 2,... (see Fig. 1). Secondly, the following result is of central importance to the geometric interpretation. In particular, an integer code series for the value of the kth k = 1,2,... integral /'*' of a function / G Wd(Ae[tm, tm+n\), f = pms£{s), s = Si...sn G /„ at the point tm+n in terms of the code gives [1] fc-i
f[k\tm+n)
= Y, akmi((m + n / s i + (m + n - 1 ) % + - + {m + lYsn)ekS. i=0
(2)
181
f
t 1
. i
i
r-— r - —hi
i
i——*t
Figure 1: Graph of a function / = pon(s),m = 0,(5 = l,e = 1, s = + 1 — 1 — 1 + 1. Comparing (1) with (2), we have / [ " ( W n ) = * ( s ) , fc = l , 2 , . . . .
(3)
Consequently, the geometric interpretation lies in the fact that the kth k = 1,2,... structural number of a sequence s € /„ is the value of the kth integral of a function / = Pm5e{s) at the point tm+n. Actually, Definition 1 is chosen to ensure the validity of this condition. It follows easily from the geometric interpretation that structural numbers are independent of m, which specifies a "frame of reference" of their consideration. Clearly, structural numbers can be seen in a way as global properties of the sequence since, as definite integrals, they carry information about it as a whole. To illustrate the geometric interpretation, consider a function / € Wi(Ai[to, U]) such that / = pon(s),m = 0,5 = l , e = l , s = + l — 1 — 1 + 1 (see Fig. 1). In this example the symmetry of the function is easily seen and we can sketch, thinking in geometric images, the graph of its first /M and second /t 2 l integral as shown in Fig. 2. From the figures we conclude that f[l]{U) = 0, fl2]{U) = 0, / [ 3 ] (t 4 ) > 0 and, by using (3), 0i(s) = O,
02(«) = 0,
tf3(s)>0.
(4)
Note that the visualization gives much more than formula (1) by blind computation. For example, from Figs. 1 and 2 condition (4) is immediate to our intuition.
3
Structural Numbers as Coordinates of a Space and a System of Linear Equations
In Sect. 2 we started to define the structure by presenting structural numbers which provide a global description for sequences. The next step leads us to consider the ability of structural numbers to completely specify sequences. In particular, it turns
182
Figure 2: Graphs of the first Z'1' and second /' 2 ' integral(sketched) of the function / depicted in Fig. 1. The visualization gives much more about fW(U),k = 1,2,... than formula (2) by blind computation. out that we should be concerned to find out whether they can be viewed as coordinates of a space. Definition 2. If for a pair of distinct sequences s,s' G /„ there is a finite integer k > 1 such that their first (k — 1) structural numbers are equal
«i(') = < i M
^ i W ^ w M
whereas the fcth structural numbers are not $A>(S) 7^ $fc(s') when k > 2 and #i(s) 7^ $i(s') when k = 1 then C(s, s') = k. Definition 2 implies that by using C(s, s') structural numbers a sequence s € /„ can be completely specified with respect to another sequence s' e In- Note that the special order is used in the specification as structural numbers are considered successively starting with the first ones. The following theorem clarifies the question about the ability of structural numbers to completely specify sequences. Theorem 1. If s = si...sn,
s' = s[...s'n £ In is a pair of distinct sequences then C{s,s')
Proof. Consider a sequence s = Si...sn € In and let N*(s,m) = {m + n)'si + (m + n-
l ) ' s 2 + ... + (m + l)'s„,
= 0,1,.
(5)
Then (1) for k = 1,2,... takes the form jt-i
fik(s) =
E i=0
^akmiKi(s,m)ek6.
(6)
183 It can be easily shown by using (6) that structural numbers i?i(s),i = l,...,k numbers Nj_i(s,m),i = l,...,k for A; > 1; and if
Ms)=Ms'),-,A-i(s)
define
= i)k-i(s'), M')*MJ)
(7)
then N 0 (s, m) = N 0 (s', m),..., Nfc_2(s, m) = Nfc_2(s', m),
(8)
K f c _i(*,ro)^K t _ 1 (s' ) m)
(9)
for A; > 2 and vice versa. Condition (8) for k > 2 by taking numbers N;(s',m),i = 0, ...,fc — 2 from the righthand side to the left-hand side of the equations and using (5) can be brought to a system of (k — 1) linear equations (m + l)°(s„ - s'n) + {m + 2) 0 (s n _! - s'^)
(m + l)*" 2 ( S n - s'n) + (m + 2)k-2(sn^
- s'^)
+ ... + (m + n)°{Sl - s[) = 0
+ ... + (m + n) fc ~ 2 ( Sl - s'J = 0 (10)
in n integer unknowns st — sj, i = l,...,n. Note that if the system (10) has n equations then at least one of its equations is nonzero. Indeed, the determinant of the Vandermonde matrix /
(m + l)° (m + 1)1
V(m+l)n_1
(m + 2)° (m + 2) 1
... ...
(m + n)° (m + n)1
\
(m + 2 ) " - 1 ... {m + n)"-1
j
is known to be written in the form det(V) = ±Il{i - j : 1 < j < i < n) ^ 0
(11)
and is nonzero. Suppose all linear equations of the system are zero, then we conclude, with the help of (11), that the unique solution to the system is the trivial one si — s'i = 0,i = l,...,n. This contradicts that the sequences s,s' are distinct. Thus not all equations of the system (10) are zero and there exists an integer 1 < k < n such that (m + i y ( s „ - s'n) + ... + (m + n)*(si - s[) = ^ ( s , m) - Xt(s', m) = 0 where i = 0,..., k — 2 and (m + \y-\sn
- s'J + ... + (m + n ) * " 1 ^ - si) = N fc -i(s, m) - N t _i(s', m) ^ 0
when 2 < fc < n, and (m + l)°(s„ - s'n) + ... + (m + n)°(si - s'x) = N 0 (s, m) - N 0 (s', m) =f= 0
184 when k = 1. The integer k specifies the nonzero equation of (10) with the minimum power of integers m 4-1,..., m + n. This implies that N 0 (s, rn) = K 0 (s', m),..., N*-2(s, m) = Kfc_2(s', m),
N*_i(s, m) / N*_i(s', m)
when 2 < fc < n and Njt_i(s, m) ^ Njfc_i(s', m) when k = 1. Then, using (8) and (9), we have (7) with ^ ( s ) = ^ ( A - A - i ( s ) = ^ - i ( s ' ) , flb(a) ^ ^ ( s ' ) when 2 < A; < n and i?i(s) ^ ^i(s') when fc = 1. Therefore, a sequence s e J n can be completely specified with respect to a sequence s' e I„ by at most n of its structural numbers. Recalling Definition 2, we conclude that C(s, s') < n. This completes the proof of the theorem. • As a result of Theorem 1 we see that numbers Nj_!(s, m),i = 1, ...,n can be equivalently considered instead of structural numbers ??i(s),z = l,...,n. We can say that there are two related sets of coordinates with the merit of structural numbers being their geometric interpretation. However, our main interest in Theorem 1 is connected with the fact established during the course of its proof. Namely, we have shown that iff k = C(s,s') > 2 then we have (8) and (9), which by taking numbers N;(s', m), i = 0,..., k — 1 from the right-hand side to the left-hand side of (8) and (9) can be brought to a system of (k — 1) linear equations (m + n)°{Sl - s'J + ... + (m + 1)°(«„ - s'n) = 0 (m + n)k-2(Sl
- s[) + ... + (rn + l)*" 2 ( S n - 0 = 0
(12)
- s[) + ... + (m + l ) * " 1 ^ - s'n) / 0.
(13)
and linear inequality (m + n)k-\Sl
We also call the system (12) a system of integer relations, because its equations can be seen as relations between 0 < i < k — 2 powers of integers m + 1,..., m + n. The system of linear equations (12) and inequality (13) constitute the main ingredient for the definition of the structure. Definition 3. Let C(s,In)
=
max
C(s,s')
< n.
Definition 3 reads that C(s, In) of a sequence s G In is the maximal number of its successive structural numbers by which it can be completely specified in /„. In Sect. 5 we present a notion which captures that C(s, s') is connected with an interesting visible phenomenon. It is conjectured that for s £ Bn [2]
C{s,Bn)< Llog2nJ+l, and the upper bound is realized by the celebrated Prouhet-Thue-Morse (PTM) sequence rj [3], [4], [5], where [log 2 nJ is the integer part of log2 n.
185
\'/ F/
v y^ A
\
\
'
*
V
Figure 3: The pattern P(f, [a, b]) (shaded) of a function / . The shadow gives to the pattern impression of a spatially-localized object.
4
Integer Patterns: Means for the Visualization of the System
In Sect. 3 we showed that a complete specification of a sequence with respect to another one in terms of structural numbers was reduced to the system of linear equations (12). These systems constitute the main ingredient for the definition of the structure. Our approach to the system is to focus on its geometric understanding. In general terms geometric understanding occurs by visual pictures that address insight as a force for generating new knowledge. In our case this means that we do not algebraically manipulate the system, but instead through visualization we try to understand phenomena involved. It turns out that the geometric interpretation reveals interesting visible phenomena. With this focus on visualization we need to use proper geometric objects to unfold the system (12) and inequality (13) into visible images. Namely, identify a function / : [a, b] —> 5R1 with a two-dimensional geometric pattern P(f, [a, b]) C 5ft2 as shown in Fig. 3, where the pattern of a function / is brought out by shading (see [6] for more details). Next, we describe geometric patterns of a special character. The consideration of these patterns arises naturally as they give a mental image of spatially-localized objects. However, for the most part these patterns attract particular interest because of their connection with integers and integer relations. To be specific, set / = /M and for a function / e Ws{Ae[tm, tm+n\), f = pm6s{s), s = S\...Sn
£
ln
• e t ^(/Jim,tm+n]) be a set of intervals [* m + t,Wt+i] C \tm,tm+n} such that si+1 / 0, i = 0,..., n - 1. Let T ( / M , [tm, tm+n]), k = 1, 2,... be a set of intervals [U,tj] C [tm,tm+n], m < i, i + 1 < j , j < m + n such that /W(
[u.tj]), A: = 0,1,... of a function f
eW6{Ae[tm,tm+n})
186 is called an integer pattern of the kth integral of the function if [ti,tj]eT(f^,[tm,tm+n])^0.
For instance, P{f,[UM), P(f, [tio,tn]) 1 - 1 e In, P(f[1],[h,t7}),
Fig. 4 shows integer patterns P(f, [t0, £1]), P{f, [t\, t2]), P{f, [t2, £3]), Ptf.foM* P(f,[U,t7]), P{f,[t7,t8}), P(f,[ts,t9}) P(/,[t 9 > tio]), of a function / = pon(s), s = + 1 - 1 - 1 0 + 1 + 1 - 1 + 1 - 1 I = { - 1 , 0 , + 1 } and integer patterns P ( / W , [t0,t2]), P(f[l],[t2,h\), [1] P(f ,[*rM) of its first integral /W.
t 1
p*1
-T
'A .t ' "
"V-
R •
vi-n M t
Figure 4: Integer patterns(shaded) of a function / and its first integral /W, where f = Pou(s), s = + l - l - 1 0 + l + l - l + l - l - l - l e / u , 7 = { - l , 0 , + l } .
W h a t Picture Appears when the System is Visualized: an Illustrative Example In this section for a particular example we show how the system of linear equations (12) and inequality (13) can be visualized to generate a heirarchical formation of integer patterns and integer relations. By visualization the formation has nonlocal order and large symmetry. The aim of the example is to demonstrate a language of integer patterns and hopefully provide a visual access to the structure when it is defined formally. Figuratively, such examples are a window through which we can view properties of the structure and see answers that may not be possible to obtain in the formal way. Consider a sequence ? e B , connected with the Fibonacci numbers, tiling and quasicrystals [7]. The sequence
187 1 - 1,... . Let s=+1-1+1+1-1+1-1+1+1-1+1+1-1+1-1 be the initial segment of length 15 of <;. There is a sequence s' = - 1 + 1 + 1 + 1 + 1 - 1 - 1 + 1 + 1 + 1 - 1 + 1 - 1 - 1 + 1 such that C(s, s') = 4, i.e., Ms)
= Ms'),
k = 1,2,3,
Ms)*
Ms1),
(14)
where e = 1,5 = 1. Clearly, condition (14) is equivalent to a system of linear equations 15°( Sl - si) + ... + l°(s 1 5 - s'15) = 0 151(s1-s'1) + ...+ l 1 ( s 1 5 - s ' 1 5 ) = 0 15 2 (s x - s[) + ... + l 2 ( S l
= 0,
(15)
153{Sl-s[)
^0.
(16)
and inequality + ... + l3{st
We are interested to show what phenomena are connected with (15) and (16).
/\S
s^s/^
/v\
' \ T 1 > t t i i 4 i'o
Figure 5: Graphs of f^ rately.
1'1 12 13 U
15
'
and // ' exhibit no visible properties when regarded sepa-
By using the geometric interpretation we graph the first integrals /W, /, of functions / = poii(s),/< = Pon(s') as shown in Fig. 5. When regarded separately the graphs exhibit no visible properties, but depicted together, properties emerge as in Fig. 6. These properties are manifested explicitly in the graph of a function
188
f ky
XX / D^ 1 4
10
s\ >
1 12 13 14 15
Figure 6: Graphs of /W and / / ' depicted together uncover visible properties. Consequently /W(ti 5 ) = / / % i 5 ) , * = 1,2,3, /W(t 15 ) / / / % s ) , or by (3)
Ms) = 0*(A * = 1,2,3,
04(«) 7^ Ms'),
and this leads us exactly to (15) and (16). Thus system of linear equations (15) and inequality (16) can be associated with a hierarchical formation of the integer patterns, that is amenable to explanation in terms of visible things. In particular, Figs. 7 and 8 may be viewed as though they were experimental results successively representing levels of this formation. It is interesting to observe that as we integrate, it appears as if corresponding integer patterns merge together in pairs to form new integer patterns belonging to the next level. Since g^k\t) > 0, t e (*o, *is], k = 4, 5,... then there are no integer patterns belonging to the (k + l)th level. Moreover, the levels combined together form a unified geometric object, in which a tree diagram can be noticed (see Fig. 9). This object displays many interesting features that address our insight. Firstly, in Fig. 9 we see immediately large symmetry involved in the formation of the integer patterns. It is worthy of note that the symmetry is exhibited without any need for further explanation. The eye at once sees the crucial image that yields it. Secondly, we witness order involved in the formation of these patterns. This order is very much connected with the symmetry and of a nonlocal character. It concerns all integer patterns of a level and across all the levels simultaneously. In particular, the order specifies how integer patterns of a level have to make the exactly right connections between themselves in terms of their positions to grow from the level to the next one. One may say that the order enfolds integer patterns of a level into integer patterns that they produce at the next level. The order is very rigid as a minor change in it leads to the collapse of the whole object. Thirdly, Fig. 9 demonstrates self-similarity that accompanies the formation. It arises as at each level topologically we have the same situation when the integer patterns join or branch. The difference is in scale and shape. But the most important is
189
Figure 7: Graphs of g and gM. As we integrate, it appears as if the integer patterns merge together in pairs to form new integer patterns belonging to the next level.
Figure 8: Graphs of g^ and g^ (sketched). that Fig. 9 give us a clear understanding that the integer patterns uncover themselves through their connections to an undivided whole as its integrated parts. The undivided whole consists of infinite hierarchical levels involving integer patterns, all of them interconnected. For example, the integer pattern of the forth level, which can be viewed as the result of the formation, is just an integrated building block of the level. This naturally motivates us to make this whole a subject of special interest and the structure is defined in Sect. 6 to formally capture this whole. We can think the integer patterns of each coming level as next in order of complexity to the integer patterns of the previous level, since every integer pattern of the successive level is formed from integer patterns of the previous one. Therefore, condition C(s, s') = 4 admits interpretation in terms of the maximum level or complexity of the integer pattern that the sequences s, s' produce. More precisely, all these four levels contain only integer patterns while next ones do not. This phenomenon may be seen to be close to intuitive understanding of what complexity is about and motivates us to use a special notion "structural complexity" as the name for C(s, s'). It is possible to translate these geometric considerations into algebraic form. This is
190
Figure 9: The hierarchical formation of integer patterns corresponding to (15) and (16). This formation has order and large symmetry. The order is nonlocal as it concerns all integer patterns of a level and across all levels simultaneously. The unified geometric object of these integer patterns is invisibly controlled by relations between integers 1,2,5,6,10,11,14,15. The structural complexity C(s,s') = 4 is the highest level of this formation. a place where a hierarchical formation of integer relations comes into view. Namely, substituting Si, s[, i = 1,..., 15 explicitly in (15) and (16), we obtain a system of integer relations, which rearranged for some elegance with 2 cancelled out gives a system of Prouhet's type identities [3] 1° + 6° + 11° + 14° = 2° + 5° + 10° + 15° 1 1 + 6 1 + l l 1 + 141 = 2 1 + 5 1 + 101 + 151 1 2 + 6 2 + l l 2 + 142 = 2 2 + 5 2 + 102 + 152
(17)
1 3 + 6 3 + l l 3 + 143 ^ 2 3 + 5 3 + 103 + 153
(18)
and an inequality
between integers 1,2,5,6,10,11,14,15. Inequality (18) may be viewed to suggest that relations between the integers presented in (17) are exhausted. System of integer relations (15) viewed by itself does not show that in fact we have a hierarchical formation of these relations behind it. The geometric interpretation helps to identify this formation shown in Fig. 10. In Figs. 9 and 10 we observe a match between the integer patterns and the integer relations, and that the formation of integer patterns corresponds to the formation of integer relations. Overall, in these figures we witness a connection between two different modes of mathematical
191
Figure 10: The hierarchical formation of integer relations corresponding to (15) and (16). There is a one-to-one correspondence between this formation and the formation of integer patterns in Fig. 9. The figure gives us a clear understanding that everything develops from the integers, i.e., "ground" level, as one undivided whole and the formation is its integrated part. understanding, that is, geometry and integers, which comes to light in an interesting manner. In particular, it tells us that geometric patterns may be just relations between integers and the integers themselves can be considered as the ultimate building blocks from which hierarchical formations start. Consequently, the system of linear equations (15) and inequality (16) have two faces, one is geometric and the other is algebraic. With this example in mind we proceed with formal descriptions of the structure.
6
Definition of the Structure and its Isomorphic Representations : Web of Relations
In Sect. 5 for a particular case we have seen how the system of linear equations (12) and inequality (13) can be visualized to generate a heirarchical formation. In the formation an element of each level can be viewed as composed of elements from lower levels and a building block for elements of higher levels. The formation permits two distinctive representations. In the first one elements are integer relations suggesting that ultimate building blocks are just integers from which everything develops as one undivided whole. In the second one elements are two-dimensional geometric patterns. This gives the possibility to visually inspect level by level compositions of these elements and reveals nonlocal order and large symmetry. The example helps to
192 understand that the formation constitutes one possible formation among many others admitted by a whole structure. This naturally motivates us to consider and define this whole structure. It is shown in [8] that the system (12) and inequality (13) can be associated with a heirarchical set WR(s, s', m, In) of k = C(s, s') levels whose elements are integers and integer relations. Elements of the set belonging to a level may be viewed to transform into elements of the neighbouring levels. For example Fig. 11 shows a hierarchical set of integer relations WR(r)(16),f}(16),0,Bie), where 77(16) = + 1 - 1 - 1 + 1 - 1 + 1 + 1 - 1 - 1 + 1 + 1 - 1 + 1 - 1 - 1 + 1 is the initial segment of length 16 of the PTM sequence T] and for convenience the common multiple 2 is omitted in the elements' representation. However, transformations of integer relations are not viewed as a good way to describe how things change. They are more perceived visibly through changes in geometric objects. A result in [8] demonstrates that the set WR(s, s', m, In) is isomorphic to a set of integer patterns WPse(s, s', m, In). This puts the integer relations in a geometric form and gives a picture for their transformations. The language of integer patterns in this picture implies phenomena with changes (see Fig. 12). Interestingly, the integer relation acquires a spatial shape through the isomorphism and can be measured by its area.
Figure 11: A hierarchical set of integer relations W R(r/(16), rj (16), 0, B 1 6 ). Note that relations +16 1 - 15 1 - 141 + 13 1 = 0 and - 1 6 1 + 15 1 + 141 - 13 1 = 0 are different elements. Therefore, based on the system of linear equations (12) and inequality (13) two heirarchical sets of integer relations WR(s, s', m, In) and integer patterns WPSe(s, s', m, In)
193 are defined. We can view elements of the sets as those composed of elements from the lower levels. There is an isomorphism ipSe • WR(s, s', m, /„) ->• WP5e{s, s', m, /„).
(19)
The system (12) viewed by itself exhibits not much grounds to conclude that it is related with a hierarchical formation of two-dimensional geometric patterns. Without the geometric context (12) looks like a curious result of algebra that expresses a property of integers. It seems reasonable to suggest that formal methods have limited applications in deriving properties of the formations such as nonlocal order and large symmetry, because algebraic manipulations of symbols cannot generate their visible images. At the same time when the formations are visualized these properties can be observed directly and each time for a particular pair of sequences we witnessed them. In particular, the system (12) can be easily brought into a form which is well-known in many areas. For simplicity, let s — s\...sn, s' = s[...s'n 6 Bn, s ^ s' then Si — s\, i = l,...,n takes values —2,0,2. This gives that powers (m + n + 1 —i)J', j = 0,..., k — 2 of an integer m+n+1—i, i = 1, ...,n are included in (12) with -I- sign if s; — s[ = +2, with — sign if Si — s[ = —2, and can be seen not to be included if S,— s\ = 0, i = 1,..., n. Let xt € {m + 1,..., m + n} and j/j e {m + 1,..., m + n}, 1 < i < n denote an integer that is the zth (counting from left) integer included in (12) with + and — sign respectively. Then by using the notation and cancelling 2 out, the system (12) can be turned into x\ + ...+xlp = y\ + ... + ylp
xk1-2 + ... + xkp-2 = yk1-2 + - + ykp-2, (20) where in addition it is supposed that C(s, s') = k > 3 and p = q, 1 < p < [f J • The system (20) showed up in many different contexts, under many different guises (for example [9], [10], [11]). However, as known to the authors, there exist no geometric interpretations of (20) leading to the connection with a hierarchical formation of two-dimensional geometric patterns that, when visualized, is believed to exhibit nonlocal order and large symmetry. In this respect the geometric interpretation presented in Sect. 2 indeed plays the crucial role in understanding the system (12).
Now, we finally define a structure that constitutes the main subject of the paper. This structure incorporates all these integer relations into one whole. In particular, the whole structure of these integer relations, denoted WR, is defined by Definition 5. Let
WR{In)= U U U m£Z s£ln
s'£ln
WR(s,s',m,In),
194
Figure 12: Two hierarchical formations of integer patterns and integer relations are displayed together to underline the unified character of elements of the web of relations. The figure allows us to see how integer relations transform. Note that the integer relations can be measured by the area of the integer patterns.
WR{I)
= lim WP6e{In),
WR =
WR{Z),
where Z is the set of integers. By using the isomorphism (19), a structure that for given 6 and e incorporates all integer patterns into one whole can be defined by Definition 6. Let
rPss : WR - • WPSe.
We call WR and WPg£ a web of integer relations and integer patterns accordingly, to informally capture their character. For convenience, due to the isomorphism, we can view both these structures as one structure, called a web of relations, whose elements can be equally seen as integer relations or integer patterns.
195
7
On Descriptive Potentialities of the Structure: Simple Examples of Global Optimization Problems
In this section we show descriptive potentialities of the structure. The descriptions appear as hierarchical formations of the elements of the structure. On one side these formations are represented by trees whose vertices are two-dimensional geometric patterns, i.e., integer patterns. Therefore, means of specification of these trees can be seen as parameters of the descriptions. For example, the area of the integer pattern naturally appears as an important parameter. To make the potentialities more accessible, simple global optimization problems formulated in terms of the structure are considered. We show that well-known geometric objects appear in a new way as solutions to the problems. Due to the simplicity, the geometric interpretation demonstrates how the hierarchical formations arise and allows us to actually see them. Namely, we consider configurations of points in threedimensional cartesian space when they are a pair, equilateral triangle and regular tetrahedron. It is known that these geometric objects are connected with global minimum energy conformations of the scaled Lennard-Jones pair potential of 2,3,4 atoms [12]. This conveniently enables us to view them as molecules with points being van der Waals atoms. We are interested to know how descriptions of these configurations given by the distances between points may be connected with their descriptions in the web of relations. It is supposed that in the web of relations a configuration can be described by using structural numbers ifc-i
Ms)
= T,akmi({m
+ n)is1 + ... + {m + l)isn)ek060,
A; = 1,2,...,
(21)
i=0
where rn,n are integers, s = si...sn € In and <5o,£o are appropriate scales of the ground level of a web of integer patterns WPsoeo- The structural complexity C(s, s'), s' € /„ encodes hierarchical formations corresponding to the configuration. In finding a connection between the descriptions the idea is to identify in the right-hand side of (21) constituents whose mathematical form can also be revealed in the cartesian description. Let h(s) = # { i : st =£ 0, i = 1,..., n} and r(j) denote the value of index i such that the condition Sj ^ 0 is the j t h occurrence when components of s = Si...sn € In are nonzero as the index takes successive values 1, ...,n. For simplicity the dependence of the function r on s in its notation is implicit. Assume that for s = Si...sn £ In in (21) we have s, > 0, i = 1,..., n and Sj ^ 0 for all the indices at once. Then we can transform (21) into the following form
196
h(s) = £ otkmi{x[ + ... + 4)e^o,
(22)
i=0
where for j = 1,..., h(s) we have Xi = (m + n — r(j) + 1) when j-\ 1
j
+ J2 ST(1)
h(s) S
T(1)
and
V = Y. sr(i)-
1=0
j=l
For each integer i = 0,1,... the ith power-sum of x\, ...,xp is 7rjp(i) = x\ + ... + xj, and then (22) can be written as fc-i
^*( s ) = Yl akmiKip(x)£a50-
(23)
i=0
From (23), recalling (6) and Theorem 1, we have a correspondence i?i(s),... ) i? J t(s)^7ro p (x),...,7r t _i J ,(i),
A; = 1,2,....
(24)
Thus, instead of structural numbers of a configuration, we can be interested, due to (24), in their related power-sums. This fact is used as the key as we look for power-sums associated with the cartesian description of a configuration. In particular, let D = {dy, i,j = 1,...,N} be a square symmetric matrix of order N with nonnegative entries, where d^ is the distance between points i and j of a configuration. It follows directly that the trace of D, due to da = 0,i = 1,..., JV, is zero and if A is an eigenvalue of D , then A is real, since the matrix is real and symmetric. D is a matrix and its eigenvalues are rather complicated functions of its entries. The elementary symmetric functions of the eigenvalues however, being the coefficients of the characteristic polynomial P(A) = dei(AI - D) = \N + aiX"-1 + ... + aN,
(25)
are polynomials in the entries of D. Namely, if we denote the roots of (25) by Ax,..., Aw, which are the eigenvalues of D , so that (A - Ai)...(A - Aw) = XN + a i A " - 1 + ... + aN, then as well-known OI,...,OAT are the polynomial functions in \i,...,\N, called the elementary symmetric polynomials, a\ = auv(A), ...,a^ —CT/vw(A)where
a homogeneous polynomial of degree k in Ai,..., A^ and k = 1,..., N. The power-sum 7r*:jv(A) = A* + ... + A^, fc = l,2,...
197 is clearly a symmetric polynomial and it can be expressed by the fundamental theorem on symmetric polynomials as a unique polynomial in the elementary symmetric polynomials. Moreover, the elementary symmetric functions and the power-sums are connected by Newton's identities, which provide a recursive scheme for expressing Tfcjv^), k = 1,..., N in terms ofCTIAT(A),..., o-fcjv(A) [13]. Thus O-lJv(A), ..., OAW(A) <"> TTliv(A), ..., TTjVjvM and there is a connection between D and the power-sums D -> P(A) <-• CTIJV(A), ..., aNN{X) O 7r1JV(A),..., irNN{\).
(26)
, 3
-2
0
-1 4
3
1 2
2 1
h
3 0
4 -
1
5
-
2
Figure 13: The spectral function encodes information about the spectrum (—2, — 1, —1, —1, + 1 , +4) of a matrix. This suggests, by using 7TIAT(A), ..., 7TJVJV(A) and (24) as the key, to define numbers that can be interpreted as structural numbers of the configuration. They are specified by eigenvalues of the distance matrix D , i.e., the spectrum Spec(D) of D. Therefore, the spectrum of the distance matrix is proposed as a means to describe a configuration in the web of relations. We consider configurations when eigenvalues are integers in terms of a length scale. These configurations are a pair, equilateral triangle and regular tetrahedron with distance 1 apart. In these examples, the geometric interpretation serves as the main tool to show how hierarchical formations corresponding to these configurations arise from the spectrum. As the result a connection between the descriptions is found. In particular, interestingly, it turns out that the area of an integer pattern corresponding to a configuration equals the absolute value of the determinant of its distance matrix. Now, we show how structural numbers can be developed from an integer spectrum. It is possible to make it with the help of a spectral function. In particular, given a
198 spectrum Spec(D) = (Ai£,..., A^e) of a distance matrix D describing a configuration of N points, where e > 0 is a length scale of distances in D and Xt £ Z, i = 1,..., N, we define a function, called spectral and denoted S ( D , t ) , that represents information about it. Consider the partition Ae[tm,tm+n] of an interval [tm,tm+n], where n=
max A; — min A; + 1 > 2, t=l,...,AT
i=l,...,AT
m=
_
min A; — 1, i=l,...,JV
then the spectral function S(D,t)
: [tm,tm+n]
—>• 5ft
of D is defined in an interval [t2m+n-k,hm+n-k+i),
k = max Aj, max Aj - 1,..., min A* 1=1,...,N
1=1,. ..,N
1=1,.. .,N
as the number of eigenvalues that are equal to k. The value of the function at the point tm+n is equal to its value in the interval [tm+n-i,tm+n). For example, in Fig. 13 a spectral function S(D,t) of a matrix D such that Spec(D) = ( - 2 , - l , - l , - l , + l , + 4 ) is shown. It follows from the definition that S(D, t) £ H / i(A e [i m , tm+n]) and the code of the spectral function is a sequence s = S\...sn € In given by S(D,t) = pmu(s), where 8 is dimensionless and equals 1. Then structural numbers of the sequence s = si...sn € /„ are *-i
Ms)
= Y,akrni{si(m
+ n)i + ... + sn(m + l)i)£k,
k = l,2,...
(27)
i=0
where S; is the number of eigenvalues which are equal to (m + n — i + 1), i = 1,..., n. This implies that (27) can be rearranged to give fc-l
Ms)
= E
k-l
a
kmi(K + - + XN)ek = X) akmTriN{X)ek,
i=0
k = 1,2,... .
i=0
Therefore, by taking into account (26), we have D -> P(A) ^am(\),...,aNN{\) where <7ON{^) = ^ON{^)
^7r0N(\),...,7rNN(X)
«t? 1 (s),...,i) J V + i(s),
= ^V- This leads to a connection between the descriptions.
Let the length scale of distances between points be 1, i.e., e = 1. We have the following matrix of distances between two points of the pair
199
Figure 14: Two different descriptions in one picture. The structural complexity of the sequence s = 101 uncovers a heirarchical formation corresponding to the pair.
and eigenvalues of the matrix are Ai = —1,A2 = + 1 . The graph of the spectral function 5 ( D 2 , t), t € [—2,1] is given in Fig. 14 and its code is a sequence s = 101 £ ^3, where / = {0,1,2}. Consider hierarchical formations that in the web of relations WP\\ correspond to D2 and are encoded by the structural complexity C(s,s') = max C(s,s")
(28)
s"£h
of the sequence s. It can be found by calculations that a sequence s' = 020 € J3 is the unique solution to the global optimization problem (28). This sequence s' together with the sequence s give the hierarchical formation shown in Fig. 14. The figure exhibits an integer pattern P ( / ' 2 ' , [—2,1]), / = p_ 2 n(s — s') which can be seen as the result of the formation. The system (12) in this case takes the form + l ° - 2 * 0 ° + (-l)° = 0 + l 1 - 2 * 0 1 + ( - l ) 1 = 0.
(29)
We show that the descriptions are closely related in this case. Note, that there is an interesting connection between the product of the eigenvalues, i.e., the determinant of the distance matrix D 2 , and the area of P ( / [ 2 ] , [-2,1]). Namely, we have det(D 2 ) = AiA2 = (-1)(+1) = - 1 .
200 The area of P(f^2\ [—2,1]) in its turn is the third structural number of a sequence a" = s — s' = s'ls'^s'z = + 1 — 2 + 1 which, according to (1), equals 2
Figure 15: This hierarchical formation arises from pinpointing the structural complexity of the sequence s = 1002. This is a global optimization problem. It looks like the sequences s, s' are defined by their correspondence to the resulting integer pattern in the web of relations. Making use of (29), we obtain
Ms") = «3,-2,2(i2 + o2 * (-2) + (-i)2) = I Q ( ( - I A - I ) 1 + ( - I ) 1 ^ ) 1 ) = i. Thus |dct(D 2 )| = Ms")
= area of(P(/M, [-2,1])).
We have the following matrix of distances triangle /0 D3 = 1 \ i
(30)
between three points of the equilateral 1 1\ 0 1 i o;
and eigenvalues of the matrix are Ai = —1, A2 = —1, A3 = 2. The graph of the spectral function 5 ( D 3 , i ) , i G [—2,2] is given in Fig. 15. The code of the spectral function is a sequence s = 1002 € 74, where I = {0,1,2,3}. Consider hierarchical formations
201 that in the web of relations WPu correspond to D 3 and are encoded by the structural complexity C{s,s') = max C(s,s") (31) of the sequence s. It can be found by calculations that a sequence s' = 0111 G I\ is the unique solution to the global optimization problem (31) and together with the sequence s gives the hierarchical formation shown in Fig. 15. The figure exhibits an integer pattern .P(/[ 2 ', [—2, 2]), / = p_2ii(s — s')> which can be seen as the result of the formation. The system (12) in the case takes the form + 2
°-l0-0
0
+ (-l)0 = 0
+ 2 1 - l 1 - 0 1 + ( - l ) 1 = 0. We have det(D3) = AiA2A3 = ( - l ) ( - l ) ( + 2 ) = 2. The area of P ( / ' 2 ' , [—2,2]) is the third structural number of a sequence s" = s - s' = s s s s 'i 2 3 4 = + 1 - 1 - 1 + 1, which equals
Us") =J2a3^2,i(2's'l + 1*4 + 0V3' + (-l)X') = 2. 2=0
Thus |det(D 3 )| = Ms")
= area of(P(/l 2 l, [-2,2])).
(32)
Along similar lines, we have the following matrix of distances between four points of the tetrahedron f0111 \ 11 10 D4 1 1 0 1 Vi i i o / and eigenvalues of the matrix are Ai = —1, A2 = —1, A3 = —1, A4 = 3. The graph of the spectral function 5(D4, t ) , i £ [—2,3] is given in Fig. 16. The code of the spectral function is a sequence s = 10003 € ^5, where / = {0,1,2,3, 4}. Consider hierarchical formations that in the web of relations WPu correspond to D 4 and are encoded by the structural complexity C(s,s') = rriax C(s,s") (33) of the sequence s. It can be found by calculations that a sequence s' = 01012 e IA is a solution to the global optimization problem (33) and together with the sequence s gives the hierarchical formation shown in Fig. 16. The figure exhibits an integer
202 pattern P(fl2\ [—2,3]), / = p_2ii(s — s'), which can be seen as the result of the formation. The system (12) in this case takes the form +3° - 2° - 0° + (-1)° = 0 + 3 1 - 2 1 - 0 1 + (-l)1 =0.
Figure 16: The picture shows a connection between the tetrahedron, i.e., the geometric object, and the relation + 3 1 — 2 1 — 0 1 + (—l) 1 = 0. In all these cases the area of the upper integer pattern equals the absolute value of the determinant of the distance matrix. We have det(D 4 ) = A!A2A3A4 = - 3 . 2
The area of P(p ', [—2,3]) is the third structural number of a sequence s" = s — s' = s'(s'i4s'l4 = + 1 - 1 0 - 1 + 1, which equals Ms")
= £ a3,-2,i(3V; + ?4 + 1*4' + Ws'l + (-1)V 5 ') = 3.
Thus \det(D4)\ = Ms")
= area of(P(/M, [-2,3])).
(34)
In all these cases we find an interesting connection between the descriptions. The connection consists of the fact that the important general invariant of the distance matrix of a configuration, i.e., the determinant, is expressed, as we observed in (30), (32) and (34), in terms of the web of relations. Namely, the area of an integer pattern belonging to the highest level equals the absolute value of the determinant of the distance matrix.
203
Acknowledgements This work was supported in part by CQU URG Grant "Computational Optimisation of Large Molecular Structures".
References V. Korotkich, Integer Code Series: Applications in Dynamic Systems and Complexity, The Computing Center of Russian Academy of Sciences, Moscow, 1993. V. Korotkich, Multicriteria Analysis in Problem Solving and Structural Complexity, in Advances in Multicriteria Analysis, P. Pardalos, Y. Siskos and C Zopounidis (editors), Kluwer Academic Publishers, 1995, pp. 81-90. E. Prouhet, Memoire sur Quelques Relations entre Its Puissances des Nombres, C. R. Acad. Sci. Paris, 33, 1851, p. 225. A. Thue, Uber unendliche Zeichenreihen, Norske vid. Selsk. Skr. I. Mat. Nat. Kl. Christiana, 7, 1906, p. 1 (Reprinted in Selected Mathematical Papers of Axel Thue, T. Nagell (editor), Universitetaforlaget, Oslo, 1977. M. Morse, Recurrent Geodesies on a Surface of Negative Curvature, Trans. Amer. Math. Soc, 22, 1921, p. 84. V. Korotkich, Symmetry in Structural Complexity and a Model of Formation, in From Local Interactions to Global Phenomena, IOS Press, Amsterdam, 1996, pp. 84-95. M. Gardner, Penrose Tiles to Trapdoor Ciphers, W. H. Freeman and Company, New York, 1989. V. Korotkich, On a Structure of Integer Relations and its Geometric Interpretation, Central Queensland University, Technical Report 97-004, 1997. S. Ramanujan, Note on a Set of Simultaneous Equations, J. Indian Math. Soc, 4, 1912, pp. 94-96. L. Mordell, On a Sum Analogous to a Gauss's Sum, Quart. J. Math., 3, 1932, pp. 161-169. N. Korobov, Trigonometric Sums and their Applications, Nauka, Moscow, 1989. M. Hoare, Structure and Dynamics of Simple Microclusters, Advances in Chemical Physics, 40, (1979), p. 49.
204
[13] I. MacDonald, Symmetric Functional and Hall Polynomials, The Camelot Press Ltd., Southhampton, 1979.
Combinatorial and Global Optimization, pp. 205-236 P.M. Pardalos, A. Migdalas and R. Burkard, Editors © 2002 World Scientific Publishing Co.
Heuristic Solutions of Vehicle Routing Problems in Supply Chain Management Yannis Marinakis ([email protected]) DSS Laboratory Department of Production Engineering and Management Technical University of Crete 73100 Chania, Greece
Athanasios Migdalas ( s a k i s @ v e r e n i k e . e r g a s y a . t u c . g r ) DSS Laboratory Department of Production Engineering and Management Technical University of Crete 73100 Chania, Greece
Abstract The distribution of commodities, known by the generic name vehicle routing problem, is one of the most important components of supply chain. The vehicle routing problem, which is a hard combinatorial problem, has therefore attracted considerable research attention and a number of algorithms has been proposed for its solution. In this paper we present an extensive review of heuristic solution techniques for the vehicle routing and traveling salesman problems. Keywords: Supply Chain Management, Vehicle Routing Problem, Traveling Salesman Problem, Distribution Problem, Combinatorial Optimization, Heuristics
206
1
Introduction
In this paper we present a review of heuristic algorithms for vehicle routing and traveling salesman problems. We focus on heuristic algorithms because no exact algorithm can be guaranteed to find optimal tours within reasonable computing time when the number of cities is large. This is due to NP-hardness of the problems. The structure of the paper is as follows: In Section 2, we present some basic characteristics of the supply chain management and we focused on the role of transportation problems. In Section 3, we present the traveling salesman problem, the vehicle routing problem and a few extensions of the latter. In Section 4, we describe classic heuristics for the solution of the mentioned problems. In Section 5, we review modern techniques. Section 6 is devoted to computational comparisons between the reviewed algorithms.
2
Supply Chain Management
A complete logistics [5, 25] system covers the entire process of shipping raw materials and input requirements from suppliers to plants, the conversion of the inputs into products at certain plants, the shipping of the products to various warehouses or depots, and the eventual delivery of these products to the final customers. The distribution activities of a firm comprise all shipping and storage of goods downstream from the plants. We classify the decisions for supply chain management into two broad categories — strategic and operational. Strategic decisions are made typically over a longer time horizon. On the other hand, operational decisions are short term, and focus on activities over a day-to-day basis. There are four major decision areas in supply chain management, and there are both strategic and operational elements in each of these areas: 1. Location. 2. Production. 3. Inventory. 4. Transportation — Distribution.
2.1
Transportation decisions
Transport decisions [5] can involve mode selection, shipment size, and routing and scheduling. These decisions are influenced by the proximity of warehouses to cus-
207 tomers and plants, which, in turn, influence warehouse location. Inventory levels also respond to transport decisions through shipment size. Transportation often represents the most important single element in logistics costs for most firms. Transportation is a key decision area within the logistics mix. Except for the cost of purchased goods, transportation absorbs, on the average, a higher percentage of logistics costs than any other relevant activity. Because transportation costs typically range between one third and two thirds of total logistics costs, improving efficiency through the maximum utilization of transportation equipment and personnel is of a major interest. The time that goods are transit reflects on the number of shipments that can be made with a vehicle within a given period of time and on the total transportation costs for all shipments. To reduce transportation costs and also to improve customer services finding the best — in terms of time or distance minimization — routes that a vehicle should follow through a network of roads, rail and other shipping lines is frequently an important decision problem.
3
The Vehicle Routing Problem
The distribution or vehicle routing problem (VRP) is often described as the problem in which vehicles based on a central depot are required to visit geographically dispersed customers in order to fulfill known customer demands. The problem is to construct a low cost, feasible set of routes — one for each vehicle. A route is a sequence of locations that a vehicle must visit along with the indication of the serve it provides [9]. The vehicle must start and finish its tour at the depot. We can say that the problem arises as a generalization of the traveling salesman problem. The traveling salesman problem (TSP) requires the determination of a minimal cost cycle that passes through each node of a given graph exactly once [46]. If costs are symmetric, that is, if the cost of traveling between two locations does not depend on the direction of travel, we have a symmetric TSP, otherwise we have an asymmetric TSP. The multiple traveling salesman problem arises if many salesmen or vehicles in the fleet are to leave from and return to the same depot. There are no restrictions on the number of nodes that each vehicle must visit except that each vehicle must visit at least one node. The vehicle routing problem has a variety of additional constraints and extensions that are often found in real-world problems. These include the following [16] • Each vehicle can operate on more than one route, provided that the total time spent on these routes is less than a given bound T. • Each customer must be visited within a specific time interval, known as time
208 window. • The problem may involve both deliveries to and collections from customers. In addition, it may be possible to mix deliveries and collections on a single route, or alternatively, it may be required from a vehicle to first perform all the deliveries in the route before performing the collections. • Vehicles may also be associated with time windows within which they are allowed to operate.
3.1
Variants of the vehicle routing problem
In this section we present some variants of the vehicle routing problem, namely, the multiple depot vehicle routing problem, the multiple commodities vehicle routing problem, the stochastic vehicle routing problem, the vehicle scheduling problem and others. Many applications [50, 60] involve pickup and delivery services between the depot and peripheral locations (warehouses, stores, stations). 'Delivery' refers to transportation of goods from the depot to customers, and 'pickup' means shipment in the opposite direction (to the depot). In the literature this problem is known as pick-up and delivery problem (PDP) or vehicle routing problem with back-hauls (VRPB) [63, 65]. The objective is to find a set of vehicle routes that serve the delivery and backhaul customers so that vehicle capacities are not violated and the total distance traveled is minimized. In companies [8, 16] with more than one depot, it is often the case that each depot is autonomous, with its own fleet of vehicles and its own geographical customer area to serve. In such cases, the company would simply face a number of similar single depot vehicle routing problems. In other cases depot operations are interdependent and vehicles leaving one depot may, after delivering to customers, end up at another depot. These problems are called multiple depot vehicle routing problem. In some cases [16], the vehicles are compartmented so that different commodities are stored in segregated compartments. Each customer may require specific quantities of different types of commodities. The problem is then called multiple commodities vehicle routing problem. The problem of managing routing and dispatch operations under conditions of random demand fluctuations, is often referred to as the stochastic routing problem [34]. The stochastic vehicle routing problem is usually formulated under the following conditions [9]: 1. Customer demand is a random variable with a known probability distribution.
209 2. Routes must be designed before the actual demands become known. 3. The objective is to minimize expected travel distance. Vehicle scheduling problems can be thought of as routing problems with additional constraints imposed by time periods during which various activities may be carried out [9]. Three constraints commonly determine the complexity of the vehicle scheduling problems: • The length of the time that a vehicle may be in operation before it must return to the depot for service or refueling. • The fact that certain tasks can only be carried out by certain vehicle types. • The presence of a number of depots where vehicles may be housed. In the school bus routing and scheduling problem [8, 10], there are a number of schools, each having being assigned a set of bus stops, with a given number of students assigned to each stop, and time windows for the pickup and the delivery of the students. The problem is to minimize the number of buses used or total transportation costs while serving all the students and satisfying all the time windows. In the problem of routing and scheduling with full loads and time windows, a set of demands is specified for a number of origin-destination pairs. Each demand is a full trailer which must be loaded onto a tractor at an origin and unloaded at a destination. These stops must satisfy prespecified time window constraints and the goal is to design routes and schedules for the fleet of tractors. In most cases, the objective is to minimize total transportation costs or the number of tractors used. In the multi-vehicle covering tour problem [33] we are given two sets of locations. The first set, V, consists of potential locations at which some vehicles may stop, and the second set, W are locations not actually on the vehicle routes but within an acceptable distance of a vehicle route. The problem is to construct several vehicle routes through a subset of V, all starting and ending at the same locations, subject to some side constraints, having a total minimum length, and such that every location of W is within a reasonable distance of a route. An important aspect [7] of the vehicle routing problem that has been largely overlooked is the use of satellite facilities to replenish vehicles during a route. When possible, satellite replenishment allows the drivers to continue making deliveries until the close of their shift without necessarily returning to the central depot. This situation arises primarily in the distribution of fuel and certain retail items. The vehicle routing problem with trailers [28] is concerned with the case where a vehicle consists of a truck and a trailer. Both can carry goods. The use of truck and
210 trailer may cause problems when serving customers that are located in the center of a city or customers who have little space nearby for maneuvering the vehicle. Time and trouble could be saved if these customers were served by the trucks only.
4
Classic Heuristics for the Traveling Salesman and t h e Vehicle Routing Problems
In this section we examine three different classes of heuristics — tour construction procedures, two phases algorithms and tour improvement procedures. Tour construction procedures generate an approximately optimal tour for the distance matrix. Two phases algorithms, like cluster first - route second procedures, which during the first phase group or cluster demand nodes and then, during a second phase, design economical routes. Tour improvement procedures attempt to find a better tour given an initial one.
4.1 4.1.1
Constructive methods Nearest Neighbor Procedure
In this heuristic, the salesman starts at some city and then visits the city nearest to the starting one. From there, he visits the nearest unvisited city, until all cities are visited, and then returns to the starting city [54, 59]. Step 1. Start with any node as the beginning of a path. Step 2. Find a node, not already on the path, which is nearest to the last added node. Add it to the path. Step 3. Repeat Step 2 until all nodes belong to the path. Then, join the first and the last nodes of the path. For symmetric, complete graphs, the worst case behavior of the algorithm is length of nearest neighbor tour 1 length of optimal tour ~ 21
0g2(n)l +
1 2'
where \x] is the smallest integer > x, and n is the number of nodes in the network. The nearest neighbor algorithm is of 0(n2) time complexity.
211 4.1.2
Insertion Procedures
The insertion procedures [30, 59] takes a subtour of k nodes and attempts to determine which node (not already in the subtour) should join the subtour next (the selection step) and then determines where in the subtour it should be inserted (the insertion step). The most known of these algorithms is the nearest insertion algorithm: Step 1 Start with a subgraph consisting of only one node, say i. Step 2 Find node k such that en- is minimal and form the subtour i — k — i. Step 3 (Selection) Given a subtour, find node k, not already in the subtour, closest to any subtour node. Step 4 (Insertion) Find the arc (i, j) in the subtour which minimizes Cik + c^j — ci}•. Insert k between i and j . Step 5 Go to Step 3 unless a Hamiltonian cycle has been formed. The worst case behavior of the algorithm is length of nearest insertion tour length of optimal tour ~~ and its time complexity
0(n2).
Similar to the nearest insertion procedure are the cheapest insertion [54], the arbitrary insertion [9], the farthest insertion [54], the quick insertion [9], and the convex hull insertion [9] algorithm.
4.1.3
Clarke and Wright Algorithm
This savings algorithm is an exchange procedure [17, 30, 44], that was originally developed for the VRP. It can, however, be applied to the TSP as well: Step 1. Select any node as the central depot and denote it as node 1. Step 2. Compute the savings Sy = Cy + cu — Cij for i, j = 2,3, • • •, n. Step 3. Order the savings from largest to smallest. Step 4. Starting at the top of the savings list and moving downwards, form larger subtours by linking appropriate nodes i and j . Repeat until a tour is formed.
212 Next the Clarke and Wright algorithm for the solution of the VRP [16] is given: Step 1. Calculate the savings s,j = cy + Cu — Cij for all pairs of customers i and j . Note that s^ is the saving in cost that would result if the link (i,j) is made to produce the route (l,i,j,l) instead of supplying i and j on two routes (1, i, 1) and ( l , j , 1). Step 2. Order the savings in descending order. Step 3. Starting at the top of the list do the following. Parallel version Step 4. If a given link results in a feasible route according to the constraints of the VRP, then append this link to the solution, if not reject the link. Step 5. Try the next link in the list and repeat Step 4 until no more links can be chosen. Sequential version Step 4. Find the first feasible link in the list which can be used to extend one of the two ends of the currently constructed route. Step 5. If the route cannot be expanded further, terminate the route. Choose the first feasible link in the list to start a new route. Step 6. Repeat Steps 4 and 5 until no more links can be chosen. The worst case behavior for this approach is bounded by a linear function in log2(n). The calculation of the matrix sy- in Step 2 requires about en2 computations for some constant c. Next, in Step 3 savings can be sorted into nonincreasing order via Heapsort method in a maximum of cn2lg(n) computations. Step 4 involves at most n 2 computations. Thus, the Clark and Wright savings procedure requires an order of n 2 log 2 (n) computations. 4.1.4
Nearest Merger Algorithm
The nearest merger method [59] when applied to a TSP of n nodes constructs a sequence Si, • • •, Sn such that each Si is a set of n — i + 1 disjoint subtours covering all the nodes. Step 1. Si consists of n subtours, each containing a single node.
213 Step 2. For each i < n, find an edge (a^bi) such that caj&; = min{cxy for x and y in different subtours in Si}. Then S i + 1 is obtained from S* by merging the subtours containing aj and 6j.
The worst case behavior of the algorithm is length of nearest merger tour length of optimal tour ~~ This approach requires an order of n 2 computations.
4.1.5
Christofides Algorithm
Christofides [9, 54] suggested a method of transforming spanning trees to Eulerian graphs. For this, it is sufficient to add a perfect matching on the odd-degree nodes of the tree. After adding the perfect matching edges, all node degrees are even and hence the graph is Eulerian. The following procedure for the solution of the TSP can be based on this:
Step 1. Find a minimal spanning tree T of the given graph G. Step 2. Identify all the odd degree nodes in T. Solve a minimum cost perfect matching on the odd degree nodes using the original cost matrix. Add the edges from the matching to the edges of T to obtain an Euler cycle. In this subgraph every node is of even degree although some nodes may have degree greater than 2. Step 3. Remove polygons over the nodes with degree greater than 2 and transform the Euler cycle into a Hamiltonian cycle.
The worst case behavior of the algorithm is length of Christofides tour length of optimal tour ~~ Since most of the computation time is consumed by the minimum matching subroutine, this heuristic is 0(n3). However, the number of odd nodes may be considerable less than n.
214
4.2
2-phase Algorithms
In two-phase methods, the first phase consists of clustering the customers by assigning them to vehicles, and the second phase routes these clusters. The best known algorithms of this category are the sweep algorithm, the Mole and Jameson algorithm and the two-phase method of Fisher and Jaikumar.
4.2.1
The Sweep Algorithm
The sweep algorithm was originally devises by Gillet and Miller [9]. This approach constructs a solution in two stages. First, it assigns nodes to vehicles and then it sequences the order in which each vehicle visits the nodes assigned to it. The procedure is as follows [17, 16, 44]: Phase I Step 1. Choose an unused vehicle k. Step 2. Starting from the unrouted customer i with smallest angle #;, include consecutive customers i + 1, i + 2, • • • in the route until the capacity constraint of the vehicle k is reached. Step 3. If all customers are swept or if all vehicles have been used, go to phase II, else return to Step 1. Phase II Step 4. Solve a TSP for every set of customers assigned to a vehicle to form the final routes.
4.2.2
M e t h o d of Mole & Jameson
The algorithm of Mole and Jameson is a sequential procedure [17] in which, for a given value of two parameters A and fi, the following two criteria are used to expand a route under construction: e(M,J) a{i,l,j) The algorithm proceeds as follows:
= ca + ctj - \xcij = Xcoi-e(i,l,j)
215 Step 1. For each unrouted customer x; compute the feasible insertion in the emerging route R as e(ii,l,ji)
= min[e(r,/, s)]
for all adjacent customers xr,xs £ R, where Xj, and Xjt are customers between which xi results in the best insertion. Step 2. The best customer xi* to be inserted in the route is computed as the one for which <7(*i*.**.>)
=
max[a(ii,l,ji)},
where xi is unrouted and feasible. Step 3 . Insert xi* in route R between x^ and Xj*. Step 4. Optimize route R using r-optimal methods (§4.3). Step 5. Return to Step 1 to start a new route JR either until all customers are routed or no more customers can be routed.
4.2.3
M e t h o d of Fisher & Jaikumar
The first [22, 23] phase of this algorithm performs a parallel clustering by solving optimally a generalized assignment problem. Phase I Step 1. Choose m customers to be seeds of clusters and allocate a vehicle to each. Step 2. For each customer i and for each cluster k, compute an insertion cost dik relative to the seed of the cluster. Step 3. Solve a corresponding generalized assignment problem (GAP) to obtain clusters. Phase II Step 4. Solve a TSP for every set of customers in the clusters implied by the solution to GAP.
216
4.3
Improving heuristics or local search heuristics
A local search algorithm [1] is built around a neighborhood search procedure. Given a tour the algorithm examines all the tours that are "neighboring" to it and tries to find a shorter one. The definition of "neighbor" varies with the details of the particular local search heuristic [39]. The overall process goes as follows: Starting with some initial tour, chosen arbitrary or generated by some other heuristic, if there is no neighboring tour which is shorter than the original one, the process halts. The initial tour is a local optimum with respect to the chosen neighborhood. Otherwise, a shorter neighbor of the original tour is used as a new starting point, and the process is repeated. The method must eventually terminate as there is only a finite number of possible tours. A neighborhood N for a problem instance (5, / ) , where S is the set of feasible points and / the objective function, can be defined as a mapping from S to its power set, i.e., N : S —» 2 5 . N(s) is called the neighborhood of s € 5 , and contains all the solutions that can be reached from s by a single move. Here, the meaning of a move is that of an operator which transforms one solution to another with small modifications. A solution x is called a local minimum of / with respect to the neighborhood N if
f(x)
4.3.1
2-opt Method
A 2-opt procedure [47, 54], in general, consists of eliminating two edges and reconnecting the two resulting paths in order to obtain a new tour. Note that there is only one way to reconnect the paths. The 2-opt procedure was introduced by Lin (1965) for the TSP: Step 1. Let T be the current tour. Step 2. For every node i = 1, 2, • • •, n: Examine all 2-opt moves involving the edge between i and its successor in the tour. If it is possible to decrease the tour length this way, then choose the best such 2-opt move and update T.
217 Step 3. If no improving move could be found, then stop. In the worst case, it can only be guaranteed that an improving move decreases the tour length by at least one unit. No polynomial worst case bound for the number of iterations to reach a local optimum can be given. Checking whether an improving 2-opt move exists takes 0(n2) time.
4.3.2
3-opt Method
The 3-opt heuristic is quite similar to the 2-opt but it introduces more flexibility in modifying the current tour, because it uses a larger neighborhood. The tour breaks into three parts instead of only two. There are eight ways to connect the resulting three paths in order to form a tour. There are I „ I ways to remove three edges from a tour.
4.3.3
Lin - Kerningham Algorithm
This algorithm was developed by Lin and Kernigham (LK) [48, 35] and for many years was considered to be the best heuristic for the TSP. The LK-algorithm decides dynamically at each iteration the number of edges that should be exchanged. The procedure works as follows [32, 48]: Step 1. Generate a random initial solution T. Step 2. Step 2.1. Set the iteration counter i = l. Step 2.2. Select xt and yt as the most out of place pair at the i-th iteration. This generally means that X{ and yi are chosen to maximize the improvement when X\, • • • ,Xi are exchanged with j/i, • • •, y,. Xi is chosen from T — {xi, • • •, x^} and y, from S — T — {x%, • • •, X(}. Step 2.3. If it appears that no more gain can be made, go to Step 3. Otherwise, set i = i + 1 and go back to Step 2.2. Step 3. If the best improvement is found for i = k, exchange x\,---,Xk with V\i' •• iVki to obtain a new T, and go to Step 2. If no improvement was found, go to Step 4. Step 4 (Multistart). Repeat from Step 1 if desired. Otherwise, terminate.
218 4.3.4
Or-opt
The Or-opt procedure, well known as node exchange heuristic, was first introduced by Or [66]. It removes a sequence of up-to-three adjacent nodes and inserts it at another location within the same route. Or-opt can be considered as a special case of 3-opt (three arcs exchanges) where three arcs are removed and substituted by three other arcs. When removing a chain of consecutive nodes in Or-opt, two arcs are deleted, and the third is eliminated when inserting the chain back into the route. However, the number of possible Or-exchanges is far less than that of possible 3-exchanges. Or-opt is also shown to produce improved solutions of comparable quality to those produced by 3-opt, while requiring significantly less computational time. Or-opt algorithm can be described as follows [43]: Step 1. Consider an initial tour and set t = 1 and s = 3. Step 2. Remove from the tour a chain of s consecutive vertices, starting with the vertex in position t, and tentatively insert it between all remaining pairs of consecutive vertices on the tour. Step 2 . 1 . If the tentative insertion decreases the cost of the tour, implement it immediately, thus defining a new tour. Set t = 1 and repeat Step 2. Step 2.2. If no tentative insertion decreases the cost of the tour, set t = t + 1. If £ = n + 1 then proceed to Step 3, otherwise repeat Step 2. Step 3 . Set t = 1 and s = s — 1. If s > 0 go to Step 2, otherwise stop.
4.3.5
G E N I and G E N I U S
The GENI algorithm was presented by Gendreau, Hertz and Laporte (1992) [40]. GENI is a hybrid of tour construction and local optimization. Suppose that cx, Ci, • • •, c/v is an arbitrary ordering of the cities. Starting with the partial tour consisting of the first three cities, ci, C2, c3, new cities are added to the current tour in the order given, starting with c 4 . To add city ct, possible ways of inserting it into the tour are considered as well as a 3-opt or 4-opt move with the C; as the endpoint of one of the deleted edges. The range of possibilities is restricted by requiring that certain of the inserted edges link cities to members of their nearest neighbor lists, where only cities currently in the tour qualify to be in such lists. Also the list lengths are constrained to a maximum p. GENIUS is a true local optimization algorithm based on principles similar to GENI's. Given a starting tour generated by GENI, it cycles through the cities looking for improvements.
219
5
5.1
Metaheuristics for the Traveling Salesman and t h e Vehicle Routing Problems Simulated annealing
Simulated annealing (SA) belongs [19] to a class of local search algorithms that are known as threshold algorithms. These algorithms play a special role within local search for two reasons. First, they appear to be quite successful when applied to a broad range of practical problems. Second, some threshold algorithms such as SA have a stochastic component, which facilitates a theoretical analysis of their asymptotic convergence. The approach of SA originates from theoretical physics, where Monte-Carlo methods are employed to simulate phenomena in statistical mechanics. Its predecessor is the so-called Metropolis filter. This simulation method can be motivated as follows. Consider a huge number of particles of fixed volume at some temperature #. Since the particles move, the system can be in various states. The probability that the system is in a state of certain energy E is given by the Boltzmann distribution f(E) = —S& 5 , where z($) is a normalization factor and KB is the Boltzmann constant. This distribution characterizes the statistical equilibrium of the system at temperature i9. In the following we present a simulated annealing procedure for the TSP [54]: Step 1. Compute an initial tour T and choose an initial temperature d > 0 and a repetition factor r. Step 2. If the stopping criterion is not satisfied: Step 2.1. Do the following r times. Step 2.1.1. Perform a random modification of the current tour to obtain the tour (V) and let A = c(T') - c(T). Step 2.1.2. Compute a random number x, 0 < x < 1. Step 2.1.3. If A < 0 or x < e x p ( ^ ) . Step 2.2. Update d and r. Step 3. Output the current tour T as solution. 5.1.1
Algorithm by Alfa, Heragu and Chen
The method described by Alfa, Heragu and Chen [26] can be viewed as route first cluster second algorithm. A giant tour is first constructed without considering weights and vehicle capacity, and then it is partitioned into segments of consecutive vertices
220 whose total weight does not exceed a given capacity Q. It is implicitly assumed that the number of vehicles is not fixed a priori. Starting with the initial tour, at iteration t, three edges are randomly selected and removed from the tour, as in classic 3-opt. 5.1.2
Osman's Simulated Annealing Algorithm
The simulated annealing implementation proposed by Osman [26] is substantially more involved in several aspects: 1. It uses a better starting solution (using the Clarke and Wright algorithm). 2. Some parameters of the algorithm are adjusted in a trial phase. 3. Richer solution neighborhoods are explored using A-interchanges (See §5.2.2). 4. The cooling scheduling is more sophisticated. The reader can find other algorithms based on simulated annealing in the articles [11, 12, 41, 64].
5.2
Tabu search
Tabu search (TS) was introduced by Glover [29]. Computational experience has shown that TS is a well established approximation technique [36], which can compete with almost all known techniques and which, by its flexibility, can beat many classic procedures. TS combines [1] the deterministic iterative improvement algorithm with a possibility to accept cost-increasing solutions. Thus, the search is directed away from local minima so that other parts of the search space can be explored. The next solution visited is always chosen to be a legal neighbor of the current solution with the best cost, even if that cost is worse than that of the current solution. The set of legal neighbors is restricted by a tabu list designed to prevent the search from cycling. The tabu list is dynamically updated during the execution of the algorithm. The tabu defines solutions that are not acceptable in the next few iterations. However, a solution in the tabu list may be accepted if its quality is in some sense good enough, in which case it is said to attain a certain aspiration level. A general description of TS is the following: Step 1. Choose an initial solution x. Set x* — x (the best solution so far) and k =0 Step 2. Set k = k + 1 and generate a subset V* of solutions in N(x,k), neighborhood of x at iteration k.
the
221 Step 3. Choose a best y e V* with respect to the objective function / or some modified objective criterion Z1 and set x = y. Step 4. If f(x) < f(x*), then set x* = x. Step 5. If a stopping condition is met, then stop. Else go to Step 2.
5.2.1
Willard's Algorithm
One of the first attempts to apply tabu search to the VRP is due to Willard [26]. Here the solution is first transformed into a giant tour by replication of the depot, and neighborhoods are defined as all feasible solutions that can be reached from the current solution using 2-opt or 3-opt exchanges. The next solution is determined by the best non-tabu move.
5.2.2
Osman's Algorithm
Osman [26] again defines neighborhoods using a A-interchange scheme. This includes a combination of 2-opt moves, vertex reassignments to different routes, and vertex interchanges between two routes. In one version of the algorithm, called best-admissible, the whole neighborhood is explored and the best non-tabu feasible move is selected. In another version, called first-best-admissible, the first admissible improving move is selected if one exists.
5.2.3
TABUROUTE
The TABUROUTE [26] algorithm involves several innovative features: • The neighborhood structure is defined by all solutions that can be reached from the current solution by removing a vertex from it and inserting it into another route in its neighborhood using GENI (see section 4.3.5). • The search procedure examines solutions that may be infeasible with respect to the capacity or maximum route length constraints. • TABUROUTE does not actually use a tabu list, but random tabu tags. • TABUROUTE uses a diversification strategy. This is achieved by penalizing vertices that have been moved frequently in order to increase the probability of considering slow-moving vertices.
222 5.2.4
Taillard's Algorithm
The Tailard tabu search [26] implementation contains some of the features of the algorithm presented in section 5.2.3, namely, random tabu durations and diversification. It defines neighborhoods using A-interchanges like the algorithm presented in section 5.2.2. Rather than executing insertions with GENI, the algorithm uses standard insertions. 5.2.5
T h e X u and Kelly Algorithm
Xu and Kelly [45] used a more sophisticated neighborhood structure. They consider swaps of vertices between two routes, a global repositioning of some vertices into other routes, and local route improvements. The global repositioning strategy solves a network flow model to optimally relocate given numbers of vertices into different routes. Route re-optimizations are performed by means of 3-opt exchanges and a TS improvement routine. The algorithm is governed by several parameters which are dynamically adjusted through the search. A pool of best solutions are memorized and periodically used to re-initiate the search with new parameter values.
5.2.6
Adaptive M e m o r y Procedure
One of the most interesting developments to have occurred in the area of TS in recent years is the concept of Adaptive Memory developed by Rochat and Taillard [31, 45, 57]. It is mostly used in TS, but its applicability is not limited to this type of metaheuristic. An adaptive memory is a pool of good solutions that is dynamically updated throughout the search process. Periodically, some elements of these solutions are extracted from the pool and combined differently to produce new good solutions. In the VRP, vehicle routes selected from several solutions will be used as a starting point. The extraction process gives a larger weight to those routes belonging to the best solutions.
5.2.7
Other Algorithms Based on Tabu Search
Many algorithms based on TS have been proposed during the last five years for the solution of VRPs. Gendreau, Laporte Musaraganyi and Taillard proposed an algorithm [27] for the solution of the heterogeneous vehicle routing problem. Firstly, the algorithm makes use of GENIUS (section 4.3.5), secondly it uses TS, and then it is itself embedded within a so-called Adaptive Memory Procedure (section 5.2.6). Renaud, Laporte and Boctor proposed an algorithm [55] for the solution of the multidepot vehicle routing problem. The algorithm consists of three phases which are: fast
223 improvement, intensification and diversification. The reader can find other algorithms based on tabu search in the articles [3, 24, 12, 14, 52].
5.3
Genetic algorithms
This search strategy uses concepts from population genetics and evolution theory to construct algorithms [1, 51, 62] that try to optimize the fitness of a population of elements through recombination and mutation of their genes. The general idea of genetic local search is given by the following procedure: Step 1 (Initialize): Construct an initial population of n solutions. Step 2 (Improve): Use local search to replace the n solutions in the population by n local optima. Step 3 (Recombine): Augment the population by adding m offspring solutions, the population size now equals n + m. Step 4 (Improve): Use local search to replace the m offspring solutions by m local optima. Step 5 (Select): Reduce the population to its original size by selecting n solutions from the current population. Step 6 (Evolute): Repeat Step 3 through 5 until a stop criterion is satisfied.
5.3.1
GIDEON
GIDEON [26, 61] is a genetic algorithm for VRPs with time windows and capacity constraints, based on a cluster first - route second strategy. The genetic algorithm is only applied during the clustering phase: a procedure, called genetic sectoring, partitions the vertices into sector or clusters centered at the depot, as in the sweep algorithm (section 4.2.1).
5.3.2
GENEROUS
GENEROUS [26] is a genetic algorithm for VRPs with time windows. This algorithm avoids the difficulties related to the encoding of a solution into a chromosome by applying the crossover and mutation operators on the solutions themselves. In this algorithm, a new solution is created from two parent solutions 1 and 2 by linking the first customers on a route of parent 1 to the last customers on a route of parent 2,
224 as in the 2-opt exchange procedure. The new route replaces the old one in parent solution 1. A second offspring solution can be created by inverting the roles of the parents solutions.
5.4
Neural nets algorithms
The use of artificial neural networks to find good solutions to combinatorial optimization problems has recently caught some attention. A neural network consists of a network [1, 15] of elementary nodes (neurons) that are linked through weighted connections. The nodes represent computational units, which are capable of performing a simple computation, consisting of a summation of the weighted inputs, followed by the addition of a constant called the threshold or bias, and the application of a nonlinear response function. The result of the computation of a unit constitutes its output. This output is used as an input for the nodes to which it is linked through an outgoing connection. The overall task of the network is to achieve a certain network configuration, for instance a required input-output relation, by means of the collective computation of the nodes. This process is often called self-organization.
5.4.1
Hopfield Neural N e t s
The first attempt to produce a solution for the TSP with neural nets was based on a Hopfield Neural Network [2]. Hopfield networks can be used as associative memories for information storage and retrieval, and to solve combinatorial problems. They belong to the class of recurrent neural networks, that is, outputs of a neural networks are fed back to inputs of previous layers of the network. The implementation of a Hopfield Neural Network requires: • A transformation of the cost and the constraints of the problem into one function, known as Hopfield energy function. • The determination of Lagrange parameters. The above steps are often the key decisive factors on the success of the application. These, are also, the reasons why researchers have difficulty in duplicating Hopfield's result for solving the traveling salesman problem. To map the traveling salesman problem onto the Hopfield framework, a scheme is needed to represent the final state of the network as a tour list. Hopfield and Tank [2] adopted a representation scheme in which the precedence (sequence) of cities in a tour list is encoded by the final states of a set of neurons. For example, for an n-city problem, the network needs n 2 neurons — one neuron for each possible tour position for each city. Since there are n cities, each with n possible tour positions, n x n neurons are needed.
225 5.4.2
Self-Organized Maps
Self-organized maps are instances of so-called competitive neural networks models [26, 49]. They self-organize by gradually adjusting the weights on their connections.
5.5
Ant algorithms
The ant system, introduced by Colorni, Dorigo and Maniezzo [13, 18], is a new distributed metaheuristic for hard combinatorial optimization problems and was first used on the traveling salesman problem. Observations on real ants searching for food were the inspiration to imitate the behavior of ant colonies. Real ants are able to communicate information concerning food sources via an aromatic essence, called pheromone. They mark the path they walk on by laying down pheromone in a quantity that depends on the length of the path and the quality of the discovered food source. Other ants can observe the pheromone trail and are attracted to follow it. The described behavior of real ant colonies can be used to solve combinatorial optimization problems by simulation: artificial ants searching the solution space simulate real ants searching their environment, the objective values correspond to the quality of the food sources and an adaptive memory corresponds to the pheromone trails. In addition, the artificial ants are equipped with a local heuristic function to guide their search through the set of feasible solutions.
5.6
Iterated Lin-Kernigham algorithm
The Iterated Lin Kernigham (ILK) [67, 53] has been proposed by Johnson [40] and it is considered to be one of the best for the TSP. ILK uses LK to obtain a first local minimum. To improve this local minimum, the algorithm examines other local minimum tours 'near' the current local minimum. To generate these tours, ILK first applies a random and unbiased nonsequential 4-opt exchange to the current local minimum and then optimizes this 4-opt neighbor using the LK algorithm. If the tour obtained by this process is better than the current local minimum then Iterated LK makes this tour the current local minimum and continues from there using the same neighbor generation process. Otherwise, the current local minimum remains as it is and further random 4-opt moves are tried. The algorithm stops when a stopping criterion based either on the number of iterations or the computational time is satisfied. The random 4-opt exchange performed by ILK is called double-bridge move and plays a diversification role for the search process; It tries to propel the algorithm into
226 a different area of the search space while preserving at the same time large parts of the structure of the current local minimum. The ILK procedure is as follows: Step 1. Generate a random tour T. Step 2. Do the following for some prespecified number, M, of iterations: Step 2.1. Perform an (unbiased) random 4-opt move on T, obtaining T1. Step 2.2. Run LK on T \ obtaining T 2 . Step 2.3. If length(T2)
< length{T),
set T = T 2 .
Step 3 Return T.
5.7
Guided local search
Guided local search (GLS) originally proposed by Voudouris and Chang [67, 4] is a general optimization technique suitable for a wide range of combinatorial optimization problems. The main focus is on the exploitation of problem and search-related information to effectively guide local search heuristics in the vast search spaces of NPhard optimization problems. This is achieved by augmenting the objective function of the problem to be minimized with a set of penalty terms which are dynamically manipulated during the search process to steer the heuristic to be guided. GLS augments the cost function of the problem to include a set of penalty terms and passes this, instead of the original one, for minimization by the local search procedure. Local search is confined by the penalty terms and focuses attention on promising regions of the search space. Iterative calls are made to local search. Each time local search gets caught in a local minimum, the penalties are modified and local search is called again to minimize the modification cost function.
5.8
Fast local search
Fast local search (FLS) originally proposed by Voudouris and Chang [67]. FLS works as follows. The current neighborhood is broken down into a number of small subneighborhoods and an activation bit is attached to each of them. The idea is to scan continuously the sub-neighborhoods in a given order, searching only those with the activation bit set to 1. These sub-neighborhoods are called active sub-neighborhoods. Sub-neighborhoods with the bit set to 0 are called inactive sub-neighborhoods and they are not being searched. The neighborhood search process does not restart whenever we find a better solution but it continues with the next sub-neighborhoods in the given order. This order may be static or dynamic.
227
5.9
GRASP
The Greedy Randomized Adaptive Search Procedure (GRASP) [20, 21, 38, 37, 56] is a two-phase local search. This randomized technique provides a feasible solution within every iteration. The final result is simply the best solution found over all iteration (multi-start local search). Each iteration consists of two phases, a construction phase and a local search procedure. In the construction phase a randomized greedy function is used to build up an initial solution. This solution is then exposed for improvement attempts in the local search phase. The construction phase can be described as stepwise adding one element at a time to the partial (incomplete) solution. The choice of the next element to be added is determined by ordering all elements in a candidate list with respect to a greedy function. The heuristic is adaptive because the benefits associated with every element are updated at each iteration of the construction phase to reflect the changes brought on by the selection of the previous element. The probabilistic component of a GRASP is characterized by randomly choosing one of the best candidate in the list but not necessary the top candidate. A generic GRASP algorithm is given below: Step 1. Input the problem instance. Step 2. Do until some GRASP stopping criterion is satisfied: Step 2 . 1 . Execute the construction phase of GRASP in order to obtain a greedy random solution. Step 2.2. Apply the local search phase to the obtained greedy solution. Step 2.3. Update the best solution if needed. Step 3. Return the best solution found. Although there are many GRASP applications to combinatorial optimization the only known application in the context of the vehicle routing problem is that of Kontoravdis et al. ([6, 42]). Kontoravdis and Bard addressed the problem of finding the minimum number of vehicles needed to serve n customers subject to time windows and capacity constraints: Step 1. Find a lower bound u on the number of routes needed. Step 2. Select u seed customers to form a set of initial routes. Step 3 . Calculate the cost of inserting each customers into these routes.
228 Step 4. Calculate penalties for each insertion. Step 5. Construct a list L that contains r largest penalties. From L, randomly choose one customer. Step 6. Try to insert the customer into the corresponding route. If successful, then go to Step 7. If insertion leads to a time or capacity violation, start a new route between the depot and the customer and go to Step 4. Step 7. If every customer is routed, then stop, otherwise update the costs and go to Step 5. Step 8. Repeat Steps 1 to 7 a predetermined number, M, of times and save the best results. During these M iterations, run a post-processor every N (N < M) times and keep the best results.
6
Computational Results
In order to compare the various algorithms we report the best solution values obtained using each algorithm on the 14 benchmark problems described by Christofides et al. [17]. The algorithms for which we have comparable results are: • A. The Clarke and Wright algorithm [17]. • B. The Mole and Jameson algorithm [17]. • C. The sweep algorithm [17]. • D. The Fisher & Jaikumar algorithm [45]. • TA. Willard's tabu search algorithm [26]. • TB. Osman's tabu search algorithm — Best admissible strategy [26]. • TC. Osman's tabu search algorithm — First best admissible strategy [26]. • TD. TABUROUTE [26]. • TE. Taillard's tabu search algorithm [26]. • T F . The Xu & Kelly tabu search algorithm [45]. • SAO. Osman's simulated annealing algorithm [45].
229 In Table 1 the numbers of each box give the total mileage. The best known solutions are indicated with an asterisk. The computational results suggest that the best known solutions for these bench mark problems are always found using Taillard's algorithm, followed closely by the Xu and Kelly algorithm and TABUROUTE. The classic heuristic algorithms give sufficiently good solutions but none of them is competitive with the tabu search implementations. SA implementations produce good quality results, although not so good as the results obtained by Taillard's algorithm. Concerning genetic algorithms, GIDEON minimizes the total distance and GENEROUS minimizes the total route time [26]. GENEROUS is computationally more expensive than GIDEON as it handles a population of solutions rather than a population of sector angles. For both algorithms the results are less favorable, when compared to TS algorithms. Besides the 14 benchmark problems by Christofides et al. [17], a large number of problem instances are in the TSPLIB library (www. e l i b . z i b . de) and others may be found from the VRP site (www.geocities.com/ResearchTriangle/7279/vrp.html).
230
,_ CM LO CO 00 LO
£
im ^ oco g o j o Sc o L o ' o i S CM i-H LO O *? oo oo LO OS OO £
T-H «P ^f CM in
* CD =i m CO oo
* -* -1 co CM oo
. . CO CO CM
00
*-. oq to ^ c>
sK
^
CO co m o5 Oi S co oo cs O «H?0)00 I-H CM OS * I* T. OS Tjl i-H Tf N CO ^ oo o; T-H •V 0O ID Ol 00 00 CM IV CM
^ 3 Jg "5 cS 3 o s S S 55
CMCOCMOCMinOCOSSS 1 0 0 0 0 0 H H « 3 0 ) 0 0 ^ 5 S co
IV t-LO
0 D 0 0
i-HT-HLO
C n
LO LO * • • _ — co in co * 3 M H ! H ! 't •^ co ^ ^
tco
s ^ CO 00 J5 00 o ho
CO OO 10 CO
00T-H^HT-H00T-H00
Sr
0 0 Tj< T f
• TTf
"
o o § o s ; s s S S S m 2 2 2 2 S ^
CO in LO
^Hi-HT-HT-ii-Ht^ 00 00
m
CM "* CO
3 £ S LO' ^
m
N
»
m
CM £ S
N ^ 3
3
^
io to 3
00 00 CO CO 00 O ) O LO OO Oi
o
^ i > c o ^ f S o c o i n 2 ° °
_,
CM CO
m
fe 53 08 s "5 OS
CO 0> SK
fc
§
*
U )
°
BS^2S5SS22SooS CMco^fmcoivoooi eu
oo
3
OJ OJ
CM T j i H N t o < 30i03I.N£U2J Si.>l co iv m O CO CO CO O in oo oo in os oo T-H T-H T—I
CO
231
References [1] E. Aarts and J.K. Lenstra. Local Search in Combinatorial Optimization. and Sons, 1997. [2] N. Ansari and E. Hou. Computational intelligence for optimization. Academic Publishers, first edition, 1997.
Wiley
Kluwer
[3] P. Augerat, J.M. Belenguer, E. Benavent, A. Corberan, and D. Nannef. Separating capacity constraints in the cvrp using tabu search. European Journal of Operational Research, 106:546-557, 1998. [4] B.D. Baker, V. Furnon, P. Shaw, P. Kilby, and P. Prosser. Solving vehicle routing problems using constraint programming and metaheuristics. Journal of Heuristics, 6:501-523, 2000. [5] R. Ballou. Business Logistics Management, Planning, Organizing and Controlling the Suply Chain. Prentice-Hall International, Inc., fourth edition, 1999. [6] J. Bard, L.Huang, P. Jaillet, and M. Dror. A decomposition approach to the inventory routing problem with satelite facilities. Transportation science, 32(2):189-203, 1998. [7] J.F. Bard, L. Huang, M. Dror, and P. Jaillet. A branch and cut algorithm for the vrp with satelite facilities. HE Transactions, 30:821-834, 1998. [8] L.D. Bodin and B.L. Golden. Classification in vehicle routing and scheduling. In B. Golden and L. Bodin, editors, Proceedings of the International Workshop on Current and Future Directions in the Routing and Scheduling of Vehicles and Crews, pages 97-108. Wiley and Sons, 1979. [9] L.D. Bodin, B.L. Golden, A. A. Assad, and Michael O. Ball. Routing and scheduling of vehicles and crews. Computers and Operations Research, 10:63-211, 1983. [10] J. Braca, J. Bramel, B. Posner, and D. Simchi Levi. A computerized approach to the new york city school bus routing problem. Technical report, Columbia University, 1994. [11] A. Van Breedam. Improvement heuristics for the vehicle routing problem based on simulated annealing. European Journal of Operational Research, 86:480-490, 1995. [12] A. Van Breedam. Comparing descent heuristics and metaheuristics for the vehicle routing problem. Computers and Operations Research, 28:289-315, 2001.
232 [13] B. Bullnheimer, R. Hartl, and C. Strauss. An improved ant system algorithm for the vehicle routing problem, pages 1-11, 1997. preprint. [14] V. Campos and E. Mota. Heuristics procedures for the capacitaded vehicle routing problem. Computational Optimization and Applications, 16:265-277, 2000. [15] Bo Sodererg Carsten Peterson. Artificial neural networks. In E. Aarts and J.K. Lenstra, editors, Local Search in Combinatorial Optimization, pages 173-214. Wiley and Sons, 1997. [16] N. Christofides. Vehicle routing. In E.L. Lawer, J.K. Lenstra, A.H.G. Rinnoy Kan, and D.B. Shmoys, editors, The Travelling Salesman Problem: A Guided Tour of Combinatorial Optimization, pages 431-448. Wiley and Sons, 1985. [17] N. Christofides, A. Mignozzi, and P. Toth. The vehicle routing problem. In N. Christofides, editor, Combinatorial Optimization, pages 315-338. Wiley and Sons, 1979. [18] M. Dorigo, V. Maniezzo, and A. Colorni. The ant system: Optimization by a colony of cooperating agents. IEEE Transactions on Systems, 26:1-13, 1996. [19] Peter J.M. van Laarhoven Emile H.L. Aarts, Jan H. M. Korst. Simulated annealing. In E. Aarts and J.K. Lenstra, editors, Local Search in Combinatorial Optimization, pages 91-120. Wiley and Sons, 1997. [20] T.A. Feo and M.G.C. Resende. Greedy randomized adaptive search procedure. Journal of Global Optimization, 6:109-133, 1995. [21] P. Festa and M.G.C. Resende. Grasp: An annotated bibliography. Look again, 2000. [22] M. Fisher. The langrangean relaxation method for solving integer programming problems. Management Science, 27(1):1-18, 1981. [23] M.L. Fisher and R. Jaikumar. A generalized assignment heuristic for vehicle routing. In B. Golden and L. Bodin, editors, Proceedings of the International Workshop on Current and Future Directions in the Routing and Scheduling of Vehicles and Crews, pages 109-124. Wiley and Sons, 1979. [24] D. Ozgur G. Barbarosoglu. A tabu search algorithm for the vehicle routing problem. Computers and Operations Research, 26:255-270, 1999. [25] R. Ganeshan. An introduction to supply chain management, pages 1-7, 2001.
233 [26] M. Gendrau, G. Laporte, and J.Y. Potvin. Vehicle routing: Modern heuristics. In E. Aarts and J.K. Lenstra, editors, Local Search in Combinatorial Optimization, pages 311-336. Wiley and Sons, 1997. [27] M. Gendreau, G. Laporte, C. Musaraganyi, and E.D. Taillard. A tabu search heuristic for the heterogeneous fleet vehicle routing problem. Computers and Operations Research, 26:1153-1173, 1999. [28] J.H. Gerdessen. Vehicle routing problem with trailers. European Journal of Operational Research, 93:135-147, 1996. [29] F. Glover. Tabu search: A tutorial, pages 1-47, 1989. [30] B.L. Golden. Introduction to and recent advances in vehicle routing methods. In M. Florian, editor, Transportation Planning Models, pages 383-418. Elsevier Science Publishers, 1991. [31] B.L. Golden, G. Laporte, and E.D. Taillard. An adaptive memory heuristic for a class of vehicle routing problems with minmax objective. Computers and Operations Research, 24:445-452, 1997. [32] B.L. Golden and W. Stewart. Empirical analysis of heuristic. In E.L. Lawer, J.K. Lenstra, A.H.G. Rinnoy Kan, and D.B. Shmoys, editors, The Travelling Salesman Problem: A Guided Tour of Combinatorial Optimization, pages 207250. Wiley and Sons, 1985. [33] M. Hachicha, M.J. Hodgson, G. Laporte, and Frederic Semet. Heuristics for the multi-vehicle covering tour problem. Computers and Operations research, 27:29-42, 2000. [34] M. Haughton and A. Stenger. Semi-variable delivery routes and the efficiency of outbound logistics. International Journal of Physical Distribution and Logistics Management, 27:459-474, 1997. [35] K. Helsgaum. An effective implementation of the lin-kernigham travelling salesman heuristic. European Journal of Operational Research, 126:106-130, 2000. [36] Alan Hertz, Eric Taillard, and Dominique de Werra. Tabu search. In E. Aarts and J.K. Lenstra, editors, Local Search in Combinatorial Optimization, pages 121-136. Wiley and Sons, 1997. [37] K. Holmqvist, A. Migdalas, and P.M. Pardalos. Parallel continuous non-convex optimization. In A. Migdalas, P.M. Pardalos, and S. Stor0y, editors, Parallel Computing in Optimization, pages 471-528. Kluwer Academic Publishers, 1997.
234 [38] K. Holmqvist, A. Migdalas, and P.M. Pardalos. Parallelized heuristics for combinatorial search. In A. Migdalas, P.M. Pardalos, and S. Stor0y, editors, Parallel Computing in Optimization, pages 269-294. Kluwer Academic Publishers, 1997. [39] D. Johnson and C. Papadimitriou. Performance guarantees of heuristic. In E.L. Lawer, J.K. Lenstra, A.H.G. Rinnoy Kan, and D.B. Shmoys, editors, The Travelling Salesman Problem: A Guided Tour of Combinatorial Optimization, pages 145-180. Wiley and Sons, 1985. [40] D.S. Johnson and L.A. McGeoch. The travelling salesman problem: A case study. In E. Aarts and Lenstra J.K, editors, Local Search in Combinatorial Optimization, pages 215-310. Wiley and Sons, 1997. [41] H. Kokubugata, H. Itoyama, and H. Kawashima. Vehicle routing methods for city logistics operations. In IFAC Transportation Systems, 1997. [42] G. Kontoravdis and J.F. Bard. A grasp for the vehicle routing problem with time windows. ORSA Journal on Computing, 7(l):10-23, 1995. [43] G. Laporte. The travelling salesman problem: An overview of exact and approximate algorithms. European Journal of Operational Research, 59:231-247, 1992. [44] G. Laporte. The vehicle routing problem: An overview of exact and approximate algorithms. European Journal of Operational Research, 59:345-358, 1992. [45] G. Laporte, M. Gendreau, J.Y. Potvin, and F. Semet. Classical and modern heuristics for the vehicle routing problem. International Transactions in Operational Research, 7:285-300, 2000. [46] E.L. Lawer, J.K. Lenstra, A.H.G. Rinnoy Kan, and D.B.-Shmoys. The Travelling Salesman Problem: A Guided Tour of Combinatorial Optimization. Wiley and Sons, 1985. [47] S. Lin. Computer solutions of the travelling salesman problem. Bell Technical Journal, 44:2245-2269, 1965.
System
[48] S. Lin and B.W. Kernigham. An effective implementation for the traveling salesman problem. Operations Research, 21:498-516, 1973. [49] A. Modares, S. Somhom, and T. Enwaka. A self - organizing neural network approach for multiple travelling salesman and vehicle routing problems. International Transactions in Operational Research, 6:591-606, 1999. [50] G. Mosheiov. Vehicle routing with pickup and delivery: Tour - partitioning heuristics. Computers and Industrial Engineering, 34:669-684, 1998.
235 [51] Heinz Muhlenbein. Genetic algorithms. In E. Aarts and J.K. Lenstra, editors, Local Search in Combinatorial Optimization, pages 137-172. Wiley and Sons, 1997. [52] W.P. Nanry and J.W. Barnes. Solving the pickup and delivery problem with time windows using reactive tabu search. Transportation Research Part B, 34:107-121, 2000. [53] David Neto. Efficient cluster compensation for Lin - Kernigham heuristics. PhD thesis, Computer science university of Toronto, 1999. [54] G. Reinelt. The Travelling Salesman Problem, Computational solutions for TSP Applications. Springer-Verlag, 1994. [55] J. Renaud, G. Laporte, and F. Boctor. A tabu search heuristic for the multi depot vehicle routing problem. Computers and Operations Research, 23:229-235, 1996. [56] M.G.C. Resende. Greedy randomized adaptive search procedure. Technical report, 1998. [57] Y. Rochat and I. Taillard. Probabilistic diversification and intensification in local search for vehicle routing problem. Journal of Heuristics, 1:147-167, 1995. [58] P. Rodriguez, M. Nusbaum, R. Baeza, G. Leon, M. Sepulveda, and A. Cobian. Using global search heuristics for the capacity vehicle routing problem. Computers and Operations Research, 25(5):407-417, 1998. [59] D.J. Rosenkratz, R.E. Stearns, and P.M. Lewis. An analysis of several heuristics for the travelling salesman problem. SIAM Journal on Computing, 6:563-581, 1977. [60] K.S. Ruland and E.Y. Rodin. The pickup and delivery problem: Faces and branch-and-cut algorithm. Computers Mathematical Applications, 33(12):1-13, 1997. [61] S. Thangiah. Vehicle routing with time windows using genetic algorithms. Technical Report, pages 1-23, 1993. [62] S.R. Thangiah, I. Osman, T. Sung, and R. Vinayagamoorthy. Algorithms for the vehicle routing problems with deadlines. American Journal of Mathematical and Management Science, 13:323-355, 1995. [63] S.R. Thangiah, J.Y. Potvin, and T. Sung. Heuristics approaches to vehicle routing with backhauls and time windows. Computers and Operations Research, 23:1043-1057, 1996.
236 [64] P. Tian, J. Ma, and D.M. Zhang. Application of the simulated annealing algorithm to the combinatorial optimisation problem with permutation property: An investigation of generation mechanism. European Journal of Operational Research, 118:81-94, 1999. [65] P. Toth and D. Vigo. An exact algorithm for the vehicle routing problem with backhauls. Transportation Science, 31(4):372-385, 1997. [66] D.V. Tung and A. Pinnoi. Vehicle routing-scheduling for waste collection in hanoi. European Journal of Operational Research, 125:449-468, 2000. [67] C. Voudouris and E. Tsang. Guided local search and its application to the travelling salesman problem. European Journal of Operational Research, 113:469-499, 1999.
Combinatorial and Global Optimization, pp. 237-249 P.M. Pardalos, A. Migdalas and R. Burkard, Editors © 2002 World Scientific Publishing Co.
A New Finite Cone Covering Algorithm for Concave Minimization Christophe Meyer ([email protected] ) Ecole Polytechnique de Montreal Depariement de Mathematiques et de Genie Industriel C. P. 6079, succ. Centre-ville Montreal (Quebec) Canada H3C 3A 7
Brigitte Jaumard ( b r i g i t t O c r t . umontreal. ca) GERAD and Ecole Polytechnique de Montreal Depariement de Mathematiques et de Genie Industriel C. P. 6079, succ. Centre-ville Montreal (Quebec) Canada H3C 3A7
Abstract We propose a new finite cone covering algorithm for concave minimization over a polytope, in which the cones are defined by extreme points of the polytope. The main novelties are the use of cones defined by an arbitrary number of edges, and the subdivision process. This latter is shown to have a "descent property", i.e., all subcones are strictly better in some sense than the subdivided cone, which eliminates the possibility of cycling. The main task in the subdivision process consists in expressing a given point of the polytope as a convex combination of extreme points of a face of the polytope. Keywords: concave minimization, cone covering, finite convergence
238
1
Introduction
We consider the following concave minimization problem (CP)
min{f(x)\x
<= P}
where / is a concave function defined on R™ and P is a full dimensional polytope of R". The first conical algorithm for concave minimization was proposed by Tuy [17] in 1964. The main idea was to cover the polytope by polyhedral cones, each of which having exactly n edges corresponding to extreme points of the polytope. If it can be shown that a cone cannot contain a better solution than the one at hand, the cone is fathomed; otherwise it is replaced by a set of subcones that cover it. Unfortunately, Zwart [22] showed that this algorithm is not finitely convergent as it was first expected by exhibiting a small example on which the algorithm cycles. In order to avoid the possibility of cycling, Bali [1] and Zwart [23] proposed a small modification which has the effect to transform Tuy's algorithm into a cone partitioning one. During the next two decades, numerous cone partitioning algorithms were developed and shown convergent ([4, 5, 6, 8, 16, 18, 19, 20, 21] to cite only a few; for additional references, see the surveys of Horst and Tuy [7], Benson [2]). Note however that up to now only infinite convergence could be shown when an exact solution is sought; the fact that the cones are not anymore defined by extreme points of the original polytope complicates significantly the search of a finitely convergent cone partitioning algorithm. In contrast, the only paper on cone covering is due to Gallo [3] who proposed a modification of Tuy's 1964 algorithm that results in a search tree which is a subtree of Tuy's one. However no convergence proof was given. It is only recently that Meyer [12] proved the infinite convergence of Gallo's [3] and Tuy's [17] algorithms. Only infinite convergence can be shown because cycling can occur in both algorithms. Two methods were proposed to transform these algorithms into finite ones. These methods are passive with respect to cycling: the first one does not attempt to identify if cycling occurs but still has a stopping criterion that guarantees an optimal solution after however a very large number of iterations. The second ensures that a cone is not generated twice by keeping a list of all generated cones. In this paper, we propose a finite cone covering algorithm which prevents cycling. The key point is a subdivision process: it was initially proposed in [11] and used to develop a cone covering algorithm that can be shown to be finitely convergent under the (rather unrealistic) assumption that the best simplicial lower bound is computed exactly for each cone (the notion of best simplicial lower bound was introduced in [4]; the computation of the best simplicial lower bound amounts to solve a convex program as shown in [11]). The nonexistence of cycles is proved by defining a function whose value depends only on the cone and on the current incumbent value, and that is shown to decrease strictly when going from one cone to another. This function is essentially equal to the optimal
239 value of the linear program that is solved in cone partitioning algorithms in order to check if the cone can be fathomed. The paper is organized as follows. In Section 2, the basic operations of the cone covering algorithm are recalled, i.e., the construction of an initial cover, the fathoming test, and the subdivision procedure with its descent property. In Section 3, we give the algorithm and prove its finite convergence. Conclusions are drawn in the last section.
2
Basic operations
In this section, we recall the basic operations needed to define the algorithm, namely the construction of an initial conical cover (Section 2.1), the fathoming test (Section 2.2) and the subdivision procedure (Section 2.3). The descent property satisfied by the subdivision process is given in Section 2.3.3.
2.1
Initial cover
Assume that a nondegenerate vertex of P is available. By performing a change of variables if necessary, we may assume that this vertex is the origin O. Let K° be the polyhedral cone of origin O defined by the n adjacent vertices to O. This cone is used to define the initial cover of P. If no nondegenerate vertex is available, we compute an interior point of P and use it to decompose P into n + 1 subpolytopes. This point defines a nondegenerate vertex for each of the n + 1 subpolytopes (see, e.g., Meyer [12])In this paper, we consider polyhedral cones of origin 0 that may have more than n edges. Moreover, each edge corresponds to an extreme point of P, i.e., the edge intersects the boundary of P at an extreme point. The set of cones of origin O and whose edges correspond to extreme points of P will be denoted by K. liu^,j = l...p are the directions of a cone, this cone will be noted conefw1, w 2 , . . . , up} (since the cone is vertexed at O, uj can be viewed indifferently as a point or as a vector of E n , for j = 1 , . . . ,p). For a subset X