Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen
Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen University of Dortmund, Germany Madhu Sudan Microsoft Research, Cambridge, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max-Planck Institute of Computer Science, Saarbruecken, Germany
5792
Osamu Watanabe Thomas Zeugmann (Eds.)
StochasticAlgorithms: Foundations and Applications 5th International Symposium, SAGA 2009 Sapporo, Japan, October 26-28, 2009 Proceedings
13
Volume Editors Osamu Watanabe Tokyo Institute of Technology Department of Mathematical and Computing Sciences W8-25, Tokyo 152-8552, Japan E-mail:
[email protected] Thomas Zeugmann Hokkaido University Division of Computer Science N-14, W-9, Sapporo 060-0814, Japan E-mail:
[email protected]
Library of Congress Control Number: 2009935906 CR Subject Classification (1998): F.2, F.1.2, G.1.2, G.1.6, G.2, G.3 LNCS Sublibrary: SL 1 – Theoretical Computer Science and General Issues ISSN ISBN-10 ISBN-13
0302-9743 3-642-04943-5 Springer Berlin Heidelberg New York 978-3-642-04943-9 Springer Berlin Heidelberg New York
This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. springer.com © Springer-Verlag Berlin Heidelberg 2009 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper SPIN: 12773497 06/3180 543210
Preface
The 5th Symposium on Stochastic Algorithms, Foundations and Applications (SAGA 2009) took place during October 26–28, 2009, at Hokkaido University, Sapporo (Japan). The symposium was organized by the Division of Computer Science, Graduate School of Computer Science and Technology, Hokkaido University. It offered the opportunity to present original research on the design and analysis of randomized algorithms, random combinatorial structures, implementation, experimental evaluation and real-world application of stochastic algorithms/heuristics. In particular, the focus of the SAGA symposia series is on investigating the power of randomization in algorithms, and on the theory of stochastic processes especially within realistic scenarios and applications. Thus, the scope of the symposium ranges from the study of theoretical fundamentals of randomized computation to experimental investigations on algorithms/heuristics and related stochastic processes. The SAGA symposium series is a biennial meeting. Previous SAGA symposia took place in Berlin, Germany (2001, LNCS vol. 2264), Hatfield, UK (2003, LNCS vol. 2827), Moscow, Russia (2005, LNCS vol. 3777), and Z¨ urich, Switzerland (2007, LNCS vol. 4665). This year 22 submissions were received, and the Program Committee selected 15 submissions for presentation. All papers were evaluated by at least three members of the Program Committee, partly with the assistance of subreferees. The present volume contains the texts of the 15 papers presented at SAGA 2009, divided into groups of papers on learning, graphs, testing, optimization, and caching as well as on stochastic algorithms in bioinformatics. The volume also contains the paper and abstract of the invited talks, respectively: – Werner R¨ omisch (Humboldt University, Berlin, Germany): Scenario Reduction Techniques in Stochastic Programming – Taisuke Sato (Tokyo Institute of Technology, Tokyo, Japan): Statistical Learning of Probabilistic BDDs We would like to thank the many people and institutions who contributed to the success of the symposium. Thanks to the authors of the papers for their submissions, and to the invited speakers, Werner R¨omisch and Taisuke Sato, for presenting exciting overviews of important recent research developments. We are very grateful to the sponsors of the symposium. SAGA 2009 was supported in part by two MEXT Global COE Programs: Center for Next-Generation Information Technology based on Knowledge Discovery and Knowledge Federation, at the Graduate School of Information Science and Technology, Hokkaido University, and Computationism as a Foundation of the Sciences at Tokyo Institute of Technology.
VI
Preface
We are grateful to the members of the Program Committee for SAGA 2009. Their hard work in reviewing and discussing the papers made sure that we had an interesting and strong program. We also thank the subreferees assisting the Program Committee. Special thanks go to Local Chair Shin-ichi Minato (Hokkaido University, Sapporo, Japan) for the local organization of the symposium. Last but not least, we thank Springer for the support provided by Springer’s Online Conference Service (OCS), and for their excellent support in preparing and publishing this volume of the Lecture Notes in Computer Science series. October 2009
Osamu Watanabe Thomas Zeugmann
Organization
Conference Chair Thomas Zeugmann
Hokkaido University, Sapporo, Japan
Program Committee Andreas Albrecht Amihood Amir Artur Czumaj Josep Dıaz
Queen’s University Belfast, UK Bar-Ilan University, Israel University of Warwick, UK Universitat Polit`ecnica de Catalunya, Barcelona, Spain Walter Gutjahr University of Vienna, Austria Juraj Hromkoviˇc ETH Z¨ urich, Switzerland Mitsunori Ogihara University of Miami, USA Pekka Orponen Helsinki University of Technology TKK, Finland Kunsoo Park Seoul National University, Korea Francesca Shearer Queen’s University Belfast, UK Paul Spirakis University of Patras, Greece Kathleen Steinh¨ ofel King’s College London, UK Masashi Sugiyama Tokyo Institute of Technology, Japan Osamu Watanabe (Chair) Tokyo Institute of Technology, Japan Masafumi Yamashita Kyushu University, Fukuoka, Japan Thomas Zeugmann Hokkaido University, Sapporo, Japan
Local Arrangements Shin-ichi Minato
Hokkaido University, Sapporo, Japan
Subreferees Marta Arias Mituhiro Fukuda
Alex Fukunaga Masaharu Taniguchi
Sponsoring Institutions MEXT Global COE Program Center for Next-Generation Information Technology based on Knowledge Discovery and Knowledge Federation at the Graduate School of Information Science and Technology, Hokkaido University. MEXT Global COE Program Computationism as a Foundation of the Sciences at Tokyo Institute of Technology.
Table of Contents
Invited Papers Scenario Reduction Techniques in Stochastic Programming . . . . . . . . . . . . Werner R¨ omisch
1
Statistical Learning of Probabilistic BDDs . . . . . . . . . . . . . . . . . . . . . . . . . . Taisuke Sato
15
Regular Contributions Learning Learning Volatility of Discrete Time Series Using Prediction with Expert Advice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Vladimir V. V’yugin
16
Prediction of Long-Range Dependent Time Series Data with Performance Guarantee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mikhail Dashevskiy and Zhiyuan Luo
31
Bipartite Graph Representation of Multiple Decision Table Classifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kazuya Haraguchi, Seok-Hee Hong, and Hiroshi Nagamochi
46
Bounds for Multistage Stochastic Programs Using Supervised Learning Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Boris Defourny, Damien Ernst, and Louis Wehenkel
61
On Evolvability: The Swapping Algorithm, Product Distributions, and Covariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimitrios I. Diochnos and Gy¨ orgy Tur´ an
74
Graphs A Generic Algorithm for Approximately Solving Stochastic Graph Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ei Ando, Hirotaka Ono, and Masafumi Yamashita How to Design a Linear Cover Time Random Walk on a Finite Graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yoshiaki Nonaka, Hirotaka Ono, Kunihiko Sadakane, and Masafumi Yamashita
89
104
X
Table of Contents
Propagation Connectivity of Random Hypergraphs . . . . . . . . . . . . . . . . . . . Robert Berke and Mikael Onsj¨ o Graph Embedding through Random Walk for Shortest Paths Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yakir Berchenko and Mina Teicher
117
127
Testing, Optimization, and Caching Relational Properties Expressible with One Universal Quantifier Are Testable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Charles Jordan and Thomas Zeugmann
141
Theoretical Analysis of Local Search in Software Testing . . . . . . . . . . . . . . Andrea Arcuri
156
Firefly Algorithms for Multimodal Optimization . . . . . . . . . . . . . . . . . . . . . Xin-She Yang
169
Economical Caching with Stochastic Prices . . . . . . . . . . . . . . . . . . . . . . . . . . Matthias Englert, Berthold V¨ ocking, and Melanie Winkler
179
Stochastic Algorithms in Bioinformatics Markov Modelling of Mitochondrial BAK Activation Kinetics during Apoptosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C. Grills, D.A. Fennell, and S.F.C. Shearer
191
Stochastic Dynamics of Logistic Tumor Growth . . . . . . . . . . . . . . . . . . . . . . S.F.C. Shearer, S. Sahoo, and A. Sahoo
206
Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
221
Scenario Reduction Techniques in Stochastic Programming Werner R¨ omisch Humboldt-University Berlin, Department of Mathematics, Rudower Chaussee 25, 12489 Berlin, Germany
Abstract. Stochastic programming problems appear as mathematical models for optimization problems under stochastic uncertainty. Most computational approaches for solving such models are based on approximating the underlying probability distribution by a probability measure with finite support. Since the computational complexity for solving stochastic programs gets worse when increasing the number of atoms (or scenarios), it is sometimes necessary to reduce their number. Techniques for scenario reduction often require fast heuristics for solving combinatorial subproblems. Available techniques are reviewed and open problems are discussed.
1
Introduction
Many stochastic programming problems may be reformulated in the form f0 (x, ξ)P (dξ) : x ∈ X , min E(f0 (x, ξ)) =
(1)
Rs
where X denotes a closed subset of Rm , the function f0 maps from Rm × Rs to the extended real numbers R = R ∪ {−∞, +∞}, E denotes expectation with respect to P and P is a probability distribution on Rs . For example, models of the form (1) appear as follows in economic applications. Let ξ = {ξt }Tt=1 denote a discrete-time stochastic process of d-dimensional random vectors ξt at each t ∈ {1, . . . , T } and assume that decisions xt have to be made such that the total costs appearing in an economic process are minimal. Such an optimization model may often be formulated as T t−1 ft (xt , ξt ) : xt ∈ Xt , Atτ (ξt )xt−τ = ht (ξt ), t = 1, . . . , T . (2) min E t=1
τ =0
Typically, the sets Xt are polyhedral, but they may also contain integrality conditions. In addition, the decision vector (x1 , . . . , xT ) has to satisfy a dynamic constraint (i.e. xt depends on the former decisions) and certain balancing conditions. The matrices Atτ (ξt ), τ = 0, . . . , t − 1, (e.g. containing technical parameters) and the right-hand sides ht (ξt ) (e.g. demands) are (partially) random. The functions ft describe the costs at time t and may also be (partially) random O. Watanabe and T. Zeugmann (Eds.): SAGA 2009, LNCS 5792, pp. 1–14, 2009. c Springer-Verlag Berlin Heidelberg 2009
2
W. R¨ omisch
(e.g. due to uncertain market prices). The objective in (2) consists in minimizing the expected total costs. The time t = 1 represents the present, hence, we assume that ξ1 is deterministic and require that the decision x1 is deterministic, too. The latter condition is modeled by the constraint x1 = E(x1 ). Then we may reformulate the stochastic program (2) as optimization model ˆ : x1 = E(x1 ), x1 ∈ X1 , A10 x1 = h1 (ξ1 ) min f1 (x1 .ξ1 ) + E Φ(x1 , ξ) (3) for the decision x1 at time t = 1. With ξˆ denoting ξˆ := (ξ2 , . . . , ξT ) the uncertain future input process the function Φ is defined by T t−1 ˆ := inf E ft (xt , ξt ) : xt ∈ Xt , Atτ xt−τ = ht (ξt ), t = 2, . . . , T . Φ(x1 , ξ) t=2
τ =0
(4) A solution of (3) minimizes the first period costs and also the minimal expected future costs. It is called first-stage solution while a (stochastic) solution (x2 , . . . , xT ) of (4) is a second-stage solution. Consequently, the model (3) is named two-stage stochastic program. We note that any first-stage solution depends on the probability distribution of the stochastic process ξ. An often more realistic condition is to require that the decision xt at time t depends only on the available information (ξ1 , . . . , ξt ) and that the information flow evolves over time. This requirement is modeled by the constraints xt = E(xt |ξ1 , ξ2 , . . . , ξt ) (t = 1, . . . , T )
(5)
which has to be incorporated into the constraint set of (2) and (4), respectively. The expression E(· |ξ1 , ξ2 , . . . , ξt ) on the right-hand side of (5) is the conditional expectation with respect to the random vector (ξ1 , ξ2 , . . . , ξt ) (and is assumed to be well defined). The constraints (5) are often called nonanticipativity constraints and (2) including (5) is a multi-stage stochastic program. We note that the constraint for t = 1 in (5) coincides with the former condition x1 = E(x1 ) in (2). If we set x = x1 and X := {x ∈ X1 : x = E(x), A10 x = h1 (ξ1 )} ˆ , if x ∈ X and Φ(x, ξ) ˆ is finite f1 (x, ξ1 ) + Φ(x, ξ) f0 (x, ξ) := +∞ , else the optimization model (2) is of the form (1). Suitable assumptions on ξ together ˆ > −∞. In that case, finiteness of with duality arguments often imply Φ(x, ξ) ˆ Φ(x, ξ) is guaranteed if a feasible decision (x2 , . . . , xT ) exists for given x1 . Most of the approaches for solving (1) numerically are based on replacing the probability distribution P by a discrete distribution with finite support {ξ 1 , . . . , ξ N }, where the atom or scenario ξ i appears with probability pi > 0, N i = 1, . . . , N , and it holds i=1 pi = 1. The approximate stochastic program associated to (1) then is of the form
Scenario Reduction Techniques in Stochastic Programming
min
N
pi f0 (x, ξ i ) : x ∈ X .
3
(6)
i=1
In Section 2 we discuss how the discrete distribution should approximate the original probability distribution P . It turns out that properties of the integrand f0 (x, · ) as function of ξ together with properties of P characterize the kind of approximation needed in stochastic programming. When looking at (3), (4) we recall that evaluations of f0 at a pair (x, ξ) may be expensive. This leads us to one of the main numerical challenges in stochastic programming: A good approximation of P requires a large N , but solving (6) in a reasonable running time prefers a small(er) N . Hence, when solving applied stochastic programming models, one might start with a large(r) N , but running time requirements might force a (much) smaller number n of scenarios. Should ξ n+1 , . . . , ξ N just be thrown away ? In this paper, we argue that one should take advantage of the information contained in {ξ 1 , . . . , ξ N } and look for a better approximation of P based on n scenarios out of {ξ 1 , . . . , ξ N } by scenario reduction. The concept of scenario reduction and recent results in this area are reviewed in Section 3. In this paper, we restrict our attention to two-stage stochastic programs. We take a look at the underlying theory, review available algorithms for linear and mixed-integer two-stage stochastic programs, and point out which heuristic algorithms are important in this context and where we see possibilities for improvements. Multi-stage models and the tree structure of scenarios (due to the constraint (5)) require a (slightly) different theoretical basis and will not be considered in this paper, although scenario reduction techniques may be very helpful in this respect (see [8, 9]). Instead we refer to the literature, e.g., [2, 3, 13, 14, 15, 16, 18, 19, 25], to our recent work [8, 9] and to excellent sources of material on theory and applications of stochastic programs in [24, 26].
2
Stability and Distances of Probability Distributions
Solving stochastic programs computationally requires to replace the original probability distribution P by a probability measure Q having a finite number of scenarios and associated probabilities. Of course, Q has to be determined in such a way that infima and solutions of (1) do not change much if P is replaced by Q. In the following, we use the notations f0 (x, ξ)P (dξ) : x ∈ X v(P ) := inf Rs S(P ) := x ∈ X : f0 (x, ξ)P (dξ) = v(P ) Rs
for the infimum and solution set, respectively, of (1) with respect to P . If the set X is compact, the following estimates are valid
4
W. R¨ omisch
|v(P ) − v(Q)| ≤ sup
f0 (x, ξ)(P − Q)(dξ)
s x∈X R
−1 sup d(y, S(P )) ≤ ψP sup
f0 (x, ξ)(P − Q)(dξ) . x∈X
y∈S(Q)
(7) (8)
Rs
In (7) d(y, S(P )) denotes the distance of y ∈ S(Q) to S(P ). The mapping ψP−1 denotes the inverse of the growth function ψP of the objective in a neighborhood of the solution set S(P ), i.e., ψP (t) := inf f0 (x, ξ)P (dξ) − v(P ) : d(x, S(P )) ≥ t, x ∈ X . Rs
The growth function ψP is monotonically increasing on [0, +∞) and it holds ψP (0) = 0. The estimates (7) and (8) elucidate that the distance
f0 (x, ξ)P (dξ) − f0 (x, ξ)Q(dξ)
d(P, Q) := sup
(9) x∈X
Rs
Rs
of the two probability distributions P and Q becomes important. But, this distance is difficult to evaluate numerically since the function f0 is often very involved (e.g. an infimum function of an optimization model as in the example in Section 1). However, in several important model instances of functions f0 it is possible to derive estimates of d(P, Q) by other distances of probability distributions that are easier to evaluate. We mention the following possibilities: (1) The first one is the classical Koksma-Hlawka inequality (see [17, Section 2.2] and [5]) if the functions f0 (x, · ) are of bounded variation in the sense of Hardy and Krause (uniformly with respect to x ∈ X) on the support [0, 1]s of both probability distributions P and Q. The relevant distance of probability measures is the so-called ∗-discrepancy and is defined in (14). (2) A second idea is to determine a set F of functions from a closed subset Ξ of Rs containing the support of both probability distributions P and Q to R such that Cf0 (x, · ) ∈ F for some constant C > 0 not depending on x. Then it holds 1 dF (P, Q) C
dF (P, Q) := sup
f (ξ)P (dξ) − f (ξ)Q(dξ) . d(P, Q) ≤
f ∈F
Ξ
Ξ
Of course, the set F should be selected such that the distance dF (P, Q) is easier to evaluate than d(P, Q). For two-stage stochastic programs that are (i) linear or (ii) mixed-integer linear the following sets F of functions are relevant (cf. [21, 23, 22]): ˜ ≤ cr (ξ, ξ), ˜ ∀ξ, ξ˜ ∈ Ξ}, (i) Fr (Ξ) = {f : Ξ → R : f (ξ) − f (ξ)
Scenario Reduction Techniques in Stochastic Programming
5
(ii) Fr,B (Ξ) = {f 1lB : f ∈ Fr (Ξ), |f (ξ)| ≤ max{1, |ξ|r }, ∀ξ ∈ Ξ, B ∈ B}, where r ∈ N, B denotes a set of polyhedral subsets of Ξ, | · | is a norm on Rs and 1lB is the characteristic function of the set B, i.e., 1lB (ξ) = 1 for ξ ∈ B and 1lB (ξ) = 0 for ξ ∈ Ξ \ B. Furthermore, the function cr is defined by ˜ := max{1, |ξ|r−1 , |ξ| ˜ r−1 }|ξ − ξ| ˜ (ξ, ξ˜ ∈ Ξ). cr (ξ, ξ) We note that the choice of r and of B depends on the structure of the stochastic program. While the second stage infima are locally Lipschitz continuous as functions of ξ in case of linear two-stage models and belong to Fr (Ξ) for some r ∈ N, they are discontinuous on boundaries of certain polyhedral sets for mixed-integer linear models. Hence, the set Fr,B (Ξ) is relevant in the latter case. An important special case of the class B is the set Brect of all rectangular sets in Rs , i.e. Brect = {I1 × · · · × Is : ∅ = Ij is a closed interval in R, j = 1, . . . , s} , which is relevant for the stability of pure integer second stage models. In case (i) so-called Fortet-Mourier metrics of order r
ζr (P, Q) = sup f (ξ)P (dξ) − f (ξ)Q(dξ)
f ∈Fr (Ξ)
Ξ
(11)
Ξ
appear as special instances of dF , and in case (ii) the metrics
ζr,B (P, Q) = sup f (ξ)P (dξ) − f (ξ)Q(dξ) .
f ∈Fr (Ξ),B∈B
(10)
B
(12)
B
Since a sequence of probability measures on Rs converges with respect to ζr,B if and only if it converges with respect to ζr and with respect to the B-discrepancy αB (P, Q) := sup |P (B) − Q(B)|, B∈B
respectively (cf. [12]), one may consider the ‘mixed’ distance dλ (P, Q) = λ αB (P, Q) + (1 − λ) ζr (P, Q)
(13)
for some λ ∈ (0, 1] instead of the more complicated metric ζr,B . Two specific discrepancies are of interest in this paper. The first one is the rectangular discrepancy αBrect which is needed in Section 3.2. The second one is the ∗-discrepancy α∗ (P, Q) = sup |P ([0, x)) − Q([0, x))|, x∈[0,1]s
(14)
where P and Q are probability measures on [0, 1]s and [0, x) = ×si=1 [0, xi ). As is known from [11] discrepancies are (much) more difficult to handle in scenario reduction compared to Fortet-Mourier metrics (see also Section 3).
6
W. R¨ omisch
If both P and Q are discrete probability distributions (with finite support), the Fortet-Mourier distance ζr (P, Q) may be computed as optimal value of certain linear programs (see also [7]). We show in Section 3.2 that the distance dλ (P, Q) may be computed as well by linear programming (see also [10, 12]). If the set Ξ is compact the Fortet-Mourier metric ζr admits a dual representation as Monge-Kantorovich transportation functional. In particular, it holds for all probability measures on Ξ with finite rth moment (see [20, Section 4.1]): ˜ ˜ : η ∈ M (P, Q) , ζr (P, Q) := inf cˆr (ξ, ξ)η(dξ, dξ) (15) Ξ×Ξ
where M (P, Q) denotes the set of all probability measures η on Ξ × Ξ whose marginal distributions on Ξ are just P and Q, respectively. Furthermore, the function cˆr is the reduced cost function associated with the cost cr and is defined by n+1 ˜ ˜ zj ∈ Ξ, n ∈ N . cˆr (ξ, ξ) := inf cr (zj−1 , zj ) : z0 = ξ, zn+1 = ξ, (16) j=1
The function cˆr is a metric on Ξ and it holds cˆr ≤ cr (cf. [20, Chapter 4.1.3]).
3
Optimal Scenario Reduction
Let P be a probability distribution on Rs with finite support consisting of N scenarios ξ i and their probabilities pi , i ∈ I := {1, . . . , N }. The basic idea of optimal scenario reduction consists in determining a probability distribution Qn which is the best approximation of P with respect to a given distance d of probability measures and whose support consists of a subset of {ξ 1 , . . . , ξ N } with n < N elements. This means d(P, Qn ) = inf{d(P, Q) : Q(Rs ) = 1, supp(Q) ⊂ supp(P ), |supp(Q)| = n}. (17) An equivalent formulation of (17) may be obtained as follows: Let QJ denote a probability measure on Rs with supp(QJ ) = {ξ i : i ∈ {1, . . . , N } \ J} for some index set J ⊂ {1, . . . , N } and let qi , i ∈ {1, . . . , N } \ J, be the probability of scenario indexed by i. Then the minimization problem min d(P, QJ ) : J ⊂ I, |J| = N − n, qi ≥ 0, i ∈ I \ J, qi = 1 (18) i∈I\J ∗
determines some index set J and qi∗ ∈ [0, 1] such that the probability measure with scenarios ξ i and probabilities qi∗ for i ∈ {1, . . . , N } \ J ∗ solves (17). The second formulation (18) of the optimal scenario reduction problem has the advantage to provide a decomposition into an inner and an outer minimization problem, namely, min inf d(P, QJ ) : qi ≥ 0, i ∈ I \ J, qi = 1 : J ⊂ I, |J| = N − n . (19) J
q
i∈I\J
As suggested in [4] the distance d has to be selected such that the stochastic program (1) behaves stable with respect to d (cf. Section 2).
Scenario Reduction Techniques in Stochastic Programming
3.1
7
Two-Stage Stochastic Linear Programs
Here, the distance d is just the Fortet-Mourier metric ζr for some r ∈ N which depends on the structure of the underlying stochastic programming model. For a discussion of the choice of r it is referred to [21, 23]. Since the distance ζr (P, QJ ) has a dual representation as mass transportation problem for any index set J (see (15)), the inner minimization problem in (19) can be solved explicitly. Namely, it holds qi = 1 = pj min cˆr (ξ i , ξ j ), (20) DJ := min ζr (P, QJ ) : qi ≥ 0, i ∈ J, j∈J
i∈J
i∈J
where the reduced costs cˆr have the simplified representation cˆr (ξ i , ξ j ) := min
n−1
cr (ξ lk , ξ lk+1 ) : n ∈ N, lk ∈ I, l1 = i, ln = j
(21)
k=1
as shortest path problem compared with the general form (16). The optimal probabilities qi∗ , i ∈ J, are given by the redistribution rule qi∗ = pi + pj and i(j) ∈ arg min cˆr (ξ i , ξ j ), i ∈ / J, (22) i∈J
j∈J i(j)=i
The redistribution rule consists in adding the probability of a deleted scenario j ∈ J to the probability of a scenario that is nearest to ξ j with respect to the distance cˆr on Rs . The outer minimization problem is of the form pj min cˆr (ξ i , ξ j ) : J ⊂ I, |J| = N − n (23) min DJ = j∈J
i∈I\J
and represents a combinatorial optimization problem of n-median type. It is N P-hard and suggests to use heuristic or approximation algorithms. We also refer to [1] where a branch-and-cut-and-algorithm is developed and tested on large scale instances. In our earlier work [6, 7] we proposed two simple heuristic algorithms: the forward (selection) and the backward (reduction) heuristic. To give a short description of the two algorithms, let cij := cˆr (ξ i , ξ j )
(i, j = 1, . . . , N ).
The basic idea of the forward algorithm originates from the simple structure of (23) for the special case n = 1. It is of the form min
u∈{1,...,N }
N
pj cuj .
j=1 j=u
If the minimum is attained at u∗ , the index set J = {1, . . . , N }\{u∗} solves (23). ∗ The scenario ξ u is taken as the first element of supp(Q). Then the separable
8
W. R¨ omisch
structure of DJ is exploited to determine the second element of supp(Q) while the first element is fixed. The process is continued until n elements of supp(Q) are selected. Forward algorithm for scenario reduction Step 0: J [0] := {1, . . . , N }. Step k: uk ∈ arg min
u∈J [k−1]
pj
j∈J [k−1] \{u}
min
i∈J [k−1] \{u}
cij ,
J [k] := J [k−1] \ {uk } . Step n+1: Redistribution with J := J [n] via (22).
Fig. 1. Illustration of selecting the first, second and third scenario out of N = 5
The idea of the backward algorithm is based on the second special case of (23) for n = N − 1. It is of the form min
l∈{1,...,N }
pl min cil . i=l
If the minimum is attained at l∗ , the index set J = {l∗ } solves (23) in case n = N − 1. After fixing the remaining index set I \ {l∗ } a second scenario is reduced etc. The process is continued until N − n scenarios are reduced. Backward algorithm of scenario reduction Step 0: J [0] := ∅ . Step k: lk ∈ arg min
l∈J [k−1]
j∈J [k−1] ∪{l}
pj
min
i∈J [k−1] ∪{l}
cij ,
J [k] := J [k−1] ∪ {lk } . Step N-n+1: Redistribution with J := J [N −n] via (22). It is shown in [6] that both heuristics are polynomial time algorithms and that the forward heuristics is recommendable at least if n ≤ N4 . 3.2
Two-Stage Stochastic Mixed-Integer Programs
The relevant probability metric is now ζr,B , where r ∈ N and B is a set of polyhedral subsets of a (polyhedral) set Ξ that contains the support of P . It was
Scenario Reduction Techniques in Stochastic Programming
9
Fig. 2. Illustration of the original scenario set with N = 5 and of reducing the first and the second scenario
1000
500 500
0
0
-500 -500
-1000 0
24
48
72
96
120
144
168
0
24
48
72
96
120
144
168
Fig. 3. Left: N = 729 equally distributed scenarios in form of a regular ternary tree. Right: Reduced scenario set with n = 20 and line width proportional to scenario probabilities obtained by the forward heuristic with r = 2.
mentioned in Section 2 that both r and B depend on the structure of the particular stochastic program. Since the complexity of scenario reduction algorithms increase if the sets in B get more involved, we follow here [12] and consider only the set Brect (see (10)) and the distance dλ (P, Q) := λαBrect (P, Q) + (1 − λ)ζr (P, Q) for some λ ∈ (0, 1]. As in Section 3.1 we are interested in computing DJ := min dλ (P, QJ ) : qi ≥ 0, i ∈ / J, qi = 1 .
(24)
(25)
i∈J /
In the following, we show that DJ can be computed as optimal value of a linear program (but a closed formula for DJ as in Section 3.1 is not available in general). To this end, we assume without loss of generality that J = {n + 1, . . . , N }, i.e., supp(QJ ) = {ξ 1 , . . . , ξ n } for some 1 ≤ n < N , consider the system of index sets IBrect := {I(B) := {i ∈ {1, . . . , N } : ξ i ∈ B} : B ∈ Brect } and obtain the following representation of the rectangular discrepancy
10
W. R¨ omisch
αBrect (P, QJ ) = sup |P (B) − QJ (B)| = max
pi − I∈IBrect
B∈Brect
i∈I
qj (26)
j∈I∩{1,...,n}
− j∈I∩{1,...,n} qj ≤ tα − i∈I pi , I ∈ IBrect
(27) = min tα
j∈I∩{1,...,n} qj ≤ tα + i∈I pi , I ∈ IBrect Since the set IBrect may be too large to solve the linear program (27) numerically, we consider the system of reduced index sets IB∗ rect := {I(B) ∩ {1, . . . , n} : B ∈ Brect } and the quantities ∗
γ I := max
pi : I ∈ IBrect , I ∩ {1, . . . , n} = I ∗
i∈I
γI ∗ := min
pi : I ∈ IBrect , I ∩ {1, . . . , n} = I ∗
i∈I
for every I ∗ ∈ IB∗ rect . Since any such index set I ∗ corresponds to some left-hand ∗ side of the inequalities in (27), γ I and γI ∗ correspond to the smallest right-hand sides in (27). Hence, the rectangular discrepancy may be rewritten as I∗
− , I ∗ ∈ IB∗ rect ∗ qj ≤ tα − γ αBrect (P, QJ ) = min tα
j∈I . (28) ∗ ∗ j∈I ∗ qj ≤ tα + γI ∗ , I ∈ IBrect Since the number of elements of IB∗ rect is at most 2n (compared to 2N in IBrect ), passing from (27) to (28) indeed drastically reduces the maximum number of inequalities and may make the linear program (28) numerically tractable. Due to (15) the Fortet-Mourier distance ζr (P, QJ ) allows the representation as linear program ⎫ ⎧
N N n ⎬ ⎨
η = qj , j = 1, . . . , n ηij cˆr (ξ i , ξ j )
ηij ≥ 0, ni=1 i,j ζr (P, QJ ) = inf ⎩
j=1 ηi,j = pi , i = 1, . . . , N ⎭ i=1 j=1
where the reduced cost function cˆr is given by (21). Hence, extending the representation (28) of αBrect we obtain the following linear program for determining DJ and the probabilities qj , j = 1, . . . , n, of the discrete reduced distribution QJ , ⎧
⎫
tα , tζ ≥ 0, qj ≥ 0, nj=1 qj = 1, ⎪ ⎪ ⎪
⎪ ⎪ ⎪ ⎪
ηij ≥ 0, i = 1, . . . , N, j = 1, . . . , n, ⎪ ⎪ ⎪ ⎪
⎪ ⎪ ⎪ N n ⎪
⎪ i j ⎪ ⎪ ⎪
tζ ≥ i=1 j=1 cˆr (ξ , ξ )ηij , ⎪ ⎨ ⎬
n (29) DJ = min λ tα + (1 − λ)tζ
j=1 ηij = pi , i = 1, . . . , N, ⎪ ⎪
N ⎪ ⎪ ⎪ ⎪
i=1 ηij = qj , j = 1, . . . , n, ⎪ ⎪ ⎪ ⎪
⎪ ⎪ ⎪ ⎪ I∗ ∗ ∗ ⎪
⎪ ⎪ − q ≤ t − γ , I ∈ I ∗ j α Brect ⎪ ⎪ ⎪
j∈I ⎩ ⎭
∗ ∗ j∈I ∗ qj ≤ tα + γI ∗ , I ∈ IBrect
Scenario Reduction Techniques in Stochastic Programming
11
Fig. 4. Non-supporting rectangle (left) and supporting rectangle (right). The dots represent the remaining scenarios ξ 1 , . . . , ξ n for s = 2.
While the linear program (29) can be solved efficiently by available software, ∗ determining the index set IB∗ rect and the coefficients γ I , γI ∗ is more intricate. ∗ It is shown in [10, Section 3] that the parameters IB∗ rect and γ I , γI ∗ can be determined by studying the set R of supporting rectangles. A rectangle B in Brect is called supporting if each of its facets contains an element of {ξ1 , . . . , ξn } in its relative interior (see also Fig. 4). Based on R the following representations are valid according to [10, Prop. 1 and 2]: I ∗ ⊆ {1, . . . , n} : ∪j∈I ∗ {ξ j } = {ξ 1 , . . . , ξ n } ∩ int B IB∗ rect = B∈R
γ
I∗
γI ∗
= max P (int B) : B ∈ R ∪j∈I ∗ {ξ j } = {ξ 1 , . . . , ξ n } ∩ int B = pi where I := i ∈ {1, . . . , N } : min∗ ξlj ≤ ξli ≤ max∗ ξlj , l = 1, . . . , s i∈I
j∈I
j∈I
for every I ∗ ∈ IB∗ rect . Here, int B denotes the interior of the set B. An algorithm is developed in [12] that constructs recursively l-dimensional supporting rectangles for l = 1, . . . , s. Computational experiments show that its running grows linearly with N , but depends on n and s via the expression time s n+1 . Hence, while N may be large, only moderately sized values of n given 2 s seem to be realistic. Since an algorithm for computing DJ is now available, we finally look at determining a scenario index set J ⊂ I = {1, . . . , N } with cardinality |J| = N −n such that DJ is minimal, i.e., at solving the combinatorial optimization problem min{DJ : J ⊂ I, |J| = N − n}
(30)
which is known as n-median problem and as N P-hard. One possibility is to reformulate (30) as mixed-integer linear program and to solve it by standard software. Since, however, approximate solutions of (30) are sufficient, heuristic algorithms like forward selection are of interest, where uk is determined in its kth step such that it solves the minimization problem
12
W. R¨ omisch
min DJ [k−1] \{u} u ∈ J [k−1] , where J [0] = {1, . . . , N }, J [k] := J [k−1] \ {uk } (k = 1, . . . , n) and J [n] := {1, . . . , N } \ {u1 , . . . , un } serves as approximate solution to (30). Recalling that the complexity of evaluating DJ [k−1] \{u} for some u ∈ J [k−1] is proportional to
1
0.8
0.6
0.4
0.2
0 0
0.2
0.4
0.6
0.8
1
Fig. 5. N = 1 000 samples ξ i from the uniform distribution on [0, 1]2 and n = 25 points ξ uk , k = 1, . . . , n, obtained via the first 25 elements zk , k = 1, . . . , n, of the Halton sequence (in the bases 2 and 3) (see [17, p. 29]). The probabilities qk of ξ uk , k = 1, . . . , n, are computed for the distance dλ with λ = 1 (gray balls) and λ = 0.9 (black circles) by solving (29). The diameters of the circles are proportional to the probabilities qk , k = 1, . . . , 25.
s k+1 shows that even the forward selection algorithm is expensive. 2 Hence, heuristics for solving (30) become important that require only a low number of DJ evaluations. For example, if P is a probability distribution on [0, 1]s with independent marginal distributions Pj , j = 1, . . . , s, such a heuristic can be based on Quasi-Monte Carlo methods (cf. [17]). The latter provide sequences of equidistributed points in [0, 1]s that approximate the uniform distribution on the unit cube [0, 1]s . Now, let n Quasi-Monte Carlo points z k = (z1k , . . . , zsk ) ∈ [0, 1]s , k = 1, . . . , n, be given. Then we determine (k = 1, . . . , n), y k := F1−1 (z1k ), . . . , Fs−1 (zsk ) where Fj is the (one-dimensional) distribution function of Pj , i.e.,
Scenario Reduction Techniques in Stochastic Programming
Fj (z) = Pj ((−∞, z]) =
N
13
(z ∈ R)
pi
i=1, ξji ≤z
and Fj−1 (t) := inf{z ∈ R : Fj (z) ≥ t} (t ∈ [0, 1]) its inverse (j = 1, . . . , s). Finally, we determine uk as solution of min |ξ u − y k |
u∈J [k−1]
and set again J [k] := J [k−1] \ {uk } for k = 1, . . . , n, where J [0] = {1, . . . , N }. Figure 5 illustrates the results of such a Quasi-Monte Carlo based heuristic and Figure 6 shows the discrepancy αBrect for different n and the running times of the Quasi-Monte Carlo based heuristic.
discrepancy
time 600
0.8 400
0.6 0.4
200 0.2 0
10
20
30
40
50
Fig. 6. Distance αBrect between P (with N = 1000) and equidistributed QMC-points (dashed), QMC-points, whose probabilities are adjusted according to (29) (bold), and running times of the QMC-based heuristic (in seconds)
Acknowledgements. The work of the author is supported by the DFG Research Center Matheon “Mathematics for key technologies” in Berlin.
References [1] Avella, P., Sassano, A., Vasil’ev, I.: Computational study of large scale p-median problems. Math. Program. 109, 89–114 (2007) [2] Casey, M., Sen, S.: The scenario generation algorithm for multistage stochastic linear programming. Math. Oper. Res. 30, 615–631 (2005) [3] Dupaˇcov´ a, J., Consigli, G., Wallace, S.W.: Scenarios for multistage stochastic programs. Ann. Oper. Res. 100, 25–53 (2000) [4] Dupaˇcov´ a, J., Gr¨ owe-Kuska, N., R¨ omisch, W.: Scenario reduction in stochastic programming: An approach using probability metrics. Math. Program. 95, 493– 511 (2003)
14
W. R¨ omisch
[5] G¨ otz, M.: Discrepancy and the error in integration. Monatsh. Math. 136, 99–121 (2002) [6] Heitsch, H., R¨ omisch, W.: Scenario reduction algorithms in stochastic programming. Comp. Optim. Appl. 24, 187–206 (2003) [7] Heitsch, H., R¨ omisch, W.: A note on scenario reduction for two-stage stochastic programs. Oper. Res. Let. 35, 731–736 (2007) [8] Heitsch, H., R¨ omisch, W.: Scenario tree modeling for multistage stochastic programs. Math. Program. 118, 371–406 (2009) [9] Heitsch, H., R¨ omisch, W.: Scenario tree reduction for multistage stochastic programs. Comp. Manag. Science 6, 117–133 (2009) [10] Henrion, R., K¨ uchler, C., R¨ omisch, W.: Discrepancy distances and scenario reduction in two-stage stochastic mixed-integer programming. J. Ind. Manag. Optim. 4, 363–384 (2008) [11] Henrion, R., K¨ uchler, C., R¨ omisch, W.: Scenario reduction in stochastic programming with respect to discrepancy distances. Comp. Optim. Appl. 43, 67–93 (2009) [12] Henrion, R., K¨ uchler, C., R¨ omisch, W.: A scenario reduction heuristic for twostage stochastic mixed-integer programming (in preparation) [13] Hochreiter, R., Pflug, G.C.: Financial scenario generation for stochastic multistage decision processes as facility location problems. Ann. Oper. Res. 152, 257– 272 (2007) [14] Høyland, K., Kaut, M., Wallace, S.W.: A heuristic for moment-matching scenario generation. Comp. Optim. Appl. 24, 169–185 (2003) [15] Kuhn, D.: Generalized bounds for convex multistage stochastic programs. LNEMS, vol. 548. Springer, Berlin (2005) [16] Kuhn, D.: Aggregation and discretization in multistage stochastic programming. Math. Program. 113, 61–94 (2008) [17] Niederreiter, H.: Random number generation and Quasi-Monte Carlo methods. CBMS-NSF Regional Conference Series in Applied Mathematics Vol. 63. SIAM, Philadelphia (1992) [18] Pag`es, G., Pham, H., Printems, J.: Optimal quantization methods and applications to numerical problems in finance. In: Rachev, S.T. (ed.) Handbook of Computational and Numerical Methods in Finance, pp. 253–297. Birkh¨ auser, Boston (2004) [19] Pennanen, T.: Epi-convergent discretizations of multistage stochastic programs via integration quadratures. Math. Program. 116, 461–479 (2009) [20] Rachev, S.T., R¨ uschendorf, L.: Mass Transportation Problems, vol. I, II. Springer, Berlin (1998) [21] R¨ omisch, W.: Stability of Stochastic Programming Problems. In: Ruszczy´ nski, A., Shapiro, A. (eds.) Stochastic Programming. Handbooks in Operations Research and Management Science, vol. 10, pp. 483–554. Elsevier, Amsterdam (2003) [22] R¨ omisch, W., Vigerske, S.: Quantitative stability of fully random mixed-integer two-stage stochastic programs. Optim. Lett. 2, 377–388 (2008) [23] R¨ omisch, W., Wets, R.J.-B.: Stability of ε-approximate solutions to convex stochastic programs. SIAM J. Optim. 18, 961–979 (2007) [24] Ruszczy´ nski, A., Shapiro, A. (eds.): Stochastic Programming. Handbooks in Operations Research and Management Science, vol. 10. Elsevier, Amsterdam (2003) [25] Shapiro, A.: Inference of statistical bounds for multistage stochastic programming problems. Math. Meth. Oper. Res. 58, 57–68 (2003) [26] Wallace, S.W., Ziemba, W.T. (eds.): Applications of Stochastic Programming. MPS-SIAM Series in Optimization (2005)
Statistical Learning of Probabilistic BDDs Taisuke Sato Department of Computer Science Graduate School of Information Science and Engineering Tokyo Institute of Technology
[email protected]
Abstract. BDDs (binary decision diagrams) are a graphical data structure which compactly represents Boolean functions and has been extensively used in VLSI CAD. In this talk, I will describe how BDDs are applied to machine learning, or more concretely how they are used in probability computation and statistical parameter learning for probabilistic modeling. The basic idea is that of PPC (propositionalized probabilistic computation) which considers “X = x”, a random variable X taking a value x, as a probabilistic proposition. A complex (discrete) probabilistic model such as a Bayesian network is representable by way of PPC as a probabilistic Boolean formula, thus by a probabilistic BDD. We developed an efficient sum-product probability computation algorithm on BDDs and also derived a parameter learning algorithm called the BDD-EM algorithm for maximum likelihood estimation in statistics. We applied the BDD-EM algorithm to the problem of finding a most probable hypothesis about a metabolic pathway in systems biology.
O. Watanabe and T. Zeugmann (Eds.): SAGA 2009, LNCS 5792, p. 15, 2009. c Springer-Verlag Berlin Heidelberg 2009
Learning Volatility of Discrete Time Series Using Prediction with Expert Advice Vladimir V. V’yugin Institute for Information Transmission Problems, Russian Academy of Sciences, Bol’shoi Karetnyi per. 19, Moscow GSP-4, 127994, Russia
[email protected]
Abstract. In this paper the method of prediction with expert advice is applied for learning volatility of discrete time series. We construct arbitrage strategies (or experts) which suffer gain when “micro” and “macro” volatilities of a time series differ. For merging different expert strategies in a strategy of the learner, we use some modification of Kalai and Vempala algorithm of following the perturbed leader where weights depend on current gains of the experts. We consider the case when experts onestep gains can be unbounded. New notion of a volume of a game vt is introduced. We show that our algorithm has optimal performance in the case when the one-step increments Δvt = vt − vt−1 of the volume satisfy Δvt = o(vt ) as t → ∞.
1
Introduction
In this paper we construct arbitrage strategies which suffer gain when “micro” and “macro” volatilities of a time series differ. For merging different expert strategies in a strategy of the learner, we use methods of the theory of prediction with expert advice. Using Cheredito [2] results in mathematical finance, Vovk [14] considered two strategies which suffer gain when prices of a stock follow fractional Brownian motion: the first strategy shows a large return in the case when volatility of the time series is high, the second one shows a large return in the opposite case. The main peculiarity of these arbitrage strategies is that their one-step gains and losses can be unrestrictedly large. Also, there is no appropriate type of a loss function for these strategies, and we are forced to consider general gains and losses. We construct a learning algorithm merging online these strategies into a one Learner’s strategy protected from these unbounded fluctuations as much as possible. Prediction with Expert Advice considered in this paper proceeds as follows. A “pool” of expert strategies (or simply, experts) i = 1, . . . N is given. We are asked to perform sequential actions at times t = 1, 2, . . . , T . At each time step t, experts i = 1, . . . N receive results of their actions in form of their gains or losses sit . In what follows we refer to sit as to gains. At the beginning of the step t Learner, observing cumulated gains si1:t−1 = si1 + . . . + sit−1 of all experts i = 1, . . . N , assigns non-negative weights wti (summing O. Watanabe and T. Zeugmann (Eds.): SAGA 2009, LNCS 5792, pp. 16–30, 2009. c Springer-Verlag Berlin Heidelberg 2009
Learning Volatility of Discrete Time Series
17
to 1) to each expert i and suffers a gain equal to the weighted sum of experts N gains s˜t = i=1 wti sit . T The cumulative gain of the learner on first T steps is equal to s˜1:T = t=1 s˜t . This can be interpreted in probabilistic terms as follows. On each time step t, Learner choose to follow an expert i according to the internal distribution P {It = i} = wti , i = 1, . . . N ; at the end of step t Learner receives the same gain sit as the ith expert and suffers Learner’s cumulative gain s1:t = s1:t−1 + sit . Let E(st ) denote the Learner’s expected one-step gain according to this randomization; it coincides with the weighted sum s˜t of the experts gains. The cumulative expected gain of our learning algorithm on first T steps is equal to T E(s1:T ) = t=1 E(st ). Evidently, s˜1:T = E(s1:T ) for all T . The goal of the learner’s algorithm is to perform in terms of s˜1:T = E(s1:T ) almost as well as the best expert in hindsight in the long run. In the traditional framework of prediction with expert advice, it is supposed that one-step gains of experts are bounded, for example, 0 ≤ sit ≤ 1 for all i and t. We allow gains to be unbounded. We consider also the case when the notions of loss and gain functions are not used. In this paper we use the method of following the perturbed leader. This method was discovered by Hannan [5]. Kalai and Vempala [7] rediscovered this method and published a simple proof of the main result of Hannan. They called the algorithm of this type FPL (Following the Perturbed Leader). Hutter and Poland [6] presented a further developments of the FPL algorithm for countable class of experts, arbitrary weights and adaptive learning rate. Also, FPL algorithm is usually considered for bounded one-step losses: 0 ≤ sit ≤ 1 for all i and t. The similar results can be achieved by other aggregate strategies, like Weighted Majority (WM) algorithm of Littlestone and Warmuth [8] or algorithm “hedge” of Freund and Schapire [4]. The FPL √algorithm has the same performance as the WM-type algorithms up to a factor 2. A major advantage of the FPL algorithm is that its analysis remains easy for an adaptive learning rate, in contrast to WMderivatives (see remark in [6]). Most papers on prediction with expert advice either consider bounded losses or assume the existence of a specific loss function. The setting allowing unbounded one-step losses (or gains) do not have wide coverage in literature; we can only refer reader to [1], [10], where polynomial bounds on one-step losses were considered. In this paper, we present some modification of Kalai and Vempala [7] algorithm of following the perturbed leader (FPL) for the case of unrestrictedly large one-step expert gains sit ∈ (−∞, +∞) not bounded in advance. This algorithm uses adaptive weights depending on past cumulative gains of the experts. t We introduce new notions of volume of a game vt = maxi |sij | and scaled j=1
fluctuation of the game fluc(t) = Δvt /vt , where Δvt = vt − vt−1 for t ≥ 1. In Section 2 we present a game with two zero-sum experts which suffer gain or loss when micro and macro volatilities of stock prices differ. These gains and
18
V.V. V’yugin
losses cannot be bounded in advance. The notion of the volume of a game is natural from this financial point of view. In Section 3 we consider a more general problem - we consider a game with N experts, where these experts suffer unbounded gains sit ∈ (−∞, +∞). We present some probabilistic learning algorithm merging the experts decisions and prove that this algorithm is performed well under very broad assumptions. In Section 4 we discuss a derandomized version this algorithm for the case of two zero-sum experts with arbitrary one-step gains from Section 2. We show in Theorem 1 (Section 3) that if fluc(t) ≤ γ(t) for all t, where γ(t) is a non-increasing computable real function such that 0 < γ(t) ≤ 1 for all t, then the algorithm of following the perturbed leader with adaptive weights constructed in Section 3 has the performance E(s1:T ) ≥ max si1:T − 2 i=1,...N
T 6(1 + ln N ) (γ(t))1/2 Δvt . t=1
If γ(t) → 0 as t → ∞ this algorithm is asymptotically consistent in a modified sense lim inf T →∞
1 E(s1:T − max si1:T ) ≥ 0, i=1,...N vT
(1)
where s1:T is the total gain of our algorithm on steps ≤ T . Proposition 1 of Section 3 shows that if the condition Δvt = o(vt ) is violated the cumulative gain of any probabilistic prediction algorithm can be much less than the gain of some expert of the pool.
2
Arbitrage Strategies
We consider a time series as a series of some stock prices. Then each expert from the pool will represent a method of playing on the stock market. Let K, M and T be positive integer numbers and let the time interval [0, KT ] be divided on a large number KM of subintervals. Let also S = S(t) be a function representing a stock price at time t. Define a discrete time series of stock prices S0 = S(0), S1 = S(T /(KM )), S2 = S(2T /(KM )) . . . , SKM = S(T ). In this paper, volatility is an informal notion. We say that the sum K−1
(S(i+1)T − SiT )2
i=0
represents the macro volatility and the sum KT −1
(ΔSi )2 ,
i=0
(2)
Learning Volatility of Discrete Time Series
19
where ΔSi = Si+1 − Si , i = 1, . . . KT , represent the micro volatility of the time series (2). In this paper for simplicity we consider the case K = 1. Informally speaking, the first strategy will show a large return if (ST −S0 )2 T −1 (ΔSi )2 ; the second one will show a large return when (ST − S0 )2 i=0 T −1 i=0
(ΔSi )2 . There is an uncertainty domain for these strategies, i.e., the case
when both and do not hold.1 We consider the game between an investor and the market. The investor can use the long and short selling. At beginning of time step t Investor purchases the number Ct of shares of the stock by St−1 each. At the end of trading period the market discloses the price St+1 of the stock, and the investor incur his current income or loss st = Ct ΔSt at the period t. We have the following equality T −1
(ST − S0 )2 = (
ΔSt )2 =
t=0 T −1
2(St − S0 )ΔSt +
t=0
T −1
(ΔSt )2 .
(3)
t=0
The equality (3) leads to the two strategies for investor which are represented by two experts. At the beginning of step t Experts 1 and 2 hold the number of shares Ct1 = 2C(St − S0 ),
(4)
−Ct1 ,
(5)
Ct2
=
where C is an arbitrary positive constant. These strategies at step t earn the incomes s1t = 2C(St −S0 )ΔSt and s2t = −s1t . The strategy (4) earns in first T steps of the game the income s11:T
=
T t=1
s1t
= 2C((ST − S0 ) − 2
T −1
(ΔSt )2 ).
t=1
The strategy (5) earns in first T steps the income s21:T = −s11:T . The number of shares Ct1 in the strategy (4) or number of shares Ct2 = −Ct1 in the strategy (5) can be positive or negative. The one-step gains s1t and s2t = −s1t are unbounded and can be positive or negative: sit ∈ (−∞, +∞). 1
The idea of these strategies is based on the paper of Cheredito [2] (see also Rogers [11], Delbaen and Schachermayer [3]) who have constructed arbitrage strategies for a financial market that consists of money market account and a stock whose price follows a fractional Brownian motion with drift or an exponential fractional Brownian motion with drift. Vovk [14] has reformulated these strategies for discrete time. We use these strategies to define a mixed strategy which incur gain when macro and micro volatilities of time series differ. There is no uncertainty domain for continuous time.
20
V.V. V’yugin
We analyze this game in the follow leader framework. We introduce Learner that can choose between two strategies (4) and (5). The following simple example of zero sum experts shows that even in the case where one-step gains of experts are bounded Learner can perform much worse than each expert: let the current gains of two experts on steps t = 0, 1, . . . , 6 be s10,1,2,3,4,5,6 = (1/2, −1, 1, −1, 1, −1, 1) and s20,1,2,3,4,5,6 = (0, 1, −1, 1, −1, 1, −1). Suppose that s10 = s20 = 0. The “Follow Leader” algorithm always chooses the wrong prediction; its income is s1:6 = −5.5. We solve this problem in Section 3 using randomization of the experts cumulative gains.
3
The Follow Perturbed Leader Algorithm with Adaptive Weights
We consider a game of prediction with expert advice. Let at each step t of the game all N experts receive arbitrary one-step gains sit ∈ (−∞, +∞), i = 1, . . . N , and the cumulative gain of the ith expert after step t is equal to si1:t = si1:t−1 +sit . A probabilistic learning algorithm of choosing an expert presents for any i the probabilities P {It = i} of following the ith expert given the cumulative gains si1:t−1 of the experts i = 1, . . . N in hindsight. Probabilistic algorithm of choosing an expert. FOR t = 1, . . . T Given past cumulative gains of the experts si1:t−1 choose the expert i, where i = 1, . . . N , with probability P {It = i}. Receive the one-step gains at step t of the expert sit and suffer one-step gain st = sit of the master algorithm. ENDFOR The performance of this probabilistic algorithm is measured in its expected regret E( max si1:T − s1:T ), i=1,...N
where the random variable s1:T is the cumulative gain of the master algorithm, si1:T , i = 1, . . . N , are the cumulative gains of the expert algorithms, E is the mathematical expectation. In this section we explore asymptotic consistency of probabilistic learning algorithms in case of unbounded one-step gains. A probabilistic algorithm is called asymptotically consistent if lim inf T →∞
1 E(s1:T − max si1:T ) ≥ 0. i=1,...N T
(6)
Notice that then 0 ≤ sit ≤ 1 all expert algorithms have total gain ≤ T on first T steps. This is not true for the unbounded case, and there are no reasons to divide the expected regret (6) on T . We modify the definition (6) of the normalized expected regret as follows. Define the volume of a game at step t
Learning Volatility of Discrete Time Series
vt =
t j=1
21
max |sij |. i
Evidently, vt−1 ≤ vt for all t ≥ 1. Put v0 = 0. A probabilistic learning algorithm is called asymptotically consistent (in the modified sense) in a game with N experts if lim inf T →∞
1 E(s1:T − max si1:T ) ≥ 0. i=1,...N vT
(7)
A game is called non-degenerate if vt → ∞ as t → ∞. Denote Δvt = vt − vt−1 for t ≥ 1. The number fluc(t) =
Δvt maxi |sit | = , vt vt
(8)
is called scaled fluctuation of the game at the step t (put 0/0 = 0). By definition 0 ≤ fluc(t) ≤ 1 for all t ≥ 1. The following simple proposition shows that any probabilistic learning algorithm cannot be asymptotically optimal for some game such that fluc(t) → 0 as t → ∞. For simplicity, we consider the case of two experts. Proposition 1. For any probabilistic algorithm of choosing an expert and for any > 0 two experts exist such that fluc(t) ≥ 1 − , 1 1 E(max si1:t − s1:t ) ≥ (1 − ) vt i=1,2 2 for all t, where s1:t is the cumulative gain of this algorithm. Proof. Given a probabilistic algorithm of choosing an expert and such that 0 < < 1, define recursively one-step gains s1t and s2t of expert 1 and expert 2 at any step t = 1, 2, . . . as follows. By s11:t−1 and s21:t−1 denote the cumulative gains of these experts incurred at steps ≤ t − 1; let vt−1 be the corresponding volume, where t = 1, 2, . . .. Define v0 = 1 and Mt = 4vt−1 / for t ≥ 1. For t ≥ 1, define s1t = 0 and s2t = Mt if P {It = 1} > 12 , and define s1t = Mt and s2t = 0 otherwise. Let for t ≥ 1, vt be the volume of the game. Let st be one-step gain of the master algorithm and s1:t be its cumulative gain at step t ≥ 1. By definition for all t ≥ 1, E(st ) = s1t P {It = 1} + s2t P {It = 2} ≤
1 Mt . 2
Evidently, E(s1,1 ) = E(s1 ) and E(s1:t ) = E(s1:t−1 ) + E(st ) for all t ≥ 2. Then E(s1:t ) ≤ 12 (1 + /2)Mt for all t ≥ 1. Evidently, vt = vt−1 + Mt = Mt (1 + /4) and Mt ≤ max si1:t for all t ≥ 1. i
22
V.V. V’yugin
Therefore, the normalized expected regret of the master algorithm is bounded from below 1 (1 − /2)Mt 1 1 ≥ (1 − ). E(max si1:t − s1:t ) ≥ 2 i vt Mt (1 + /4) 2
for all t. Let ξ 1 ,. . . ξ N be a sequence of i.i.d random variables distributed according to the exponential law with the density p(x) = exp{−x}. Let γ(t) be a non-increasing real function such that 0 < γ(t) ≤ 1 for all t; for example, γ(t) = t−δ , where δ > 0 and t ≥ 1. Define N ln 1+ln 1 6 1− and (9) αt = 2 ln γ(t) 6 αt μt = (γ(t)) = (γ(t))1/2 . (10) 1 + ln N for all t.2 We consider an FPL algorithm with a variable learning rate t =
1 , μt vt−1
(11)
where μt is defined by (10) and the volume vt−1 depends on experts actions on steps < t. By definition vt ≥ vt−1 and μt ≤ μt−1 for t = 1, 2, . . .. Also, if γ(t) → 0 as t → ∞ then μt → 0 as t → ∞. We suppose without loss of generality that si0 = v0 = 0 for all i and 0 = ∞. The FPL algorithm is defined as follows: FPL algorithm. FOR t = 1, . . . T Define It = argmaxi {si1:t−1 + 1t ξ i }, where t is defined by (11) and i = 1, 2, . . . N .3 Receive one-step gains sit for experts i = 1, . . . , N , and receve one-step gain It st of the FPL algorithm. ENDFOR T sIt t be the cumulative gain of the FPL algorithm. Let s1:T = t=1
The following theorem gives a bound for regret of the FPL algorithm. It shows also that if the game is non-degenerate and Δvt = o(vt ) as t → ∞ with algorithmic bound then the FPL-algorithm with variable parameters μt is asymptotically consistent. 2
3
The choice of the optimal value of αt will be explained later. It will be obtained by minimization of the corresponding member of the sum (40) below. The definition (9) is invalid for γ(t) = 1. In that follows for γ(t) = 1, we will use the values (γ(t))αt and (γ(t))1−αt defined by (10). If the maximum is achieved for more then one different i choose the minimal such i.
Learning Volatility of Discrete Time Series
23
Theorem 1. Let γ(t) be a non-increasing computable real function such that 0 < γ(t) ≤ 1 and fluc(t) ≤ γ(t)
(12)
for all t. Then the expected gain of the FPL algorithm with variable learning rate (11) is satisfied T E(s1:T ) ≥ max si1:T − 2 6(1 + ln N ) (γ(t))1/2 Δvt . i
(13)
t=1
for all T . If the game is non-degenerate and γ(t) → 0 as t → ∞ this algorithm is asymptotically consistent lim inf T →∞
1 E(s1:T − max si1:T ) ≥ 0. i=1,...N vT
(14)
Proof. The analysis of optimality of the FPL algorithm is based on an intermediate predictor IFPL (Infeasible FPL). IFPL algorithm. FOR t = 1, . . . T Define t =
1 , μt vt
(15)
where vt is the volume of the game at step t and μt is defined by (10). Also, define Jt = argmaxi {si1:t + 1 ξ i }. t
Receive the one step gain sJt t of the IFPL algorithm. ENDFOR The IFPL algorithm predicts under the knowledge of t and si1:t , i = 1, . . . N , which may not be available at beginning of step t. Using unknown value of t and si1:t is the main distinctive feature of our version of IFPL. For any t we have 1 i ξ }, t 1 1 Jt = argmaxi {si1:t + ξ i } = argmaxi {si1:t−1 + sit + ξ i }. t t It = argmaxi {si1:t−1 +
The expected one-step and cumulated gains of the FPL and IFPL algorithms at step t are denoted lt = E(sIt t ) and rt = E(sJt t ), l1:T =
T t=1
lt and r1:T =
T t=1
rt ,
24
V.V. V’yugin
respectively, where sIt t is the one-step gain of the FPL algorithm at step t and sJt t is the one-step gain of the IFPL algorithm, and E denotes the mathematical expectation. Lemma 1. The total expected gains of the FPL and IFPL algorithms satisfy the inequality l1:T ≥ r1:T − 6
T
(γ(t))1−αt Δvt
(16)
t=1
for all T . Proof. For any j = 1, . . . n and fixed reals c1 , . . . cN define 1 ci }, t 1 1 mj = max{si1:t + ci } = max{si1:t−1 + sit + ci }. i=j i=j t t mj = max{si1:t−1 + i=j
1 2 Let mj = sj1:t−1 + 1t cj 1 and mj = sj1:t−1 + sjt2 + and since j1 = j we have
1 t cj2 .
By definition of mj
1 1 1 cj ≥ sj1:t−1 + cj 1 = t 2 t 1 1 1 1 sj1:t−1 + cj1 + − cj1 = t t t 1 1 mj + − cj1 . t t
2 mj = sj1:t−1 + sjt2 +
(17)
We compare conditional probabilities P {It = j|ξ i = ci , i = j} and P {Jt = j|ξ i = ci , i = j}. The following chain of equalities and inequalities is valid: P {It = j|ξ i = ci , i = j} = 1 P {sj1:t−1 + ξ j ≥ mj |ξ i = ci , i = j} = t P {ξ j ≥ t (mj − sj1:t−1 )|ξ i = ci , i = j} = P {ξ j ≥ t (mj − sj1:t−1 ) + (t − t )(mj − sj1:t−1 )|ξ i = ci , i = j} = 1 1 P {ξ j ≥ t (mj − sj1:t−1 ) + (t − t )(sj1:t−1 − sj1:t−1 + cj1 )|ξ i = ci , i = j} = t 1 exp{−(t − t )(sj1:t−1 − sj1:t−1 )} × 1 P {ξ j ≥ t (mj − sj1:t−1 ) + (t − t ) cj1 |ξ i = ci , i = j} ≥ t 1 exp{−(t − t )(sj1:t−1 − sj1:t−1 )} × 1 1 1 P {ξ j ≥ t (mj − sj1:t + sjt − − cj1 ) + (t − t ) cj1 |ξ i = ci , i = j} = t t t
(18) (19) (20)
(21)
Learning Volatility of Discrete Time Series 1 exp{−(t − t )(sj1:t−1 − sj1:t−1 ) − t sjt } ×
j
exp −
t (mj
P {ξ ≥
1 1 − μt vt−1 μt vt
−
sj1:t )|ξ i
= ci , i = j} = sj 1 − sj1:t−1 ) − t (sj1:t−1 × μt vt
1 (sj ) − mj |ξ i = ci , i = j} ≥ P {ξ j > μt vt 1:t
j j1 ) Δvt Δvt (s1:t−1 − s1:t−1 − × exp − μt vt vt−1 μt vt P {ξ j > Δvt exp − μt vt
1+
25 (22) (23)
(24)
1 (mj − sj1:t )|ξ i = ci , i = j} = μt vt j
1 − s1:t−1 sj1:t−1 vt−1
P {Jt = 1|ξ i = ci , i = j}.
(25)
Here we have used twice, in (18)-(19) and in (21)-(22), the equality P {ξ > a + b} = e−b P {ξ > a} for any random variable ξ distributed according to the exponential law. The equality (20)-(21) follows from (17) and t ≥ t for all t. We have used in (24) the equality vt − vt−1 = maxi |sit |. The expression in the exponent (25) is bounded j sj1 1:t−1 − s1:t−1 (26) ≤ 2, vt−1 i s since v1:t−1 ≤ 1 for all t and i. t−1 Therefore, we obtain P {It = j|ξ i = ci , i = j} ≥
3 Δvt exp − P {Jt = j|ξ i = ci , i = j} ≥ μt vt exp{−3(γ(t))1−αt }P {Jt = j|ξ i = ci , i = j}.
(27) (28)
Since, the inequality (28) holds for all ci , it also holds unconditionally P {It = j} ≥ exp{−3(γ(t))1−αt }P {Jt = j}. for all t = 1, 2, . . . and j = 1, . . . N . Since sjt + Δvt ≥ 0 for all j and t, we obtain from (29) lt + Δvt = E(sIt t + Δvt ) =
N
(sjt + Δvt )P (It = j) ≥
j=1
exp{−3(γ(t))1−αt }
N
(sjt + Δvt )P (Jt = j) =
j=1
exp{−3(γ(t))1−αt }(E(sJt t ) + Δvt ) =
(29)
26
V.V. V’yugin
exp{−3(γ(t))1−αt }(rt + Δvt ) ≥ (1 − 3(γ(t))1−αt )(rt + Δvt ) = rt + Δvt − 3(γ(t))1−αt (rt + Δvt ) ≥ rt + Δvt − 6(γ(t))1−αt Δvt .
(30)
In the last line of (30) we have used the inequality |rt | ≤ Δvt for all t and the inequality exp{−3r} ≥ 1 − 3r for all r. Subtracting Δvt from both sides of the inequality (30) and summing it by t = 1, . . . T , we obtain l1:T ≥ r1:T − 6
T
(γ(t))1−αt Δvt
t=1
for all T . Lemma 1 is proved. The following lemma gives a lower bound for the gain of the IFPL algorithm. Lemma 2. The expected cumulative gain of the IFPL algorithm with the learning rate (15) is bounded by r1:T ≥ max si1:T − (1 + ln N ) i
T
(γ(t))αt Δvt
(31)
t=1
for all T . Proof. The proof is along the line of the proof from Hutter and Poland [6] with an exception that now the sequence t is not monotonic. Let in this proof, st = (s1t , . . . sN t ) be a vector of one-step gains and s1:t = 1 (s1:t , . . . sN ) be a vector of cumulative gains of the experts algorithms. Also, 1:t let ξ = (ξ 1 , . . . ξ N ) be a vector whose coordinates are i.i.d according to the exponential law random variables. Recall that t = 1/(μt vt ) and v0 = 0, 0 = ∞. Define s˜1:t = s1:t + 1 ξ for t = 1, 2, . . .. Consider the one-step gains s˜t = t for the moment. For any vector s and a unit vector d denote st + ξ 1 − 1 t
t−1
M (s) = argmaxd∈D {d · s}, where D = {(0, . . . 1), . . . (1, . . . 0)} is the set of N unit vectors of dimension N and “·” is the inner product of two vectors. We first show that T
M (˜ s1:t ) · s˜t ≥ M (˜ s1:T ) · s˜1:T .
(32)
t=1
For T = 1 this is obvious. For the induction step from T − 1 to T we need to show that s1:T ) · s˜1:T − M (˜ s1:T −1 ) · s˜1:T −1 . M (˜ s1:T ) · s˜T ≥ M (˜
Learning Volatility of Discrete Time Series
27
This follows from s˜1:T = s˜1:T −1 + s˜T and s1:T −1 ) · s˜1:T −1 . M (˜ s1:T ) · s˜1:T −1 ≤ M (˜ We rewrite (32) as follows T
M (˜ s1:t ) · st ≥ M (˜ s1:T ) · s˜1:T −
t=1
T
M (˜ s1:t ) · ξ
t=1
1 1 − t t−1
.
(33)
By the definition of M we have
ξ = M (˜ s1:T ) · s˜1:T ≥ M (s1:T ) · s1:T + T ξ max{d · s1:T } + M (s1:T ) · . d∈D T
(34)
We have T 1
t
t=1
−
1 t−1
M (˜ s1:t ) · ξ =
T
(μt vt − μt−1 vt−1 )M (˜ s1:t ) · ξ.
(35)
t=1
We will use the inequality 0 ≤ E(M (˜ s1:t ) · ξ) ≤ E(M (ξ) · ξ) = E(max ξ i ) ≤ 1 + ln N. i
(36)
The proof of this inequality uses an idea of Lemma 1 from [6]. We have for the exponentially distributed random variables ξ i , i = 1, . . . N , i
i
P {max ξ ≥ a} = P {∃i(ξ ≥ a)} ≤ i
N
P {ξ i ≥ a} = N exp{−a}.
Since for any non-negative random variable η, E(η) = we have E(maxi ξ i − ln N ) = ∞
(37)
i=1
∞
∞
P {η ≥ y}dy, by (37)
0
P {maxi ξ i − ln N ≥ y}dy ≤
0
N exp{−y − ln N }dy = 1. Therefore, E(maxi ξ i ) ≤ 1 + ln N .
0
By (36) the expectation of (35) has the upper bound T t=1
E(M (˜ s1:t ) · ξ)(μt vt − μt−1 vt−1 ) ≤ (1 + ln N )
T
μt Δvt .
t=1
Here we have used the inequality μt ≤ μt−1 for all t, Since E(ξ i ) = 1 for all i, the expectation of the last term in (34) is equal to ξ 1 (38) E M (s1:T ) · = = μT vT . T T
28
V.V. V’yugin
Combining bounds (33)-(35) and (38), we obtain T r1:T = E M (˜ s1:t ) · st ≥ t=1
max si1:T + μT vT − (1 + ln N ) i
T
μt Δvt ≥
t=1
max si1:T − (1 + ln N ) i
T
μt Δvt .
(39)
t=1
Lemma is proved. . We finish now the proof of the theorem. The inequality (16) of Lemma 1 and the inequality (31) of Lemma 2 imply the inequality E(s1:T ) ≥ max si1:T − i
T
(6(γ(t))1−αt + (1 + ln N )(γ(t))αt )Δvt .
(40)
t=1
for all T . The optimal value (9) of αt can be easily obtained by minimization of each member of the sum (40) by αt . In this case μt is equal to (10) and (40) is equivalent to (13). Let γ(T ) → 0 as T → ∞ and the game is non-degenerate, i.e., vT → ∞ as T → ∞. We have Tt=1 Δvt = vT for all T . Then by Toeplitz lemma [13] 1 vT
T (γ(t))1/2 Δvt 2 6(1 + ln N )
→0
t=1
as T → ∞. This limit and the inequality (13) imply the expected asymptotic consistency (14). Theorem is proved. In [1] and [10] polynomial bounds on one-step losses were considered. We also present a bound of this type. Corollary 1. Suppose that maxi |sit | ≤ ta for all t and and lim inf t→∞ where i = 1, . . . N and a, δ are positive real numbers. Then 1 E(s1:T ) ≥ max si1:T − O( (1 + ln N ))T 1− 2 δ+a
vt ta+δ
> 0,
i
as T → ∞, where γ(t) = t−δ and μt is defined by (10).
4
Zero-Sum Game
In this section we derandomize the zero-sum game from Section 2. We interpret the expected one-step gain E(st ) gain as the weighted average of one-step gains
Learning Volatility of Discrete Time Series
29
of experts strategies. In more detail, at each step t, Learner divide his investment in proportion to the probabilities of expert strategies (4) and (5) computed by the FPL algorithm and suffers the gain Gt = 2C(St − S0 )(P {It = 1} − P {It = 2})ΔSt
T at any step t, where C is an arbitrary positive constant; G1:T = t=1 Gt = gain. E(s1:T ) is the Learner’s cumulative t If the game satisfies |s1t |/ i=1 |s1i | ≤ γ(t) for all t then by (13) we have the lower bound G1:T ≥ |s11:T | − 8
T
(γ(t))1/2 |s1t |
t=1
for all T . Assume that γ(t) = μ for all t. Then G1:T ≥ |s11:T | − 8μ1/2 vT for all T .
5
Conclusion
In this paper we study two different problems: the first of them is how use the fractional Brownian motion of prices to suffer gain on financial market; the second one consists in extending methods of the theory of prediction with expert advice for the case when experts one-step gains are unbounded. Though these problems look independent, the first of them serves as a motivating example to study averaging strategies in case of unbounded gains and losses. The FPL algorithm with variable learning rates is simple to implement and it is bringing satisfactory experimental results when prices follow fractional Brownian motion. There are some open problems for further research. How to construct a defensive strategy for Learner in sense of Shafer and Vovk’s book [12]? This means that Learner starting with some initial capital never goes to debt and suffer a gain when macro and micro volatilities differ. Also, a general problem is to develop another strategies which suffer gain when prices follow fractional Brownian motion. In the theory of prediction with expert advice, it is useful to analyze the performance of the well known algorithms (like WM) for the case of unbounded gains and losses in terms of the volume of a game. There is a gap between Proposition 1 and Theorem 1, since we assume in this theorem that the game satisfies fluc(t) ≤ γ(t) → 0, where γ(t) is computable. Does there exists an asymptotically consistent learning algorithm in case where fluc(t) → 0 as t → ∞ with no computable upper bound?
Acknowledgments This research was partially supported by Russian foundation for fundamental research: 09-07-00180-a and 09-01-00709-a. Author thanks Volodya Vovk and Yuri Kalnishkan for useful comments.
30
V.V. V’yugin
References [1] Allenberg, C., Auer, P., Gyorfi, L., Ottucsak, G.: Hannan consistency in on-Line learning in case of unbounded losses under partial monitoring. In: Balc´ azar, J.L., Long, P.M., Stephan, F. (eds.) ALT 2006. LNCS (LNAI), vol. 4264, pp. 229–243. Springer, Heidelberg (2006) [2] Cheredito, P.: Arbitrage in fractional Brownian motion. Finance and Statistics 7, 533 (2003) [3] Delbaen, F., Schachermayer, W.: A general version of the fundamental theorem of asset pricing. Mathematische Annalen 300, 463–520 (1994) [4] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55, 119–139 (1997) [5] Hannan, J.: Approximation to Bayes risk in repeated plays. In: Dresher, M., Tucker, A.W., Wolfe, P. (eds.) Contributions to the Theory of Games 3, pp. 97– 139. Princeton University Press, Princeton (1957) [6] Hutter, M., Poland, J.: Prediction with expert advice by following the perturbed leader for general weights. In: Ben-David, S., Case, J., Maruoka, A. (eds.) ALT 2004. LNCS (LNAI), vol. 3244, pp. 279–293. Springer, Heidelberg (2004) [7] Kalai, A., Vempala, S.: Efficient algorithms for online decisions. In: Sch¨olkopf, B., Warmuth, M.K. (eds.) COLT/Kernel 2003. LNCS (LNAI), vol. 2777, pp. 26– 40. Springer, Heidelberg (2003); Extended version in Journal of Computer and System Sciences 71, 291–307 (2005) [8] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and Computation 108, 212–261 (1994) [9] Lugosi, G., Cesa-Bianchi, N.: Prediction, Learning and Games. Cambridge University Press, New York (2006) [10] Poland, J., Hutter, M.: Defensive universal learning with experts for general weight. In: Jain, S., Simon, H.U., Tomita, E. (eds.) ALT 2005. LNCS (LNAI), vol. 3734, pp. 356–370. Springer, Heidelberg (2005) [11] Rogers, C.: Arbitrage with fractional Brownian motion. Mathematical Finance 7, 95–105 (1997) [12] Shafer, G., Vovk, V.: Probability and Finance. It’s Only a Game! Wiley, New York (2001) [13] Shiryaev, A.N.: Probability. Springer, Berlin (1980) √ [14] Vovk, V.: A game-theoretic explanation of the dt effect, Working paper #5 (2003), http://www.probabilityandfinance.com
Prediction of Long-Range Dependent Time Series Data with Performance Guarantee Mikhail Dashevskiy and Zhiyuan Luo Computer Learning Research Centre Royal Holloway, University of London Egham, Surrey TW20 0EX, UK {mikhail,zhiyuan}@cs.rhul.ac.uk
Abstract. Modelling and predicting long-range dependent time series data can find important and practical applications in many areas such as telecommunications and finance. In this paper, we consider Fractional Autoregressive Integrated Moving Average (FARIMA) processes which provide a unified approach to characterising both short-range and long-range dependence. We compare two linear prediction methods for predicting observations of FARIMA processes, namely the Innovations Algorithm and Kalman Filter, from the computational complexity and prediction performance point of view. We also study the problem of Prediction with Expert Advice for FARIMA and propose a simple but effective way to improve the prediction performance. Alongside the main experts (FARIMA models) we propose to use some naive methods (such as Least-Squares Regression) in order to improve the performance of the system. Experiments on publicly available datasets show that this construction can lead to great improvements of the prediction system. We also compare our approach with a traditional method of model selection for the FARIMA model, namely Akaike Information Criterion.
1 Introduction Many real-life applications such as those related to telecommunication network traffic and stock markets have the property of long-range dependence, i.e. the sample autocovariance function is a slowly varying function ([14]). Modelling and predicting such processes can find practical and important applications in many areas such as telecommunications, signal processing, finance, econometrics and climate studies. For example, predicting network traffic demand is a fundamental objective of network resource management algorithms. A few models have been proposed for representing long-range dependent processes. One of such models, Fractional Autoregressive Integrate Moving Average (FARIMA) has proved useful in the analysis of time series with long-range dependency and has been used in various applications [5, 16, 22]. In this paper, we focus on FARIMA models for time series data and investigate various ways to improve the quality of prediction for such series. Firstly, we address the problem of model selection and discuss ways of selecting the best possible model (or to find alternative ways of using such a model without having to choose only one set of initial parameters). In [4] it was shown that O. Watanabe and T. Zeugmann (Eds.): SAGA 2009, LNCS 5792, pp. 31–45, 2009. c Springer-Verlag Berlin Heidelberg 2009
32
M. Dashevskiy and Z. Luo
the Aggregating Algorithm (a method of Prediction with Expert Advice) could be effectively used to avoid model selection and provide performance guarantee. Further improvements of the method are discussed and presented. Secondly, we consider the problem of making predictions for a FARIMA process. Two algorithms, namely the Innovations Algorithm described in [3] and Kalman Filter ([9, 10]) are investigated. Both algorithms have been used extensively but the problem of choice between them is still open. Experiments are carried out to compare these algorithms in terms of prediction performance and computational efficiency. Finally, we consider an approach for dealing with practical model selection problem by using the Aggregating Algorithm. One of the advantages of the proposed method is a performance guarantee through a theoretical bound on the total loss of the algorithm used. The rest of the paper is organised as follows. Section 2 briefly describes the FARIMA model and Subsection 2.1 is devoted to Akaike Information Criterion (AIC) which is often used as a model selection tool. Section 3 describes the Innovations Algorithm and Kalman Filter, two algorithms which can be used for prediction of linear processes. Section 4 presents a short summary of the Aggregating Algorithm, a method which can be applied to guarantee the performance of prediction for long-range dependent processes. Section 5 discusses the experimental setup and obtained results. Finally Section 6 concludes the paper.
2 Fractional Auto-Regressive Integrated Moving Average (FARIMA) Consider we have observations x1 , x2 , . . . ; xi ∈ R ∀i and we are interested in knowing a sequence of functions:fn : x1 , . . . , xn −→ xn+1 , such that its outputs are close (in a sense) to observed values in the future. This is the problem of predicting a discrete time series. A time series process {Xt , t ∈ Z} is called weakly stationary if it has a finite mean and the covariance depends only on the difference between the parameters. Definition 1. The autocovariance function of process {Xt } is γ(s, t) = E[(Xs − μ)(Xt − μ)], where μ = E Xt . Since for weakly stationary processes the covariance depends only on the difference between two parameters, we can write γ(s, t) = E[(Xs − μ)(Xt − μ)] = E[(X0 − μ)(Xt−s − μ)] = γ(t − s). In this paper we are most interested in long-range dependent time series processes. Definition 2. Time series process {Xt , t ∈ Z} is called long-range dependent if ∞ |γ(h)| = ∞. h=−∞ Another definition of long-range dependence states that if a process’s autocovariance function can be written in the form γ(h) ∼ h2d−1 l1 (h), where l1 (h) is a slowly varying function, then this process is long-range dependent. In this definition d reflects the order of long-range dependency. In this section we describe the FARIMA model, which was introduced in [6, 7]. The model has been applied to many applications where long-range dependent processes
Prediction of Long-Range Dependent Time Series Data
33
had to be predicted. These applications include prediction of network traffic demand, electricity demand and financial data. The FARIMA model is a generalization of the Auto-Regressive Moving Average (ARMA) which is itself a generalization of the Moving Average and the Autoregressive models. The ARMA model was designed to model and predict processes with short-range dependence in linear form. Previous research ([20, 22]) showed that the FARIMA model can model network traffic demand well and we assume that this model can be successful in predicting other time series processes with long-range dependence. A FARIMA process {Xt } can be defined by φ(B)Xt = θ(B)(1 − B)−d t , where B is the backshift operator defined as B k Xt = Xt−k , φ(B) = 1 + φ1 B + . . . + φp B p and θ(B) = 1 + θ1 B + . . . + θq B q are the autoregressive and moving average operators respectively; φ(B) and θ(B) have no common roots, (1−B)−d is a fractional differentiating operator defined by binomial expansion (1 − B)−d =
∞ j=0
ηj B j = η(B) , where ηj =
Γ (j + d) Γ (j + 1)Γ (d)
∞ = 0, −1, −2, . . ., and where Γ (z) = 0 tz−1 e−t dt is the Gamma funcfor d < tion (see [14, 3]) and t is a white noise sequence with finite variance. Process {Xt } is called a FARIMA(p, d, q) process. To fit a FARIMA model into some data it is essential to start with parameter estimation. The parameter estimation problem for FARIMA involves estimation of three parameters: d, φ(B) and θ(B). Of these three parameters d describes the order of longrange dependency and φ(B) and θ(B) describe the short-range dependency. Knowing parameter d we can convert a FARIMA process into an ARMA process and the other way round. There exist different techniques to estimate the parameters associated with a FARIMA process. In [4] it was shown that it is not necessary to estimate the model orders p and q to use the FARIMA model. An important characteristic of a time series is Hurst parameter H which is related to parameter d via the equation H = d + 12 . The following procedure is used to estimate parameter d and the polynomial coefficients of φ(B) and θ(B) with fixed parameters p and q: 1 2, d
1. Estimate Hurst parameter using Dispersional analysis (also known as the Aggregated Variance method), see [17, 2] for more details; parameter d can be found using d = H − 12 . 2. Convert the data from a FARIMA process into an ARMA process using differentiation with estimated parameter d. 3. Apply the Innovations Algorithm ([3]) to fit a Moving Average model into the data. 4. Estimate the coefficients of φ(B) and θ(B) ([3]). As the model orders p and q increase, the computational complexity increases dramatically. This is one of the major drawbacks of the FARIMA model. In this paper, we will only consider FARIMA models with small values of p and q. This is a standard approach to using FARIMA models since usually the most successful models have parameters p and q not greater than 2.
34
M. Dashevskiy and Z. Luo
2.1 Akaike Information Criterion When the problem of model selection arises, there is a standard technique which helps to choose a good model. Model selection among fractionally integrated auto-regressive moving average (FARIMA) models is commonly carried out with the likelihood-ratio test. Akaike Information Criterion is a widely used method for model identification ([14]) and has become a standard tool in time series modelling. The criterion calculates a value (AIC) for each suggested model and chooses the model with its lowest AIC. AIC trades off the goodness-of-fit against the number of parameters required in the model. A large number of parameters is needed to be penalised in order to avoid overfitting, i.e. a situation where the model fits the data too well and cannot predict the data well due to particularities of the available data. For the FARIMA model the criterion calculates the value: 2 ) + 2(p + q + 1), AIC = N ln(ˆ σN
where N is the size of the training set, p and q are the parameters of the FARIMA model 2 and σ ˆN is an approximation of the log-likelihood function. In this paper we used the Whittle approximation ([14]) to this function which states 2 σ ˆN =
1 2π
(N −1)/2
j=1
IN (λj ) , f (λj , θ, φ, d)
where IN (λj ) denotes the periodogram and f (λj , θ, φ, d) is the spectral density of the FARIMA process with the parameters’ vector (θ, φ, d) at the Fourier frequency λj = 2πj N . The periodogram of series yj is N 1 iλj I(λ) = y e j 2πN j=1 and the spectral density of the FARIMA model is −2d |θ(e−iλ )|2 σ2 λ , f (λ) = 2 sin 2π 2 |φ(e−iλ )|2 where φ(·) and θ(·) are the autoregressive and moving average polynomials of the FARIMA model and d is a parameter of the FARIMA model (as in FARIMA(p, d, q)) and σ is the variance of the white noise. The model with the smallest AIC over the set of considered models is chosen as the preferred model. AIC value is, however, not a direct measure of predictive power. A good study on the FARIMA model identification is given in [1]. As it will be shown in Section 5 AIC does not usually identify the best model and there exist alternative methods of using FARIMA models.
3 Prediction for ARMA Processes As it has been mentioned in Section 2, the problem of prediction for FARIMA processes can be transformed into the problem of prediction for ARMA processes by splitting a
Prediction of Long-Range Dependent Time Series Data
35
FARIMA model into its “fractional differentiated” and “ARMA” part. Fractional differentiating of the data can be carried out using the following (see [21]): Wt =
t i=0
i
(−1) Γ (1 + d) Xi . Γ (1 + i)Γ (1 + d − i)
We can transform the process back into a FARIMA process using the following: Xt =
t i=0
i
(−1) Γ (1 − d) Wi . Γ (1 + i)Γ (1 − d − i)
The converted process Wt can be fitted with an ARMA model and different linear prediction algorithms have been developed. More formally, if we have some time series data Xt and we want to use ARMA models to predict future observations of the series. We can follow the steps: 1. Transform the data into an ARMA process Wt 2. Make a prediction for the transformed process Wt 3. Calculate the prediction of the Xt process using the transformed model In the rest of this section we describe two algorithms which can be used for recursive one-step ahead predictions of ARMA processes, namely the Innovations Algorithm and Kalman Filter. The former can also be used for modelling ARMA processes and fitting the ARMA model into data. 3.1 Innovations Algorithm One of the algorithms used to fit an ARMA process into data is called the Innovations Algorithm. We first start with fitting a Moving Average (MA) model with a high order into the data. When such a model is fitted, we can find coefficients of φ and θ of an ARMA model which fits the data. From the ARMA model we can move on to FARIMA models. The algorithms provided in this section are described more fully in [3]. Suppose we have an ARMA process Wt . To make predictions on the future observations we can follow the steps: 1. Fit a MA model of a high order into the data using the Innovations Algorithm 2. Use the MA model to find coefficients of polynomials φ and θ and the white noise variance σ 2 of an ARMA model which fits the data well 3. Use the polynomials φ and θ to predict the future value of Wt Let us now consider each step separately. Consider the process generated by Wt − φ1 Wt−1 − . . . − φp Wt−p = Zt + θ1 Zt−1 + . . . + θq Zt−q , Zt ∼ WN(0, σ 2 ). (1)
36
M. Dashevskiy and Z. Luo
If we assume that data series Xt is generated by the ARM A(p, q) model with known parameters p and q and use properties of ARMA models, we can represent the data series Wt as ∞ Wt = ψj Zt−j , j=0
where Zj is a white noise sequence and coefficients ψj satisfy ψ0 = 1, . min(j,p) ψj = θj + i=1 φi ψj−i To fit a model ∞ j=0 ψj Zt−j into the data Wt with zero mean we can use Algorithm 1: Algorithm 1. Innovations Algorithm for M A(r) model Require: Parameter r k(i, j) = E(Wi Wj ), 1 ≤ i, j ≤ r + 1 v0 = k(1, 1) for n = 1, 2, . . . , r do for l = 0, 1, . . . , n − 1 do
θn,n−l = vl−1 k(n + 1, l + 1) − l−1 j=0 θl,l−j θn,n−j vj end for 2 vn = k(n + 1, n + 1) − n−1 j=0 θn,n−j vj end for
By taking the order of the Moving Average model r high enough (in our experiments we used r = max(p, q) + p − 1 which proved to be giving a good fit) we can estimate the coefficients of polynomial φ by solving ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ θr,q+1 θr,q φ1 θr,q−1 · · · θr,q+1−p ⎜ θr,q+2 ⎟ ⎜ θr,q+1 ⎜ ⎟ θr,q · · · θr,q+2−p ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ φ2 ⎟ ⎜ .. ⎟ = ⎜ ⎟ × ⎜ .. ⎟ . .. .. .. ⎝ . ⎠ ⎝ ⎠ ⎝ . ⎠ . . . θr,q+p θr,q+p−1 θr,q+p−2 · · · θr,q φp Coefficients θj of the polynomial of the ARMA process represented by equation (1) can be estimated using min(j,p)
θj = θr,j −
φi θr,j−i , j = 1, 2, . . . , q
i=1
and the white noise variance in equation (1) is estimated using σ 2 = vr , where vr are calculated by the algorithm. Once polynomials φ and θ and variance σ 2 of equation (1) are known, Algorithm 2 ˆ n+1 on Wn+1 . can be used to make a one-step ahead prediction W
Prediction of Long-Range Dependent Time Series Data
37
Algorithm 2. Recursive one-step ahead prediction of an ARM A(p, q) process Require: Parameters N (the total size of “training set + test set”), p and q (the orders of the model) m = max(p, q) v0 = E(W12 ) ˆ1 = 0 W for n = 1, 2, . . . , N do γ(n) = E(W1 Wn ) for 1 ≤ i, j, ≤ n + 1 do k(i, j) = σ −2 γ(i − j),if 1 ≤ i, j ≤ m p −2 =σ q γ(i − j) − r=1 φr γ(r − |i − j|) , if min(i, j) ≤ m < max(i, j) ≤ 2m = r=0 θr θr+|i−j| , if min(i, j) > m = 0 otherwise end for for l = 0, 1, . . . , n − 1 do
θn,n−l = vl−1 k(n + 1, l + 1) − l−1 j=0 θl,l−j θn,n−j vj end for 2 vn = k(n + 1, n + 1) − n−1 j=0 θn,n−j vj if n < m then ˆ n+1−j ) ˆ n+1 = n θn,j (Wn+1−j − W W j=1 else ˆ n+1−j ) ˆ n+1 = φ1 Wn + . . . + φp Wn+1−p + n θn,j (Wn+1−j − W W j=1 end if end for
3.2 Kalman Filter Kalman Filter is a technique for smoothing, filtering and prediction for linear time series data. In this paper we will use Kalman Filter only to predict future observations of a time series for a given model. To apply Kalman Filter the model needs to be represented in the state-space form, which can be described by the discrete-time equations: yt = Zt αt + Gt t , (2) αt+1 = Tt αt + Ht t where yt ∈ R is the observation sequence, αt is the state vector of size s × 1 for all time t, the system matrices Zt , Gt , Tt and Ht are non-random matrices of sizes 1 × s, 1 × l, s × s and s × l respectively, t ∼ (0, σ 2 I) is the errors vector of size l × 1, α1 ∼ (a1 , σ 2 P1 ) and t and α1 are mutually uncorrelated, a1 and P1 are the parameters of prior distribution of the state vector and σ 2 is the variance of the errors. The state-space model and Kalman Filter were first introduced by Kalman and Bucy (see [9, 10]). To represent the ARM A(p, q) model as a linear state-space system we follow [8] and take s = max(p, q + 1), l = 1 and
38
M. Dashevskiy and Z. Luo
⎛
φ1 φ2 .. .
⎜ ⎜ ⎜ Tt = T = ⎜ ⎜ ⎝ φs−1 φs
⎞ 0 0⎟ ⎟ .. ⎟ , .⎟ ⎟ 0 0 ··· 1⎠ 0 0 ··· 0 1 0 ··· 0 1 ··· .. .. . . . . .
where φi = 0 for i = p + 1, . . . , s and Zt = Z = (1, 0, . . . , 0), Gt = G = 0, Ht = H = (1, θ1 , . . . , θq , 0, . . . , 0) . In general, observations yt are converted into innovations vt . Let us assign α1 = (y0 , 0, . . . , 0)T , where y0 is the last observation from the training set and P1 = I, the identity matrix. At each step t = 1, . . . , n the following five steps are performed to update the model: 1. 2. 3. 4. 5.
vt = yt − Zt αt Ft = Zt Pt Zt + Gt Gt Kt = (Tt Pt Zt + Ht Gt )Ft−1 αt+1 = Tt αt + Kt vt Pt+1 = Tt Pt (Tt − Kt Zt ) + Ht (Ht − Kt Gt )
Intuitively αt is a state estimate, Ft is innovations’ covariance, Kt is the optimal Kalman gain and Pt is a covariance’s estimate. In our particular case we can formulate the aforementioned sequence of steps in the form of an algorithm (see Algorithm 3). Since the matrices Z, G, T and H are constant in time and G = 0 : Algorithm 3. Kalman Filter for an ARMA process P1 = I T, Z and H are as given above for t = 1, 2, . . . do vt = yt − Zαt {These 5 steps update the model} Ft = ZPt Z Kt = (T Pt Z )Ft−1 αt+1 = T αt + Kt vt Pt+1 = T Pt (T − Kt Z) + HH yˆt+1 = Zαt+1 {Give a prediction} end for
Algorithm 3 outputs predictions for future values of yt . It is interesting to note that the number of steps required to calculate predictions at each round of the algorithm does not grow with the number of observations being processed, as opposed to the number of steps required by Algorithm 2. Experiments will be carried out to compare empirically the computational complexity of the two algorithms on publicly available datasets.
4 Aggregating Algorithm To predict future observations of a time series, typically one has to select a model. If it is possible to make an assumption that FARIMA models, which can efficiently represent
Prediction of Long-Range Dependent Time Series Data
39
time series data which exhibit both long-range and short-range dependence, can fit the data well, it is essential to choose three parameters p, q and d. It is still an open question on how to choose the best set of parameters given the data. In [4] it was proposed to use several models with different sets of parameters and to mix their outcomes according to previous respective performances in order to avoid making the choice between different sets of parameters. The aim here is not to find the best model predicting time series data, but to find an efficient way to use several models so that we can have a guaranteed high performance. The idea of using several prediction algorithms in order to achieve high performance has been studied extensively and is called the problem of Prediction with Expert Advice. The problem can be described in the form of a game. In the game, there are experts which give individual predictions of the possible outcome and it is unknown which expert can perform well in advance. At each round of the game, experts give their predictions based on the past observations and we can use the experts’ predictions in order to make our prediction. Once our prediction is made, the outcome of the event is known. As the result, the losses suffered by each expert and our algorithm are calculated. Our aim is to find a strategy which can guarantee that the total loss (i.e. the cumulative loss after all rounds of the game) is not much worse than the total loss of the best expert during the game. It is useful to describe the game of prediction with expert advice more formally. Consider a game between three players, Nature, Experts, and Learner. Let Σ be a set called signal space, Ω be a set called sample space, Γ be a set called the decision space, and Θ be a measurable space called the parameter space. A signal is an element of the signal space, an outcome is an element of the sample space and a decision is an element of the decision space. The function that measures the performance of Experts and Learner is called the loss function, λ : Ω × Γ → [0, ∞], and is also part of the game. In the perfect-information game at each round T = 1, 2, . . .: 1. Nature outputs a signal xT ∈ Σ. 2. Experts make measurable predictions ξT : Θ → Γ ; where ξT (θ) is the prediction corresponding to the parameter θ ∈ Θ. 3. Learner makes his own prediction γT ∈ Γ. 4. Nature chooses an outcome ωT ∈ Ω. 5. Experts’ loss calculation LT (θ) = Tt=1 λ(ωt , ξt (θ)). T 6. Learner’s loss calculation LT = t=1 λ(ωt , γt ). The aim of Learner is to minimize the regret term, i.e. the difference between the total loss of the algorithm and the total loss of the best expert. Many strategies have been developed for Learner to make his prediction γT such as following the perturbed leader, gradient descent and weighted majority algorithm. In this paper we consider the case Ω ⊂ R, Γ ⊂ R and apply one of the methods solving the problem, namely the Aggregating Algorithm (AA). This algorithm’s main advantage consists in a theorem (see [18]) stating that its performance cannot be improved, i.e. it has guaranteed optimal performance. When the experts are fixed the only step of the game of prediction with expert advice which remains unclear is step 3. At each step T the Aggregating Algorithm re-computes the weights of the experts (represented by their probability distribution) and mixes all
40
M. Dashevskiy and Z. Luo
experts’ predictions according to this distribution. Thus the AA gets a mixture of the experts’ predictions (a generalized prediction), then it finds the best (in some sense) prediction from Θ. There are two parameters in the AA: the learning rate is η > 0 (β is defined to be e−η ) and P0 is a probability distribution on set Θ of experts. Intuitively P0 is the prior distribution, which specifies the initial weights assigned to the experts. More formally, at each step T the algorithm performs the following actions to make the prediction at step 3 of the above game: 3.1 Updating experts’ weights according to the previous losses: PT −1 (dθ) = β λ(ωT −1 , ξT −1 (θ)) PT −2 (dθ), θ ∈ Θ. 3.2 Predictions’ mixing according to their weights: β λ(ω, ξT (θ)) PT∗−1 (dθ), gT (ω) = logβ
(3)
Θ
where PT∗−1 (dθ) is a normalised probability distribution, i. e. PT∗−1 (dθ) = PT −1 (dθ) PT −1 (Θ) and if PT −1 (Θ) = 0, the AA is allowed to choose any prediction. On this step we get gT (ω) which is a function Ω → R. 3.3 Looking for an appropriate prediction from Γ : γT (gT ). An important equation which describes how to apply the AA to our problem was proved in [18] (Lemma 2). It states that in the case of the square-loss function, i. e. λ(ω, γ) = 2 (ω − γ) , η ≤ 2Y1 2 (where Y is a bound for |yt |) γT =
gT (−Y ) − gT (Y ) 4Y
(4)
is a prediction from Θ. Formula (4) gives us the rule for making a prediction at step T . In this paper we use FARIMA models as experts and ξT (θ) corresponds to the prediction at step T given by a specific FARIMA model denoted by θ. As we use only a finite number of experts the integration in the formula (3) becomes the operation of summation. Preprocessing of the datasets can ensure |Y | = 1. In this paper we consider square loss which is a standard measure of the performance of prediction algorithms. Since the number of experts is finite in our case, we can use the fact ([19]) that for each round T of the game and each expert θ the accumulative loss of the AA LT (AA) is bounded by as LT (AA) ≤ LT (θ) + ln(n) where n is the total number of experts.
5 Experiments and Results In this section we use FARIMA models with different parameters and Least-Squares Regression to make predictions on a time series and apply the Aggregating Algorithm in various settings to several datasets. Our aim is to find an efficient and fast algorithm with high performance and compare it with existing methods. All the algorithms were
Prediction of Long-Range Dependent Time Series Data
41
implemented in Matlab and the experiments were run on a laptop with Intel Core Duo 2.40GHz, 4.0 GB RAM. We ran our experiments on publicly available datasets related to network traffic. Dataset Z contains an hour’s worth of all wide-area traffic between the Lawrence Berkeley Laboratory and the rest of the world and was made publicly available by Vern Paxson (see [15]). Dataset M was collected at Bellcore in August of 1989 and contains one “normal” hour’s traffic demand (see [12]). We transformed both the datasets in such a way that each observation is the number of packets in the corresponding network during a fixed time interval, one second. The datasets Z and M are then each divided into two sub-datasets of equal size Z1 (d = 0.34) and Z2 (d = 0.24), and M1 (d = 0.32) and M2 (d = 0.38) respectively. Dataset DEC-PKT (d = 0.33) contains an hour’s worth of all wide-area traffic between Digital Equipment Corporation and the rest of the world. Dataset LBL-PKT-4 (d = 0.29) consists of observations of another hour of traffic between the Lawrence Berkeley Laboratory and the rest of the world. On each of these datasets the following operations were performed: subtraction of the mean value and division by the maximum absolute value. Table 1. Running times (in seconds) for the two algorithms Alg. stats M1 M2 Z1 Z2 DEC-PKT LBL-PKT-4 mean 1.37 1.37 1.38 1.37 1.39 1.36 KF std 0.05 0.03 0.03 0.03 0.04 0.02 mean 1156.68 1181.57 1136.84 1137.30 1140.42 1136.08 IA std 19.81 37.53 8.69 1.71 28.71 4.35
Table 1 shows measured running times for all 15 FARIMA models (with d estimated using Dispersional Analysis and 0 ≤ p, q ≤ 3, p2 + q 2 = 0) based on Kalman Filter (KF) and the Innovations Algorithm (IA) as the underlying prediction algorithm for the ARMA model. The times are given in seconds. We repeated our experiments on 5 subdatasets of 300 examples with 200 training examples and 100 test examples from all original datasets and then averaged the time over the sub-datasets. The performance of these algorithms was quite similar for all experiments. The mean value of the relative difference between the error achieved by the best model (according to the cumulative errors) of the Innovations Algorithm and the best model of Kalman Filter is 3.92%, the median is 0.20% and the standard deviation is 0.10. We also applied the runs test (see [11]) to check that Kalman Filter and the Innovations Algorithm are not performancecomparable (i.e. that we cannot predict which algorithm will have lower error rate for its respective best model). To apply the test we computed a series of 1s and 0s, where 1 corresponded to the case where the best model output by Kalman Filter performed better and 0 to the case where the best model output by the Innovations Algorithm performed better. We then checked the hypothesis that the series is random. With 99% confidence neither one of the two algorithms outperforms the other one. Therefore, since it works much faster, we will use Kalman Filter in our further experiments.
42
M. Dashevskiy and Z. Luo Table 2. Cumulative square errors for different algorithms and datasets Algorithm FARIMA(0, 1) FARIMA(0, 2) FARIMA(0, 3) FARIMA(1, 0) FARIMA(1, 1) FARIMA(1, 2) FARIMA(1, 3) FARIMA(2, 0) FARIMA(2, 1) FARIMA(2, 2) FARIMA(2, 3) FARIMA(3, 0) FARIMA(3, 1) FARIMA(3, 2) FARIMA(3, 3) LSR AA-15 AA-16 FARIMA-AIC AA-AIC-5 Th. bound:AA-5 Th. bound:AA-15 Th. bound:AA-16
M1 1.50 1.21 1.21 1.16 1.20 1.21 0.94 1.21 36.59 1.60 22.47 1.21 97.85 100.39 101.84 2.16 1.14 1.06 1.16 1.24 2.76 3.65 3.72
M2 26.41 26.32 26.40 26.46 26.18 26.51 44.50 26.30 55.12 46.97 98.80 26.30 35.45 33.07 33.52 3.51 22.74 4.63 26.46 25.57 27.79 28.89 6.28
Z1 6.28 6.23 6.12 6.21 6.20 7.18 4.97 6.17 4.95 4.76 54.65 6.05 91.51 105.17 104.73 4.56 5.46 5.05 6.21 6.16 7.65 7.47 7.33
Z2 DEC-PKT LBL-PKT-4 0.13 0.45 1.51 0.13 0.45 1.53 0.15 0.47 1.52 0.14 0.45 1.50 0.14 0.44 1.53 4.24 1.39 8.38 0.16 0.44 1.51 0.14 0.45 1.52 0.28 1.54 1.62 21.60 68.72 59.13 0.17 88.66 1.51 0.14 0.46 1.49 0.21 92.74 7.90 0.19 90.52 53.11 0.20 99.94 29.57 0.25 1.62 1.46 0.13 0.41 1.47 0.11 0.30 1.37 0.16 0.45 1.50 0.15 0.43 1.56 1.75 2.05 3.10 2.84 3.15 4.20 2.91 3.22 4.23
Table 2 summarises the cumulative square errors for different algorithms. Firstly, the FARIMA model with different sets of parameters (p, q) and Least-Squares Regression (LSR) are considered. Secondly, the results for the Aggregating Algorithm with 15 FARIMA models as its experts (AA-15) and the Aggregating Algorithm with 15 FARIMA models and the LSR algorithm as its experts (AA-16) are given. Then the FARIMA model selected by Akaike Information Criterion (FARIMA-AIC) and the Aggregating Algorithm with 5 FARIMA models with lowest AIC values as its experts (AA-AIC-5) are considered. In addition, Table 2 gives the theoretical bounds on the cumulative square-loss for AA-AIC-5 (AA-5), AA-15 and AA-16. For each dataset we used only the first 300 examples where 200 examples were used for training and the remaining 100 examples as testing examples. From Table 2 it can be seen that the best FARIMA model for each dataset is different (i.e. we can’t choose one set (p, q) such that the FARIMA model with this set of parameters is better than the other models on all datasets) even though the data is of similar nature. In case of dataset M2 all FARIMA models perform much worse than a simple regression method, Least-Squares Regression. The Aggregating Algorithm is a method which can avoid model selection and provide a guaranteed output. As it can be seen the theoretical bound for the AA holds and that adding a simple prediction algorithm as an expert improves the performance of the prediction system in most cases. It can also be seen that AA-15 and AA-16 perform not much worse than the best model (expert) and the
Prediction of Long-Range Dependent Time Series Data
43
difference between losses of individual experts and AA
differences between the total losses of the best experts and the worst experts are large. For example, the total losses of the best model (FARIMA(2,2)) for dataset Z1 is 4.76 and the worst model (FARIMA(3,2)) suffered the total loss of 105.17. However, AA-15 achieved 5.46. A traditional way of dealing with the problem of model selection is to use Akaike Information Criterion (AIC). As it can be seen from Table 2 AIC failed to identify the best FARIMA model during our experiments. A natural way of combining the AA and AIC is to use AIC to choose “promising” experts, i.e. experts that according to AIC should perform well. In our experiments we used AIC to determine the five most promising experts out of 15 but in most cases (5 out of 6) the results were worse than those obtained by the AA based on all experts. A conclusion which can be drawn from these results is that using AIC might not be able to select the best possible model. In [4] it was shown that methods of Prediction with Expert Advice can be successfully used to mix FARIMA models in order to achieve a guaranteed performance. We have shown that adding a naive method can improve the performance even further and that the constructed prediction system outperforms traditional systems which employ AIC for model selection. Figure 1 shows the differences between the cumulative errors of 12 experts (the rest of the experts performed much worse and therefore their results are not included in the
8
theoretical bound on the values difference between losses of individual experts and AA
6
4
2
0
−2
−4 0
10
20
30
40
50 examples
60
70
80
90
100
Fig. 1. The differences between the cumulative errors of 12 best experts and the AA based on these experts (dataset Z1). The thick line shows the theoretical bound for this value.
44
M. Dashevskiy and Z. Luo
graph) and the AA based on all (16) experts for the dataset Z1. The AA guarantees that this difference is bounded by the logarithm of the number of experts (when the number of experts is finite). The thick line shows the theoretical bound for this value. As it can be seen from Figure 1 the bound holds.
6 Conclusions In this paper, we studied how to model and predict long-range dependent time series with FARIMA models and investigated various ways to improve prediction performance. We have compared two widely used linear prediction algorithms, namely the Innovations Algorithm and Kalman Filter, in terms of prediction performance and computational complexity. Empirical results have shown that Kalman Filter can achieve good performance while maintaining high efficiency due to its lightweight mechanism for model update. We have shown that by using the Aggregating Algorithm we can guarantee the quality of predictions and outperform the model selected by Akaike Information Criterion. The experiments have also shown that by adding a naive method such as LeastSquares Regression we can improve the performance of the prediction system and deal with the cases where the FARIMA model does not perform well. The problems considered in this paper give several directions for further research. Since we ran the experiments on network traffic demand time series datasets we need to validate the results for other types of applications. We also plan to find a way of improving the overall performance of the prediction system by assigning different weights to the experts before running the algorithm. This can be done, for example, by using different information criteria, such as AIC. Another approach is to look at how model selection for other models, such as Generalized Autoregressive Conditional Heteroscedasticity (GARCH) model can be avoided by aggregating several models with different parameters. It may turn out that this approach can achieve better performance than the methods used currently.
Acknowledgements This work has been supported by the EPSRC through grant EP/E000053/1 “Machine Learning for Resource Management in Next-Generation Optical Networks” and Royal Society International Joint Project “Explosives Trace Detection with an Odour Capture Hybrid Sensor System”.
References [1] Bisaglia, L.: Model selection for long-memory models. Quaderni di Statistica 4 (2002) [2] Blok, H.J.: On The Nature Of The Stock Market: Simulations And Experiments. PhD Thesis, University of British Columbia, Canada (2000) [3] Brockwell, P.J., Davis, R.A.: Time Series: Theory and Methods. Springer, Heidelberg (1991)
Prediction of Long-Range Dependent Time Series Data
45
[4] Dashevskiy, M., Luo, Z.: Guaranteed Network Traffic Demand Prediction Using FARIMA Models. In: Fyfe, C., Kim, D., Lee, S.-Y., Yin, H. (eds.) IDEAL 2008. LNCS, vol. 5326, pp. 274–281. Springer, Heidelberg (2008) [5] Dethe, C.G., Wakde, D.G.: On the prediction of packet process in network traffic using FARIMA time-series model. J. of Indian Inst. of Science 84, 31–39 (2004) [6] Granger, C.W.J., Joyeux, R.: An introduction to long-memory time series models and fractional differencing. J. of Time Series Analysis 1, 15–29 (1980) [7] Hosking, J.R.M.: Fractional differencing. Biometrica 68, 165–176 (1981) [8] de Jong, P., Penzer, J.: The ARMA model in state space form. Statistics & Probability Letters 70(1), 119–125 (2004) [9] Kalman, R.E.: A new approach to linear filtering and prediction problems. Journal of Basic Engineering 82, 34–45 (1960) [10] Kalman, R.E., Bucy, R.S.: New results in linear filtering and prediction theory. Journal of Basic Engineering 83, 95–108 (1961) [11] Laurencelle, L., Dupuis, F.-A.: Statistical Tables, Explained and Applied: Explained and Applied. World Scientific, Singapore (2002) [12] Leland, W.E., Taqqu, M.S., Willinger, W., Wilson, D.V.: On the self-similar nature of Ethernet traffic. IEEE/ACM Trans. on Networking. 2, 1–15 (1994) [13] de Lima, A.B., de Amazonas, J.R.A.: State-space Modeling of Long-Range Dependent Teletraffic. In: Mason, L.G., Drwiega, T., Yan, J. (eds.) ITC 2007. LNCS, vol. 4516, pp. 260–271. Springer, Heidelberg (2007) [14] Palmo, W.: Long-Memory Time Series. Theory and Methods. Wiley Series in Probability and Statistics (2007) [15] Paxson, V.: Fast Approximation of Self-Similar Network Traffic. Technical report LBL36750/UC-405 (1995) [16] Shu, Y., Jin, Z., Zhang, L., Wang, L., Yang, O.W.W.: Traffic prediction using FARIMA models. In: IEEE International Conf. on Communication, vol. 2, pp. 891–895 (1999) [17] Taqqu, M.S., Teverovsky, V., Willinger, W.: Estimators for long-range dependence: An empirical study. Fractals 3, 785–788 (1995) [18] Vovk, V.: Competitive On-line Statistics. Int. Stat. Review 69, 213–248 (2001) [19] Vovk, V.: Prediction with expert advice for the Brier game (2008), http://arxiv.org/abs/0710.0485 [20] Willinger, W., Paxson, V., Riedi, R.H., Taqqu, M.S.: Long-range dependence and data network traffic. Theory And Applications Of Long-Range Dependence, 373–407 (2003) [21] Xue, F.: Modeling and Predicting Long-range Dependent Traffic with FARIMA Processes. In: Proc. of 1999 International Symposium on Communication (1999) [22] Xue, F., Liu, J., Zhang, L., Yang, O.W.W.: Traffic Modelling Based on FARIMA Models. In: Proc. IEEE Canadian Conference on Electrical and Computer Eng. (1999)
Bipartite Graph Representation of Multiple Decision Table Classifiers Kazuya Haraguchi1, Seok-Hee Hong2 , and Hiroshi Nagamochi3 1
Faculty of Science and Engineering, Ishinomaki Senshu University, Japan
[email protected] 2 School of Information Technologies, University of Sydney, Australia
[email protected] 3 Department of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Japan
[email protected]
Abstract. In this paper, we consider the two-class classification problem, a significant issue in machine learning. From a given set of positive and negative samples, the problem asks to construct a classifier that predicts the classes of future samples with high accuracy. For this problem, we have studied a new visual classifier in our previous works, which is constructed as follows: We first create several decision tables and extract a bipartite graph structure (called an SE-graph) between the given set of samples and the set of created decision tables. We then draw the bipartite graph as a two-layered drawing by using an edge crossing minimization technique, and the resulting drawing acts as a visual classifier. We first describe our background and philosophy on such a visual classifier, and then consider improving its classification accuracy. We demonstrate the effectiveness of our methodology by computational studies using benchmark data sets, where the new classifier outperforms our older version, and is competitive even with such standard classifiers as C4.5 or LibSVM.
1
Introduction
1.1
Classification Problem
Our goal is to establish the theory of a new visual classifier to a mathematical learning problem called classification, which has been a significant research issue from classical statistics to modern research fields on learning theory and data analysis [1, 2]. For a positive integer m, let [m] = {1, 2, . . . , m}. A sample s is an n dimensional vector with entries on n domains (or n attributes) D1 , . . . , Dn , and s belongs to either positive (+) class or negative (−) class. We assume that Dj (∀j ∈ [n]) is a finite set of at least two elements. In classification, we are given a training set S = S + ∪ S − of samples, where S + and S − are the sets of samples s ∈ S belonging to the positive and negative classes, respectively. Table 1
This work is partially supported by Grant-in-Aid for Young Scientists (Start-up, 20800045) from Japan Society for the Promotion of Science (JSPS).
O. Watanabe and T. Zeugmann (Eds.): SAGA 2009, LNCS 5792, pp. 46–60, 2009. c Springer-Verlag Berlin Heidelberg 2009
Bipartite Graph Representation of Multiple Decision Table Classifiers
47
Table 1. A training set S = S + ∪ S − with S + = {s1 , s2 , s3 , s4 } and S − = {s5 , s6 , s7 } over three attributes with D1 = {yes, no}, D2 = {high, med}, and D3 = {high, med, low}
S + (malignant) s1 s2 s3 s4 − S (benign) s5 s6 s7
Att. 1 (headache) yes yes yes no no yes no
Att. 2 (temperature) high med high high med high med
Att. 3 (blood pressure) high med high med high med low
shows an example of training set S, where a sample corresponds to a patient for some disease, an attribute represents the result of an examination, and the class represents diagnosis. A classifier is a function from the sample space S = D1 × · · · × Dn to the class {+, −}. The aim of classification is to construct a classifier that predicts the class of a future sample s ∈ S with high accuracy, where s is possibly unseen, i.e., s ∈ S. 1.2
Background on Visual Classifier
Visualization plays an important role as an effective analysis tool for huge and complex data sets in many application domains such as financial market, computer networks, biology and sociology. Graph drawing has been extensively studied over the last twenty years due to its popular application for visualization in VLSI layout, computer networks, software engineering, social networks and bioinformatics [3]. To draw graphs automatically and nicely, we need to give a mathematical definition of aesthetic criteria (e.g., the number of edge crossings, symmetry, or bends) for 2D and 3D drawings. In our companion paper [4, 5], we proposed a new mathematical measurement of occlusion for 2.5D drawing [6] of pairwise trees (where one dimension of the 2.5 dimensions is used for essentially different purpose from the others) as the sum of the number of edge crossings of the bipartite graphs in the projected images. We then presented algorithms to the problem of finding a 2.5D drawing that minimizes the occlusion. As a case study, we observed the pairwise trees which represent samples and entries, supposing that the minimum occlusion drawing supports visual analysis of classification or clustering. Given a well-visualized image, one may expect that a hidden mathematical structure has been revealed. We are also inspired by the fact that algorithms for reducing edge crossings in graph drawings are recently used in some data analysis such as the rank aggregation problem [7, 8]. We hypothesize that good visualization (e.g., visual objects with low visual complexity) itself can discover
48
K. Haraguchi, S.-H. Hong, and H. Nagamochi
essential or hidden structure of data without relying on data analysis techniques, which can lead to a novel learning technique. Based on our hypothesis, we designed a prototype visual classifier using visualization in our preliminary research [9, 10]. The classifier was constructed from a training set S of samples and a family P = {P1 , . . . , PN } of N partitions of the sample space S (i.e., each Pj ∈ P is such a set Pj = {Q1 , . . . , QK } of K K subspaces of S that satisfies k=1 Qk = S and Qk ∩ Qk = ∅ for k = k ) as follows: We first represented the relationship between the samples of the training set S and the subspaces of P as a bipartite graph. We visualized the bipartite graph using a two-layered drawing [11], as shown in Fig. 1(a). Next, starting with the family P = {D1 , . . . , Dn } of attribute domains, we tried to reduce the number of edge crossings by changing the drawing (i.e., the ordering of nodes on each side) based on local search, and to change the structure of the bipartite graph by replacing two partitions into their product as a new partition until a termination criteria is satisfied (see Fig. 1(b)). The resulting drawing was used as a visual classifier in the following way. Given a future sample s ∈ S, we placed s on the sample side of the drawing so that the position of s was the average of the positions of adjacent nodes in the entry side, and then judged s as positive (negative) if the nearest neighbor of s was a positive (negative) sample.
Fig. 1. The prototype visual classifier [9, 10] for MONKS-1 data set
Bipartite Graph Representation of Multiple Decision Table Classifiers
49
Table 2. Decision tables T1 = (A1 , 1 ), T2 = (A2 , 2 ) and T3 = (A3 , 3 ) with attribute sets A1 = {1, 2}, A2 = {1, 3} and A3 = {3} T1 = (A1 , 1 ) v ∈ D1 × D2 1 (v) yes, high + yes, med + no, high + no, med −
T2 = (A2 , 2 ) v ∈ D1 × D3 2 (v) yes, high + yes, med + yes, low + no, high − no, med + no, low −
T3 = (A3 , 3 ) v ∈ D3 3 (v) high + med + low −
Although we demonstrated the effectiveness of the preliminary version of our visual classifier by empirical studies, the mechanism or reason why such a classifier could successfully work was not analyzed. To clarify the reason, we recently investigated a relationship between the above bipartite graph representation of data and the majority vote classifier (MV ) with multiple decision tables [12]. Among the classifiers developed so far, decision table is simple and comprehensible. Table 2 shows three decision tables for the training set in Table 1. Formally, a decision table T = (A, ) is defined by a subset A ⊆ [n] of n attributes and a label function . The label function assigns either (+) or (−) to each entry of A which corresponds to a row in the table. For a future sample, a decision table estimates its class as the label of the matched entry. For example, (no, high, high) is classified as (+) by T1 , (−) by T2 and (+) by T3 . Assume that a set T = {T1 , . . . , TN } of N decision tables is given. We note that one can use T as a single classifier by applying such ensemble technique as majority vote (MV). The MV classifies a future sample as the majority class among the N outputs given by the N decision tables in T . For example, MV with T = {T1 , T2 , T3 } in Table 2 classifies (no, high, high) as (+). In [12], we considered a generalization of MV (GMV ), and found that GMV classifier can be represented as a drawing of bipartite graph called sample-entry graph (SE-graph), where the number of edge crossings is to be minimized. We proposed a scheme to decide the parameters of GMV for “fixed” decision table set T , and demonstrated its effectiveness by experiments using randomly generated T ; However, it was left open to develop an algorithm to obtain a “good” decision table set. 1.3
New Contribution
In this paper, following our recent work [12], we consider GMV with a given set T = {T1 , . . . , TN } of decision tables, and represent it as an SE-graph. In an SE-graph, one node set corresponds to the samples in S, the other set does to the rows (i.e., entries) of the decision tables in T , and a sample s and an entry v are joined by an edge if and only if s matches v. See Fig. 2(a) for an example of SE-graph drawn as two-layered drawing. The layout of SE-graph
50
K. Haraguchi, S.-H. Hong, and H. Nagamochi
Fig. 2. SE-graph for GLASS data set
is defined by permutations on the samples and the entries. We perform edge crossing minimization of a drawing of SE-graph, and we divide the sample nodes into positive and negative sides by a suitably chosen threshold θ, based on which a future sample will be judged as positive or negative (see the θ dividing the sample nodes in Fig. 2(b)). These are the major difference from our preliminary classifier [9, 10]. The new contribution of this paper is summarized as follows. (i) We assert that edge crossing minimization of SE-graph “stretches” the training set to a total order through observation on the layout. This opens a new research direction for future research of visual classifier. (ii) To address the open problem in [12], we present a new simple procedure that collects a subset of “good” decision tables from a given larger set. Also, we decide the parameters of GMV in more detailed way. (iii) Performance of classifier is usually evaluated by error rate on test set of samples. Our new version of visual classifier (SEC-II) outperforms the
Bipartite Graph Representation of Multiple Decision Table Classifiers
51
previous version (SEC-I) [12], where the improvement is 25% on average. Surprisingly, it is competitive with decision tree constructed by C4.5 [13] and support vector machine (SVM ) constructed by LibSVM package [14] of WEKA [15], and even outperforms them for some benchmark instances. The paper is organized as follows. In Sect. 2, we prepare terminologies mainly on decision table and SE-graph. In Sect. 3, we discuss the meaning of edge crossing minimization on SE-graph (for (i)) and describe the construction algorithm of our classifier, along with the idea of its improvement via crossing minimization (for (ii)). Then we present computational results in Sect. 4 (for (iii)), followed by concluding remarks in Sect. 5.
2 2.1
Preliminaries Decision Table
Let A = {j1 , . . . , jq } ⊆ [n] denote a subset of n attributes. Formally, we call an element v ∈ Dj1 × · · · × Djq of the Cartesian product of the domains an entry of A. For s ∈ S, let s|A = (sj1 , . . . , sjq ) denote the projection of s onto the attribute set A. For a decision table T = (A, ), we define the label function by DTM (Decision Table Majority), which was introduced in [16]. For an entry v of A, let − m+ A (v) (resp., mA (v)) denote the number of positive (resp., negative) samples matching v; + + + m+ A (v) = |{s ∈ S | s |A = v}|,
− − − m− A (v) = |{s ∈ S | s |A = v}|.
Then the value (v) ∈ {+, −} is defined tive/negative samples matching v; ⎧ ⎨+ (v) = − ⎩ majority class in S
(1)
according to distribution of posi− if m+ A (v) > mA (v), + if mA (v) < m− A (v), otherwise.
(2)
The empirical error rate of a classifier is its error rate on the training set. Then the empirical error rate of T = (A, ) is computed by: 1 |S|
v is an entry of A
− min{m+ A (v), mA (v)}.
(3)
Recall a given set T = {T1 , . . . , TN } of N decision tables. For any decision table Tk = (Ak , k ) (k ∈ [N ]), let Ak = {j1 , . . . , jq } ⊆ [n]. We denote the set of all entries of Ak by Dk = Dj1 × · · · × Djq , and denote the union of all Dk ’s by D = D1 ∪ · · · ∪ DN .
52
K. Haraguchi, S.-H. Hong, and H. Nagamochi
We define the generalized majority vote (GMV) with T [12]. GMV is defined by a tuple (c1 , . . . , cN , θ) of N functions ck : Dk → [−1, 1] (k ∈ [N ]) and a threshold θ ∈ [−1, 1]. Function ck assigns a real value in [−1, 1] to each entry of decision table Tk , and is called a class level . Given a future sample s ∈ S, a GMV classifier first computes the barycenter b(s ) of sample s by N 1 ck (s |Ak ), b(s ) = N
(4)
k=1
and estimates the class of s as the sign of b(s ). One can readily see that MV is a special case of GMV as follows; for each decision table Tk ∈ T and its entry v ∈ Dk , set ck (v) = 1 (resp., −1) if k (v) = + (resp., −), and θ = 0. 2.2
Sample-Entry Graph (SE-Graph)
We define sample-entry graph (SE-graph) [12], which is a bipartite graph G = (S, D, E) defined as follows: – Each sample s ∈ S is called a sample node, and each entry v ∈ D = D1 ∪ · · · ∪ DN is called an entry node. – For each k ∈ [N ], a sample node s and an entry node v ∈ Dk are joined by an edge (s, v) ∈ S × Dk if and only if s matches the entry v, i.e., s|Ak = v. Thus the edge set is given by E = E1 ∪E2 ∪· · ·∪EN such that Ek = {(s, v) ∈ S × Dk | s|Ak = v}. For each k ∈ [N ], let Gk = (S, Dk , Ek ) denote the subgraph of G induced by S and Dk . We visualize the SE-graph in 3D space shown in Fig. 3: We assume a cylinder standing on the x, y-plane. Suppose that its base has radius of the unit length and that the height is sufficiently large. Then we take N + 1 lines orthogonal to the x, y-plane so that one of them penetrates the center of the base and the others are along with the side of the cylinder. Finally, we place the training set S on the central line and the entry sets D1 , . . . , DN on the N side lines, respectively. Formally, we define a layout of SE-graph G = (S, D, E) by (σ, Π), where σ : S → [|S|] is an ordering on the samples in S, and Π = {π1 , . . . , πN } is a set of orderings on the entries, πk : Dk → [|Dk |] (∀k ∈ [N ]). We are not concerned with how to assign the sets D1 , . . . , DN to the N lines on the side of the cylinder here. The assignment can be chosen arbitrarily. Observe that each subgraph Gk appears as a two-layered drawing on a 2D plane. We say that two edges (s, v), (s , v ) ∈ Ek cross if and only if (σ(s) > σ(s ) and πk (v) < πk (v )) or (σ(s) < σ(s ) and πk (v) > πk (v )). We denote by χ(Gk ; σ, πk ) the number of edge crossings of Gk when the samples in S are ordered by σ and the entries in Dk are ordered by πk . We denote by χ(G; σ, Π)
Bipartite Graph Representation of Multiple Decision Table Classifiers
S yes high yes med no high no med
+
s1
+s
2 3
+s s4 + s5 − s6 − s7 −
yes high
D1
yes med yes low no high no med no low
z y
53
high
med low
D3
shadow of nodes on the x, y-plane
D2
x Fig. 3. The SE-graph G = (S, D, E) for the training set S in Table 1 and the set T = {T1 , T2 , T3 } of decision tables in Table 2
the number of edge crossings of G in the layout (σ, Π), defined as the sum of the edge crossings over all the N induced subgraphs; χ(G; σ, Π) =
N
χ(Gk ; σ, πk )
k=1
=
N {(s, s ) ∈ S × S | σ(s) < σ(s ), πk (s|A ) > πk (s |A )}. (5) k k k=1
For given G = (S, D, E), we refer to the problem of finding (σ, Π) that minimizes (5) as two-sided edge crossing minimization on SE-graph (2CMSE ). Furthermore, if Π is fixed, we call the problem one-sided edge crossing minimization on SE-graph (1CMSE ) for fixed Π. We can regard 2CMSE (and 1CMSE) as a special case of the crossing minimization problem on a two-layered bipartite graph on 2D plane: We place the samples on one layer and the entries on the other layer, and for the latter, we restrict ourselves to the permutations where the entries from the same decision table are arranged consecutively. Under the above restriction, one can easily see that the total number of edge crossings to the sum of edge crossings Nequals × , which is the number of crossings over all subgraphs G1 , . . . , GN plus |S| 2 2 between edges from different edge sets Ek , Ek (k, k ∈ [N ], k = k ) and is a constant for given S and T . Therefore, the edge crossing minimization on G drawn as two-layered drawing on 2D plane is equivalent to minimization of the sum of edge crossings over all subgraphs G1 , . . . , GN , i.e., χ(G; σ, Π).
54
K. Haraguchi, S.-H. Hong, and H. Nagamochi
The two-sided crossing minimization problem (2CM) on a two-layered bipartite graph is NP-hard [17] even if the permutation of one node set is fixed, i.e., one-sided crossing minimization problem (1CM) [18]. However, there are several approximation algorithms available for 1CM. For example, Eades and Wormald [18] proposed a median method , which produces a 3-approximate solution. The √ barycenter heuristic by Sugiyama et al. [11] is an O( m)-approximation algorithm (where m denotes the number of nodes) [18]. Currently, the best known approximation algorithm is given by Nagamochi [19] that delivers a drawing with a 1.4664 factor approximation, based on a random key method. 2.3
Partially Ordered Set (Poset)
In the next section, we deal with partial order on samples which are decided by a set of orderings Π = {π1 , . . . , πN } on the entries. For any set P of elements, we call a binary relation a partial order if it satisfies reflexivity, antisymmetry and transitivity. We call (P, ) a partially ordered set (poset ). For any p, q ∈ P , we say that p and q are comparable if at least one of p q and q p holds and that they are incomparable if neither p q nor q p holds. We call (P, ) a chain (resp., an antichain) if any p, q ∈ P (p = q) are comparable (resp., incomparable). For example, denoting by m a positive integer, the poset ([m], ≤) is a chain where [m] = {1, 2, . . . , m} is a set of integers and ≤ denotes the sign of inequality on numbers.
3 3.1
Construction of Classifier via Crossing Minimization Poset and Crossing Minimization on SE-Graph
As mentioned in the literature (e.g., [20, 21]), crossing minimization may improve readability of SE-graph. In this subsection, we describe our observation on crossing minimization of SE-graph. Observe that a set Π = {π1 , . . . , πN } of orderings on the entries defines a partial order on samples as follows: For any two samples s, s , we write s Π s if and only if πk (s|Ak ) ≤ πk (s |Ak ) for all k’s. The binary relation Π is clearly a partial order on samples. Whether the poset (S, Π ) is close to a chain or an antichain depends on the choice of Π. We prefer that (S, Π ) is close to a chain rather than an antichain since we can expect meaningful analysis by making use of comparability defined on the samples. In particular, it would be interesting if positive and negative samples are separated clearly in the chain. Such a chain may contain essential information on discrimination of positive and negative samples. We then evaluate the poset (S, Π ) by whether there exists a “good” bijection from (S, Π ) to the chain ([|S|], ≤), i.e., an ordering σ : S → [|S|] on the samples. The goodness of σ should take into account how σ preserves comparability of the samples in the poset (S, Π ). It should be penalized for such a conflict:
Bipartite Graph Representation of Multiple Decision Table Classifiers
55
σ(s) < σ(s ) but πk (s|Ak ) > πk (s |Ak ). Then the following equation can serve as an evaluator of the poset (S, Π ); min σ
N {(s, s ) ∈ S × S | σ(s) < σ(s ), πk (s|A ) > πk (s |A )} k k k=1
= min χ(G; σ, Π). σ
(6)
Compare (5) and (6). The problem of computing (6) is no more than 1CMSE for fixed Π. We then claim that 2CMSE finds such a poset (S, Π ) that is the closest (in our sense) to a chain among all Π’s. 3.2
Construction Algorithm of SE-Graph Classifier
Recall GMV (c1 , . . . , cN , θ) defined in Sect. 2.1. We now compute the layout (σ, Π) of SE-graph along with GMV. Consider the 1CMSE that asks to decide the optimum ordering σ on the samples, where the orderings π1 , . . . , πN on the entries are fixed as a nondecreasing order of class level. To be more precise, by denoting Dk = {v1 , v2 , . . . , vK } (∀k ∈ [N ]) and ck (v1 ) ≤ ck (v2 ) ≤ · · · ≤ ck (vK ), we assign πk (v1 ) = 1, πk (v2 ) = 2, · · · , πk (vK ) = K. To solve the problem, let us adopt barycenter heuristic for 1CM [11] here. Barycenter heuristic permutes the samples in S in a nondecreasing order of barycenter (see (4)). Computing its layout by barycenter heuristic, we assert that the SE-graph appears as a good visualization of GMV in the following sense: (i) The number of edge crossings is (approximately) minimized. Indeed, barycenter heuristic has been recognized as an effective approximation algorithm in practice; e.g., [22]. (ii) The resulting drawing can enable some meaningful analysis on GMV. For example, the computed string of samples (which we call the sample string) is split into two substrings according to whether barycenter is larger than θ or not. The samples in the former (resp., latter) substring are estimated as positive (resp., negative) by GMV. We call a GMV visualized as above an SE-graph based classifier (SEC ). Construction of SEC consists of four steps, and we describe their details and computational complexity as follows. Step 1 : Generation of Decision Table Set. In our previous work [12], we did not consider how to generate T , and conducted the experiments for randomly generated T . In this paper, we consider collecting “good” decision tables from a given larger set of decision tables. More specifically, we decide T = {T1 , . . . , TN } as follows: We first generate N ≥ N decision tables by choosing the attribute sets at random. From the generated N decision tables, we take T as the set of the N decision tables having the smallest empirical error rate (see (3)).
56
K. Haraguchi, S.-H. Hong, and H. Nagamochi
To generate a decision table Tk = (Ak , k ), it takes O(n) time to choose an − attribute set Ak and O(|Dk | + n|S|) time to compute m+ Ak (v) and mAk (v) (see (1)) for all entries v’s, by which the label k (v) is immediate by (2). Then it
N is required O( k=1 |Dk | + nN |S|) time to obtain N decision tables. Further, it takes O(N log N ) time to select the N decision tables having the smallest
N empirical error rate, which is fewer than O( k=1 |Dk | + nN |S|). Step 2 : Determination of Class Levels. We prefer such a sample string where positive and negative samples are separated well, rather than a patchy string. Recall that the barycenter of a sample is the average of the class levels of the matched entries. Hence we should decide the value ck (v) ∈ [−1, 1] of class level to be large (resp., small) if positive (resp., negative) class is dominant in the entry v. In this paper, we adopt the following definition of ck (v) which was proposed by Stanfill and Waltz [23]; ⎧ − ⎨0 if m+ Ak (v) = mAk (v) = 0, + 2mA (v) ck (v) = (7) k −1 otherwise. ⎩ m+ (v)+m − (v) Ak
Ak
N It requires O( k=1 |Dk |) time to compute all class levels. In (7), if the denominator is fixed, ck (v) is linear with respect to m+ Ak (v). In our previous work [12], we defined ck (v) based on statistical test where ck (v) performs like a threshold function or a sigmoid function rather than a linear func− tion. For example, for two entries v, v ∈ Dk , suppose that (m+ Ak (v), mAk (v)) = + − (7, 3) and (mAk (v ), mAk (v )) = (10, 0). Then the definition of (7) gives 0.4 to v and 1 to v , while the definition of [12] gives almost 1 to both entries. The definition of (7) discriminates the entries in more detail. Step 3 : Crossing Minimization. Since we adopt barycenter heuristic to solve 1CMSE, we need to compute the barycenters of the samples, which requires O(nN |S|) time. Step 4 : Determination of Threshold. For threshold θ ∈ [−1, 1], we choose the value so that the empirical error rate of the resulting SEC is minimized. Thus it is given by the following equality (which was introduced in [12]), and can be computed in O(|S|) time if the samples have been sorted in barycenter (which requires O(|S| log |S|) time): 1 + |{s ∈ S + | b(s+ ) ≤ θ }| + |{s− ∈ S − | b(s− ) > θ }| . (8) θ ∈[−1,1] |S|
θ = arg min
Computational Complexity. Overall, our SEC construction algorithm requires
N O( k=1 |Dk | + nN |S|) time, which is due to generation of decision tables (Step 1). To use SEC, i.e., to classify a future sample s ∈ S, we need O(nN ) time to check the matched entries, to compute its barycenter b(s ), and to decide whether b(s ) ≤ θ or b(s ) > θ.
Bipartite Graph Representation of Multiple Decision Table Classifiers
4
57
Experimental Results
In this section, we compare a new version of SEC (denoted as SEC-II) with other classifiers in terms of prediction error rate on future samples. For comparison, we take its former version (denoted as SEC-I) [12], decision tree constructed by C4.5 [13] and support vector machine (SVM) constructed by LibSVM package [14] of WEKA [15]. We use 13 benchmark data sets from UCI Machine Learning Repository [24]. Among these, 7 data sets (named CHESS, MONKS-1, MONKS-2, MONKS-3, MUSHROOM, TICTACTOE and VOTING) are discrete ones, where the domain of each attribute is a set of discrete nominals, and thus can be used in our formulation without any conversion. The other 6 data sets (named BCW, GLASS, HABERMAN, HEART, HEPATITIS and IONO) are not discrete ones, where some attributes may have numerical values. We transform each non-discrete data set into binary one by the algorithm considered in our previous work [25]. For estimation of prediction error rate, we construct a classifier from a training set and then evaluate its error rate on a test set. Among the 13 data sets, MONKS-1,2,3 and IONO have their own training and test sets, and thus we use them in the experiments. For the other data sets, we estimate the prediction error rate by 10-fold cross validation, a well-known experimental methodology to split a data set into training/test sets [26]. For SEC’s, we take the average of error rates over 100 different T ’s because its performance must heavily depend on the choice of T . Figure 2 shows construction of SEC-II for GLASS data set. In Fig. 2 (b), we can see that the entries are sorted by class level for each decision table, and that the samples are sorted by barycenter. A future sample is classified according to whether it falls on the left side (−) or the right side (+) to the threshold θ. Table 3 shows the error rates of the SEC-I/II, C4.5 and LibSVM. For SEC’s, we generate N = 100 decision tables at random, with the restriction that the cardinality of an attribute set is at most 4. Furthermore, for SEC-II, we select N = 10 decision tables that achieve the smallest empirical error rate. The standard deviations arising from different T ’s are also shown. We set all parameters of C4.5 and LibSVM to the default values, where we use RBF kernel for LibSVM. In general, SEC-II outperforms the SEC-I in almost all data sets, where the improvement is 25% on average. SEC-II wins over SEC-I in 8 out of 13 data sets by difference over standard deviation. Also, SEC-II is competitive with C4.5 and LibSVM. Surprisingly, SEC-II wins over C4.5 (resp., LibSVM) in MONKS1 and BCW (resp., MONKS-1 and HABERMAN) by difference over standard deviation, and in 7 (resp., 6) data sets, the error rate of C4.5 (resp., LibSVM) is still in the range of standard deviation from the average of SEC-II. In the field of machine learning, it is broadly recognized that processing attributes (e.g., pruning or weighting attributes) plays a significant role in constructing effective classifiers [1, 16, 13]. Our SEC-II constructs decision tables on randomly selected attribute sets and collects some of them just simply by
58
K. Haraguchi, S.-H. Hong, and H. Nagamochi
Table 3. Error rates (%) of SEC-II, in comparison with SEC-I [12], C4.5 and LibSVM Data CHESS MONKS-1 MONKS-2 MONKS-3 MUSHROOM TICTACTOE VOTING BCW GLASS HABERMAN HEART HEPATITIS IONO
Samples 3196 124 169 122 8124 958 435 699 214 306 270 155 200
SEC-II 14.79 ± 4.39 7.14 ± 6.48 34.08 ± 1.76 4.75 ± 1.40 0.08 ± 0.12 17.64 ± 1.99 5.41 ± 0.74 3.83 ± 0.39 6.17 ± 0.56 25.82 ± 0.89 18.00 ± 1.63 21.68 ± 1.35 7.48 ± 3.93
SEC-I 21.06 ± 3.04 21.25 ± 2.42 27.02 ± 1.72 9.55 ± 1.80 1.51 ± 0.68 22.56 ± 1.34 10.46 ± 1.08 3.74 ± 0.24 7.48 ± 0.44 28.06 ± 0.77 17.65 ± 1.04 22.63 ± 1.27 9.46 ± 1.34
C4.5 0.41 24.30 35.00 2.80 0.00 15.46 4.34 5.15 6.51 26.15 18.13 20.61 4.59
LibSVM 5.88 18.28 32.40 3.70 0.00 12.52 4.36 3.57 6.49 27.11 15.55 20.62 6.62
empirical error rate, while C4.5 and LibSVM uses smarter and more sophisticated strategies for processing attributes. Based on these, we consider that SEC is an effective methodology and can be improved further by detailed inspection. Let us discuss the computation time. For MUSHROOM data set, which is the largest one, our implementation takes about 20 seconds to construct an SECI/II. (Our PC carries 2.4GHz CPU.) Most of the computation time is devoted to generating decision tables, which agrees with the analysis in Sect. 3.2. We note that our preliminary classifier [9, 10] takes 120 seconds even for IONO data set, much smaller one, which indicates that our methodology makes progress in scalability. For other classifiers, C4.5 takes less than 1 second and LibSVM does 5 to 10 seconds. It is left as our future work to develop how to generate a better and smaller set of decision tables.
5
Concluding Remarks
In this paper, we have described our background and philosophy on a new visual classifier. We have considered a classifier named SEC, as a visualization of GMV, and its improvement over [12]. From viewpoint of machine learning, GMV (c1 , . . . , cN , θ) is regarded as a hyperplane classifier that transforms a discrete sample s ∈ S into a real vector x ∈ [−1, 1]N , and that classifies s according to which side of the hyperplane x1 + · · · + xN − θ the vector x exists in. This type of classifier has already been studied (e.g., [27]). However, it is our achievement to interpret GMV as a bipartite graph (SE-graph), showing new application of crossing minimization methods in graph drawing. For future work, we should make use of the structure of the sample poset, which was discussed in Sect. 3.1. In machine learning, many previous methodologies convert unordered discrete samples into numerical ones, while we embed
Bipartite Graph Representation of Multiple Decision Table Classifiers
59
them in a poset. We conjecture that our idea on sample poset may give a new direction of processing discrete data sets. Also, we need to establish an algorithm to generate a better and smaller decision table set, to extend the formulation to multi-class data sets, etc.
References [1] Friedman, J.H.: Recent advances in predictive (machine) learning. Journal of Classification 23, 175–197 (2006) [2] Kettenring, J.R.: The practice of cluster analysis. Journal of Classification 23, 3–30 (2006) [3] Battista, G.D., Eades, P., Tamassia, R., Tollis, I.G.: Graph Drawing: Algorithms for the Visualization of Graphs. Prentice Hall, Englewood Cliffs (1999) [4] Haraguchi, K., Hong, S., Nagamochi, H.: Visual analysis of hierarchical data using 2.5D drawing with minimum occlusion. In: IEEE PacificVis 2008 (2008) (poster session) [5] Haraguchi, K., Hong, S., Nagamochi, H.: Visual analysis of hierarchical data using 2.5D drawing with minimum occlusion. Technical Report 2009-010, Department of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Japan (2009) [6] Ware, C.: Designing with a 2 1/2D attitude. Information Design Journal 10(3), 171–182 (2001) [7] Dwork, C., Kumar, R., Naor, M., Sivakumar, D.: Rank aggregation methods for the web. In: WWW10, pp. 613–622. ACM, New York (2001) [8] Biedl, T.C., Brandenburg, F.J., Deng, X.: Crossings and permutations. In: Healy, P., Nikolov, N.S. (eds.) GD 2005. LNCS, vol. 3843, pp. 1–12. Springer, Heidelberg (2006) [9] Haraguchi, K., Hong, S., Nagamochi, H.: Classification by ordering data samples. RIMS Kokyuroku 1644, 20–34 (2009) [10] Haraguchi, K., Hong, S., Nagamochi, H.: Classification via visualization of samplefeature bipartite graphs. Technical Report 2009-011, Department of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Japan (2009) [11] Sugiyama, K., Tagawa, S., Toda, M.: Methods for visual understanding of hierarchical system structures. IEEE Transactions on Systems, Man, and Cybernetics SMC-11(2), 109–125 (1981) [12] Haraguchi, K., Hong, S., Nagamochi, H.: Visualization can improve multiple decision table classifiers. In: Proc. of MDAI (to appear, 2009) [13] Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993) [14] Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001), http://www.csie.ntu.edu.tw/~ cjlin/libsvm [15] Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005), http://www.cs.waikato.ac.nz/ml/weka/ [16] Kohavi, R.: The power of decision tables. In: Lavraˇc, N., Wrobel, S. (eds.) ECML 1995. LNCS, vol. 912, pp. 174–189. Springer, Heidelberg (1995) [17] Garey, M.R., Johnson, D.S.: Crossing number is NP-complete. SIAM Journal on Algebraic and Discrete Methods 4, 312–316 (1983)
60
K. Haraguchi, S.-H. Hong, and H. Nagamochi
[18] Eades, P., Wormald, N.C.: Edge crossings in drawings of bipartite graphs. Algorithmica 11, 379–403 (1994) [19] Nagamochi, H.: An improved bound on the one-sided minimum crossing number in two-layered drawings. Discrete and Computational Geometry 33(4), 569–591 (2005) [20] Purchase, H.: Which aesthetic has the greatest effect on human understanding? In: DiBattista, G. (ed.) GD 1997. LNCS, vol. 1353, pp. 248–261. Springer, Heidelberg (1997) [21] Huang, W., Hong, S., Eades, P.: Layout effects on sociogram perception. In: Healy, P., Nikolov, N.S. (eds.) GD 2005. LNCS, vol. 3843, pp. 262–273. Springer, Heidelberg (2006) [22] J¨ unger, M., Mutzel, P.: 2-layer straightline crossing minimization: Performance of exact and heuristic algorithms. Journal of Graph Algorithms and Applications 1(1), 1–25 (1997) [23] Stanfill, C., Waltz, D.L.: Toward memory-based reasoning. Commun. ACM 29(12), 1213–1228 (1986) [24] Asuncion, A., Newman, D.: UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences (2007), http://www.ics.uci.edu/~ mlearn/MLRepository.html [25] Haraguchi, K., Nagamochi, H.: Extension of ICF classifiers to real world data sets. In: Okuno, H.G., Ali, M. (eds.) IEA/AIE 2007. LNCS (LNAI), vol. 4570, pp. 776–785. Springer, Heidelberg (2007) [26] Weiss, S.M., Kulikowski, C.A.: Computer Systems that Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning, and Expert Systems. Morgan Kaufmann, San Francisco (1991) [27] Grabczewski, K., Jankowski, N.: Transformations of symbolic data for continuous data oriented models. In: Kaynak, O., Alpaydın, E., Oja, E., Xu, L. (eds.) ICANN 2003 and ICONIP 2003. LNCS, vol. 2714, pp. 359–366. Springer, Heidelberg (2003)
Bounds for Multistage Stochastic Programs Using Supervised Learning Strategies Boris Defourny, Damien Ernst, and Louis Wehenkel University of Li`ege, Systems and Modeling, B28, B-4000 Li`ege, Belgium
[email protected] http://www.montefiore.ulg.ac.be/~bdf/
Abstract. We propose a generic method for obtaining quickly good upper bounds on the minimal value of a multistage stochastic program. The method is based on the simulation of a feasible decision policy, synthesized by a strategy relying on any scenario tree approximation from stochastic programming and on supervised learning techniques from machine learning.
1
Context
Let Ω denote a measurable space equipped with a sigma algebra B of subsets of Ω, defined as follows. For t = 0, 1, . . . , T − 1, we let ξt be a random variable valued in a subset of an Euclidian space Ξt , and let Bt denote the sigma algebra def generated by the collection of random variables ξ[0: t−1] = {ξ0 , . . . , ξt−1 }, with B0 = {∅, Ω} corresponding to the trivial sigma algebra. Then we set BT −1 = B. Note that B0 ⊂ B1 ⊂ · · · ⊂ BT −1 form a filtration; without loss of generality, we can assume that the inclusions are proper — that is, ξt cannot be reduced to a function of ξ[0: t−1] . Let πt : Ξ → Ut denote a Bt -measurable mapping from the product space Ξ = Ξ0 × · · · × ΞT −1 to an Euclidian space Ut , and let Πt denote the class of such mappings. Of course Π0 is a class of real-valued constant functions. We equip the measurable space (Ω, B) with the probability measure P and consider the following optimization program, which is a multistage stochastic program put in abstract form: S:
min E {f (ξ, π(ξ))}
π∈Π
subject to πt (ξ) ∈ Ut (ξ) almost surely.
(1)
Here ξ denotes the random vector [ξ0 . . . ξT −1 ] valued in Ξ, and π is the mapping from Ξ to the product space U = U0 × · · · × UT −1 defined by π(ξ) = [π0 (ξ) . . . πT −1 (ξ)] with πt ∈ Πt . We call such π an implementable policy and let Π denote the class of implementable policies. The function f : Ξ × U → R ∪ {±∞} is B-measurable and such that f (·, ξ) is convex for each ξ. It can be interpreted as a multi-period cost function. The sets Ut (ξ) are defined as Bt -measurable closed convex subsets of Ut . A set Ut (ξ) may implicitly depend on π0 (ξ), . . . , πt−1 (ξ) viewed as Bt -measurable O. Watanabe and T. Zeugmann (Eds.): SAGA 2009, LNCS 5792, pp. 61–73, 2009. c Springer-Verlag Berlin Heidelberg 2009
62
B. Defourny, D. Ernst, and L. Wehenkel
random variables; in that context we let U0 be nonempty, and Ut (ξ) be nonempty whenever πτ (ξ) ∈ Uτ (ξ) for each τ < t — these conditions correspond to a relatively complete recourse assumption. The interpretation of the measurability restrictions put on the class Π is that def the sequence {ut } = {πt (ξ)} depends on information on ξ gradually revealed. Note the impact of these restrictions: should the mappings πt be B-measurable, and then an optimal solution would simply be given by u = π(ξ) with u = [u0 . . . uT −1 ] a real vector minimizing f (ξ, u) subject to constraints ut ∈ Ut (ξ) and optimized separately for each ξ. A usual approximation for estimating the optimal value of programs of the form (1) consists in replacing the probability space (Ω, B, P) by a simpler one, so as to replace the expectation by a finite sum and make the optimization program tractable. One then ends up with a finite number of possible realizations ξ (k) of ξ — called scenarios — and a probability P induced by probability nmeasure (k) (k) (k) masses p = P (ξ = ξ ) > 0 satisfying k=1 p = 1. The program (1) then becomes S :
min
π∈Π
n
subject to πt (ξ (k) ) ∈ Ut (ξ (k) ) ∀k . (2)
p(k) f (ξ (k) , π(ξ (k) ))
k=1 (k) def
It has a finite number of optimization variables ut = πt (ξ (k) ). Remark that the random variables ξt have actually been replaced by other random variables ξt , and the sigma subalgebras Bt actually replaced by those Bt generated by the collection ξ[0: t−1] = {ξ0 , . . . , ξt−1 }. In particular, B is now identified to BT −1 , and the class Πt is such that πt is Bt -measurable. This paper is interested in the following question. We want to generalize a solution for (2) — that is, values of πt (ξ) on a finite subset of points of Ξ — to a good candidate solution π ˆ for (1), of course in such a way that π ˆ ∈ Π and ˆ (ξ) π ˆt (ξ) ∈ Ut (implementability and feasibility). Moreover, we want that π be easy to evaluate (low complexity), so that we can, considering a test sample TS of m independent and identically distributed (i.i.d.) new scenarios ξ (j) , compute efficiently an unbiased estimator of E{f (ξ, π ˆ (ξ))} by m
1 π) = f (ξ (j) , π ˆ (ξ (j) )) , RTS (ˆ m j=1
(3)
which provides, up to the standard error of (3), and given that π ˆ is feasible but possibly suboptimal, an upper bound on the optimal value of (1). The rest of the paper is organized as follows. Section 2 motivates the question. Section 3 describes properties of the data extracted from approximate programs (2). Section 4 casts the generalization problem within a standard supervised learning framework and discusses the possible adaptations to the present context. Section 5 proposes learning algorithms. Section 6 points to related work and Section 7 concludes.
Bounds for Multistage Stochastic Programs
2
63
Motivation
Coming back to the transition from (1) to (2), we observe that there exist, in the stochastic programming literature, algorithms A(Θ), with Θ any algorithmdependent parameters, that take as input (Ω, B, P), possibly under the form of the conditional distributions of ξt given ξ[0: t−1] , and return an approximation (Ω , B , P ) under the form of n scenario-probability pairs. These pairs are generally arranged under the form of a scenario tree [1, 2, 3, 4, 5, 6, 7, 8, 9], from which a version of (2) can be formulated and solved. But this approach raises several concerns, in particular on the good choices for the parameters Θ. – It has been shown in [10] for a particular randomized algorithm A1 (Θ1 ) relying on Monte Carlo sampling — the Sample Average Approximation (SAA) [5] — that it is necessary, for guaranteeing with a certain probability 1 − α that an optimal value of (2) is -close to that of (1), to have a number of scenarios n that grows with T exponentially. Thus in practice, one might have to use unreliable approximations built with a small fraction of this number n. What happens then is not clear. It has also been shown in [11] that it is not possible to obtain an upper bound on the optimal value of (1) using statistical estimates based on A1 (Θ1 ) when T > 2. – It has been shown in [8] that there exists a particular derandomized algorithm A2 (Θ2 ) relying on Quasi-Monte Carlo sampling and valid for a certain class of models for ξ, such that the optimal value and solution set of (2) converge to those of (1) asymptotically. What happens in the non-asymptotical case is not clear. – It has been shown in [9] that there exist two polynomial-time deterministic heuristics A3 (Θ3 ) — the backward and forward tree constructions — that approximate the solution to a combinatorial problem, ideal in a certain sense, which is NP-hard. However, the choice of Θ3 is left as largely open. There actually exists a generic method for computing upper bounds on the value of (1) using any particular algorithm A(Θ). This method has been used by authors studying the value of stochastic programming models [12] or the performance of particular algorithms A(Θ) [13, 14]. It relies on an estimate of the form (3) with π ˆ replaced by a certain policy π ˜ defined implicitly. It has computational requirements which turn out to be extreme. To see this, we describe the procedure used to compute the sequence {˜ πt (ξ (j) )} corresponding (j) to a particular scenario ξ . Assuming that the solution to the A(Θ)-approximation (2) of a problem S of (1) the form (1) is available, in particular u0 ∈ Π0 ∩ U0 , one begins by setting (1) (j) π ˜0 (ξ (j) ) = u0 . Then, one considers a new problem, say S(ξ0 ), derived from (j) S by replacing the random variable ξ0 by its realization ξ0 — and possibly taking account of π ˜0 (ξ (j) ) for simplifying the formulation. This makes the sigma algebra B1 in (1) trivial and Π1 a class of constant functions. One uses again the algorithm A(Θ), possibly with an updated Θ, to obtain an approximate ver(j) (1) sion (2) of S(ξ0 ). One computes its solution, extracts u1 ∈ Π1 ∩ U1 from
64
B. Defourny, D. Ernst, and L. Wehenkel (1)
it, and sets π ˜1 (ξ (j) ) = u1 . The procedure is repeated for each t = 2, . . . , T − 1, (j) on the sequence of problems S(ξ[0: t−1] ) formed by using the growing collection (j)
of observations ξ[0: t−1] extracted from ξ (j) . It is not hard to see that the resulting sequence π ˜t (ξ (j) ) forming π ˜ (ξ (j) ) meets the implementability and feasibility conditions. But repeating the full evaluation process on a sufficient number m of scenarios is particularly cumbersome. The bottleneck may come from the complexity of the programs to solve, from the running time of the algorithm A(Θ) itself, or, if A(Θ) is a randomized algorithm, from the new source of variance it adds to the Monte-Carlo estimate (3). Now an upper bound on the value of (1) computed quickly and reflecting the quality of A(Θ) would allow a more systematic exploration of the space of parameters Θ. In the case of a randomized algorithm such as A1 (Θ1 ), it may allow to rank candidate solutions obtained through several runs of the algorithm.
3
Datasets
We consider a problem S and the approximate version (2) obtained through an algorithm A(θ). Let (p(i) , ξ (i) , u(i) ) denote a probability-scenario-decision triplet formed by extracting from an optimal solution to (2) the sequence u(i) = (i) (i) (i) (i) [u0 . . . uT −1 ] that corresponds to the scenario ξ (i) = [ξ0 . . . ξT −1 ] of probability p(i) . There are n of such triplets; let us collect them into a dataset Dn . Definition 1 (Regular datasets) In this paper, a dataset is said to be S−regular, or simply regular, if the decisions u(i) of its triplets (p(i) , ξ (i) , u(i) ) are jointly optimal with respect to a program (2) over the complete set of probability-scenario pairs (p(i) , ξ (i) ). Normal datasets have two immediate but important properties. Proposition 1 (Properties of regular datasets) 1. For any pair i, j of triplets (p(i) , ξ (i) , u(i) ) and (p(j) , ξ (j) , u(j) ) in Dn , the non(i) (j) anticipativity property holds: for t = 0, ut = ut and for 1 ≤ t ≤ T − 1, (i) (j) (i) (j) if ξ[0: t−1] = ξ[0: t−1] then ut = ut . 2. For any triplet (p(i) , ξ (i) , u(i) ), the feasibility property holds: for all t ≥ 0, (i) ut ∈ Ut (ξ (i) ). The interpretation of these properties may be better perceived through the proof. Proof. Property 1 results from the measurability restriction put on the classes Πt . Since the sigma algebras Bt are generated by the random variables ξ[0: t−1] , the decisions ut = πt (ξ) are Bt -measurable if and only if there exists a measurable map gt : Ξ0 × · · · × Ξt−1 → Ut such that ut = gt (ξ0 , . . . , ξt−1 ) in any event (Theorem 20.1 in [15]). The latter set of conditions is enforced in (2) either by (i) (j) (i) (j) imposing explicit equality constraints ut = ut whenever ξ[0: t−1] = ξ[0: t−1] , or by merging those equal optimization variables into a single one. (i) def Property 2 is directly enforced by the constraint ut = πt (ξ (i) ) ∈ Ut (ξ (i) ) in (2).
Bounds for Multistage Stochastic Programs
65
Definition 2 (NAF property). In this paper, a dataset is said to have the NAF property if its triplets mutually satisfy the Non-Anticipativity properties, and if each triplet satisfies the Feasibility properties relative to S. Remark 1. Let Dn+1 be a regular dataset with triplets having the same probability p(i) = (n + 1)−1 . Let Dn be the dataset obtained by deleting one triplet from Dn+1 and setting p(i) = n−1 for each remaining triplet. Then Dn has still the NAF property but is not necessarily regular. Any dataset with the NAF property induces a set of compatible policies, in the following sense. Proposition 2 (Compatible policies) Let Dn be a dataset of triplets (p(k) , ξ (k) , u(k) ) with the NAF property. Let P(S) denote the set of implementable and feasible policies for S. We assume that P(S) is nonempty. We define P(Dn ) as the set of policies π ∈ Π such that π(ξ (k) ) = u(k) , 1 ≤ k ≤ n. Then it holds that P(S) ∩ P(Dn ) is nonempty. In (1) (i) particular, each policy π in P(S) ∩ P(Dn ) satisfies π0 (ξ) = u0 , and πt (ξ) = ut (i) whenever ξ[0: t−1] = ξ[0: t−1] for some 1 ≤ i ≤ n and 0 < t < T −1. These policies are said to be compatible with the dataset. Proof. We recall that π ∈ P(S) iff π ∈ Π and πt (ξ) ∈ Ut (ξ) almost surely. If there exists a policy π in P(S) ∩ P(Dn ), it is clear, by the feasibility prop(k) erty of the triplets in Dn , that πt (ξ (k) ) = ut is in Ut (ξ (k) ) for each k, t. The properties stated in the proposition then follow from the Bt -measurability of πt . It remains to show that given the value of the mapping π over the points def Ξ Dn = {ξ (1) , . . . , ξ (n) }, it is still possible to specify π(ξ) for any ξ ∈ Ξ \ Ξ Dn such that πt (ξ) ∈ Ut (ξ). But this follows from the relatively complete recourse (i) assumption: for any scenario ξ such that ξ[0: t−1] = ξ[0: t−1] for some i, t, setting (i)
πτ (ξ) = uτ for each τ ≤ t cannot make the sets Ut+1 (ξ), . . . , UT −1 (ξ) empty.
4
Generalization Strategies
Proposition 2 suggests to view each pair (ξ (k) , u(k) ) of a dataset Dn with the NAF property as being generated by a single implementable and feasible policy π ∈ P(Dn ). One can thus try to predict the output π(ξ) of that policy on any new scenario, on the basis of the triplets of Dn . However, the problem is not well-posed if we cannot discriminate among the policies compatible with the dataset. There exists two classical approaches to further specify the problem: 1. Choose a class of models on which the search for π can be restricted; 2. Put a prior probability over every possible function for π, compute posterior probabilities using the observations provided by the dataset, and then make a prediction for the policy. It turns out that the two approaches raise similar issues regarding their adaptation to the context of this paper; we pursue the discussion using the first one.
66
B. Defourny, D. Ernst, and L. Wehenkel
Definition 3 (Training sets) Starting from the triplets of a dataset Dn , for each t > 0, let St be a set made def def (k) (k) (k) of the pairs (x(k) , y (k) ) with x(k) = (ξ0 , . . . , ξt−1 ) and y (k) = ut . Definition 4 (Supervised Learning — Hypothesis Class View) Consider a training set St of m pairs (x(k) , y (k) ) ∈ Xt × Yt . Assume that these pairs are i.i.d. samples drawn from a fixed but unknown distribution Pt over Xt × Yt . Consider a hypothesis class Ht of mappings ht : Xt → Yt . The goal of a supervised learning algorithm L : [Xt × Yt ]m → Ht is to select a hypothesis ht ∈ Ht such that ht (x) approximates y in the best possible way on new samples (x, y) drawn from the distribution Pt . In many approaches to supervised learning, the selection of the hypothesis ht is done by first specifying a loss function : X × Y × Y → R+ for penalizing the error between the prediction ht (x) and the target y, and a complexity measure C : H → R+ for penalizing the complexity of the hypothesis [16, 17]. Defining the empirical risk of a hypothesis ht ∈ Ht as R,St (ht ) = m−1
m
(x(k) , y (k) , ht (x(k) )) ,
k=1
one selects the hypothesis ht by minimizing a weighted sum of the empirical risk and the complexity measure (we use λ ≥ 0): h∗t (λ) = arg min R,St (ht ) + λ C(ht ) . ht ∈Ht
(4)
Choosing the right value for λ is essentially a model selection issue. One seeks to choose λ such that h∗t (λ) would also minimize the generalization error R (ht ) = E(x,y)∼Pt { (x, y, ht (x))} .
(5)
Ideally, the dataset is large, and we can partition the samples into a training set, a validation set, and a test set. Hypotheses h∗t (λ) are optimized on the training set, and then ranked on the basis of their risk on the validation set. The best hypothesis h∗t (λ∗ ) is retained and its generalization error is estimated on the test set. Otherwise, when the dataset is small, there exist computationally-intensive techniques to estimate the generalization error by working on several random partitions of the dataset [18, 19]. To actually produce an upper bound in (3), the prediction of the output of π(ξ) must be implementable and feasible. While it is not hard to decompose the learning of π into a sequence of learning tasks for π0 , π1 (ξ1 ), . . . , πT −1 (ξ[0: T −2] ), more care is needed for enforcing the feasibility of a prediction. Let Ξ[0: t] be a shorthand for Ξ0 × · · · × Ξt and U[0: t] be a shorthand for U0 × · · · × Ut . Definition 5 (Feasibility mappings) We call Mt : Ξ[0: t−1] × U[0: t] → Ut a Ut -feasibility mapping if its range is always contained in the feasible set Ut (ξ) whenever Ut (ξ) is nonempty.
Bounds for Multistage Stochastic Programs
67
Example 1 (Projections) Let || · || denote a strictly convex norm on Ut . For the sake of concreteness, let Ut (ξ) be represented as {ut ∈ Ut : gk (ξ0 , . . . , ξt−1 , u0 , . . . , ut ) ≤ 0, 1 ≤ k ≤ m} with gk jointly convex in u0 , . . . , ut . The mapping Mt defined by Mt (ξ[0: t−1] , u[0: t] ) = arg min ||z − ut || z∈Ut
subject to
gk (ξ0 , . . . , ξt−1 , u0 , . . . , ut−1 , z) ≤ 0 ,
1≤k≤m
is a Ut -feasibility mapping. Proposition 3 (Representation of implementable and feasible policies) Let Mt be feasibility mappings for the sets Ut , t = 0, . . . , T − 1. Let h0 denote a constant, and let ht : Ξ[0: t−1] → Ut be some mappings, t = 1, . . . , T − 1. Then the mapping π ˆ : Ξ → U defined by π ˆ (ξ) = [ˆ π0 , . . . , π ˆT −1 ] , π ˆt = Mt (ξ0 , . . . , ξt−1 , π ˆ0 , π ˆ1 , . . . , π ˆt−1 , ht (ξ0 , . . . , ξt−1 )) π ˆ0 = M0 (h0 ) , corresponds to an implementable and feasible policy π ˆ for S. It can be used in (3). The computational complexity of π ˆt depends on the complexity of evaluating ht and then Mt . The mappings ht of Prop. 3 can be defined as the best hypothesis of a standard supervised learning algorithm for the learning set St . The model selection can be performed in two ways (referred in the sequel as M1 and M2): 1. We follow the classical approach of supervised learning outlined above, and keep some triplets of the dataset Dn apart for testing purpose; 2. Instead of using the loss function of the supervised learning algorithm in (5), we look for good choices of the parameters λ of the learning problems by directly computing (3) for each candidate set of parameters. (It is also possible to perform the model selection on (3) on a large validation set of new scenarios, and then recompute (3) on a large test set of new scenarios.) Remark 2. It is possible to combine the two model selection approaches in a same policy π ˆ . More generally, it is also possible to form π ˆ by combining learned models ˜t of π ˆt on t = 0, 1, . . . , t0 , and implementing on t = t0 + 1, . . . , T − 1 the models π Section 2 based on standard stochastic programming. The two classes of models are complementary: as t increases, the learning problems for ht become harder (larger input space, fewer samples), but the optimization problems S(ξ[0:t−1] ) for π ˜t become easier. Up to now, we have viewed the generalization problem as the one of selecting a good model π that explains the generation of a regular dataset Dn , and yields good predictions on new scenarios ξ in terms of a loss function from supervised learning (if we know the desired output) or in terms of the objective function of the problem (1).
68
B. Defourny, D. Ernst, and L. Wehenkel
Nevertheless, at the same time there is no guarantee that any policy π compatible with a dataset Dn having the NAF property was optimal for the original program S. We could then interpret u(k) as a noisy observation of the output π ∗ (ξ (k) ) of an optimal policy π ∗ for S. In this respect, we could still use the supervised learning approach of Prop. 3. We could even legitimately try to work with learning sets St formed from a dataset without the NAF property, for example a dataset obtained by merging the solution to several approximations (2). We can also view the generalization problem differently. Proper algorithms A(θ) come with the guarantee that the solution set of (2) converges to the solution set of (1). Thus, we could consider two regular datasets Dn and Dq , with n < q, and learn hypotheses ht from Dn that generalize well to the examples from Dq . We will refer to this generalization approach as M3.
5
Learning Algorithms
In this section, we develop algorithms for learning the hypotheses ht ∈ Ht of Prop. 3. We start by noting that if ut ∈ Ut is vector-valued with components [ut ]1 , . . . , [ut ]d , the hypothesis ht must be vector-valued as well. Of course, Proposition 4. Any standard regression algorithm suitable for one-dimensional outputs can be extended to multi-dimensional outputs. Proof. A training set St = {(x(i) , y (i) )}1≤i≤n can be split into d training sets [St ]j = {(x(i) , [y (i) ]j )}1≤i≤n , j = 1, . . . , d, from which hypotheses [ht ]1 , . . . , [ht ]d can be learned. The components [ht ]j can then be concatenated into a single vector-valued hypothesis ht . The following algorithm implements the approach M2 of Section 4. Algorithm 1 (Joint selection approach) Let S0 be an approximation of the form (2) to a problem (1). Let St denote the training sets built from a solution to S0 . Assume that the model selection in (4) is parameterized by a single parameter λ common to every component j of ht and every index t. Assume that the feasibility mappings Mt are fixed a priori. Then, for each λ in some finite set Λ, 1. Learn [ht ]j (λ) using training sets [St ]j and the λ-dependent criterion (4). ˆ (λ) (Proposition 3). 2. From [ht ]j (λ) form a policy π 3. Evaluate the score of π ˆ (λ) with (3) on m scenarios. Let v(λ) be that score. Select λ∗ = arg minλ∈Λ v(λ) and return the hypotheses h0 (λ∗ ), . . . , hT −1 (λ∗ ). Remark 3. For datasets with the NAF property, h0 (λ∗ ) = M0 (h0 (λ∗ )) = u0 . (1)
The purpose of the assumptions of Algorithm 1 is to reduce the complexity of the search in the parameter space Λ. To see this, let cA (Θ0 ) denote the expected time for forming the approximation S0 to S using algorithm A(Θ0 ), and let cS (0) denote the expected time for solving S0 . Let |Λ| denote the cardinality of Λ, cL (t) the expected running time of the learning algorithm on a dataset St , and cE (t) the expected running time of the combined computation of Mt and ht .
Bounds for Multistage Stochastic Programs
69
Proposition 5. Algorithm 1 runs in expected time T −1 T −1 T −1 |Λ| · cL (t) + m · cE (t) = |Λ| · [cL (t) + m cE (t)] , t=1
t=1
t=1
starting from data obtained in expected time cA (Θ0 ) + cS (0). In general cL (t) and cE (t) grow with the dimensions of Ξ0:t−1 and Ut , and with the cardinality of the dataset St . These dependences, and the ratio between cL (t) and cE (t), depend largely on the algorithms associated with the hypothesis spaces Ht and the feasibility mappings Mt . The value of λ ∈ Λ may actually also influence cL (t) and cE (t) but we neglect this dependence in first approximation. Clearly Algorithm 1 can be run on N parallel processes, so as to replace |Λ| in Prop. 5 by |Λ|/N . Relaxing in Algorithm 1 the assumption of a single λ and considering paramˆ eters λt ∈ Λt proper to each t could improve the quality of the learned −1policy π |Λt | inbut would also imply an expected running time proportional to Tt=1 stead of |Λ| in Prop. 5, if one attempts a systematic search over all possible values of [λ1 . . . λT −1 ] in Λ1 × · · · × ΛT −1 . In that context it might be better to switch to another model selection strategy (compare to remark 2 and M3): Algorithm 2 (Sequential selection approach) Let S0 , . . . , ST −1 be T approximations of the form (2) to a problem (1), with S0 being the only solved one. 1. From the solution to S0 , extract h0 = u0 and the training set S1 . Set t = 1. 2. Assume that the model selection for ht is parameterized by a single λt common to every component [ht ]j of ht . For each value of λt in a finite set Λt , learn from the training set St the hypothesis ht (λt ). 3. Consider the optimal values v(λt ) of programs St (λt ) derived from St as (k) follows. For each τ = 1, . . . , t and each realization ξ[0: τ −1] in St , compute (1)
(k) def
successively, using u ˆ0 def
(k)
= h0 , (k)
(k)
(k)
(k)
(k)
= Mt (ξ0 , . . . , ξτ −1 , u ˆ0 , . . . , uˆτ −1 , hτ (ξ0 , . . . , ξτ −1 )) uˆ(k) τ where hτ depends on λt when τ = t. Then define St (λt ) as the program St (k) (k) to which is added the whole set of equality constraints uτ = uˆτ . 4. Select λ∗t = arg minλt ∈Λt v(λt ) and set ht = ht (λ∗t ). 5. If t < T − 1, extract from the solution to St (λ∗t ) the training set St+1 , set t to t + 1, and go to step 2. Otherwise, return the hypotheses h0 (λ∗0 ), . . . , hT −1 (λ∗T −1 ). Given π ˆ0 , . . . , π ˆt−1 , the impact of π ˆt on the generalization abilities of π ˆ is evalˆ1 , . . . , π ˆt−1 uated on an approximation St to S distinct from those on which π have been determined. The program St (λt ) acts as a proxy for a test, on an independent set of scenarios, of a hybrid policy π † such that (i) πτ† = π ˆτ for
70
B. Defourny, D. Ernst, and L. Wehenkel
τ ≤ t, with π ˆt depending on λt , and (ii) πτ† = πτ∗ for τ > t with π ∗ an optimal policy for S. Optimizing St (λt ) yields the decisions of a policy for ut+1 , . . . , uT −1 perfectly adapted to the scenarios of St — a source of bias with respect to π ∗ — and to the possibly suboptimal decisions u0 , . . . , ut imposed on these scenarios by π ˆ0 , . . . , π ˆt . Whether v(λt ) is lower or higher than the value of π † on a large test sample cannot be predicted, but the ranking among the values v(λt ) should be fairly representative of the relative merits of the various models for π ˆt . The complexity of Algorithm 2 is expressed as follows. Let cA (Θj ) denote the expected time for forming the approximation Sj to S using algorithm A(Θj ). Let m(t, τ ) denote the number of distinct realizations ξ[0: τ −1] of ξ[0: τ −1] in St . Let cS (t) denote the expected running time for solving a program St (λt ). (k)
Proposition 6. Algorithm 2 runs in expected time t T −1 |Λt | · cL (t) + m(t, τ ) cE (τ ) + cS (t) , t=1
τ =1
starting from data obtained in expected time
T −1 j=0 cA (Θj )
+ cS (0).
The term cS (t) is usually large but decreases with t, since the step 3 of Algorithm 2 sets decisions up to time t and thus reduces St (λt ) to m(t, t) problems (k) (k) over disjoint subsets of the optimization variables ut+1 , . . . , uT −1 of St . The values m(t, τ ) are typically far smaller than m in Prop. 5. The term cS (t) is thus essentially the price paid for estimating the generalization abilities of a policy π ˆ ˆT −1 still unspecified. that leaves π ˆt+1 , . . . , π For the sake of comparison, we express the expected running time of the generic upper bounding technique of Section 2. Let cS (t) denote the expected (j) time for solving the approximation to the program S(ξ[0: t−1] ) of Section 2 obtained through A(Θt ). Let cA (Θt ) denote the expected running time of the (j) algorithm A(Θt ). Note that here Θt for t > 0 is such that there is in S(ξ[0: t−1] ) (j)
a single realization of ξ[0: t−1] corresponding to the actual observation ξ[0: t−1] of ξ[0: t−1] . Proposition 7. The bounding scheme of Section 2 runs in expected time T −1
m · [cA (Θt ) + cS (t)] ,
t=1
starting from data obtained in expected time cA (Θ0 ) + cS (0). Of course using N parallel processes allows to replace m by m/N .
6
Related Work
The need for generalizing decisions on a subset of scenarios to an admissible policy is well recognized in the stochastic programming literature [11]. Some authors
Bounds for Multistage Stochastic Programs
71
addressing this question propose to assign to a new scenario the decisions associated to the nearest scenario of the approximate solution [20, 21], thus essentially reducing the generalization problem to the one of defining a priori a measurable similarity metric in the scenario space (we note that [21] also proposes several variants for a projection step restoring the feasibility of the decisions). Put in perspective of the present framework, this amounts to adopt, without model selection, the nearest neighbor approach to regression [22] — arguably one of the most unstable prediction algorithm [17]. Our work differs in that it is concerned with the complexity of exploiting the induced policies, seeks to identify the nature of the generalization problem, and makes the connection with model selection issues well studied in statistics [23, 24, 25, 26] and statistical learning [16, 27]. As a result, guidance is provided on how to improve over heuristical methods, and on which tools to use [28, 29, 30, 31, 32, 33]. The model selection techniques developed in this paper are proper to the stochastic programming context. There is a large body of work applying supervised learning techniques to the context of Markov Decision Processes (MDP) [34] and Reinforcement Learning problems (RL) [35, 36]. In these problems, one seeks to maximize an expected sum of rewards by finding a policy that maps states to decisions. The state space is often large, but the decision space is often reduced to a finite number of possible actions. Supervised learning is used to represent the value function of dynamic programming, or to represent the policy directly [37, 38, 39, 40, 41]. We observe however that the hypothesis spaces relevant for representing policies adapted to multistage stochastic programs differ from those used in the context of Markov Decision Processes, and that the datasets are obtained in a very different way.
7
Conclusion
This paper has proposed to compute bounds on the optimal value of a multistage stochastic program by generalizing to an admissible policy the solution to a scenario-tree approximation of the program. Such bounds could be helpful to select the parameters of the algorithm that generates the scenario tree, an important aspect in practical applications. We have aimed to adapt to this task the concepts and model selection techniques from machine learning and statistical learning theory. We have outlined the special structure of the data and of the output of admissible policies. The results established in this paper have also shed light on the type of feasibility constraints with which our approach can be expected to be sound. Learning algorithms for the policies have been proposed, offering various quality-complexity tradeoffs.
Acknowledgments This paper presents research results of the Belgian Network DYSCO (Dynamical Systems, Control, and Optimization), funded by the Interuniversity Attraction Poles Programme, initiated by the Belgian State, Science Policy Office. The scientific responsibility rests with its authors. Damien Ernst is a Research Associate of the Belgian FNRS of which he acknowledges the financial support.
72
B. Defourny, D. Ernst, and L. Wehenkel
References [1] Frauendorfer, K.: Barycentric scenario trees in convex multistage stochastic programming. Mathematical Programming 75, 277–294 (1996) [2] Dempster, M.: Sequential importance sampling algorithms for dynamic stochastic programming. Annals of Operations Research 84, 153–184 (1998) [3] Dupacova, J., Consigli, G., Wallace, S.: Scenarios for multistage stochastic programs. Annals of Operations Research 100, 25–53 (2000) [4] Høyland, K., Wallace, S.: Generating scenario trees for multistage decision problems. Management Science 47(2), 295–307 (2001) [5] Shapiro, A.: Monte Carlo sampling methods. In: Ruszczy´ nski, A., Shapiro, A. (eds.) Stochastic Programming. Handbooks in Operations Research and Management Science, vol. 10, pp. 353–425. Elsevier, Amsterdam (2003) [6] Casey, M., Sen, S.: The scenario generation algorithm for multistage stochastic linear programming. Mathematics of Operations Research 30, 615–631 (2005) [7] Hochreiter, R., Pflug, G.: Financial scenario generation for stochastic multistage decision processes as facility location problems. Annals of Operations Research 152, 257–272 (2007) [8] Pennanen, T.: Epi-convergent discretizations of multistage stochastic programs via integration quadratures. Mathematical Programming 116, 461–479 (2009) [9] Heitsch, H., R¨ omisch, W.: Scenario tree modeling for multistage stochastic programs. Mathematical Programming 118(2), 371–406 (2009) [10] Shapiro, A.: On complexity of multistage stochastic programs. Operations Research Letters 34(1), 1–8 (2006) [11] Shapiro, A.: Inference of statistical bounds for multistage stochastic programming problems. Mathematical Methods of Operations Research 58(1), 57–68 (2003) [12] Golub, B., Holmer, M., McKendall, R., Pohlman, L., Zenios, S.: A stochastic programming model for money management. European Journal of Operational Research 85, 282–296 (1995) [13] Kouwenberg, R.: Scenario generation and stochastic programming models for asset liability management. European Journal of Operational Research 134, 279–292 (2001) [14] Hilli, P., Pennanen, T.: Numerical study of discretizations of multistage stochastic programs. Kybernetika 44, 185–204 (2008) [15] Billingsley, P.: Probability and Measure, 3rd edn. Wiley, Chichester (1995) [16] Vapnik, V.: Statistical Learning Theory. Wiley, Chichester (1998) [17] Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer, Heidelberg (2009) [18] Wahba, G., Golub, G., Heath, M.: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21, 215–223 (1979) [19] Efron, B., Tibshirani, R.: An introduction to the bootstrap. Chapman and Hall, London (1993) [20] Th´eni´e, J., Vial, J.P.: Step decision rules for multistage stochastic programming: A heuristic approach. Automatica 44, 1569–1584 (2008) [21] K¨ uchler, C., Vigerske, S.: Numerical evaluation of approximation methods in stochastic programming (2008) (submitted) [22] Cover, T.: Estimation by the nearest neighbor rule. IEEE Transactions on Information Theory 14, 50–55 (1968) [23] Akaike, H.: Information theory and an extension of the maximum likelihood principle. In: Proceedings of the Second International Symposium on Information Theory, pp. 267–281 (1973)
Bounds for Multistage Stochastic Programs
73
[24] Schwartz, G.: Estimating the dimension of a model. Annals of Statistics 6, 461–464 (1978) [25] Rissanen, J.: Stochastic complexity and modeling. Annals of Statistics 14, 1080– 1100 (1986) [26] James, G., Radchenko, P., Lv, J.: DASSO: connections between the Dantzig selector and Lasso. Journal of the Royal Statistical Society: Series B 71, 127–142 (2009) [27] Chapelle, O., Vapnik, V., Bengio, Y.: Model selection for small sample regression. Machine Learning 48, 315–333 (2002) [28] Huber, P.: Projection pursuit. Annals of Statistics 13, 435–475 (1985) [29] Buja, A., Hastie, T., Tibshirani, R.: Linear smoothers and additive models. Annals of Statistics 17, 453–510 (1989) [30] Friedman, J.: Multivariate adaptive regression splines (with discussion). Annals of Statistics 19, 1–141 (1991) [31] Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural networks architectures. Neural Computation 7, 219–269 (1995) [32] Williams, C., Rasmussen, C.: Gaussian processes for regression. In: Advances in Neural Information Processing Systems 8 (NIPS 1995), pp. 514–520 (1996) [33] Smola, A., Sch¨ olkopf, B., M¨ uller, K.R.: The connection between regularization operators and support vector kernels. Neural Networks 11, 637–649 (1998) [34] Puterman, M.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, Chichester (1994) [35] Bertsekas, D., Tsitsiklis, J.: Neuro-Dynamic Programming. Athena Scientific, Belmont (1996) [36] Sutton, R., Barto, A.: Reinforcement Learning, an introduction. MIT Press, Cambridge (1998) [37] Bagnell, D., Kakade, S., Ng, A., Schneider, J.: Policy search by dynamic programming. In: Advances in Neural Information Processing Systems 16 (NIPS 2003), pp. 831–838 (2004) [38] Lagoudakis, M., Parr, R.: Reinforcement learning as classification: leveraging modern classifiers. In: Proceedings of the Twentieth International Conference on Machine Learning (ICML 2003), pp. 424–431 (2003) [39] Ernst, D., Geurts, P., Wehenkel, L.: Tree-based batch mode reinforcement learning. Journal of Machine Learning Research 6, 503–556 (2005) [40] Langford, J., Zadrozny, B.: Relating reinforcement learning performance to classification performance. In: Proceedings of the Twenty-Second International Conference on Machine Learning (ICML 2005), pp. 473–480 (2005) [41] Fern, A., Yoon, S., Givan, R.: Approximate policy iteration with a policy language bias: solving relational Markov Decision Processes. Journal of Artificial Intelligence Research 25, 85–118 (2006)
On Evolvability: The Swapping Algorithm, Product Distributions, and Covariance Dimitrios I. Diochnos1 and Gy¨ orgy Tur´ an1,2 1
Dept. of Mathematics, Statistics, and Computer Science, University of Illinois at Chicago, Chicago IL 60607, USA 2 Research Group on Artificial Intelligence of the Hungarian Academy of Sciences, University of Szeged
[email protected],
[email protected]
Abstract. Valiant recently introduced a learning theoretic framework for evolution, and showed that his swapping algorithm evolves monotone conjunctions efficiently over the uniform distribution. We continue the study of the swapping algorithm for monotone conjunctions. A modified presentation is given for the uniform distribution, which leads to a characterization of best approximations, a simplified analysis and improved complexity bounds. It is shown that for product distributions a similar characterization does not hold, and there may be local optima of the fitness function. However, the characterization holds if the correlation fitness function is replaced by covariance. Evolvability results are given for product distributions using the covariance fitness function, assuming either arbitrary tolerances, or a non-degeneracy condition for the distribution and a size bound on the target. Keywords: learning, evolution.
1
Introduction
A model combining evolution and learning was introduced recently by Valiant [14]. It assumes that some functionality is evolving over time. The process of evolution is modelled by updating the representation of the current hypothesis, based on its performance for training examples. Performance is measured by the correlation of the hypothesis and the target. Updating is done using a randomized local search in a neighborhood of the current representation. The objective is to evolve a hypothesis with a close to optimal performance. As a paradigmatic example, Valiant [14] showed that monotone conjunctions of Boolean variables with the uniform probability distribution over the training examples are evolvable. Monotone conjunctions are a basic concept class for learning theory, which has been studied from several different aspects [7, 8, 10]. Valiant’s algorithm, which is referred to as the swapping algorithm in this paper, considers mutations obtained by swapping a variable for another one, and adding and deleting a variable1 , and chooses randomly among beneficial mutations (or among neutral ones if there are no beneficial mutations). 1
These mutations may be viewed as swapping a variable with the constant 1.
O. Watanabe and T. Zeugmann (Eds.): SAGA 2009, LNCS 5792, pp. 74–88, 2009. c Springer-Verlag Berlin Heidelberg 2009
The Swapping Algorithm, Product Distributions, and Covariance
75
Valiant also established a connection between the model and learning with statistical queries (Kearns [7], see also [8]), and studied different versions such as evolution with and without initialization. Valiant noted that concept learning problems have been studied before in the framework of genetic and evolutionary algorithms (e.g., Ros [11]), but his model is different (more restrictive) in its emphasis on fitness functions which depend on the training examples only through their performance, and not on the training instances themselves. His model excludes, e.g., looking at which bits are on or off in the training examples. Feldman [2, 3] gave general results on the model and its different variants, focusing on the relationship to statistical queries. He showed that statistical query algorithms of a certain type can be translated into evolution algorithms. The translation, as noted by Feldman, does not lead to the most efficient or natural2 evolution algorithms in general. This is the case with monotone conjunctions: even though their evolvability follows from Feldman’s result, it is still of interest to find simple and efficient evolution procedures for this class. Michael [9] showed that decision lists are evolvable under the uniform distribution using the Fourier representation. In general, exploring the performance of simple evolution algorithms is an interesting direction of research; hopefully, leading to new design and analysis techniques for efficient evolution algorithms. The swapping algorithm, in particular, appears to be a basic evolutionary procedure (mutating features in and out of the current hypothesis) and it exhibits interesting behavior. Thus its performance over distributions other than uniform deserves more detailed study. In this paper we continue the study of the swapping algorithm for evolving monotone conjunctions. A modified presentation of the algorithm for the uniform distribution is given, leading to a simplified analysis and an improved complexity bound (Theorem 2). We give a simple characterization of best approximations by short hypotheses, which is implicit in the analysis of the algorithm. We then consider the swapping algorithm for product distributions. Product distributions generalize the uniform distribution, and they are studied in learning theory in the context of extending learnability results from the uniform distribution, usually under non-degeneracy conditions (see, e.g. [4, 5, 6, 12]). We show that the characterization of best approximations does not hold for product distributions in general, and that the fitness function may have local optima. It is shown that the picture changes if we replace the correlation fitness function with covariance. (Using fitness functions other than correlation has also been considered by Feldman [3] and Michael [9]; the fitness functions discussed in those papers are different from covariance.) In this case there is a characterization of best approximations similar to the uniform distribution with correlation. This leads to two positive results for the evolvability of monotone conjunctions under product distributions. Theorem 4 shows that in the unbounded-precision model of evolution, the swapping algorithm using covariance as the fitness function, is an efficient 2
Of course, we do not use the term ‘natural’ here to suggest any actual connection with evolutionary processes in nature.
76
D.I. Diochnos and G. Tur´ an
algorithm for monotone conjunctions over arbitrary product distributions. Thus this result applies to a very simply defined (though clearly not realistic) evolution model, and analyzes a very simple and natural evolution algorithm (swaps using covariance) over a whole class of distributions (product distributions without any restrictions). Therefore, it may be of interest as an initial example motivating further models and algorithms, such as the introduction of short and long hypotheses in order to work with polynomial sample size estimates of performances. Theorem 5 shows that the swapping algorithm works if the target is short and the underlying product distribution is μ-nondegenerate. The paper is structured as follows. Section 2 has an informal description of the swapping algorithm. As we are focusing on a single algorithm and its variants, we do not need to define the evolution model in general. The description of the swapping algorithm and some additional material given in Section 3 contain the details of the model that are necessary for the rest of the paper. Section 4 contains the analysis of the swapping algorithm for the case of uniform distribution. The performance of the swapping algorithm for product distributions is discussed in Section 5. In Section 6 we turn to the swapping algorithm using covariance as fitness function. Finally, Section 7 contains some further remarks and open problems.
2
An Informal Description of the Swapping Algorithm
Given a set of Boolean variables x1 , . . . , xn , we assume that there is an unknown target c, a monotone conjunction of some of these variables. The possible hypotheses h are of the same class. The truth values true and false are represented by 1 and −1. The performance of a hypothesis h is 1 h(x) · c(x), (1) PerfUn (h, c) = n 2 n x∈{0,1}
called the correlation of h and c. Here Un denotes the uniform distribution over {0, 1}n. The evolution process starts with an initial hypothesis h0 , and produces a sequence of hypotheses using a random walk type procedure on the set of monotone conjunctions. Each hypothesis h is assigned a fitness value, called the performance of h. The walk is performed by picking randomly a hypothesis h from the neighborhood of the current hypothesis h which seems to be more fit (beneficial) compared to h, or is about as fit (neutral) as h. Details are given in Section 3. Some care is needed in the specification of the probability distribution over beneficial and neutral hypotheses. Moreover, there is a distinction between short and long conjunctions, and the neighborhoods they induce. Valiant uses a threshold value q = O (log(n/ε)) for this distinction. Section 4.2 has details. Valiant showed that if this algorithm runs for O(n log(n/ε)) stages, and evaluates performances using total sample size O((n/ε)6 ) and different tolerances for short, resp. long conjunctions, then with probability at least 1 − ε it finds a hypothesis h with PerfUn (h, c) 1 − ε.
The Swapping Algorithm, Product Distributions, and Covariance
3
77
Preliminaries
The neighborhood N of a conjunction h is the set of conjunctions that arise by adding a variable, removing a variable, or swapping a variable with another one, plus the conjunction itself3 . The conjunctions that arise by adding a variable form the neighborhood N+ , the conjunctions that arise by dropping a variable form the neighborhood N− , and the conjunctions that arise by swapping a variable form the neighborhood N+− . In other words we have N = N− ∪ N+ ∪ N+− ∪ {h}. As an example, let our current hypothesis be h = x1 ∧ x2 , and n = 3. Then, N− = {x1 , x2 }, N+ = {x1 ∧ x2 ∧ x3 }, and N+− = {x3 ∧ x2 , x1 ∧ x3 }. Note that |N| = O(n2 ) in general. Similarity between two conjunctions h and c in an underlying distribution Dn is measured by the performance function4 PerfDn (h, c) which is evaluated 1 approximately, by drawing a random sample S and computing |S| x∈S h(x) · c(x). The goal of the evolution process is to evolve a hypothesis h such that: Pr [PerfDn (h, c) < PerfDn (c, c) − ε] < δ.
(2)
The accuracy parameter ε and the confidence δ are treated as one in [14]. Given a target c, we split the neighborhood in 3 parts by the increase in performance that they offer. There are beneficial, neutral, and deleterious mutations. In particular, for a given neighborhood N and real constant t (tolerance) we are interested in the sets Bene = N ∩ h | PerfDn h , c PerfDn (h, c) + t (3) Neut = N ∩ h | PerfDn h , c PerfDn (h, c) − t \ Bene. A mutation is deleterious if it is neither beneficial nor neutral. The size (or length) |h| of a conjunction h is the number of variables it contains. Given a target conjunction c and a size q, we will be interested in the best size q approximation of c. Definition 1 (Best q-Approximation). A hypothesis h is called a best qapproximation of c if |h| q and ∀h = h, |h | q : PerfDn h , c PerfDn (h, c) . Note that the best approximation is not necessarily unique. In this paper the following performance functions are considered; the first one is used in [14] and the second one is the covariance of h and c5 : PerfDn (h, c) =
h(x)c(x)Dn (x) = E [h · c] = 1 − 2 · Pr [h = c]
(4)
x∈{0,1}n
Cov [h, c] = PerfDn (h, c) − E [h] · E [c] . 3 4 5
(5)
As h will be clear from the context, we write N instead of N(h). See the end of this section for the specific performance functions considered in this paper. For simplicity, we keep the notation Perf for a specific performance function. A related performance function, not considered here, is the correlation coefficient.
78
4
D.I. Diochnos and G. Tur´ an
Monotone Conjunctions under the Uniform Distribution
Given a target conjunction c and a hypothesis conjunction h, the performance of h with respect to c can be found by counting truth assignments. Let h=
m i=1
xi ∧
r =1
y
and c =
m
xi ∧
i=1
u
wk .
(6)
k=1
Thus the x’s are mutual variables, the y’s are redundant variables in h, and the w’s are undiscovered, or missing variables in c. Variables in the target c are called good, and variables not in the target c are called bad. The probability of the error region is (2r + 2u − 2)2−m−r−u and so PerfUn (h, c) = 1 − 21−m−u − 21−m−r + 22−m−r−u .
(7)
For a fixed threshold value q, a conjunction h is short (resp., long), if |h| q (resp., |h| > q). The following lemma and its corollary show that if the target conjunction is long then every long hypothesis has good performance, as both the target and the hypothesis are false on most instances. Lemma 1 (Performance Lower Bound). If |h| q and |c| q + 1 then PerfUn (h, c) > 1 − 3 · 2−q . Corollary 1. Let q lg(3/ε). If |h| q, |c| q + 1 then PerfUn (h, c) > 1 − ε. 4.1
Properties of the Local Search Procedure
Local search, when switching to h from h, is guided by the quantity Δ = PerfUn h , c − PerfUn (h, c) .
(8)
We analyze Δ using (7). The analysis is summarized in Figure 1, where the node good represents good variables and the node bad represents bad variables. Note that Δ depends only on the type of mutation performed and on the values of the parameters m, u and r; in fact, as the analysis shows, it depends on the size of the hypothesis |h| = m + r and on the number u of undiscovered variables. Comparing h ∈ N+ with h. We introduce a variable z in the hypothesis h. If z is good, Δ = 2−|h| > 0. If z is bad, Δ = 2−|h| (1 − 21−u ). Comparing h ∈ N− with h. We remove a variable z from the hypothesis h. If z is good, Δ = −21−|h| < 0. If z is bad, Δ = −21−|h| (1 − 21−u ). Comparing h ∈ N+− with h. Replacing a good with a bad variable gives Δ = −21−|h|−u. Replacing a good with a good, or a bad with a bad variable gives Δ = 0. Replacing a bad with a good variable gives Δ = 22−|h|−u .
The Swapping Algorithm, Product Distributions, and Covariance
good
bad
(a) u 2
good
bad
(b) u = 1
good
79
bad
(c) u = 0
Fig. 1. Arrows pointing towards the nodes indicate additions of variables and arrows pointing away from the nodes indicate removals of variables. Note that this is consistent with arrows indicating the swapping of variables. Thick solid lines indicate Δ > 0, simple lines indicate Δ = 0, and dashed lines indicate Δ < 0. Usually Figure 1a applies. When only one good variable is missing we have the case shown in Figure 1b. Once all good variables are discovered, Figure 1c applies; hence two arrows disappear. Note that an arrow with Δ > 0 may correspond to a beneficial or neutral mutation, depending on the value of the tolerance t.
Correlation produces a perhaps unexpected phenomenon already in the case of the uniform distribution: adding a bad variable can result in Δ being positive, 0 or negative, depending on the number of undiscovered variables. Now we turn to characterizing the best bounded size approximations of concepts, implicit in the analysis of the swapping algorithm. The existence of such characterizations seems to be related to efficient evolvability and so it may be of interest to formulate it explicitly. Such a characterization does not hold for product distributions in general, as noted in the next section. However, as shown in Section 6, the analogous characterization does hold for every product distribution if the fitness function is changed from correlation to covariance. Theorem 1 (Structure of Best Approximations). The best q-approximation of a target c is c if |c| q, or any hypothesis formed by q good variables if |c| > q. Proof. The claim follows directly from the definitions if |c| q. Let |c| > q. Let h be a hypothesis consisting of q good variables. Then both deleting a variable or swapping a good variable for a bad one decrease performance. Thus h cannot be improved among hypotheses of size at most q. If h has fewer than q variables then it can be improved by adding a good variable. If h has q variables but it contains a bad variable then its performance can be improved by swapping a bad variable for a good one. Hence every hypothesis other than the ones described in the theorem can be improved among hypotheses of size at most q. 4.2
Evolving Monotone Conjunctions under the Uniform Distribution
The core of the algorithm for evolving monotone conjunctions outlined in Section 2 is composed by the Mutator function, presented in Algorithm 1. The role of Mutator is, given a current hypothesis h, to produce a new hypothesis h which has better performance than h if Bene is nonempty, or else a hypothesis
80
D.I. Diochnos and G. Tur´ an
Algorithm 1. The Mutator Function under the Uniform Distribution
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Input: q ∈ N∗ , samples sM,1 , samples sM,2 , a hypothesis Output: a new hypothesis h if |h| > 0 then Generate N− else N− ← ∅; if |h| < q then Generate N+ else N+ ← ∅; if |h| q then Generate N+− else N+− ← ∅; vb ← GetPerformance(h); Initialize Bene, Neutral to ∅; if |h| q then t ← 2−2q else t ← 21−q ; for x ∈ N+ , N− , N+− do SetWeight(x, h, N+ , N− , N+− ); if |x| q then SetPerformance(x, c, sM,1 ); else SetPerformance(x, c, sM,2 ); vx ← GetPerformance(x); if vx vb + t then Bene ← Bene ∪ {x}; else if vx vb − t then Neutral ← Neutral ∪ {x};
h, a target c.
/* set tolerance */ /* sM,1 examples */ /* sM,2 examples */
SetWeight(h, h, N+ , N− , N+− ); Neutral ← Neutral ∪ {h}; if Bene = ∅ then return RandomSelect(Bene); else return RandomSelect(Neutral);
h with about the same performance as h, in which case h arises from h by a neutral mutation. Hence, during the evolution, we have g calls to Mutator throughout a sequence of g generations. We pass some slightly different parameters in the Mutator from those defined in [14], to avoid ambiguity. Hence, Mutator receives as input q, the maximum allowed size for the approximation, sM,1 , the sample size used for all the empirical estimates of the performance of each conjunction of size up to q, sM,2 the sample size used for conjunctions of length greater than q, and the current hypothesis h. We view conjunctions as objects that have two extra attributes, their weight and the value of their performance. GetPerformance returns the value of the performance, previously assigned by SetPerformance. The performance of the initial hypothesis has been determined by another similar call to the SetPerformance function with the appropriate sample size. Weights are assigned via SetWeight. Hence, SetWeight assigns the same weight to all members of {h} ∪ N− ∪ N+ so that they add up to 1/2, and the same weight to all the members of N+− so that they add up to 1/2. Finally, RandomSelect computes the sum W of weights of the conjunctions in the set that it has as argument, and returns a hypothesis h from that set with probability wh /W, where wh is the weight of h . Note that the neighborhoods and the tolerances are different for short and long hypotheses, where a hypothesis h is short if |h| q, and 3 . (9) q = lg ε
The Swapping Algorithm, Product Distributions, and Covariance
81
Theorem 2. For every targetconjunction c and every initial hypothesis h0 it holds that after O q + |h0 | ln δ1 iterations, each iteration evaluating the performance of O (nq) hypotheses, andeach performance being evaluated using sample 4 per iteration, equation (2) is satisfied. ln n + ln δ1 + ln ε1 size O ε1 Proof. The analysis depends on the size of the target and the initial hypothesis. Short Initial Hypothesis and Short Target. Note first that for any hypothesis h and for any target c such that |h| = m + r q and |c| = m + u q, for the nonzero values of the quantity Δ it holds that |Δ| 21−m−r−u 21−(m+r)−(m+u) = 21−(|h|+|c|) 21−2q . Tolerance for short hypotheses is t = 12 21−2q = 2−2q . Hence as long as the estimate of the performance is within t of its exact value, beneficial mutations are identified as beneficial. Therefore it is sufficient to analyze the signs of Δ along the arrows in Figure 1. Note that deleting a good variable is always deleterious, and so u is non-increasing. If there are at least two undiscovered variables (i.e., u 2, corresponding to Figure 1a), then beneficial mutations can only add or swap variables. Each swap increases the number of good variables, and so after |c| − 1 many swaps there is at most one undiscovered variable. Hence, as long as u 2, there can be at most q − |h0 | additions and at most |c| − 1 swaps. If there is one undiscovered variable (i.e., u = 1, corresponding to Figure 1b), then, in 1 step, the first beneficial mutation brings this variable into the hypothesis, and all variables become discovered. If all variables are discovered (i.e., u = 0 , corresponding to Figure 1c) then beneficial mutations are those which delete bad variables from h0 . After we get to the target, there are no beneficial mutations, and the only neutral mutation is the target itself, hence there is no change. Thus the number of steps until getting to the target is at most q − |c|. Summing up the above, the total number of steps is at most 2q − |h0 | 2q. Short Initial Hypothesis and Long Target. As long as |h| < q, we have u 2, corresponding to Figure 1a. Therefore adding any variable is beneficial. Note that replacing a bad variable by a good one may or may not be so, depending on the size of c. The same analysis as above implies that after at most 2q beneficial mutations we reach a hypothesis of size q. If |c| q + 2 then u 2 continues to hold, and so all beneficial or neutral mutations will keep hypothesis size at q. However, by Corollary 1, all those hypotheses have performance at least 1 − ε. If |c| = q + 1 then after reaching level q there is one undiscovered variable, corresponding to Figure 1b. Swaps of bad variables for good ones are beneficial. Combining these mutations with the ones needed to reach level q, we can bound the total number of steps until reaching a hypothesis of q good variables by 2q (using the same argument as above). After that, there are only neutral mutations swapping a good variable with another good one, and again all those hypotheses have performance at least 1 − ε.
82
D.I. Diochnos and G. Tur´ an
As a summary, if we start from a short hypothesis and all the empirical tests perform as expected, then, we are always at a good hypothesis after 2q iterations. This will not be the case when we start from a long hypothesis. Long Initial Hypothesis. For long hypotheses the neighborhood consists of hypotheses obtained by deleting a literal, and the hypothesis itself. We set the tolerance in such a way that every hypothesis in the neighborhood is neutral. This guarantees that with high probability in O |h0 | ln δ1 iterations we arrive at a hypothesis of size at most q, and from then on we can apply the analysis of the previous two cases. The model assumes that staying at a hypothesis is always a neutral mutation, hence it is possible to end up in a hypothesis of size bigger than q. Computing sample sizes is done by standard Chernoff bound arguments and the details are omitted.
5
Monotone Conjunctions under Product Distributions Using Correlation
A product distribution over {0, 1}n is specified by the probabilities p = (p1 , . . . , pn ), xi to 1. The probability of a truth where pi is the probability of setting the variable ai 1−ai assignment (a1 , . . . , an ) ∈ {0, 1}n is n . For the uniform i=1 pi · (1 − pi ) distribution Un the probabilities are p1 = . . . = pn = 1/2. We write Pn to denote a fixed product distribution, omitting p for simplicity. Let us consider a target c and a hypothesis h as in (6). Let index (z) be a function that returns the set of indices of the participating variables in a hypothesis z. We define the sets M = index (h) ∩ index (c) , R = index (h) \ M, and U = index (c) \ M. We can now define pi , R= p , and U= pk . M= i∈M
∈R
k∈U
Finally, set |M| = m, |R| = r, and |U| = u. Then (7) generalizes to PerfPn (h, c) = 1 − 2M(R + U − 2RU).
(10)
We impose some conditions on the pi ’s in the product distribution. Definition 2 (Nondegenerate Product Distribution). A product distribution Pn given by p = (p1 , . . . , pn ) is μ-nondegenerate if – min{pz , 1 − pz } μ for every variable z – the difference of any two members of the multiset {p1 , 1 − p1, . . . , pn , 1 − pn } is zero, or has absolute value at least μ. The following Lemma and its Corollary are analogous to Lem. 1 and Cor. 1. Lemma 2 (Performance Lower Bound). Let a hypothesis h such that |h| q − 1 and a target c such that |c| q + 1. Then, PerfPn (h, c) > 1 − 6.2 · e−μq . Corollary 2. Let q μ1 ln 6.2 , |h| q−1, |c| q+1 ⇒ PerfUn (h, c) > 1 −ε. ε
The Swapping Algorithm, Product Distributions, and Covariance
good
83
bad
good
bad
good
bad
bad
good
bad
good
bad
bad
good
bad
good
bad
?
good
? good
(a) U < 1/2
(b) U = 1/2
(c) U > 1/2
Fig. 2. The style and the directions of arrows have the same interpretation as in Figure 1. The middle layer represents variables that have the same probability of being satisfied under the distribution; i.e. pgood = pbad . A node x that is one level above another one y indicates higher probability of satisfying the variable x; i.e. px > py . Here we distinguish the three basic cases for U; for two arrows in the first case we have a ? to indicate that Δ can not be determined by simply distinguishing cases for U.
5.1
Properties of the Local Search Procedure
We want to generalize the results of Section 4.1 by looking at the quantity (11) Δ = PerfPn h , c − PerfPn (h, c) which corresponds to (8). We use (10) for the different types of mutations. The signs of Δ depend on the ordering of the probabilities pi . A variable xi is smaller (resp., larger ) than a variable xj if pi < pj (resp., xi > xj ). If pi = pj then xi and xj are equivalent. Analyzing Δ, we draw the different cases in Figure 2. However, when U < 1/2, two arrows can not be determined. These cases refer to mutations where we replace a bad variable with a bigger good one, or a good variable with a smaller bad one. Both mutations depend on the distribution; the latter has Δ = −2MR(pin /pout − 1 + 2U(1 − pin )), where out is a good variable and in is the bad smaller variable. One application of this equation is that the Structure Theorem 1 does not hold under product distributions. The other application is the construction of local optima; example follows. Example 1. Let Pn be a distribution such that p1 = p2 = 1/3, and the rest of the n − 2 variables are satisfied with probability 1/10. Set the target c = x1 ∧ x2 . A hypothesis h formed by q bad variables has performance PerfPn (h, c) = 1 − 2Pr [error region] < 1 − 2/9 = 7/9. Note that, for the nonzero values of Δ, it holds |Δ| 2μq+2 . Hence, by setting the tolerance t = μq+2 , and the accuracy on the empirical tests on conjunctions of size at most q, equal to M = t = μq+2 , all the arrows in the diagrams can be determined precisely. Starting from h0 = ∅, there are sequences of beneficial mutations in which the algorithm inserts a bad variable in each step, e.g. h0 = ∅ h1 = x3 . . . hq = q+2 =3 x . This is a local optimum, since swapping a bad variable
84
D.I. Diochnos and G. Tur´ an
with a good one yields Δ < 0. Note that μ = 1/10, q = 10 ln(62) = 42, and for ε = 1/10 the algorithm is stuck in a hypothesis with PerfPn (hq , c) < 1 − ε. Under the setup of the example above, the algorithm will insert q bad variables (n−q)(n−q−1) 2 . Rein the first q steps, with probability Γ = q−1 1 − r=0 n−r = n(n−1)
2q quiring n δ we have Γ 1 − δ. Hence, starting from the empty hypothesis, the algorithm will fail for any ε < 2/10, with probability 0.9, if we set n 840. 5.2
Special Cases
Although for arbitrary targets and arbitrary product distributions we can not guarantee that the algorithm will produce a hypothesis h such that (2) is satisfied, we can however, pinpoint some cases where the algorithm will succeed with the correct setup. These cases are targets of size at most 1 or greater than q, and heavy targets; i.e. targets that are satisfied with probability at least 1/2.
6
Covariance as a Fitness Function
The discussion in the previous section shows that there are problems with extending the analysis of the swapping algorithm from the uniform distribution to product distributions. In this section we explore the possibilities of handling product distributions with a different fitness function, covariance, given by (5). Using the same notation as in (6), and with M, R, and U representing the sets of indices as in the previous section, the first term is given by (10). Furthermore, E [h] = −1 + 2 · i∈M pi · ∈R p = −1 + 2MR (12) E [c] = −1 + 2 · i∈M pi · k∈U pk = −1 + 2MU Thus from (10) and (12) we get Cov [h, c] = 4MRU (1 − M) .
We use (13) to examine the difference Δ = Cov h , c − Cov [h, c] .
(13)
Comparing h ∈ N+ with h. We introduce a literal z in the hypothesis h. If z is good, then Δ = 4M2 RU (1 − pz ) > 0. If z is bad, then Δ = (pz − 1) Cov [h, c] 0. We have equality if m = 0; i.e. M = 1. Comparing h ∈ N− with h. We remove a literal z from the hypothesis h. If z is good, then Δ = −4M2 RU(1/pz − 1) < 0. If z is bad, then Δ = (1/pz − 1)Cov [h, c] 0. We have equality if m = 0; i.e. M = 1. Comparing h ∈ N+− with h. We swap a literal out with a literal in. If out is good and in is good, then Δ = 4M2 RU(1 − pin /pout ). If pout pin , then Δ 0, with Δ = 0 if pout = pin . If pout > pin ⇒ Δ > 0. If out is good and in is bad, then Δ = 4MRU · ((pin − 1) + M · (1 − pin /pout )).
The Swapping Algorithm, Product Distributions, and Covariance
good
bad
good
bad
good
bad
good
bad
good
bad
good
bad
(a) M = 1
85
(b) M < 1
Fig. 3. The style and the directions of arrows have the same interpretation as in the previous figures. Similarly, the hierarchy of nodes on levels has the same interpretation. Some arrows are missing in the left picture since there are no good variables in the hypothesis; i.e. M = 1.
We now examine the quantity κ = (pin − 1) + M · (1 − pin /pout ): pout pin : Then (1 − pin /pout ) 0, and (pin − 1) < 0. Therefore Δ < 0. pout > pin : Note M pout . Hence, κ < pin −1 +1 −pin /pout = pin (1 −1/pout ) < 0. If out is bad and in is bad, then Δ = (pin /pout − 1) · Cov [h, c] and Cov [h, c] 0: pout pin : In this case, Δ 0, and Δ = 0 when m = 0, or pout = pin . pout > pin : In this case Δ 0, and Δ = 0 when m = 0. If out is bad and in is good, then Δ = 4MRU(1/pout − 1 + M(1 − pin /pout )). We examine the quantity κ = 1/pout − 1 + M(1 − pin /pout ): pout < pin : Note M 1. Hence, κ > 1/pout − 1 + 1 − pin /pout = (1 − pin )/pout > 0. pout pin : In this case pin /pout 1 ⇒ κ > 0 ⇒ Δ > 0. The effects of the different mutations are summarized in Figure 3. Compared to Figure 2, it is remarkably simple and uniform, and can be summarized as follows. If there are some mutual variables (i.e. good) in the hypothesis, then – Δ > 0 if a good variable is added, a bad variable is deleted, a bad variable is replaced by a good one, a good variable is replaced by a smaller good one, and a bad variable is replaced by a larger bad one, – Δ < 0 if a good variable is deleted, a bad variable is added, a good variable is replaced by a bad one, a good variable is replaced by a larger good one, and a bad variable is replaced by a smaller bad one, – Δ = 0 if a good variable is replaced by an equivalent good one, and a bad variable is replaced by an equivalent bad one. If there are no mutual variables in the hypothesis, then – Δ > 0 if a good variable is added, or a good variable replaces a bad one. – Δ = 0 if a bad variable is added, deleted, or replaced by a bad one. Note that the beneficiality or neutrality of a mutation is not determined by these observations; it also depends on the tolerances. Nevertheless, these properties are
86
D.I. Diochnos and G. Tur´ an
sufficient for an analogue of Theorem 1 on the structure of best approximations to hold for product distributions and the covariance fitness function. Theorem 3 (Structure of Best Approximations). The best q-approximation of a target c, such that |c| 1, is c itself if |c| q, or any hypothesis formed by q good variables, such that the product q i=1 pi is minimized if |c| > q. As mentioned earlier, the existence of characterizations of best approximations is related to evolvability. This relationship is now illustrated for product distribution and the covariance fitness function. First we introduce an idealized version of evolution algorithms, where beneficiality depends on the precise value of the performance function. Definition 3 (Unbounded-Precision Evolution Algorithm). An evolution algorithm is unbounded-precision if, instead of (3) it uses Bene = N ∩ h | PerfDn h , c > PerfDn (h, c) , (14) Neut = N ∩ h | PerfDn h , c = PerfDn (h, c) or, equivalently, arbitrary tolerance to determine which hypotheses are beneficial, neutral or deleterious. All other parts of the definition are unchanged. Consider the following unbounded-precision evolution algorithm: starting from an arbitrary initial hypothesis, apply beneficial mutations as long as possible. Then beneficial mutations can add a good variable, delete a bad variable, replace a bad variable by a good one, replace a good variable by a smaller good one or replace a bad variable by a larger bad one. The amortized analysis argument of Theorem 5 in the next section shows that the number of steps is O(n2 ). Hence the following result holds. Theorem 4. The swapping algorithm using the covariance fitness function is an efficient evolution algorithm for monotone conjunctions over product distributions. 6.1
Evolving Short Monotone Conjunctions under µ-Nondegenerate Product Distributions
The problem with applying the unbounded-precision algorithm to the boundedprecision model is that the presence of the U factor in Δ may make the performance differences superpolynomially small. If we assume that the product distribution is non-degenerate and the target is short then this cannot happen, and an analysis similar to Theorem 2 shows that we indeed get an efficient evolution algorithm. In Section 7 we give some remarks on possibilities for handling long targets. We set 1 1 ln q=O . μ ε
The Swapping Algorithm, Product Distributions, and Covariance
87
Theorem 5. Let Pn be a μ-nondegenerate product distribution. The swapping algorithm, using the covariance fitness function, evolves non-empty short (1 |c| q) monotone conjunctions starting from an initial hypothesis h0 in O nq + |h0 | ln δ1 iterations, each iteration examining the performance of O (nq) hypotheses, and each performance being evaluated using sample size 4 (4/μ) ln(1/μ) 1 1 1 1 1 . ln n + ln + ln + ln O μ ε δ μ ε Proof. The analysis of the proof is similar to that of Theorem 2. Short Initial Hypothesis. Again, we are interested in the non-zero values of Δ so that, given representative samples, we can characterize precisely all the mutations. It holds that |Δ| 4μ2q+2 . Therefore, we set the tolerance t = 2μ2q+2 , and require accuracy for the empirical estimates M = t = 2μ2q+2 . Without loss of generality, we are in the state of Figure 3b, otherwise, in 1 step a good variable appears in the hypothesis, and we move from Figure 3a to Figure 3b. Throughout the evolution, implicitly, we have two arrays; one with good variables, and one with bad variables. The array of bad variables shrinks in size, while the array of good variables expands in size. Due to mutations that swap good with good variables, and bad with bad variables, each entry of those arrays can take at most n different values, since the entries of the bad array increase in value, while the entries of the good array decrease in value; i.e. we have O (nq) such mutations overall. Mutations that change the parameters m, r increase the number of good variables or decrease the number of bad variables, hence we can have at most O (q) such mutations. LongInitial Hypothesis. The arguments are similar to those in Theorem 2, and in O |h0 | ln δ1 stages the algorithm forms a short hypothesis of size q. Then, we apply the analysis above.
7
Further Remarks
We are currently working on completing the results of Section 6, by handling the case of long targets. The problem with long targets is that the term U appearing in Δ can be small. This may lead to the increase in fitness being smaller than the tolerance. The following observation may provide a starting point. It may happen that there are no beneficial mutations, but all mutations are neutral. In this case the evolution process is similar to a random walk on the set of short hypotheses. However, most short hypotheses have maximal size. Therefore, after sufficiently many steps in the random walk, the hypothesis will have maximal size with high probability. Such hypotheses have good performance, and so the evolution process leads to a good hypothesis with high probability. The evolvability of monotone conjunctions for more general classes of distributions, using the swapping algorithm with the covariance fitness function, or
88
D.I. Diochnos and G. Tur´ an
using some other simple evolutionary mechanism, is an important open problem [14]. There is a similar swapping-type learning algorithm for decision lists, where a single step exchanges two tests in the list [13, 1]. Can such an algorithm be used in the evolution model? A positive answer could give an alternative to Michael’s Fourier-based approach [9]. In summary, it appears that from the perspective of learning theory, one of the remarkable features of Valiant’s new model of evolvability is that it puts basic, well-understood learning problems in a new light and poses new questions about their learnability. One of these new questions is the performance of basic, simple evolution mechanisms, like the swapping algorithm for monotone conjunctions. The results of this paper suggest that the analysis of these mechanisms may be an interesting challenge.
References [1] Castro, J., Balc´ azar, J.L.: Simple PAC Learning of Simple Decision Lists. In: Zeugmann, T., Shinohara, T., Jantke, K.P. (eds.) ALT 1995. LNCS, vol. 997, pp. 239–248. Springer, Heidelberg (1995) [2] Feldman, V.: Evolvability from learning algorithms. In: STOC 2008, pp. 619–628. ACM, New York (2008) [3] Feldman, V.: Robustness of Evolvability. In: COLT 2009 (2009) [4] Furst, M.L., Jackson, J.C., Smith, S.W.: Improved learning of AC0 functions. In: COLT 1991, pp. 317–325 (1991) [5] Hancock, T., Mansour, Y.: Learning monotone ku DNF formulas on product distributions. In: COLT 1991, pp. 179–183 (1991) [6] Kalai, A.T., Teng, S.-H.: Decision trees are PAC-learnable from most product distributions: a smoothed analysis. CoRR, abs/0812.0933 (2008) [7] Kearns, M.: Efficient noise-tolerant learning from statistical queries. J. ACM 45(6), 983–1006 (1998) [8] Kearns, M.J., Vazirani, U.V.: An introduction to computational learning theory. MIT Press, Cambridge (1994) [9] Michael, L.: Evolvability via the Fourier Transform (2009) [10] Reischuk, R., Zeugmann, T.: A Complete and Tight Average-Case Analysis of Learning Monomials. In: Meinel, C., Tison, S. (eds.) STACS 1999. LNCS, vol. 1563, pp. 414–423. Springer, Heidelberg (1999) [11] Ros, J.P.: Learning Boolean functions with genetic algorithms: A PAC analysis. In: FGA, San Mateo, CA, pp. 257–275. Morgan Kaufmann, San Francisco (1993) [12] Servedio, R.A.: On learning monotone DNF under product distributions. Inf. Comput. 193(1), 57–74 (2004) [13] Simon, H.-U.: Learning decision lists and trees with equivalence-queries. In: Vit´ anyi, P.M.B. (ed.) EuroCOLT 1995. LNCS, vol. 904, pp. 322–336. Springer, Heidelberg (1995) [14] Valiant, L.G.: Evolvability. In: Kuˇcera, L., Kuˇcera, A. (eds.) MFCS 2007. LNCS, vol. 4708, pp. 22–43. Springer, Heidelberg (2007)
A Generic Algorithm for Approximately Solving Stochastic Graph Optimization Problems Ei Ando1, , Hirotaka Ono1,2 , and Masafumi Yamashita1,2 1
Dept. Computer Sci. and Communication Eng., Kyushu University, 744 Motooka, Nishi-ku, Fukuoka, Fukuoka 819-0395, Japan 2 Institute of Systems, Information Technologies and Nanotechnologies, Fukuoka SRP Center Building 7F 2-1-22, Momochihama, Sawara-ku, Fukuoka 814-0001, Japan
Abstract. Given a (directed or undirected) graph G = (V, E), a mutually independent random variable Xe obeying a normal distribution for each edge e ∈ E that represents its edge weight, and a property P on graph, a stochastic graph maximization problem asks the distribution function FMAX (x) of random variable XMAX = maxP ∈P e∈A Xe , where property P is identified with the set of subgraphs P = (U, A) of G having P. This paper proposes a generic algorithm for computing an elementary function F˜ (x) that approximates FMAX (x). It is applicable to any P and runs in time O(TAMAX (P) + TACNT (P)), provided the existence of an algorithm AMAX that solves the (deterministic) graph maximization problem for P in time TAMAX (P) and an algorithm ACNT that outputs an upper bound on |P| in time TACNT (P). We analyze the approximation ratio and apply it to three graph maximization problems. In case no efficient algorithms are known for solving the graph maximization problem for P, an approximation algorithm AAPR can be used instead of AMAX to reduce the time complexity, at the expense of increase of approximation ratio. Our algorithm can be modified to handle minimization problems.
1
Introduction
1.1
Motivation and Contribution
Given a (directed or undirected) graph G = (V, E), a function w : E → R that assigns a real number w(e) to each edge e ∈ E as its weight, and a property P on graph, a typical graph optimization problem asks a subgraph P = (U, A) of G having P that maximizes (or minimizes) its weight w(P ), where w(P ) = e∈A w(e). Examples of P include path, matching, spanning tree, edge cut and so on. The set of subgraphs P = (U, A) of G having P is called the feasible set. In this paper, we identify a property P with its feasible set and abuse P to denote it, so |P| is the cardinality of the feasible set. A promising approach to solve a computation problem that we encounter in real world is to model it as a weighted graph and then to solve it using a
Supported by Research Fellowships of the Japan Society for the Promotion of Science for Young Scientists.
O. Watanabe and T. Zeugmann (Eds.): SAGA 2009, LNCS 5792, pp. 89–103, 2009. c Springer-Verlag Berlin Heidelberg 2009
90
E. Ando, H. Ono, and M. Yamashita
known graph optimization algorithm. However, there are two main concerns. First, many of the graph optimization problems are known to be NP-hard, and there can be no known efficient algorithms for solving the graph optimization problem that we want to solve. In such a case, an approximation algorithm is frequently used instead. Second, quantities in real applications suggested to model as edge weights tend to vary or to be uncertain. In order to model variable and uncertain quantities, we can introduce for each edge e ∈ E a random variable Xe denoting its edge weight and formulate the problem as a stochastic graph optimization problem, which typically asks the distribution function FMAX (x) (or the value FMAX (d) −1 or FMAX (a) for a constant d or a) of a random variable XMAX = max(U,A)∈P e∈A Xe (or the distribution function FMIN (x) of a random variable XMIN = min(U,A)∈P e∈A Xe ), given a graph G = (V, E), a distribution that Xe obeys for each e ∈ E, and a graph property P. This paper proposes a generic algorithm ALG to compute an elementary function F˜ (x) that approximates FMAX (x), provided that the random variables Xe are mutually independent and obey normal distributions N (μe , σe2 ). ALG is applicable to any property P. It assumes the existence of two algorithms: an algorithm AMAX to solve the graph maximization problem for P in time TAMAX (P) set and an algorithm ACNT to approximate |P|, i.e., the cardinality of the feasible 2 for P, in time T (P). We compute μ = max μ and σ max (U,A)∈P max = e∈A e ACNT max(U,A)∈P e∈A σe2 using AMAX , and an upper bound c on |P| using ACNT . 2 Then an elementary formula representing F˜ (x) is determined from μmax , σmax ˜ and c. Once the formula is computed, we can evaluate F (d) in O(log c)-time for any d. A basic approximation property of F˜ (x) is an inequality F˜ (x) ≤ F (x) for any x ≥ μmax , or equivalently, F −1 (y) ≤ F˜ −1 (y) for any y ≥ 1/2. The value F˜ −1 (a) is more important than F˜ (d) in many applications. To see this, let us consider a typical application of circuit delay analysis. In this context, F (x) is the probability that the circuit delay of a product is less than or equal to x, and a main target of the delay analysis is to estimate F −1 (a) for a given a, since F −1 (a) = d means that the delay of a product is less than or equal to d with the probability at least a. Since F −1 (a) ≤ F˜ −1 (a), taking F˜ −1 (a) as an estimation of F −1 (a) is a safe choice to guarantee the success ratio a. Interestingly, we can calculate F˜ −1 (a) even faster than F˜ (d); F˜ −1 (a) is computable in O(1)-time. Of course, applications require not only that F˜ (x) ≤ F (x) holds, but also an approximation ratio guarantee. Let h = max(U,A)∈P |A|, ν = mine∈E μe , ρ2 = maxe∈E σe2 , and c be the output of ACNT . We show that, for any constant −1 ˜ −1 a ≥ 1/2, the approximation ratio F −1(a)/F (a) is bounded from above by O((ρ/ν) log c/h). For guaranteeing F (a) = 0, we will assume that μe > 0 for any e ∈ E. Note that in O((ρ/ν) log c/h) we consider a as a constant (although we can evaluate the ratio as a function of a, we omit it for the simplicity). One might ask the reason we do not adopt F (d)/F˜ (d) but F˜ −1 (a)/F −1 (a) for the approximation measure. An immediate reason is that many applications are
Approximately Solving Stochastic Graph Optimization Problems
91
more interested in F −1 (a) than F (d) as explained in the previous paragraph. Another reason is that a constant approximation ratio is trivially achievable, if we define it as F (d)/F˜ (d) (or F˜ (d)/F (d)) since F (x) ≤ 1. The third reason is that this definition of approximation ratio can be considered as a natural extension of that for the (deterministic) maximization problems (see e.g., [21]). A graph maximization problem is a special case of the corresponding stochastic graph maximization problem in which all variances of edge weights are 0. The distribution function FMAX (x) is now a step function, and the optimal cost is the point at which FMAX (x) jumps from 0 to 1. The counterpart of the point in −1 the general stochastic problem is FMAX (y) for y (0 < y < 1). Note that we do −1 not deal with the case y < 1/2, since FMAX (y) may be 0 or negative. The following four facts are worth emphasizing. We discuss Items 3 and 4 further in Section 6. 1. The stochastic graph optimization problem is very hard to solve. The problem of correctly computing the distribution function has been discussed only for a couple of properties under very restricted conditions, and usually Monte Carlo simulation is conducted to evaluate (i.e., to approximate) FMAX (d) at a single point d. Monte Carlo simulation is very slow to guarantee the precision. Algorithm ALG, which is applicable to any property, outputs a function F˜ (x) that approximates FMAX (x), whose quality is formally guaranteed. 2. For many properties P, we can derive a good upper bound h of h such that h = O(h). In such cases, since m c = (me/h )h ≥ h is an upper bound can guarantee a near√constant approxima on |P|, ALG √ tion ratio (ρ/ν) log c/h ≤ (ρ/ν) h log(me)/h = O( log m) = O( log n), where n = |V |, m = |E| and e is the Napier’s constant. 3. If the graph optimization problem for P is difficult, we can use an approximation algorithm AAPR for the problem instead of AMAX . Since 2m is a trivial upper bound on |P|, we can tune ALG so that it meets the given time constraint, e.g., linear time, by using a linear time approximation algorithm AAPR instead of AMAX . 4. This paper discusses maximization problems under the assumption that each random variable Xe obeys a normal distribution. However, we can modify ALG to solve minimization problems. Also, we can handle other distributions like the gamma distributions. To demonstrate how ALG is applied to properties, we investigate three graph maximization problems; the longest path problem for directed acyclic graphs (DAGs), the maximum spanning tree problem, and the maximum weight perfect matching problem for planar graphs. We choose these problems since they have efficient algorithms AMAX and ACNT , but as was emphasized in above, the existence of efficient algorithms AMAX and ACNT is not a necessary condition for ALG to be applicable to the problem.
92
1.2
E. Ando, H. Ono, and M. Yamashita
Related Works
The study of stochastic optimization problems has a considerably long history starting at least in 1950’s (see e.g., [22]). Nevertheless, not so many works have been done for computing (or approximating) the distribution function of the optimal value. PERT [17] and Critical Path Planning [13] are classical approaches to approximately solve the stochastic longest path problem for DAGs. Critical Path Planning transforms the stochastic problem into a (deterministic) longest path problem by taking some definite values as edge weights, and PERT tries to approximate the expected longest path length, whose accuracy is later discussed in [6]. Martin [16] proposed an algorithm for symbolically computing the accurate distribution function, provided that the random variables obey distributions such that their density functions are polynomials. His algorithm needs to consider all distinct paths and hence its time complexity is Ω(2n ) for some DAG of order n. There are several works for approximating the distribution function of the longest path length in a DAG, from the view point of estimating the project completion time of a PERT network. Kleindorfer [14] proposed an algorithm that computes upper and lower bounds of the distribution function. Dodin [7] improved the Kleindorfer’s bounds. However, it is not clear that how close their bounds are to the exact value. Badinelli [3] proposed to measure the quality of approximation F˜ (x) in an interval [xU , xL ] by xU ˜ |F (t) − F (t)| 1 dt, xU − xL xL F (t) where F (x) is the function to be approximated. Ludwig et al. [15] evaluated the project completion time of stochastic project networks empirically. These results again do not give any theoretical approximation guarantee. A natural idea is to discretize the ranges of random variables and then to apply a combinatorial optimization algorithm. Hagstrom [8] however showed that computing a value of the distribution function of project duration in PERT problem is #P-complete, and hence this approach does not work efficient for large instances. Motivated by the importance of the circuit delay analysis, many heuristic algorithms to approximate the distribution function of the longest path length have been proposed [4, 9, 19]. They run fast, but have no guarantee about the approximation also. Recently, Ando et al. [1]proposed an algorithm for approximating the distribution function of the longest path length in a DAG. This runs in linear time and formally guarantees its approximation ratio. Another work on stochastic graphs asks an algorithm to compute a feasible set that optimizes a given measure. For example, Ishii et al. [11] proposed a polynomial time algorithm for the following problem: Given a graph G and a real number a, compute a spanning tree T of G that minimizes the value subject to the probability that the weight of T is at most is at least a. Recently a substantial number of works have been done on the recourse model (see e.g., [18]).
Approximately Solving Stochastic Graph Optimization Problems
93
All of the methods and algorithms mentioned above are designed for some particular problems, and their purposes are different from ours in this paper, which presents a generic algorithm. The only general method we have in mind is the Monte Carlo simulation. It has many advantages, but still has drawbacks; it takes a long computation time to guarantee stable behavior, cannot formally guarantee an approximation ratio, cannot give us a formula that approximates the distribution function, and so on. The paper is organized as follows: Section 2 prepares definitions and notations. In Section 3, we first give an inequality that bounds FMAX (x) from below by utilizing an enumeration of all P ∈ P, and then, using this inequality, we present an algorithm ALG to calculate an elementary formula F˜ (x) that approximates FMAX (x). We also propose an efficient method to compute F˜ −1 (a) in O(1)-time. Section 4 derives the approximation ratio. Three application examples are given in Section 5. In Section 6, we conclude the paper by discussing how to extend algorithm ALG to more general cases and giving some open problems.
2
Preliminaries
Let X be a random variable. The probability P (X ≤ x) is called the (cumulative) distribution function of X. We call the derivative of the distribution function with respect to x the (probability) density function. A random variable X is normally distributed or obeys a normal distribution when its distribution function is given as, 2 x 1 t−μ 1 √ exp − dt, P (X ≤ x) = 2 σ −∞ σ 2π where μ and σ 2 are the mean and the variance of X, respectively. A normal distribution with mean μ and variance σ 2 is denoted by N (μ, σ 2 ). A normal distribution N (0, 1) is called the standard normal distribution and its distribution function and density function are denoted by Φ(x) and φ(x), respectively. The distribution function F (x) of N (μ, σ 2 ) is given as F (x) = Φ ((x − μ)/σ). It is well known that the sum of two normally distributed random variables also obeys a normal distribution. Let random variables X1 and X2 obey N (μ1 , σ12 ) and N (μ2 , σ22 ), respectively. The distribution of the sum X1 + X2 is given by N (μ1 + μ2 , σ12 + σ22 ). The distribution of max{X1 , X2 } is not normal even if X1 and X2 are normally distributed. The distribution function of max{X1 , X2 } is the product of distribution functions of X1 and X2 . Let F1 (x) and F2 (x) be the distribution functions of X1 and X2 . Suppose that F3 (x) is the distribution function of max{X1 , X2 }. Then we have F3 (x) = F1 (x)F2 (x). Sometimes we assume that mutually independent random variables X and X have distribution functions F (x) and F (x) that satisfies F (x) = F (x) for any x. In this case, X and X are mutually independent random variables with a common distribution. We denote an undirected edge between two vertices a and b by {a, b}. A directed edge from a vertex a to b is denoted by (a, b). An undirected graph is denoted by
94
E. Ando, H. Ono, and M. Yamashita
a pair G = (V, E), where V is its vertex set V = {v0 , v1 , . . . , vn−1 } and E is its undirected edge set E ⊆ {{vi , vj } | vi , vj ∈ V }. A directed graph is denoted by → − a pair G = (V, E), where V is also its vertex set and E is its directed edge set E ⊆ {(vi , vj ) | vi , vj ∈ V }. Each edge e ∈ E is associated with its edge weight, a random variable Xe that obeys a normal distribution N (μe , σe2 ), where μe > 0 and σe > 0. We identify a graph property P with its feasible set, i.e., the set of subgraphs P = (U, A) of G that have P. We also identify a subgraph P = (U, A) with its edge set A, as long as it is clear from the context. Let G = (V, E) be a graph with edge weights. This paper investigates the problem of approximating the distri
bution function FMAX (x) of a random variable XMAX = maxP ∈P e∈P Xe .
3 3.1
Algorithm ALG A Lower Bound on FMAX (x)
We start with a series of lemmas to obtain a lower bound on FMAX (x). Lemma 1. Let X1 , X2 , Z and Z be mutually independent random variables. Suppose that Z and Z are with a common distribution. We have P (max {X1 + Z, X2 + Z } ≤ x) ≤ P (max {X1 + Z, X2 + Z} ≤ x) .
(1)
Since this Lemma 1 seems to be well known, we omit the proof due to the space limit (for the proof see e.g. [1]). Lemma 2. For each e ∈ E, let Xe and Xe be mutually independent random variables with a common distribution N (μe , σe2 ). For any P1 , P2 ⊆ E, we have
P max Xe , Xe ≤ x ≤ P max Xe , Xe ≤ x . (2) e∈P1
e∈P2
e∈P1
e∈P2
Proof. We give the proof by applying Lemma 1. Let X1 = e∈P1 \(P1 ∩P2 ) Xe , X2 = e∈P2 \(P1 ∩P2 ) Xe , X2 = e∈P2 \(P1 ∩P2 ) Xe , Z = e∈P1 ∩P2 Xe and Z = e∈P1 ∩P2 Xe . The right hand side of (2) is equal to P (max{X1 +Z, X2 +Z} ≤ x). The left hand side of (2) is equal to P (max{X1 + Z, X2 + Z } ≤ x). Since X2 and X2 are mutually independent random variables with a common distribution, we have P (max{X1 + Z, X2 + Z } ≤ x) = P (max{X1 + Z, X2 + Z } ≤ x). Since these random variables X1 , X2 , Z and Z are mutually independent, the proof completes by Lemma 1. Let P = {P1 , P2 , . . . , P|P| }. We define mutually independent random variables X , X , . . . , XP|P| ; for i = 1, . . . , |P|, XPi obeys N (μi , σi2 ), where μi = P1 P2 2 2 e∈Pi μe and σi = e∈Pi σe . Notice that XPi ’s and Xe ’s are mutually independent, although the distributions of XPi ’s are defined by using μe ’s and σe ’s.
Approximately Solving Stochastic Graph Optimization Problems
Lemma 3
P
max
Pi ∈P
Xe
≤x
(1)
(2)
≥P
e∈Pi
95
max {XPi } ≤ x .
Pi ∈P
(|P|)
Proof. For each e ∈ E, let Xe , Xe , . . . , Xe and Xe be mutually indepen(k) dent random variables with a common distribution. Note that Xe ’s are mutually independent for any combination of 1 ≤ k ≤ |P| and e ∈ E. Since XPi ’s and Xe ’s are mutually independent, we have
(i) ≤x . Xe P max {XPi } ≤ x = P max Pi ∈P
Pi ∈P
e∈Pi
Partition P into two subsets P (i) and Q(i) , where P (i) = {P1 , P2 , . . . , Pi } and Q(i) = P \ P (i) . By Lemma 2, we have
P max Xe ≤ x = P max Xe , max Xe ≤x Pi ∈P
≥P
e∈Pi
max
Xe(1) , max Pi
e∈P1
∈Q(1)
Pi ∈Q(1)
e∈P1
Xe
e∈Pi
≤x
e∈Pi
By repeating the same argument, we can prove this lemma. Since we have
FMAX (x) = P max P ∈P
Xe ≤ x ≥ P max{XP } ≤ x = P (XP ≤ x) = FP (x),
e∈P
P ∈P
P ∈P
P ∈P
by Lemma 3, we state a lower bound on FMAX (x). Theorem 1. For any P ∈ P, let FP (x) be the distribution function of XP . Then FP (x). (3) FMAX (x) ≥ P ∈P
The inequality (3) does not directly lead to an efficient calculation of a lower bound on FMAX (x), since calculating (3) requires enumerating all P ∈ |P| and the cardinality of P can be exponential of n or more. For example, the number of spanning trees can be nn−2 . 3.2
Constructing F˜ (x)
Let FB (x)= Φ((x − μmax )/σmax ) be a distribution function where μmax = 2 maxP ∈P { e∈P μe } and σmax = maxP ∈P { e∈P σe2 }. For any P ∈ P, it is easy to show that FB (x) ≤ FP (x) if x ≥ μmax . Since FP (x) is the distribution function
96
E. Ando, H. Ono, and M. Yamashita
have FP (x) = Φ((x − μP )/σP ), where μP = e∈P μe and σP2 = Φ(x) is monotonically increasing, FP (x) = Φ((x − μP )/σP ) ≥ Φ((x − μmax )/σP ) for all x. Again, since Φ(x) is monotonically increasing, we have Φ((x − μmax )/σP ) ≥ Φ((x − μmax )/σmax ) = FB (x) for x ≥ μmax , which implies that FB (x) ≤ FP (x) for x ≥ μmax . By Theorem 1, we have (FB (x))|P| ≤ FMAX (x) if x ≥ μmax . We now have a straightforward algorithm to approximate FMAX (d) from below for a given d ≥ μmax ; first calculate μmax , σmax and an upper bound c on |P|, and then calculate (FB (d))c , by accessing the table of standard normal distribution. There are however two issues that we need to discuss further. The first issue is about algorithms to compute μmax , σmax and c. Recall that we assume the existence of an algorithm AMAX that solves the graph maximization problem for P in time TAMAX (P) and an algorithm ACNT that outputs an upper bound on |P| in time TACNT (P). To compute μmax (resp. σmax ), we solve the graph maximization problem for G = (V, E), whose edge weight w(e) of each edge e ∈ E is μe (resp. σe2 ) using AMAX , and c is computed by ACNT . Thus, in time O(TAMAX (P) + TACNT (P)), we can calculate μmax , σmax and c. The second is about the complexity of computing (FB (d))c . We introduce a 2 new family of elementary functions L(x; μ, σ 2 ) and show that L(x; μmax , σmax ) 2 ) is a achieves a lower bound on FB (x) if x ≥ μmax , and hence Lc (x; μmax , σmax 2 lower bound on FMAX (x) if x ≥ μmax . Furthermore, Lc (x; μmax , σmax ) is easy to calculate, since L(x; μ, σ 2 ) is an elementary function given as an explicit formula. Lemma 4 can be proved by standard calculus technique. of
e∈P Xe , we 2 e∈P σe . Since
Lemma 4. Let F (x; μ, σ 2 ) = Φ ((x − μ)/σ) and define L(x; μ, σ 2 ) by
1 L(x; μ, σ ) = exp − exp − 2 2
x−μ σ
2 + ln ln 2
.
Then L(x; μ, σ 2 ) ≤ F (x; μ, σ 2 ) for any x ≥ μ. 2 2 We propose Lc (x; μmax , σmax ) to use as F˜ (x). Since μmax and σmax are computed by AMAX in TAMAX (P)-time and c is computed by ACNT in TACNT (P)-time, (an explicit elementary representation of) F˜ (x) is composed in (TAMAX (P) + 2 TACNT (P))-time. Once μmax , σmax and c have been computed, evaluation of ˜ F (d) for any d is possible in O(log c)-time.1
3.3
Efficient Evaluation of F˜ −1 (a)
In Introduction, we explained why efficient calculation of F˜ −1 (a) is required in many applications. In this section, we show that F˜ −1 (a) can be efficiently evaluated; it takes a constant time. 1
Some approximation formulas proposed so far (see e.g., [10]) take mostly the same amount of time.
Approximately Solving Stochastic Graph Optimization Problems
97
Equation L(x; μ, σ 2 ) = a has two real solutions for 1/2 ≤ a < 1. We denote the larger solution by L−1 (a; μ, σ 2 ). By an easy calculation, L−1 (a; μ, σ 2 ) = μ + σ −2 ln(− ln a) + 2 ln ln 2. 2 Then F˜ −1 (a) = L−1 (a1/c ; μmax , σmax ), where 2 L−1 (a1/c ; μmax , σmax ) = μmax + σmax 2(ln c − ln(− ln a)) + 2 ln ln 2.
(4)
Unlike the evaluation of F˜ (d), evaluation of F˜ −1 (a) is possible in O(1)-time, 2 once we have computed μmax , σmax and c. Note also that ln c (not c) is required here, which suggests a possibility of using a more efficient algorithm to compute an upper bound on ln |P| instead of ACNT .
4
Bounding Approximation Ratio
This section shows the following theorem. Let h = maxP ∈P |P |, ν = mine∈E μe , ρ2 = maxe∈E σe2 , and c be the output of ACNT . Theorem 2. For any a ≥ 1/2, F˜ −1 (a) =O −1 FMAX (a)
ρ log c . ν h
We first prepare three lemmas. Let F (x) and f (x) be a normal distribution function and its normal density function with mean μ and variance σ 2 , respectively. Lemma 5. Let
1 U (x) = exp − exp − 2
x−μ σ
2 − 2 ln
x−μ +1 −3 . σ
Then for x ≥ μ, U (x) > F (x) holds. Lemma 5 can be proved by standard calculus technique. Lemma 6. Let W (y) = μ + σ
2((− ln(− ln y) + 3) − 2 ln(− ln(− ln y) + 3)).
Then for 1/2 ≤ y < 1, U −1 (y) ≥ W (y) holds. Proof. We define a function W1 (x) by its inverse function W1−1 (y) = μ + σ 2((y + 3) − 2 ln(y + 3)), for W1 (x) ≥ −1, and observe that W1−1 (y) is monotone if y ≥ −1 and hence function W1 (x) is well-defined. Note that W (y) = W1−1 (− ln(− ln y)). We show
98
E. Ando, H. Ono, and M. Yamashita
that U −1 (y) ≥ W1−1 (− ln(− ln y)) = W (y) for any y ≥ −1. Since − ln(− ln y) ≥ −1, i.e., y ≥ exp(−e) = 0.06598 . . . this proves the lemma. We define a function W2 (x) as follows: 1 W2 (x) = 2
x−μ σ
2 − 2 ln
x−μ + 1 − 3. σ
Since W2 (W1−1 (y)) = y − 2 ln(y + 3) − 2 ln( 2((y + 3) − 2 ln(y + 3)) + 1) < y, by replacing y with W1 (x), we have W2 (x) < W1 (x). Thus, exp(− exp(−W1 (x))) > exp(− exp(−W2 (x))) = U (x), which implies that U −1 (y) ≥ W1−1 (− ln(− ln y)) = W (y). Lemma 7. For a real number r, let r x−μ F (x) = Φ . σ Then for 2−r ≤ y < 1, F −1 (y) = Θ(μ + σ
2(ln r − ln(− ln y)) + 2 ln ln 2).
Proof. By Lemma 4, 5 and 6, we have F −1 (y) = Ω(W (y 1/r )) and F −1 (y 1/r ) = O(L−1 (y 1/r )) for 2−r ≤ y < 1. Since the ratio W (y 1/r )/L−1 (y 1/r ) converges to 1 as r tends to ∞, we have W (y 1/r ) = Ω(L−1 (y 1/r )), which shows the lemma. Proof of Theorem 2 √ 2 ≤ hρ2 by definition, F˜ −1 (a) = Θ(μmax + ρ h log c) by Lemma 7, Since σmax −1 where ρ2 = maxe∈E σe2 . We show 1/FMAX (a) = O(1/μmax ), which is sufficient to complete the proof, since μmax ≥ hν > 0, where ν = mine∈E μe . Let Q ∈ P be a subset thatachieves μmax = e∈Q μe . Let XQ = e∈Q Xe . Since FMAX (x) = P (maxP ∈P e∈P Xe ≤ x), we have ⎛ ⎞ FQ (x) = P ⎝ Xe ≤ x⎠ ≥ FMAX (x), e∈Q −1 (y) for all y. Since FQ−1 (a) = Ω(μmax ) for all x, which implies that FQ−1 (y) ≤ FMAX −1 (a) = O(1/μmax ). for a ≥ 1/2 by Lemma 7, which implies that 1/FMAX
Approximately Solving Stochastic Graph Optimization Problems
5 5.1
99
Three Examples The Stochastic Longest Path Problem in a DAG
→ − Let us consider the stochastic longest path problem. Given a DAG G = (V, E) and normal distributions N (μe , σe2 ) that random variables Xe assigned to edge → − e ∈ E obeys, we consider a DAG G with a weight function w : E → R defined by w(e) = μe (resp. σe2 ) for any e ∈ E and run a known longest path algorithm for a DAG (like PERT [17]) to compute μmax (resp. σmax ) in O(n + m), where n = |V | and m = |E|. As for an upper bound c on |P|, mn is a trivial upper bound. However, we can use the dynamic programming technique to compute the exact value of |P| in O(n + m)-time. We first topologically sort V = {v1 , v2 , . . . , vn } and assume that vi < vi+1 for 1 ≤ i ≤ n − 1. That is, v1 is the source and vn is the sink. Since the number of paths from source to vi is the sum of the number of paths from source to the parents of vi , a dynamic programming algorithm can count the number of paths from the source to the sink. Thus our algorithm runs in O(n√ + m)-time. h log d), the Whereas the approximation ratio that is proved in [1] is O((ρ/ν) approximation ratio of our algorithm is O((ρ/ν) (log |V |)/h). This result is also mentioned in the author’s previous paper [2]. 5.2
The Stochastic Maximum Spanning Tree Problem
Next consider the stochastic maximum spanning tree problem. Currently the fastest maximum spanning tree algorithm is shown by Chazelle [5], and it runs in O(mα(m, n))-time, where α(m, n) is the inverse of Ackermann’s function. We can compute μmax and σmax using this algorithm. As for an upper bound c on |P|, a trivial upper bound is nn−2 . We can also count it by Matrix-Tree Theorem in O(TDET (n − 1))-time, where TDET (n − 1), which is O(n3 ), is the time complexity to calculate the determinant of an (n − 1) × (n − 1) matrix. Thus our algorithm runs in O(n3 )-time. 5.3
The Stochastic Maximum Weight Perfect Matching Problem in a Planar Graph
Finally, consider the stochastic maximum weight perfect matching problem for planar graphs. Currently the fastest maximum weight perfect matching algorithm for planar graphs is due to Varadarajan [20], and it runs in O(n3/2 log5 n)-time. The number of perfect matching can be obtained by an algorithm due to [12] as a square root of a determinant of (n − 1) × (n − 1) matrix, whose elements are 1, −1 or 0. Thus our algorithm runs in O(n3 )-time.
6
Discussions and Conclusions
Given a graph G = (V, E), a mutually independent random variable Xe obeying a normal distribution N (μe , σe2 ) for each edge e ∈ E, and a property P on graph, we
100
E. Ando, H. Ono, and M. Yamashita
proposed a method ALG to compute an elementary function F˜ (x) (as an explicit formula) that approximates the distribution function FMAX (x) of random variable XMAX = max(U,A)∈P e∈A Xe in time O(TAMAX (P) + TACNT (P)), provided the existence of an algorithm AMAX that solves the graph maximization problem for P in time TAMAX (P) and an algorithm ACNT that outputs an upper bound on |P| in time TACNT (P). Given formula F˜ (x), F˜ (d) is computed in O(log c)-time for any d, and interestingly F˜ −1 (a) is computable in O(1)-time (faster than F˜ (d)) for any a, where c is the upper bound on |P| computed by ACNT . We showed that, for any a ≥ 1/2, the approximation ratio F˜ −1 (a)/F −1 (a) is bounded from above by O((ρ/ν) log c/h), where h = max(U,A)∈P {|A|}, ν = mine∈E {μe } and ρ2 = maxe∈E {σe2 }. There are several issues that we did not discuss in this paper. 1. a method to modify ALG so that it can handle minimization problems 2. a method to use an approximation algorithm AAPR instead of AMAX when AMAX is slow 3. a method to handle cases in which edge weights obey other distributions 4. a method to handle cases in which edge weights are not mutually independent 5. a method to use a combination of a wider variety of elementary functions to construct a better approximation F˜ (x) of F (x) As we promised in Section 1, we briefly discuss items (1)-(3) in below. However, the discussions are still on a preliminary stage, and we leave thorough discussions as future works. 6.1
Handling Minimization Problems
Given a graph G = (V, E), a distribution that Xe obeys for each e ∈ E, and a graph property P, consider how to approximate the distribution function FMIN (x) of a random variable XMIN = min(U,A)∈P e∈A Xe . Observe that the complementary distribution function of the weight of any edge set P ∈ P is an upper bound on the complementary distribution function of XMIN . We thus consider how to bound FMIN (x) from above. For constructing the algorithm to approximate FMIN (x) from above, we must show an upper bound on FMIN (x). Let P = {P1 , . . . , P|P| }, and XP1 , . . . , XP|P| be random variables that obey N (μi , σi2 ) where μi = e∈Pi μe and σi2 = e∈Pi σe2 . The following theorem is symmetric to Theorem 1. Theorem 3. Let FP (x) be the distribution function of XP . Then we have FMIN (x) ≤ 1 − (1 − FP (x)).
P ∈P
Let μmin = min(U,A)∈P { e∈A μe } and F (x; μ, σ 2 ) = Φ((x − μ)/σ). For any P ∈ 2 P, F (x; μmin , σ 2 ) ≥ FP (x) if x ≤ μmin , and hence 1 − (1 − F (x; μmin , σmax ))|P| ≥ FMIN (x) if x ≤ μmin by Theorem 3. The following is thus a straightforward approximation algorithm for FMIN (x): First we calculate μmin , σmax and an upper bound c on |P|, and then output 1 −
Approximately Solving Stochastic Graph Optimization Problems
101
2 (1 − F (x; μmin , σmax ))c . We however make use of a more sophisticated function L 2 given in Section 3 and propose 1 − (1 − L(x; μmin , σmax ))c as F˜ (x). By the same arguments in the proofs of Lemmas 4 and 5, we can show the next lemma.
Lemma 8. Let L(x; μ, σ 2 ) be a function defined in Lemma 4. Then for any x ≤ μ, L(x; μ, σ 2 ) ≤ 1 − F (x; μ, σ 2 ). 1/c 2 2 Let L−1 ; μmin , σmax ) be the smaller root of equation L(x; μmin , σmax ) = a. min (a −1 1/c 2 Then Lmin (1 − (1 − a) ; μmin , σmax ) gives a lower bound on FMIN (x) if x ≤ 2 F −1 (a1/c ; μmin , σmax ). As for estimating the approximation ratio, for convenience, we assume that all μe for e ∈ E are negative, which allows us to have exactly the same approximation ratio as in the maximization problem.
6.2
Using AAPR Instead of AMAX
Let us discuss the effect of using an approximation algorithm AAPR instead of AMAX . AAPR is used to compute μmax and σmax . Let μapr and σapr be the out2 2 2 puts and assume that μmax ≤ μapr ≤ αμmax and σmax ≤ σapr ≤ ασmax , where α > 1. We should be careful about this approximation since an “approximation algorithm” in usual sense for a maximization problem are for approximating μmax or σmax from below (see e.g., [21]). However, we must have μapr and σapr greater than μmax and σmax , respectively. That is, we ask AAPR to approximate μmax and σmax from above. This is possible as long as there is an approximation algorithm, because approximation ratio is shown as the ratio between approximation and an upper bound that can be obtained in polynomial time; we utilize the upper bound as AAPR since we use the value of μmax and σmax (we do not need the structures 2 that gives the values). We then propose Lc (x; μapr , σapr ) as F˜ (x). Since by (4), 2 L−1 (a1/c ; μapr , σapr ) ≤ α, −1 1/c 2 L (a ; μmax , σmax )
the approximation ratio is estimated as follows: F˜ −1 (a) ρ log c . =O α −1 ν h FMAX (a) As an example, we consider the following stochastic knapsack problem. Given a set S = {a1 , . . . , an } of items. Each item ai ∈ S is associated with profit Xi , that is a normally distributed random variable and obeys N (μi , σi2 ). We assume that the profits are mutually independent. Each item ai ∈ S is also associated with size s(ai ) ∈ R, that is a static value. Let ν be the maximum mean of a profit of an item. According to [21], we can 2 approximately compute μmax and σmax in O(n2 ((α − 1)ν )/n) and O(n2 ((α − 1)ρ)/n) time, respectively, with approximation ratio α. The upper bound on the
102
E. Ando, H. Ono, and M. Yamashita
number of possible can be obtained trivially as 2n . By applying our algorithm for the stochastic optimization problem, we can approximately compute the value of −1 FMAX (y) with approximation ratio O(α(ρ/ν) n/h) for the stochastic knapsack problem. 6.3
Handling Other Distributions
The Gamma Distribution. Our approach seems to be applicable to many other cases in which the edge weights obey other distributions, although the approximation guarantee has to be examined separately for each distribution. Let us first consider the gamma distribution with a shape k and a scale θ. Then its density function f (x; k, θ) is given as 0 x≤0 f (x; k, θ) = xk−1 exp(−x/θ) x > 0. θ k Γ (k) If every |P | for P ∈ P is sufficiently large and all edge weights are random variables with a common distribution, the central limit theorem works, and the algorithm in this paper becomes applicable and guarantees the same approximation ratio. However, if |P | is small, or edge weights are not with a common distribution, for gamma distribution function F (x;k, θ), we need to discover a func2 2 ) such that (1) L(x; μmax , σmax ) < F (x; k, θ) for x that tion L(x; μmax , σmax is larger than certain known value, and (2) the exact or approximate value of 2 ) is efficiently computable. We obtain the approximation raL−1 (a1/c ; μmax , σmax 2 )/ FMAX (x; k, θ). tio by evaluating L−1 (a1/c ; μmax , σmax This method would produce a good approximation ratio when the distribution functions of edge weights do not have “heavy tails,” and investigation of the cases in which some distribution functions have heavy tails is left as a future work.
References [1] Ando, E., Nakata, T., Yamashita, M.: Approximating the Longest Path Length of a Stochastic DAG by a Normal Distribution in Linear Time. Journal of Discrete Algorithms (2009), doi:10.1016/j.jda.2009.01.001 [2] Ando, E., Ono, H., Sadakane, K., Yamashita, M.: A Counting-Based Approximation the Distribution Function of the Longest Path Length in Directed Acyclic Graphs. In: Proceedings of the 7th Forum on Information Technology, pp. 7–8 (2008) [3] Badinelli, R.D.: Approximating Probability Density Functions and Their Convolutions Using Orthogonal Polynomials. European Journal of Operational Research 95, 211–230 (1996) [4] Berkelaar, M.: Statistical delay calculation, a linear time method. In: Proceedings of the International Workshop on Timing Analysis (TAU 1997), pp. 15–24 (1997) [5] Chazelle, B.: A Minimum Spanning Tree Algorithm with Inverse-Ackermann Type Complexity. Journal of the ACM 47(6), 1028–1047 (2000)
Approximately Solving Stochastic Graph Optimization Problems
103
[6] Clark, C.E.: The PERT model for the distribution of an activity time. Operations Research 10, 405–406 (1962) [7] Dodin, B.: Bounding the Project Completion Time Distribution in PERT Networks. Networks 33(4), 862–881 (1985) [8] Hagstrom, J.N.: Computational Complexity of PERT Problems. Networks 18, 139–147 (1988) [9] Hashimoto, M., Onodera, H.: A performance optimization method by gate sizing using statistical static timing analysis. IEICE Trans. Fundamentals, E83-A 12, 2558–2568 (2000) [10] Hastings, C.: Approximations for Digital Computers. Princeton University Press, Princeton (1955) [11] Ishii, H., Shiode, S., Nishida, T., Namasuya, Y.: Stochastic spanning tree problem. Discrete Appl. Math. 3, 263–273 (1981) [12] Kasteleyn, P.W.: Graph theory and crystal physics. In: Harary, F. (ed.) Graph Theory and Theoretical Physics, pp. 43–110. Academic Press, London (1967) [13] Kelley Jr., J.E.: Critical-path planning and scheduling: Mathematical basis. Operations Research 10, 912–915 (1962) [14] Kleindorfer, G.B.: Bounding Distributions for a Stochastic Acyclic Network. Operations Research 19, 1586–1601 (1971) [15] Ludwig, A., M¨ ohring, R.H., Stork, F.: A Computational Study on Bounding the Makespan Distribution in Stochastic Project Networks. Annals of Operations Research 102, 49–64 (2001) [16] Martin, J.J.: Distribution of the time through a directed, acyclic network. Operations Research 13, 46–66 (1965) [17] Malcolm, D.G., Roseboom, J.H., Clark, C.E., Fazar, W.: Application of a technique for research and development program evaluation. Operations Research 7, 646–669 (1959) [18] Ravi, R., Sinha, A.: Hedging Uncertainly: Approximation algorithms for stochastic optimization problems. Mathematical Programming 108(1), 97–114 (2006) [19] Tsukiyama, S., Tanaka, M., Fukui, M.: An Estimation Algorithm of the Critical Path Delay for a Combinatorial Circuit. In: IEICE Proceedings of 13th Workshop on Circuits and Systems, pp. 131–135 (2000) (in Japanese) [20] Varadarajan, K.R.: A divide-and-conquer algorithm for min-cost perfect matching in the plane. In: Proc. 38th Annual IEEE Symposium on Foundations of Computer Science (1998) [21] Vazirani, V.V.: Approximation Algorithms. Springer, Berlin (2001) [22] Wets, R.J.-B.: Stochastic Programming. In: Nemhauser, G.L., Rinnooy Kan, A.H.G., Todd, M.J. (eds.) Handbooks in Operations Research and Management Science. Optimization, vol. 1, North-Holland, Amsterdam (1991)
How to Design a Linear Cover Time Random Walk on a Finite Graph Yoshiaki Nonaka1 , Hirotaka Ono1,2 , Kunihiko Sadakane3 , and Masafumi Yamashita1,2 1
2
Department of Informatics, Kyushu University, Japan
[email protected], {ono,mak}@csce.kyushu-u.ac.jp Institute of Systems, Information Technologies and Nanotechnologies (ISIT), Japan 3 Principles of Informatics Research Division, National Institute of Informatics, Japan
[email protected] Abstract. Arandom walk on a finite graph G = (V, E) is random token circulation on vertices of G. A token on a vertex in V moves to one of its adjacent vertices according to a transition probability matrix P . It is known that both of the hitting time and the cover time of the standard random walk are bounded by O(|V |3 ), in which the token randomly moves to an adjacent vertex with the uniform probability. This estimation is tight in a sense, that is, there exist graphs for which the hitting time and cover times of the standard random walk are Ω(|V |3 ). Thus the following questions naturally arise: is it possible to speed up a random walk, that is, to design a transition probability for G that achieves a faster cover time? Or, how large (or small) is the lower bound on the cover time of random walks on G? In this paper, we investigate how we can/cannot design a faster random walk in terms of the cover time. We give necessary conditions for a graph G to have a linear cover time random walk, i,e., the cover time of the random walk on G is O(|V |). We also present a class of graphs that have a linear cover time. As a byproduct, we obtain the lower bound Ω(|V | log |V |) of the cover time of any random walk on trees.
1 1.1
Introduction Random Walks
Given a finite undirected and connected graph G = (V, E) and a transition probability matrix P = (puv )u,v∈V such that puv > 0 only if (u, v) ∈ E, a random walk ω on G starting at a vertex u ∈ V under P is an infinite sequence ω = ω0 , ω1 , · · · of random variables ωi whose domain is V , such that ω0 = u with probability 1 and the probability that ωi+1 = w provided that ωi = v is pvw for i = 0, 1, · · · . The standard random walk on a graph is the one in which the transition probability is uniform, that is, the transition probability matrix P0 is defined by, for any u, v ∈ V , 1 v ∈ N (u), puv = deg(u) 0 otherwise, O. Watanabe and T. Zeugmann (Eds.): SAGA 2009, LNCS 5792, pp. 104–116, 2009. c Springer-Verlag Berlin Heidelberg 2009
How to Design a Linear Cover Time Random Walk on a Finite Graph
105
where N (u) and deg(u) are the set of vertices adjacent to a vertex u and the degree of u respectively. The hitting time HG (P ; u, v) from u ∈ V to v ∈ V is the expected number of transitions necessary for random walk ω starting from u to reach v for the first time under a P , and the hitting time of the HG (P ) of G is defined to be HG (P ) = maxu,v∈V HG (P ; u, v). The cover time CG (P ; u) from u ∈ V is the expected number of transitions necessary for random walk ω starting from u to visit all vertices in V under a P , and the cover time CG (P ) of G is defined to be CG (P ) = maxu∈V CG (P ; u). Let n = |V | and m = |E|. Then the cover time of the standard random walk on any graph G with n vertices and m edges are CG (P0 ) ≤ 2m(n − 1) holds[1, 2], whose result was later refined by Feige [5, 6]: (1 − o(1))n log n ≤ CG (P0 ) ≤ (1 + o(1))
4 3 n . 27
Since there is a graph L (called a Lollipop) such that HL (P0 ) = (1 − o(1))
4 3 n , 27
both the hitting and cover times of standard random walks are Θ(n3 ) [4]. 1.2
Related Work
Previous studies of the standard random walk have shown a tight analysis of the hitting time and the cover time. The result implies that to achieve a faster random walk we need to adopt a different transition probability from the one of the standard random walk. Ikeda et. al. [11] proposed a transition probability matrix P1 = (puv )u,v∈V defined by, for any u, v ∈ V , ⎧ ⎨ deg(v)−1/2 if v ∈ N (u), −1/2 w∈N (u) deg(w) puv = ⎩0 otherwise, and showed that the hitting and cover times of the random walk for any graph G are bounded by O(n2 ) and O(n2 log n) respectively, where n is the order of G. In addition, they proved that the hitting and the cover times of any random walk on path graph are bounded by Ω(n2 ). It should be noted that the above random walk is defined only by local degree information. As another random walk using only local degree information, the Metropolis walk is known. Given a graph G and a target (stationary) distribution (πv )v∈V on vertices, we can construct a random walk that attains the stationary distribution (πv )v∈V by applying the Metropolis-Hastings algorithm [9] to the standard random walk. The constructed random walk is called the Metropolis walk. Although we omit the precise definitions, the transition probability of the Metropolis walk also depends only on the local degree information. Recently the authors of this paper have shown that the hitting and cover times of Metropolis walk are bounded by O(f n2 ) and O(f n2 log n) respectively, where f = maxu,v πu /πv , and also there
106
Y. Nonaka et al.
exist pairs of a graph and its stationary distribution for which the hitting and cover times of the Metropolis walk take Ω(f n2 ) and Ω(f n2 log n), respectively. By relaxing the restriction that the transition probability is defined only by local degree information, we may obtain a faster random walk. In fact, it is easy to see that, for any graph G, there exists a random walk whose cover time is O(n2 ), by considering a random walk on its spanning tree. The paper follows this research direction: Given the whole topological information of graph G, by carefully designing a transition probability of a random walk on G, is it possible to attain a faster cover time? 1.3
Our Results
In this paper, we investigate the conditions for which a graph G has a linear cover time random walk, i.e., a random walk whose cover time is bounded by O(n). We first show that for any tree with n vertices and any transition probability matrix, the cover time is Ω(n log n). This result implies that any tree does not have a linear cover time random walk and we need to well utilize cycle structures of a graph to obtain a linear cover time random walk. Thus, we next focus on biconnected component decomposition of graph G, which gives necessary conditions for the existence of a linear cover time random walk. Finally, we consider Hamiltonian component decomposition, which yields a sufficient condition for a linear cover time random walk.
2
Preliminaries
We assume a standard terminology of graph theory. Suppose that G = (V, E) is a finite, undirected, simple, connected graph with the order n = |V | and the size m = |E|. For u ∈ V , let N (u) = {v|(u, v) ∈ E} and deg(u) = |N (u)| be the set of vertices adjacent to a vertex u ∈ V and the degree of u, respectively. A vertex v is a pendant-vertex if deg(u) = 1. A vertex v is a cut-point if G − v is not connected. A biconnected component is a maximal subgraph that has no cut-point. An edge e ∈ E is a bridge if G − e is not connected. A graph is Hamiltonian if it has a spanning cycle. Let diam(G) be the diameter of G, that is, maxu,v∈V dist(u, v), where dist(u, v) denote the shortest path length between u and v. Given graph G = (V, E), let biconnected component decomposition be a partition of V to V1 , V2 , · · · , Vk such that each induced subgraph of G on Vi (i = 1, · · · , k) is biconnected. Let P = (puv )u,v∈V be a transition probability matrix on G. That is, for any u ∈ V and v ∈ V , puv ≥ 0, and puv > 0 only if (u, v) ∈ E, and v∈V puv = 1. Let π = (πv )v∈V be the stationary distribution vector of P . That is, for any u∈V, v∈V
πv pvu = πu .
How to Design a Linear Cover Time Random Walk on a Finite Graph
107
Now, for any graph G and a transition probability matrix P and its stationary distribution π, since HG (P ; u, v) = puw (1 + H(P ; w, v)) − puv HG (P ; v, v), w∈V
we have
πu HG (P ; u, v) =
u∈V
πw HG (P ; w, v) +
w∈V
πu puw − πv HG (P ; v, v),
u∈V w∈V
which implies that πv HG (P ; v, v) = 1
3
for any v ∈ V.
(1)
Lower Bound of the Cover Time on Tree
In this section, we prove that for any tree with n vertices and for any transition probability matrix, the cover time of a random walk is Ω(n log n). In Section 3.1, we show that the tight lower bound of the cover time of a random walk on the star graph is n · h(n) ≈ n log n. Then, in Section 3.2, the star graph has the minimum cover time random walk among trees with n vertices. 3.1
Lower Bound on the Cover Time for a Star Graph
A star graph is a tree whose diameter is 2. We define a star graph with n vertices as Sn = ({v1 , · · · , vn }, {(v1 , vi )|i = 2, · · · , n}), that is, n − 1 vertices are connected to a vertex v1 as pendant-vertices. We first show that for any random walk on Sn the cover time is Ω(n log n). To prove this, we consider the transition probabilities between a vertex v0 and two pendant-vertices v1 , v2 connected to v0 in a general setting. Let G = (V, E) be a graph that has two pendant-vertices v1 and v2 . These pendant-vertices are both connected to a vertex v0 . Let P be a transition probability matrix of a
Fig. 1. Star graph
Fig. 2. A graph with two pendantvertices
108
Y. Nonaka et al.
random walk on G, and p1 and p2 denote the transition probabilities from v0 to v1 and from v0 to v2 , respectively. By fixing the probabilities in P other than p1 and p2 , we can consider another transition probability P (q1 , q2 ) by changing p1 and p2 with q1 and q2 , respectively. (Of course, q1 and q2 should satisfy that 0 ≤ q1 ≤ 1, 0 ≤ q2 ≤ 1 and q1 + q2 = p1 + p2 .) Lemma 1. The cover time CG (P (q1 , q2 )) is minimized at q1 = q2 . Proof. Let V = V \ {v1 , v2 }, and let pi be the transition probability from v0 to wi ∈ {N (v0 ) ∩ V } for i = 3, · · · , deg(v0 ). The hitting times from v0 to pendant-vertices v1 and v2 are represented as follows: 1 (1 + q2 + (pH)∗ ), q1 1 H(P (q1 , q2 ); v0 , v2 ) = (1 + q1 + (pH)∗ ), q2 H(P (q1 , q2 ); v0 , v1 ) =
(2) (3)
def deg(v ) where (pH)∗ = i=3 0 pi H(wi , v0 ). We consider the following two cases: (i) Starting at u ∈ V Hitting times between two vertices other than v1 and v2 do not depend on q1 and q2 . This implies that in case (ii) the expected number of transitions for the token to visit all the vertices other than v1 and v2 is fixed. Thus, the cover time is minimum if and only if the expected number of transitions for the token to visit both v1 and v2 is minimum. Let C be the expected number of transitions that the token starting from v0 takes to visit v1 and v2 . Then,
C = 2 +
2 p∗ + (pH)∗ 1 , (q1 + q22 )(1 + (pH)∗ ) + q13 + q23 + ∗ (1 − p )q1 q2 1 − p∗
(4)
def deg(v ) where p∗ = i=3 0 pi In the following, we consider to find q1 and q2 minimizing C , at which the cover time C(P (q1 , q2 ); u) is minimum. Since q1 + q2 = 1 − p∗ by the definition of p∗ , we have
q12 + q22 = (q1 + q2 )2 − 2q1 q2 = (1 − p∗ )2 − 2q1 q2 , q13 + q23 = (1 − p∗ )((1 − p∗ )2 − 3q1 q2 ). Since p∗ is fixed, Eq (4) is minimized when q1 q2 takes the maximum, i.e., q1 = q2 . (ii) Starting at u ∈ {v1 , v2 } We simply denote H(P (q1 , q2 ); v0 , v1 ) and H(P (q1 , q2 ); v0 , v2 ) by H(v0 , v1 ) and H(v0 , v2 ), respectively. The cover time C(P (q1 , q2 ; u)) is minimized when max{H(v0 , v1 ), H(v0 , v2 )} is minimum. Let qm be max{q1 , q2 }, and max{H(v0 , v1 ), H(v0 , v2 )} =
1 (1 + qm + (pH)∗ ). 1 − p∗ − qm
Eq (5) is minimized when qm is minimum, that is q1 = q2 .
(5)
How to Design a Linear Cover Time Random Walk on a Finite Graph
109
Lemma 1 can be generalized to the case in which v0 is connected with k pendantvertices for any positive integer k ≥ 2. Let G = (V, E) be a graph that has k pendant-vertices v1 , · · · , vk , and all of these pendant-vertices are connected to vertex v0 . Given a transition probability matrix P on G, we can define P (q1 , q2 , . . . , qk ) similarly to P (q1 , q2 ), that is, P (q1 , q2 , . . . , qk ) is the transition probability matrix obtained from P by replacing the transition probabilities from v0 to vi in P with a variable qi . Then we have the following theorem: Theorem 2. The cover time CG (P (q1 , q2 , . . . , k)) is minimized at q1 = · · · = qk . By applying Theorem 2 to a star graph repeatedly, we can see that the standard random walk is the fastest random walk for star graph in terms of the cover time. Since the cover time of the standard random walk on the star graph can be evaluated as the coupon collector problem, we obtain the following: Theorem 3. The cover time of the standard random walk on star graph Sn is 2n · h(n − 1), where h(i) is the i-th harmonic number. Corollary 1. For any random walk of transition probability matrix P on star graph Sn with n vertices, CSn (P ) = Ω(n log n). 3.2
Lower Bound of the Cover Time on Tree
As mentioned at the beginning of Section 3, we will show that Sn is the graph that has the fastest cover time random walk among the trees of n vertices in this subsection. To this end, we first present a way to “compare” two graphs having a similar structure