To my wife Zoja and To my parents Nadezda and Todor Rachevi.
To my wife Gabi.
Svetlozar (Zari) Rachev
Preface to Volume II
The second volume of the Mass Transportation Problems is devoted to applications in a variety of fields of applied probability, queueing theory, mathematical economics, risk theory, tomography, and others. In Volume I we encompassed the general mathematical theory of mass transportation, concentrating our attention on: • the general duality theory of the transportation and transshipment problem; • explicit optimality results; • applications to minimal probability metrics, stochastic ordering, approximation and extension problems; • applications to functional analysis and mathematical economics (the Debreu theorem, utility theory, dynamical systems, choice theory, and convex and nonconvex analysis were dicsussed in this context). In Volume II we expand the scope of applications of mass transportation problems. Some of them arise from modifications of the admissible transportation plans. In fact, for applications to mathematical economics it is of interest to consider relaxations of the marginal constraints, such as upper or lower bounds on the supply and demand distributions, or additional constraints like capacity bounds for the transportation plans. In mathematical tomography the basic problem is to reconstruct the multivariate
viii
Preface to Volume II
probability distribution based on some information about the marginal distributions in a certain finite number of directions. This information may be represented by additional constraints on the support functions or distributional moments, or it may be contained in only partial information on the marginals. Thus there is a close relationship between a class of problems in mathematical tomography and the classical theory on moment problems, which again can be viewed as a relaxation on the set of constraints in mass transportation problems. We discuss in detail applications to approximation problems for stochastic processes and to rounding problems based on moment-type characteristics. A particular example will be the approximation of queueing models. The minimal metrics allow us to compare various rounding rules and to determine optimal ones from an asymptotic point of view. An important field of applications of mass transportation problems we shall consider in this second volume is to probabilistic limit theorems. This approach was introduced in the seventies by the Russian school of probability theory, headed by V.M. Zolotarev. By inherent regularity properties of probability metrics defined via certain mass transportation problems, there are streamlined proofs for central limit theorems on Banach spaces yielding sharp quantitative estimates of Berry–Esseen type for the convergence rate. The probability metric approach will be applied to general stable and operator stable limits theorems, martingale-type limit theorems, limit behavior of summability methods, and compound Poisson approximation. A particular application is to the classical problem in mathematical risk theory dealing with sharp approximation of the individual risk model by the collective risk model. The probability metric approach will also be applied to the quantitative asymptotics in rounding problems. A new field of application of probability metrics arising as solutions of mass transportation problems is the analysis of deterministic and stochastic algorithms. This research area is of increasing importance in computer science and various fields of stochastic modeling. Based on regularity properties of probability metrics, a general “contraction” method for the asymptotic analysis of algorithms has been developed. The contraction method has been applied successfully to a variety of search, sorting, and other tree algorithms. Furthermore, the recursive structure in iterated functions systems (image encoding), fractal measures, bootstrap statistics, and time series (ARCH) models has been analyzed by this method. It becomes clear that there are many interesting probabilistic applications of this method to be rigorously developed in the future. In the final chapter we consider applications to stochastic differential equations (SDEs) and to convergence of empirical measures. SDEs will be interpreted as continuous recursive structures. From this point of view we provide a detailed discussion on the approximative solution of nonlinear stochastic differential equations of McKean–Vlasov type by interactive par-
Preface to Volume II
ix
ticle systems with application to the Kac theory of chaos propagation. The probability metrics approach allows us to establish approximation results for various modifications of the diffusion system, some of them of “nontraditional” type. In a general context we establish approximation results for empirical measures and give applications to the approximation of stochastic processes. As final applications we discuss a weak approximation of SDEs of Itˆ o type by a combination of the time discretization methods of Euler and Milshtein with a chance discretization based on the strong invariance (embedding) principle. This approximation is given in terms of minimal Lp -metrics and thereby based on regularity properties of the solutions of the corresponding mass transportation problem.
Preface to Volume I
The subject of this book, mass transportation problems (MTPs), concerns the optimal transfer of masses from one location to another, where the optimality depends upon the context of the problem. Mass transportation problems appear in various forms and in various areas of mathematics and have been formulated at different levels of generality. Whereas the continuous case of the transportation problem may be cast in measure-theoretic terms, the discrete case deals with optimization over generalized transportation polyhedra. Accordingly, work on these problems has developed in several separate and independent directions. The aim of this monograph is to investigate and to develop, in a systematic fashion, the Monge–Kantorovich mass transportation problem (MKP) and the Kantorovich–Rubinstein transshipment problem (KRP). We consider several modifications of these problems known as the MTP with partial knowledge of the marginals and the MTP with additional constraints (MTPA). We also discuss extensively a variety of stochastic applications. In the first volume of Mass Transportation Problems we concentrate on the general mathematical theory of mass transportation. In Volume II we expand the scope of applications of mass transportation problems. In 1781 Gaspard Monge proposed in simple prose a seemingly straightforward problem of optimization. It was destined to have wide ramifications. He began his paper on the theory of “clearings and fillings” as follows: When one must transport soil from one location to another, the custom is to give the name clearing to the volume of the soil that one
xii
Preface to Volume 1 must transport and the name filling (“remblai”) to the space that it must occupy after transfer. Since the cost of transportation of one molecule is, all other things being equal, proportional to its weight and the interval that it must travel, and consequently the total cost of transportation being proportional to the sum of the products of the molecules each multiplied by the interval traversed; given the shape and position, the clearing and the filling, it is not the same for one molecule of the clearing to be moved to one or another spot of the filling. Rather, there is a certain distribution to be made of the molecules from the clearing to the filling, by which the sum of the products of molecules by intervals travelled will be the least possible, and the cost of the total transportation will be a minimum. (Monge, (1781, p. 666)).
In mathematical language Monge proposed the following nonlinear varational problem. Given two sets A, B of equal volume, find an optimal volume-preserving map between them; the optimality is evaluated by a cost function c(x, y) representing the cost per unit mass for transporting material from x ∈ A to y ∈ B. The optimal map is the one that minimizes the total cost of transferring the mass from A to B. Monge considered this problem with cost function equal to the Euclidean distance in IRd : c(x, y) = |x − y|. Monge’s problem turned out to be the prototype for a class of problems arising in various fields such as mathematical economics, functional analysis, probability and statistics, linear and stochastic programming, differential geometry, information theory, cybernetics, and ma trix theory. The optimization function A c(x, t(x)) dx is nonlinear in the transportation function t, and moreover, the set of admissible transportations is a nonconvex set. This explains why it took a long time until even existence results for optimal solutions could be established. The first general existence result was given in 1979 by Sudakov. On the second page of his paper Monge himself had remarked that to obtain a minimum, the intervals traversed by two different molecules should not intersect. This simple observation applied to the discrete case—where there are only a finite number of molecules—leads to a “greedy” algorithm, the so-called northwest corner rule. The totality of mass transferences plans in the discrete case is a polytope that arises in the transportation problem of mathematical programming, where it is treated in specialized form as an assignment problem and in generalized form as a network-flow problem. The northwest corner rule solves transportation problems having a particular structure on the costs and is, moreover, at the heart of many seemingly different problems having an “easy” solution (cf. Hoffman (1961), Barnes and Hoffman (1985), Derigs, Goecke, and Schrader (1986), Hoffman and Veinott (1990), Olkin and Rachev (1991), and Rachev and R¨ uschendorf (1994); see also Burkard, Klinz, and Rudolf (1994) and the references therein). The Academy of Paris offered a prize for the solution of Monge’s problem, which was claimed by the differential geometer P. Appell (1884–1928), who
Preface to Volume I
xiii
established some geometric properties of optimal maps in the plane and in IR3 . But it took a long time until a real breakthrough in the transportation problem came, originating in the seminal 1942 paper of L.V. Kantorovich entitled “On the transfer of masses.” Kantorovich stated the problem in a new, abstract, and in more easily accessible setting and without knowledge of Monge’s work. Kantorovich learned of Monge’s work only later (cf. his 1948 paper). In the Kantorovich formulation of the mass transportation problem (the so-called “continuous” MTP), the initial mass (the clearing) and the final mass (the filling) can be considered as probability measures on a metric space. The essential step in this formulation is the replacement of the class of transportation map by the wider class of generalized transportation plans, that are identifiable with the convex set of all probability measures on the product space with fixed marginals. The difficult nonlinear Monge problem was thereby replaced by a linear optimization problem over an abstract convex set. This made it possible to put this problem in the framework of linear optimization theory and encouraged the development of general duality theory for the solution of the Kantorovich formulation of the transportation problem as the basic tool. Accordingly, these problems and their generalizations will be referred to as Monge–Kantorovich Mass Transportation Problems (MKPs). Kantorovich’s measure theoretic formulation made the problem accessible to various areas of the mathematical sciences and other scientific fields. Kantorovich himself received a Nobel Prize in Economics for related work in mathematical economics.(1) Here is a list of some references in the mathematical sciences: • Functional analysis: Kantorovich and Akilov (1984) • Probability theory: Fr´echet (1951), Cambanis et al. (1976), Dudley (1976, 1989), Kellerer (1984), Rachev (1991c), R¨ uschendorf (1991) • Statistics: Gini (1914, 1965), Hoeffding (1940, 1955), Kemperman (1987), Huber (1981), Bickel and Freedman (1981), R¨ uschendorf (1991) • Linear and stochastic programming: Hoffman (1961), Barnes and Hoffman (1985), Anderson and Nash (1987), Burkard, Klinz and Rudolf (1994) • Information theory and cybernetics: Wasserstein (1969), Gray et al. (1975), Gray and Ornstein (1979), Gray et al. (1980) • Matrix theory: Lorentz (1953), Marcus (1960), Olkin and Pukelsheim (1982), Givens and Shortt (1984) (1) L.V.
Kantorovich together with T.C. Koopmans received the Nobel Memorial Prize in Economic Science in 1975 for “contributions to the theory of optimum allocation of resources”; see Dudley (1989, p. 342).
xiv
Preface to Volume 1
Many practical problems arising in various scientific fields have led mathematicians to solve MKPs: e.g., in • Statistical physics: Tanaka (1978), Dobrushin (1979) • Reliability theory: Barlow and Proschan (1975), Kalashnikov and Rachev (1990), Bene˘s (1985) • Quality control: Jirina and Nedoma (1957) • Transportation: Dantzig and Ferguson (1956) • Econometrics: Shapley and Shubik (1972), Pyatt and Round (1985), Gretsky, Ostroy, and Zame (1992) • Expert systems: Perez and Jirousek (1985) • Project planning: Haneveld (1985) • Optimal models for facility location: Ermoljev, Gaivoronski, and Nedeva (1983) • Allocation policy: Rachev and Taksar (1992) • Quality usage: Rachev, Dimitrov and Khalil (1992) • Queueing theory: Rachev (1989), Anastassiou and Rachev (1992a, 1992b) There are several surveys in the vast literature about MKP, among them Rachev (1984b), Rachev and R¨ uschendorf (1990), Burkard, Klinz, and Rudolf (1994), Cuesta-Albertos, Matr´ an, Rachev, and R¨ uschendorf (1996), and Gangbo and McCann (1996) related to dual solutions and applications of MKP; Shorack and Wellner (1985, Sect. 3.6) on optimal processes; Benes and Stepan (1987, 1991) on extremal mass transportation plans; R¨ uschendorf (1981, 1991, 1991a), Kellerer (1984), Rachev (1991c) on multivariate transportation problems; Dudley (1989) on distances in the space of measures; Talagrand (1992) and Yukich (1991) on matching problems. In recent years, characterizations of the solutions of the Monge–Kantorovich problem have been given in terms of c-subgradients of generalized convex functions defined in terms of the cost functions c(x, y) (cf. Knott and Smith (1984, 1992), Brenier (1987), R¨ uschendorf and Rachev (1990), R¨ uschendorf (1991, 1991a, 1995), Cuesta-Albertos, Matr´ an, Rachev, and R¨ uschendorf (1996), and Gangbo and McCann (1996)). For the case of squared Euclidean costs c(x, y) = |x − y|2 , the generalized convexity property is equivalent to convexity, and c-subgradients are identical to the usual subgradients of convex analysis. From this characterization
Preface to Volume I
xv
a series of explicit solutions of the transportation problem could be established. It also implies that the solutions of the MKP are under continuity assumptions given by mappings. Therefore, the solutions of the “easier” MKP imply as well the existence and characterizations of solutions of the original Monge problem, and so the MKP turns out to be the fundamental formulation of the transportation problem. For this reason, we concentrate in this book on the Kantorovich-type mass tranportation problems. For a discussion of interesting analytic aspects of the Monge problem, we refer to Gangbo and McCann (1996). Another type of MTP appears in probability theory, even if it leaves the framework of probability measures as transportation plans. Its solutions are bounded measures on a product of two spaces with the difference of marginals equal to the difference of two given probability measures. It will be called the Kantorovich–Rubinstein Problem (KRP), since the first results were obtained by Kantorovich and Rubinstein (1958). In its relation to the practical task of mass transportation it is sometimes referred to as the transshipment problem; see Kemperman (1983), and Rachev and Shortt (1990). The KRP has been developed to a great extent in the Russian school of probabilists and functional analysts, in particular by V.L. Levin, A.A. Milyutin, and A.M. Vershik and their students. For metric cost functions the KRP coincides with the corresponding MKP; for general cost functions it can be reduced to the MKP for a corresponding reduced cost function. For the duality theory of the KRP a specific detailed theory with many results that are of value in themselves has been developed with wide-ranging applications to mathematical economics. For a different approach to the KRP as introduced in Dudley (1976) and as further extended in Rachev and Shortt (1990) we refer to the book of Rachev (1991c). A problem related to both MKP and KRP is the Mass Transportation Problem with Partial Knowledge of the Marginals (MTPP), which is expressed by stating finitely many moment conditions. Problems of this type were formulated and extensively studied by Rogosinski (1958), Kemperman (1983), and Kuznezova-Sholpo and Rachev (1989). Barnes and Hoffman (1985) considered mass tranportaion problems with capacity constraints on the admissible transportation plans as an example of Mass Transportation Problems with Additional Constraints (MTPA) (see Rachev (1991b) and Rachev and R¨ uschendorf (1994)). In this book we give an extensive account of the duality theory of the MKP and the KRP, including the known results on explicit constructions and characterizations of optimal solutions. In Chapters 2 and 3 we present important duality theorems for the Monge–Kantorovich problem based on work of H. Kellerer, L. R¨ uschendorf, S.T. Rachev, and D. Ramachandran.
xvi
Preface to Volume 1
In Chapters 4 and 5 we present basically work of V.L. Levin; we analyze measure-theoretic methods for infinite-dimensional linear programs developed in context with the KRP as well as applications to general utility theorems (the Debreu theorem), extension theorems, choice theory, and set-valued dynamical systems.(2) In Chapters 6 and 8 we discuss new material on applications of the MKP and the KRP to the representation of ideal metrics and on various probabilistic approximation and limit theorems. This supplements the earlier results in this direction as described in the book of Rachev (1991) on probability metrics and stochastic models. In particular, we show that probability metrics allow us to find unified proofs for central limit theorems for martingales, (operator) stable limit theorems, and to more specific problems like compound Poisson approximation or rounding problems. Chapter 7, the first chapter in the second volume, is concerned with modifications of the MKP by additional or relaxed constraints. We discuss various types of moment problems and applications to the tomography paradoxon and to the approximation of queueing systems. A wide range of applications of metrics based on the transportation problem has been established in recent years in connection with recursive stochastic equations. We discuss algorithms of informatics (sorting, searching, branching, search trees) as well as applications to the approximation of stochastic differential equations, to the propagation of the chaos property of particle systems with applications to the approximation of nonlinear PDEs, as well as to the rate of convergence of empirical measures, which is of interest for matching problems in Chapters 9 and 10. From the technical point of view, MKPs can be subdivided into the discrete and continuous cases, according to the nature of their basic spaces and to the supports of the initial and the final masses. In the discrete case, the totality of the mass transference plans is the polytope that arises in the transportation problem of mathematical programming. There is, of course, a vast literature on the transportation problem, its specialization to the assignment problem, and its generalization to network flow problems. It turns out, as will be elaborated further in the book, that the northwest corner rule in the discrete case corresponds to a closed form for the solution in the continuous case. Indeed, the discrete analogue of a result known in the continuous case provides a new result in the discrete case; and its simple proof in the discrete case provides a new proof for the continuous case, see Rachev and R¨ uschendorf (1994c) and the references therein. Another approach in the discrete linear case prefers to exploit the special structure of supplies and demands (or clearings and fillings) and permits a particularly simple combinatorial algorithm for finding an optimal solution as developed (2) These two chapters were written following closely the notes kindly provided to us by V.L. Levin.
Preface to Volume I
xvii
by Balinski (1983), Balinski and Russakoff (1984), Balinski (1985, 1986), Goldfarb (1985), Kleinschmidt, Lee, and Schannath (1987), and Burkard, Klinz, and Rudolf (1994). MTPs may be viewed as an analogue and a unifying framework of a problem considered by probabilists at the beginning of the twentieth century: How does one measure the difference between two random quantities? Many specific contributions to the analysis of this problem have been made, including Gini’s (1914) notion of concordance, Kendall’s τ , Spearman’s , the analysis of greatest possible differences by Hoeffding (1940) and others, by Fr´echet (1951, 1957), Robbins (1975), and Lai and Robbins (1976), and the generalizations of these results by Cambanis, Simons, and Stout (1976), R¨ uschendorf (1980), Tchen (1980), and Cambanis and Simons (1982). These (and others) offer piecemeal answers to basic questions that arise from different stochastic models; they give no guidance as to the question of what concept should be used where: There is no general theory underlying the diverse approaches. We refer to Kruskal (1958), Gini (1965), and Rachev (1984b, 1991c). In this book we investigate, develop, and exploit the connections between the discrete and continuous versions of the mass transportation problems (MTP) as well as study systematically the relationships between the methods and results from different versions of the MTP. The MTPs are the basis of many problems related to the question of stability of stochastic models, to the question of whether a proposed model yields a satisfactory approximation to the phenomenon under consideration, and to the problem of approximation of stochastic and deterministic algorithms. It is our belief that MTPs hold great promise in stochastic analysis as well as in mathematical analysis. The MTP is full of connections with geometry, (partial) differential equations, (generalized) convex analysis, moment problems, infinite-dimensional linear programming, measurable choice theory, and extension problems, and it has many open problems. It has a great potential for a series of applications in several scientific fields. This book grew out of joint work and lectures delivered by the authors at the Steklov Mathematical Institute, Universit¨at M¨ unster, Universit¨at Freiburg, the Ecole Polytechnique, SUNY at Stony Brook, and the University of California, Santa Barbara, over many years. Many colleagues provided helpful suggestions after reading parts of the manuscript. All chapters were rewritten several times, and preliminary versions were circulated among friends, who eliminated many inaccuracies and obscurities. We would like to thank H.G. Kellerer, V.L. Levin, M. Balinski, D. Ramachandran, G.A. Anastassiou, M. Maejima, M. Cramer, I. Olkin, M. Gelbrich, W. R¨ omisch, V. Bene˘s, L. Uckelmann, and many other friends and colleagues who encouraged us to complete the work. We are indebted to Mrs. M. Hattenbach and Ms. A. Blessing for their superb typing; the appearance of this monograph owes much to them. We are grateful to the publisher
xviii
Preface to Volume 1
and especially to J. Kimmel for support and patience. We are particularly thankful to J. Gani for his invaluable suggestions concerning improvements of this work, his help with the organization of the material, and his encouragement to continue the project. Finally, we thank the Alexander von Humboldt Foundation for its generous financial support of S.T. Rachev in 1995 and 1996, which made this joint work possible. (3)
(3) The
work of S.T. Rachev was also partially supported by NSF Grants. The joint work of the authors was supported by NATO-Grant CRG900798.
Contents to Volume II
Preface to Volume II
vii
Preface to Volume I
xi
7 Relaxed or Additional Constraints 7.1 Mass Transportation Problem with Relaxed Marginal Constraints . . . . . . . . . . 7.2 Fixed Sum of the Marginals . . . . . . . . . . . . . . 7.3 Mass Transportation Problems with Capacity Constraints . . . . . . . . . . . . . . . 7.4 Local Bounds for the Transportation Plans . . . . . 7.5 Closeness of Measure on a Finite Number of Directions . . . . . . . . . . . 7.6 Moment Problems of Stochastic Processes and Rounding Problems . . . . . . . . . . . . . . . . 7.6.1 Moment Problems and Kantorovich Radius . . . . . 7.6.2 Moment Problems Related to Rounding Proportions 7.6.3 Closeness of Random Processes with Fixed Moment Characteristics . . . . . . . . . . 7.6.4 Approximation of Queueing Systems with Prescribed Moments . . . . . . . . . . . . . . . 7.6.5 Rounding Random Numbers with Fixed Moments . .
1 . . . . . .
2 10
. . . . . .
17 36
. . .
42
. . . . . . . . .
52 54 57
. . .
62
. . . . . .
71 80
xx
Contents to Volume II
8 Probabilistic-Type Limit Theorems
85
8.1
Rate of Convergence in the CLT with Respect to Kantorovich Metric . . . . . . . . . . . .
8.2
Application to Stable Limit Theorems . . . . . . . . . . . 102
8.3
Summability Methods, Compound Poisson Approximation 126
8.4
Operator-Stable Limit Theorems . . . . . . . . . . . . . . 131
8.5
Proofs of the Rate of Convergence Results . . . . . . . . . 153
8.6
Ideal Metrics in the Problem of Rounding . . . . . . . . . 178
9 Mass Transportation Problems and Recursive Stochastic Equations
85
191
9.1
Recursive Algorithms and Contraction of Transformations . . . . . . . . . . . . . . . . . . . . . . 191
9.2
Convergence of Recursive Algorithms . . . . . . . . . . . . 204
9.2.1 Learning Algorithm . . . . . . . . . . . . . . . . . . . . . . 204 9.2.2 Branching-Type Recursion . . . . . . . . . . . . . . . . . . 206 9.2.3 Limiting Distribution of the Collision Resolution Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 9.2.4 Quicksort . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 9.2.5 Limiting Behavior of Random Maxima . . . . . . . . . . . 231 9.2.6 Random Recursion Arising in Probabilistic Modeling: Limit Laws . . . . . . . . . . . . . . . . . . . . . . . . . . 236 9.2.7 Random Recursion Arising in Probabilistic Modeling: Rate of Convergence . . . . . . . . . . . . . . . . . . . . . 248 9.3
Extensions of the Contraction Method . . . . . . . . . . . 254
9.3.1 The Number of Inversions of a Random Permutation . . . 254 9.3.2 The Number of Records . . . . . . . . . . . . . . . . . . . 257 9.3.3 Unsuccessful Searching in Binary Search Trees
. . . . . . 260
9.3.4 Successful Searching in Binary Search Trees . . . . . . . . 263 9.3.5 A Random Search Algorithm . . . . . . . . . . . . . . . . 269 9.3.6 Bucket Algorithm
. . . . . . . . . . . . . . . . . . . . . . 272
10 Stochastic Differential Equations and Empirical Measures 10.1
277
Propagation of Chaos and Contraction of Stochastic Mappings . . . . . . . . . . . . . . . . . . . . 277
10.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 277
Contents to Volume II
xxi
10.1.2 Equations with p-Norm Interacting Drifts . . . . . . . . . 279 10.1.3 A Random Number of Particles
. . . . . . . . . . . . . . 290
10.1.4 pth Mean Interactions in Time: A Non-Markovian Case . 293 10.1.5 Minimal Mean Interactions in Time
. . . . . . . . . . . . 307
10.1.6 Interactions with a Normalized Variation of the Neighbors: Relaxed Lipschitz Conditions . . . . . . . . . . . . . . . . 308 10.2
Rates of Convergence in the Kantorovich Metric . . . . . . . . . . . . . . . . . . 322
10.3
Stochastic Differential Equations . . . . . . . . . . . . . . 332
References
351
Abbreviations
395
Symbols
397
Index
409
Contents to Volume I
Preface to Volume I
vii
Preface to Volume II
xv
1 Introduction
1
1.1
Mass Transportation Problems in Probability Theory . . .
1
1.2
Specially Structured Transportation Problems . . . . . . .
21
1.3
Two Examples of the Interplay Between Continuous and Discrete MTPs . . . . . . . . . . . . . . . . . . . . . .
23
Stochastic Applications . . . . . . . . . . . . . . . . . . . .
27
1.4
2 The Monge–Kantorovich Problem 2.1
57
The Multivariate Monge–Kantorovich Problem: An Introduction . . . . . . . . . . . . . . . . . . . . . . . .
58
2.2
Primal and Dual Monge–Kantorovich Functionals . . . . .
64
2.3
Duality Theorems in a Topological Setting . . . . . . . . .
76
2.4
General Duality Theorem . . . . . . . . . . . . . . . . . .
82
2.5
Duality Theorems with Metric Cost Functions . . . . . . .
86
2.6
Dual Representation for Lp -Minimal Metrics . . . . . . . .
96
xxiv
Contents to Volume I
3 Explicit Results for the Monge–Kantorovich Problem
107
3.1
The One-Dimensional Case . . . . . . . . . . . . . . . . . 107
3.2
The Convex Case . . . . . . . . . . . . . . . . . . . . . . . 112
3.3
The General Case . . . . . . . . . . . . . . . . . . . . . . . 123
3.4
An Extension of the Kantorovich L2 -Minimal Problem . . 132
3.5
Maximum Probability of Sets, Maximum of Sums, and Stochastic Order . . . . . . . . . . . . . . . . . . . . . 144
3.6
Hoeffding–Fr´echet Bounds . . . . . . . . . . . . . . . . . . 151
3.7
Bounds for the Total Transportation Cost . . . . . . . . . 158
4 Duality Theory for Mass Transfer Problems
161
4.1
Duality in the Compact Case . . . . . . . . . . . . . . . . 161
4.2
Cost Functions with Triangle Inequality . . . . . . . . . . 172
4.3
Reduction Theorems . . . . . . . . . . . . . . . . . . . . . 190
4.4
Proofs of the Main Duality Theorems and a Discussion . . 207
4.5
Duality Theorems for Noncompact Spaces . . . . . . . . . 219
4.6
Infinite Linear Programs . . . . . . . . . . . . . . . . . . . 241
4.6.1 Duality Theory for an Abstract Scheme of Infinite-Dimensional Linear Programs and Its Application to the Mass Transfer Problem
. . . . 241
4.6.2 Duality Theorems for the Mass Transfer Problem with Given Marginals . . . . . . . . . . . . . . . . . . . . . 245 4.6.3 Duality Theorem for a Marginal Problem with Additional Constraints of Moment-Type . . . . . . . 251 4.6.4 Duality theorem for a Further Extremal Marginal Problem . . . . . . . . . . . . . . . . . . . . . . . 258 4.6.5 Duality Theorem for a Nontopological Version of the Mass Transfer Problem . . . . . . . . . . . . . . . . 265 5 Applications of the Duality Theory
275
5.1
Mass Transfer Problem with a Smooth Cost Function—Explicit Solution . . . . . . . . . . . . . . 275
5.2
Extension and Approximate Extension Theorems . . . . . 290
5.2.1 The Simplest Extension Theorem the Case X = E(S) and X1 = E(S1 ) . . . . . . . . . . . 290 5.2.2 Approximate Extension Theorems . . . . . . . . . . . . . 292 5.2.3 Extension Theorems . . . . . . . . . . . . . . . . . . . . . 295
Contents to Volume I
xxv
5.2.4 A continuous selection theorem . . . . . . . . . . . . . . . 302 5.3
Approximation Theorems . . . . . . . . . . . . . . . . . . 306
5.4
An Application of the Duality Theory to the Strassen Theorem . . . . . . . . . . . . . . . . . . . 319
5.5
Closed Preorders and Continuous Utility Functions . . . . 322
5.5.1 Statement of the Problem and the Idea of the Duality Approach . . . . . . . . . . . . . . . . . . . 322 5.5.2 Functionally Closed Preorders . . . . . . . . . . . . . . . . 324 5.5.3 Two Generalizations of the Debreu Theorem . . . . . . . . 329 5.5.4 The Case of a Locally Compact Space . . . . . . . . . . . 335 5.5.5 Varying preorders and a universal utility theorem
. . . . 337
5.5.6 Functionally Closed Preorders and Strong Stochastic Dominance . . . . . . . . . . . . . . 341 5.6
Further Applications to Utility Theory . . . . . . . . . . . 344
5.6.1 Preferences That Admit Lipschitz or Continuous Utility Functions . . . . . . . . . . . . . . . 344 5.6.2 Application to Choice Theory in Mathematical Economics . . . . . . . . . . . . . . . . . 352 5.7
Applications to Set-Valued Dynamical Systems . . . . . . 354
5.7.1 Compact-Valued Dynamical Systems: Quasiperiodic Points . . . . . . . . . . . . . . . . . . . . . 354 5.7.2 Compact-Valued Dynamical Systems: Asymptotic Behavior of Trajectories . . . . . . . . . . . . 358 5.7.3 A Dynamic Optimization Problem . . . . . . . . . . . . . 363 5.8
Compensatory Transfers and Action Profiles . . . . . . . . 367
6 Mass Transshipment Problems and Ideal Metrics
371
6.1
Kantorovich–Rubinstein Problems with Constraints . . . . 372
6.2
Constraints on the κth Difference of Marginals . . . . . . 383
6.3
The General Case . . . . . . . . . . . . . . . . . . . . . . . 402
6.4
Minimality of Ideal Metrics . . . . . . . . . . . . . . . . . 414
References
429
Abbreviations
473
Symbols
475
Index
487
7 Modifications of the Monge–Kantorovich Problems: Transportation Problems with Relaxed or Additional Constraints
In this chapter we study modifications of the usual transportation problem by allowing additional constraints on the admissible supply—resp. demand—distributions. In particular, we consider the case that the marginal distribution function of the supply is bounded below by a d.f. F1 , while the marginal d.f. of the demand is bounded above by a d.f. F2 . We also examine transportation plans with constraints of a local type concerning the densities of the marginals, and finally, we study transportation problems with additional moment-type constraints. For the solution of these problems we make use of some methods arising in the theory of marginal and moment problems, duality theory, and stochastic ordering results. The next part is concerned with a solution of the tomography paradox. With respect to some weak metrics, two distributions are getting close if they coincide on an increasing number of directions. In the final sections we review results on the closeness of distributions under given momenttype characteristics and discuss applications to the rounding problem. Most of the results in these sections are contained in Rachev and R¨ uschendorf (1993, 1994c), Levin and Rachev (1989), Klebanov and Rachev (1995), and Anastassiou and Rachev (1992). A survey on related discrete transportation problems is given in Burkard, Klinz, and Rudolf (1994).
2
7. Relaxed or Additional Constraints
7.1 Mass Transportation Problem with Relaxed Marginal Constraints For distribution functions F1 , F2 let F(F1 , F2 ) denote the set of all d.f.s F on IR2 with marginals F1 , F2 (i.e., F (x, ∞) = F1 (x), F (∞, y) = F2 (y)). Then the transportation problem with nonnegative cost function c(x, y), x, y ∈ IR, has the form minimize c(x, y) dF (x, y) over all F ∈ F(F1 , F2 ). (7.1.1) IR2
Usually, in the linear programming setting, F1 is viewed as the supply distribution and F2 as the demand distribution. Clearly, (7.1.1) is an infinitedimensional of the discrete transportation problem: Given ai ≥ analogue m n a = 0, bj ≥ 0, i i=1 j=1 bj , minimize
m n
cij xij , subject to the constraints
(7.1.2)
i=1 j=1 n j=1
xij = ai , 1 ≤ i ≤ m,
m
xij = bj , j = 1, . . . , n, xij ≥ 0, ∀i, j.
i=1
Suppose c(x, y) (resp. (cij )) satisfies the “Monge” conditions, i.e., c is right continuous, and c(x , y ) − c(x, y ) − c(x , y) + c(x, y) ≤ 0
for all x ≥ x, y ≥ y;
(7.1.3)
in the discrete case these conditions are of the form cij + ci+1,j+1 − ci,j+1 − ci+1,j ≤ 0, ∀1 ≤ i < m, 1 ≤ j < n.
(7.1.4)
Then the solution of (7.1.1), (7.1.4) is well known and based on the “northwest corner rule,” which leads to a greedy algorithm; see Hoffman (1961). For (7.1.1) the solution is given by the d.f. F ∗ , F ∗ (x, y) = min{F1 (x), F2 (y)}.
(7.1.5)
F ∗ is the upper Fr´echet bound, see (3.6.2). Recall that the Fr´echet bounds provide the following characterization of F(F1 , F2 ): F ∈ F(F1 , F2 )
if and only if
(7.1.6) ∗
F∗ (x, y) := (F1 (x) + F2 (y) − 1)+ ≤ F (x, y) ≤ F (x, y) (here (·)+ = max(0, ·)); the lower Fr´echet bound yields the solution of the maximization problem corresponding to (7.1.1).
7.1 Mass Transportation Problem with Relaxed Marginal Constraints
3
In terms of random variables an equivalent formulation of the transportation problem is the following: minimize Ec(X, Y ),
subject to FX = F1 , FY = F2 ,
(7.1.7)
where X, Y are random variables on a rich enough (e.g., atomless) probability space (Ω, A, P ). The solutions (7.1.5), resp. (7.1.6), then can be represented as distributions of r.v.s X ∗ , Y ∗ : X ∗ = F1−1 (U ),
Y ∗ = F2−1 (U )
X ∗ = F1−1 (U ),
Y ∗ = F2−1 (1 − U )
(for (7.1.1), (7.1.5)),
(7.1.8)
resp. (for F∗ ),
(7.1.9)
where U is uniformly distributed on (0, 1), and F1−1 (u) = inf{y; F1 (y) ≥ u} is the generalized inverse of F1 . We next consider the mass transportation problem (7.1.1), but with relaxed marginal constraints. For d.f.s F1 , F2 the set H(F1 , F2 ) = {F ; F is a d.f. on IR2 with marginal d.f.s F1 ≤ F1 , F2 ≥ F2 }
(7.1.10)
of all d.f.s F with F1 (x) = F (x, ∞) ≤ F1 (x), ∀x ∈ IR1 , and F2 (y) = F (∞, y) ≥ F2 (y), ∀y ∈ IR1 . We consider the transportation problem: minimize
c(x, y) dF (x, y),
subject to F ∈ H(F1 , F2 ),
(7.1.11)
R2
or, equivalently, minimize Ec(X, Y ),
subject to FX ≤ F1 , FY ≥ F2 .
(7.1.12)
In the discrete case the problem is to minimize
cij xij ,
(7.1.13)
where for some “supplies” s1 , . . . , sn , a1 ≤ s1 , a1 +a2 ≤ s1 +s2 , . . . , and for some demands d1 , . . . , dn , b1 ≥ d1 , b1 +b2 ≥ d1 +d2 , . . . (ai , bi as in (7.1.2)). This describes a production process and a consumption process subject to some priorities (e.g., queueing priorities) with capacities s1 , . . . , sn having the following property: Every remaining free capacity at stage i of the production (resp. consumption) process can be transferred to some of the next stages i + 1, . . . , n.
4
7. Relaxed or Additional Constraints
Theorem 7.1.1 Let the cost function c(x, y) be symmetric in x, y, let c(x, y) satisfy the Monge condition (7.1.3), and let c(x, x) = 0 for all x ∈ IR. Set H ∗ (x, y) = min{F1 (x), max{F1 (y), F2 (y)}},
x, y ∈ IR.
(7.1.14)
Then (a)
H ∗ ∈ H(F1 , F2 ),
(b)
H ∗ solves the relaxed transportation problem (7.1.11),
(c)
(7.1.15)
1 c(x, y) dH (x, y) = c F1−1 (u), min F1−1 (u), F2−1 (u) du. ∗
0
IR2
Remark 7.1.2 Setting G1 (y) = max{F1 (y), F2 (y)}, we see from Theorem 7.1.1 that the relaxed transportation problem (7.1.11) is equivalent to the transportation problem (7.1.1) with marginals F1 , G1 . In terms of random variables the solution can be expressed by the joint distribution of X ∗ = F1−1 (U ) and −1 −1 Y ∗ = G−1 1 (U ) = min F1 (U ), F2 (U )
(7.1.16)
(cf. (7.1.8)). Proof: The Monge condition implies that we can view the function −c(x, y) as a “distribution function” corresponding to a nonnegative measure µc on IR2 . Let X, Y be any real r.v.s, and for x, y ∈ IR1 set x ∨ y = max{x, y}, x ∧ y = min{x, y}. Theorem 7.1.1 is a consequence of the following two claims. Claim 7.1.3 (Cambanis, Simons, and Stout (1976); see also Dall’Aglio (1956) for the special case c(x, y) = |x − y|p ) (P (X < x ∧ y, Y ≥ x ∨ y) 2Ec(X, Y ) = IR2
+P (X ≥ x ∨ y, Y < x ∧ y))µc ( dx, dy).
(7.1.17)
For the proof of Claim 7.1.3 define the function f (x, y, w) : IR2 × Ω → IR by ⎧ ⎨ 1 if X(w) < x, y ≤ Y (w) or Y (w) < x, y ≤ X(w), f (x, y, w) = ⎩ 0 otherwise.
7.1 Mass Transportation Problem with Relaxed Marginal Constraints
5
Using Fubini’s theorem, Ew f (x, y, w)µc ( dx, dy) = (Ew f (x, y, w))µc ( dx, dy). (7.1.18) IR2
IR2
Next, the symmetry of c(x, y) and c(x, x) = 0 yield f (x, y, w) dµc
(7.1.19)
IR2
= − [c (Y (w), Y (w)) + c (X(w), X(w)) − c (X(w), Y (w)) −c (Y (w), X(w))] = 2c (X(w), Y (w)) . Clearly, Ew f (x, y, w)
(7.1.20)
= P (X < x ∧ y, Y ≥ x ∨ y) + P (X ≥ x ∨ y, Y < x ∧ y). Combining (7.1.18), (7.1.19), and (7.1.20) we obtain (7.1.17). Claim 7.1.4 Define X ∗ = F1−1 (U ), Y ∗ = min F1−1 (U ), F2−1 (U ) . Then Ec(X ∗ , Y ∗ ) = min (Ec(X, Y ); FX ≤ F1 , FY ≥ F2 ) ,
(7.1.21)
and the value of the expectation in (7.1.21) is given by 1 Ec(X ∗ , Y ∗ ) = max (0, F2 ((x ∧ y)−) − F1 ((x ∨ y)−)) µc ( dx, dy) 2 IR2
1 c F1−1 (t), min F1−1 (t), F2−1 (t) dt. =
(7.1.22)
0
For the proof of Claim 7.1.4 let X, Y be any r.v.s with d.f.s FX ≤ F1 , FY ≥ F2 . Using Claim 7.1.3 we obtain P (X ≥ x ∨ y, Y < x ∧ y)µc ( dx, dy) (7.1.23) 2Ec(X, Y ) ≥ IR2
(P (Y < x ∧ y) − P (X < x ∨ y, Y < x ∧ y)) µc ( dx, dy)
= IR2
≥
(P (Y < x ∧ y) − min {P (X < x ∨ y), IR2
6
7. Relaxed or Additional Constraints
P (Y < x ∧ y)}) µc ( dx, dy) (P (Y < x ∧ y) − P (X < x ∨ y))+ µc ( dx, dy)
= IR2
≥
(F2 ((x ∧ y)−) − F1 ((x ∨ y)−))+ µc ( dx, dy). IR2
∗ Next, we check that the lower bound −1 in (7.1.23) is attained for X ∗ = −1 −1 ∗ F1 (U ), Y = min F1 (U ), F2 (U ) . In fact, by Claim 7.1.3 using X ≥ Y ∗ and {U < F2 (z)} = F2−1 (U ) < z a.s. we get
2Ec(X ∗ , Y ∗ ) = (P (X ∗ ≥ x ∨ y, Y ∗ < x ∧ y) IR2
(7.1.24)
+P (X ∗ < x ∧ y, Y ∗ ≥ x ∨ y)) µc ( dx, dy) P (X ∗ ≥ x ∨ y, Y ∗ < x ∧ y)µc ( dx, dy)
= IR2
= IR2
=
P F1−1 (U ) ≥ x ∨ y, min F1−1 (U ), F2−1 (U ) < x ∧ y µc ( dx, dy) P F1−1 (U ) ≥ x ∨ y, F2−1 (U ) < x ∧ y µc ( dx, dy)
IR2
P (U ≥ F1 (x ∨ y), U < F2 (x ∧ y))+ µc ( dx, dy)
= IR2
=
(F2 ((x ∧ y)−) − F1 ((x ∨ y)−))+ µc ( dx, dy). IR2
Obviously, F(X ∗ ,Y ∗ ) = H ∗ ∈ H(F1 , F2 ), and the proof of Theorem 7.1.1 is complete. 2
Remark 7.1.5 The optimal coupling (7.1.16) leads to the following “greedy” algorithm for solving the finite discrete transportation problem with relaxed side conditions: minimize
n n
cij xij
i=1 j=1
subject to:
xij ≥ 0,
(7.1.25)
7.1 Mass Transportation Problem with Relaxed Marginal Constraints j n
xrs ≥
s=1 r=1 i n
j
bs =: Gj ,
1 ≤ j ≤ n,
ar =: Fi ,
1 ≤ i ≤ n,
7
s=1
xrs ≤
r=1 s=1
i r=1
n where the sum of the “demands” s=1 bs equals the sum of the “supplies” n r=1 ar , assuming that (cij ) are symmetric, cii = 0, and c satisfies the Monge condition (7.1.4). Set =
Hi δ1
max(Fi , Gi ), 1 ≤ i ≤ n,
(7.1.26)
= H1 , δi+1 = Hi+1 − Hi , 1 ≤ i ≤ n − 1.
Then (7.1.25) is equivalent to the standard transportation problem (7.1.2) with side conditions (ai ), (δi ). 7In the following example we compare the solution of problem (7.1.25) with inequality constraints with the “greedy” solution of the standard transportation problem with equality constraints (7.1.2). For the problem with inequality constraints we first calculate the new artificial demands δj as in (7.1.26) and then apply the northwest corner rule. supply a1
Example 7.1.6 yij xij
20 10
10
20 20
20 10
10 20 20 10 10 10 10
demand b1
10
30
10
40
0
10
10
40
50
90
90
100
Hj = Fj ∨ Gj
20
40
60
90
90
100
δ 1 = H1 ,
20
20
20
30
0
10
Gj =
j s=1
bs
δj+1 = Hj+1 − Hj
Fi =
i r=1
20
20
0
20
40
60
20
80
10
90
10
100
ar
“artificial” demands
xij = solution of the standard transportation problem (7.1.2), using the classical northwest corner yij = solution of the transportation problem with relaxed side conditions
8
7. Relaxed or Additional Constraints
We next extend the solution to the nonsymmetric case. We relax the symmetry condition, assuming that for any x, y the functions c(x, ·), c(·, y) are unimodal: c(x, y1 ) ≤ c(x, y2 ) if x ≤ y1 ≤ y2 or y2 ≤ y1 ≤ x,
(7.1.27)
c(x1 , y) ≤ c(x2 , y) if x2 ≤ x1 ≤ y or y ≤ x1 ≤ x2 . Theorem 7.1.7 If c(x, x) = 0 for all x ∈ IR and c satisfies the Monge condition and the unimodality condtion (7.1.27), then the relaxed transportation problem minimize Ec(X, Y )
subject to FX ≥ F1 , FY ≤ F2
has a solution, given by the coupling X ∗ = F1−1 (U ), Y ∗ = max F1−1 (U ), F2−1 (U )
(7.1.28)
(7.1.29)
with joint distribution
FX ∗ ,Y ∗ (x, y) = min F1 (x), min (F1 (y), F2 (y)) , and the optimal value is given by 1 Ec(X , Y ) = c F1−1 (u), max F1−1 (u), F2−1 (u) du. ∗
∗
0
Proof: Let X, Y be r.v.s with FX ≥ F1 , FY ≤ F2 . Then by (7.1.8), −1 Ec(X, Y ) ≥ Ec FX (U ), FY−1 (U ) . (7.1.30) −1 −1 −1 −1 −1 Let G(y) X (y), FY (y)). Then FX ≤ F1 , FY ≥ F2 , and G = −1= min(F −1 max FX , FY . We now need the following
Claim 7.1.8 1 1 −1 −1 c FX (u), FY−1 (u) du ≥ c FX (u), G−1 (u) du. 0
(7.1.31)
0
−1 (u), y1 = To show Claim 7.1.8 set (for a fixed u ∈ (0, 1)), x = FX −1 −1 −1 ∨ FY (u) = G (u), and y2 = FY (u).
−1 (u) FX
Case 1: x < y2 . In this case, x ≤ y1 ≤ y2 , and therefore, the unimodality condition (7.1.27) implies c(x, y2 ) ≥ c(x, y1 ). Case 2: y2 ≤ x. In this case, y1 = x, and therefore, y2 ≤ y1 = x. Again by the unimodality condition, c(x, y2 ) ≥ c(x, y1 ). So Claim 7.1.8 holds.
7.1 Mass Transportation Problem with Relaxed Marginal Constraints
9
Claim 7.1.9 The following bound holds for every coupling (X, Y ) : 1 −1 −1 c FX (u), FY−1 ∨ FX (u) du
(7.1.32)
0
1 ≥ c F1−1 (u), F2−1 (u) ∨ F1−1 (u) du. 0 −1 (u), x 2 = FY−1 (u), x1 = F1−1 (u), x2 = For the proof define x 1 = FX for a fixed u. Then x 1 ≤ x1 , x2 ≤ x 2 .
F2−1 (u)
2 , then x 1 ≤ x 2 ∨ x2 ≤ x 2 . If x 1 < x if
x 1 ≥ x 2 , then x 1 = x 1 ∨ x2 ≥ x 2 .
(7.1.33)
From (7.1.33) we obtain Claim 7.1.10 c( x1 , x 1 ∨ x2 ) ≥ c(x1 , x1 ∨ x2 ).
(7.1.34)
For the proof of Claim 7.1.10 we use the relation x1 ≥ x 1 . 1 . Then c( x1 , x2 ) = c( x1 , x 1 ∨ x2 ) ≥ c(x1 , x2 ) = Case 1: x2 > x1 > x c(x1 , x1 ∨ x2 ) by the unimodality condition. Case 2: (a) x1 ≥ x2 ≥ x 1 . Then, trivially, c( x1 , x2 ) = c( x1 , x2 ∨ x 1 ) ≥ c(x1 , x1 ∨ x2 ) = c(x1 , x1 ) = 0. 1 ≥ x2 . Then again, c( x1 , x 1 ) = c( x1 , x 1 ∨x2 ) ≥ c(x1 , x1 ∨x2 ) = (b) x1 ≥ x c(x1 , x1 ) = 0. Claims 7.1.8, 7.1.9, and 7.1.10 imply (7.1.28).
2
Remark 7.1.11 (a) The unimodality assumption (7.1.27) is quite natural from an application point of view. Note that the transportation problem in Theorem 7.1.7 is the same as in Theorem 7.1.1 (where only the indices 1,2 have been changed). We used this to demonstrate that the solution F ∗ is not unique. Without the symmetry, resp. the unimodality condition, the solution may differ substantially. Given a right continuous function f = f (y) ≥ 0, consider the cost function c(x, y) = f (y). Then
10
7. Relaxed or Additional Constraints
c satisfies the Monge condition, and so (7.1.28) is equivalent to the following problem: minimize
f (y) dFY (y)
subject to FY ≤ F2 .
(7.1.35)
Equivalently, we are seeking a d.f. F2 ≤ F2 such that the distribution of f with respect to F2 has a minimal first moment. Obviously, the solution (7.1.31) of Theorem 7.1.7 is not a solution of (7.1.35). (b) In the proof of Theorem 7.1.7, the assumption c(x, x) = 0 can be replaced with a weaker one, c(x, x) ≤ c(x, y) ∧ c(y, x),
for all x, y ∈ IR.
(7.1.36)
7.2 Mass Transportation Problem with Fixed Sum (Difference) of the Marginals and with Stochastically Ordered Marginals Consider a flow in a network with n nodes, i = 1, . . . , n, and let xij be the flow from node i to node j. Assume that for all nodes k the value of x + is fixed and equal to hk . As motivation, suppose ai = ik j xkj in n x , b = i k=1 xki to be the amount of workload corresponding to k=1 ik the outflow, resp. to the inflow, in node i. Assume that the total work capacity at node i is given by hi (in a certain time period). Then every admissible flow (xij ) should satisfy the condition hi = ai + bi ,
1 ≤ i ≤ n.
(7.2.1)
k k k Set A(k) = i=1 ai , B(k) = i=1 bi , and H(k) = i=1 hi . Then hk = A(k) + B(k) − (A(k − 1) + B(k − 1)), and (7.2.1) is equivalent to H(k) = A(k) + B(k),
1 ≤ k ≤ n.
(7.2.2)
Let cij be the transportation cost of a unit from node i to node j. Then the problem is to minimize the total cost cij xij subject to the admissibility condition (7.2.1) and xij ≥ 0. The general formulation of this problem is the following. For two d.f.s A, B define G(x) = 12 (A(x) + B(x)). For a given cost function c(x, y), c(x, y) dF (x, y), subject to F ∈ FA+B . (7.2.3) minimize IR2
7.2 Fixed Sum of the Marginals
11
Here FA+B is the set of all d.f.s F (x, y) with marginal d.f.s F1 , F2 satisfying F1 (x) + F2 (x) = A(x) + B(x). Consider next the special case c(x, y) = |x − y|. Let X, Y be real r.v.s with joint d.f. F . Then by the triangle inequality, E|X − Y | ≤
inf (E|X − a| + E|Y − a|).
a∈IR1
(7.2.4)
Since E|X − a| + E|Y − a| = |x − a| d(FX + FY )(x) depends only on the sum of the marginals, (7.2.3) is the best possible improvement of (7.2.4), provided that the sum of the marginal FX + FY is known. Rachev (1984d) showed that sup {E|X − Y |p ; FX + FY = A + B} = 1 −1 G (t) − G−1 (1 − t)p dt, p ≥ 1.
(7.2.5)
0
The following result gives an explicit solution of the general problem in (7.2.3). Proposition 7.2.1 Suppose c ≥ 0 is symmetric and satisfies the Monge condition: c(x , y ) − c(x, y ) − c(x , y) + c(x, y) ≤ 0
∀x ≥ x, y ≥ y.
(7.2.6)
1 = c(G−1 (u), G−1 (u)) du,
(7.2.7)
Then inf
c(x, y) dF (x, y); F ∈ FA+B
0
and sup
c(x, y) dF (x, y); F ∈ FA+B
=
1 c(G−1 (u), G−1 (1−u)) du.(7.2.8) 0
The corresponding optimal pairs of r.v.s (couplings) are given by (G−1 (U ), G−1 (U )), resp. (G−1 (U ), G−1 (1 − U )). Proof: Since c is symmetric, we obtain for any F ∈ FA+B , 1 c(x, y) dF (x, y) = (c(x, y) + c(y, x)) dF (x, y) 2 F (x, y) + F (y, x) . = c(x, y) d 2
12
7. Relaxed or Additional Constraints
(y,x) On the other hand, Fs (x, y) = F (x,y)+F ∈ F(G, G). Consequently, we 2 obtain (7.2.7), (7.2.8) by making use of (7.1.8) and (7.1.9) with F1 = F2 = G. 2
We have the following analogue of the above proposition for nonsymmetric cost functions. Proposition 7.2.2 If c(x, y) satisfies the Monge condition (7.2.6) and furthermore, x1 ≤ y ≤ x2 implies that c(x1 , x2 ) ≥ c(y, y), then
inf
c(x, y) dF (x, y); F ∈ FA+B
1 c(G−1 (u), G−1 (u)) du. =
(7.2.9)
0
Proof: Applying the Monge conditions for every X, Y with FX,Y ∈ FA+B , −1 (U ), FY−1 (U )). Since FX (x)+FY (x) = 2G(x), it follows Ec(X, Y ) ≥ Ec(FX −1 −1 ∧ FY−1 ≤ G−1 ≤ FX ∨ that FX ∧ FY ≤ G ≤ FX ∨ FY , and therefore, FX −1 −1 −1 −1 −1 FY . Consequently, we have c(FX (U ), FY (U )) ≥ c(G (U ), G (U )), which proves (7.2.9). 2 Remark 7.2.3 The marginals of the class FA+B have largest and smallest elements, defined by ⎧ ⎨ 2 G(x), x < x0 , F1∗ (x) = ⎩ 1, x≥x , 0
and
⎧ ⎨ 2 G(x) − 1, F2∗ (x) = ⎩ 1,
x < x0 , x ≥ x0 ,
with x0 := inf{y; 2G(y) ≥ 1}. Note that there is no smallest d.f. in FA+B . To show this let F1 (x), F2 (x) be the marginal d.f.s of the smallest elements F ∈ FA+B and let G1 , G2 be d.f.s such that G1 (x) + G2 (x) = 2G(x). If the lower Fr´echet bounds satisfy (F1 (x) + F2 (y) − 1)+ ≤ (G1 (x) + G2 (y) − 1)+ , then F1 ≤ G1 and F2 ≤ G2 , which implies that F1 = G1 , F2 = G2 . In particular, this implies that (G−1 (U ), G−1 (1 − U )) is in the general nonsymmetric case no longer a solution to the problem of maximizing c(x, y) dF (x, y) over the class FA+B . For example, let G be the d.f. of 1 4
4 i=1
ε(i) . Then P1 = P (G ∗ −1
while P2 = P ((F1 )
−1
(U ),G−1 (1−U ))
(U ),(F2∗ )−1 (1−U ))
=
= 14 (ε(1,4) + ε(2,3) + ε(3,2) + ε(4,1) ), 1 2 (ε(1,4)
+ ε(2,3) ). For c1 (x, y) =
7.2 Fixed Sum of the Marginals
13
1(−∞,(3,2)] (x, y), we have EP1 c1 = 14 , EP2 c1 = 0, while for c2 = 1[(2,3),∞) , we have EP1 c1 = 14 , EP2 c2 = 12 . Note that both functions, −c1 , −c2 , are Monge functions (but are not unimodal). We next consider the case where in the network example we fix the total outflow minus the inflow of each node. This problem is known in the literature as the minimal network flow problem (cf. for example Barnes and Hoffman (1985, Section 9) or Anderson and Nash (1987)). Assume that the outflow minus the inflow of each node is fixed; i.e., the following Kirchhoff equations hold xik − xki = ai − bi = hi for all i, k
k
or equivalently, H(k) = A(k) − B(k), with A(k) =
k j=1
aj , B(k) =
1 ≤ k ≤ n, k
bj , and H(k) =
j=1
(7.2.10) k
hj . Consider now the
j=1
general case: Let A, B be distribution functions and let FA−B be the set of all “generalized” d.f.s of finite measures on IR2 with marginals F1 , F2 satisfying F1 − F2 = A − B. We consider the following transportation problem: (7.2.11) minimize c(x, y) dF (x, y) subject to F ∈ FA−B , with c(x, y) satisfying the Monge condition (7.2.6). To solve (7.2.11) we make use of the following dual representation (cf. (6.1.23)): inf c(x, y) dF (x, y); F ∈ FA−B (7.2.12) = sup f d(A − B)(x); f (x) − f (y) ≤ c(x, y), ∀x, y . We first consider a particular type of cost function. Proposition 7.2.4 Let c(x, y) = |x−y| max(1, h(|x−a|), h(|y−a|)), where h is a monotonically nondecreasing function on IR+ . Then (7.2.13) inf c(x, y) dF (x, y); F ∈ FA−B = max(1, h(|x − a|))|A − B|(x) dx, provided that h(|x − a|) is locally integrable.
14
7. Relaxed or Additional Constraints
Proof: We first note that the duality constraints condition f (x) − f (y) ≤ c(x, y), for all x, y, holds if and only if f is absolutely continuous and moreover, |f (x)| ≤ max(1, h(|x − y|)) a.s. Consequently, by the dual representation (7.2.12), we obtain inf c(x, y) dF (x, y); F ∈ FA−B = sup f d(A − B)(x); |f | ≤ max(1, h(|x − a|)), ∀x = sup f (x) d(A − B)(x) dx; |f | ≤ max(1, h(|x − a|)), ∀x = max(1, h(|x − a|))|A − B|(x) dx. 2 To handle the general case set c(x, y) = |x − y|ζ(x, y)
i.e., ζ(x, y) =
c(x, y) |x − y|
.
(7.2.14)
Theorem 7.2.5 Assume that for any x < t < y, ζ(t, t) ≤ ζ(x, y), ζ(x, y) = ζ(y, x). Moreover, let ζ(x, y) be right continuous in y, and also assume that t → ζ(t, t) is locally bounded. Then the optimal value in the minimization problem (7.2.11) is equal to inf c(x, y) dF (x, y); F ∈ FA−B = ζ(t, t)|A − B|(t) dt. (7.2.15) Proof: Let F = {f ; f (x)−f (y) ≤ c(x, y), ∀x, y}, and let F ∗ = {f absolutely continuous and |f (t)| ≤ ζ(t, t), ∀t}. Then F ⊂ F ∗ , and for f ∈ F (y) (y) ≤ ζ(x, y), and therefore, lim f (x)−f ≤ ζ(x, x). Also, we have f (x)−f |x−y| |x−y| y→x
f (x) − f (y) f (y) − f (x) = − lim ≥ − lim ζ(y, x) = −ζ(x, x). |x − y| |x − y| y→x lim
Since f is locally Lipschitz, it is absolutely continuous, so the inequalities above imply that |f (t)| ≤ ζ(t, t) a.s. If, conversely, f ∈ F ∗ , then y y f (x) − f (y) = f (t) dt, and therefore, |f (x) − f (y)| ≤ |f (t)| dt ≤ y
x
x
ζ(t, t) dt ≤ |x − y|ζ(x, y) = c(x, y). The dual representation (7.2.12) again
x
implies (7.2.13) (by the same arguments as in the proof of Proposition 7.2.4). 2
7.2 Fixed Sum of the Marginals
15
Next, we consider the following transportation problem with stochastically ordered marginals posed by Rogers (1992). Let F, G be real distribution functions, F ≤st G; here as usual ≤st stands for the stochastic order. Let C := {(x, y) ∈ IR2 ; x ≤ y}, and let MC (F, G) := M (F, G) ∩ {µ ∈ M 1 (IR2 , IB2 ); µ(C) = 1}
(7.2.16)
be the set of all measures with marginals F, G that are concentrated on the order cone C. The problem is to determine, for a given strictly convex function ϕ, the bound sup ϕ(x − y)µ( dx, dy); µ ∈ MC (F, G) . (7.2.17) The motivation for problem (7.2.17) is to get a good monotone coupling of random walks (Sn ), (Sn ) with S0 = x ≥ X0 = 0, Sn ≥ Sn for all n, and Sn = Sn for all large enough n. Without the order restriction, a solution of (7.2.17) is given by the random variables X = F −1 (U ), Y = G−1 (1 − U ) for a uniform (0, 1) distributed r.v. U. It is intuitively clear that a solution of (7.2.17) should concentrate as much mass on the diagonal as possible. This is indeed true. Theorem 7.2.6 (Rogers (1992)) Each solution (X, Y ) of (7.2.17) has the property that P (X = Y ) = |F ∧ G| = f ∧ g dm, (7.2.18) when F = f m, G = gm. There exists a solution of (7.2.17). We next characterize the optimal solutions by an order-type relation. Theorem 7.2.7 Let X, Y be r.v.s with d.f.s F, G and X ≤ Y a.s. Then (X, Y ) defines a solution of (7.2.17) iff X(ω) < X(ω ) ≤ Y (ω) ≤ Y (ω )
implies Y (ω ) = Y (ω)
(7.2.19)
a.s. (for (ω, ω ) and with respect to the product measure). Proof: If (X, Y ) is an optimal admissible coupling and if on a set of pairs (ω, ω ) with positive measure X(ω) < X(ω ) ≤ Y (ω) < Y (ω ) holds, then let us define Y (ω ) := Y (ω), Y (ω) := Y (ω ) and set Y = Y otherwise. Then Y has d.f. G and Eϕ(X − Y ) > Eϕ(X − Y ) because ϕ is strictly convex. Since there is essentially up to simultaneous rearrangements only one pair of r.v.s X, Y with d.f.s F, G satisfying the order relation (7.2.18), the opposite direction follows from the first part of the proof. 2
16
7. Relaxed or Additional Constraints
In terms of measures µ ∈ MC (F, G), the characterization of optimality of µ in (7.2.19) can be formulated as µ ⊗ µ ({(x1 , y1 , x2 , y2 ); x1 < x2 ≤ y1 < y2 }) = 0.
(7.2.20)
We remark that the characterization of optimal pairs in (7.2.19), resp. (7.2.20), implies the “maximal concentration on the diagonal” property in (7.2.18). For finite discrete distributions one can explicitly construct optimal pairs with the ordering property given in (7.2.19). We consider at nfirst the case of equiprobable atoms in each distribution. So let µ1 = n1 i=1 εai , µ2 = n 1 i=1 εbi be the measures corresponding to F, G, where a1 ≤ · · · ≤ an , n b1 ≤ · · · ≤ bn , and ai ≤ bi for all i. Problem (7.2.17) is equivalent to the following problem: Find a permutation π ∈ Υn such that n
ϕ(bi − aπ(i) ) is maximal.
(7.2.21)
i=1
Here, the maximum is considered over all permuations π ∈ Υn such that aπ(i) ≤ bi , 1 ≤ i ≤ n. Permutations with this property are called admissible permutations. An optimal admissible permutation is essentially unique (up to indices with equal values of ai ), and it is given in the following theorem. Theorem 7.2.8 Define π ∗ ∈ Υn inductively: π ∗ (1)
:= max{k ≤ n; ak ≤ b1 } (7.2.22) ∗ ∗ π (k) := max{ ≤ n; ∈ {π (1), . . . , π (k − 1)}, a ≤ bk }, 2 ≤ k ≤ n. ∗
Then π ∗ ∈ Υ is the optimal admissible permutation. Proof: Define on Ω = {1, . . . , n} (supplied with the uniform distribution P ) random variables X(i) := ai and Y (i) := bπ∗ (i) , 1 ≤ i ≤ n. Then X ≤ Y , since π ∗ is admissible and X, Y satisfy the order relation (7.2.19). Therefore, they are optimal couplings. Equivalently, π ∗ is the optimal admissible permutation. 2 It is clear from the construction in Theorem (7.2.8) that up to a simultaneous permutation of the probability space, an optimal pair of r.v.s is essentially unique. Remark 7.2.9 Theorem 7.2.8 can be extended to the case that µ1 = n n p ε , µ = q 2 i=1 i ai i=1 i εbi with rational pi , qi , by representing pi , qi in the formal equiprobable case. By an approximation argument—as given in Rogers (1992)—one can approximate the optimal couplings for F, G with
7.3 Mass Transportation Problems with Capacity Constraints
17
couplings having compact support. The general case then can be approximated via the ordering criterion (7.2.19) using a truncation technique. Thus, applying Theorem 7.2.8, we are able to construct explicit approximate solutions in the general case.
7.3 Mass Transportation Problems with Capacity Constraints In this section we obtain explicit solutions of Monge–Kantorovich mass transportation problems with capacity constraints. The Hoeffding–Fr´echet inequality is extended for bivariate distribution functions having fixed marginals and satisfying additional constraints. In the discrete case, our results lead to “greedy” algorithms similar to the classical northwest corner rule. Let us start with recalling the abstract version of the MKP: Given two Borel measures µ and ν on a separable metric space S with equal total mass λ = µ(S) = ν(S) < ∞ and a measurable cost function c on S × S, find Lc (µ, ν) = inf c(x, y)P ( dx, dy), (7.3.1) Uc (µ, ν) = sup c(x, y)P ( dx, dy), (7.3.2) where the infimum and supremum are taken over all Borel measures P on S × S having projections (marginals) P (· × S) = µ(·),
P (S × ·) = ν(·).
(7.3.3)
As shown in Section 3.1, the explicit solutions of MKP are based on the Hoeffding–Fr´echet inequality (referred to as upper and lower Fr´echet bounds): max(0, F µ (x) + F ν (y) − λ) ≤ F P (x, y) ≤ min(F µ (x), F ν (y)),
(7.3.4)
for any P on IR2 that satisfies (7.3.3) with S = IR. (In (7.3.4) and in the sequel, F P stands for the distribution function of P .) If c is a lattice superadditive (equivalently, −c is a Monge function): c(x , y ) + c(x, y) ≥ c(x , y) + c(x, y )
for all x ≥ x, y ≥ y,
(7.3.5)
then under mild moment conditions on µ and ν the explicit values of Lc and Uc were given in Section 3.1. In this section we consider two marginal problems with additional constraints on the joint distribution functions. Suppose µ and ν are two nonnegative Borel measures on IR, µ(IR) = ν(IR) = λ < ∞. Suppose c : IR2 →
18
7. Relaxed or Additional Constraints
IR is a right-continuous Monge function generating a nonnegative measure on IR2 . Let σ be a nonnegative bounded Borel measure on IR2 . (Note that the total mass of σ may be different from λ.) Problem I.
Find maximum c(x, y)P ( dx, dy)
(7.3.6)
IR2
subject to the constraints P is a nonnegative Borel measures on IR2
(7.3.7)
with marginals µ and ν, and P ((−∞, x] × (−∞, y]) ≤ σ((−∞, x] × (−∞, y]) for all x, y ∈ IR.
(7.3.8)
Problem II.
Find minimum c(x, y)P ( dx, dy)
(7.3.9)
IR2
subject to (7.3.7) and P ((−∞, x] × [y, ∞)) ≤ σ((−∞, x] × [y, ∞))
for all x, y ∈ IR. (7.3.10)
Problem I with discrete µ and ν was studied by Barnes and Hoffman (1985). Olkin and Rachev (1990) extended their results by completing the characterization of the “optimal feasible” P; i.e., P satisfies (7.3.7), (7.3.8) and attains the maximum in (7.3.6). This method is extended to solve Problem II as well. We start with a refinement of the Fr´echet bounds (7.3.4). We shall do this by determining the exact bounds for a d.f. F P (x, y) with marginals F µ and F ν assuming that P satisfies the constraint (7.3.8) or (7.3.10). Then we shall apply the extended Fr´echet bounds to solve Problems I and II. Whereas in the discrete case the solution of Problem I leads to the Barnes– Hoffman greedy algorithm, the solution of Problem II implies a new greedy algorithm for a transportation problem with capacity constraints (7.3.10). We begin with some notation. For two nonnegative Borel measures µ and ν on IR with equal total mass λ denote by M (µ, ν) the set of all nonnegative Borel measures on IR2 with projections µ and ν. Without loss of generality set λ = 1. Given a nonatomic probability space, the set F(A, B) of joint d.f.s F (x, y) = FX,Y (x, y) = P (X ≤ x, Y ≤ y) with fixed
7.3 Mass Transportation Problems with Capacity Constraints
19
marginals FX = A and FY = B is the set of d.f.s of the probability laws in M (µ, ν). Thus, the Fr´echet bounds (7.3.4) can be rewritten as max
F (x, y) = F ∗ (x, y) :=
min(A(x), B(y)),
(7.3.11)
max
G(x, y) = G∗ (x, y) :=
min(A(x), B(y)),
(7.3.12)
F ∈F (A,B) F ∈F (A,B)
where B(y) := ν([y, ∞)) and G(x, y) := GX,Y (x, y) := P (X ≤ x, Y ≥ y). Clearly, the laws corresponding to F ∗ and G∗ are in M (µ, ν). Furthermore, given a nonnegative bounded Borel measure σ on IR2 , set F σ (x, y) := σ((−∞, x] × (−∞, y]), (7.3.13) σ G (x, y) := σ((−∞, x] × [y, ∞)) F(A, B, F σ ) := {F ∈ F(A, B); F ≤ F σ }, G(A, B, Gσ ) := {GX,Y ; FX,Y ∈ F(A, B), GX,Y ≤ Gσ } . Our objective in the next two theorems is to extend the Fr´echet bounds; we shall characterize the bounds max
F (x, y) =: F(x, y),
x, y ∈ IR,
(7.3.14)
max
y), G(x, y) =: G(x,
x, y ∈ IR,
(7.3.15)
F ∈F (A,B,F σ ) G∈G(A,B,Gσ )
and shall examine the conditions implying F ∈ F(A, B, F σ )
∈ G(A, B, Gσ ). and G
(7.3.16)
Theorem 7.3.1 If F σ (x, y) ≥ max(0, A(x) + B(y) − 1),
(7.3.17)
then the maximum in (7.3.14) is attained: F(x, y) = =
inf {F σ (t, s) + µ((t, x]) + ν((s, y])}
t≤x s≤y
(7.3.18)
inf {F σ (t, s) + µ((t, x]) + ν((s, y])} ∧ (A(x) ∧ B(y)),
t≤x s≤y
and F ∈ F(A, B, F σ ), where ∧ := min. Remark 7.3.2 Condition (7.3.17) is necessary and sufficient for F(A, B, F σ ) = Ø, cf. Fr´echet (1951), Kellerer (1964). Remark 7.3.3 The second equality in (7.3.18) follows from the fact that F σ (t, s) = 0 for t = −∞ or s = −∞.
20
7. Relaxed or Additional Constraints
Remark 7.3.4 From (7.3.18) F is not greater than the Hoeffding–Fr´echet upper bound F ∗ (7.3.11). Remark 7.3.5 By (7.3.4) the maximum in (7.3.11) is attained for the pair X ∗ = A− (U ), Y ∗ = B − (U ), where A− is the generalized inverse of A, and U is uniformly distributed on [0, 1]. In contrast, for F given in (7.3.18), Y ) with joint d.f. given by F is the explicit form of the optimal pair (X, not known. However, in the discrete case one can use the Barnes–Hoffman greedy algorithm to compute F. Suppose µ, ν, and σ are discrete measures,
ai
:= µ({xi }),
bj
:= ν({yj }), j ∈ N = {1, 2, . . . , }, = bj = 1;
ai
i∈M
i ∈ M = {1, 2, . . . , m},
(7.3.19)
j∈N
σij
:= F σ (xi , yj ),
i ∈ M, j ∈ N.
(7.3.20)
Then F(xi , yj ) =
j i
prs ,
(7.3.21)
r=1 s=1
where the probabilities prs are determined by the following variant of the northwest corner rule (see Hoffman (1961), Barnes and Hoffman (1985)); in fact, we set p11 pij
:=
min(a1 , b1 , σ11 ); (7.3.22) ⎧ ⎫ ⎪ ⎪ j−1 i−1 ⎨ ⎬ := min ai − pis , bj − prj , σij − prs , ⎪ ⎪ ⎭ ⎩ r≤i s≤j s=1 r=1 (r,s)=(i,j)
if prs is determined for r ≤ i < m and s ≤ j < n, and we let j−1 i−1 pis , bj − prj , if i = m or j = n. pij := min ai − s=1
r=1
In other words, taking discrete versions of µ, ν, and σ in (7.3.19) one can apply the greedy algorithm (7.3.22) to approximate F in (7.3.18) by means of (7.3.21). Proof of Theorem 7.3.1: The proof is based on three assertions. Claim 7.3.6 (Fr´echet (1951)) The condition F σ (x, y) ≥ H− (x, y) = max(0, A(x) + B(y) − 1) is necessary and sufficient for F(A, B, F σ ) = Ø.
7.3 Mass Transportation Problems with Capacity Constraints
21
Suppose F(A, B, F σ ) = Ø. Then, by (7.3.4) H− (x, y) ≤ F (x, y) < F σ (x, y),
F ∈ F(A, B, F σ ).
(7.3.23)
On the other hand, if H− ≤ F σ , then H− ∈ F(A, B, F σ ). Claim 7.3.7 F defined by (7.3.18) has marginal d.f.s A and B and for all x, y ∈ IR, sup F ∈F (A,B,F σ )
F (x, y) ≤ F(x, y).
(7.3.24)
For any F ∈ F(A, B, F σ ) and any t ≤ x, s ≤ y, we have F (x, y) ≤ F (t, s) + µ((t, x]) + ν((s, y]), which clearly implies (7.3.24). Invoking Remark (7.3.3), F(x, y) ≤ H+ (x, y) where H+ is the upper Hoeffding–Fr´echet bound, H+ (x, y) := min(A(x), B(y)). Since F ≥ H− (cf. (7.3.23), (7.3.24)), F ∈ [H− , H+ ] has marginals A and B. σ
Theorem 7.3.1 is now a consequence of the following assertion. Claim 7.3.8 F is a d.f. To this end, we choose −∞ = x0 < x1 < · · · < xm−1 < xm = ∞, −∞ = y0 < y1 < · · · < yn−1 < yn = ∞ such that µ((xi−1 , xi )) < ε, ν((yn−1 , y1 )) < ε, and σ((xi−1 , xi ) × (yj−1 , yj )) < ε for all i ∈ M = {1, . . . , m} and j ∈ N = {1, . . . , n}. Set ai := µ((xi−1 , xi ]), bj := ν((yj−1 , yj ]), and σij := F σ (xi , yj ). Consider the convex polygon ⎧ ⎨ pij = ai , (7.3.25) p = (pij ) i∈M ; pij ≥ 0, pi· := j∈N ⎩ j∈N j i pij = bj , prs ≤ σij , for all i ∈ M, j ∈ N p·j := =
i∈M
r=1 s=1
p; pij ≥ 0, pi· = ai , p·j = bj ,
j i
prs ≤ σij , i = 1, . . . , m − 1,
r=1 s=1
j = 1, . . . , n − 1,
j s=1
p·s ≤ σmj , j ∈ N,
i
pr· ≤ σin , i ∈ M
.
r=1
By the Fr´echet condition (7.3.17) (cf. Claim 7.3.6) the marginals of F σ j majorize A and B, respectively, and thus σmj ≥ s=1 p·s and σin ≥ i p for all j ∈ N, i ∈ M . The polygon (7.3.25) becomes r=1 r· p; pij ≥ 0, pi· = ai , p·j = bj
for all i ∈ M, j ∈ N,
(7.3.26)
22
7. Relaxed or Additional Constraints j i
prs ≤ σij
for all i = 1, . . . , m − 1, j = 1, . . . , n − 1 .
r=1 s=1
Consider now the discrete analogue of F in (7.3.18): dij
:=
dij
:=
min {σrs + ar+1 + · · · + ai + bs+1 + · · · + bj },
0≤r≤i 0≤s≤j
0
(7.3.27)
if i = 0 or j = 0,
where σrs = 0 if r = 0 or s = 0. Our aim now is to show that d = (dij ) i∈M j∈N
determines a bivariate d.f. with support on X × Y, X = (xi )i∈M , Y = (yj )j∈N . Claim 7.3.9 The greedy algorithm (7.3.22) is determined uniquely by (7.3.27); i.e., j i
dij :=
prs ,
i ∈ N, j ∈ M.
(7.3.28)
r=1 s=1
Proof: Consider the discrete version of F (cf. (7.3.21), (7.3.25)). Let σr,s := 0 if r = 0 or s = 0, and define dij
:=
dij
=
min {σrs + ar+1 + · · · + ai + bs+1 + · · · + bj },
0≤r≤i 0≤s≤j
(7.3.29)
0 if i = 0 or j = 0.
We need to check the equality dij =
j i
prs ,
i ∈ M, j ∈ N,
(7.3.30)
r=1 s=1
where the pij ’s are determined by the greedy algorithm (7.3.22). If i = j = 1, then p11 = min(a1 , b1 , σ11 ) (cf. (7.3.22)), and by (7.3.29) d11 = min{σ11 + a1 + b1 , σ11 + a1 , σ10 + b1 , σ11 } = p11 . Suppose we have proved that d1,j−1 = p11 + · · · + p1,j−1 . Then p11 + · · · + p1j
(7.3.31)
7.3 Mass Transportation Problems with Capacity Constraints
=
j−1
p1s + min a1 −
s=1
= =
j−1
p1s , bj , σ1j −
s=1
j−1
23
p1s
s=1
min{a1 , bj + d1,j−1 , σ1,j } min{a1 , b1 + · · · + bj , σ11 + b2 + · · · + bj , . . . , σ1,j−1 + bj , σi,j }
= d1,j . These equalities hold due to (7.3.22), (7.3.31), and (7.3.29), respectively. By symmetry, di,1 = p11 + · · · + pi1 . Suppose next that drs =
s r
for all r ≤ i, s ≤ j, (r, s) = (i, j).
pkl
(7.3.32)
k=1 l=1
Then for 1 ≤ i ≤ m − 1, 1 ≤ j ≤ n − 1, j j j−1 i i−1 i prs = min ai + prs , bj + prs , σij = dij , r=1 s=1
r=1 s=1
r=1 s=1
where the equalities follow from (7.3.22) and (7.3.32). Thus dij =
j i
prs
for all 1 ≤ i ≤ m − 1, 1 ≤ j ≤ n − 1.
(7.3.33)
r=1 s=1
Consider now the case i = m. Then, m m−1 m−1 pr1 = pr1 + min am , b1 − pr1 r=1
r=1
=
r=1
min{am + dm−1,1 , b1 } = dm,1 ,
which follows from (7.3.22) and (7.3.33). Suppose that dm,j−1 =
j−1 m
prs .
(7.3.34)
r=1 s=1
Then using (7.3.22), (7.3.33), and (7.3.34), for 1 ≤ j ≤ n, j j−1 m m−1 m n prs = min am + prs , bj + prs r=1 s=1
r=1 s=1
=
r=1 s=1
min{σrs + ar+1 + · · · + am bs+1 + · · · + bj },
for 0 ≤ r ≤ m, 0 ≤ s ≤ j, (r, s) = (m, j); m m r=1 s=1
prs = dm,j ,
for r = m, s = j, σm,j = F σ (∞, yj ) ≥ bj .
24
7. Relaxed or Additional Constraints
Similarly, di,n =
i n
prs , for all i ∈ M , which proves Claim 7.3.9.
2
r=1 s=1
The greedy algorithm (7.3.22) defines nonnegative pij ’s (cf. Barnes and Hoffman (1985, Lemma 3.2)). Define the probability P (ε) on X × X by P (ε) ((−∞, xi ], (−∞, yj ]) := dij ,
i ∈ M, j ∈ N.
(7.3.35)
Similarly, (ai )i∈M and (bi )i∈N determine probabilites µ(ε) and ν (ε) with supports X and Y , respectively. If is the Kolmogorov (uniform) distance (µ, ν) := sup |F µ (x) − F ν (x)| ,
(7.3.36)
x∈IR
then the sequences µ(ε) ε>0 and ν (ε) ε>0 are -relatively compact, and thus there exists εn ↓ 0 such that
µ(εn ) , µ → 0 and ν (εn ) , ν → 0. (7.3.37) (For more facts on -relative compactness cf. Rachev (1984a) and Kakosjan, Klebanov, and Rachev (1988, Sec. 2.5).) Similarly, by definition of σij := F σ (xi , yj ), σ((xi−1 , xi )×(yj−1 , yj )) < ε we have that (σij ) i∈M determines a measure σ (ε) on X × Y . Again, the i∈N family σ (ε) ε>0 is -relatively compact. Thus, without loss of generality, we may assume that as εn → 0, (ε )
n (7.3.38) σ (εn ) , σ = sup F σ (x, y) − F σ (x, y) → 0. x,y∈R
(ε) As inClaim has marginals µ(ε) and ν (ε) , and 7.3.7, we conclude that P (ε) is tight. By (7.3.37), (7.3.38), (7.3.26), and (7.3.18), there thus P ε>0 exists a subsequence {εn } ⊂ {εn } such that P (εn ) weakly converges to a measure P with d.f. F. The proof of Theorem 7.3.1 is now complete. 2
The next theorem provides an explicit expression for the Fr´echet type bound (7.3.15). We recall the notations (7.3.11)–(7.3.13). Theorem 7.3.10 Suppose Gσ (x, y) := σ((−∞, x] × [y, ∞)) (cf. (7.3.13)) satisfies the condition
◦ ◦ B(y) := ν((−∞, y)) . (7.3.39) Gσ (x, y) ≥ max 0, A(x) − B(y)
7.3 Mass Transportation Problems with Capacity Constraints
25
Then the maximum in (7.3.15) is attained, and y) = inf {Gσ (t, s) + µ((t, x] + v([y, s))} . G(x, t≤x s≥y
(7.3.40)
Conclusions similar to those in Remarks 7.3.2–7.3.5 can be made. Here we shall only point out the greedy algorithm that can be used to approximate We use the notations (7.3.19) again and let the optimal distribution G. λij := Gσ (xi , yj ),
i ∈ M, j ∈ N.
(7.3.41)
has the form Then, in this discrete case, G i , yj ) = G(x
i n
prs ,
(7.3.42)
r=1 s=j
where the probabilites pij are determined by the following southwest corner rule: pin
:=
pij
:=
min{a1 , bn , λ1n }; (7.3.43) ⎧ ⎫ ⎪ ⎪ n i−1 ⎨ ⎬ min ai − pis , bj − prj , λij − prs , (7.3.44) ⎪ ⎪ ⎩ ⎭ r≤i s≥j r=1 s=j+1 (r,s)=(i,j)
if i = m or j = 1. if prs is determined for r ≤ i ≤ m − 1 and s ≥ j > 1; moreover, ⎧ ⎫ n i−1 ⎨ ⎬ pij := min ai − pis , bj − prj , if i = m or j = 1. ⎩ ⎭ s=j+1
(7.3.45)
r=1
We now give explicit solutions of the marginal problems I and II. Theorem 7.3.11 Suppose (i)
c : IR2 → IR is a right-continuous lattice superadditive function (−c is a Monge function);
(ii)
µ and ν are two Borel nonnegative measures on IR with µ(IR) = ν(IR) = λ < ∞ and d.f.s F µ and F ν , and such that
c(x0 , y)ν( dy) < ∞
c(x, y0 )µ( dx) + IR
IR
for some x0 , y0 ∈ IR;
(7.3.46)
26
7. Relaxed or Additional Constraints
(iii)
σ is a nonnegative bounded Borel measure on IR2 and F σ (x, y) ≥ max(0, F µ (x) + F ν (y) − λ)
for all x, y ∈ IR. (7.3.47)
Then the maximum in (7.3.6) is attained at the “optimal” measure P. P satisfies the feasibility conditions (7.3.7), (7.3.8) and is determined by
F P (x, y) := inf {F σ (t, s) + µ((t, x]) + ν((s, y])} , t≤x s≤y
x, y ∈ IR.
(7.3.48)
Proof: We need Theorem 3.1.2 (cf. Cambanis, Simons, and Stout (1976, Theorem 1); see also Rachev (1991c, Section 7.3)). If (7.3.5) holds, then for measures P1 and P2 on IR2 with marginals µ and ν, c dP1 ≤ c dP2 , (7.3.49) F P1 ≤ F P2 ⇒ IR2
IR2
2
which with an appeal to Theorem 7.3.1 yields the result.
Remark 7.3.12 The assumption (7.3.46) can be replaced by one of the following assumptions:
c(x, x)(µ + ν)( dx) < ∞.
(a)
c(x, y) is symmetric, and
(b)
c(x, y) is uniformly integrable for all P with marginals µ and ν.
That (a) implies (7.3.49) follows from Cambanis, Simons, and Stout (1976); that (b) implies (7.3.49) follows from Tchen (1980, Corollary 2.1); see also Rachev (1991c, Theorem 7.3.2). Remark 7.3.13 Condition (7.3.47) guarantees that the set of feasible solutions P determined by (7.3.7), (7.3.8) is not empty. Remark 7.3.14 If F σ (x, y) ≥ min(F µ (x), F ν (y)) =: H+ (x, y),
(7.3.50)
then F P in (7.3.48) equals H+ (cf. Remark 7.3.3). Thus, Theorem 7.3.11 (see also the next Theorem 7.3.17) can be considered as a generalization of Theorem 2 of Cambanis, Simons, and Stout (1976) and Corollary 2.2 of Tchen (1980). In this case, Hoffman’s (1962) northwest corner rule gives a greedy algorithm to determine an “optimal” measure P, provided that µ and ν have finite discrete support.
7.3 Mass Transportation Problems with Capacity Constraints
27
Remark 7.3.15 Consider the discrete version of Problem I (see (7.3.6)). Suppose c(i, j), i ∈ M, j ∈ N , is a lattice superadditive sequence c(i, j) + c(i + 1, j + 1) ≥ c(i, j + 1) + c(i + 1, j), (7.3.51) i = 1, . . . , m − 1, j = 1, . . . , n − 1. Hoffman (1961) and Barnes and Hoffman (1985) treat c(i, j) as the (negative) cost of shipping a unit commodity from origin i to destination j. Suppose the discrete measures µ and ν with supports M and N are given. Then ai = µ{i} and bj = ν{j} are interpreted as the amount of a product available at i and the amount required at destination j. Suppose the (m − 1) × (n − 1) matrix (σij ) satisfies ⎧ ⎫ i n ⎨ ⎬ σij ≥ max 0, ar − bs , (7.3.52) ⎩ ⎭ r=1
s=j+1
σij ≤ σis , σij ≤ σrj , σij + σrs ≥ σis + σrj , r ≥ i, s ≥ j. (These conditions are related to what is called a uniformly tapered matrix; see Marshall and Olkin (1979).) Barnes and Hoffman (1985) consider the following transportation problem: maximize c(i, j)pij (7.3.53) i∈M j∈N
subject to
j i
pij
≥ 0, pi· = ai , p·j = bj
pij
≤ σij ,
for all i ∈ M, j ∈ N, (7.3.54)
i = 1, . . . , m − 1, j = 1, . . . , n − 1.
r=1 s=1
Clearly, (7.3.54) is a special case of Problem I. Following Barnes and Hoffman, (7.3.54) can be viewed as the capacity restrictions on the amount that can be shipped from the first i origins to the first j destinations. Theorem 7.3.11 is completed by showing that the greedy algorithm of Barnes and Hoffman (1985) for determining the solution (pij ) i∈M of (7.3.53) is also j∈N characterized by F P (i, j) :=
j i
prs
(7.3.55)
r=1 s=1
=
σrs
=
min {σrs + ar+1 + · · · + ai + bs+1 + · · · + bj } ,
0≤r≤i 0≤s≤j
⎧ ⎨ 0 if r = 0 or s = 0, ⎩ +∞ if r = m or s = n.
28
7. Relaxed or Additional Constraints
Remark 7.3.16 One can determine the extremal value in (7.3.6):
max µ ν
F ∈F (F ,F
;F σ )
R2
c dF P ,
c dF =
(7.3.56)
IR2
where F P is given by (7.3.48). By (7.3.46) and Cambanis, Simons, and Stout (1976, p. 288, (9)), c dF = IR2
(7.3.57)
IR
c(x0 , y)F ν ( dy) − c(x0 , y0 ) +
c(x, y0 )F µ ( dx) + IR
B(x, y)µc ( dx, dy) IR2
for any bivariate d.f. F with marginals F µ and F ν . (Since F P ∈ µ ν· σ F(F , F , F ) by Theorem 7.3.1, (7.3.57) can be used to compute the value
c dF P . In (7.3.57) the points x0 and y0 are the same as in condition
of
(7.3.23), the measure µc is generated by c (see condition (i) in Theorem 7.3.11), and we also assume that c is a nondecreasing function in both arguments.) Finally, := B1 − B2 (7.3.58) ⎧ ⎪ ⎪ 1 + F (x, y) − F µ (x) − F ν (y) if x0 < x, y0 < y, ⎪ ⎨ B1 (x, y) := F (x, y) if x ≤ x0 , y ≤ y0 , ⎪ ⎪ ⎪ ⎩ 0 otherwise; ⎧ µ ⎪ ⎪ ⎪ F (x) − F (x, y) if x ≤ x0 , y0 ≤ y, ⎨ B2 (x, y) := F ν (y) − F (x, y) if x0 < x, y ≤ y0 , ⎪ ⎪ ⎪ ⎩ 0 otherwise. B
Theorem 7.3.17 Suppose conditions (i) and (ii) of Theorem 7.3.11 hold, and in addition ∗
(iii )
σ is a nonnegative bounded Borel measure on IR2 satisfying
Gσ := σ ((−∞, x], [y, ∞)) ≥ max (0, F µ (x) − ν((−∞, y)))
(7.3.59)
for all x, y ∈ IR. Then, the minimum in (7.3.9) is attained at an optimal measure Q satisfying the feasibility conditions (7.3.7) and (7.3.10); Q is
7.3 Mass Transportation Problems with Capacity Constraints
29
determined by GQ (x, y) = Q((−∞, x] × [y, ∞)) = inf {Gσ (x, y) + µ((t, x]) + ν([y, s))} .
(7.3.60)
t≤x s≥y
All the Remarks 7.3.12–7.3.16 can be easily reformulated regarding Theorem 7.3.17. In particular, consider the transportation problem c(i, j)pij (7.3.61) minimize i∈M j∈N
subject to
n i
pij
≥ 0, pi· = ai , p·j = bj
for all i ∈ M, j ∈ N, (7.3.62)
prs
≤ λij , i = 1, . . . , m − 1, j = 1, . . . , n − 1.
r=1 s=j
bj and c(·, ·) is a lattice superadditive sequence
j i (cf. (7.3.51)). Suppose also that λij ≥ max 0, r=1 ar − s=1 bs and for any r < i and s > j the inequalities
Suppose
i∈M
ai =
j∈N
λrj ≤ λij ≥ λis , λij + λrs ≥ λis + λrj ≥ 0, r < i, s > j,
(7.3.63)
hold. Then the greedy algorithm (7.3.43)–(7.3.45) realizes the minimum in (7.3.61). Moreover, the optimal pij ’s are determined by pij = fij − fi,j+1 − fi−1,j + fi−1,j+1 , where fij
:=
min {λrs + (ar+1 + · · · + ai ) + (bj + · · · + bs−1 )}
1≤r≤i j≤s≤n
∧
i r=1
ar ∧
n
bs .
(7.3.64)
s=j
The rest of this section is devoted to a generalization of the MKP with additional constraints stated in Problems I and II; see (7.3.6)–(7.3.10). The results are motivated by Hoffman and Veinott (1990), where the discrete version of the problem has been considered. We shall only state the results. The proofs are similar to those of Theorems 7.3.1 and 7.3.10 and will therefore be omitted. The abstract form of the problem is the following. Suppose that (i)
µ and ν are two nonnegative Borel measures on IR, µ(IR) = ν(IR) = λ < ∞;
30
7. Relaxed or Additional Constraints
(ii) L is a union of disjoint sublattices Li ⊂ IR2 , i ∈ S, and the projections of L on each axis equal IR; (iii) (σi )i∈S are nonnegative σ-finite Borel measures on Li . Then the problem is to find min c dP,
(7.3.65)
L
where the minimum is subjet to the following constraints: (i)
P ’s are nonnegative Borel measures on L with marginals µ and ν;
(7.3.66)
(ii)
P (A) ≤ σi (A)
(7.3.67)
for any A = Li ∩ (−∞, x] × (−∞, y],
(x, y) ∈ Li , i ∈ S.
As before, see (7.3.1)–(7.3.3), the measures µ and ν are viewed as initial and final mass distributions, and P in (7.3.66), (7.3.67) are the (feasible) transportation plans. Here the generalization of problems I and II is that L describes the path of the transportation flow and σi ’s are capacity constraints on the cumulative supply–demand flow. Finally, c : L → R is a cost function, and therefore, the integral in (7.3.65) represents the total cost of mass transportation applying the plan P . Suppose c is subadditive on the lattice L; that is; for all x, y ∈ L, f (x) + f (y) ≥ f (x ∧ y) + f (x ∨ y). Then we shall call a feasible plan of transportation achieving the minimum in (7.3.65) an optimal measure P ∗ . As in problems I and II we start with extensions of the classical Hoeffding– Fr´echet bounds (7.3.4), assuming that P meets the constraints (7.3.66) and (7.3.67), or their alternatives: P is a nonnegative Borel measure on L = Li (7.3.68)
i∈S
Li := {(x, y); (x, −y) ∈ L} with marginals µ and ν ;
P (B) ≤ σi (B) for any B = Li ∩ ((−∞, x] × [y, ∞)). The restriction on the support of P given in (7.3.66) has the form L = Li , where S = {0, 1, . . . , s}, i∈S
(7.3.69)
7.3 Mass Transportation Problems with Capacity Constraints
31
+ − + or S = IN, and each sublattice Li is a rectangle (x− i , xi ] × (yi , yi ], where − − − + − + − − + − x0 = y0 = −∞, xi < xi , yi < yi , xi−1 ≤ xi ≤ xi−1 ≤ x+ i , yi−1 ≤ + + yi− ≤ yi−1 ≤ yi+ , x+ s = ys = ∞. Write PL (resp. PL ) to denote the class of all P ’s on L with (7.3.66) and (7.3.67) (resp. (7.3.68), (7.3.69)). Recall that for any measure P on IR2 , F P stands for the d.f. of P , and GP (x, y) = P ((−∞, x]×[y, ∞)). In the next two theorems we shall compute the bounds
F ∗ (x, y) = max F P (x, y)
(7.3.70)
G∗ (x, y) = max GP (x, y).
(7.3.71)
P ∈PL
and P ∈PL
For L = IR2 and σi = +∞, F ∗ is indeed the upper Hoeffding–Fr´echet bound H+ (x, y) = min {F µ (x), F ν (y)}
(F µ (x) := µ((−∞, x]) . (7.3.72)
On the other hand, G∗ (x, y) = min {F µ (x), Gν (y)} determines a measure with d.f.
(Gν (x) := ν([y, ∞)))
H− (x, y) = max (0, F µ (x) + F ν (y) − λ) ,
(7.3.73)
which is the lower Hoeffding–Fr´echet bound. Theorem 7.3.18 Suppose that F ∗ : L → IR is defined iteratively as follows: F ∗ (x, y) =
min
[µ((u, x]) + ν((v, y]) + F σ0 (u, v)]
(7.3.74)
min
[µ((u, x]) + ν((v, y]) + F σi (u, v)
(7.3.75)
+ −∞
for (x, y) ∈ L0 , and F ∗ (x, y) =
− +
x
+ + ∗ + + F ∗ (x+ i−1 , v ∧ yi−1 ) ∨ F (xi−1 ∧ u, yi−1 )
for (x, y) ∈ Li , i ≥ 1. Suppose also that F ∗ satisfies the inequalities + ∗ F σi (x.y) + F ∗ (x+ i−1 , y) ∨ F (x, yi−1 ) ≥ H− (x, y)
(7.3.76)
for (x, y) ∈ Li , i ∈ S. Then the equality (7.3.70) holds, and moreover, F ∗ is a d.f. of some P ∗ ∈ PL .
32
7. Relaxed or Additional Constraints
The proof is similar to that of Theorem 7.3.1. A slightly different approach (see Olkin and Rachev (1990)) can be used based on the following result of Topkis and Veinott (1973): The minimum of subadditive functions over a sublattice with respect to some variables is subadditive in the remaining variables; see also Hoffman and Veinott (1990) for the discrete version of Theorem 7.3.18. In the next theorem we evaluate the bound G∗ in (7.3.71). Recall that − + + − Li := (xi , xi ] × [−yi , −yi ), L := i∈S Li . Theorem 7.3.19 Suppose that G∗ : L → IR is defined iteratively by G∗ (x, y) = min [µ((u, x]) + ν([y, v)) + Gσ0 (u, v)] , for (x, y) ∈ L0 (7.3.77) u≤v v≥y
and G∗ (x, y)
=
min
− + x
v≥y≥−y i i
µ((u, x]) + ν([y, v)) + Gσi (u, v)
(7.3.78)
+ + + ∗ + G∗ (x− i−1 , v ∨ (−yi−1 )) ∨ G (u ∧ xi−1 , −yi−1 )
for (x, y) ∈ Li . Suppose also that G∗ satisfies the inequalities + ∗ Gσ (x, y) + G∗ (x+ i−1 , y) ∨ G (x, −yi−1 ) ≥ F µ (x) + Gν (y) − λ for any (x, y) ∈ Li , i ∈ S.
(7.3.79)
Then the inequality (7.3.71) holds, and G∗ defines P ∗ ∈ PL by G∗ (x, y) = P ∗ ((−∞, x] × [y, ∞)) for (x, y) ∈ L. Condition (7.3.79) is necessary for P ∗ ∈ PL . Next, we shall formulate a multivariate analogue of Theorem 7.3.18. (In general, Theorem 7.3.19 does not admit a multivariate extension by the well-known reason that the lower Hoeffding–Fr´echet bound for d.f.s on IRr (r > 2) with given one-dimensional projections does not generate a measure.) Let µ = µ(1) , . . . , µ(r) be a vector of r Borel nonnegative measures on IR with one and the same total mass λ < ∞. Suppose L is a complete Borel sublattice on IRr whose projection on every axis x(i) (i ∈ R := {1, . . . , r}) is the entire real line IR. Suppose also that L is a union of disjoint nonempty sublattices Li , i ∈ S (S = {0, . . . , s} or S = IN) and each Li is a rectangle in IRr ,
(j)− (j)+ + = ⊗rj=1 xi , xi Li = x− , i ∈ S, i , xi
7.3 Mass Transportation Problems with Capacity Constraints
33
− + − − + + + with x− 0 = −∞, xs = +∞, xi < xi , xi−1 ≤ xi ≤ xi−1 ≤ xi . (For representations of sublattices on a product of r lattices we refer to Veinott (1989, Section 4).)
Given σi , a nonnegative Borel measure on Li , and a measure P on L with vector of one-dimensional projections µ we write PLi ≺ σi to denote that the restriction of P on Li is less concordant that σi ; that is, for any x ∈ Li , − P ((x− i , x]) ≤ σi ((xi , x]),
i ∈ S.
Note that in contrast with the usual definition of concordance (Kruskal (1958), Tchen (1980), Stoyan (1983)) we allow σi to have total mass different from that of PLi . For example, assuming that σi vanishes on a subset of Li , we in fact impose additional restrictions on the support of P . Write PL (µ, σ) (σ := (σi )i∈S ) to denote the class of all P ’s on L possessing the properties (i)
P is a nonnegative Borel measure on L with vector of one-dimensional marginals µ.
(7.3.80)
(ii)
PLi ≺ σi
(7.3.81)
for all i ∈ S.
Define the mapping F ∗ : L → IR iteratively as follows: For x ∈ L0 , let F ∗ (x) =
⎧ ⎨
min
−∞
⎩
µ(j)
u(j) , x(j)
j∈R
+ F σ0 (u)
⎫ ⎬ ⎭
,
and for x ∈ Li , let F ∗ (x) =
min
⎧ ⎨
+ x− i
⎩
µ(j)
u(j) , x(j)
j∈R
⎫ ⎬ + F σi (u) + fLi (u) , ⎭
(1)− (2) (r) (1) (2) (r) where fLi (u) = max F ∗ xi , vi , . . . , vi , F ∗ vi , xi , . . . , vi ,
(1) (2) (r)− (j) (j)+ and vi := u(j) ∧ xi−1 , u ∈ Li . Denote by H− F ∗ vi , vi , . . . , xi the lower Hoeffding–Fr´echet bounds for multivariate d.f. with prescribed one-dimensional marginals: ⎛ H− (u) = max ⎝0,
j∈IR
µ(j)
−∞, u(j)
⎞ − (r − 1)λ⎠ .
34
7. Relaxed or Additional Constraints
Theorem 7.3.20 Suppose F ∗ defined above satisfies the inequality F σi (u) + fLi (u) ≥ H− (u)
for every u ∈ Li , i ∈ S.
Then max F P = F ∗ ,
P ∈PL
and F ∗ is a d.f. of some P ∗ ∈ PL . The proof is similar to that of Theorem 7.3.1. It requires a multivariate analogue of the greedy algorithm similar to that in Barnes and Hoffman (1985) and Olkin and Rachev (1990). A multivariate version of Hoffman’s (1963) northwest corner rule is given in Balinski and Rachev (1989), where the interplay between greedy algorithms and MKPs is emphasized. We are now ready to state the solution of MKP (7.3.65) with constraints (7.3.66) and (7.3.67). Theorem 7.3.21 Suppose that the assumptions of Theorem 7.3.18 hold, and c : L → IR is subadditive and left-continuous on L with c(x, y0 )µ( dx) + c(x0 , y)ν( dy) > −∞ (7.3.82) IR
IR
for some (x0 , y0 ) ∈ L. Then the minimum in (7.3.65) is attained at P ∗ ∈ PL defined in Theorem 7.3.18. The next theorem gives the solution of the following MKP: minimize c dP
(7.3.83)
L
under the constraints (7.3.68), (7.3.69), and assuming that c is superadditive on L; that is, (−c) is subadditive on L. Theorem 7.3.22 Suppose that the assumptions of Theorem 7.3.19 hold, and c : L → IR is superadditive and right-continuous with c(x, y0 )µ( dx) + c(x0 , y)ν( dy) < ∞. (7.3.84) IR
IR
Then the minimum in (7.3.84) is attained at P ∗ ∈ PL defined in Theorem 7.3.19.
7.3 Mass Transportation Problems with Capacity Constraints
35
The proof of the above two theorems is the same as that of Theorem 7.3.11. Example 7.3.23 (The discrete case) Suppose µ and ν are discrete measures with supports I = {1, . . . , m} and J = {1, . . . , n} and L = L0 +· · ·+Ls is a sublattice of I ×J with projections I and J, respectively. Then Theorem 7.3.18 corresponds to the main theorem in Hoffman and Veinott (1990). Example 7.3.24 (MKP with capacity constraints) Suppose that in Theorems 7.3.21 and 7.3.22, L = L = L0 = L0 = IR2 . Then we obtain the solution of problems I and II; see (7.3.6)–(7.3.10). In fact, Theorems 7.3.21 and 7.3.22 reduce to Theorems 7.3.11 and 7.3.17. We shall complete this section with another possible extension of Problems I and II. Consider a finite measure µ on (IR2 , B 2 ) and define for two probability measures P1 , P2 on (IR1 , B 1 ) and Ai × Bi ∈ B 1 ⊗ B 1 , i ∈ I, M µ (P1 , P2 ) (7.3.85) 1 = P ∈ M (P1 , P2 ); P (Ai × Bi ) ≤ µ(Ai × Bi ), i ∈ I , where M 1 (P1 , P2 ) denotes the set of all probability measures P on IR2 with marginals P1 , P2 . As in Theorem 7.3.1 (see (7.3.17)), we assume that µ(Ai × Bi ) ≥ (P1 (Ai ) + P2 (Bi ) − 1)+ .
(7.3.86)
Theorem 7.3.25 Under the assumption (7.3.86) let us define P ∗ (A × B) = inf {µ (A1 × Bi ) + (P1 (A) − P1 (Ai )) + (P2 (B) − P2 (Bi ))} Ai ⊂A Bi ⊂B
∧ min(P1 (A), P2 (B)),
A, B ∈ B 1 .
(7.3.87)
Then hµ (A × B) := sup{P (A × B); P ∈ M µ (P1 , P2 )} ≤ P ∗ (A × B). (7.3.88) If P ∗ determines a measure, then hµ (A × B) = P ∗ (A × B),
and P ∗ is a solution of (7.3.87).
(7.3.89)
Remark 7.3.26 The proof of Theorem 7.3.25 is similar to that of Theorem 7.3.1. In contrast to Theorem 7.3.1 it allows us to consider “local” bounds in the transportation problem. Observe that in the finite discrete case, bounds of the type xij ≤ µij
for some (i, j)
(7.3.90)
are of this “local” type. As far as we know, in the literature there is no result concerning the solution of (7.3.90) with local bounds. See the next section for a possible approach to the problem.
36
7. Relaxed or Additional Constraints
7.4 Local Bounds for the Transportation Plans While in the preceding three sections the additional constraints were formulated mainly in terms of the d.f.s, we now consider “local” constraints formulated in terms of the probability densities. These local-type restrictions are stronger than those in the previous section, and generally they are much more difficult to handle; see Remark 7.3.26. Our first result deals with a transportation problem with “indicator” cost function ⎧ ⎨ 1 if x = y, (7.4.1) c(x, y) = I(x = y) = ⎩ 0 if x = y; i.e., the cost of transportation is one for any unit mass that has to be moved, and zero otherwise. The cost function c does not satisfy a Mongetype condition. We formulate this transportation problem on a general measure space (S, U) assuming only that {(x, y); x = y} ∈ U ⊗ U.
(7.4.2)
Let Mf (S), Mf (S×S) be the set of all finite measures on (S, U), respectively (S ×S, U ⊗U), and for µ ∈ Mf (S ×S), let πi µ, i = 1, 2, denote the marginals of µ. (This transportation problem leads to an extension of Dobrushin’s result on optimal couplings.) Theorem 7.4.1 (Optimal couplings with local restrictions) Assume that (7.4.2) holds and let µ1 , µ2 ∈ Mf (S) with µ1 (S) ≤ µ2 (S). Then (a) inf{µ ({(x, y); x = y}) ; µ ∈ Mf (S × S), π1 µ ≥ µ1 , π2 µ ≤ µ2 } (7.4.3) = λ− (S) := sup (µ1 (C) − µ2 (C)). C∈U
(b) Moreover, the infimum in (7.4.3) is attained at µ∗ (A × B) = γ(A ∩ B) +
λ− (A)λ+ (B) , λ+ (S)
(7.4.4)
where λ+ (A) = supC⊂A (µ2 − µ1 )(C), λ− (A) = supC⊂A (µ1 − µ2 )(C) and γ(A) = µ2 (A) − λ+ (A) = µ1 (A) − λ− (A). Proof: For any µ ∈ Mf (S × S), µ(x = y) ≥ sup µ(C × (S \ C)) = sup{µ(C × S) − µ(C × C)} C
C
≥ sup{µ(C × S) − µ(S × C)} ≥ sup{µ1 (C) − µ2 (C)} C
=
C
sup{λ− (C) − λ+ (C)} = λ− (supp λ− ) = λ− (S). C
7.4 Local Bounds for the Transportation Plans
37
On the other hand, µ∗ (A × S) = γ(A) + λ− (A)λ+ (S)/λ+ (S) = µ1 (A) and µ∗ (S × B) = γ(B) + λ− (S)λ+ (B)/λ+ (S) ≤ γ(B) + λ+ (B) = µ2 (B). Finally, we have
∗
µ (x = y) = =
I(x = y)(γ( dx, dy) + λ− ( dx)λ+ ( dy)/λ+ (S)) I(x = y) λ− ( dx)λ+ ( dy)/λ+ (S)
= λ− (S)λ+ (S)/λ+ (S) = λ− (S). 2 Consider next some finite measures µ1 , µ2 on IR with densities h1 , h2 with respect to a dominating measure µ on IR1 . Define Pµµ12 := {P ∈ M 1 (IR2 , B 2 ); π1 P ≥ µ1 , π2 P ≤ µ2 }.
(7.4.5)
Any P ∈ Pµµ12 has marginals P1 = π1 P, P2 = π2 P with densities f1 ≥ h1 and f2 ≤ h2 with respect to µ. We assume that 1 = µ1 (IR1 ) ≤ µ2 (IR1 ); i.e., µ1 is a probability measure, and so f1 = h1 . ⎧ ⎪ ⎨ Theorem 7.4.2 Let z0 = inf y; ⎪ ⎩
f2∗ (y) =
⎧ ⎪ ⎪ h2 (y) ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ 1 − h2 (u) du ⎪ ⎪ ⎪ ⎨ (z0 ,∞)
⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ 0 ⎪ ⎪ ⎩
(y,∞)
⎫ ⎪ ⎬ h2 dµ ≤ 1 , ⎪ ⎭
if y > z0 ,
if y = z0 and µ{z0 } > 0,
(7.4.6)
µ(z0 ) otherwise,
and let P ∗ be the corresponding probability measure with µ-density f2∗ . Then the following characterizations of the optimal coupling hold: (a) sup{F P (x, y); P ∈ Pµµ12 } = 1−max(Fµ1 (x), FP ∗ (y)), for all x, y where F P (x, y) = P ([x, ∞) × [y, ∞)) is the survival function.
38
7. Relaxed or Additional Constraints
(b) The sup in (a) is attained for the distribution F ∗ = FX ∗ ,Y ∗ , where (U ), Y ∗ = FP−1 X ∗ = Fµ−1 ∗ (U ). 1 (c) If c is a cost function that is componentwise antitone and satisfies the Monge condition (cf. (7.1.3), (7.1.4)), then inf
c(x, y) dFP (x, y); P ∈ Pµµ12
=
c(x, y) dF ∗ (x, y).
(7.4.7)
Proof: (a), (b) For P ∈ Pµµ12 with marginals Fµ1 , G2 , we know that F P (x, y) ≤ P Fµ−1 (U ) ≥ x, G−1 2 (U ) ≥ y = P (U ≥ max(Fµ1 (x), G2 (y))) = 1 1 − max(Fµ1 (x), G2 (y)). By the definition of P ∗ , FP ∗ (y) ≤ G2 (y) for all y, and therefore, F P (x, y) ≤ 1 − max(Fµ1 (x), FP ∗ (y)). (c) Applying (a), (b), and Theorem 3.1.2 we obtain (7.4.7). The conditions on the cost function c were studied by R¨ uschendorf (1980). In that terminology (−c) is a -monotone function. Applying the results in R¨ uschendorf (1980), it is easy to check that (c) follows from (a), (b). 2 The “antitone” assumption in (c) of Theorem 7.4.2 does not have a transparent interpretation in terms of cost functions. Moreover, under some additional assumptions on the bounding measures we can construct solutions for more “natural” cost functions. Again, let µ1 have densities hi with respect to µ, 1 = µ1 (IR1 ) ≤ µ2 (IR1 ). Theorem 7.4.3 Assume that for some y0 ∈ IR1 , h1 (u) ≤ h2 (u) for u < y0
and
h1 (u) ≥ h2 (u) for u ≥ y0 .
(7.4.8)
Define
x0
⎧ ⎪ ⎨ = inf y; ⎪ ⎩
⎫ ⎪ ⎬
h1 (u) dµ(u) ≥
(y,∞)
h2 (u) dµ(u) (y,∞)
⎪ ⎭
,
and let ⎧ h2 (u) ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ h1 (u) dµ(u) − h2 (u) dµ(u) ⎪ ⎪ ⎪ ⎨ f2 (u) :=
[x0 ,∞)
⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ h1 (u)
(x0 ,∞)
µ(x0 )
if u > x0 ,
if u = x0 and µ{x0 } > 0, if u < x0 .
(7.4.9)
7.4 Local Bounds for the Transportation Plans
39
Then for any cost function c satisfying the Monge condition (7.1.3) and the unimodality condition (7.1.27) we have inf
c(x, y) dfP (x, y); P ∈
Pµµ12
1 (u), F2−1 (u) du, = c Fµ−1 1
(7.4.10)
0
where F2 is the d.f. of the measure with density f2 with respect to µ. The op(U ), Y ∗ = F2−1 (U ). timal distribution is determined by the r.v.s X ∗ = Fµ−1 1 Proof: Invoking the Monge condition, for any P ∈ Pµµ12 with marginals 1 Fµ1 , G2 , we have c(x, y) dFP (x, y) ≥ c Fµ−1 (u), G−1 2 (u) du. 1 0
By the definition of F2 , G2 (y) F2 (y)
≥ F2 (y) ≥ Fµ1 (y) =
for all y ≥ x0 ,
and
(7.4.11)
for all y ≤ x0 ;
Fµ1 (y)
in fact, (7.4.11) implies that Fµ−1 (u) ≥ F2−1 (u) ≥ G−1 2 (u) for u > F2 (x0 ) 1 −1 −1 and F2 (u) = Fµ1 (u) for u ≤ F2 (x0 ). Our assumptions on c imply that −1 −1 , G−1 2 c Fµ−1 2 (u) ≥ c Fµ1 (u), F2 (u) for all u. 1 Remark 7.4.4 It is not difficult to extend the solution of Theorem 7.4.3 to the case µ1 (IR1 ) < 1 and to the case f1 ≥ h1 , f2 ≤ h2 for the densities (here, f1 and f2 are the marginal densities of an admissible plan P ), if we still keep the assumption (7.4.8). To see this, choose x0 as in (7.4.9), and define ⎧ h2 (x) ⎪ ⎪ ⎪ ⎪ ⎪ 1− h2 (x) dµ(x) ⎪ ⎪ ⎪ ⎨ f2 (x) =
where z0
y0
(z0 ,∞)
⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩
0 ⎧ ⎪ ⎨ = inf x; ⎪ ⎩ ⎧ ⎪ ⎨ = inf y; ⎪ ⎩
if x > z0 ,
if x = z0 and µ(z0 ) > 0,
µ(z0 ) otherwise, ⎫ ⎪ ⎬ h2 (x) dµ(x) ≤ 1 . Define next ⎪ ⎭
(x,∞)
h2 (x) dµ(x) ≤ (y,∞)
⎫ ⎪ ⎬
h1 (x) dµ(x) (y,∞)
⎪ ⎭
(7.4.12)
40
7. Relaxed or Additional Constraints
and ⎧ h1 (x) ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ f2 (x) ⎪ ⎪ ⎪ ⎨ f1 (x) = (h2 (x) − h1 (x)) dµ(x) ⎪ ⎪ ⎪ ⎪ ⎪ [y0 ,∞) ⎪ ⎪ ⎪ ⎪ µ(y0 ) ⎩
if x > y0 , if x < y0 , (7.4.13) if µ(y0 ) > 0.
Then for a cost function c defined as in Theorem 7.4.3, we have inf c(x, y) dFP (x, y); π1 P ≥ µ1 , π2 P ≤ µ2
(7.4.14)
1 = c F1−1 (u), F2−1 (u) du, 0
where Fi have densities fi with respect to µ, i = 1, 2. Let us return to the comment we made in Remark 7.3.26. Consider transportation problems with local upper bounds on the transportation plans xij ≤ µij in the discrete case, while P ≤ µ for some finite measure µ in the general case. The following framework allows us to handle quite general transportation problems. On a measurable space (X, B), let Bi ⊂ B, be sub-σ-algebras, 2 ≤ i ≤ n, Pi ∈ M 1 (X, Bi ). Further, let µ be a finite measure on (X, B), and define Mµ := P ∈ M 1 (X, B); P/Bi = Pi , 1 ≤ i ≤ n, P ≤ µ . (7.4.15) Assume that Mµ = Ø and define the set of generalized transportation plans with local upper bound µ as follows: Uµ (ϕ) := inf U (ϕ0 ) + h dµ; h ≥ 0, ϕ0 + h ≥ ϕ , (7.4.16) where U (ϕ0 ) := inf
n i=1
1
fi dPi ; fi ∈ L (Bi , Pi ), ϕ0 ≤
n
fi
.
i=1
We view U as the dual operator for the “pure” transportation problem, and typically, sup ϕ0 dP ; P ∈ M (P1 , . . . , Pn ) = U (ϕ0 ) (7.4.17)
7.4 Local Bounds for the Transportation Plans
41
will hold (cf. Chapter 2). The duality principle allows us to infer the corresponding “minimization” problem. Similarly, Uµ is the dual operator for the local majorized transportation problem. A linear operator S is majorized by Uµ , S ≤ Uµ if and only if S ≥ 0, S/Bi = Pi , 1 ≤ i ≤ n, and S ≤ µ. (7.4.18) Therefore, the approach developed in Chapter 2 yields the duality theorem ϕ dP ; P ∈ Mµ (7.4.19) =: Mµ (ϕ) Uµ (ϕ) = sup for any upper semicontinuous or uniformly approximable integrable functions ϕ in the case of a compactly approximable measure space (X, Bi , Pi ) with countable topological basis. In some sense (7.4.19) gives the duality result for the general case of order restrictions as considered, for example, in Sections 3.5 and 5.5. In particular, we obtain upper bounds n Mµ (ϕ) ≤ f dP i i for any admissible system of functions (fi ) with i=1 n ϕ ≤ i=1 fi . We next consider the question of more explicit evaluations of the dual operator Uµ for the case ϕ = 1B , B ∈ B; or equivalently, we wish to establish sharp upper Fr´echet bounds in the class Mµ . Define Mµ (B) := Mµ (1B ), and assume the duality (7.4.19) for φ = 1B . Theorem 7.4.5 Mµ (B) =
sup P ∈M (P1 ,...,Pn )
P ∧ µ(B),
(7.4.20)
where P ∧ µ is the infimum in the lattice of measures. Proof: From (7.4.19), Mµ (B) = inf{µ(h) + U (ϕ); h ≥ 0, ϕ + h ≥ 1B } = inf{µ(h) + U (1B − h); 0 ≤ h ≤ 1B }.
(7.4.21)
To see the second equality in (7.4.4), take ϕ = (1B − h)+ . Thus, 0 ≤ ϕ, and it is possible to assume that h ≤ 1B . Next, we make use of the “integration” approach in Strassen (1965), Mµ (B) =
=
inf
sup
{µ(h) + P (1B − h)}
⎧ 1 ⎨
1
0≤h≤1B P ∈M (P1 ,...,Pn )
inf
sup
0≤h≤1B P
⎩
µ(h > t) dt + 0
0
(7.4.22)
⎫ ⎬ P (1B − h ≥ 1 − t) dt . ⎭
42
7. Relaxed or Additional Constraints
With Ct := {h > t} ⊂ B we see that {x; h(x) ≤ 1B (x) − 1 + t} = {x ∈ B; h(x) ≤ t} = B \ Ct . Therefore, 1 Mµ (B) =
inf
(µ(Ct ) + P (B \ Ct )) dt
sup
0≤h≤1B P
0
≥ sup inf {µ(C) + P (B \ C)} = sup µ ∧ P (B). P
C⊂B
P
On the other hand, Mµ (B) = sup{P (B); P ∈ M (P1 , . . . , Pn ), P ≤ µ} = sup{P ∧ µ(B); P ∈ M (P1 , . . . , Pn ), P ≤ µ} ≤
sup P ∈M (P1 ,...,Pn )
P ∧ µ(B). 2
Theorem 7.4.5 allows us to reduce the problem of the majorized Fr´echet boundsto a problem of “usual” Fr´echet bounds, but for a more complicated functional. It remains an open problem to determine more explicit formulas for Mµ (B) in the general case.
7.5 Closeness of Measure with Joint Marginals on a Finite Number of Directions In this section we follow the work of Kakosjan and Klebanov (1984), Khalfin and Klebanov (1990), Klebanov and Rachev (1995a, 1995b, 1995c), on the application of marginal problems to computer and diffraction tomography. Here, estimates of the closeness between probability measures defined on IRn that have the same marginals on a finite number of arbitrary directions will be provided. The estimates show that the probability laws get closer in a certain metric when the number of coinciding marginals increases. The results offer a solution to the computer tomography paradox stated in Gutman, Kemperman, Reeds, and Shepp (1991). We start with some historical remarks and with the statement of the problem. Let Q1 and Q2 be a pair of probabilities on IR, i.e., probability measures defined on the Borel σ-field of IR. Lorentz (1949) studied conditions for the existence of a probability density function g(·) on IR2 taking
7.5 Closeness of Measure on a Finite Number of Directions
43
only two values, 0 or 1, and having Q1 and Q2 as marginals. In his 1961 paper Kellerer generalized this result and gave necessary and sufficient conditions for the existence of a density f (·) on IR2 that satisfies the inequalities 0 ≤ f (·) ≤ 1 and has Q1 and Q2 as marginals (see also Strassen (1965) and Jacobs (1987)). Fishburn et al. (1990) were able to show that Kellerer’s and Lorentz’s conditions are equivalent; i.e., for any density 0 ≤ f ≤ 1, on IR2 there exists a density taking the values 0 and 1 only that has the same marginals. In general, similar results hold for probability densities on IRm , m ≥ 2, when the (m − 1)-dimensional marginals are prescribed. Gutmann et al. (1991) show that for any probability density 0 ≤ f ≤ 1 on IRm and for any finite number of directions, there exists a probability density taking the values 0, 1 only that has the same marginals in the chosen directions. It follows that densities having the same marginals in a finite number of arbitrary directions may differ considerably in the uniform metric between densities, which is indeed a very strong metric; recall that convergence in the uniform metric implies convergence in total variation. The goal in this section is to show that under moment-type conditions, measures having a “large” number of coinciding marginals are close to each other in the weakmetrics.(1) The method is based on techniques used in the classical moment problem. On the other hand, most of our results will make use of relationships between different probability metrics, analyzed in the monograph by Kakosjan, Klebanov, and Rachev (1988), referred to below as KKR (1988). The key idea in showing that measures with a large number of common marginals are close to each other in the weak metrics is best understood by comparing three results. The first is the theorem of Gutman et al. (1991) mentioned above. The second (see Karlin and Studden (1966, p. 265)) states that if a finite number of moments µ1 , . . . , µn of a function f , 0 ≤ f ≤ 1, are given, then there exists a function g that takes the values 0 or 1 only and possesses the moments µ1 , . . . , µn . Finally, the third result (see KKR (1988, pp. 170–197)) gives estimates of the closeness in terms of a weak metric (the so-called λ-metric) on IR for measures having a finite number of common moments. Of course, since the condition of common marginals seems to be more restrictive than the condition of equal moments, one should be able to construct a similar estimate expressed in terms of the common marginals only. Furthermore, the technique should be similar to that used here. For simplicity, let us consider the 2-dimensional case. Let θ1 , . . . , θn be n unit vectors in the plane and P1 , P2 be two probabilities on IR2 having the same marginals in the directions θ1 , . . . , θn . To estimate the distance (1) Here weak metric stands for a metric metrizing the weak convergence in the space of probability measures on a Euclidean space.
44
7. Relaxed or Additional Constraints
between P1 and P2 , various weak metrics can be used; however, it seems that the λ-metric is the most convenient for this purpose. This metric is defined as follows (see, for example, Zolotarev (1986)): Let ϕi (t) = ei(t,x) Pi ( dx), i = 1, 2, IR2
be the characteristic function of Pi . Then define the λ-distance between P1 and P2 as 1 λ(P1 , P2 ) = min max max |ϕ1 (t) − ϕ2 (t)|, ; (7.5.1) T >0 T t≤T here (·, ·) is the inner product and · is the Euclidean norm. Clearly, λ metrizes the weak convergence. Our first result concerns the important case where one of the probability measures considered has compact support. Lemma 7.5.1 Let θ1 , . . . , θn be n ≥ 2 unit vectors in IR2 , no two of which are collinear. Let the support of the probability P1 be a subset of the unit disk, and let the probability P2 have the same marginals as P1 in the directions θ1 , . . . , θn . Set(2) " # n−1 s = 2 . (7.5.2) 2 Then 1 2 s+1 λ(P1 , P2 ) ≤ . s!
(7.5.3)
Remark 7.5.2 We can replace the right-hand side of (7.5.3) by C/s, where 1 2 s+1 C is a constant; note that as s → ∞, 2! ∼ e/s. The difference 1 2 s+1 e − s is plotted in figures 7.1 and 7.2. s! Proof of Lemma 7.5.1: The λ-metric is invariant under rotations of the coordinate system, so without loss of generality we assume that (a) the directions θj (j = 1, . . . , n) are not parallel to the axis; (b) there exists at least one pair of directions, say θj1 and θj2 , such that θj1 = (a, b), θj2 = (a, −b), where a = 0, b = 0; i.e., the vectors θj1 and θj2 are symmetric about the horizontal axis. (2) Here
and in what follows [r] denotes the integer part of the number r.
45
-0.6
-0.4
-0.2
0.0
7.5 Closeness of Measure on a Finite Number of Directions
-1.2
-1.0
-0.8
FIGURE 7.1. Plot of the difference (2/s!)1/(s+1) − e/s for s = 1, . . . , 100
20
40
60
80
100
0.0
0.001
0
-0.002
-0.001
FIGURE 7.2. Plot of the difference (2/s!)1/(s+1) − e/s for s = 10, . . . , 100
20
40
60
80
100
The law P1 has bounded support, and so, since the marginals on the directions θ1 , . . . , θn of P1 and P2 coincide, then for all j = 1, . . . , n,
(x, θj )k P1 ( dx) =
IR2
(x, θj )k P2 ( dx).
(7.5.4)
IR2
To see that P2 has moments of any order, consider (7.5.4) with j = j1 , j = j2 , and x = (x1 , x2 ). Then (x1 a ± x2 b)k (P1 − P2 )( dx) = 0, IR2
(x1 a + x2 b)k + (x1 a − x2 b)k (P1 − P2 )( dx) = 0,
(7.5.5)
IR2
and all integrals are finite. If k is even, then (ax1 + bx2 )k + (ax1 − bx2 )k ≥ ak xk1 + bk xk2 , and thus (7.5.5) implies the existence of all moments of P2 of even order.
46
7. Relaxed or Additional Constraints
The next step is to show that all moments of P1 and P2 of order ≤ n − 1 agree. Set µr,t (P ) = xr1 xt2 P ( dx), = 1, 2. IR2
Then setting θj = (uj , vj ) in (7.5.4) yields k k =0
uj vjk− [µ,k− (P1 ) − µ,k− (P2 )] = 0,
j = 1, . . . , n; k ≥ 0. Now, setting zj = vj /uj in the last equation leads to k k =0
zjk− [µ,k− (P1 ) − µ,k− (P2 )] = 0,
(7.5.6)
j = 1, . . . , n. Since no two of the directions θ1 , . . . , θn are collinear, the points z1 , . . . , z2 are distinct. Hence from (7.5.6) we find that the following polynomial of degree k of the variable z, k k l=0
z k− [µ,k− (P1 ) − µ,k− (P2 )] ,
(7.5.7)
has n distinct roots z1 , . . . , zn . If n ≥ k + 1, then this is possible only if all coefficients of (7.5.7) are equal to zero, that is, µ,k− (P1 ) = µ,k− (P2 ), = 0, . . . , k; k = 0, . . . , n − 1. So, for any unit vector t, and k = 0, 1, . . . , n − 1, k (t, x) P1 ( dx) = (t, x)k P2 ( dx). (7.5.8) IR2
IR2 (t)
Denote by P the marginal of P ( = 1, 2) in the direction t, and by ϕ (τ ; t)(τ ∈ IR) its characteristic function. By assumption, the support of (t)
P1
is in the segment [−1, 1]. Then (7.5.8) is equivalent to (k)
(k)
ϕ1 (τ ; t)|τ =0 = ϕ2 (τ ; t)|τ =0 ,
k = 0, . . . , n − 1,
(7.5.9)
(k)
where ϕ (τ ; t) is the kth derivative of ϕ (τ ; t) with respect to τ ( = 1, 2). A Taylor expansion now gives ϕ1 (τ ; t) − ϕ2 (τ ; t) =
(7.5.10)
s−1 (k) (s) ϕ (0; t) − ϕ (0; t) 1
k=0
2
k!
(k)
τk +
(s)
τ ; t) − ϕ2 ( τ ; t) s ϕ1 ( τ s!
7.5 Closeness of Measure on a Finite Number of Directions
47
for some τ ∈ (0, τ ). From (7.5.9), the first sum on the right-hand side of (7.5.10) is equal to zero. Since s is an even number, (s) |ϕ ( τ ; t)|
≤
z
s
(t) P ( dx)
1 (t) = z s P1 ( dz) ≤ 1,
= 1, 2.
−1
IR
Thus for all τ ∈ IR, |ϕ1 (τ ; t) − ϕ2 (τ ; t)| ≤ 2
τs . s!
1
s+1 ;(3) then Choose T = ( s! 2)
1 2 s+1 sup |ϕ1 (τ ; t) − ϕ2 (τ ; t)| ≤ . s! |τ |≤T 2 Corollary 7.5.3 Let θ1 , . . . , θn , n ≥ 2, be directions in IR2 no two of which are collinear. Suppose that the marginals of the probabilities P1 and P2 with respect to the directions θ1 , . . . , θn have moments up to the even order k ≤ n − 1. Then the marginals of P1 and P2 with respect to any direction t have the same moments up to order k. Corollary 7.5.4 Lemma 7.5.1 still holds if we replace the assumption that P1 and P2 have coinciding marginals with respect to the directions θj (j = 1, . . . , n) with the assumption that these marginals have the same moments up to order n − 1. To prove our main result we must relax the condition that the support of P1 is compact, assuming only the existence of all moments together with Carleman’s conditions for the definiteness of the moments problem. Set µk = sup (x, θ)k P1 ( dx), k = 0, 1, . . . , θ∈S 1 IR2
where S 1 is the unit circle, and let (s−1)/2
βs =
−
1
µ2j2j ,
j=1
where the number s is determined in Lemma 7.5.1; see (7.5.2). (3) This
s
choice of T is optimal, since 2 Ts! =
1 T
; see the definition (7.5.1) of λ-metric.
48
7. Relaxed or Additional Constraints
Theorem 7.5.5 Let θ1 , . . . , θn be n ≥ 2 directions in IR2 , no two of which are collinear. Suppose that the measure P1 has moments of any order. Suppose also that the marginals of the measures P1 and P2 in the directions θ1 , . . . , θn have the same moments up to order n − 1. Then there exists an absolute constant C such that(4) − 14
λ(P1 , P2 ) ≤ Cβs
(µ0 +
√
1/4
µ2 )
.
Proof: Let t be an arbitrary vector of the unit circle. From Corollary 7.5.3 (t) (t) we have that the marginals P1 and P2 have the same moments up to order s. From KKR (1988, p. 180) and Klebanov and Mkrtchian (1980), it follows that ⎞ ⎛ −1/4 (s−1)/2
1/4 $ (t) (t) −1/(2j) ⎠ µ0 (t) + µ2 (t) , ≤ C⎝ µ2j (t) λ P1 , P2 j=1
where µk (t) =
∞ −∞
(t)
uk Pi ( du), k = 0, . . . , s, i = 1, 2. The theorem now
follows from the obvious inequality µ2j (t) ≤ µ2j
(j = 0, 1, . . . , s/2).
2
Let us now consider the situation where the marginals of P1 and P2 in the directions θ1 , . . . , θn are not the same but are close in the metric λ. Theorem 7.5.6 Let θ1 , . . . , θn , n ≥ 2, be directions in IR2 , no two of which are collinear. Suppose that the supports of the measures P1 and P2 are in the unit disk, and that P1 and P2 have ε-coinciding marginals with respect to the directions θj (j = 1, . . . , n); i.e.,
(θ ) (θ ) (7.5.11) λ P1 j , P2 j := min max max |ϕ1 (τ ; θj ) − ϕ2 (τ ; θj )|, 1/T ≤ ε. T >0
|τ |≤T
Then there exists a constant C depending on the directions θj (j = 1, . . . , n) such that for sufficiently small ε > 0, we have 1 + 1/s , (7.5.12) λ(P1 , P2 ) ≤ C 1/ ln ε . where s = 2 n−1 2 (4) That
is, C is independent of s, P1 , and P2 .
7.5 Closeness of Measure on a Finite Number of Directions
49
Proof: Set ψj (τ ) := ϕ1 (τ ; θj ) − ϕ2 (τ ; θj ), j = 1, . . . , n. For 0 < ε ≤ 1 we have sup|τ |≤1 |ψj (τ )| ≤ ε, cf. (7.5.11). Since the supports of the measures (θj )
(θj )
are subsets of [−1, 1], for any even number k ≥ 2 we have
(k) (k) |ϕ1 (0; θj )| + |ϕ2 (0; θj )| 2 (k) sup |ψj (τ )| ≤ ≤ . (7.5.13) k k! |τ |≤1 P1
and P2
Now we apply Corollary 1.5.1 in KKR (1988), which states that there exist constants Ck such that ()
sup |ϕj (τ )|
|τ |≤1
(7.5.14)
≤ Ck
k− k sup |ϕj (τ )|
|τ |≤1
sup |τ |≤1
(k) |ϕj (τ )|
k1 ,
= 0, 1, . . . , k.
Choosing k ≥ 2s, ≤ s, and applying (7.5.13), we obtain ()
sup |ϕj (τ )| ≤ Cs ε1/2 ,
= 0, 1, . . . , s; j = 1, . . . , n,
|τ |≤1
where Cs is a new constant depending on s only. In particular, ()
()
|ϕ1 (0; θj ) − ϕ2 (0; θj )| ≤ Cs ε1/2 ,
= 0, 1, . . . , s; j = 1, . . . , n,
or equivalently, (x, θj )k (P1 − P2 )( dx) ≤ Cs ε1/2 , 2
(7.5.15)
IR
k = 0, 1, . . . , s; j = 1, . . . , n. Following the notation in Lemma 7.5.1, we can rewrite (7.5.15) in the form for k = 0, . . . , s and j = 1, . . . , n, k k k− uj vj [µ,k− (P1 ) − µ,k− (P2 )] ≤ Cs ε1/2 .
=0
Thus, setting Rkj =
k k =0
zjk− [µ,k− (P1 ) − µ,k− (P2 )] ,
(7.5.16)
k = 2, . . . , s; j = 1, . . . , n; zj = vj /uj , we obtain 1/2 , |Rkj | ≤ Cε
(7.5.17)
depends on the directions θ1 , . . . , θn only. For any fixed k (k = where C 2, . . . , s) consider
50
7. Relaxed or Additional Constraints (k)
(i) the matrix Ak with elements aj =
k −1
k−(−1)
zj
, , j = 1, . . . , k+1;
(ii) the vector Bk with elements (k)
b
= µ−1,k−+1 (P1 ) − µ−1,k−+1 (P1 ),
= 1, . . . , k + 1;
(iii) the vector Dk with elements dj = Rkj , j = 1, . . . , k + 1. Then (7.5.16) has the form Ak Bk = Dk (k = 1, . . . , s − 1), while (7.5.17) 1/2 . The matrices Ak are invertible, and so yields Dk ≤ Cε 1/2 Bk ≤ A−1 , k Dk ≤ Cε
(7.5.18)
where the constant C depends on the directions θ1 , . . . , θn only. Inequality (7.5.18) shows that the first s − 1 moments of the two-dimensional distributions are close when ε > 0 is sufficiently small. Such an evaluation of closeness holds for the first s − 1 moments of the marginals corresponding to an arbitrary direction t; i.e., (x, t)k (P1 − P2 )( dx) ≤ Cε1/2 , 2 IR
k = 0, . . . , s − 1. Now we have
s−1 (j) ϕ1 (0; t) − ϕ2(j) (0; t) j 2 s |ϕ1 (τ ; t) − ϕ2 (τ ; t) ≤ τ + |τ | j! j=0 s! ≤
s−1 Cε1/2 j=0
j!
|τ |j +
2|τ |s 2|τ |s ≤ Cε1/2 e|τ | + . s! s!
% 1 & 1/2 s! s−1 Choose T = min ln 1 + Cε1/2 , 2 . Since t is arbitrary on the unit circle, we obtain 1 s−1 2 1/2 1/4 1/2 λ(P1 , P2 ) ≤ max C ε + Cε + , 1/T s! ≤ C [1/ ln(1/ε) + 1/s] , which proves the theorem.
2
Remark 7.5.7 The statement in Theorem 7.5.6 still holds if instead of the ε-coincidence of the marginals as in (7.5.11), we require the ε-coincidence of the moments up to order s of these marginals.
7.5 Closeness of Measure on a Finite Number of Directions
51
Theorems 7.5.5 and 7.5.6 can be generalized for probability measures defined on IRm . However, we cannot choose the directions θ1 , . . . , θn in an arbitrary way. Furthermore, to obtain the same order of precision in IRm , m > 2, corresponding to the n directions in IR2 , we need nm−1 directions. The results can be obtained by induction on the dimension m. We define next the set of directions we are going to use. Choose n ≥ 2 distinct real numbers u1 , . . . , un , all different from zero, and first construct the set of n two-dimensional vectors (1, u1 ), (1, u2 ), . . . , (1, un ). Then construct n2 three-dimensional vectors (1, uj1 , uj2 ), j1 , j2 = 1, . . . , n. Repeating this process, by the last step we have constructed a set of m-dimensional vectors (1, uj1 , uj2 , . . . , ujm−1 ),
j = 1, . . . , n; = 1, . . . , m − 1.
(7.5.19)
Denote these m-dimensional vectors by θ1 , . . . θN , where N = nm−1 . These inductive arguments lead to the following extensions of Theorems 7.5.5 and 7.5.6. Theorem 7.5.8 The results in Theorems 7.5.5 and 7.5.6 still hold if we consider the measures P1 and P2 in IRm , and we choose as directions the N = nm−1 vectors in (7.5.19). Further, s = 2[(n − 1)/2]. To prove this, it is sufficient to note that instead of the m-dimensional vectors, we can first consider a pair of one-dimensional probabilities; the first component is the distribution of the inner product of the projections of the vector x and the vector θj upon the (m − 1)-dimensional subspace, while the second is the law of the last coordinate of the vector x. This allows us to decrease the dimensionality by one. To complete the proof it is sufficient to apply inductive arguments. The bounds of the deviation between probability measures with coinciding marginals offers a solution to the computer tomography paradox as stated in Gutman et al. (1991): “It implies that for any human object and corresponding projection data there exist many different reconstructions, in particular, a reconstruction consisting only of bone and air (density 1 or 0), but still having the same projection data as the original object. Related nonuniqueness results are familar in tomography and are usually ignored because CT machines seem to produce useful images. It is likely that the ‘explanation’ of this apparent paradox is that point reconstruction in tomography is impossible.” Lemma 7.5.1 shows that although the densities of the probability measures P1 and P2 (given that such densities exist) can be quite distant from each other for any “large” number of coinciding marginals, yet the measures P1 and P2 themselves are close in the weak metric λ. Khalfin and Klebanov (1990) have analyzed this paradox and obtained some bounds for the closeness of probability measures with coinciding
52
7. Relaxed or Additional Constraints
marginals for specially chosen directions for the case of uniform distance between the smoothed densities of these measures. In tomography the observations are, in fact, integrals of body densities along some straight lines. Using quadratic formulas enables us to evaluate the moments of a set of marginals; these in turn make it possible to apply the results in this section (see Remark 7.5.7) to evaluate the precision of the reconstruction for densities. The classical theory of moments makes it possible to give numerical methods for reconstructing the probability measures using the moments (see, for example, Ahiezer (1961)).
7.6 Moment Problems with Applications to Characterization of Stochastic Processes, Queueing Theory, and Rounding Problems The theory of moment has a long history, which originated in the pioneering works of Shohat and Tamarkin (1943), Hoeffding (1955), Rogosinsky (1958), Ahiezer and Krein (1962), Karlin and Studden (1966), Kemperman (1968). It was also in the 1950s and ’60s that moment theory became a separate mathematical discipline. Currently, it is appropriate to talk about the moment problems as beeing a whole range of problems with applications to many mathematical theories. We refer to the monograph of Annastassiou (1993) for a recent survey on the developments in the theory of moments. In this section we present some applications of moment theory to probabilistic-statistical models. The results presented here are due to Anastassiou and Rachev (1992). For the proofs of the theorems, which are only stated but not proved in this section, we refer to Anastassiou (1993). First we shall state results on the following five moment problems: Moment problem 1: Find (7.6.1) sup |x − y|p µ( dx, dy), S ⊂ IR2 , p ≥ 1, µ
S
and
|x − y|p µ( dx, dy),
inf µ
S ⊂ IR2 ,
(7.6.2)
S
where the supremum (resp. infimum) is taken over the set of all probability measures µ with support in S having fixed marginal moments xi µ( dx, dy) = αi , y i µ( dx, dy) = βi , i = 1, 2, . . . , n. (7.6.3) S
S
7.6 Moment Problems of Stochastic Processes and Rounding Problems
53
Remark 7.6.1 Problem (7.6.2) with fixed marginal distributions µ(· × S) = µ1 (·),
µ(S × ·) = µ2 (·)
(7.6.4)
is indeed the Lp -Kantorovich problem on mass transportation (see Chapters 2 and 3). Moment problem 2: For given x0 ∈ IR and positive α find the Kantorovich radius sup E|X − x0 |α ,
(7.6.5)
where the supremum is over all random variables X with fixed moments EX = p and EX 2 = q. Remark 7.6.2 Problem 2 will be used in approximation of complex queueing models by means of deterministic models. Moment problem 3: Find sup [t]c µ( dt), A = [0, a] or [0, ∞),
(7.6.6)
A
and
inf
[t]c µ( dt),
A = [0, a] or [0, ∞),
(7.6.7)
A
over the set of all probability measures with support A having fixed rth moment tr µ( dt) = dr , r > 0, dr > 0, (7.6.8) A
where for a given nonnegative x the c-rounding (0 ≤ c ≤ 1) of x is defined by ⎧ ⎨ m if m ≤ x ≤ m + c, [x]c = ⎩ m + 1 if m + c < x ≤ m + 1. Remark 7.6.3 Moment problem 3 can be applied to the problems of rounding and apportionment; see Mosteller, Youtz, and Zahn (1967), Diaconis and Freedman (1979), Balinski and Young (1982), and Balinski and Rachev (1993). In the apportionment theory, c = 0 corresponds to the Adams method; c = 1/2 corresponds to the Webster method (or conventional rounding, or Mosteller–Youtz–Zahn “broken stick” rule of rounding); c = 1 corresponds to the Jefferson method; see Balinski and Young (1982).
54
7. Relaxed or Additional Constraints
Moment problem 4: Find (7.6.6) and (7.6.7) subject to (7.6.8) and tµ( dt) = d1 . (7.6.9) A
We next consider some infinite-dimensional analogues of the moment problems 1 and 2. Let C[0, 1] be the space of continuous functions on [0, 1] with the usual sup-norm x, and let X (C[0, 1]) be the space of r.v.s on a nonatomic probability space (Ω, A, P ) with values in C[0, 1]. Let M be the class of all strictly increasing continuous functions f : [0, ∞] → [0, ∞], f (0) = 0, f (∞) = ∞. Finally, let T be a set of finitely many points in [0, 1], 0 ≤ t1 < t2 < · · · ≤ tN ≤ 1.
(7.6.10)
Moment problem 5: Given h, gi ∈ M(i = 1, . . . , N ) find inf Eh(X − Y ),
(7.6.11)
where the infimum is over the set of all possible joint distributions of X and Y subject to the moment constraints Egi (|X(ti )|) = ai ,
Egi (|Y (ti )|) = bi .
(7.6.12)
Remark 7.6.4 This problem can be interpreted as follows. Having observations of two random processes (more precisely, we suppose the moments (7.6.12) are known), the goal is to evaluate the minimal possible distance Eh(||X −Y ||) between the processes X and Y . We shall determine the minimum in (7.6.11) and show that essentially this minimum can be achieved.
7.6.1
Moment Problems and Kantorovich Radius
In this section we state the solutions of moment problems 1 and 2; the proofs are given in Anastassiou and Rachev (1992) and the monograph Anastassiou (1993). Let S = [a, b] × [c, d] ⊂ IR2 and ϕ(x1 , x2 ) = |x1 − x2 |p , p ≥ 1. Suppose (α, β) ∈ S, and denote by U = U (ϕ, α, β) the supremum in (7.6.1) subject to x1 µ( dx1 , dx2 ) = α, x2 µ( dx1 , dx2 ) = β. (7.6.13) S
S
Theorem 7.6.5 The supremum U in (7.6.1) is given by U = Dδ + T,
(7.6.14)
7.6 Moment Problems of Stochastic Processes and Rounding Problems
where
|b − d|p + |a − c|p − |b − c|p + |a − d|p ,
D
:=
T
(1 − B)|b − c|p + (B + C − 1)|a − c|p + (1 − C)|a − d|p , d−β b−α , C := , := b−a d−c := max(0, 1 − B − C).
B δ
55
:=
Remark 7.6.6 Since ϕ is convex on any S ⊂ IR2 , then given (7.6.13), inf ϕ dµ = |α − β|p . µ
S
Next, consider nonbounded regions S. Namely, define for b ≥ 0 the following stripes in IR2 : S1b S2b Sb
:= {(x, y); y = x + b , where 0 ≤ b ≤ b}, := {(x, y); y = x − b , where 0 ≤ b ≤ b}, := S1b ∪ S2b .
We extend Theorem 7.6.5 to this type of unbounded region. Theorem 7.6.7 Assume that 0 < p ≤ 1. (i) If S = S1b or S2b , (α, β) ∈ S, then the supremum U in (7.6.1) is equal to U := U (ϕ, α, β) = |α − β|p . (ii) If S = S b , then U = bp . (iii) Let L be the lower bound (7.6.2) subject to (7.6.13). Then if S = S1b or S = S2b or S = S b , (α, β) ∈ S, we have L := L(ϕ; α, β) = bp−1 |α − β|. Next, consider another type of stripe in IR2 : For b, γ > 0, S1b S2γ S b,γ
:= {(x, y); y = x + b , where 0 ≤ b ≤ b}, := {(x, y); y = x − γ , where 0 ≤ γ ≤ γ}, := S1b ∪ S2γ .
Theorem 7.6.8 Let p ≥ 1. (i) If S = S1b , (α, β) ∈ S, then U := U (ϕ, α, β) = bp−1 (β − α).
56
7. Relaxed or Additional Constraints
(ii) If S = S2γ , (α, β) ∈ S, then U = U (ϕ, α, β) = γ p−1 (α − β). (iii) If S = S b,γ , (α, β) ∈ S, then U =
(bp − γ p )(β − α − b) + bp (b + γ) . b+γ
Next, we shall state the explicit solutions of Moment problem 2. So we will be interested in the following problem. For given x0 ∈ IR, α > 0, p ∈ IR, q > 0 (p2 ≤ q), −∞ ≤ a < b ≤ +∞, find the Kantorovich radius K
(7.6.15) := K(x0 ; α, p, q, a, b) := sup{E|X − x0 |α ; X ∈ [a, b] a.s., EX = p, EX 2 = q}.
Theorem 7.6.9 (Case (A): α ≥ 2, −∞ < a < b < +∞) Let x0 = (a + b)/2, a ≤ p ≤ b, and 0 ≤ q ≤ b2 + (a + b)(p − b). Then the Kantorovich radius K admits the following bound: α−2 " # b−a (a + b)2 q − p(a + b) + K ≤ . 2 4 Moreover, if there exist λ1 , λ2 ≥ 0, λ1 + λ2 ≤ 1, such that p
=
q
=
a+b b−a + (λ1 − λ2 ), 2 2 2 2 b − a2 (a + b) (b − a)2 + (λ1 − λ2 ) + (λ1 + λ2 ), 4 2 4
then K =
b−a 2
α−2 " # (a + b)2 q − p(a + b) + . 4
The next theorem gives an analogue of Theorem 7.6.9 when the X’s in (7.6.15) have unbounded support. Theorem 7.6.10 (Case (B): 0 < α ≤ 2, a = −∞, b = +∞) For any x0 ∈ IR, p ∈ IR, q > 0, p2 ≤ q, the Kantorovich radius K is given by K = K(x0 ; α; p, q) = (q − 2x0 p + x20 )α/2 . The rest of the results in this section treat various versions of Theorems 7.6.9 and 7.6.10. Theorem 7.6.11 (Case (C): 0 < α ≤ 2, −∞ < a < b < +∞) For any x0 ∈ (a, b); p ∈ IR, p2 ≤ q, K = sup{E|X − x0 |α ; X ∈ [a, b] a.s., EX = p, EX 2 = q}. Set P = p − x0 , Q = q − 2x0 p + x20 , A(x0 ) = a − x0 , B(x0 ) = b − x0 , C(x0 ) = min(−A(x0 ), B(x0 )).
7.6 Moment Problems of Stochastic Processes and Rounding Problems
57
(i) If 0 ≤ Q ≤ C 2 (x0 ), then K = Qα/2 . (ii) If Q > C 2 (x0 ) and (A(x0 ) + B(x0 ))P − Q − A(x0 )B(x0 ) ≥ 0, then K ≤ Qα/2 . Theorem 7.6.12 (Case (D): 1 ≤ α ≤ 2, −∞ < a < b ≤ +∞) For any p ∈ IR, p2 ≤ q, a ≤ p ≤ b, set P = p − a, Q = q − 2ap + a2 , B = b − a. Suppose Q ≤ BP . Then K := sup{E|X − a|α ; X ∈ [a, b], EX = p, EX 2 = q} = p2−α Qα−1 . Theorem 7.6.13 (Case (E): 1 ≤ α ≤ 2, −∞ ≤ a < b < +∞) For any p ∈ IR, p2 ≤ q, a ≤ p ≤ b, set P = p − b, Q = q − 2bp + b2 , θ = a − b. Suppose Q ≤ θP . Then K := sup{E|X − b|α ; X ∈ [a, b], EX = p, EX 2 = q} = p2−α Qα−1 .
7.6.2
Moment Problems Related to Rounding Proportions
Here we state results on explicit solutions of Moment see (7.6.6) and (7.6.9). To this end recall the definition c ≤ 1): For any x ≥ 0, ⎧ ⎨ m, if m ≤ x < m + c, [x]c = ⎩ m + 1, if m + c ≤ x < m + 1, ⎧ ⎨ 0, if 0 ≤ x < c, = ⎩ m, if m − 1 + c < x ≤ m + c,
problems 3 and 4; of c-rounding (0 ≤
m ∈ IN ∪ {0},
m ∈ IN.
The next four theorems deal with problem 3.(5) Theorem 7.6.14 Let c ∈ (0, 1), r > 0, 0 < a < ∞, d > 0, and U := U[·]c (a, r, d) := sup{E[X]c ; 0 ≤ X ≤ a a.s., (EX r )1/r = d}. Set n = [a]. (I) If n + c < a, n + c ≤ d ≤ a, then U = n + 1. (II) If n + c ≥ a, n − 1 + c ≤ d ≤ a, then U = n. (5) In
the sequel, the underlying probability space is assumed to be nonatomic, and thus the space of laws of nonnegative r.v.s coincides with the space of all Borel probability measures on IR+ .
58
7. Relaxed or Additional Constraints
(III) If 0 < a ≤ c, then U = 0. ln 2 (< 1), n + c < a, and 0 ≤ d ≤ n + c, then ln(1 + 1/c) U = (n + 1)dr (n + c)−r .
(IV) If 0 < r ≤
ln 2 , n + c ≥ a, and 0 ≤ d ≤ n − 1 + c, then ln(1 + 1/c) U = ndr (n − 1 + c)−r .
(V) If 0 < r ≤
(VI) If r ≥ 1 and 0 ≤ d ≤ c, then U = dr c−r . (VII) Suppose r ≥ 1. If either (a) n + c < a and k ∈ {1, . . . , n} is determined by k − 1 + c ≤ d < k + c, or (b) n + c ≥ a and k ∈ {1, . . . , n} is determined by k − 1 + c ≤ d < k + c, then U = k +
dr − (k − 1 + c)r ≤ 1 − c + d. (k − c)r − (k − 1 + c)r
The next theorem extends Theorem 7.6.14 to the case a = +∞. Theorem 7.6.15 Let 0 < c < 1, r > 0, d > 0, and U := U[·]c (r, d) := sup{E[X]c ; X ≥ 0 a.s., (EX r )1/r = d}. (I) If 0 < r < 1, then U = +∞. (II) If r ≥ 1 and 0 ≤ d ≤ c, then U = dr c−r . (III) Suppose r ≥ 1. Define k ∈ N by k − 1 + c ≤ d < k + c. Then U = k+
dr − (k − 1 + c)r ≤ 1 − c + d. (k + c)r − (k −+ c)r
The next two theorems are versions of the previous two; here we consider the lower bounds in c-rounding. Theorem 7.6.16 Let c ∈ (0, 1), r > 0, 0 < a < ∞, d > 0, and L := 1/r L[·]c (a, r, d) := inf{E[X]c ; 0 ≤ X ≤ a a.s., (EX r ) = d}. Set n = [a]. (I) If 0 < d ≤ c, then L = 0. (II) If c < a ≤ 1 + c and c ≤ d ≤ a, then L = (dr − cr )/(ar − cr ).
7.6 Moment Problems of Stochastic Processes and Rounding Problems
59
(III) Let 0 < r ≤ 1, n + c < a, and determine k ∈ {0, 1, . . . , n − 1} by k + c ≤ d < k + 1 + c. Then L = k+
dr − (k + c)r . (k + 1 + c)r − (k + c)r
(IV) If 0 < r ≤ 1, n + c < a, and n + c ≤ d ≤ a, then L = n+
dr − (n + c)r . ar − (n + c)r
The case a = +∞ is extended as follows. Theorem 7.6.17 Let c ∈ (0, 1), r > 0, d > 0, and L := L[·]c (r, d) := inf{E[X]c : X ≥ 0 a.s., (EX r )1/r = d}. (I) If r > 1, then L = 0. (II) If 0 ≤ r ≤ 1, 0 < d ≤ c, then L = 0. (III) If r = 1, c ≤ d < ∞, then L = d − c. (IV) If 0 < r ≤ 1, define k ∈ IN ∪ {0} by k + c ≤ d < k + 1 + c. Then L := k +
dr − (k + c)r . (k + 1 + c)r − (k + c)r
Next we pass to Moment problem 4, see (7.6.6)–(7.6.9). For simplicity we shall consider only special cases of c-rounding. For the general case we refer to Anastassiou and Rachev (1992) and Anastassiou (1993). First consider the conventional (Webster) rounding, or MYZ-rounding, [x] := [x]1 . Theorem 7.6.18 Let a > 0, 0 < r = 1, d1 > 0, dr > 0, and U = U[·] (a, r, d1 , d2 ) := sup{E[X]; 0 ≤ X ≤ a a.s. and EX = d1 , EX r = dr }. Let θ = [a]. (I) Set ∆r,a := ar +
ar − θr (d1 − θ). a−θ
Suppose that a = θ, θ ≤ d1 ≤ a, and either r > 1, dr1 ≤ dr ≤ ∆r,a , or 0 < r < 1 and ∆r,a ≤ dr ≤ dr1 . Then U = θ.
60
7. Relaxed or Additional Constraints
(II) Suppose 0 < θ = a and there are λ1 , λ2 ≥ 0 with λ1 + λ2 ≤ 1 and such that d1 = λ1 θ + λ2 a and dr = λ1 θr + λ2 ar . Then U =
(θr − ar )d1 + (a − θ)dr . a(θr−1 − ar−1 )
(III) Let θ ≥ 1 and suppose there exists k ∈ {0, 1, . . . , θ − 1} such that k ≤ d1 < k+1 and either r > 1 and dr,k := k r +[(k+1)r −k r ](d1 −k) ≤ dr ≤ θr−1 d1 or 0 < r < 1 and θr−1 d1 ≤ dr ≤ dr,k . Then U = d1 . For the case a = +∞ we have the following version of the above theorem. Theorem 7.6.19 Let 0 < r = 1, d1 > 0, dr > 0, and U := U[·] (r, d1 , dr ) := sup{E[X]; X ≥ 0 a.s. and EX = d1 , EX r = dr }. Suppose there exists a nonnegative integer k such that k ≤ d1 ≤ k + 1 and either r > 1 and dr ≥ dr,k := k r + [(k + 1)r − k r ](d1 − k) or 0 < r < 1 and 0 < dr ≤ dr,k . Then U = d1 . If we change in Theorems 7.6.18 and 7.6.19 the upper bound U to the corresponding lower bound, we obtain the following two theorems. Theorem 7.6.20 Let a > 0, 0 < r = 1, d1 > 0, dr > 0, and L := L[·] (a, r, d1 , dr ) := inf{E[X]; 0 ≤ X ≤ a a.s., EX = d1 , EX r = dr }. (I) Suppose there exist t1 , t2 , λ such that 0 ≤ t1 ≤ t2 ≤ 1, and d1 = (1 − λ)t1 + λt2 and dr = (1 − λ)tr1 + λtr2 . Then L = 0. (II) If 0 < a ≤ 1, then L = 0. (III) If 1 < a < 2 and there exist λ1 , λ2 > 0 with λ1 + λ2 ≤ 1 and such (dr − d1 ) that d1 = λ1 + λ2 a, dr = λ1 + λ2 ar , then L = . (ar − a) From now on assume that a ≥ 2, and let θ = [a]. (IV) Suppose ∆θ :=
(θ − 1)(ar − a) ≤ θ. (θr − θ)
(i) If d1 = λ1 + λ2 θ, dr = λ1 + λ2 θr for some λ1 , λ2 ≥ 0, λ1 + λ2 ≤ 1, then L = ∆θ . (ii) If d1 = λ1 θ+λ2 a and dr = λ1 θr +λ2 θr for some λ1 , λ2 ≥ 0, λ1 +λ2 ≤ 1, then L =
(θ − a(θ − 1))dr − (θr+1 − ar (θ − 1))d1 . θar − aθr
7.6 Moment Problems of Stochastic Processes and Rounding Problems
61
(V) Suppose ∆θ > θ. (i) If d1 = λ1 +λ2 θ+λ3 a and dr = λ1 +λ2 θr +λ3 ar for some λ1 , λ2 , λ3 ≥ 0, λ1 + λ2 + λ3 = 1, then L=
(θ − 1)(θ − a + 1)(dr −1) − ((θr −1)θ − (ar −1)(θ − 1))(d1 − 1) . (θ − 1)(ar − 1) − (a − 1)(θr − 1)
(ii) If d1 = λ1 + λ2 a, dr = λ1 + λ2 ar for some λ1 , λ2 ≥ 0, λ1 + λ2 ≤ 1, θ(dr − d1 ) . then L = ar − a (VI) Suppose θ > 1 and one of the following holds. (i) r > 1 and there exists k ∈ {1, . . . , θ − 1} such that k ≤ d1 k + 1 and dr,k
:= k r + ((k + 1)r − k r ) (d1 − k) ≤ dr (θr − 1)(d1 − 1) ; ≤ ∆r,θ := 1 + θ−1
(ii) 0 < r < 1 and there exists k ∈ {1, . . . , θ − 1} such that k ≤ d1 ≤ k + 1 and ∆r,k ≤ dr ≤ dr,k . Then L = d1 − 1. The special case a = +∞ is treated as follows. Theorem 7.6.21 Let 0 < r = 1, d1 > 0, dr > 0, and L := L[·] (r, d1 , dr ) := inf{E[X] : X ≥ 0 a.s., EX = d1 , EX r = dr }. (I) Suppose there exist 0 ≤ t1 ≤ t2 ≤ 1, 0 ≤ λ ≤ 1 such that d1 = (1 − λ)t1 + λt2 and dr = (1 − λ)tr1 + λtr2 . Then L = 0. (II) Suppose 0 < r < 1 and either d1 = λ1 + λ2 , dr = λ1 , λ1 ≥ 0, λ1 + λ2 ≤ 1 or d1 ≥ 1, 0 < d1 ≤ 1. Then L = d1 − dr . (III) Suppose 0 < r < 1 and for some integer k, k ≤ d1 < k + 1, and 1 ≤ dr ≤ (k r + ((k + 1)r − k r ))(d1 − k). Then L = d1 − 1. (IV) Suppose r > 1 and either d1 = λ1 , dr = λ1 + λ2 , λi ≥ 0, i = 1, 2, λ1 + λ2 ≤ 1 or 0 < d1 ≤ 1 and 1 ≤ dr . Then L = 0. (V) Suppose r > 1 and for some integer k, k ≤ d1 < k + 1, and (k r + ((k + 1)r − k r ))(d1 − k) ≤ dr . Then L = d1 − 1. Similar results are valid for Adams (c = 0) and Jefferson (c = 1) rules of roundings; see Anastassiou and Rachev (1992) and Anastassiou (1993).
62
7. Relaxed or Additional Constraints
7.6.3
Closeness of Random Processes with Fixed Moment Characteristics
The moment problems we are going to consider in this section may be viewed as extensions of Moment problem 1 (on page 52) for measures µ generated by the joint distribution of random processes. Namely, let the class M, the space X (C[0, 1]), and the set T = {t1 , . . . , tN } be defined by (7.6.10). The subject of this section is the following general version of problem (7.6.10)–(7.6.12). Moment problem 6: Given h, gi,j ∈ M (i = 1, . . . , N, j = 1, . . . , n) find the set of valued Eh(||X − Y ||) subject to the moment constraints Egi,j (|X(ti )|) = ai,j ,
Egi,j (|Y (ti )|) = bi,j ,
i = 1, . . . , N.
(7.6.16)
In particular, determine the bounds inf Eh(||X − Y ||), sup Eh(||X − Y ||),
(7.6.17)
given the constraints (7.6.16). One interpretation of the problem is as follows: Suppose we observe two continuous processes X and Y , only through the “window” T . Suppose for each point of the “window” we know some moment characteristics of X and Y . The problem is to determine the possible deviations between the processes outside the window. In particular, “What is the minimal distance between X and Y with given moment information (7.6.16)?” is just a special case of Moment problem 7.6.3. We start with the case n = 1 in (7.6.16); i.e., given h, gi ∈ M(i = 1, . . . , N ) and assuming that Egi (|X(ti )|) = ai ,
Egi (|Y (ti )|) = bi ,
i = 1, . . . , N,
(7.6.18)
we are interested in the range of Eh(||X − Y ||). The solution of this problem will be given under some assumptions of the following type: Assumption A(h, g): h ◦ g −1 (t) (t ≥ 0) is a convex function (here and in the sequel, f −1 stands for the inverse of f ∈ M). Assumption B(g): g −1 (Eg(|ξ + η|)) ≤ g −1 (Eg(|ξ|)) + g −1 (E(|η|)) for any ξ, η ∈ X (R) (here, X (R) is the set of all real-valued r.v.s.). Assumption C(g): Eg(|ξ + η|) ≤ Eg(|ξ|) + Eg(|η|) for any ξ, η ∈ X (R). Assumption D(h, g): limt→∞ h(t)/g(t) = 0.
7.6 Moment Problems of Stochastic Processes and Rounding Problems
63
Remark 7.6.22 Take the most interesting case: h(t) = tp , g(t) = tq (p > 0, q > 0). Then A(h, g) ⇔ p ≥ q, B(g) ⇔ q ≥ 1, C(g) ⇔ q ≤ 1, D(h, g) ⇔ q > p. Now let T = {0 ≤ t1 ≤ · · · ≤ tN ≤ 1} and take a = (a1 , . . . , aN ) ∈ N IRN + , b = (b1 , . . . , bN ) ∈ IR+ and g = (g1 , . . . , gN ) ∈ M to be fixed vectors. Denote by X (T, g, a), the space of all X ∈ X (C[0, 1]) satisfying the marginal moment conditions Egi (|X(ti )|) = ai (i = 1, . . . , N ), and let I{h, g, T, a, b} :=
inf{Eh(||X − Y ||); X ∈ X (T, g, a), Y ∈ X (T, g, b)}.
(7.6.19)
In the next four theorems we describe the exact range of values of Eh(||X − Y ||) under different conditions of type A–D. Theorem 7.6.23 Let A(h, gi ) and B(gi ) hold for any i = 1, 2, . . . , N . Then (7.6.20) (i) I{h, g, T, a, b} = sup h |gi−1 (ai ) − gi−1 (bi )| ; 1≤i≤N
(ii) for any ν ≥ I{h, g, T, a, b} there exist random processes Xν ∈ X (T, g, a) and Yν ∈ X (T, g, b) such that Eh(||Xν − Yν ||) = ν.
(7.6.21)
Proof: We shall split the proof into three claims. Claim 1: I{h, g, T, a, b} ≥ sup φi (ai , bi ), where φi (ai , bi ) := h(|gi−1 (ai )− gi−1 (bi )|).
1≤i≤N
Proof of Claim 1: Let X, Y ∈ X (C[0, 1]) and ξ := g(|X(ti ) − Y (ti )|). Then, by the Jensen’s inequality and A(h, gi ), h−1 (Eh(||X − Y ||)) ≥ h−1 (Eh(|X(ti ) − Y (ti )|)) = h−1 (Eh ◦ gi−1 (ξ)) ≥ h−1 ◦ h ◦ gi−1 E(ξ) = gi−1 E(ξ). Further, by B(gi ), h ◦ gi−1 (E(ξ)) ≥ h
gi−1 (Egi (|X(ti )|) − gi−1 (Egi (|Y (ti )|)
= h(|gi−1 (ai ) − gi−1 (bi )|),
64
7. Relaxed or Additional Constraints
which proves the claim. Claim 2: The infimum in the left-hand side of (7.6.19) is attained, and (7.6.20) holds. Y ∈ X (C[0, 1]) to be random polygonal Proof of Claim 2: Define X, lines with vertices at points 0, t1 , . . . , tn , 1 given by ⎧ ⎪ i , ω) = g −1 (ai ), Y (ti , ω) = g −1 (bi ), i = 1, . . . , N, ⎪ X(t ⎪ ⎨ ω) = Y (0, ω) = 0, (7.6.22) X(0, if t1 > 0, ⎪ ⎪ ⎪ ⎩ X(1, ω) = Y (1, ω) = 0 if tN < 1, ω ∈ Ω. For any ω ∈ Ω, ' ' ' ' 'X(·, ω) − Y (·, ω)' = sup |gi−1 (ai ) − gi−1 (bi )|, 1≤i≤N
and hence − Y ||) = Eh(||X
sup φi (ai , bi ).
(7.6.23)
1≤i≤N
Further, by (7.6.22), ∈ X (T, g, a), X
Y ∈ X (T, g, b).
(7.6.24)
Invoking (7.6.23), (7.6.24), and Claim 1, we complete the proof of the claim. Claim 3: (ii) is satisfied. Proof of Claim 3: Let τ ∈ (0, 1), t ∈ T . Define the r.v.s Xν and Yν in X (C[0, 1]) as follows: Xν (t) = X(t),
Yν (t) = Y (t)
for t = 0, t1 , . . . , tN , 1,
(7.6.25)
Y are determined by (7.6.22), and where X, Xν (τ ) = h−1 (ν),
Yν (τ ) = 0.
(7.6.26)
Next, let Xν (t), Yν (t) be a random polygonal lines with vertices at 0, t1 , . . . , tN , 1 and τ . Making use of Claim 2, we have ||Xν − Yν || = h−1 (ν) ≥ and thus (7.6.21) holds.
sup |gi−1 (ai ) − gi−1 (bi )|,
1≤i≤N
2
7.6 Moment Problems of Stochastic Processes and Rounding Problems
65
Theorem 7.6.24 Let A(h, gi ) and C(gi ) hold for any i = 1, . . . , N . Then (i)
I{h, g, T, a, b} = sup h ◦ gi−1 (|ai − bi |),
(ii)
for any ν ≥ I{h, g, T, a, b} there exist Xnν ∈ X (T, g, a) and Ynν ∈ X (T, g, b) such that Eh(||Xnν − Ynν ||) → Y as n → ∞.
1≤i≤N
Proof: Claim 1: I{h, g, T, a, b} ≥ sup ϕi (ai , bi ), where ϕi (ai , bi ) := h◦gi−1 (|ai − 1≤i≤N
bi |).
Proof of Claim 1: Let X, Y ∈ X (C[0, 1]). Then, as in Claim 1 of Theorem 7.6.23, by A(h, gi ), C(gi ), and Jensen’s inequality, we have h−1 (Eh(||X − Y ||)) ≥ gi−1 E[gi (|X(ti ) − Y (ti )|)] ≥ gi−1 (|Egi (|X(ti )|) − Egi (|Y (ti )|)|) = h−1 ◦ ϕi (ai , bi ), which proves the claim. Claim 2: For any ε > 0 there exists a pair (Xε , Yε ) ∈ X (T, g, a)×X (T, g, b) such that Eh(||Xε − Yε ||) = sup h ◦ gi−1 (|ai − bi | + ε) sup 1≤i≤N
1≤i≤N
|ai − bi | . (7.6.27) |ai − bi | + ε
Proof of Claim 2: Without loss of generality we can assume that ai > bi , i = 1, . . . , N . Let pi := (ai −bi )/(ai −bi +ε) and qi := 1−pi , i = 1, . . . , N . We rearrange the indices i so that p1 ≤ p2 ≤ · · · ≤ pN . Choose sets Ai ∈ A such that A1 ⊂ · · · ⊂ AN and P (Ai ) = pi by using the assumption of (Ω, A, P ) being a nonatomic space. More precisely, since (Ω, A, P ) is nonatomic, then for any B ∈ A and any λ ∈ [0, P (B)] there exists C = C(B, λ) ∈ A, C ⊆ B such that P (C) = λ (see Loeve (1977, p. 101)). Then the required sets Ak (k = 1, . . . , N ) are given by Ak = C(Ak+1 , Pk ),
k = 1, . . . , N
(AN +1 := Ω).
Further, for any ω ∈ Ω, define ⎧ ⎨ c := g −1 (a − b + ε), i i i i Xε (ti , ω) := ⎩ d := g −1 (b /q ), i i i i and
⎧ ⎨ 0, Yε (ti , ω) := ⎩ d, i
if ω ∈ Ai , if ω ∈ Ai .
if ω ∈ Ai , if ω ∈ Ai ,
66
7. Relaxed or Additional Constraints
We define (t, Xε (t))t∈[0,1] to be a random polygonal line with vertices (ti , Xε (ti )) and let Xε (0, ω) = 0 if ti > 0 and Xε (1, ω) = 0 if tN < 1 for any ω ∈ Ω. Analogously define the process Yε (t). Then Egi (|Xε (ti )|) = gi (ci )pi + gi (di )qi = ai , and Egi (|Yε (ti )|) = gi (di )qi = bi for any i = 1, . . . , N ; i.e., Xε ∈ X (T, g, a) and Yε ∈ X (T, g, b). Further, Eh(||Xε − Yε ||) = Eh[sup |Xε (t) − Yε (t)|],
(7.6.28)
t∈T
where
⎧ ⎨ c, i |Xε (ti , ω) − Yε (ti , ω)| = ⎩ 0,
if ω ∈ Ai ,
(7.6.29)
if ω ∈ Ai .
Since g ∈ M, then for any i = 1, . . . , N − 1, pi ≤ pi+1 ⇔ ai − bi ≤ ai+1 − bi+1 ⇔ ci ≤ ci+1 ; i.e., c1 ≤ c2 ≤ · · · ≤ cN . Hence, by (7.6.29) and A1 ⊂ A2 ⊂ · · · ⊂ AN , ⎧ ⎨ c , if ω ∈ A , N N (7.6.30) sup |Xε (t, ω) − Yε (t, ω)| = ⎩ 0, t∈T if ω ∈ A . N
Now, (7.6.28) and (7.6.30) imply that Eh(||Xε − Yε ||) = h(cN )pN , which is in fact the right-hand side of equality (7.6.27), and thus the claim is proved. Claims 1 and 2 prove the desired equality (i). Claim 3: (ii) is satisfied. Proof of Claim 3: Let τ ∈ (0, 1), τ ∈ T . Using the same notations as in Claims 1 and 2 we define Xν (ti , ω) Xν (τ, ω)
:= Xε (ti , ω), Yν (ti , ω) := Yε (ti , ω), ⎧ ⎨ h−1 (ν), if ω ∈ A , N = ⎩ 0, if ω ∈ AN ,
Yν (τ, ω)
=
ν > h(cN )
=
0
ω ∈ Ω,
for any ω ∈ Ω, and ε > 0 is chosen so small that sup h ◦ gi−1 (|ai − bi | + ε).
1≤i≤N
We define the random broken lines Xν and Yν with vertices Xν (ti ), Yν (ti ), i = 1, . . . , N and Xν (0) = Yν (0) if ti > 0; Xν (1) = Yν (1) = 0 if tN < 1. Hence, as in Claim 2 we conclude that ⎧ ⎨ max(c , h−1 (ν)), if ω ∈ A , N N ||Xν (·, ω) − Yν (·, ω)|| = ⎩ 0, if ω ∈ A . N
Hence, Eh(||Xν − Yν ||) = νpN = ν aNaN−b−bNN+ε . This proves the claim.
2
7.6 Moment Problems of Stochastic Processes and Rounding Problems
67
Theorem 7.6.25 Let D(h, gi ) hold for any i = 1, . . . , N . Then I{h, g, T, a, b} = 0,
(7.6.31)
and for any ν > 0 there exist random processes Xnν ∈ X (T, g, a) and Ynν ∈ X (T, g, b) such that Eh(||Xnν − Ynν ||) → ν. Proof: Claim 1: For any n = 1, 2, . . . there exist Xn ∈ X (T, g, a) and Yn ∈ X (T, g, b) such that
Eh(||Xn − Yn ||) ≤
n
h(nbi )
i=1
bi ai + h(nai ) gi (nbi ) gi (nai
. (7.6.32)
Since gi ∈ M for n large enough, say n ≥ n0 , we can define disjoint sets Ain , Bin , Cin , and such that Ain + Bin + Cin = Ω and
P (Ain ) = cin :=
ai , gi (nai )
P (Bin ) = din :=
bi . gi (nbi )
Now, for any i = 1, . . . , N , n ≥ n0 , define ⎧ ⎪ ⎪ X (t , ω) = nai , ⎪ ⎨ n i
Yn (ti , ω) = 0,
if ω ∈ Ain ,
Xn (ti , ω) = 0, Yn (ti , ω) = nbi , ⎪ ⎪ ⎪ ⎩ X (t , ω) = Y (t , ω) = 0, n i n i
if ω ∈ Bin ,
(7.6.33)
if ω ∈ Cin .
Then Egi [|Xn (ti )] = gi (nai )cin = ai and Egi [|Yn (ti )|] = gi (nbi )din = bi ; i.e., X ∈ X (T, g, a) and Y ∈ X (T, g, b). Further, we define the random broken lines Xn (t), Yn (t) (t ∈ [0, 1]) in the way we already did in Theorems 7.6.23 and 7.6.24. Without loss of generality we can assume that a1 ≤ a2 ≤ · · · ≤ aN ≤ b1 ≤ b2 ≤ · · · ≤ bN . Then ||Xn − Yn || =
sup |Xn (t) − Yn (t)|, t∈T
(7.6.34)
68
7. Relaxed or Additional Constraints
⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ =
⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩
nbN
if ω ∈ BN,n ,
nbN −1 .. .
if ω ∈ BN −1,n \ BN,n ,
nb1
if ω ∈ B1,n \
naN
if ω ∈ AN,n \
N (
Bj,n ,
j=2 N (
Bj,n ,
j=1
.. .
)
na1
if ω ∈ A1,n \
0
if ω ∈
N (
N (
Aj,n ∪
j=2
Aj,n ∪
j=1
N (
N (
* Bj,n ,
j=1
Bj,n .
j=1
Hence, Eh(||Xn − Yn ||) ≤
N
h(nbj,n )dj,n +
j=1
N
h(naj,n )cj,n ,
j=1
which proves (7.6.32) and the claim. By D(h, gi ) (i = 1, . . . , N ) it follows that the right-hand side of (7.6.32) goes to 0 as n → ∞. Hence, (7.6.31) holds true. Claim 2: (ii) is valid. ∈ (0, 1), t ∈ T , and define νn
Let τ
=
1−P
ν
+
N j=1
, Cj,n
Xn,ν (τ ) = h−1 (νn ), Yn,ν (τ ) = 0. We define the random broken line Xn,ν (t) with vertices Xn,ν (tj ) = Xn (tj ) (see (7.6.33)) and Xn,ν (τ ) (cf. Claim 2 of Theorem 7.6.23). Following the same notations as in Claim 1, we have ⎛
N ,
h(nbi )P ⎝Bi,n \
⎞ Bj,n ⎠ ≤ h(nbi )din → 0
as n → ∞
(7.6.35)
Bj,n ⎦⎦ ≤ h(nai )cin → 0
(7.6.36)
j=i+1
and ⎡
⎡
h(nai )P ⎣Ai,n \ ⎣
N ,
j=i+1
Aj,n ∪
N ,
⎤⎤
j=1
as n → ∞. Hence, by (7.6.34)–(7.6.36), for n large enough,
7.6 Moment Problems of Stochastic Processes and Rounding Problems
Eh(||XN,ν − YN,ν ||) =
N
⎛
i=1
+
N
⎛
=
⎛
νn P ⎝Bi,n \
j=1
+
N
Bj,n ⎠
j=i+1
⎡
max(νn , h(nai ))P ⎝Ai,n \ ⎣
i=1 N
⎞
N ,
max(νn , h(nbi ))P ⎝Bi,n \
⎛
⎡
νn P ⎝Ai,n \ ⎣
= νn P ⎝
N ,
Aj,n ∪
j=i+1 N ,
Aj,n ∪
N ,
⎤⎞ Bj,n ⎦⎠
j=1
Bj,n ⎠
j=i+1
i=1
⎛
N ,
j=i+1
⎞
N ,
69
⎞
N ,
⎤⎞ Bj,n ⎦⎠
j=1
(Aj,n ∪ Bj,n )⎠ .
2
j=1
Theorem 7.6.26 For any gi ∈ M (i = 1, . . . , N ), (i)
I{0, g, a, b}
(7.6.37) := inf{P (X = Y ); X ∈ X (T, g, a), y ∈ X (T, g, b)} = 0;
(ii) for any ν ∈ (0, 1) there exists a sequence (Xnν , Ynν ) ∈ X (T, g, a)× X (T, g, b) such that P (Xnν = Ynν ) → ν Proof: (i)
as n → ∞.
Let cn ∈ A and P (cn ) =
1 n.
For any i = 1, . . . , N define
⎧ ⎨ X (t , ω) := g −1 (na ), Y (t , ω) := g −1 (nb ), if ω ∈ C , n i i n i i n i i (7.6.38) ⎩ X (t , ω) := Y (t , ω) = 0, if ω ∈ Cn . n i n i Then (7.6.38) determines the random polygonal lines Xn ∈ X (T, g, a) and Yn ∈ X (T, g, b). Since Xn (ti , ω) = Yn (ti , ω) = 0 whenever ω ∈ Cn and i = 1, . . . , N , then Xn (t, ω) = Yn (t, ω) = 0 if ω ∈ Cn and t ∈ [0, 1]. Hence, P (Xn = Yn ) ≥ P (Ω \ Cn ) = as desired.
n−1 → 1 n
70
7. Relaxed or Additional Constraints
(ii) Let 0 < ν < 1 and τ ∈ (0, 1) \ T . Choose A ∈ A with P (A) = ν and let ⎧ ⎪ ⎪ for any ω ∈ A, X (τ, ω) = 1, Ynν (τ, ω) = 0, ⎪ ⎨ nν (7.6.39) Xnν (τ, ω) = Ynν (τ, ω) = 0, for any ω ∈ A, ⎪ ⎪ ⎪ ⎩ X (t ) = X (t ), Y (t ) = Y (t ), i = 1, . . . , N, nν
i
n
i
nν
i
n
i
where Xn (ti ) and Yn (ti ) are given by (7.6.38). We construct the random broken lines Xnν and Xnν by using (7.6.39) (cf. Claim 3 of Theorem 7.6.23). From the implications
Xnν (·, ω) = Ynν (·, ω) ⇔
⇔
⎧ ⎪ ⎪ X (t , ω) = Ynν (ti , ω), ⎪ ⎨ nν i
i = 1, . . . , N,
⎪ ⎪ ⎪ ⎩ X (τ, ω) = Y (τ, ω) nν nν ω ∈ A ∩ Cn ,
it follows that P (Xnν = Ynν ) = P (Ω \ (A ∪ Cn )) → 1 − ν, which proves (ii), and the theorem as well.
2
As a consequence of Theorems 7.6.23–7.6.26 we obtain the following solution of Moment problem 7.6 for qi (t) = tqi (yi > 0) and h(t) = tp (P ≥ 0, with the convention t0 := I{t = 0}). Corollary 7.6.27 Let q = (q1 , . . . , qN ) (qi > 0), p ≥ 0, and
I{p, q, a, b} :=
inf {E||X − Y ||p ; X, Y ∈ X (C[0, 1]), E|X(ti )|qi = ai , E|Y (ti )|qi = bi , i = 1, . . . , N } .
Then ⎧ p 1/q 1/q ⎪ sup ai i − bi i , if p ≥ qi ≥ 1, i = 1, . . . , N, ⎪ ⎪ ⎪ ⎪ ⎨ 1≤i≤N p/q sup |ai − bi | i , if p ≥ qi , 0 < qi < 1, (7.6.40) I{p, q, a, b} = ⎪ 1≤i≤N ⎪ i = 1, . . . , N. ⎪ ⎪ ⎪ ⎩ 0, if 0 ≤ p ≤ qi , i = 1, . . . , N.
7.6 Moment Problems of Stochastic Processes and Rounding Problems
71
Moreover, for any ν > I{p, q, a, b} there exists a sequence (Xnν , Ynν ) ∈ X ([0, 1]) such that E||Xnν − Ynν ||p → ν
as n → ∞
and E|X(ti )|qi = ai ,
E|Y (ti )|qi = bi ,
i = 1, . . . , N.
Remark 7.6.28 Corollary 7.6.27 gives an explicit expression for I{p, q, a, b} if p and qi are subject to certain inequalities (cf. (7.6.40)) or if qi = q for all i = 1, . . . , N . The problem of an explicit description of I{p, q, a, b} for any p ≥ 0 and qi > 0 is still open.
7.6.4
Approximation of Queueing Systems with Prescribed Moments
In this section we discuss applications of Moment problem 1 (on page 52) to the problem of best approximation of a queueing system with known moment characteristics. As an example, suppose our “real” queueing system is of type G|G|1|∞ (for some acquaintance with the usual notations in queueing theory we refer to Borovkov (1984), Kalashnikov and Rachev (1990)). For this system, the sequences of nonnegative r.v.s (possibly dependent and nonidentically distributed) e = {en }n∈IN , s = {sn }n∈IN (IN = (1, 2, . . .)) are viewed as sequences of interarrival and service times. Looking at e and s as “input” of laws, we define (as the “output” flow) the sequence of waiting times w1 = 0,
wn+1 = (wn + sn − en )+ ,
n ∈ IN,
(7.6.41)
where (·)+ = max(0, ·). Since the distribution of w = {wn }n∈IN is not known, the aim is to approximate, model, or simulate the “real” system determined by the triplet (e, s, w) with a “simpler” queueing model (e∗ , s∗ , w∗ ). Assuming that the marginal distributions (the laws of ei , si ) are known, Borovkov (1984, Chapter 4) and Kalashnikov and Rachev (1990) examine different approximating models (e∗ , s∗ , w∗ ) and estimate the possible discrepancy between the “real” system (e, s, w) and the “ideal” model (e∗ , s∗ , w∗ ). Further, we shall relax the constraints “the laws of ei ’s and si ’s are known” by “certain moment characteristics of ei ’s and si ’s are fixed.” In this setup the solutions of Moment problem 1 are used in cases when the “ideal” model is not deterministic, say G|G|1|∞ but with simpler structure. We invoke Moment problem 2 (on page 53) when the approximation model has some deterministic components, like D|G|1|∞ (i.e., e∗j ’s are constants), or D|D|1|∞ (i.e., e∗j ’s and s∗j ’s are constants). Summarizing, we shall consider here the following two problems:
72
7. Relaxed or Additional Constraints
(a) Bounds for the deviation of output characteristics of two dependent queueing models. (b) Approximation of queueing systems by deterministic-type queueing models. Consider the following problem, which occurs in investigations stability of queueing models (see Kalashnikov and Rachev (1990, Chapter 5)). Suppose two queueing models of type G|G|1|∞, (e, s, w) and (e∗ , s∗ , s∗ ), with dependent characteristics are given. Here e = {en }n∈IN , s = {sn }n∈IN , w = {wn }n∈IN are, respectively, the sequences of interarrival, service, and waiting times. Assume that the components dj , sj , j ∈ IN of the “input flows” e and s are dependent and nonidentically distributed. The “output” flow is given by the sequence of waiting times (7.6.41). Suppose that the distribution of ej (resp. sj ) is concentrated on a compact interval [aj , bj ] (resp. [cj , dj ]). While this assumption is quite natural from the practical point of view, it is not used frequently in the literature, simply because it is easier to analyze queueing models with input distributions having unbounded support. We make similar assumptions for the model (e∗ , s∗ , w∗ ); in particular, it is assumed that a∗j ≤ e∗j ≤ b∗j , c∗j ≤ s∗j ≤ d∗j a.s. for all j ∈ IN. The input pairs (ej , e∗j ), (sj , s∗j ) of the two models are arbitrarily mutually dependent, the distributions of ej ’s, e∗j ’s, sj ’s, s∗j ’s are unknown. We assume that only the moments Eej = αj ,
Ee∗j = αj∗ ,
Esj = βj ,
Es∗j = βj∗
(7.6.42)
are given. Our problem to find a sharp bound for the deviation between the waiting times in both models. Let ϕk (en−k,n−1 , sn−k,n−1 ) (en−k,n−1 := (en−k , . . . , en−1 ), sn−k,n−1 := (sn−k , . . . , sn−1 )) be the waiting time for the nth arrival, assuming that the system is “free” at the moment n − k. In other words, ϕ(en−k,n−1 , sn−k,n−1 ) :=
(7.6.43)
max [0, en−1 − sn−1 , (en−1 − sn−1 ) + (en−2 − sn−2 ), . . . , (en−1 − sn−1 ) + (en−2 − sn−2 ) + · · · + (en−k − sn−k )] .
As a measure of deviation between the waiting times of the both systems we shall use δp (T ) = sup max Lp [ϕk (en,n+k−1 , sn,n+k−1 ), ϕk (e∗n,n+k−1 , s∗n,n+k−1 )], n∈IN 1≤k≤T
where p ≥ 1 and T ≥ 2 are fixed, and Lp (X, Y ) := {E|X − Y |p }1/p , p ≥ 1, X, Y ∈ X (R).
(7.6.44)
For random vectors we extend (7.6.44) as follows: Lp (X, Y ) = {E||X − Y ||p }1/p , X, Y ∈ X (RT ),
(7.6.45)
7.6 Moment Problems of Stochastic Processes and Rounding Problems
73
T where ||(x1 , . . . , xT )|| = i=1 |xj |. Since ϕk is a Lipschitz function with respect to the Minkowski norm || · ||, we have that for any k = 1, . . . , T , Lp [ϕk (en,n+k−1 , sn,n+k−1 ), ϕk (e∗n,n+k−1 , s∗n,n+k−1 )]
(7.6.46)
≤ Lp [(en,n+T −1 , sn,n+T −1 ), (e∗n,n+T −1 , sn,n+T −1 )] + Lp [(e∗n,n+T −1 , sn,n+T −1 ), (e∗n,n+T −1 , s∗n,n+T −1 )] ≤
n+T −1
Lp (ej , e∗j ) + Lp (sj , s∗j ) .
j=n
Now we invoke Theorem 7.6.5 to obtain sharp estimates of Lp (ej , e∗j ) and Lp (sj , s∗j ). Namely, Lp (ej , e∗j ) ≤ (Dj δj + sj )1/p ,
(7.6.47)
where Dj
:= Dj (aj , bj , a∗j , bj ∗) := |bj − b∗j |p + |aj − a∗j |p − |bj − a∗j |p − |aj − b∗j |p ;
Tj
:= Tj (aj , a∗j , bj , b∗j , αj , αj∗ ) (7.6.49) ∗ p ∗ p := (1 − Bj )|bj − aj | + (Bj + Cj − 1)|aj − aj | + (1 − Cj )|aj − b∗j |p ;
Bj
:=
δj
bj − αj , bj − aj
Cj :=
b∗j − αj∗ ; b∗j − a∗j
:= δj (aj , a∗j , bj , b∗j , αj , αj∗ ) := max(0, 1 − Bj − Cj ).
(7.6.48)
(7.6.50) (7.6.51)
Remark 7.6.29 If ej and e∗j are unknown, i.e., aj = a∗j = 0, bj = b∗j = +∞, then sup{Lp (dj , e∗j ); Eej = αj , Ee∗j = αj∗ } = ∞ (cf. Kuznezova-Sholpo and Rachev (1989)). In a similar way, j δj + Tj )1/p , Lp (sj , s∗j ) ≤ (D
(7.6.52)
j , δj , Tj are defined by (7.6.48)–(7.6.51), exchanging bj with dj , where D b∗j with d∗j , aj with cj , and a∗j with c∗j . In this way we have proved the following theorem. Theorem 7.6.30 For any p ≥ 1 and T = 2, 3, . . . , j δj + Tj )1/p . δp (T ) ≤ T sup (Dj δj + Tj )1/p + (D j≥1
(7.6.53)
74
7. Relaxed or Additional Constraints
The estimate is sharp or nearly sharp, since the inequalities (7.6.53) and (7.6.52) are the best possible bounds under the moment assumptions (7.6.42) (cf. Theorem 7.6.5), and also, the inequality (7.6.46) cannot be improved in the set of all possible input flows e, e∗ , s, s∗ . Next, we shall consider a much more general case than the single-channel models discussed above. Suppose the dynamics of a queueing system are determined by the transformation F from the set U of input flows U to the set V of output flows V . Let V0 represent the output at moment zero; V0 is assumed to be an -dimensional vector; i.e., V0 ∈ X (R ). It is quite general to assume that the input and the output flows have the form U = (V0 , U0 , U1 , . . .) and V = (V0 , V1 , . . .), where Uj ∈ X (Rk ). We endow U and V with the norms ||U ||U :=
∞
2−j ||Uj ||k,p + ||V0 ||,p
(7.6.54)
2−j ||Vj ||,p ,
(7.6.55)
j=0
and ||V ||V :=
∞ j=0
where p ≥ 1, ||Uj ||k,p
:=
(E||Uj ||pk )1/p ,
||Uj ||k
=
||(Uj , . . . , Uj || = |Uj | + · · · + |Uj |,
(1)
(k)
(1)
(k)
and ||Vj ||,p is defined in a similar way. Suppose the transformation F : U → V is determined by the set of mappings Fj : R × Rkj → R ,
j ∈ IN,
(7.6.56)
such that the output at “time” j is defined recursively: Vj = Fj (V0 , U0 , . . . , Uj−1 ).
(7.6.57)
A smoothness assumption on Fj is given by the Lipschitz condition ⎡ ⎤ j−1 (7.6.58) ||βj ||k ⎦ . ||Fj (α0 , β0 , . . . , βj−1 || ≤ cj ⎣||α0 || + j=0
A reasonably large number of queueing models meet conditions (7.6.56)– (7.6.58). Among them are the single-channel models G|G|1|∞, the multichannel models G|G|J|∞, and the multichannel–multiphased model (G|G|J1 ) → (G|J2 ) → · · · → (G|Jn ) (cf. Kalashnikov and Rachev (1990,
7.6 Moment Problems of Stochastic Processes and Rounding Problems
75
Chapter 5)). By (7.6.55), (7.6.57), and (7.6.58), ||Vj ||,p ≤ cj ||V0 ||,p + j−1 i=0 ||Ui ||k,p , and thus ||V ||V
≤ 2cj ||U ||U ⎡ ⎤ ∞ k (i) (i) ≤ 2cj ⎣ ||V0 ||,p + 2−1 ||Vj ||,p ⎦ . i=1
(7.6.59)
i=1 j=0
Combining (7.6.59) with Theorem 7.6.5 gives us a sharp bound on the deviation of two queueing models V = FU, V ∗ = FU ∗ , whose dynamics are determined by (7.6.54)–(7.6.58). Theorem 7.6.31 Suppose V = FU,
U ∈ U,
V ∈U
(7.6.60)
is a queueing model satisfying (7.6.54)–(7.6.58) such that (i)
a0
(i) cj
(i)
≤ V0
(i) Vj
≤
(i)
≤ b0 ≤
(i) dj
(i)
a.s.,
EV0
a.s.,
(i) EVj
(i)
= L0 , =
(i) βj ,
i = 1, . . . , , j = 0, 1, . . . , i = 1, . . . , k.
In addition to model (7.6.60) consider the same type model indexed by ∗ and satisfying the above two sets of inequalities with constants indexed by ∗. Then ||V − V ∗ ||V ⎡ ⎤ k ∞ 1/p
1/p (i) (i) (i) (i) (i) (i) δ + T ⎦, ≤ 2cj ⎣ + 2−j D D0 δ0 + T0 j j j i=1
i=1 j=0
where the D’s, δ’s, and T ’s are determined by the same formula as in (7.6.58)–(7.6.60), and (i)
D0
(i)
T0
(i)
δ0
(i) D j (i) Tj (i) δj
(i) (i) (i)∗ (i)∗ , = Di a0 , b0 , a0 , b0
(i) (i) (i)∗ (i)∗ (i) (i)∗ = Ti a0 , b0 , a0 , b0 , α0 , α0 ,
(i) (i) (i)∗ (i)∗ (i) (i)∗ = δi a0 , b0 , a0 , b0 , α0 , α0 ,
(i) (i) (i)∗ (i)∗ = Dj cj , dj , cj , dj ,
(i) (i) (i)∗ (i)∗ (i) (i)∗ = Tj cj , dj , cj , dj , βj , βj ,
(i) (i) (i)∗ (i)∗ (i) (i)∗ = δj cj , dj , cj , dj , βj , βj .
76
7. Relaxed or Additional Constraints
The rest of this section deals with Problem (b) (on page 72). Suppose again that the “real” queueing system is determined by the triplet (e, s, w), where w is given by the recursive equation (7.6.41). Often in practice one models the random input characteristics by replacing their random values with constants, usually equal to the corresponding means. In doing so, it is natural to investigate the deviation between the “real” output w and the modeled (“ideal”) output w∗ . (In the sequel, all quantities related to the approximating model will have the same notations as in the “real” system but superscribed with ∗.) The deviation between w and w∗ will be expressed by the Kantorovich metric p , defined here as follows: For X, Y ∈ X (IR∞ ),
p (X, Y ) := p (P X , P Y )
(7.6.61)
Y ); X, Y ∈ X (IR ), X = X, Y = Y }, p > 0, min{Lp (X, q Y ) := E dp (X, Y ) , q = min(1, 1/p) is the Lp -metric. In the where Lp (X, above definition, the space X (IR∞ ) consists of all random ∞sequences taking values in the metric space (IR∞ , d), where d(x, y) := j=1 2−j ||xj − yj ||. Since we have assumed that the underlying probability space is not atomic, the minimum in the right-hand side of (7.6.61) is equal to ⎧ ⎨ d(x, y)P ( dx, dy); P s are probabilities on IR∞ × IR∞ min ⎩ ⎫ IR∞ ×IR∞ ⎬ with fixed projections P X and P Y . ⎭ :=
∞
d
d
(n) (n) = X1 , X2 , . . . ∈ Xp (IR∞ ), X = (X1 , X2 , . . .) ∈ Xp (IR∞ ),
(n)
(n) (n) we have p X , X ≥ 2−j p Xj , Xj , and thus p (X , X) → 0 imFor X
(n)
d
(n)
plies the weak convergence of any j-component Xj = X and E|Xj |p → E|Xj |p . Further, we consider two types of approximating queues D|G|1|∞ (i.e., e∗j are constants) and D|D|1|∞ (i.e., e∗j and s∗j are constants). Similar results can be obtained if one examines the model G|D|1|∞ (i.e., s∗j are constants) as an approximation of the “real” queue G|G|1|∞. In both queues D|G|1|∞ and G|G|1|∞, the sequences of service times s∗ and s consist of dependent nonidentically distributed random variables. The next lemma shows that the outputs for the ideal and real models meet a lower bound of deviation if s∗ is chosen to have independent components. Let ε > 0 and X = (X1 , X2 , . . .) ∈ Xp (IR∞ ). The components of X are said to be ( p , ε)-independent if IND(X) := p (X, X) ≤ ε,
(7.6.62)
7.6 Moment Problems of Stochastic Processes and Rounding Problems
77
d
where the X i ’s (the components if X) are independent and X i = Xi (i ∈ IN). Lemma 7.6.32 Let the approximating model be of type D|G|1|∞. Assume that the sequences e and s of the queueing model G|G|1|∞ are independent. Then 1
p (w, w∗ ) 2
(7.6.63)
≤ IND(s) + IND(s∗ ) +
∞
2−j ( p (ej , e∗j ) + p (sj , s∗j )).
j=1
Proof: Using the recursive equations (7.6.41) for w and w∗ , we obtain 1 1 ∗ ∗ ∗ ∗ ∗ ∗ 2 d(w, w ) ≤ d(e, e ) + d(s, s ). Hence, 2 Lp (w, w ) ≤ Lp (e, e ) + Lp (s, s ). ∗ ∗ Since e and s (resp. e and s ) are independent, we have, passing to the minimal metrics, that 1
p (w, w∗ ) ≤ p (e, e∗ ) + p (s, s∗ ). 2
(7.6.64)
By (7.6.61) and since ej (j ∈ IN) are constants, we obtain the bound
p (e, e∗ ) =
∞
2−j Lp (ej , e∗j ) =
j=1
∞
2−j p (ej , e∗j ).
(7.6.65)
j=1
To estimate p (s, s∗ ) in (7.6.64) we use the ( p , ε)-independence characteristic defined in (7.6.62):
p (s, s∗ ) ≤ IND(s) + IND(s∗ ) + p (s, s∗ ),
(7.6.66) d
d
where s (resp. s∗ ) has independent components and sj = sj (resp. s∗j = s∗j ). We now invoke the “regularity” property of the Kantorovich metric: 1∞ 2 ∞ ∞
p X (n) , Y (n)
p X (n) , Y (n) ≤ (7.6.67) n=1
n=1
n=1
for sequences {X (n) }n≥1 ⊂ Xp (IR∞ ), {Y (n) }n≥1 ⊂ Xp (IR∞ ) of independent components. Let E j be a sequence with components all equal to zero except for the jth component, which equals 1. Then by (7.6.67), ⎛ ⎞ ∞ ∞
p (s, s∗ ) = p ⎝ (7.6.68) sE j , s∗ E j ⎠ j=1
≤
∞ j=1
j=1 j
p sE , s∗ E
j
=
∞ j=1
2−j p (sj , s∗j ).
78
7. Relaxed or Additional Constraints
Combining (7.6.64), (7.6.65), (7.6.66), and (7.6.68) proves the lemma.
2
The estimate (7.6.63) suggests that the approximating model should be chosen with s∗ having independent components. If this is the case, then IND(s∗ ) = 0, and the first problem is to estimate IND(s). Lemma 7.6.33 (a) Suppose that the only information known about the “real” service times are the moments ESjq1 = βj ,
Esqj 2 = γj ,
j ∈ IN,
(7.6.69)
and that the support of Fsj is [0, ∞). Then IND(s) ≤
∞
2−j ∆j ,
(7.6.70)
j=1
where
∆j :=
⎧ pq ⎪ ⎨ 2β 1/q1 , if 0 < p ≤ q1 , 1 ≤ q1 < q2 , j ⎪ ⎩ +∞,
1/q1
if 0 < q1 < q2 < p and βj
1/q2
= γj
(7.6.71) ,
and q = min(1, 1/p). (b) Suppose the support Fsj is the compact interval [cj , dj ], and βj = Esj . Then (7.6.70) holds with
∆j Tj
1/p j δj + Tj j = −2(dj − cj )p , D , p ≥ 1, where D d j − βj dj − β j = 2 1− , and δj = max 0, 1 − 2 . dj − cj dj − c j =
(7.6.72)
Proof: Assertion (a) follows from Corollary 2 of Kuznezova-Sholpo and Rachev (1989) and (b) from Theorem 7.6.5. 2
Lemma 7.6.34 Suppose that for every j ∈ IN the first two moments of ej are known: mj := Eej ,
(2)
mj
and let aj ≤ ej ≤ bj a.s.
:= Ee2j ,
σj2 := Var ej ,
(7.6.73)
7.6 Moment Problems of Stochastic Processes and Rounding Problems
79
(i) If p ≥ 2 and −∞ < aj < bj < ∞ and if e∗j is chosen to be the midpoint of [aj , bj ], then
p (ej , e∗j )
)" ≤
bj − aj 2
#p−2
(2) mj
− mj (aj + bj ) +
e∗2 j
* 1/p . (7.6.74)
(ii) Suppose 0 < p ≤ 2 and either −∞ = aj , +∞ = bj , or −∞ < aj < bj < ∞ and σj ≤ min[mj − aj , bj − mj ].
(7.6.75)
Then the “optimal” d∗j for the approximating model is given by e∗j = mj , and in this case
p (ej , e∗j ) ≤ σjpq ,
q = min(1, 1/p).
(7.6.76)
Proof: This follows from Theorems 7.6.10, 7.6.11, and 7.6.12 after some obvious arguments. The estimates (7.6.74) and (7.6.76) are sharp. 2 Lemma 7.6.35 (a) If 0 < p ≤ q1 , 1 ≤ q1 < q2 , then % & (2) ∗(2) 1 2 = Es∗q sup p (sj , s∗j ); nj = Esqj 1 , nj = Esqj 2 , n∗j = Es∗q j , nj j pq
1/q ∗1/q , q = min(1, 1/p). (7.6.77) = nj 1 + nj 1 (b) Suppose p ≥ 1, cj ≤ sj ≤ dj , c∗j ≤ s∗j ≤ d∗j a.s., and nj = Esj , n∗j = Es∗j . Then
p (sj , s∗j ) ≤
j δj + Tj D
1/p
,
(7.6.78)
j = Dj (cj , dj , c∗ , d∗ ), δj = δj (cj , dj , c∗ , d∗ , nj , n∗ ), where D j j j j j Tj = Tj (cj , dj , c∗j , d∗j , nj , n∗j ) are given by (7.6.48)–(7.6.51). Proof: Assertion (a) follows from Corollary 2 of Kuznezova-Sholpo and Rachev (1989) and (b) from Theorem 7.6.5. 2 Lemmas 7.6.32–7.6.35 lead us to the main result. Theorem 7.6.36 Let the approximating queueing model be of type D|G|1|∞. Assume that the sequences e and s of the “real” queueing model are independent. Then the Kantorovich metric between the sequences of waiting
80
7. Relaxed or Additional Constraints
times of the “approximating” and “real” models is bounded as follows:
p (w, w∗ )
(7.6.79)
≤ 2 IND(s) + 2 IND(s∗ ) +
∞
2−j+1 ( p (ej , e∗j ) + p (sj , s∗j )).
j=1
Each term in the right-hand side of (7.6.79) can be estimated as follows: (a) An appropriate choice for the approximating sequence of service times will be IND(s∗ ) = 0. (b) If (7.6.69) holds, a bound for IND(s) is given by (7.6.70). (c) If the means and variances of the ej ’s are known, then p (ej , e∗j ) can be estimated from above by (7.6.74). (d) The last term in (7.6.78), p (sj , s∗j ), can be estimated by (7.6.77) (resp. (7.6.78)), provided that the corresponding moment conditions hold. In the next theorem we shall omit the restriction that e and s are independent, but we shall assume that the approximating model is of completely deterministic type D|D|1|∞. Theorem 7.6.37 If the approximation queueing model is of type D|D|1|∞, then
p (w, w∗ ) ≤
∞
2−j+1 ( p (ej , e∗j ) + p (sj , s∗j )).
(7.6.80)
j=1
If the first moments of ej and sj are fixed, then p (ej , e∗j ) and p (sj , s∗j ) can be estimated as in Lemma 7.6.34. The proof is similar to that of Theorem 7.6.36.
7.6.5
Rounding Random Numbers with Fixed Moments
In this part we shall discuss the interplay between Moment problems 3 and 4 (on pages 53, 54) and the problem of rounding of random proportions. Given a vector X = (X1 , . . . , Xn ) of r.v.s consider the sum X1 +· · ·+Xn . If the Xi ’s are uniformly distributed on the simplex {(si ) ≥ 0; s1 +· · ·+sn = 1}, then they can be treated as proportions, and clearly Sn := X1 + · · · + Xn = 1. If Sn∗ is the sum of conventional roundings [X1 ]1/2 + · · · + [Xn ]1/2 , then Mosteller, Youtz, and Zahn (1967), Diaconis and Freedman (1979), and Balinski and Rachev (1993) have estimated the probability that Sn =
7.6 Moment Problems of Stochastic Processes and Rounding Problems
81
Sn∗ . Here we shall examine the closeness between Sn and Sn∗ in the case of i.i.d. observations Xi where only one or two moments are known. Suppose {Xi }i∈IN are nonnegative i.i.d. r.v.s with known moments EX1 = d1 ,
EX1r = dr .
(7.6.81)
The c-rounding [·]c (c ∈ [0, 1]) (see Section 7.6.2) gives us the sequence of i.i.d. roundings {[Xi ]c }i∈IN . Let Vi := [Xi ]c − Xi be the rounding error, and n Sn,c = i=1 Vi is the total rounding error. Then the normalized rounding error n−1 Sn,c converges by the LLN to E[X1 ]c − d1 . Our objective here is to find sharp bounds for the distribution function of n−1 Sn,c subject to (7.6.81). In other words, for a suitably chosen metric µ in the distribution functions space, the problem is to determine the “radius” of the set of probabilistic laws, i.e., Dn = Dn (µ) := sup µ n−1 Sn,c , E[X]c − d1 ; (7.6.82) r EX = d1 , EX = dr . In (7.6.82), X has the same distribution as the Xi ’s, and thus E[X]c − d1 = d
EV , where V = V1 . Clearly, there is a great variety of metrics µ from which one can choose in (7.6.82). We shall consider two metrics, one especially designed for the problem, one the ideal metric θs (s > 1) and the other the L´evy metric L. Note that Theorems 7.6.18–7.6.21 provide us with sharp bounds for E[X]c − d1 , in the case of the conventional rounding c = 12 .(6) In fact, with [X] = [X]1/2 , sup{E[X] − d1 ; EX = d1 , EX r = dr } inf{E[X] − d1 ; EX = d1 , EX r = dr }
= U − d1 ,
(7.6.83)
= L − d1 ,
(7.6.84)
where the exact values of U and L are given in Theorems 7.6.18–7.6.21. Next we can chose µ in the definition of Dn = Dn (µ) to be the L´evy metric L(X, Y ) =
inf{ε > 0; FX (x − ε) − ε ≤ FY (x) ≤ FX (x + ε) + ε for all x ∈ IR},
and thus for the distribution function Fn of n−1 Sn,c we obtain the following bounds: 0 ≤ Fn (x) ≤ Dn 0 ≤ Fn (x) ≤ 1 1 − Dn ≤ Fn (x) ≤ 1 (6) The
for 0 ≤ x ≤ L − Dn ,
(7.6.85)
for L − Dn ≤ x ≤ U − Dn , for U + Dn ≤ x.
(7.6.86) (7.6.87)
general case of c ≤ 1 was treated in Anastassiou and Rachev (1992).
82
7. Relaxed or Additional Constraints
From Theorems 7.6.18–7.6.21 it follows that the above bounds are sharp. Our next step is to find a good estimate for Dn = Dn (L). To this end we first estimate Dn (θs ) for θs (X, Y ) = sup |Ef (X) − Ef (Y )|.
(7.6.88)
Here, the supremum is taken over all bounded functions f on IR with q1 integrable second derivative |f |q ≤ 1, 1 < s < 2, q = 2−s . The next 1−s lemma shows that the θs -radius Dn (θs ) = O(n ) for all 1 < s < 2. We use the notation ∨ := max. Lemma 7.6.38 For 1 < s < 2, cn1−s , Dn (θs ) ≤
c :=
2 (c ∨ (1 − c))s . s
Proof: For any X and Y with equal means, θs (X, Y )
1 κs (X, Y ) s d d = X| s−1 − Y |Y |s−1 ; X := inf{E|X| X, Y = Y }. ≤
Therefore, from the ideality of θs (7) Dn (θs )
1 ≤ n1−s θs (V, EV ) ≤ n1−s κs (V, EV ) s 1 = n1−s E V |V |s−1 − (EV )|EV |s−1 s 1 ≤ n1−s . s
The latter follows since |V | = |X − [x]c | ∈ (0, c ∨ (1 − c)).
2
s−1 In the next theorem we bound Dn = Dn (L) in (7.6.82) as O n− 1+s . Theorem 7.6.39 For any 1 < s < 2, 0 < c < 1, 1
1−s
Dn (L) ≤ (4 c) 1+s n 1+s , where the constant c is defined as in Lemma 7.6.38.
(7) θ
s is an ideal s i |ci | θs (Xi , Yi ), for
metric of order s > 0; that is, θs ( i ci Xi , i ci Yi ) all independent Xi , Yi and constants ci ∈ IR.
≤
7.6 Moment Problems of Stochastic Processes and Rounding Problems
83
Proof: The following claim was proved by Grigorevski and Shiganov (1976) for the case s = 2; i.e., in (7.6.88) the functions f have a.e. f and |f | ≤ 1 a.e.; see also Maejima and Rachev (1987) and Rachev and R¨ uschendorf (1992). Claim: For any 1 < s < 2, θ(X, Y ) ≥
1 1+s L (X, Y ). 4
Proof of the Claim: Let L(X, Y ) > ε. Then there exists x0 such that either FX (x0 ) > FY (x0 +ε)+ε or FY (x0 ) > FY (x0 +ε)+ε. Say the first inequality takes place. Define ⎧ 1 for x ≤ x0 ; ⎪ ⎪ 2 ⎪ ⎪ 2(x − x0 ) ε ⎪ ⎪ for x0 < x ≤ x0 + ; ⎨ 1− ε 2 2 f0 (x) := ⎪ 2(x0 + ε + x) ε ⎪ ⎪ −1 + for x0 + ≤ x < x0 + ε; ⎪ ⎪ ε 2 ⎪ ⎩ 1 for x ≥ x0 + ε. Observe that |f0 (x)| ≤ 1, f (x) exists a.e., and ⎡ x +ε ⎤1/q 0 |f0 (x)|q dx⎦ = 8ε−s =: c(ε) ||f0 ||q = ⎣ x0
1 . Recalling the definition of θs , we have 2−s " # " # f0 (Y ) f0 (X) θs (X, Y ) ≥ E −E c(ε) c(ε) 1 (f0 (x) + 1) d [FX (x) − FY (x)] = c(ε) x 0 ∞ 1 = (f0 (x) + 1) dFX (x) + (f0 (x) + 1) dFX (x) c(ε)
for q =
−∞
x0
∞ (f0 (x) + 1) dFY (x) − (f0 (x) + 1) dFY (x) − −∞ x0 +ε ⎡ x ⎤ 0 ∞ 1 ⎣ (f0 (x) + 1) dFX (x) − (f0 (x) + 1) dFY (x)⎦ c(ε) x0 +ε
≥
−∞
≥
2 [FX (x0 ) − FY (x0 + ε)] c(ε)
x0 +ε
84
7. Relaxed or Additional Constraints
≥ =
2ε c(ε) 1 1+s ε . 4
Letting ε → L(X, Y ) completes the proof of the claim. Now the desired estimate follows from Lemma 7.6.38 and the claim.
2
8 Application of Kantorovich-Type Metrics to Various Probabilistic-Type Limit Theorems
We have discussed already in detail the Kantorovich metric as the solution of mass transportation and mass transshipment problems with a metric cost function; cf. Section 2.5 and Chapter 4. In Chapter 7 we studied generalized transshipment problems, leading to extensions of the Kantorovich metric to encompass a variety of ideal probability metrics. This chapter is devoted to applications of these metrics to the rate of convergence problem in the central limit theorem (CLT) and different summability methods for random vectors. We also discuss applications to the asymptotics of various rounding rules.
8.1 Rate of Convergence in the CLT with Respect to the Kantorovich Metric In this section, we investigate bounds for the rate of convergence in the CLT with respect to the Kantorovich metric for random variables with values in separable Banach spaces. In the first part, the rate in stable limit theorems for sums of i.i.d. random variables is considered. The method of proof is an extension of the Bergstr¨ om convolution method. All assumptions regarding the domain of attraction are given in a metric form. In the second part an extension is given to the martingale case. The proof is based on smoothing properties of suitable conditonal versions of the Kantorovich metric. Smoothing inequalities for the Kantorovich metric will be established, and
86
8. Probabilistic-Type Limit Theorems
the Bergstr¨om convolution method (cf. Zolotarev (1977, 1979, 1983, 1986), Senatov (1980), Sazonov (1981), Rachev and Yukich (1989, 1991), Rachev (1991c)) will be extended to the case of stable limit theorems and at the same time to the Kantorovich metric. All assumptions concerning the domain of attraction and the order of convergence are described in terms of finiteness conditions for certain convolution-type metrics. As a consequence of the results for the Kantorovich metric, one obtains rate of convergence results in stable limit theorems for martingales with respect to the Prohorov metric.(1) We start with the rate of convergence in the i.i.d. case. Consider a separable Banach space (U, · ) and the space X (U ) of U -valued r.v.s defined on a rich enough probability space. The r.v. ϑ ∈ X (U ) is said to be α-stable (0 < α ≤ 2) if −1/α
n
n
d
ϑi = ϑ,
(8.1.1)
i=1
where the ϑi ’s are i.i.d. copies of ϑ. We are interested in the rate of convergence of the normalized sum Zn = n−1/α
n
Xi
(8.1.2)
i=1
of i.i.d. r.v.s to ϑ with respect to the Kantorovich metric:
1 (X, Y ) :=
sup {|E(f (X) − f (Y ))|; f : U → IR bounded, (8.1.3) |f (x) − f (y)| ≤ x − y} .
Recall from Chapters 2 and 4 that 1 -convergence is equivalent to convergence in distribution and convergence of the moments E · (existence assumed); moreover, for U = IR, 1 (X, Y ) = |FX (x) − FY (x)| dx. The Prohorov metric π(X, Y ) :=
inf{ε > 0; P (X ∈ A) ≤ P (Y ∈ Aε ) + ε,
(8.1.4)
for all Borel sets A in U }(A := {x; |x − A|} < ε) ε
and the Kantorovich metric satisfy the well-known inequality π 2 ≤ 1 ,
(8.1.5)
(1) Some results in the literature are formulated more generally but use bounds involving moments of order ≥ 2 and, therefore, are restricted to the Gaussian case. For some recent literature we refer to Bolthausen (1982), H¨ aussler (1988), Bentkus et al. (1990), and Rackauskas (1990). Our method involves various extensions of an idea in Gudynas (1985) on suitably conditioned versions of probability metrics. The results in this section are based on Rachev and R¨ uschendorf (1994a).
8.1 Rate of Convergence in the CLT with Respect to Kantorovich Metric
87
which is in fact an immediate consequence of the Strassen and Kantorovich theorems. In particular, 1 -convergence rates imply convergence rates for π. Theorem 8.1.1 For any 0 < α < 1,
1 (Zn , ϑ) ≤ n1−1/α 1 (X1 , ϑ).
(8.1.6)
Proof: The result follows from (8.1.1)–(8.1.3) and the contraction properties of 1 ; in fact, if ϑi are i.i.d. copies of ϑ, then n −1/α E Zn − n ϑi ≤ n1−1/α E|X1 − ϑ1 |. (8.1.7) i=1
Next, we take in both sides of (8.1.7) the infimum over all joint distributions P X1 ,ϑ1 with fixed marginals P X1 and P ϑ . The result is (8.1.6) as desired. 2 Note that (8.1.6) is a general property for every ideal metric of order 1; see for example Zolotarev (1979). Recall that a probability metric µ is said to be ideal of order r if µ(X + Z, Y + Z) ≤ µ(X, Y )
(8.1.8)
for all r.v.s X, Y, Z such that Z is independent of (X, Y ) and µ(cX, cY ) = |c|r µ(X, Y )
for all c ∈ IR,
(8.1.9)
see Sections 6.3 and 6.4. Consider next the rate of convergence in
1 (Zn , ϑ) → 0
(8.1.10)
for 1 < α ≤ 2. Define the following ideal (smoothing) Kantorovich metric of order r > 1:
r (X, Y ) = sup hr−1 1 (X + hϑ, Y + hϑ),
r > 1,
(8.1.11)
h>0
and σ r (X, Y ) = sup hr σ(X + hϑ, Y + hϑ),
r > 0.
(8.1.12)
h>0
Here ϑ in (8.1.11) and (8.1.12) is assumed to be independent of X and Y , and σ is the total variation metric: σ(X, Y ) = sup{|E(f (X) − f (Y ))|; f : U → [0, 1] continuous} = 2 sup |P (X ∈ A) − P (Y ∈ A)|. A∈B(U )
(8.1.13)
88
8. Probabilistic-Type Limit Theorems
Note that r and σ r are ideal metrics of order r. Throughout this section
r stands for the smoothed 1 -metric of order r. The notion p has been used in previous sections for the minimal Lp -metric. So, we have increased the level of “ideality” for 1 and σ (recall that 1 is an ideal metric of order 1, while σ is ideal of order 0) by appropriate smoothing; see (8.1.11) and (8.1.12). The next theorem provides an estimate of the convergence rate in (8.1.10). In what follows C stands for an absolute constant that can be different in different places. Set 1 = 1 (X1 , ϑ), r = r (X1 , ϑ), σ = σ(X1 , ϑ), σ r = σ r (X1 , ϑ). We always assume r > 0. The results in this section are due to Rachev and R¨ uschendorf (1994). Theorem 8.1.2 Suppose that (a)
Eϑ < ∞;
(b)
1 + r + σ1 + σ r < ∞.
Then
1 (Zn , ϑ) ≤ C n1−r/α r + τr n−1/α ,
(8.1.14)
τr = max 1 , σ1 , σ 1/(r−α) . r
(8.1.15)
where
Remark 8.1.3 Zolotarev (1986, §5.4) provides a similar bound for
1 (Zn , ϑ) in the normal univariate case. Zolotarev’s bound contains ζr metrics in the right-hand side of (8.1.14), which can be easily estimated from above in the normal case. In the stable case, however, we need more refined bounds. The problem of finiteness of σ r was discussed in Rachev and Yukich (1989) (see also Section 8.3); for the finiteness of r see the next corollary. Further in this section the sum of any random variables X + Y d d + Y , where X and Y are independent and X = means X X, Y = Y . ϑ, and ϑi are defined as in (8.1.1) and satisfy (a). Proof: The proof is similar to that of Theorem 8.1.16, further in this section, which we shall give in detail. Here we give only a short sketch of the proof. It uses the following two properties of the metrics 1 , r , σ r ; see Zolotarev (1986, §5.4). Smoothing Property 1. For any X, Y ∈ X (U ),
1 (X, Y ) ≤ 1 (X + εϑ, Y + εϑ) + 2εEϑ.
(8.1.16)
8.1 Rate of Convergence in the CLT with Respect to Kantorovich Metric
89
Smoothing Property 2. For any X, Y, Z, W independent,
1 (X + Z, Y + Z) ≤ 1 (Z, W )σ(X, Y ) + 1 (X + W, Y + W ). (8.1.17) Next, let m = [ n2 ]; then by (8.1.16),
1 (Zn , ϑ1 ) ≤ 1 (Zn + εϑ, ϑ1 + εϑ) + Cε ϑ1 + X 1 + · · · + X n + εϑ ≤ 1 Zn + εϑ, n1/α m ϑ1 + · · · + ϑj + Xj+1 + · · · + Xn ·
1 + εϑ, n1/α j=1
ϑ1 + · · · + ϑj+1 + Xj+2 + · · · + Xn + εϑ n1/α ϑ1 + · · · + εm+1 + Xm+2 + · · · + Xn + 1 + εϑ, ϑ + εϑ 1 n1/α m = I0 + Ij + Im+1 . j=1
By (8.1.17),
X2 + · · · + Xn ϑ2 + · · · + ϑn −1/α −1/α , X + εϑ, n ϑ + εϑ σ n I0 ≤ 1 1 1 n1/α n1/α X1 + ϑ 2 + · · · + ϑ n ϑ 1 + · · · + ϑn + 1 + εϑ, + εϑ . n1/α n1/α Similar upper bounds are obtained for Ij , 1 ≤ j ≤ m + 1. Some of the terms obtained in this way can be estimated using the ideality properties of the For example, a term of the following form, ∆ = (m + metrics. +···+ϑn , ϑ , can be estimated by 1) 1 X1 +ϑn21/α 1 ∆ =
(m + 1) 1
≤ (m + 1) ≤ (m + 1)
n
−1/α
n n−1 n n−1
X1 +
r−1 α r−1 α
n−1 n
1/α
−1/α
ϑ, n
ϑ1 +
r (n−1/α X1 , n−1/α ϑ) n−r/α r ≤ Cn1−r/α r ,
where in the first inequality we use the obvious relation
r (X, Y ) ≥ hr−1 1 (X + hϑ, Y + hϑ).
n−1 n
1/α 2 ϑ
90
8. Probabilistic-Type Limit Theorems
X1 +···+Xj ϑ1 +···+ϑj we , j 1/α j 1/α 1−r/α −1/α Bj ≤ C( r j + τr j ).
For terms of the form Bj := 1
use an induction
argument to get the bound see the proof of Theorem 8.1.16.
For more details 2
Corollary 8.1.4 Suppose that U = IRK and that ϑ has a Fr´echet differentiable density pϑ and let C(ϑ) = sup |pϑ (y)(z)| dz < ∞. (8.1.18) z≤1
Suppose that Eϑ < ∞ and 1 + r < ∞. Then
1 (Zn , ϑ) ≤ C(n1−r/α r + τr∗ n−1/α ), where τ1∗ = max( 1 , r
1/(r−α)
(8.1.19)
).
For an integer r, r can be estimated from above by the ζr -metric (see (r) Zolotarev (1983, p. 294)): r ≤ Cζr if supz≤1 |pϑ (y)(z)| dz is finite. We shall discuss the finiteness of r in Section 8.5 in more detail. Proof: Claim 1. For any X and Y ∈ X (IRk ) and δ > 0, σ(X + δϑ, Y + δϑ) ≤ C(r)δ −r r (X, Y ),
(8.1.20)
with C(r) = 2(2−3)/α C(ϑ). To prove the claim we first use the obvious bound σ(X + δϑ, Y + δϑ) ≤ δ −r σ r (X, Y ).
(8.1.21)
Next, we show that for any δ > 0, σ(X + δϑ, Y + δϑ) ≤ δ −1 C(ϑ) 1 (X, Y ).
(8.1.22)
Indeed, by the ideality of σ and 1 it is enough to show (8.1.22) for δ = 1. Then σ(X + ϑ, Y + ϑ) ≤ sup f (x)(PX ( dx) − PY ( dx)) , |f |≤1
f (x + y)pϑ (y) dy. Since |f | ≤ 1, f (x) = sup |f (x)(z)| ≤ sup |pϑ (y)(z)| dy =: C(ϑ),
where f (x) =
z≤1
z≤1
8.1 Rate of Convergence in the CLT with Respect to Kantorovich Metric
91
and thus |f (x) − f (y)| ≤ C(ϑ)x − y, which obviously implies (8.1.22). To show (8.1.19) we use (8.1.21), (8.1.22), and the following bound: σ r (X, Y )
=
sup hr σ(X + hϑ, Y + hϑ) h>0
≤ sup hr 1 (X + 2−1/α hϑ, Y + 2−1/α hϑ) h>0
21/α C(ϑ) h
= C(ϑ)2r/α r (X, Y ). This completes the proof of the claim as well as that of (8.1.19).
2
Remark 8.1.5 (Rate of convergence in the CLT for random elements with LePage representation) Consider a symmetric α-stable U -valued random variable ϑ with LePage representation d
ϑ =
∞
−1/α
Γj
ηj Yj ,
(8.1.23)
j=1
where (i) Yj are i.i.d. with EY1 r < ∞; (ii) ηj are i.i.d. symmetric real-valued random variables with η1 α = (E|η1 |α )1/α < ∞; (iii) (Γj ) is a sequence of successive times of jump of a standard Poisson process; (iv) we assume that the three sequences are independent; see Ledoux and Talagrand (1991, Sect. 5.1) and Samorodnitsky and Taqqu (1994). Suppose X has a similar representation d
X =
∞
−1/α ∗ ∗ ηj Yj ,
Γj
(8.1.24)
j=1
where (Yj∗ ) and (ηj∗ ) are chosen as in (i) and (ii) with the only difference that they are not identically distributed. Write Zn , the normalized sum of i.i.d. copies Xi as in (8.1.2). Then Theorem 8.1.2 yields the following rate of convergence of Zn to ϑ in the 1 -metric. Corollary 8.1.6 Let 1 ∨ α < r < 2, and E||Y1 ||r + supj≥1 E||Yj∗ ||r + E||η1 ||r + supj≥1 E||ηj∗ ||r < ∞. Then
1 (Zn , ϑ) ≤ C(n1−r/α ∗r + τr∗ n−1/α ),
(8.1.25)
92
8. Probabilistic-Type Limit Theorems
% where ∗r := supj≥1 ( r (Yj∗ , Y1 ) + r (ηj∗ Y1 , η1 Y1 )) and τr∗ = max ∗1 , σ1∗ , & ∗1/(r−α) σr with σr∗ := supj≥1 (σ r (Yj∗ , Y1 ) + σ r (ηj∗ Y1 , η1 Y1 )). Proof: In view of (8.1.14), (8.1.15) we need only show the finiteness of σ r and r . For σ r = σ r (X, ϑ) we use the ideality of order r and the asymptotics −r/α EΓj ∼ j −r/α (j → ∞) to obtain σ r (X, ϑ)
=
−r/α
EΓj
r (ηj Yj , ηj∗ Yj∗ )
j≥1
⎛ ≤ ⎝
j≥1
⎞ −r/α ⎠ sup{E|ηj∗ |r EΓj j≥1
σ r (Yj∗ , Yj ) + σ r (ηj∗ Yj , ηj Yj )}
≤ C sup(σ r (Yj∗ , Y1 ) + σ r (ηj∗ Y1 , η1 Y1 )). j≥1
2
The same type estimate is valid for r .
Since in the LePage representations Yj , Yj∗ can have any high enough moment, examples with finite ∗r and τr∗ can be readily constructed. Take, d
for example, U to be a Hilbert space with basis (hm )m≥1 , and set Yj∗ = ∗ d ∗ ζj,m hm , Y1 = ζm hm , where (ζm )m≥1 , (ζj,m )m≥1 are sequences of m≥1
m≥1
independent random variables. Then, by the ideality of σ r , ∗ ∗ σ r (Yj∗ , Y1 ) ≤ σ r (ζj,m , ζm ) ≤ C κr (ζj,m , ζm ), m≥1
(8.1.26)
m≥1
where κr is the rth pseudomoment, κr (ζ ∗ , ζ) = r |x|r−1 |Fζ ∗ (x) − Fζ (x)|dx, see Zolotarev (1983). Similarly, ∗
r (Yj∗ , Y1 ) ≤ C κr (ζj,m , ζm ).
(8.1.27)
m≥1
The same example is valid if we relax the independence assumption to “independence in finite blocks,” requiring only that (ζ1+ , . . . , ζL+ ), = 0, L, 2L, . . ., are independent. Remark 8.1.7 (Finite-dimensional approximation) An alternative use of the explicit upper bounds of the smoothing metrics in the finite-dimensional case is to combine Theorem 8.1.2 with an approximation step by the finitedimensional case. To be concrete, let X, Y be C(S) valued processes, (S, d)
8.1 Rate of Convergence in the CLT with Respect to Kantorovich Metric
93
a totally bounded metric space. For ε > 0 let Vε denote a finite covering ε-net and let Pε X = (Xt )t∈Vε be the corresponding finite-dimensional approximation of X = (Xt )t∈S . If E sup |Xs − Xt | ≤ a(ε)
(8.1.28)
d(s,t)≤ε
and |Ys − Yt | ≤ b(ε),
E sup d(s,t)≤ε
then
1 (X, Y ) ≤ 1 (Pε X, Pε Y ) + a(ε) + b(ε).
(8.1.29)
So we can combine fluctuation inequalities (8.1.28) with the finite-dimensional bounds derived in (8.1.14) for the normalized sum Zn in order to choose an optimal rate of approximation ε = ε(n) → 0. A general and simple useful tool to derive fluctuation inequalities as in (8.1.28) is Pollard’s lemma, which applied to (8.1.28) yields $ 3 Nε max E sup |Xs − Xt | , (8.1.30) E sup |Xs − Xt | ≤ 1≤i≤Nε
d(s,t)≤ε
d(s,ti )<ε d(t,ti )<ε
where Nε = card (Vε ) and Vε = {ti , 1 ≤ i ≤ Nε }. The case α = 1 requires special consideration. We shall state a variant of Theorem 8.1.2 that will cover the case α = 1 but requires additional smoothing conditions on the law of the Xi ’s. The next theorem is based on the following lemma (see Rachev and Yukich (1989) or Rachev (1991c, Ch. 14)). Lemma 8.1.8 Let 0 < α ≤ 2, r > α, ar = −r/α
Ar (a) = 2
1 , 21+r/α (2r/α +3r/α )
and Ar =
ar . Suppose
δ0 := δ0 (X1 , ϑ) := max(σ, ϑr ) ≤ ar .
(8.1.31)
Then for any n ≥ 1 σ(Zn , ϑ) ≤ Ar δ0 n1−r/α ≤ 2−r/α n1−r/α .
(8.1.32)
Theorem 8.1.9 Suppose condition (8.1.23) holds and τ r := max( 1 , r ) < ∞. Then for
1 2
(8.1.33)
< α ≤ 2 and r > α,
1 (Zn , ϑ) ≤ Br,α τ r n1−r/α , where Br,α ≥ 8(r−1)/α + 2(2r/α + 3r/α ).
(8.1.34)
94
8. Probabilistic-Type Limit Theorems
The proof uses the following analogue of (8.1.17): For any independent X, Y, Z, W ,
1 (X + Z, Y + Z) ≤ 1 (X, Y )σ(Z, W ) + 1 (X + W, Y + W ). (8.1.35) The proof is similar to that of the smoothing inequality in Zolotarev (1986, §5.4) (see also Rachev (1991c, Theorem 15.2.2)) and thus is omitted. The theorem is of interest for 1 ≤ α ≤ 2, as for 0 < α < r < 1 we get from (8.1.6), 1 (Zn , ϑ) ≤ n1−1/α τ¯r . Our next objective is the extension of Theorem 8.1.2 to the martingale case. Let (Ω, A, P ) be a rich enough probability space, (Fi ) an increasing sequence of sub σ-algebras of A, and let (Xi , Fi ) be an adapted martingale difference sequence with values in a separable Banach space (U, · ); that is, E(Xi |Fi−1 ) = 0 a.s., i ∈ IN. For a given probability metric µ and a sub σ-algebra G ⊂ A define the G-dependence metric µ(·G) by µ(X, Y G) = sup µ(X + V, Y + V ), V ∈G
(8.1.36)
where V ∈ G denotes that V is a G-measurable random variable. This notion generalizes an idea due to Gudynas (1985). Lemma 8.1.10 If µ is homogeneous of order r, that is, µ(cX, cY ) ≤ |c|r µ(X, Y ),
(8.1.37)
then µ(·G) also is homogeneous of order r. We shall use the following metrics: r (·G), σ r (·G), where r , σ r are respectively the smoothed Kantorovich metric and the total variation metric (cf. (8.1.11), (8.1.12)). Lemma 8.1.11 Let the regular conditional distributions PX|G , PY |G exist. Then
r (X, Y G) ≤ E r (PX|G , PY |G )
(8.1.38)
σ r (X, Y G) ≤ Eσ r (PX|G , PY |G ).
(8.1.39)
and
Proof: Let ϑ be independent of X, Y , and G. Then
r (X, Y G) = =
sup r (X + V, Y + V ) sup sup sup hr−1 E(f (X + V + hϑ)
V ∈G
f L ≤1 h>0 V ∈G
8.1 Rate of Convergence in the CLT with Respect to Kantorovich Metric
95
−f (Y + V + hϑ))
≤ E sup sup sup hr−1 E(f (X + V + hϑ)|G)
−E(f (Y + V + hϑ)|G)
f L ≤1 h>0 V ∈G
= E sup sup sup hr−1 E(fV (X + hϑ)|G)
−E(fV (Y + hϑ)|G),
f L ≤1 h>0 V ∈G
where fV (·) = f (· + V ) is the translation by V , which is again a Lipschitz (y)| is the Lipschitz norm. We function, and where f L = supx =y |f (x)−f x−y arrive at
r (X, Y G) ≤ E sup sup hr−1 E(f (X + hϑ)|G) − E(f (Y + hϑ)|G) f L ≤1 h>0
= E r (PX|G PY |G ). The proof for the metric σ r is similar.
2
As a consequence we obtain the following regularity property of r and σr . Lemma 8.1.12 Let (Xi , Fi ) be a stochastic sequence and (Gi ) a decreasing sequence of sub σ-algebras such that Yj are Gi -measurable for j ≥ i. Suppose that the following condition holds: (c)
Xi and Gi+1 are conditionally independent given Fi−1 , and Yi and Gi+1 are conditionally independent given Fi−1 .
Then, for ci ∈ IR,
r
1 n
ci Xi ,
n
2 ≤
ci Yi
i=1
i=1
1 n
n
n
|ci |r E r (PXi |Fi−1 , PYi |Fi−1 )
(8.1.40)
i=1
and σr
i=1
c i Xi ,
i=1
2 ci Yi
≤
n
|ci |r Eσ r (PXi |Fi−1 , PYi |Fi−1 ),
i=1
assuming that the conditional distributions exist.
(8.1.41)
96
8. Probabilistic-Type Limit Theorems
Proof: By Lemma 8.1.10, 1 n 2 n
r ci Xi , ci Yi i=1
≤
n
i=1
r (c1 X1 + · · · + ci Xi + ci+1 Yi+1 + · · · + cn Yn ,
i=1
≤ = ≤
n
c1 X1 + · · · + ci−1 Xi−1 + ci Yi + · · · + cn Yn ) sup hr−1
i=1 h>0 n
sup
V ∈Fi−1 ∨Gi+1
1 (ci Xi + V + hϑ, ci Yi + V + hϑ)
r (ci Xi , ci Yi Fi−1 ∨ Gi+1 )
i=1 n
|ci |r r (X1 , Y1 Fi−1 ∨ Gi+1 ),
i=1
where Fi−1 ∨ Gi+1 is the σ-algebra generated by Fi−1 and Gi+1 . From Lemma 8.1.11 and the conditional independence assumption,
r (Xi , Yi Fi−1 ∨ Gi+1 ) ≤ E r (PXi |Fi−1 ∨Gi+1 , PYi |Fi−1 ∨Gi+1 ) = E r (PXi |Fi−1 , PYi |Fi−1 ). As for the metric σ r , the proof is similar.
2
Remark 8.1.13 If Yi are independent of Fi−1 , EYi = 0, then
r (PXi |Fi−1 , PYi ) ≤ Cr ζr (PXi |Fi−1 , PYi ) ≤ Cr κr (PXi |Fi−1 , PYi ),
(8.1.42)
where ζr is the Zolotarev metric and κr is the pseudo-difference moment (cf. (8.1.26) and Rachev (1991, p. 377)). In the α-stable case 1 < α < 2 and r = 2, the finiteness of κr implies that E(Xi |Fi−1 ) = EYi = 0,
(8.1.43)
which is fulfilled in the martingale case. In the normal case α = 2 and r = 3, the finiteness of ζr implies in the Euclidean case that the conditional covariance Cov (Xi |Fi−1 ) = Cov (Yi )
(8.1.44)
is almost surely constant. This and related conditions have been assumed in several papers on the martingale convergence theorem (cf. Basu (1976), Dvoretzky (1970), Bolthausen (1982), Butzer et al. (1983), H¨ aussler (1988), Rackauskas (1990)).
8.1 Rate of Convergence in the CLT with Respect to Kantorovich Metric
97
Lemma 8.1.14
r (X, Y ) ≥ ( 1 (X, Y ))r
r−1 r
r−1
1 2Eϑ. r
(8.1.45)
Proof: By the triangle inequality and from the definition of r ,
r (X, Y ) ≥ 1 (X + εϑ, Y + εϑ)εr−1 ≥ ( 1 (X, Y ) − 2εEϑ)εr−1 := ϕ(ε). 2
Maximizing ϕ(ε) with respect to ε, we obtain (8.1.45). In the next step we extend the smoothing inequality (8.1.17).
Lemma 8.1.15 Suppose that X, Z, Y, W are random variables with values in U such that (X, Z) is independent of (Y, W ) and Y, W are independent. Then
1 (X + Z, Y + Z) ≤ 1 (Z, W )σ(X, Y ) + 1 (X + W, Y + W ) + X), + 1 (Z + X, Z
(8.1.46)
d = is independent of X, and Z and Z where Z
σ(X + Z, Y + Z) ≤ σ(Z, W )σ(X, Y ) + σ(X + W, Y + W ) + X). + σ(Z + X, Z
(8.1.47)
Proof: By the triangle inequality,
1 (X + Z, Y + Z) =
sup |E [(f (X + Z) − f (X + W ))
f L ≤1
− (f (Y + Z) − f (Y + W ))]|
+ sup |E(f (X + W ) − f (Y + W ))|. f L ≤1
Furthermore, |E [f (X + Z) − f (X + W ) − (f (Y + Z) − f (Y + W ))]| = (E(f (Z + x)|X = x) − Ef (W + x)) dPX (x) − (Ef (Z + x) − Ef (W + x)) dPY (x) ≤ (E(f (Z + x)|X = x) − Ef (Z + x)) dPX (x) + Ef (Z + x)( dPX (x) − dPY (x))
98
8. Probabilistic-Type Limit Theorems
−
Ef (W + x)( dPX (x) − dPY (x))
+ X) + 1 (Z, W )σ(X, Y ). ≤ 1 (Z + X, Z 2
The proof of (8.1.47) is similar.
The last term in (8.1.46) is a measure of dependence of Z, X, which disappears if Z, X are independent. Making use of the smoothing properties, we next extend Theorem 8.1.2 to the martingale case. Let (Xi , Fi ) be a martingale difference sequence, n Zn = n1/α j=1 Xj , and as in (8.1.4) let ϑ, ϑi be independent, α-stable distributed r.v.s. For r > α we define
r = sup r (Xj , ϑj ), τr = sup E r (PXj |Fj−1 , Pϑj ), r = r ∨ τr , (8.1.48) j
j
τr = sup E r (PXj |Fj−1 , PXj ), τ4r = sup E r (PXj |G4j+1 , PXj ), j
j
where G4j+1 = σ(Xj+1 , Xj+2 , . . .), and σ r = supj σ r (Xj , ϑj ), the conditional distributions, are assumed to exist. Theorem 8.1.16 Suppose that Eϑ < ∞. Then
1 (Zn , ϑ) ≤ C(n1−r/α r + n−1/α tr ), 1 1 where tr = max 1 , σ1 , σ rr−α , τ4rr−α , τ1 .
(8.1.49)
Proof: Applying (8.1.16) we shall estimate 1 (Zn + εϑ, ϑ1 + εϑ). Set m = [ n2 ]. Then 2 1 n
1 Zn + εϑ, n−1/α ϑi + εϑ ≤
i=1
1 Zn + εϑ, n−1/α (ϑ1 + X2 + · · · + Xn ) + εϑ +
m
1 n−1/α (ϑ1 + · · · + ϑj + Xj+1 + · · · + Xn ) + εϑ,
j=1
⎛
n−1/α (ϑ1 + · · · + ϑj+1 + Xj+2 + · · · + Xn ) + εϑ
+ 1 ⎝n−1/α (ϑ1 + · · · + ϑm+1 + Xm+2 + · · · + Xn ) + εϑ,
8.1 Rate of Convergence in the CLT with Respect to Kantorovich Metric
n−1/α
n
99
⎞ ϑj + εϑ⎠
j=1
=: I0 +
m
Ij + Im+1 .
j=1
From the generalized smoothing inequality (8.1.46), X1 ϑ1 X2 + · · · + X n X2 + · · · + Xn I0 = 1 + + εϑ , + + εϑ n1/α n1/α n1/α n1/α X 2 + · · · + X n ϑ2 + · · · + ϑ n −1/α −1/α ≤ 1 , X + εϑ, n ϑ + εϑ σ n 1 1 n1/α n1/α X1 + ϑ 2 + · · · + ϑ n ϑ1 + · · · + ϑ n + εϑ, + εϑ + 1 1/α n n1/α 2 1 n X1 + · · · + Xn−1 + X X1 + · · · + X n + 1 + εϑ, + εϑ n1/α n1/α =: ∆1 + ∆2 + ∆3 , d n = n is independent of X1 , . . . , Xn−1 , ϑ. Similarly, where X Xn and X m j=1
Ij
≤
m j=1
1
Xj+2 + · · · + Xn ϑj+2 + · · · + ϑn , n1/α n1/α
ϑ1 + · · · + ϑj + Xj+1 ϑ1 + · · · + ϑj+1 + εϑ, + εϑ n1/α n1/α 1 m ϑ1 + · · · + ϑj + Xj+1 + ϑj+2 + · · · + ϑn
1 + εϑ, + n1/α j=1 2 n j=1 ϑj + εϑ n1/α m ϑ1 + · · · + ϑj + Xj+1 + · · · + Xn +
1 + εϑ, n1/α j=1 ·σ
j+1 + Xj+2 + · · · + Xn ϑ1 + · · · + ϑ j + X + εϑ n1/α
2
=: ∆4 + ∆5 + ∆6 . We first estimate ∆5 . By the ideality of r , 1 1/α m Xj+1 n−1 ∆5 =
1 + ϑ1 + εϑ, n n1/α j=1
(8.1.50)
100
8. Probabilistic-Type Limit Theorems
2 1/α ϑj+1 n−1 + ϑ + εϑ n n1/α 1 1/α 1/α 2 m n−1 n−1 ϑj+1 Xj+1 ≤
1 + ϑ, 1/α + ϑ n n n1/α n j=1 ≤ Cn1−r/α r . Similarly, by Lemma 8.1.12, ∆7
=: Im+1 (8.1.51) 1 1/α m+1 Xm+1 + · · · + Xn ϑ+ , ≤ 1 n n1/α 2 1/α m+1 ϑm+1 + · · · + ϑn ϑ+ n n1/α (1−r)/α n m+1 ≤ n−r/α E r PXi |Fi−1 , Pϑi n j=m+1 ≤
Cn1−r/α τr ,
and in the same way as for ∆5 , we obtain ∆2 ≤ Cn1−r/α r .
(8.1.52)
The remaining terms are dealt with by induction. Assume next that for j < n,
X1 + · · · + X j ϑ 1 + · · · + ϑ j r j 1−r/α + tr j −1/α , ≤ B
1 (8.1.53) j 1/α j 1/α and let
1/(r−α) 1/(r−α) ε = A max σ1 , σ 1/(r−α) , r , τ4r n−1/α r
(8.1.54)
with a constant A ≥ 0 that we shall fix later in the proof. Then ∆1
≤ BC(n1−r/α r + n−1/α tr )ε−1 n−1/α σ1 (X1 , ϑ) 1 ≤ BC(n1−r/α r + n−1/α tr ). A
In the same way,
∆4 ≤ CB r (n − m − 2)1−r/α + tr (n − m − 2)−1/α −r/α ∞ j Xj+1 ϑj+1 α +ε σr , · n n1/α n1/α j=1
(8.1.55)
(8.1.56)
8.1 Rate of Convergence in the CLT with Respect to Kantorovich Metric
101
≤ CB r n1−r/α + tr n−1/α ·
∞
j=1
σ r (Xj+1 , ϑj+1 ) j + Aα σ r (Xj+1 , ϑj+1 )
α/(r−α)
r/α
≤ CB( r n1−r/α + tr n−1/α )/Ar−α , using that εα ≥ Aα σ r α/(r−α) n−1 . To estimate ∆3 we apply the G-dependence metric 1 (·Fn−1 ): 1 2 n X Xn + εϑ, 1/α + εϑFn−1 (8.1.57) ∆3 ≤ 1 n1/α n 1 2 n Xn X ≤ 1 ≤ n−1/α E 1 (PXn |Fn−1 , PXn ) , Fn−1 n1/α n1/α ≤ n−1/α τ1 . Finally, we estimate ∆6 as follows: 1 1/α m j Xj+1 Xj+2 + · · · + Xn α ∆6 = +ε
1 + + ϑ, n n1/α n1/α j=1 1/α 2 j+1 j X Xj+2 + · · · + Xn + + ϑ + εα n n1/α n1/α (1−r)/α m j α ≤ +ε n j=1 1 2 j+1 Xj+1 Xj+2 + · · · + Xn X Xj+2 + . . . + Xn · r + , 1/α + n1/α n1/α n n1/α ≤
m (j + nεα )(1−r)/α
n(1−r)/α
j=1
≤ τr n−1/α
m
j+1 G4j+2 ) n−r/α r (Xj+1 , X
(j + nAα τ4rα/(r−α) n−1 )(1−r)/α
j=1
≤ Cn−1/α
1 τ41/(r−α) . Ar−1−α r
Gathering all the inequalities, we obtain B 1−r/α n
r + n−1/α tr + C2 n1−r/α r + C3 n−1/α τ1 A B + C4 r−α n1−r/α r + n−1/α tr + C5 n1−r/α r A
1 (Zn , ϑ) ≤ C1
102
8. Probabilistic-Type Limit Theorems
+ C6
1 τ41/(r−α) n−1/α + C7 n1−r/α τr Ar−1−α r
+ 2Eϑn−1/α max σ1 , σ r 1/(r−α) , r
1/(r−α)
, τ4r1/(r−α) .
4 ≤ 12 and then choose B Choose A large enough such that CA1 + ACr−α 1+α−r large enough such that C2 + C3 + C5 + C6 A + C7 + 2Eϑ ≤ B2 . Thus we obtain (8.1.49). 2
8.2 Application to Stable Limit Theorems Zolotarev (1976) introduced the ζr -metric as an extension of the Kantorovich metric. For any pair of random vectors X, Y on IRk and r = m + a it is defined by ζr (X, Y ) = sup{|E(f (X) − f (Y ))|; f ∈ Fr },
(8.2.1)
where Fr is the class of all functions f : IRk → IR such that f (m) (x) − f (m) (y) ≤ x − ya1 , 0 < a ≤ 1; f (m) denotes the mth Fr´echet derivative of f supplied with the usual supremum norm of multilinear functionals (cf. Zolotarev (1986, Section 6.3) and Rachev (1991, p. 264)), and x − y1 denotes the L1 -norm in IRk . Indeed, ζ1 is merely the Kantorovich metric; see (8.1.3). ζr is ideal of order r and therefore is suitable for analyzing the rate of convergence in various central limit theorems. (The definition of an ideal metric was given in (8.1.8) and (8.1.9).) A disadvantage of ζr is that only for integers r can ζr be estimated by difference pseudomoments from above, while for r ∈ / IN the known upper estimates involve absolute moments br = max(EXr , EY r ) or absolute pseudomoments of order r and therefore are not suitable for approximation by stable distributions of order α < 2. In IR an alternative ideal metric of order r that does not have this drawback of ζr was found by Maejima and Rachev (1987) and applied to prove convergence to self-similar processes; see also Rachev (1991c, Section 17.1). In this section we introduce a new ideal metric ϑs,p (with respect to summation of independent random vectors in IRk ), which generalizes the construction in Maejima and Rachev (1987). This ideal metric has the following properties. It is ideal of order r = s − 1 + p1 . It can be estimated from above by a Zolotarev-type metric and, what is more important, by a pseudo difference moment, which allows applications to stable distributions. Finally, it can be bound from below by the L´evy metric, and thus ϑs,p describes weak convergence of distributions. The degree of ideality of this
8.2 Application to Stable Limit Theorems
103
metric does not depend on the dimension. This is an important property, which is not satisfied by some obvious generalizations of one-dimensional ideal metrics of order greater than 1, see Sections 6.1, 6.3. We shall establish relations between ϑ1,p and ϑs,p and prove various smoothing inequalities. In the second part of this section we give an application to the rate of convergence in stable limit theorems. The upper bounds in the limit theorem are formulated in metric terms. We establish some new results ensuring the finiteness of these bounds and apply these results to show that random vectors in a neighborhood of the LePage decomposition of a stable law satisfy the central limit theorem with rate. Further applications are to the convergence of summability methods of i.i.d. random vectors and to the approximation by compound Poisson distributions. All these applications are based on the thorough analysis of the metric properties of ideal metrics having a structure close to that of the Kantorovich metric. The results in this section are due to Rachev and R¨ uschendorf (1992). We start with the construction of the ϑs,p -metric. Let X, Y ∈ X (IRk ), the class of k-dimensional random vectors, and define for s ∈ IN, 1 ≤ p ≤ ∞, ϑs,p (X, Y ) = sup{|E(f (X) − f (Y ))|; f ∈ Gs,p }.
(8.2.2)
Here Gs,p is the class of functions f : IRk → IR, such that for 1 ≤ i1 ≤ · · · ≤ is ≤ k and for 1 ≤ j ≤ k, x = (x1 , . . . , xj−1 , xj+1 , . . . , xk ) ∈ IRk−1, and q with p1 + 1q = 1,
Dis1 ,...,is f q,j (x) ⎧ ⎛ ⎞1/q ⎪ ⎪ ⎪ ⎪ q ⎨ ⎝ |Ds ⎠ i1 ,...,is f (x1 , . . . , xj , . . . , xk )| dxj = IR ⎪ ⎪ ⎪ ⎪ ess sup |Dis1 ,...,is f (x1 , . . . , xj , . . . , xk )| ⎩ xj ∈IR
≤ 1
(8.2.3) if q < ∞, if q = ∞
a.s. with respect to the Lebesgue measure.
Lemma 8.2.1 For any 1 ≤ p ≤ ∞ and s ∈ IN, the metric ϑs,p is an ideal metric of order r = s − 1 + p1 . Proof: If f ∈ Gs,p and z ∈ IRk , then f (· + z) ∈ Gs,p and hence ϑs,p (X + Z, Y + Z) ≤ ϑs,p (X, Y ) for any Z independent of X and Y . Further, when q < ∞, for any c ∈ IR, x ∈ IRk−1 , 1 ≤ j ≤ k, fc (x) := f (cx), ⎤1/q ⎡ Dis1 ,...,is fc q,j (x) = ⎣ |Dis1 ,...,is fc (x1 , . . . , xk )|q dxj ⎦ IR
(8.2.4)
104
8. Probabilistic-Type Limit Theorems
⎡ ⎤1/q = |c|s−1/q ⎣ |Dis1 ,...,is f (y1 , . . . , yk )|q dyj ⎦ IR
= |c|r Dis1 ,...,is f q,j , which yields the ideality of ϑs,p of order r. The case q = ∞ can be handled similarly. 2
Remark 8.2.2 Note that the direct generalization of the Maejima–Rachev (1987) construction leads to 1/q (s) q ϑs,p (X, Y ) = sup |E(f (X) − f (Y ))|; f (x) dx ≤ 1 , (8.2.5) which is an ideal metric of order s − kq = s − k(1 − p1 ). This unpleasant dependence on the dimensionality is avoided in the definition of ϑs,p by the restriction to one-dimensional integration in (8.2.3). We next show that ϑs,p is estimated from above by the following modification ζ r of the ζr -metric: ζ r (X, Y ) = sup{|E(f (X) − f (Y ))|; f ∈ F r },
(8.2.6)
where F r is the class of functions f : IRk → IR with f (m) (x) − f (m) (y) ≤ dα (x, y) :=
k
|xi − yi |a ,
(8.2.7)
i=1
m = 0, 1, . . . , a ∈ (0, 1], and r = m + a. In fact, (8.2.7) is equivalent to |f (x) − f (y)| ≤ dα (x, y), ∀x, y ∈ IRk ,
if m = 0,
(8.2.8)
and to sup 1≤i1 ≤···≤im ≤k
|Dim1 ,...,im f (x) − Dim1 ,...,im f (y)| ≤ da (x, y), if m ≥ 1.
Since x − ya1 ≤ da (x, y), we have ζr ≤ ζ r . Lemma 8.2.3 (a)
(8.2.9) For any integer r,
ϑr,1 = ζ r = ζr .
(8.2.10)
8.2 Application to Stable Limit Theorems
105
For any r > 0, if ϑs,p (X, Y ) < ∞, then
(b)
ϑs,p (X, Y ) ≤ ζ r ≤ where r = s − 1 +
1 p
da (x, 0)
|xi1 · · · xim | |PX − PY |( dx).
(8.2.12)
1≤i1 ,···,im ≤k
IRk
Proof: (a)
(8.2.11)
= m + a, and
v r (X, Y ) :=
Γ(1 + a) v r (X, Y ), Γ(1 + r)
It is enough to check that Gr,1 = F r . If f ∈ Gr,1 , then
sup 1≤i1 ,...,ir−1 ≤k
|Dir−1 f (x) − Dir−1 f (y)| 1 ,...,ir−1 1 ,...,ir−1
⎧ y ⎨ 1 r Di1 ,...,ir−1 ,1 f (t, x2 , . . . , xk ) dt ≤ sup 1≤i1 ,...,ir−1 ≤k ⎩ x1 y 2 r + Di1 ,...,ir−1 ,2 f (y1 , t, x3 , . . . , xk ) dt x2 y ⎫ k ⎬ r + · · · + Di1 ,...,ir−1 ,k f (y1 , . . . , yk−1 , t) dt ⎭ xk
≤ x − y1 =: d1 (x, y); i.e., f ∈ F r . Conversely, for f ∈ F r we have from (8.2.8),
sup 1≤i1 ,...,ir ≤k
=
|Dir1 ,...,ir f (x)|
sup
lim
|Dir−1 (f (x1 , . . . , xk ) − f (x1 , . . . , yr , . . . , xk ))| 1 ,...,ir−1 |xr − yr |
1≤i1 ,...,ir ≤k yr →xr
≤ 1 a.s.; i.e., f ∈ Gr,1 . (b)
If f ∈ Gs,p and 1 < p ≤ ∞, then similarly to the proof in (a),
sup 1≤i1 ,...,is−1 ≤k
|Dis−1 f (x) − Dis−1 f (y)| 1 ,...,is−1 1 ,...,is−1 k
(8.2.13)
yi
≤
sup
1≤i1 ,...,is−1 ≤k i=1 xi
|Dis1 ,...,is−1 ,i f (y1 , . . . , t, xi+1 , . . . , xk )| dt
106
8. Probabilistic-Type Limit Theorems
≤
k
Dis1 ,...,is f q,i (y1 , . . . , yi−1 , xi+1 , . . . , xk )|xi − yi |1/p
i=1
≤ da (x, y). For the second part of (b) first note that ϑs,p (X, Y ) < ∞ implies that for any 1 ≤ i1 ≤ · · · ≤ ij ≤ k, j ≤ s − 1, E(Xi1 · · · Xij − Yi1 · · · Yij ) = 0.
(8.2.14)
This follows, by taking fc (x) = c xi1 · · · xij , j ≤ s − 1, and the obvious inequality ϑs,p (X, Y ) ≥ supc>0 |E fc (X) − E fc (Y )|. Following the argument in the first part of this proof, (8.2.14) is also a consequence of the condition ζ r (X, Y ) < ∞. We obtain from the Taylor expansion and applying (8.2.14) with m = s − 1 that |E(f (X) − f (Y ))| 1 (1 − t)m−1 = (m − 1)! 1≤i1 ,...,im ≤k 0
· E Dim1 ,...,im f (tX)Xi1 · · · Xim − Dim1 ...im f (tY )Yi1 · · · Yim ≤
⎛ (1 − t)m−1 ⎝ |Dim1 ,...,im f (tx)xi1 · · · xim (m − 1)!
1
1≤i1 ,...,im ≤k 0
IRk
dt
⎞
− Dim1 ,...,im f (t0)xi1 · · · xim | |PX − PY |( dx)⎠ dt ≤
1
1≤i1 ,...,im ≤k 0
⎛
·⎝ ⎛ ≤
1 ⎝ (m − 1)! ·
(1 − t)m−1 (m − 1)!
⎞ da (tx, t0)|xi1 · · · xim | |PX − PY |(dx)⎠ dt
IRk
1
(1 − t)m−1 ta dt⎠
0
1≤i1 ,...,im ≤k
=
⎞
Γ(1 + α) v r (X, Y ). Γ(1 + r)
da (x, 0)|xi1 · · · xim | |PX − PY |( dx) IRk
2
8.2 Application to Stable Limit Theorems
107
r m a k k k Remark 8.2.4 (a) Since xr1 = |x | = |x | |x | i i i i=1 i=1 i=1
k a ≤ 1≤i1 ,...,im ≤k |xi1 . . . xim | , and on the other hand kxr1 ≥ i=1 |xi | k a 1≤i1 ,...,im ≤k |xi1 . . . xim | i=1 |xi | , we have vr (X, Y ) := xr1 |PX − PY |( dx) ≤ v r (X, Y ) ≤ k vr (X, Y ). (8.2.15) IRk
(b) By the same arguments as in (a) we have ζr ≤ ζ r ≤ k ζr .
(8.2.16)
In particular, by (8.2.11), ϑs,p is also estimated from above by the Zolotarev metric ζr (up to a constant). The following theorem gives an estimate of ϑs,p in terms of certain pseudomoments, which allows one to apply ϑs,p to stable distributions. For random vectors X, Y with densities uX , uY , define αs,p (X, Y ) =
k
i=1 1≤i1 ,...,is ≤k
·
y
1
t
,...,
IRk−1
⎡ 1 s−1 ⎣ |yi1 . . . yis | t−s−k (1 − t) (uX − uY ) (s − 1)! IR
⎤1/p
yk dt |yi |p dyi ⎦ t
0
dyi . . . dyi−1 dyi+1 . . . dyk .
If k = 1, then after some transformations we obtain αs,p (X, Y )
= =
Fs,X − Fs,Y p (8.2.17) ⎛ x ⎞1/p (x − t)s−1 ⎝ d(FX − FY )(t)|p dx ⎠ , (s − 1)! −∞
Fs,X (x) :=
1 E(x − X)s−1 + (s − 1)!
(see Maejima and Rachev (1987), Rachev and R¨ uschendorf (1990)). Indeed, αs,p is an ideal metric of order r. Representation (8.2.17) shows that αs,p depends only on the difference pseudomoments. A similar representation holds also for k ≥ 1. Theorem 8.2.5 αs,p is the upper bound for ϑs,p ; i.e., ϑs,p ≤ αs,p .
(8.2.18)
108
8. Probabilistic-Type Limit Theorems
Proof: By (8.2.14) and the Taylor expansion, E(f (X) − f (Y )) = IRk
1
1≤i1 ≤···≤is ≤k 0
(1 − t)s−1 (s − 1)!
· Dis1 ,...,is f (tx1 , . . . , txk )xi1 · · · xis dt d(FX − FY )(x) ⎡ 1 (1 − t)s−1 ⎣ = Dis1 ,...,is f (y1 , . . . , yk )yi1 . . . yis (s − 1)! 1≤i1 ,...,is ≤k 0 IRk ⎤
y −s−k · (uX − uY ) dy ⎦ dt. t t This implies, by making use of H¨older’s inequality, that |E(f (X) − f (Y ))| ≤
k s−1
i=1 j=0 1≤i1 ,...,ij ≤k,ij =i
|yi1 . . . yij | IRk−1
⎡ ⎣Di1 ,...,ij ,i,...,i f (y)
IR
1
s−1 (1 − t) y (uX − uY ) dt · t−s−k (s − 1)! t 0 ⎤ · |yi |s−j dyi ⎦ dy1 . . . dyi−1 dyi+1 . . . dyk
≤
k s−1
i=1 j=0 1≤i1 ,...,ij ≤k,ij =i
|yi1 . . . yij |Di1 ,...,ij ,i,...,i f q,i (y) IRk−1
' ' 1 ' '
y s−1 ' ' (1 − t) −s−k s−j ' (uX − uY ) |yi | dt' ·' t ' (s − 1)! t ' ' 0
p
dy1 . . . dyi−1 dyi+1 . . . dyk , 2
which is equivalent to the representation in (8.2.17). For random vectors in IRk we define the L´evy distance by L(X, Y ) =
inf{ε > 0; P (X ∈ Bx ) ≤ P (Y ∈ Bx (ε)) + ε,
(8.2.19)
P (Y ∈ Bx ) ≤ P (X ∈ Bx (ε)) + ε, ∀x ∈ IR }, k
8.2 Application to Stable Limit Theorems
109
where Bx := {y ∈ IRk ; yi ≤ xi , 1 ≤ i ≤ k} and Bx (ε) := {y ∈ IR ; y − Bx ≤ ε}, k
y =
1 k
21/2 yi2
;
i=1
note that P (X ∈ Bx ) = FX (x). L metrizes the topology of weak convergence. If X has a bounded density uX , then (X, Y ) := sup |FX (x) − FY (x)| 1 2 k ≤ 1+ sup uXi (x) L(X, Y ), i=1
x
(8.2.20) X = (X1 , . . . , Xk ).
Next, we establish that ϑs,p convergence implies weak convergence by providing a lower bound of ϑs,p in terms of L. Theorem 8.2.6 Let s = 1, 2, . . . , p ∈ [1, ∞], r = s − 1 + p1 . Then ϑs,p (X, Y ) ≥ a(s, k)Lr+1 (X, Y ), where a(s, k) :=
Vr 2s+k s!
,
Vr :=
(8.2.21)
(1 − x2 )r+1 dx.
(8.2.22)
{x≤1}
Proof: Let L(X, Y ) > ε. Then without loss of generality we can assume that for some z = (z1 , ..., zk ), P (X ∈ Bz ) − P (Y ∈ Bz (ε)) > ε.
(8.2.23)
We define gr (x) = (1 − x2 )r+1 + ,
x ∈ IRk (a+ = max(0, a)),
(8.2.24)
and “normalize” gr by g r (x) :=
gr (x) . Vr
(8.2.25)
Finally, we define the smoothed version of the indicator of Bz : %
ε & ε I x − y ∈ Bz g r (y) dy uε (x) = 2 2 IRk
=
k %
ε & 2 2 I y ∈ Bz gr (x − y) d y. ε 2 ε IRk
(8.2.26)
110
8. Probabilistic-Type Limit Theorems
Since 0 ≤ uε (x) ≤
g r (y) dy ≤ 1, we have
IRk
|fε | ≤ 1, where fε (x) := 2 uε (x) − 1; and furthermore, ⎧ ⎨ 1 if x ∈ B , z fε (x) = ⎩ 0 if x ∈ / Bz (ε).
(8.2.27)
(8.2.28)
In fact, we have for x ∈ Bz , %
ε & ε uε (x) = I x − y ∈ Bz g r (y) dy 2 2 IRk %
ε & ε = I x − y ∈ Bz , y ≤ 1 g r (y) dy 2 2 IRk = g r (y) dy = 1. IRk
Similarly, for x ∈ / Bz (ε), uε (x) = 0. In the next step we establish bounds on the derivatives of fε . To that purpose let Ls (f ) =
sup ess sup Dis1 ,...,is f q,i (x).
(8.2.29)
2ε−r . a(s, k)
(8.2.30)
1≤i≤k x∈IRk−1
Then Ls (fε ) ≤
To show (8.2.30), observe that Dis1 ,...,is fε (x) (8.2.31) k+s %
& 2 2 ε = 2 (x − y) d y Dis1 ,...,is g s I y ∈ Bz ε 2 ε IRk
s %
ε & 2 = 2 Dis1 ,...,is g s (y) dy. I x − εy ∈ Bz ε 2 IRk
By Minkowski’s inequality, we get the following bound for the norm of the above quantity: Dis1 ,...,is fε q,1 (x)
(8.2.32)
8.2 Application to Stable Limit Theorems
⎡ = ⎣
111
⎤1/q
|Dis1 ,...,is fε (x1 , x2 , . . . , xn )|q dx1 ⎦
IR
⎧ s ⎨ 2 = 2 ε ⎩
%
& I x − εy ∈ Bz ε , y ≤ 1 2 IR IRk ⎫1/q ⎬ ·I{Bz (ε)\Bz }Dis1 ,...,is g s (y) dy|q dx1 ⎭
s 2 = 2 ε ≤ 2
|Dis1 ,...,is g s (y)|
{y≤1}∩(Bz (ε)\Bz ) s
2 ε
⎧ ⎨ ⎩
I{x − εy ∈ Bz
IR
ε 2
⎫1/q ⎬ dx1 dy ⎭
|Dis1 ,...,is g s (y)| {y≤1}∩(Bz (ε)\Bz )
·
⎧ ⎨ ⎩
I{z1 ≤ x1 − εy1 ≤ z1 + ε} dx1
IR
⎫1/q ⎬ ⎭
dy.
In fact, the inequality is valid a.e. with respect to Lebesgue measure λ\k−1 . The last integrals are estimated as follows: ⎧ ⎫1/q ⎬ ⎨ I{z1 ≤ x1 − εy1 ≤ z1 + ε} dx1 = ε1/q , (8.2.33) ⎩ ⎭ IR
and
|Dis1 ,...,is g s (y)| dy
(8.2.34)
{y≤1}∩(Bz (ε)\Bz )
=
1 Vr
=
1 Vr
≤
{y≤1}
1 2s−1 n s−1 I{Bz (ε)\Bz } Di1 ,...,is−1 s 1 − yi2 2yis dy i=1 + I{Bz (ε)\Bz }s!2s |yi1 · · · yis | dy
{y≤1}
1 s!2s Vr
1 1 · · · |yi1 · · · yis | dy ≤
−1
−1
1 1 s!2s · 2k−s = s!2k . Vr Vr
Similarly, we can argue for any index 1 ≤ i ≤ k, and thus (8.2.30) follows from (8.2.33) and (8.2.34).
112
8. Probabilistic-Type Limit Theorems
From the inequality in (8.2.30) we obtain that given fε (x) , Ls (fε )
f ∗ (x) :=
≥ E(f ∗ (X) − f ∗ (Y )) εr ≥ a(s, k)E(fε (X) − fε (Y )). 2
ϑs,p (X, Y )
(8.2.35)
Applying (8.2.27), (8.2.28) we arrive at the following decomposition: E(fε (X) − fε (Y )) =
(fε (x) + 1)(PX − PY )( dx) IRk
⎛
⎜ = ⎝
+
+
⎞ ⎟ ⎠(fε (x) + 1)(PX − PY )( dx)
Bz Bz (ε)\Bz IRk \Bz (ε)
=: I1 + I2 + I3 , where (fε (x) + 1)(PX − PY )( dx) = 2(PX − PY )(Bz );
I1
=
I2
≥ −2 PY (Bz (ε)\Bz );
I3
= 0.
Bz
Thus by (8.2.23), I1 + I2 + I3 ≥ 2(PX (Bz ) − PY (Bz )) − 2(PY (Bz (ε)) − PY (Bz )) ≥ 2ε. From (8.2.35) we finally obtain ϑs,p (X, Y ) ≥ εr+1 a(s, k). With ε → L(X, Y ), this implies (8.2.21). 2
Remark 8.2.7 (a)
Let us use the polar transformation
x1 x2 x3 .. .
= cos ϑ1 = cos ϑ1 = cos ϑ1
xk
= sin ϑ1 ,
· · · cos ϑk−2 cos ϑk−1 , · · · cos ϑk−2 sin ϑk−1 , · · · sin ϑk−2 ,
(8.2.36)
8.2 Application to Stable Limit Theorems
113
where > 0, 0 ≤ ϑ1 ≤ 2π, 0 ≤ ϑj ≤ π, 2 ≤ j ≤ k − 1, and ∂(x1 , . . . , xk ) ∂(, ϑ1 , . . . , ϑk−1 )
k−1 Dk (ϑ), ⎛ sin ϑ1 ⎜ sin ϑ1 ⎜ ⎜ Dk (ϑ) := det ⎜ sin ϑ1 ⎜ .. ⎝ . =
cos ϑ1
· · · sin ϑk−1 · · · sin ϑk−2 · · · cos ϑk−2 0
···
(8.2.37) ⎞
sin ϑk−1 cos ϑk−1 0
⎟ ⎟ ⎟ ⎟. ⎟ ⎠
0
Then we have 1 Vr
= Dk
(1 − 2 )r+1 k−1 d
(8.2.38)
0
1 = Dk
(1 − 2 )r+1 (2 )
k−2 2
d2
0
1 Γ(r + 2)Γ( k2 ) , = Dk 2 Γ(r + 1 + k2 ) where
Dk :=
Dk (ϑ) dϑ.
/ IN) in terms of (b) Note that lower bound of ϑs,p (r = s − 1 + p1 ∈ the Prohorov metric exists. This follows from an example in Maejima and Rachev (1987) in the case k = 1. We next investigate smoothing inequalities, which play an important role in the proof of Berry–Ess´een-type theorems. They are also of interest for the study of intrinsic properties of probability metrics. Lemma 8.2.8 (a) Let Z be independent of X, Y, ε > 0, r = s − 1 + p1 . Then ϑs,p (X, Y ) ≤ ϑs,p (X + εZ, Y + εZ) + 2
Γ(1 + p1 ) Γ(1 + r)
εr kEZr1 . (8.2.39)
(b) If Z, W are independent of X, Y , then ϑs,p (X + Z, Y + Z) ≤ ϑs,p (X, Y )σ(W, Z) + ϑs,p (X + W, Y + W ); (8.2.40) and moreover, ϑs,p (X + Z, Y + Z) ≤ ϑs,p (W, Z)σ(X, Y ) + ϑs,p (X + W, Y + W ), (8.2.41)
114
8. Probabilistic-Type Limit Theorems
where σ is the total variation distance. Proof: (a) By the regularity of ϑs,p (cf. 8.2.1) we have ϑs,p (X, Y )
≤ ϑs,p (X + εZ, Y + εZ) + 2ϑs,p (0, εZ) ≤ ϑs,p (X + εZ, Y + εZ) + 2εr ϑs,p (0, Z).
By (8.2.11) and (8.2.15), ϑs,p (0, Z) ≤
Γ(1 + α) Γ(1 + r)
1≤i1 ,...,is−1 ≤k
E
k
|Zi |α |Zi1 · · · Zis−1 |.
i=1
(b) For any f ∈ Gs,p we have |E(f (X + Z) − f (Y + Z))| (8.2.42) ≤ |E(f (X + Z) − f (X + W )) − E(f (Y + Z) − f (Y + W ))| + |E(f (X + W ) − f (Y + W ))|. If f ∈ Gs,p , then the translates fx (z) := f (x + z) are also in Gs,p , and therefore, the first term is estimated by conditioning on X (respectively Y ): E(f (x + Z) − f (x + W )) dPX (x) − E(f (x + Z) − f (x + W )) dPY (x) = E(f (x + Z) − f (x + W )) d(PX − PY )(x) (8.2.43) ≤ ϑs,p (Z, W )σ(X, Y ). Indeed, the inequalities (8.2.42), (8.2.43) imply ϑs,p (X + Z, Y + Z) ≤ ϑs,p (Z, W )σ(X, Y ) + ϑs,p (X + W, Y + W ). The other case is derived similarly.
2
Lemma 8.2.9 If Z is independent of X, Y and PZ has a density pZ having integrable (s − 1)-fold derivatives Cs,Z := sup |Dis−1 p (x)| dx < ∞, (8.2.44) 1 ,...,is−1 Z 1≤i1 ,...,is−1 ≤k IRk
then ϑ1,p (X + Z, Y + Z) ≤ Cs,Z ϑs,p (X, Y ).
(8.2.45)
8.2 Application to Stable Limit Theorems
115
Proof: For any f ∈ G1,p , E(f (X + Z) − f (Y + Z)) =
f (x) d(FX+Z − FY +Z )(x) IRk
⎛
IRk
⎞
⎝
=
(8.2.46)
f (x)pZ (x − z) d(FX − FY )(z)⎠ dx
IRk
f ∗ (z) d(FX − FY )(z),
= IRk
where f ∗ (z) =
f (x)pZ (x − z) dx. From the Taylor expansion,
IRk
f ∗ (z) = f ∗ (0) +
s−1
j=1 1≤i1 ,...,ij ≤k
+
1
1≤i1 ,...,is ≤k 0
Dij1 ,...,ij f ∗ (0)zi1 · · · zij
(8.2.47)
(1 − t)s−1 s D f ∗ (tz)zi1 · · · zis dt. (s − 1)! i1 ,...,is
Since f ∈ G1,p , i.e., ⎞1/q
⎛
⎝
|Di1 f (x1 , . . . , xi , . . . , xn )|q dxi ⎠
≤ 1
a.s.,
(8.2.48)
IR
we have the following bound for the qth norm of f ∗ -derivatives: ⎛
⎝
IR
⎞1/q |Dis1 ,...,is f ∗ (z1 , . . . , zn )|q dzi ⎠
q ⎞1/q s = ⎝ Di1 ,...,is f (x)pZ (x − z) dx dzi ⎠ IR IRk q ⎞1/q ⎛ 1 = ⎝ Di1 f (x)Dis−1 pZ (x − z) dx dzi ⎠ 2 ,...,is IR IRk q ⎞1/q ⎛ 1 = ⎝ Di1 f (x + z)Dis−1 pZ (x) dx dzi ⎠ 2 ,...,is k ⎛
IR
IR
(8.2.49)
116
8. Probabilistic-Type Limit Theorems
q ⎞1/q 1 s−1 Di1 f (x + z) (Di2 ,...,is pZ (x)) dx dzi ⎠ = ⎝ k IR IR ⎧ ⎫1/q ⎨ ⎬ 1 q (Di f (x + z))(Ds−1 pZ (x)) dzi ≤ dx i2 ,...,is 1 ⎩ ⎭ ⎛
IR
IRk
=
⎧ ⎫1/q ⎬ ⎨ 1 q Di f (x + z) dzi p (x) dx Z i2 ,...,is 1 ⎩ ⎭
s−1 D
IR
IRk
|Dis−1 pZ (x)| dx = Cs,Z 2 ,...,is
≤
by (8.2.48)).
IRk
Summarizing the results in (8.2.46), (8.2.47), and (8.2.48), we derive the desired inequality (8.2.45). 2 As a consequence of Lemmas 8.2.8, 8.2.9 we next obtain an estimate between ϑ1,p and ϑs,p . Theorem 8.2.10 For every s = 1, 2, . . . , p ∈ [1, ∞), r := s − 1 + p1 , and random vectors X, Y on IRk we have 1
pr ϑ1,p (X, Y ) ≤ A(s, p, k)ϑs,p (X, Y ),
(8.2.50)
where 1
A(s, p, k) := a pr b
s−1 r
1
(p(s − 1)) pr
r s−1
(8.2.51)
and 1 a := √ s
2s π
1 7
s/2 ,
b := 2k k
2 π
21/p .
(8.2.52) d
Proof: Recall the inequality (8.2.39). Then for any ε > 0 and any Z = N (0, 1) independent of X, Y , we have 1/p
ϑ1,p (X, Y ) ≤ ϑ1,p (X + εZ, Y + εZ) + 2ε1/p k EZ1 ;
(8.2.53)
and furthermore, 1/p EZ1
≤ (EZ1 )
1/p
=
1 k i=1
21/p E|Zi |
1 7 21/p 2 = k . π
(8.2.54)
8.2 Application to Stable Limit Theorems
117
Now we apply Lemma 8.2.9 to get ϑ1,p (X + εZ, Y + εZ) ≤ Cs,εZ ϑs,p (X, Y ).
(8.2.55)
Next, assuming that Z is standard normal with independent components, we bound the constant in (8.2.55) as follows: x Cs,εZ = sup |Dis−1 pZ ( )|ε−s−n+1 dx (8.2.56) 1 ,...,is−1 ε i1 ,...,is−1 IRk
1−s
= ε
Cs,Z = ε1−s Cs,s−1/2 (Z1 +···+Zs ) .
Here, Z1 , . . . , Zs are i.i.d. copies of Z, and thus (see Zolotarev (1977, 1983, 1986)) Cs,εZ
1
= ε1−s s− 2 (1−s) Cs,Z1 +···+Zs s−1 2
≤ ε1−s s where C1,Z
=
sup 1≤i≤k IRk
=
sup i
IRk
=
sup i
IR
(8.2.57)
(C1,Z )s ,
|Di1 pZ (x)| dx 1 √ 2π
n
|xi |e−
x2 i 2
8 j =i
x2 1 i √ |xi |e− 2 dxi = 2π
e− 7
x2 j 2
dx
2 . π
Therefore, from (3.15), (3.17), (3.19) we obtain ϑ1,p (X, Y ) ≤ ε
1−s
1 √ s
2s π
1 7 21/p s/2 2 1/p ϑs,p (X, Y ) + 2ε k k . (8.2.58) π
Define ϕ(x) := ax1−s + bx1/p , 1 a := √ s
2s π
s/2 ϑs,p (X, Y ),
1 7 b := 2k k
Minimizing ϕ with respect to x yields (8.2.50).
2 π
21/p . 2
As a consequence of the smoothing properties we shall establish a Berry– Ess´een-type result, that will provide the right order estimate in the stable central limit theorem in terms of the metric ϑ1,1 . Let (Xi ) be an i.i.d.
118
8. Probabilistic-Type Limit Theorems
sequence of random vectors in IRk ; let (Θ, Θi ) be an i.i.d. sequence of d
symmetric α-stable distributed random vectors, i.e. n−1/α (Θ1 +· · ·+Θn ) = Θ; and define ϑr := ϑr (X1 , Θ) := sup hr−1 ϑ1,1 (X1 + hΘ, Θ1 + hΘ),
(8.2.59)
h>0
σr := σr (X1 , Θ) := sup hr σ(X1 + hΘ, Θ1 + hΘ), (8.2.60) h>0 1 ϑ := ϑ1,1 (X1 , Θ), σ := σ(X1 , Θ), τr := max ϑ, σ, σr(r−α) . (8.2.61) Theorem 8.2.11Suppose that 1 < α ≤ 2, α < r, and ϑ+ϑr +σ+σr < ∞. n Let Zn = n−1/α i=1 Xi . Then for some absolute constant C = C(k) depending only on the dimension k,
r ϑ1,1 (Zn , Θ) ≤ C n1− α ϑr + τr n−1/α . (8.2.62) Proof: We shall use the notation ϑ(X, Y ) = ϑ1,1 (X, Y ) during this proof. Note that by (8.2.10), ϑ(X, Y ) = ζ 1 (X, Y ) = ζ1 (X, Y ). From the smoothing inequality (8.2.39) we obtain the following bound: For any ε > 0 with Θi , Θ independent and identically distributed, ϑ(Zn , Θ1 ) ≤ ϑ(Zn + εΘ, Θ1 + εΘ) + Cε,
(8.2.63)
where C := 2 k EΘ1 . Our proof will be based on the Bergstr¨om convolution method (cf. Rachev (1991, Chapter 18) and the references therein). We start by making use of the triangle inequality: ϑ(Zn + εΘ, Θ1 + εΘ) Θ 1 + X2 + · · · + X n + εΘ ≤ ϑ Zn + εΘ, n1/α m Θ1 + · · · + Θj + Xj+1 + · · · + Xn + ϑ + εΘ, n1/α j=1
(8.2.64)
Θ1 + · · · + Θj+1 + Xj+2 + · · · + Xn + εΘ n1/α Θ1 + · · · + Θm+1 + Xm+2 + · · · + Xn +ϑ + εΘ, Θ1 + εΘ n1/α m =: Io + Ij + Im+1 , m = [n/2]. j=1
Applying the smoothing property (8.2.41), we obtain X2 + · · · + Xn Θ2 + · · · + Θn Io ≤ ϑ , n1/α n1/α
(8.2.65)
8.2 Application to Stable Limit Theorems
119
· σ n−1/α X1 + εΘ, n−1/α Θ1 + εΘ X 1 + Θ2 + · · · + Θ n Θ 1 + · · · + Θn + εΘ, + εΘ . +ϑ n1/α n1/α Similarly, for 1 ≤ j ≤ m, we have
Ij
Xj+2 + · · · + Xn Θj+2 + · · · + Θn ≤ ϑ , (8.2.66) n1/α n1/α Θ1 + · · · + Θj + Xj+1 Θ1 + · · · + Θj+1 ·σ + εΘ, + εΘ 1/α n n1/α Θ1 + · · · + Θj + Xj+1 + Θj+2 + · · · + Θn +ϑ + εΘ, Θ1 + εΘ . n1/α
Summarizing the above inequalities, we get the bound ϑ(Zn , Θ1 ) ≤
5
∆j ,
(8.2.67)
j=1
where
∆1 := ∆2 :=
∆3 := ∆4 := ∆5 :=
X2 + · · · + X n Θ 2 + · · · + Θ n X1 Θ1 ϑ , + εΘ, 1/α + εΘ , σ n1/α n1/α n1/α n m Xj+2 + · · · + Xn Θj+2 + · · · + Θn ϑ , n1/α n1/α j=1 Θ1 + · · · + Θj + Xj+1 Θ1 + · · · + Θj+1 + εΘ, + εΘ , ·σ n1/α n1/α X1 + Θ2 + · · · + Θ n (m + 1)ϑ , Θ , n1/α Θ1 + · · · + Θm+1 + Xm+2 + · · · + Xn ϑ + εΘ, Θ + εΘ , 1 n1/α Cε = 2k EΘ1 ε.
To estimate the terms ∆i , we introduce smoothed versions of the metrics ϑ, σ defined as follows: For r > 1, ϑr (X, Y )
:=
sup hr−1 ϑ(X + hΘ, Y + hΘ),
(8.2.68)
h>0
σr (X, Y )
:=
sup hr σ(X + hΘ, Y + hΘ)
(8.2.69)
h>0
(cf. also (8.2.59), (8.2.60)). It is easy to see that both ϑr , σr are ideal metrics of order r.
120
8. Probabilistic-Type Limit Theorems
We first estimate ∆3 and ∆4 using the ideality of ϑr . In the rest of the proof, c stands for a general constant, which may be different at different places: 1 1/α 1/α 2 n−1 n−1 −1/α −1/α X1 + Θ, n Θ1 + Θ ∆3 = (m + 1)ϑ n n n r−1
α n ϑr n−1/α X1 , n−1/α Θ ≤ (m + 1) (8.2.70) n−1 ≤ c n1− α ϑr . r
Similarly, ∆4
≤ ϑr
Xm+2 + · · · + Xn Θm+2 + · · · + Θn , n1/α n1/α
n m+1
r−1 α (8.2.71)
≤ c n1− α ϑr . r
Define
1 ε := A max σ1 , σrr−α n−1/α ,
A > 0.
(8.2.72)
The proof continues by induction on n. The induction hypothesis states that for all j < n, r X 1 + · · · + X j Θ1 + · · · + Θ j , (8.2.73) ϑ ≤ B(ϑr j 1− α + τr j −1/α ). j 1/α j 1/α (For n = 1, . . . , n0 , n0 fixed, (8.2.62) follows from τr ≥ ϑ and the ideality of ϑ1,1 .) Then, for ∆1 = ∆1 (n), we obtain ∆1 ≤ Bc(n1− α ϑr + n−1/α τr )σ(n−1/α X1 + εΘ, n−1/α Θ1 + εΘ) (8.2.74) r ≤ Bc(n1− α ϑr + n−1/α τr )ε−1 n−1/α σ1
r 1 Bc n1− α ϑr + n−1/α τr . ≤ A r
Similarly, we estimate ∆2 : ∆2
≤ cB(ϑr (n − m − 2)1− α + τr (n − m − 2)−1/α ) −r/α ∞ j
−1/α −1/α α σr n X1 , n Θ · +ε n i=1 r
∞
r ≤ cB ϑr n1− α + τr n−1/α
≤ cB ϑr n
r 1− α
+ τr n−1/α
j=1
σr (j + εα n)r/α
1 . Ar−α
(8.2.75)
8.2 Application to Stable Limit Theorems
From (8.2.70)–(8.2.75) we infer r 1 1 + r−α B ϑr n1− α + τr n−1/α ϑ(Zn , Θ) ≤ C1 A A
r + C2 ϑr n1− α + Aτr n−1/α . 1 Choosing A big enough so that C1 ( A1 + Ar−α )< B/2 > C2 (1 + A), we complete the proof.
1 2
121
(8.2.76)
and then B such that 2
Remark 8.2.12 (a) We note that the conditions concerning the domain of attraction of Θ are given solely in terms of the metrics appearing in the upper bounds. (b) Since Θ has a density pΘ with integrable derivatives, we can get, similarly to the proof of Lemma 8.2.9, that σ(X + δΘ, Y + δΘ) ≤ Cδ −1 ϑ1,1 (X, Y ),
(8.2.77)
where C = C(Θ). This implies that for any 0 < ε < 1, σr (X, Y )
sup hr σ(X + hΘ, Y + hΘ) h>0
= sup hr σ X + h(1 − ε)1/α Θ1 + hε1/α Θ2 , h>0 Y + h(1 − ε)1/α Θ1 + hε1/α Θ2
≤ sup hr ϑ1,1 X + h(1 − ε)−1/α Θ1 , Y + h(1 − ε)1/α Θ2 =
h>0
· C(Θ)/(h(1 − ε)1/α ) = C(Θ)(1 − ε)(1−r)/α ε−1/α ϑr (X, Θ). The minimum in the right-hand side is attained for ε = 1/r, implying that σr (X, Y )
≤ C(Θ)rr/α (r − 1)(1−r)/α ϑr (X, Θ) ≤ C(Θ)2
r/α
(8.2.78)
ϑr (X, Θ).
Similarly to the proof of Theorem 8.2.10, relations (8.2.77), (8.2.78) allow us to replace (8.2.62) by a bound involving only ϑ, ϑr :
r −1/α }n ϑ1,1 (Zn , Θ) ≤ C n1− α ϑr + max{ϑ, ϑ1/(r−α) . (8.2.79) r For r ∈ IN, ϑr , the r-smoothed ϑ metric, can be estimated from above by the ζr metric: ϑr ≤ cr ζr ,
(8.2.80)
122
8. Probabilistic-Type Limit Theorems
(r) where cr depends on supz≤1 |pΘ (y)(z)| dy (cf. Zolotarev (1983, p. 294, property 6)). Also, for r ∈ IN, ζr is estimated from above by the rth difference pseudomoment kr , defined by % kr (X, Y ) = sup |E(f (X) − f (Y ))|; f bounded, f : IRk → IR, (8.2.81) ' '& |f (x) − f (y)| ≤ 'xxr−1 − y yr−1 ' and kr (X, Y ) ≤ 2r vr (X, Y ) = 2r xr |PX − PY |( dx),
(8.2.82)
where vr is the absolute pseudomoment of order r. From (8.2.79)– (8.2.82) we obtain easy-to-check criteria for finiteness of the upper bounds. In particular, in the normal case α = 2, we take r = 3, and so the finiteness of the third moments of Xi implies the Berry–Ess´een result. In the case 1 < α < 2 we use the boundness of kr in (8.2.81). (c) In the case k = 1, α = 2 (normal case, dimension one) the result of Theorem 8.2.11 is due to Zolotarev (1987), based on the proof of Senatov (1980). From part (b) of the above remark, it follows that we can replace the terms ϑ, σ, ϑr , σr in the upper bound in (8.2.62) by k1 and kr . Since kr is topologically weaker than vr , it is of interest to obtain alternative bounds for kr . To this end let us recall the minimal r -metric: for r > 0, % & d d − Y r )(1/r)∧1 ; X = X, Y = Y . (8.2.83)
r (X, Y ) := inf (EX Then kr (X, Y ) =
% & d d X r−1 − Y Y r−1 |; X = inf |EX X, Y = Y (8.2.84)
= 1 (XXr−1 , Y Y |r−1 ) % & d d = inf EU − V ; U = XXr−1 , V = Y Y r−1 (cf. Rachev and R¨ uschendorf (1990)). If X and Y have densities fX and fY , respectively, then (cf. Rachev (1991, pp. 249–252)) we use that k1 = 1 to get the bound kr (X, Y )
≤
αr (X, Y ) (8.2.85) 1
x
x −k−1 − fY Y r−1 dt dx. := x1 t fXXr−1 t t k IR
0
8.2 Application to Stable Limit Theorems
123
For some examples with an equality in (8.2.85), see Rachev (1991, p. 252). The densities of XXr−1 , Y Y r−1 are obtainable from the transformation formula. In particular, this gives explicit bounds in the case r = 1, where the expression in (8.2.85) simplifies. The following upper bound for kr , r > 1, will turn out to be useful. Lemma 8.2.13 If EXr < ∞, EY r < ∞, r > 1, then kr (X, Y ) ≤ c r (X, Y ),
(8.2.86)
where the constant c depends on the rth moments of X, Y . Proof: Let r :=
r r−1
d
d
and U = X, V = Y . Then
' ' E 'U U r−1− V V r−1 '
≤ EU r−1 U −V + EV U r−1− V r−1 =: I1 + I2 .
For I1 , I2 we readily get the bounds I1 ≤ (EU r )1/r (EU − V r )1/r and ⎧ r 1/r r 1/r ⎪ ⎪ 1 < r ≤ 2, ⎪ (EV ) (EU − V ) , ⎨ r 1/r I2 ≤ (r − 1)(EU − V ) ⎪
⎪ r−2 1 ⎪ ⎩ · (EV r )1/r + (EV r ) r−1 (EU r ) r−1 , r > 2. So I1 + I2 ≤ c(EU − V r )1/r , and passing to the corresponding minimal 2 metrics, we get kr (X, Y ) ≤ c r (X, Y ), as required. In some examples one can determine kr explicitly. Suppose that for some radial transformation ⎧ ⎨ α(x) x if x = 0, x φ : IRk → IRk , φ(x) := ⎩ 0 if x = 0, d
with α monotonically nondecreasing, we have Y = φ(X). Examples of this relation include spherically invariant distributions and spherically equivalent distributions, as for example, the uniform distribution on a p-ball in IRk and the product of Weibull distributions (cf. Section 3.2). By (8.2.84), it is easy to see that the pair (X, φ(X)) is an optimal coupling with respect to kr , and so we obtain ' ' (8.2.87) kr (X, Y ) = E 'XXr−1 − φ(X)φ(X)r−1 ' = E |Xr − α(X)r | .
124
8. Probabilistic-Type Limit Theorems
(A related explicit formula was derived for the r distance in Section 3.2.) Note that α is determined by the equation FY (y) = P (α(X) ≤ y) = −1 FX (α−1 (y)), which in the case of FX continuous leads to α(t) = FY ◦ FX (t). We illustrate the above resutls, invoking again the stable limit theorem 8.2.11. Let Θ be a k-dimensional α-stable random vector with spectral measure m such that xα dm(x) < ∞; i.e., IRk
⎧ ⎫ ⎨ 1 ⎬ E exp{it, Θ} = exp − |t, s|α dm(s) . ⎩ 2 ⎭
(8.2.88)
IRk
We apply the LePage representation for symmetric α-stable laws. Let 1. (Yj ) be an i.i.d. sequence of random vectors with distribution m/|m| and let Yj := |m|1/α Yj ; 2. ( ηj ) be i.i.d. symmetric random variables with η1 α = (E| η1 |α )1/α < ∞ and let ηj := ηj/ηj α ; 3. (Γj ) be the sequence of successive times of jump of a standard Poisson process and assume that the three sequences are independent. Then d
Θ = cα
∞
−1/α
Γj
ηj Y j ,
(8.2.89)
j=1
where the constant cα is determined by the tail behavior of the law of Θ. Without loss of generality we set cα = 1 (cf. Ledoux and Talagrand (1991, Section 5.1) and Samorodnitsky and Taqqu (1994)). Suppose that the distribution of X has a similar representation in distribution d
X =
∞
−1/α ∗ ∗ ηj Yj ,
Γj
(8.2.90)
j=1
where (Yj∗ ), (ηj∗ ) are independent but not necessarily identically distributed. Recall the bound (8.2.80) to see that what we need is an estimate for ζr (X, Θ) from above. Proposition 8.2.14 Let r > max{1, α}. Suppose that Yj , Yj∗ , ηj , ηj∗ have finite rth moments with supj E|ηj∗ |r < ∞. Then ⎛ ⎞ ∞ ζr (X, Θ) ≤ C sup ⎝ r (Yj∗ , Yj ) + |x|r−1 |Fηj∗ (x) − Fηj (x)| dx⎠. (8.2.91) j≥1
−∞
8.2 Application to Stable Limit Theorems
125
Proof: By the ideality of ζr , ζr (X, Θ) ≤
∞
−r/α
E(Γj
)ζr (ηj∗ Yj∗ , ηj Yj ).
j=1
Since r > α, and for j > r/α, −r/α
EΓj
the series Sr =
=
Γ(j − r/α) ∼ j −r/α , Γ(j)
∞ j=1
−r/α
E(Γj
) converges. Furthermore,
ζr (ηj∗ Yj∗ , ηj Yj ) ≤ ζr (ηj∗ Yj∗ , ηj∗ Yj ) + ζr (ηj∗ Yj , ηj Yj ) ≤
(E|ηj∗ |r )ζr (Yj∗ , Yj )
+
k
(E|Yj,i |r )ζr (ηj∗ , ηj )
i=1
≤ C ζr (Yj∗ , Yj ) + ζr (ηj∗ , ηj ) , where Yj,i is the ith compoment of Yj . Since ζr (ηj∗ , ηj )
1 1 ≤ kr (ηj∗ , ηj ) = r! (r − 1)!
∞ |x|r−1 |Fηj∗ (x) − Fηj (x)| dx, −∞
we obtain (8.2.91) after applying the inequality ζr (Yj∗ , Yj ) ≤ and Lemma 8.2.13.
1 ∗ r! κr (Yj , Yj )
2
Under the additional assumption sup EYj∗ r < ∞
(8.2.92)
j
we obtain (by the obvious bound r (Yj∗ , Yj ) ≤ (EYj∗ r )1/r + (EYj r )1/r ) the finiteness of the upper bound in the limit theorem in (8.2.62). In this way we establish a stable limit theorem (with an estimate of the rate of convergence) for random vectors in the r -neighborhood of a stable symmetric law in the sense of the LePage representation. For r = 1 we use the estimate in (8.2.85). For r > 1 (and in particular r = 2) explicit expressions for r are known in several cases (cf. Section 3.2), for example for the distance r (X, Y ) between normal distributed random vectors X and Y , between uniform distributions on balls and multivariate normal, or Weibull, distributions, and between spherically equivalent distributions.
126
8. Probabilistic-Type Limit Theorems
8.3 Application to Summability Methods and Compound Poisson Approximation In this section we apply the ϑs,p -metric (see (8.2.2)) to obtain rate of convergence results in stable limit theorems for multivariate summability methods, thus extending some results of Maejima (1985) in the real case. We also study the approximation of sums of independent random variables by compound Poisson distributions. Let (Xn )n≥0 be an i.i.d. sequence of random vectors in IRk and consider the weighted sums T (λ) :=
∞
cj (λ)Xj ,
cj (λ) ≥ 0,
(8.3.1)
j=0
where for λ > 0 or λ ∈ IN, (cj (λ)), j ≥ 0, is a summability method Some classical summability methods are ⎧ ⎨ 1 , 0 ≤ j ≤ n, n+1 (8.3.2) “C´esaro method” cj (λ) = ⎩ 0, otherwise; “Borel method” “Euler method”
λj −λ e , λ > 0, j ∈ IN0 ; j! n j cj (λ) = λ (1 − λ)n−j , j 0 ≤ j ≤ n, 0 < λ < 1;
cj (λ) =
cj (λ) = (1 − e−1/λ )e−j/λ , 0 ≤ j < ∞;
“Abel method”
(8.3.3) (8.3.4)
(8.3.5)
“random walk method” cj (n) = P (Sn = j), 0 ≤ j < ∞, (8.3.6) where Sn is a random walk on the integers IN0 . For a review and discussion of these methods in the univariate case we refer to Maejima (1985). Let Θ(α) denote a random vector with symmetric stable distribution of index α, 0 < α ≤ 2. Recall (see Samorodnitsky and Taqqu (1994)) that for 0 < α < 2, Θ(α) is symmetric α-stable in IRk if and only if there exists a (unique) symmetric finite measure Γ on the unit sphere Sk such that ϕΘ(α) (t)
E exp it, Θ(α) ⎧ ⎫ ⎨ ⎬ = exp − |(t, s)|α Γ( ds) , ⎩ ⎭
=
Sk
(8.3.7) t ∈ IRk .
8.3 Summability Methods, Compound Poisson Approximation
127
Define then ⎞1/α ⎛ ∞ cj (λ)α ⎠ . dα (λ) = ⎝
(8.3.8)
j=0
Theorem 8.3.1 Let 0 < α < r = s − 1 + p1 . Then ϑs,p
1 T (λ), Θ(α) dα (λ)
≤ R(λ)ϑs,p ,
(8.3.9)
where ⎛ ⎞r ∞ R(λ) = ⎝ cj (λ)/dα (λ)⎠,
ϑs,p := ϑs,p (X0 , Θ(α) ).
(8.3.10)
j=0
Proof: Let (Θj ) be an i.i.d. sequence with the same distribution as Θ(α) . Let us show first that ∞
1 d cj (λ)Θj = Θ(α) . dα (λ) i=0
(8.3.11)
Consider the characteristic function of the right-hand side quantity in (8.3.11): E ei dα (λ) 1
∞
j=0
cj (λ)Θj ,t
=
∞ 8
9 i
Ee
cj (λ) Θ ,t dα (λ) j
:
(8.3.12)
j=0
⎧ ⎫ <α ⎨ ; c (λ) ⎬ j exp − = t, s Γ( ds) ⎩ ⎭ dα (λ) j=0 Sk ⎧ ⎫ α ∞ ⎨ ⎬ cj (λ) = exp − |t, s|α Γ( ds) ⎩ ⎭ dα (λ) j=0 ∞ 8
Sk
= ϕΘ(α) (t). By Lemma 8.2.3, ϑs,p is ideal of order r = s − 1 +
1 p
> α. Therefore,
1 T (λ), Θ(α) dα (λ) ⎛ ⎞ ∞ ∞ 1 1 = ϑs,p ⎝ cj (λ)Xj , cj (λ)Θj ⎠ dα (λ) j=0 dα (λ) j=0
ϑs,p
(8.3.13)
128
8. Probabilistic-Type Limit Theorems
≤ dα (λ)−r
∞
cj (λ)r ϑs,p (X0 , Θ(α) )
j=0
= R(λ)ϑs,p (X0 , Θ(α) ). 2 Note that various upper bounds for ϑs,p were established in Section 8.2. In particular, if r ∈ IN, or if X0 has a density, we have obtained upper bounds in terms of difference pseudomoments. Maejima (1985) showed that R(λ) ≤ cλ−(r−α)/α
(8.3.14)
for the C´esaro and Abel methods, and R(λ) ≤ cλ−(r−α)/2α
(8.3.15)
for the random walk method (which includes the Euler method and the Borel method as particular cases). In the Gaussian case, for r = 3 the metric ϑs,p in (8.3.9) is finite, provided that (i) if Cov (X0 ) = Ik , the unity matrix, and (ii) the components have finite third moments. Furthermore, the corresponding rate of convergence is λ−1/2 for the C´esaro and Abel methods and λ−1/4 for the random walk method. We complete this section with an application of the ideality properties of our metrics to the approximation of the distribution of sums of nonidentically distributed random vectors by a compound Poisson law. Let X1 , . . . , Xn be independent random vectors in IRk with distributions P1 , . . . , Pn of the form Pi = (1 − pi )δ0 + pi Qi ,
0 ≤ pi ≤ 1, 1 ≤ i ≤ n.
(8.3.16)
Here, δ0 stands for the one point distribution at zero. We can write Xi in the form Xi = Ci Di ,
1 ≤ i ≤ n,
(8.3.17)
where Ci has distribution Qi , Di is B(1, pi )-distributed, and Ci , Di are independent. We shall consider the approximation of S ind :=
n
Xi
(8.3.18)
i=1
by a multivariate compound Poisson distribution P(µ, Q). P(µ, Q) is defined as the distribution of S coll :=
N i=1
Zi ,
(8.3.19)
8.3 Summability Methods, Compound Poisson Approximation
129
where (Zi ) is an i.i.d. sequence; P Zi = Q, N is Poisson distributed with parameter µ, P N = P(µ); and N , (Zi ) are independent. The notation S ind , S coll is taken from risk theory. Recall that in the risk-theory framework in the “individual model” pi is the probability of a claim Ci with distribution Qi , corresponding to k different types of claims. S coll denotes the approximation of S ind by the “collective model”; we refer to the books of Gerber (1981) and Hipp and Michel (1990) for these and related notions. The usual choice of Q, µ in risk theory is µ=µ :=
n
:= Q=Q
pi ,
i=1
n pi i=1
µ
Qi .
(8.3.20)
This leads to the following representation of S coll : S coll =
n
Sicoll ,
(8.3.21)
i=1
where Sicoll ∼ P(pi , Qi ) (X ∼ Q denoting that X has distribution Q) and and {Sicoll } independent. Note that with this choice µ = µ , Q = Q, moreover, E S ind = E S coll ,
(8.3.22)
if the expectations exist. If Σi = Cov (Ci ), αi = (αi,1 , . . . , αi,k ) = ECi , then n n
pi Σi + pi qi αiT αi , Cov S ind = i=1
(8.3.23)
i=1
while n n
pi Σi + pi αiT αi . Cov S coll = i=1
(8.3.24)
i=1
As a consequence we obtain the following majorization result:
Cov S ind
In particular, ϑs,p S ind , S coll = ∞ if r = s − 1 +
(8.3.25)
1 p
≥ 3.
130
8. Probabilistic-Type Limit Theorems
In Rachev and R¨ uschendorf (1990) it is shown that a better choice of (µ, Q) is possible for k = 1 by appropriately chosen scale transformations. For an extension of this result to k ≥ 1, define µi
:=
µ
:=
(1 − pi )αi , Γi n
µi ,
Q
:=
(1 − pi )Σi ,
:=
n βi Qi , β β i=1
i=1
βi
pi ; 1 − pi n := βi,
:=
(8.3.26)
i=1
where Qi is a probability measure with mean µi and covariance Γi . We approximate Xi by a compound Poisson distributed r.v., Sicoll ∼ P(β i , Qi ). This leads to an approximation of S ind by S coll ∼ P(β, Q).
(8.3.27)
From our construction, it follows that EXi Cov (Xi )
= pi αi = E S coll ;
= Σi = Cov Sicoll = β i Γi + βi µTi µi .
(8.3.28)
The “ideal” properties of the metric ϑs,p derived in Section 8.2 yield closeness between the “individual” and the “collective” models using the following bounds: n
ϑs,p S ind , S coll ≤ ϑs,p Xi , Sicoll
(8.3.29)
i=1
and
ϑ1,p S ind , S coll
≤ A(s, p, k)
1 n
ϑs,p Xi , Sicoll
1 2 pr
.
(8.3.30)
i=1
The constant A(s, p, k) can be determined from (8.2.45). ϑs,p Xi , Sicoll is estimated from above by the metric αs,p (Theorem 8.2.5), which depends only on pseudo-difference moments. In particular, for r = s − 1 + p1 = s and bi = EXi s1 , ϑs,p (Xi , Sicoll ) ≤ pi bi
1 (s − 1)!
(cf. (8.2.11)).
(8.3.31)
Define the normalizations i = Xi − EXi , X
Yi = Sicoll − E Sicoll .
(8.3.32)
Consider the i.i.d case; we shall establish an estimate similar to that in (8.3.29) but without the dependence on n in the upper bound.
8.4 Operator-Stable Limit Theorems
131
Theorem 8.3.2 Suppose (Xn ) is an i.i.d. sequence with α = E Ci , Σ = Cov (Ci ), p = P (Di = 1). Then
(8.3.33) ϑ1,1 S ind , S coll ≤ C(ϑr + ϑr + τr + τr ), 1 , Θ), ϑr = ϑr (Y1 , Θ), τr = τr (X 1 , Θ), τr = τr (Y1 , Θ), C where ϑr = ϑr (X is as defined in (8.2.62), and Θ is an N (0, Σ)-distributed r.v. Proof: The ideality of ϑ1,1 and the triangle inequality yield 1 n 2 n
ind coll coll ϑ X, S S ,S = ϑ 1,1
1,1
= ϑ1,1
i
i=1 1 n
i , X
i=1
≤
√
1
nϑ1,1
1 √ n
i
i=1 n i=1 n i=1
2 Yi 2 i , Θ X
+
√
1 nϑ1,1
2 n 1 √ Yi , Θ , n i=1
where Θ is normally distributed with mean zero and covariance Σ. The proof of Theorem 8.2.11 (in the case α = 2 with Σ being the identity matrix) extends to the general case with the same constant, thus implying (8.3.33). 2
8.4 Application to Operator-Stable Limit Theorems: Statement of the Results and Auxiliary Lemmas In this section we generalize some of the results from Section 8.2, studying the rate of convergence problem for a more general summation scheme for random vectors. Namely, suppose that the IRd -valued random vector θ is strictly operator-stable in the sense that µ 4, the characteristic function of ∗ θ, satisfies µ 4(z)t = µ 4(tB z) for every t > 0, for some invertible linear operator B on IRd . Suppose also that for the i.i.d. random vectors {Xi } in n w IRd , n−B i=1 Xi → θ. In this and the next section we study the rate of convergence of this operator-stable limit theorem in terms of several probability metrics including the Kantorovich metric and Lp -minimal versions, see (2.6.2). The results in this and the next section are due to Maejima and Rachev (1996). We start with some definitions and notation related to operator-stable limit theorems. A probability distribution µ on IRd is said to be full if µ is
132
8. Probabilistic-Type Limit Theorems
not concentrated on a proper hyperplane in IRd . A full distribution µ on IRd is called operator-stable if there exists an invertible linear operator B on IRd and a function b : (0, ∞) → IRd such that for all t > 0, ∗
4(tB z)eib(t) , µ 4(z)t = µ
for all z ∈ IRd .
(8.4.1)
Here µ 4 is the characteristic function of µ, B ∗ is the adjoint operator of B, ∞ A and t = exp{(ln t)A} = k=0 (k!)−1 (ln t)k Ak . The distribution µ is called strictly operator-stable if we can choose b(t) ≡ 0. In this section, we always assume that µ is a full strictly operator-stable distribution on IRd . Sharpe (1969) showed that if 1 is not in the spectrum of B, then the operatorstable law can be centered so as to become strictly operator-stable. Thus, the assumption of strict operator-stability is not so restrictive. The invertible linear operator B in (8.4.1) is called an exponent of µ. When µ is operator-stable with an exponent B, µ may satisfy (8.4.1) for other B’s; i.e., the exponent of µ is not necessarily unique. Further, we fix the value of the exponent B and denote by θ the random vector in IRd having the full strictly operator-stable distribution µ with this fixed B. It is known that every eigenvalue of B has its real part not less than 12 (see Sharpe (1969)). Recall that for a given sequence X1 , X2 , . . . of i.i.d. random vectors in IRd for which the normalized sum converges to θ, namely, n−B
n
w
Xi → θ,
(8.4.2)
i=1
we say that {Xi } belongs to the domain of normal attraction of µ. As in the previous sections, we will be interested in the rate of convergence of (8.4.2). Remark 8.4.1 Some of the results in this section can be extended to Banach space–valued random variables. Also, our arguments can be used for similar rate of convergence problems of the max-operator-stable limit theorem. We start with some notation. Let · 0 be the usual Euclidean norm of IRd and let S(µ) be the symmetry group associated with µ, that is, the group of all invertible linear operators A on IRd such that for some a ∈ IRd , µ 4(z) = µ 4(A∗ z)eia . Since by assumption µ is full, S(µ) is compact, and thus there exists a Haar probability H on S(µ). We introduce the following norm · , which depends on the particular operator-stable law µ but not on the choice of exponent: 1 x =
gtB x0 S(µ) 0
dt dH(g) t
(8.4.3)
8.4 Operator-Stable Limit Theorems
133
(see Hudson et al. (1986) and Hahn et al. (1989)). It has the following properties: (i) · does not depend on the choice of the exponent B. (ii) The map t → tB x is strictly increasing on (0, ∞) for x = 0. Define the norm of the linear operator A on IRd in the usual way by A = supx=1 Ax. Then property (ii) implies (iii) The map t → tB is strictly increasing on (0, ∞); i.e., t → t−B = (t−1 )B is strictly decreasing on (0, ∞). Further, we will need estimates of the growth rate of R(t) = tB x. Meerschaert (1989) showed that for every x, the function R0 (t) = tB x0 varies regularly with index between λB and ΛB , where λB and ΛB are the minimum and the maximum of the real parts of the eigenvalues of B, respectively. Clearly, for every norm · on IRd , the function R(t) = tB x will be of the same order as the regularly varying function R0 (t). In particular, for any η > 0, there exists t0 > 0 such that for any t > t0 , tλB −η x < tB x < tΛB +η x,
(8.4.4)
t−ΛB −η x < t−B x < t−λB +η x.
(8.4.5)
and
Let X (IRd ) be the class of all random vectors in IRd , and the Kolmogorov metric in X (IRd ), (X, Y ) :=
sup |P (X ≤ x) − P (Y ≤ x)|.
(8.4.6)
x∈IRd
Here, and throughout this section, x ≤ y or x < y, x, y ∈ IRd , means component-wise inequality. Also, all the probability metrics µ that we shall use are in fact metrics in the space of probability laws: We write µ(X, Y ) instead of µ(PX , PY ) only for the sake of simplicity, where PX , PY stand for the probability distributions of X, Y , respectively. Next, we define a uniform metric depending on the exponent B, ∗ (X, Y ) := sup (tB X, tB Y ).
(8.4.7)
t>0
This metric plays a crucial role in our approach to the rate of convergence problem (8.4.2). Let Var be the total variation distance in X (IRd ),
134
8. Probabilistic-Type Limit Theorems
Var(X, Y ) := 2
sup A∈B(IRd )
|P (X ∈ A) − P (Y ∈ A)|
(8.4.8)
|PX − PY |( dx)
= IRd
=
sup{|Ef (X) − Ef (Y )|; f : IRd → IR, continuous, |f (x)| ≤ 1 for all x ∈ IRd }.
Remark 8.4.2 It is not difficult to check that ∗ is topologically “between” and Var; that is, top
top
≺ ∗ ≺ Var . top
Here we use the standard notation µ ≺ ν, meaning that ν-convergence implies µ-convergence, but the inverse is not generally valid. Remark 8.4.3 Our aim is to present a general approach to the rate of convergence problems associated with (8.4.2) that is designed to work for different metrics in terms of which we want to obtain estimates of the rate of convergence. We start with uniform-type metrics (, ∗ , Var), and then we will proceed with Kantorovich-type minimal distances. For r > 0, define a convolution-type metric associated to Var: µr (X, Y ) := sup tB −r Var (tB X + θ, tB Y + θ).
(8.4.9)
t>0
Here and in what follows, the notation X1 + X2 means the sum of two independent random vectors X1 and X2 . We shall first list our results and then prove them, extending the general method we have outlined in Section 8.1. Theorem 8.4.4 Let θ be a full strictly operator-stable random vector in B (≥ λ1B ) and take p such that IRd , and B an exponent of θ. Let r > Λ λ2 1 λB
λB ΛB r.
B
d Let {Xi }∞ i=1 be a sequence of i.i.d. random vectors in IR
∗ = ∗ (X1 , θ),
µ1 = µ1 (X1 , θ),
µr = µr (X1 , θ),
satisfying the moment-type condition 1 r−p ∗ < ∞. τr = τr (X1 , θ) := max , µ1 , µr
(8.4.10)
(8.4.11)
8.4 Operator-Stable Limit Theorems
135
Then for some absolute constant K = K(d, B, r, p) > 0, 1 n−B
n
1
2 Xi , θ
i=1
≤ ∗
n−B
≤ K nn
n
2 Xi , θ
i=1 −B r
µr + n−B τr
for all n ≥ 1.
Remark 8.4.5 In our theorem, we do not explicitly assume that {Xi } belongs to the domain of normal attraction of θ. However, since λB > 12 and rλB > 1, nn−B r µr + n−B τr → 0
as
n → ∞,
because of (8.4.5). Consequently, conditions (8.4.10) and (8.4.11) are sufficient for {Xi } to be in the domain of normal attraction of θ. As to the decreasing rate of n−B , by (8.4.5), for every η > 0, there exists n0 such that n−B ≤ n−λB +η for every n ≥ n0 . However, we also see that for any η > 0, n−B ≤ M n−λB +η for all n ≥ 1, where M = supt≥1 t−B+(λB −η)I (< ∞). Note, however, that rate of convergence theorems typically describe only a relatively small subset of that domain of attraction. Letting B =
1 αI
, 0 < α ≤ 2, we have the following:
Corollary 8.4.6 Let θ be a strictly α-stable random vector with index 0 < α ≤ 2 . Let α < p < r and {Xi } be a sequence of i.i.d. random vectors in IRd satisfying τr < ∞. Then for some absolute constant K = K(d, α, p) > 0, 1 ∗
−1/α
n
n
2 Xi , θ
r 1 ≤ K n1− α µr + n− α τr
for all n ≥ 1. (8.4.12)
i=1
Resnick and Greenwood (1979) studied the limit theorem for (α1 , α2 )stable laws, which corresponds to the operator-stable limit theorem with exponent ⎞ ⎛ 1/α1 0 ⎠. B=⎝ 0 1/α2 Theorem 8.4.4 provides a bound for the rate of convergence in this particular case. Corollary 8.4.7 Let θ = (θ(1) , θ(2) ) be a strictly (α1 , α2 )-stable bivariate α2 vector, 0 < α1 ≤ α2 ≤ 2. Let r > α21 (≥ α2 ) and take p such that α2 <
136
8. Probabilistic-Type Limit Theorems (1)
(2)
1 p< α α2 r. Let {Xi = (Xi , Xi )}i≥1 be a sequence of i.i.d. random vectors satisfying τr < ∞. Then for all n ≥ 1, 11 2 2 n n
(1) (2) ∗ −1/α1 −1/α2 Xi , n Xi n , θ ≤ K n1−r/α1 µr + n−1/α1 τr .
i=1
i=1
We next state our results on the rates of convergence in another type of uniform metric: the total variation distance Var and the uniform distance between characteristic functions. Let ' ' 'r ' ' −B 'r b = 54 '2−B ' ' 25 r > λ1B , ' , (8.4.13) ' B 'r ' B 'r ' ' r 1 ' −B ' 2 c = '2 ' + '3 ' , a = bc , and
' 1 'r ' ' M = sup 'x r I−B ' (< ∞).
(8.4.14)
x≥1
d Theorem 8.4.8 Let {Xi }∞ i=1 be a sequence of i.i.d. random vectors in IR satisfying
νr = νr (X1 , θ) := max{Var(X1 , θ), µr } ≤ Then
1 −B
Var n
n
2 Xi , θ
a . M
' 'r ≤ cn 'n−B ' νr
i=1
≤
' ' ' 1 ' '2−B 'r n 'n−B 'r bM
(8.4.15)
(8.4.16) for all n ≥ 1.
It would be interesting to have a version of this theorem without condition (8.4.15). Our next theorem concerns the rate of convergence of a third type of uniform metric χ that lies “between” and Var, top
top
≺ χ ≺ Var, namely, the uniform distance between characteristic functions: χ(X, Y ) := sup |φX (s) − φY (s)|, φX (s) := E eis,X , (8.4.17) s∈IRd
where ·, · is the inner product in IRd . The corresponding “tB -uniform” (recall the definition of ∗ (8.4.7)) and “smoothed” versions of χ are defined by χ∗ (X, Y ) := sup χ(tB X, tB Y ) t>0
(8.4.18)
8.4 Operator-Stable Limit Theorems
137
and ' '−r χr (X, Y ) := sup 'tB ' χ∗ tB X + θ, tB Y + θ .
(8.4.19)
t>0 d Theorem 8.4.9 Let {Xi }∞ i=1 be a sequence of i.i.d. random vectors in IR satisfying
νr∗ = νr∗ (X1 , θ) := max{χ∗ (X1 , θ), χr (X1 , θ)} ≤ Then for all n ≥ 1, 2 1 n ' 'r ∗ −B χ n Xi , θ ≤ cn 'n−B ' νr∗ ≤ i=1
a . M
' ' ' 1 ' '2−B 'r n 'n−B 'r . bM
Let us now denote the density of the random vector X by pX (x) (when it exists) and define the fourth type of uniform metric d(X, Y ) := ess sup |pX (x) − pY (x)|.
(8.4.20)
x∈IRd
This is “topologically” the strongest: top
top
top
≺ χ ≺ Var ≺ d. Let K(d, B) :=
max
max xB ij
2≤x≤3 1≤i,j≤d
(8.4.21)
(< ∞),
where Aij is the (i, j) component of the matrix A, and put C(d, B) = d! K(d, B)d . Let dr be the smoothed version of d, ' '−r dr (X, Y ) := sup 'tB ' d tB X + θ, tB Y + θ .
(8.4.22)
(8.4.23)
t>0
Applying Theorem 8.4.8, we obtain the following rate of convergence bound in the local central limit theorem for operator-stable random vectors. Theorem 8.4.10 Suppose X1 has a density. Let ' 'r 6 ' 'r A = max C(d, B) '2B ' + '3B ' , 1 5 and ' 'r D ≥ C(d, B) '3B ' .
138
8. Probabilistic-Type Limit Theorems
If Tr = Tr (X1 , θ) := max{d(X1 , θ), dr (X1 , θ)} < ∞ and νr ≤ min
a 1 , M M cD
,
then 1 −B
d n
n
2 Xi , θ
≤ Ann−B r Tr
for all n ≥ 1.
i=1
Remark 8.4.11 Operator-stable random vectors have bounded densities (Hudson (1980)). The rest of our rate of convergence results are concerned with the minimal Lp -metrics, 0 ≤ p ≤ ∞, and in particular with 1 , the Kantorovich metric, 4 1 ; see Section 8.1. Recall that the total variation distance Var is
1 = L in fact the minimal L0 -metric. Recall the definition of the Lp -compound metric: For any X, Y ∈ X (IRd ), (8.4.24) Lp (X, Y ) := {E[X − Y p ]}min(1,1/p) , 0 < p < ∞, L0 (X, Y ) := E[I[X = Y ]] = P (X = Y ), (8.4.25) L∞ (X, Y ) := ess sup X −Y = inf{ε > 0; P (X −Y > ε) = 0}, (8.4.26) where I[A] is the indicator function of a set A. As always in this book, we assume that all random vectors X ∈ X (IRd ) are defined on a nonatomic probability space (Ω, A, P ); in this way the space of all joint laws PX,Y coincides with the space of all probability measures on IR2d . The Lp -minimal metric for 0 ≤ p ≤ ∞ was defined in (8.2.23): 4 p (X, Y ) = L 4 p (PX , PY ) := inf Lp (X, Y ), (8.4.27)
p (X, Y ) = L d where the infimum is taken over all PX, Y with fixed marginals X ∼ X, d Y ∼ Y . Remark 8.4.12 For every p ∈ [0, ∞] fixed, we shall be interested in the 4 p n−B n Xi , θ → 0. As a consequence, we rate of convergence of L i=1 shall derive the rate of convergence results in terms of the Prohorov metric π(X, Y ) = inf{ε; P (X ∈ A) ≤ P (Y ∈ Aε ) + ε for all A ∈ B(IRd )}, (8.4.28) where Aε := {x; x − A ≤ ε}.
8.4 Operator-Stable Limit Theorems
139
4 0 = 1 Var, and so Theorem 8.4.8 gives the Case p = 0. For p = 0, L 2 desired bound for the rate of convergence. Case 0< p ≤ 1. Suppose first that B, the exponent of θ, satisfies ΛB 1 ≤ < p ≤ 1. (8.4.29) λB λ2B Then by (8.4.5), nn−B p → 0 as n → ∞. Theorem 8.4.13 Suppose 0 < p ≤ 1 and (8.4.29) holds. Let X, X1 , X2 , . . . be a sequence of i.i.d. random vectors satisfying 4 p := L 4 p (X, θ) < ∞. L
(8.4.30)
Then 1 4p L
n
−B
n
2 Xi , θ
' 'p 4p , ≤ n 'n−B ' L
i=1
and furthermore, 2 1 n 1 ' ' p 1 −B 4 pp+1 . Xi , θ ≤ n p+1 'n−B ' p+1 L π n i=1
In the case where (8.4.29) is not satisfied, we shall prove a result similar to that in Theorem 8.4.10. Define the convolution-type metric associated 4 p : For r > 0, with L 4 p (tB X + θ, tB Y + θ). 4 p,r (X, Y ) := sup tB −r L L
(8.4.31)
t>0
Theorem 8.4.14 Let 0 < p ≤ 1 and X, X1 , X2 , . . . be a sequence of i.i.d. random vectors in IRd . Let r > λ1b , ' 'p ' 'r 6 ' 'r A = max '2−B ' '2B ' + '3B ' , 1 5 and ' 'p ' 'r D ≥ '2−B ' '3B ' . Suppose 4 p (X, θ), L 4 p,r (X, θ)} < ∞ Rp,r = Rp,r (X, θ) := max{L
140
8. Probabilistic-Type Limit Theorems
and
νr ≤ min
a 1 , M M cD
,
where a, c, and M are defined in (8.4.13) and (8.4.14). Then, for all n ≥ 1, 1 π n−B
n
2p+1 Xi , θ
1 4p ≤ L
n−B
i=1
n
2 Xi , θ
≤ Ann−B r Rp,r .
i=1
a In the next theorem we shall relax the assumption νr ≤ min M , M1cD at the cost of losing a little of the order of convergence nn−B r . The next result has a form resembling Theorem 8.4.4. Theorem 8.4.15 Let 0 < p ≤ 1. Let r > 1 λB
λB ΛB r.
Let
{Xi }∞ i=1
ΛB (≥ λ1B ), λ2B
and take q such that
be a sequence of i.i.d. random vectors in IRd
4 p,r = L 4 p,r (X1 , θ) < ∞ L and
Qp,r
1
4 p , µ1 , µrr−q = Qp,r (X1 , θ) := max L
< ∞,
4 p (X1 , θ), µ1 = µ1 (X1 , θ), and µr = µr (X1 , θ). Then for some 4p = L where L absolute constant K = K(d, B, r, p, q) > 0, 1 2 n
' ' 'r ' 4 p,r + 'n−B 'p Qp,r , 4 p n−B L Xi , θ ≤ K n 'n−B ' L i=1
for all n ≥ 1. Case 1< p ≤2. For 1 < p ≤ 2, we use a completely different method in the rate of convergence problem, which relies on the minimality property 4 p and was introduced in Rachev and R¨ uschendorf (1992). of L B (≥ Theorem 8.4.16 Suppose 1 < p ≤ 2 and p > Λ λ2B X2 , . . . be a sequence of i.i.d. random vectors satisfying
4p = L 4 p (X, θ) < ∞ L
and
E[X − θ] = 0.
Then there exists Cr > 0 such that for all n ≥ 1, 2 1 n ' 1 ' −B 4p n 4p , L Xi , θ ≤ Cp n p 'n−B ' L i=1
1 λB ).
Let X, X1 ,
8.4 Operator-Stable Limit Theorems
141
and moreover, the right-hand side vanishes as n → ∞. Furthermore, 2 1 n p p ' ' p 1 −B 4 pp+1 . Xi , θ ≤ Cpp+1 n p+1 'n−B ' p+1 L π n i=1
Corollary 8.4.17 Let θ = (θ(1) , θ(2) ) be a strictly (α1 , α2 )-stable bivariα2 ate vector, 0 < α1 ≤ α2 ≤ 2. Let 2 ≥ p > α21 (≥ α2 ). Let {Xi = (1) (2) 4p = (Xi , Xi )}i≥1 be a sequence of i.i.d. random vectors satisfying L 4 p (X1 , θ) < ∞, and if p > 1, we additionally assume that E[X1 − θ] = 0. L Then for all n ≥ 1, 2 1 n n (1) (2) −1/α1 −1/α2 Xi , n Xi ), θ π (n i=1
i=1
11 4p ≤ L
n−1/α1
n
(1)
Xi , n−1/α2
i=1
≤
⎧ 1 p ⎪ ⎨ n1− α1 L 4 p p+1 p
⎪ ⎩ Cp n p1 − α11 L 4 p p+1
n
2 (2)
Xi
2 max(1,p) p+1 , θ
i=1
for
0 < p ≤ 1,
for
1 < p ≤ 2.
4 p can be Remark 8.4.18 Our approach based on the use of “ideality” of L =n extended to bound the distance between the maxima MX (n) := n−B k=1 k = = n k n −B i=1 Xi and Mθ (n) := n k=1 i=1 θi . (Here k=1 stands for the componentwise maximum, and {θi } are i.i.d. copies of θ.) Also, we can >n k >n k −B compare>mX (n) := n−B k=1 i=1 Xi with mθ (n) := n k=1 i=1 θi n (where k=1 stands for the componentwise minimum) and aX (n) := n−B ' ' k =n ' =n ' −B ' ' ' k θi '. k=1 i=1 Xi with aθ (n) := n k=1 i=1 Theorem 8.4.19 Let 1 < p ≤ 2 and θ be a full strictly operator-stable random vector with exponent B such that n2 n−B p → 0
as
n → ∞.
Let X, X1 , X2 , . . . be i.i.d. random vectors in IRd with E[X − θ] = 0 and such that Lp (X, θ) < ∞. Then there exists Cp,d > 0 such that for every n ≥ 1, % & 4 p (M (n), M (n)) , L 4 p (m (n), m (n)) , L 4 p (a (n), a (n)) max L X X X θ θ θ ' ' 4 p (X, θ). ≤ Cp,d n2/p 'n−B ' L
142
8. Probabilistic-Type Limit Theorems
Remark 8.4.20 Note that aX (n) and aθ (n) are positive random variables, and therefore ⎛ 1 ⎞1/p p 4 p (a (n), a (n)) = ⎝ F −1 (t) − F −1 (t) dt⎠ , L X θ a (n) a (n) X
θ
0 −1 is the generalized inverse of FX ; cf. Theorem 3.1.2. Also, from where FX the above bound we can get the rate of convergence for π by making use of p
the bound π ≤ Lpp+1 . Let us compare Theorem 8.4.16 with a similar result on the rate of convergence in terms of the Zolotarev metric ζr , r > 0; see (8.2.1). Theorem 8.4.21 Let X, X1 , X2 , . . . be i.i.d. random vectors in IRd , and r a positive constant satisfying the conditions ζr := ζr (X, θ) < ∞
and
nn−B r → 0 as n → ∞.
Then for every n ≥ 1, 2 1 n −B Xi , θ ≤ nn−B r ζr , ζr n i=1
and for some Cr > 0, 2 1 n 1 1 ' ' r 1 Xi , θ ≤ Crr+1 n r+1 'n−B ' r+1 ζrr+1 . π n−B i=1
It is known that if r is an integer, then ζr on the right-hand sides of the above bounds can be estimated by the rth difference pseudomoment κr from above (see Zolotarev (1993)). Namely, if all mixed moments of order less than or equal to r − 1 for X and Y agree, then ζr (X, Y ) ≤
1 κr (X, Y ), r!
r ∈ IN,
(8.4.32)
where κr is rth difference pseudomoment % κr (X, Y ) = sup |E[f (X) − f (Y )]| ; f : IRd → IR, (8.4.33) ' ' & ' r−1 r−1 ' |f (x) − f (y)| ≤ 'x x − y y ' for all x, y ∈ IRd . For arbitrary r > 0, ζr is bounded from above by the absolute pseudomoment, namely, if all mixed moments of X and Y of order less than or equal to m (r = m + α, m ∈ IN, α ∈ (1, 2]) agree, then ζr ≤
Γ(1 + α) ξr , Γ(1 + r)
(8.4.34)
8.4 Operator-Stable Limit Theorems
where ξr is the rth absolute pseudomoment xr |PX − PY |( dx). ξr (X, Y ) :=
143
(8.4.35)
IRd
Let us now compare the rate of convergence in Theorem 8.4.21 with that in Theorem 8.4.16 for r = p ∈ (1, 2]. Recall that (8.3.32) is true only for r ∈ IN, and the known estimates for ζr from above by κr (r being noninteger) involve E[Xr ] and E[Y r ]. However, for any p ≥ 1, 4 pp (X, Y ) ≤ 2p κp (X, Y ) ≤ 2p ξp (X, Y ). L
(8.4.36)
4 p (X, θ) < ∞ in Theorem 8.4.16 is preferable Therefore, the restriction L θ) < ∞ in Theorem 8.4.21. On the other hand, the estimate for to ζr (X, n ζr (n−B i=1 Xi , θ) holds for any r > 0 and provides us with the exact order of convergence (as n → ∞) under the assumption ζr (X1 , θ) < ∞. Case 2< p ≤ ∞. Theorem 8.4.22 T8.4.21 Let θ be a full strictly operator-stable random vector that does not have a Gaussian component, or equivalently, whose exponent B satisfies n1/2 n−B → 0
as
n → ∞.
4 p (X, θ) < ∞. Then Let X, X1 , X2 , . . . be i.i.d. random vectors such that L for some C(d, p) > 0, 1 2 n −B 4 4 p (X, θ). Lp n Xi , θ ≤ C(d, p)n1/2 n−B L i=1
Before starting with the proof of our theorems, we introduce a notion of ideality for a probability metric, designed for problem (8.4.2). Definition 8.4.23 A metric ζ : X (IRd ) × X (IRd ) → [0, ∞) is called operator-ideal of order r ≥ 0 if (i) (homogeneity) ζ(aB X, aB Y ) ≤ aB r ζ(X, Y ) for any a > 0, and (ii) (regularity) ζ(X + Z, Y + Z) ≤ ζ(X, Y ) for any Z independent of X and Y . We next show a few lemmas needed for the proof of our main results.
144
8. Probabilistic-Type Limit Theorems
Lemma 8.4.24 , κr , and are regular (that is, (ii) holds); ∗ , Var, and χ∗ are operator-ideal of order r = 0; and ≤ ∗ ≤ 12 Var, χ ≤ χ∗ ≤ Var. This follows directly from the definitions of the metrics. 4 p,r , and ζr are operator-ideal of order r > 0. Lemma 8.4.25 µr , χr , dr , L 4 Lp is operator-ideal of order p ∧ 1. Proof: We first show the operator-ideality of µr . For any a > 0, ' '−r µr (aB X, aB Y ) = sup 'tB ' Var((ta)B X + θ, (ta)B Y + θ) t>0
' '−r 1 2 B B ' t B' t t ' ' a X + θ, a Y +θ = sup ' ' Var ' a a t>0 ' a ' '−r ≤ aB r sup 'tB ' Var(tB X + θ, tB Y + θ), t>0
since ' B ' ' B '−1 't ' 'a '
' ' ' ' '−1 ' = 'tB a−B aB ' 'aB ' ≤ 'tB a−B ' = aB r µr (X, Y ),
which shows the homogeneity of µr of order r > 0. We also have µr (X + Z, Y + Z) = sup tB −r Var(tB (X + Z) + θ, tB (Y + Z) + θ) t>0
≤ sup tB −r Var(tB X + θ, tB Y + θ) t>0
= µr (X, Y ), since tB Z is independent of tB X and θ, and Var is regular. This demon4 p,r strates the regularity of µr . One can check the ideality of χr , dr , and L in a similar fashion. We next show the operator-ideality of ζr . The regularity of ζr is known. As for the homogeneity, we have % ζr (aB X, aB Y ) = sup |E[f (aB X) − f (aB Y )]|; & f (r−1) (x) − f (r−1) (y) ≤ x − y . (8.4.37) Let fa (x) := f (aB x). Then (r−1) fa(r−1) (x)(h)(r−1) = f (r−1) aB x aB h ,
8.4 Operator-Stable Limit Theorems
145
implying that ' ' ' 'r−1 ' ' ' (r−1) ' ' (r−1) B ' a x − f (r−1) aB y ' . (x) − fa(r−1) (y)' ≤ 'aB ' 'fa 'f Then the side condition in (8.4.37), ' ' ' (r−1) ' (x) − f (r−1) (y)' ≤ x − y, 'f results in ' ' ' ' ' 'r−1 ' B ' ' (r−1) ' 'a x − aB y ' ≤ 'aB 'r x − y. (x) − fa(r−1) (y)' ≤ 'aB ' 'fa Consequently, by (8.4.37), ζr aB X, aB Y % & ≤ sup |E[fa (X) − fa (Y )]| ; fa(r−1) (x) − fa(r−1) (y) ≤ aB r x − y = aB r ζr (X, Y ), which shows the regularity of ζr .
2
Lemma 8.4.26 If Z, W ∈ X (IRd ) are independent of X, Y ∈ X (IRd ), then ∗ (X + Z, Y + Z) ≤ ∗ (Z, W ) Var(X, Y ) + ∗ (X + W, Y + W ) and ∗ (X + Z, Y + Z) ≤ ∗ (X, Y ) Var(Z, W ) + ∗ (X + W, Y + W ). Lemma 8.4.27 If Z, W ∈ X (IRd ) are independent of X, Y ∈ X (IRd ), then Var(X + Z, Y + Z) ≤ Var(Z, W ) Var(X, Y ) + Var(X + W, Y + W ) and χ∗ (X + Z, Y + Z) ≤ χ∗ (Z, W )χ∗ (X, Y ) + χ∗ (X + W, Y + W ). Lemma 8.4.28 If Z, W ∈ X (IRd ) are independent of X, Y ∈ X (IRd ), then d(X + Z, Y + Z) ≤ d(Z, W ) Var(X, Y ) + d(X + W, Y + W ), d(X + Z, Y + Z) ≤ d(X, Y ) Var(Z, W ) + d(X + W, Y + W ), 4p . and for 0 < p ≤ 1 both inequalities hold with d replaced by L
146
8. Probabilistic-Type Limit Theorems
Proof: The proofs are very similar to those in Lemma 8.1.15; cf. also Lemma 2 in Senatov (1980) or Lemmas 14.3.3 and 14.3.6 in Rachev (1991). 4p , 0 < We shall demonstrate only the proof of the smoothing inequality for L 4p : p ≤ 1. We use the dual representation for L 4 p (X + Z, Y + Z) L = sup |E[f (X + Z) − f (Y + Z)]| f ∈ Lipb (p)
(recall that Lipb (p) consists of all bounded continuous functions on IRd satisfying |f (x) − f (y)| ≤ x − yp for all x, y ∈ IRd ) = sup PZ ( dz)(E[f (X + z)] − E[f (Y + z)]) f ∈ Lipb (p)
≤
sup (PZ − PW )( dz) (E[f (X + z)] − E[f (Y + z)]) f ∈ Lipb (p) + sup PW ( dz) (E[f (X + z)] − E[f (Y + z)])
≤
f ∈ Lipb (p)
|PZ − PW |( dz)
sup f ∈ Lipb (p)
|(E[f (X + z)] − E[f (Y + z)])|
+ Lp (X + W, Y + W ) =
Var(Z, W )Lp (X, Y ) + Lp (X + W, Y + W ),
as desired.
2
4 p , 0 < p ≤ 1) Let θ Lemma 8.4.29 (Smoothing inequalities for ∗ and L d and θ1 be independent random vectors in IR having the same full strictly operator-stable distribution with exponent B. Then for any X ∈ X (IRd ) independent of θ1 and δ > 0, ∗ (X, θ) ≤ C1 ∗ (X + δ B θ1 , θ + δ B θ1 ) + C2 δ B , and for 0 < p ≤ 1, if E[θp ] < ∞, 4 p (X, θ) ≤ Lp (X + δ B θ1 , θ + δ B θ1 ) + 2δ B p E[θp ]. L Here and in what follows, the Ci ’s are absolute constants depending only on d and B, unless stated otherwise explicitly. B = tB Proof: Fix ε ∈ (0, 1) and choose X ε X, θ = tε θ for some tε > 0 such ∗ that (X, θ) ≤ (X, θ) + ε. We first show the inequality
≤ C1 (X + δ B θ1 , θ + δ B θ1 ) + C2 δ B θ) (X,
(8.4.38)
8.4 Operator-Stable Limit Theorems
147
d = (cX, for any c > 0, we can shrink X, θ) cθ) θ, for θ1 ∼ θ. Since (X, and θ1 without any loss of generality. So we assume
< 1) > P (θ
2 . 3
(8.4.39)
For brevity we shall delete the “ ” from now on. Let θ(i) ∈ IR, i = 1, . . . , d, be the ith component of the operator-stable random vector θ ∈ IRd . Then for each i = 1, . . . , d, θ(i) has a bounded density; that is, for some M < ∞, d (i) P (θ ≤ x) =: sup |pθ(i) (x)| ≤ M for all i. sup x∈IR dx x∈IR (See Hudson (1976).) The idea of the following proof is taken from Lemma 12.1 in Bhattacharya and Rao (1976). First consider the case := (X, θ)
=
sup |P (X ≤ x) − P (θ ≤ x)|
(8.4.40)
x∈IRd
= − inf (P (X ≤ x) − P (θ ≤ x)) . x∈IRd
Given η ∈ (0, ), there exists x0 ∈ IRd such that P (X ≤ x0 ) − P (θ ≤ x0 ) < − + η.
(8.4.41)
We then have I := P (X + δ B θ1 ≤ x0 − δ B e) − P (θ + δ B θ1 ≤ x0 − δ B e) P (X + z ≤ x0 − δ B e)−P (θ + z ≤ x0 − δ B e) P (δ B θ1 ∈ dz) = IRd
=
+
E
,
Ec
where E := {z ∈ IRd − δ B e < z < δ B e} and e = (1, 1, . . . , 1)t ∈ IRd . Then estimating both integrals in the representation for I, we get P (X ≤ x0 ) − P (θ ≤ x0 − z − δ B e) P (δ B θ1 ∈ dz) I ≤ E
+ P (δ B θ1 ∈ E c ).
(8.4.42)
148
8. Probabilistic-Type Limit Theorems
To estimate the last term observe that β := P (δ B θ1 ∈ E) ≥ P (δ B θ1 < δ B ) >
2 , 3
(8.4.43)
by (8.4.39). On the other hand, denoting the distribution function of θ by F (x), x = (x(1) , x(2) , . . . , x(d) )t ∈ IRd , and ε = (ε(1) , ε(2) , . . . , ε(d) )t ∈ IRd , we have |P (θ ≤ x + ε) − P (θ ≤ x)| d ≤ F (x(1) , . . . , x(i−1) , x(i) + ε(i) , . . . , x(d) + ε(d) ) i=1
≤
d
−F (x(1) , . . . , x(i) , x(i+1) + ε(i+1) , . . . , x(d) + ε(d) ) P (θ(i) ∈ Ii ).
i=1
Here Ii := (x(i) , x(i) + ε(i) ] or := (x(i) + ε(i) x(i) ] depending on the sign of ε(i) . Therefore, d
P (θ(i) ∈ Ii ) ≤
i=1
d
|ε(i) | sup |pθ(i) (x)| ≤ M ε1 ,
i=1
x∈IR
where · 1 is the L1 -norm. Hence, −P (θ ≤ x + ε) ≤ −P (θ ≤ x) + M ε1 .
(8.4.44)
Thus we have, by (8.4.41), (8.4.42), and (8.4.44) with ε = −z − δ B e, that I ≤ P (X ≤ x0 ) − P (θ ≤ x0 ) + M (z1 + dδ B ) P (δ B θ1 ∈ dz) E
≤
+ P (δ B θ1 ∈ E c ) − + η + M (z1 + dδ B ) P (δ B θ1 ∈ dz) + P (δ B θ1 ∈ E c ).
E
Since z1 ≤ dδ B on E, it follows that I
≤ (− + η + 2M dδ B )P (δ B θ1 ∈ E) + P (δ B θ1 ∈ E c ) ≤ (1 − 2β) + η + 2M dδ B .
Consequently, (2β − 1) ≤ (X + δ B θ1 , θ + δ B θ1 ) + 2M dδ B + η.
8.4 Operator-Stable Limit Theorems
149
Since η can be taken arbitrarily small, we have 1 (X + δ B θ1 , θ + δ B θ1 ) + 2M dδ B . 2β − 1 Since β > 23 by (8.4.43), ≤ 3 (X + δ B θ1 , θ + δ B θ1 ) + 2M dδ B . This proves the inequality (X, θ) ≤ C1 (X + δ B θ1 , θ + δ B θ1 ) + C2 δ B with C1 = 3 and C2 = 6M d, provided that (8.4.40) holds. ≤
If, on the other hand, = supx∈IRd (P (X ≤ x) − P (θ ≤ x)), then given η ∈ (0, ), there exists x0 such that P (X ≤ x0 ) − P (θ ≤ x0 ) > − η. Then we similarly have P (X + δ B θ1 ≤ x0 + δ B e) − P (θ + δ B θ1 ≤ x0 + δ B e) = P (X + z ≤ x0 + δ B e) − P (θ + z ≤ x0 + δ B e) P (δ B θ1 ∈ dz) IRd
+
= E
≥
Ec
P (X ≤ x0 ) − P (θ ≤ x0 − z + δ B e) P (δ B θ1 ∈ dz)
E
− P (δ B θ1 ∈ E c ) ≥
P (X ≤ x0 ) − P (θ ≤ x0 ) − M (z1 + dδ B ) P (δ B θ1 ∈ dz)
E
− P (θB θ1 ∈ E c ) ≥ (2β − 1) − η − 2M dδ B . B B B Hence (2β − 1)B ≤ (XB+ δ θ1 , θ + Bδ θ1 ) + 2M dδ + η, so that ≤ 3 (X + δ θ1 , θ + δ θ1 ) + 2M dδ . This completes the proof of the inequality (8.4.28), and reintroducing the symbol “ ”, we write
≤ C1 (X + δ B θ1 , θ + δ B θ1 ) + C2 δ B . θ) (X, and θ, Therefore, by the definition of ∗ , X, ∗ (X, θ)
+ε θ) ≤ (X, + δ B θ1 , θ + δ B θ1 ) + C2 δ B + ε ≤ C1 (X B B B B ≤ C1 (tB ε (X + δ θ1 ), tε (θ + δ θ1 )) + C2 δ + ε
≤ C1 ∗ (X + δ B θ1 , θ + δ B θ1 ) + C2 δ B + ε,
150
8. Probabilistic-Type Limit Theorems
which yields the first smoothing inequality of Lemma 8.4.29. 4 p (0 < p ≤ 1). By Now let us show an inequality of a similar type for L 4 the triangle inequality and the regularity of Lp , 4 p (X, θ) ≤ L 4 p (X, X + δ B θ1 ) + L 4 p (X + δ B θ1 , θ+ δ B θ1 ) + L 4 p (θ, θ+ δ B θ1 ) L 4 p (X + δ B θ1 , θ + δ B θ1 ) + 2L 4 p (0, δ B θ1 ). ≤ L 4 p as a minimal metric with respect to the Lp -metric, From the definition of L it follows that 4 p (0, δ B θ) = E[δ B θp ] ≤ δ B p E[θp ], L which completes the proof of Lemma 8.4.29.
2
The proof of the next two lemmas is obvious. Lemma 8.4.30 For any a > 0 and r > 0, Var(aB X + θ, aB Y + θ) ≤ aB r µr (X, Y ), χ∗ (aB X + θ, aB Y + θ) ≤ aB r χr (X, Y ), d(aB X + θ, aB Y + θ) ≤ aB r dr (X, Y ), and for 0 < p ≤ 1, 4 p (aB X + θ, aB Y + θ) ≤ aB r L 4 p,r (X, Y ), L where X, Y ∈ X (IRd ) are independent of θ. Lemma 8.4.31 Let Aut(IRd ) be the set of all invertible linear operators (automorphisms). Then, for any A ∈ Aut(IRd ), d(AX, AY ) = |JA−1 |d(X, Y ), where JA is the Jacobian of the matrix A. Lemma 8.4.32 For x ∈ [2, 3], |JxB | ≤ C(d, B), where C(d, B) is defined in (8.4.22). Proof: Note that |JxB | = | det xB |.
8.4 Operator-Stable Limit Theorems
151
If A is d × d matrix, then | det A| ≤ d! | max Aij |d , 1≤i,j≤d
which proves the lemma.
2
Lemma 8.4.33 Let θ be a full strictly operator-stable random vector in IRd with exponent B. Then for any two independent copies θ1 and θ2 of θ and for any t, s > 0, d
tB θ1 + sB θ2 ∼ (t + s)B θ. Proof: By (8.4.1) with b(t) ≡ 0, B B B B E eiz,t θ1 +s θ2 = E eiz,t θ1 E eiz,s θ2
∗ ∗ = µ 4 tB z µ 4 sB z = µ 4(z)t µ 4(z)s
∗ = µ 4(z)t+s = µ 4 (t + s)B z B = E eiz,(t+s) θ . 2 The following lemmas are proved in Sections 2.5 and 2.6. Further, Cb (IRd ) stands for the set of all bounded continuous functions on IRd . 4p ) L 4 p admits the following repreLemma 8.4.34 (Duality theorems for L sentation: (i) For p = 0, 4 0 (X, Y ) L
=
=
sup{|E[f (X) − f (Y )]|; f ∈ Cb (IRd ) such that |f (x) − f (y)| ≤ 1 for all x, y ∈ IRd } 1 Var(X, Y ). 2
(ii) For 0 < p ≤ 1, 4 p (X, Y ) L
=
sup{|E[f (X) − f (Y )]|; f ∈ Cb (IRd ) such that |f (x) − f (y)| ≤ x − yp for all x, y ∈ IRd }.
152
8. Probabilistic-Type Limit Theorems
(iii) For 1 < p < ∞, 4 p (X, Y ) L
=
sup{E[f (X)] − E[g(Y )]; f, g ∈ Cb (IRd ) such that |f (x) − g(y)| ≤ x − yp for all x, y ∈ IRd }.
(iv) For p = ∞, 4 ∞ (X, Y ) L
inf{ε > 0; P (X ∈ A) ≤ P (Y ∈ Aε ) for all A ∈ B(IRd )}.
=
4 p -convergence 4 p -convergence) (i) For any 0 ≤ p ≤ ∞, L Lemma 8.4.35 (L implies weak convergence. Moreover, if π is the Prohorov metric, then 1
4 pp+1 π ≤ L
for all 0 ≤ p ≤ 1
and p
4 pp+1 π ≤ L
for all 1 ≤ p ≤ ∞.
(ii) Let 0 < p < ∞ and E[Xn p ] + E[Xp ] < ∞. Then 4 p (Xn , X) → 0 L if and only if w
Xn → X
E[Xn p ] → E[Xp ].
and
4 p in X (IR)) For d = 1, 1 ≤ Lemma 8.4.36 (Explicit representations for L p ≤ ∞, ⎛ 4 p (X, Y ) = ⎝ L
1
⎞1/p −1 |FX (t) − FY−1 (t)|p dt⎠
,
1 ≤ p < ∞,
0
4 ∞ (X, Y ) L
=
−1 sup |FX (t) − FY−1 (t)|.
0≤t≤1
4 p ) For 1 ≤ p < ∞, Lemma 8.4.37 (Upper bounds for L 4 p ≤ κp ≤ ξp , 2−p+1 L where κp (resp. ξp ) is the pth difference (resp. absolute) pseudomoment.
8.5 Proofs of the Rate of Convergence Results
153
8.5 Application to Operator-Stable Limit Theorems: Proofs of the Rate of Convergence Results In this section we give the proofs of the rate of convergence results stated in Section 8.4. Proof of Theorem 8.4.4: All probability metrics for the random vectors in this section are defined by their marginal distributions and are consequently independent of their joint distributions. So, without loss of generality, we assume that {Xi } and θ are independent of each other. Let {θi } be independent copies of θ and assume that {θi } are independent of {Xi } and θ. Then by the definition of θ (or by (8.4.1) with b(t) ≡ 0), n
n−B
d
θi ∼ θ
for any n = 1, 2, . . . .
(8.5.1)
i=1
Now, by Lemma 8.4.29 and (8.5.1), for any δ > 0, 1 2 n ∗ n−B Xi , θ i=1
1
≤ C1
∗
−B
n
= C1
2 B
+ C2 δ B
B
Xi + δ θ1 , θ + δ θ1
i=1
1 ∗
n
(8.5.2)
−B
n
n
B
Xi + δ θ, n
−B
i=1
n
2 + C2 δ B .
B
θi + δ θ
i=1
Furthermore, by the triangle inequality, 1 2 n n ∗ −B B −B B Xi + δ θ, n θi + δ θ n i=1
i=1
1 ∗
≤
n
−B
n i=1
+
m
⎛
+
⎛
j i=1
1 ∗
Xi + δ θ, n